Compare commits

..

313 Commits

Author SHA1 Message Date
Joseph Doherty 5776021970 ci(rust): allow generated result size lint
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m18s
ci / java (push) Successful in 2m24s
ci / portable (push) Successful in 9m31s
2026-08-25 12:28:12 -04:00
Joseph Doherty a11786cbea docs: record batch N portable gates
ci / portable (push) Failing after 17m19s
ci / java (push) Successful in 2m29s
ci / windows-x86 (push) Successful in 5m55s
ci / nightly-windev (push) Has been skipped
2026-08-25 12:09:40 -04:00
Joseph Doherty aea2c74fdf docs: start batch N release gate
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m18s
ci / java (push) Successful in 2m11s
ci / portable (push) Failing after 8m43s
2026-08-25 12:06:34 -04:00
Joseph Doherty 8b147a32a2 docs: record batch M verification
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m19s
ci / java (push) Successful in 2m26s
ci / portable (push) Failing after 5m27s
2026-08-25 12:02:19 -04:00
Joseph Doherty cc143b52cf ci(TST-35): run gateway suite on windev nightly
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m44s
ci / windows-x86 (push) Successful in 5m56s
ci / portable (push) Failing after 9m21s
2026-08-25 11:56:35 -04:00
Joseph Doherty ad79238c84 docs: record batch L verification
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m17s
ci / java (push) Successful in 2m28s
ci / portable (push) Failing after 10m12s
2026-08-25 11:50:48 -04:00
Joseph Doherty 986c6c0ce4 fix(IPC-36): import obsolete attribute namespace
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 2m4s
ci / java (push) Successful in 2m22s
ci / portable (push) Failing after 9m44s
2026-08-25 11:49:02 -04:00
Joseph Doherty 484028e721 feat(GWC-37,WRK-32): refresh live alarm watch lists
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 51s
ci / java (push) Successful in 2m8s
ci / portable (push) Failing after 1h1m54s
2026-08-25 11:46:07 -04:00
Joseph Doherty 1b11e89f29 docs: record batch K verification 2026-08-25 11:26:12 -04:00
Joseph Doherty 5cdb3da8c0 fix(CLI-47,CLI-48,CLI-49,CLI-50): align alarm feed clients 2026-08-25 11:25:47 -04:00
Joseph Doherty feadcf5740 docs: record batch J verification 2026-08-25 11:14:33 -04:00
Joseph Doherty d6c5a11ae6 fix(IPC-36,IPC-37): guard secondary contracts 2026-08-25 11:14:14 -04:00
Joseph Doherty f3827c71c9 docs: record batch I verification 2026-08-25 11:01:38 -04:00
Joseph Doherty 1875426e4b fix(SEC-38,SEC-39): preserve aggregate and fallback throttles 2026-08-25 11:00:29 -04:00
Joseph Doherty 780720165a fix(GWC-35): repair re-raised alarm snapshots 2026-08-25 10:58:21 -04:00
Joseph Doherty b0156c685e fix(GWC-33): unregister terminal worker queue gauge 2026-08-25 10:56:23 -04:00
Joseph Doherty f2090fe1b5 docs: record batch H verification 2026-08-25 10:53:03 -04:00
Joseph Doherty 92aae28f07 chore(GWC-36,IPC-38,TST-33,TST-34,TST-36): close cycle-3 currency gaps 2026-08-25 10:52:39 -04:00
Joseph Doherty 4614080e24 docs: normalize continuation design formatting 2026-08-25 10:41:22 -04:00
Joseph Doherty 8e30ec5be9 docs: plan and track cycle-3 P2 continuation 2026-08-25 10:40:55 -04:00
Joseph Doherty 30548da71c docs: design cycle-3 P2 continuation 2026-08-25 10:37:30 -04:00
Joseph Doherty 914495a9f3 chore: ignore project-local worktrees 2026-08-20 12:14:23 -04:00
Joseph Doherty fa364ce139 chore(archreview): cycle-3 remediation integration gate recorded (gateway 1183/1183, windev worker 524/524)
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m23s
ci / java (push) Successful in 2m27s
ci / portable (push) Successful in 9m25s
2026-08-20 09:41:49 -04:00
Joseph Doherty f2ab9cc879 Merge fix/c3-tracker-hygiene: GWC-31 perf-pass comments relabeled PERF-nn, archreview prefixes reserved in REVIEW-PROCESS.md, cycle-1/2 tracker housekeeping (GWC-16 Won't fix, IPC-08/IPC-31 Done, TST-06/17 refreshed, WRK-29..31 reserved)
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m57s
ci / java (push) Successful in 2m11s
ci / portable (push) Successful in 9m23s
2026-08-20 09:40:05 -04:00
Joseph Doherty cb820bb666 chore(archreview): tracker row G → Done with outcome notes 2026-08-20 09:39:59 -04:00
Joseph Doherty c6650c156b Merge fix/c3-session-lifecycle: GWC-32 WorkerClient dispose is terminal, GWC-34 attach-after-dispose throws SessionNotReady, GWC-37 subtag watch-list coverage graded on provider status + re-resolved at failover edge 2026-08-20 09:39:53 -04:00
Joseph Doherty 193221773d fix(GWC-37): grade the alarm subtag watch-list instead of latching it silently
The subtag fallback's coverage was fixed at SubscribeAlarms time and never
re-examined. AlarmWatchListResolver deliberately swallows a Galaxy Repository
outage and returns whatever configuration contributed, so a discovery failure at
startup left the monitor subscribed with an empty watch-list for the rest of its
life. In Auto mode the alarmmgr primary keeps working, so nothing looks wrong
until the failover that needs the standby finds it covering zero alarms — with a
single LogWarning emitted possibly weeks earlier as the only evidence.

ResolveAsync now returns an AlarmWatchListResolution (targets + whether discovery
was attempted and whether it failed, with the failure message), which is what the
caller needed to tell "discovery produced nothing" from "nothing was configured".
GatewayAlarmMonitor grades that resolution against the effective mode and folds
the verdict into AlarmProviderStatus, so it reaches the feed, the dashboard badge
and every late-joining subscriber:

- zero targets while the mode can need them (Auto/ForceSubtag) -> degraded with
  SubtagWatchListEmpty; discovery enabled but failing adds SubtagDiscoveryFailed
  and the message. ForceAlarmManager builds no subtag consumer and is not graded.
- degraded is now the OR of "subtag is serving" and "the watch-list covers
  nothing", so the Auto case where alarmmgr is healthy and the standby is empty
  is visible. DashboardAlarmProviderStatus renders that combination as the amber
  "Alarm Manager (no subtag standby)" badge rather than the failover label, which
  would have named the wrong live provider.
- on the failover edge (ApplyProviderModeChangeAsync, toMode == Subtag) the
  monitor re-resolves BEFORE publishing the status and reports any difference as
  SubtagWatchListStale with both counts.

The re-resolution is deliberately NOT re-sent. The worker rejects a second
SubscribeAlarms while subscribed (AlarmCommandHandler.Subscribe), and the only
expressible replacement — UnsubscribeAlarms then SubscribeAlarms — rebuilds
FailoverAlarmConsumer, which restarts on the primary that just failed and
discards the warm standby state, so it would flap the feed rather than repair it.
The finding's other alternative (treat empty coverage as a lifecycle failure) was
rejected too: it would crash-loop the monitor every 5 s on a persistent GR outage
and take the working alarmmgr feed down with it. A monitor restart re-resolves and
re-sends, and the status now says so.

Docs: docs/AlarmClientDiscovery.md gains the grading rules and corrects two stale
claims (the resolver method name, and "re-runs discovery on its reconcile cadence
and pushes an updated watch-list", which never happened); gateway.md gains the
standby-coverage visibility paragraph; docs/GatewayConfiguration.md records the
consequence on the UseGalaxyRepository and Mode rows.

Tests: SubtagCoverage_WhenDiscoveryFailsAtSubscribe_ProviderStatusIsDegradedWith-
Reason and SubtagCoverage_WhenDiscoveryRecoversByFailover_ReResolvesAndReportsDrift
drive the fake-worker monitor; two resolver cases pin the new provenance fields.
The pre-existing dashboard case for degraded+ALARMMGR now asserts the new label.
2026-08-20 09:38:46 -04:00
Joseph Doherty 06cd8b844c chore(archreview): tracker rows D/E → Done with outcome notes 2026-08-20 09:33:59 -04:00
Joseph Doherty cfbc99581b Merge fix/c3-ipc-boundaries: IPC-33 drain budget bounded under the gRPC send cap + validator upper coupling, IPC-39 pre-check sequence allowance, IPC-34 WorkerCancel on caller cancellation + CallerCancelled code, IPC-35 handshake WorkerFault surfaces category 2026-08-20 09:33:55 -04:00
Joseph Doherty ab41362a5c fix(IPC-35): surface a handshake-phase WorkerFault with its category and diagnostic
The worker reports startup failures as structured WorkerFault frames --
MxAccessCreationException carries the COM HRESULT/ProgID context and a
WorkerFaultCategory -- so the gateway can tell "MXAccess is not installed / not
registered / access denied" apart from a pipe-level protocol problem.
ReadHandshakeEnvelopeAsync did not look inside: any body other than the expected
one became a ProtocolViolation naming only the body type, and that text is what
SessionManager puts in the operator log and the client's OpenFailed detail. The
information was lost exactly in the window where it matters most -- first
session open on a freshly deployed host, where the answer is almost always an
MXAccess installation or permission problem.

Special-case BodyCase == WorkerFault and throw WorkerClientException(
WorkerFaulted, CreateWorkerFaultMessage(...)) -- the same formatter the
steady-state fault path already uses, so both windows read alike. Every other
unexpected body still gets ProtocolViolation.

Test: StartAsync_WhenWorkerFaultsDuringHandshake_SurfacesFaultCategoryAndDiagnostic
has the fake worker answer the hello and then write a WorkerFault
{MxaccessCreationFailed} in place of WorkerReady; the existing
FakeWorkerHarness.EmitFaultAsync covered it, so the harness needed no change.
2026-08-20 09:32:01 -04:00
Joseph Doherty b27035b3a6 fix(IPC-34): forward a WorkerCancel when the caller abandons a command, not only on gateway timeout
IPC-08 closed with WorkerCancel sent from the gateway's own timeout arm only.
Its rationale -- the gateway has stopped waiting but the worker has not stopped
working, so a queued-but-not-started command should be dropped instead of
occupying the single STA behind a caller that is already gone -- applies exactly
as well to caller cancellation, which is the common abandonment path: gRPC
clients routinely set per-call deadlines shorter than the gateway's
CommandTimeout, and a client disconnect cancels the RPC's token. Both landed in
arms that removed the pending entry and returned silently, leaving the
correlation to execute in its turn. Under a retry storm against a slow provider
the STA works through dead commands while live callers queue behind them --
precisely the backlog the cancel exists to shorten.

- TrySendCancelForTimedOutCommand is now TrySendCancelForAbandonedCommand and
  takes the reason string, since it serves both arms.
- Both caller-cancellation arms route through AbandonCommandOnCallerCancellation,
  which sends the cancel under the same `removed` ownership gate as the timeout
  arm (so a reply that won the race is never chased) and skips it once _stopCts
  has fired, where the outbound channel is being completed and every pending
  command is failed anyway.
- New WorkerClientErrorCode.CallerCancelled replaces the GatewayShutdown label
  those arms used, so metrics and logs stop recording a shutdown that did not
  happen. It maps to StatusCode.Cancelled, the same status GatewayShutdown maps
  to, so nothing changes for clients.

Docs: docs/Grpc.md mapping line, gateway.md cancellation rules,
docs/GatewayProcessDesign.md pending-command section.

Tests: InvokeAsync_WhenCallerCancels_SendsWorkerCancelForThatCorrelation (twin
of the timeout test, driven by the caller token; verified failing without the
send) and InvokeAsync_WhenClientStopsUnderInFlightCommand_DoesNotSendWorkerCancel.
2026-08-20 09:31:29 -04:00
Joseph Doherty 9f0c3fd9fa Merge fix/c3-dashboard-acl: SEC-41 ACL re-resolves tag grants from group claims, SEC-40 alarms hub/page behind non-bypassable authenticated-role policy, GWC-38 browse scope fails closed without identity under ApiKey mode 2026-08-20 09:31:21 -04:00
Joseph Doherty a71663a2a2 fix(IPC-39): make the CommandTooLarge pre-check allow for the sequence stamp
The pre-check measures the command envelope while Sequence is still 0, and an
unset protobuf field costs nothing to encode -- so the measured size is a lower
bound on the frame WriteLoopAsync emits after GWC-28 stamps the sequence. Being
a lower bound is the right kind of conservative for the knownSize question the
comment answers, and the wrong kind for the pre-check itself: it admits an
envelope in the last eleven bytes of the range that the writer then rejects with
a session-fatal MessageTooLarge.

Compare against MaxMessageBytes - MaxSequenceEncodingBytes (1 tag byte + a
10-byte varint, the widest 64-bit field encoding) and say in the comment that
the pre-check is a lower bound relying on that allowance.

Not reachable under the shipped validator (caller-controlled commands arrive
through gRPC capped at least 64 KiB below the pipe max, gateway-authored
commands are tiny), so this closes the argument rather than a live defect. The
existing CommandTooLarge test still passes unchanged.
2026-08-20 09:30:56 -04:00
Joseph Doherty 8257a0333e docs(SEC-40): correct the hub-mapping comment now that AlarmsHub uses its own policy 2026-08-20 09:30:45 -04:00
Joseph Doherty 577a0db862 fix(IPC-33): bound the DrainEvents byte budget against the gRPC send cap, not just the pipe frame max
The worker budgeted a DrainEvents reply against the negotiated pipe frame
maximum less one 64 KiB reserve, but the gateway returns that same
MxCommandReply verbatim through Invoke, where MaxSendMessageSize applies. The
two limits were coupled only from below (pipe >= gRPC + reserve), so any
configuration the validator permitted with a wider gap let the worker pack a
reply the pipe carried and the gRPC layer then refused with ResourceExhausted --
after Drain had already removed the events from the worker queue, losing them to
both the caller and the event stream.

Option (a) from the finding: close the coupling at both ends.

- GatewayOptionsValidator adds the upper half of the headroom rule
  (Worker:MaxMessageBytes <= Protocol:MaxGrpcMessageBytes + 2 x reserve), with a
  message naming the DrainEvents budget as the reason and the two knobs to move.
- WorkerPipeSession.ResolveDrainReplyByteBudget subtracts the reserve twice --
  once for the pipe frame, once so the MxCommandReply wrapper clears the gRPC
  send cap -- keeping the MaxMessageBytes/2 floor that makes small frame maxima
  usable.

The two changes are load-bearing for each other: the validator band is what
makes "frame max - 2 x reserve" provably at or below the gRPC cap, and the
doubled subtraction is what makes the band sufficient.

Docs updated in the same change (GatewayConfiguration.md both option rows plus
the headroom prose, WorkerFrameProtocol.md drain budget, gateway.md DrainEvents
byte axis). Tests: new GatewayOptionsValidatorTests cases for the upper bound
(reject) and both edges of the band (accept); GatewayOptionsTests' override case
now moves both message-size knobs together, since the band makes moving one
alone illegal.

The worker half cannot be built on macOS; verified on windev separately.
2026-08-20 09:30:39 -04:00
Joseph Doherty 2ef7b93761 fix(SEC-41, SEC-40): re-resolve dashboard tag grants; gate the alarm feed on a real role
Committed together because the two changes share the same doc paragraphs (the ACL
section of GatewayDashboardDesign.md, the GroupToTag / AllowAnonymousLocalhost rows
of GatewayConfiguration.md, and CLAUDE.md's authentication paragraph), and splitting
them would leave one of the two describing behaviour that does not yet exist.

SEC-41 — tag grants no longer trusted from login-stamped claims
---------------------------------------------------------------
DashboardSessionAcl read the zb:dashboardtag claims stamped into the cookie at
login. GatewayOptions binds once at startup, so applying a GroupToTag edit needs a
restart — but the restart does not clear the auth cookie (data-protection keys
persist), so a browser returning afterwards re-authenticated with tag claims
resolved under the OLD map. A revoked grant therefore survived on the cookie seam
for the sliding 8-hour idle life, renewable indefinitely by activity, while the
hub-token seam had no such gap because its mint re-resolves.

CanViewSession now re-resolves the grant on every subscribe from the caller's
mxgateway:ldap_group claims through DashboardGroupTagMapping.MapGroupsToTags
against the currently bound map — the same one call HubTokenService.Issue makes.

Fallback (documented on the class and in the design doc): a principal carrying NO
group claims falls back to its stamped tag claims. That arm exists for the
hub-token principal — HubTokenService.Validate rehydrates roles and tags but not
groups, so re-resolution already happened at mint, one lifetime ago at worst. It
cannot re-open the cookie hole: DashboardAuthenticator stamps group and tag claims
from one group list, so a cookie principal holding a tag claim always holds the
groups it came from, and a login yielding no groups yields no tags. A principal
with neither kind of claim stays an empty-grant Viewer.

SEC-40 — the alarm feed requires a real authenticated dashboard role
--------------------------------------------------------------------
AlarmsHub sat behind HubClientsPolicy, which the read-only environment bypasses
satisfy, so an anonymous-localhost caller or a Mode=Disabled deployment received
galaxy-wide alarm metadata — alarm reference (a full tag path), state, severity,
category, area, acknowledging operator — for the whole subscription expression.

Decided: tighten the gate, keep the scope. New AuthenticatedDashboardRoleRequirement
+ its handler, and a MxGateway.Dashboard.Alarms policy built from it. Because
DashboardAuthorizationHandler is typed to DashboardAuthorizationRequirement, it is
never invoked for the new type, so neither Authentication:Mode=Disabled nor
anonymous localhost can satisfy the policy — the same structural device the mounted
Secrets UI already relies on. The policy keeps both schemes so external hub clients
can still authenticate with a bearer token, and DisableLogin auto-login is
unaffected (it mints a genuinely authenticated both-roles principal).

The /alarms page carries the same policy, not just the hub: it reads the same
central monitor in process, so gating only the hub would have left the disclosure
one navigation away. The side rail hides the Alarms link for callers the policy
denies, matching the Secrets link precedent. AlarmsHubPublisher is unchanged and
stays unfiltered by design — audience is decided at hub admission, and the monitor
carries no per-session provenance to narrow by.

Alarm metadata deliberately remains galaxy-wide for authenticated Viewers: an
AlarmFeedMessage has no session id or owning key for the SEC-25 tag intersection to
work against, and equating a visibility tag with an alarm-reference prefix would
turn GroupToTag into the data-access constraint it explicitly is not.

Tests
-----
DashboardSessionAclTests: the SEC-41 regression — a cookie-style principal whose
stale tag claim is no longer granted by any of its groups under the current map is
denied; plus the allowing direction (including with no stamped tag at all, which
proves the decision comes from the groups), the ignore-case corner, group claims
that map to nothing, and the hub-token fallback asserted against an EMPTY map so
dropping the fallback would fail loudly.

AlarmFeedAuthorizationTests (new): both handlers run together in the most permissive
environment (Mode=Disabled + loopback + AllowAnonymousLocalhost) do not satisfy the
alarm requirement, with the bypassable requirement as the control so the negative
cannot pass because the bypass was misconfigured; authenticated Viewer/Admin do
satisfy it; an unauthenticated role claim does not; the registered policy carries
the non-bypassable requirement and both schemes; and both readers — the hub type's
[Authorize] and the /alarms endpoint metadata — carry it.

SecretsNavRenderTests: registers the new policy (an unregistered policy name throws
rather than denying in an AuthorizeView) and gains the Alarms-link negative/control
pair.

Verified: dotnet build src/ZB.MOM.WW.MxGateway.NonWindows.slnx (zero warnings);
filtered run of the touched classes — 92 passed, 0 failed.
2026-08-20 09:29:44 -04:00
Joseph Doherty b177636d7a fix(GWC-38): deny Galaxy browse when the ambient identity is absent under API-key auth
GatewayBrowseScopeProvider resolved `identityAccessor.Current?.EffectiveConstraints
?? ApiKeyConstraints.Empty`, so an absent ambient identity produced an empty
BrowseSubtrees list — which the shared GalaxyRepository library reads as "no
scoping". The control's fail state was therefore open: any future path reaching the
library's galaxy service without GatewayGrpcAuthorizationInterceptor having pushed
the identity would silently hand a browse-scoped key the full hierarchy, with
nothing failing and nothing logged. A constraint-narrowing control's absence is
invisible in normal operation, which is what made the honest "safe only because the
global interceptor runs first" comment insufficient on its own.

The provider now distinguishes the two states the old expression collapsed:

- authenticated key with no BrowseSubtrees constraint -> empty list, unscoped
  (unchanged, and now asserted in its own test rather than implied);
- no ambient identity with Authentication:Mode=ApiKey -> log a warning naming the
  broken ordering invariant and throw RpcException(PermissionDenied);
- no ambient identity with Authentication:Mode=Disabled -> unscoped, because
  nothing authenticates there so no key and no constraints exist to scope by.

The registration-order comment at GatewayApplication.cs now says the ordering is
still required but no longer silently trusted.

Tests: GatewayBrowseScopeProviderTests gains the PermissionDenied case and the
Disabled-mode case alongside the unconstrained-identity one.
GalaxyRepositoryHostWiringTests phase 1 now pushes an identity carrying no
BrowseSubtrees instead of pushing none — that, not an absent identity, is
production's unscoped case.

Docs: docs/Authorization.md's constraint section states the two-state distinction
and both fail behaviours under browse_subtrees.
2026-08-20 09:28:54 -04:00
Joseph Doherty 282a0e70ad fix(GWC-34): fail an attach that races session disposal instead of hanging it
EnsureDistributorCreated read a null _eventDistributor as "not created yet",
which is true before MarkReady and false after DisposeAsync nulled it. An
attach that passed its readiness gate while the session was still Ready and
then lost the CPU across the whole close + dispose therefore built a
REPLACEMENT distributor: startNow came back false because the pump-start flag
is a one-way latch, so the subscriber registered on a distributor whose pump
never starts and whose channel is never completed, and EventStreamService
blocked in WaitToReadAsync until the client's own deadline — no error, no log,
no metric, the backlog gauge reporting 0 forever. This is the disposal-side
twin of the premature-attach poison GWC-27 closed.

DisposeAsync now sets _eventDistributorDisposed under _syncRoot in the same
section that nulls the distributor, and EnsureDistributorCreated throws
SessionManagerException(SessionNotReady) when it is set. The attach paths'
existing catch/rollback releases the subscriber slot, so a second attempt
still reports SessionNotReady rather than EventSubscriberAlreadyActive.

StartDashboardMirror is the one caller that is not an attach: it abandons the
mirror (releasing its loop token source) instead of propagating, so MarkReady
stays total. Losing that race previously left a started orphan distributor
behind, so this path strictly improves too.

Test: AttachAfterDisposeThrowsSessionNotReadyInsteadOfRegisteringOnANeverPumped-
Distributor covers all three attach entry points plus the rollback, and fails
on the pre-fix code. docs/Sessions.md records the disposal end of the
one-distributor-per-session rule.
2026-08-20 09:25:21 -04:00
Joseph Doherty 3619152b9f chore(GWC-31): relabel perf-pass GWC-3x comments to PERF-nn; tracker housekeeping
Sweep code/doc comments citing unregistered GWC-30/GWC-31 (2026-08-15 perf
pass) to PERF-02/PERF-07/PERF-22, keyed to task numbers in
docs/plans/2026-08-15-perf-review-remediation.md; add a PERF-ID register
table to that plan. Genuine cycle-2 GWC-30 citations (gateway-side frame-
reader 4-byte prefix fix in Server/Workers/WorkerFrameReader.cs and its
test) are left untouched.

Add a line to REVIEW-PROCESS.md reserving the GWC/WRK/IPC/SEC/CLI/TST
prefixes for the archreview domain registers.

Tracker housekeeping per archreview/2026-08-19/00-overall.md:
- GWC-16 -> Won't fix (rationale: PERF(followup) comment documents the
  double session lookup as measured-negligible, deliberately not collapsed)
- IPC-08 -> Done (closed by 7c1ea12/a1a38b5; residual tracked as IPC-34)
- IPC-31 -> Done (superseded by GWC-28's write-time sequence stamping)
- TST-06, TST-17 rows refreshed per cycle-3 60-testing-docs-gaps.md
- WRK-31 (and defensively WRK-29/30) registered as reserved/do-not-reuse
  interim IDs burned by e913dab, in both the cycle-2 tracker and the
  cycle-2 worker remediation doc

Flip Batch C to Done in archreview/2026-08-19/remediation/00-tracking.md.

Worker-side files (Worker/Ipc/WorkerFrameReader.cs, WorkerFrameWriter.cs)
are comment-only edits, unverified here (net48 x86, windev-only build).
2026-08-20 09:23:36 -04:00
Joseph Doherty ee68cc3f72 Merge fix/c3-client-versions: CLI-46 client versions bumped past the published train (0.2.1 / Java 0.2.2), pypi 0.2.0 broken-connect() note, check-client-versions.ps1 drift guard in CI 2026-08-20 09:22:48 -04:00
Joseph Doherty 7200023bcc fix(CLI-46): bump client versions past the already-published train + add PR-time drift guard
.NET/Go/Python/Rust clients (plus Contracts, which releases in lockstep
with the .NET Client) move 0.2.0 -> 0.2.1; Java moves 0.2.1 -> 0.2.2. This
starts the next release train the finding recommended, since post-publish
source changes (including the Python connect() event-loop fix, a8f86b5)
had been sitting at already-published version numbers for nine days with
nothing recording the divergence.

- Note the pypi 0.2.0 connect() defect and its supersession by 0.2.1 in
  clients/python/README.md and docs/ClientPackaging.md.
- Add scripts/check-client-versions.ps1: a PR-time guard that finds each
  client's latest publish tag (clients/go/vX.Y.Z for Go, the only client
  with a git tag; the other four publish straight to a registry with no
  tag, so the check reports "no tags found" and passes for them) and fails
  if clients/<lang> changed since that tag while the version constant
  still equals the tag's version. Wired into .gitea/workflows/ci.yml's
  portable job next to the codegen freshness checks (needs fetch-depth: 0
  for tag history).
- Mark remediation batch A Done in
  archreview/2026-08-19/remediation/00-tracking.md.

Verified on macOS: dotnet build clients/dotnet/...slnx (pass), cargo check
--workspace (pass), python -m pytest (168 passed, 1 skipped), gradle test
with JDK 17 (pass, no generated-tree churn to revert this time), gofmt -l
+ go build ./... + go test ./... (clean), and
scripts/check-client-versions.ps1 via pwsh (pass; also verified it fails
correctly when a version constant is rolled back).
2026-08-20 09:22:28 -04:00
Joseph Doherty 058d8d529f fix(GWC-32): make WorkerClient disposal a terminal transition
DisposeAsync is the third path that completes the outbound, staging and event
channels, but unlike MarkClosed and SetFaulted it left _state untouched. A
client disposed while still Ready therefore kept reporting
IsTerminalState() == false, so a read loop already inside DispatchEnvelope when
the staging channel was completed took StageWorkerEvent's overflow arm: a
spurious ProtocolViolation fault, a worker-event-staging overflow metric, a
redundant kill and a WorkerStopped("ProtocolViolation") metric during an
intentional teardown — on exactly the path an operator inspects after a failed
SessionManager.OpenSessionAsync.

DisposeAsync now takes _syncRoot once, claims _disposed there, and transitions
to Closed only when the state is not already Closed or Faulted, so a
dispose-after-fault keeps Faulted and a double dispose is a no-op. Because the
transition is now real, the dispose path records the worker stop through the
existing RecordWorkerStoppedOnce latch, which also stops the running-worker
gauge leaking when a Ready client is disposed without a prior close or fault.
MarkClosed is deliberately not reused: it would overwrite Faulted with Closed
and would re-run a teardown sequence DisposeAsync performs itself.

The StageWorkerEvent XML doc now names all three completion paths.

Test: DisposeAsync_WhileWorkerIsEmittingEvents_ClosesWithoutFaultingOrOverflowing
disposes a Ready client mid-event-stream and asserts Closed state, zero faults,
zero queue overflows, zero kills and exactly one recorded worker exit. It fails
on the pre-fix code.
2026-08-20 09:22:17 -04:00
Joseph Doherty aa07497cba Merge fix/c3-worker-residuals: WRK-33 observed command-task wrapper, WRK-32 pre-baseline pump + recency contract documented, WRK-34 Watchdog CommandMethod label 2026-08-20 09:21:42 -04:00
Joseph Doherty f1353dda86 chore(plans): mark batch F (WRK-32, WRK-33, WRK-34) done-pending-windev 2026-08-20 09:21:16 -04:00
Joseph Doherty 3679e59eed fix(WRK-34): stop writing the watchdog correlation id into CommandMethod
ReportWatchdogFaultIfNeededAsync built the StaHung fault with
CommandMethod = snapshot.CurrentCommandCorrelationId, so the wire
fault's command_method field carried an opaque correlation id (or
empty) instead of a method name -- a consumer reading CommandMethod as
a method name got a confusing value. Use a stable "Watchdog" label and
fold the stuck command's correlation id (when present) into the
diagnostic message instead. No test asserts CommandMethod for the
StaHung fault; the existing assertions only check Category and that
DiagnosticMessage contains "STA activity is stale", both unchanged.
2026-08-20 09:20:34 -04:00
Joseph Doherty f59c34a544 Merge fix/c3-glauth-doc: TST-31 glauth.md reconciled against config.toml, live WriteSecured credential re-derived, auth outcome logged 2026-08-20 09:20:28 -04:00
Joseph Doherty 20a3eefb1e fix(WRK-32): pre-baseline pump for write-completion correlation + honest contract docs
Write-completion correlation is recency-based per (server_handle,
item_handle): OnWriteComplete carries no per-call token, so the executor
attributes the first completion whose version exceeds a pre-call baseline
to that write. A stale completion from an earlier timed-out write to the
same handle could ride the next write's reply if it was already queued
when the baseline was captured.

Add CaptureCompletionBaseline (Write, Write2, WriteSecured,
WriteSecured2) to run one pumpStep() immediately before snapshotting
CurrentVersion, so an already-queued stale completion folds into the
baseline instead of being available to the next wait. This narrows, but
per the finding cannot close, the correlation gap -- a completion queued
after the pump step and before the COM call returns (or during the wait)
is still indistinguishable from the new write's own completion; closing
it needs an MXAccess-side correlation token that does not exist.

Rewrite the AwaitWriteCompletion doc comment to state the recency-based
contract honestly instead of implying exact per-call matching, and add
the same contract to docs/MxAccessWorkerInstanceDesign.md's
write-completion section.

Deferred: the reply-field comment in
src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_gateway.proto is not
edited here. A comment-only proto change still requires regenerating
every generated artifact (C#, Go, Python, Java, Rust, protoset) in the
same commit per repo convention, and this macOS box cannot verify all
five client generator pins together for a Low-severity doc-only change.
The contract is documented in MxAccessWorkerInstanceDesign.md instead;
the existing proto comment ("concurrent writes... can swap rows") is not
wrong, just less precise than the new doc section.
2026-08-20 09:20:28 -04:00
Joseph Doherty 22d5d52c14 fix(TST-31): reconcile glauth.md with the shared GLAuth directory, fix propagated live-test credential
glauth.md's OPC-perm group table and pre-provisioned-users table described a
directory that no longer exists: all five OPC-perm gids were off by 100
(5501-5505 vs actual 5601-5605), and the user table listed
readonly/writeop/writetune/writeconfig/alarmack accounts that scadaproj's
shared GLAuth (config.toml) never provisioned. The real per-role testers are
opc-readonly/opc-writeop/opc-writetune/opc-writeconfig/opc-alarmack (renamed
to avoid a cn= case-collision with their group), and every dev user's
password is "password" per config.toml's own comment and passsha256 (not
admin123). admin itself is no longer a member of any OPC-perm group in the
shared directory (only GwAdmin + OtOpcUa-Admins) - multi-role is the actual
"covers every gw capability" account now.

The wrong admin123 default had propagated into
MXGATEWAY_LIVE_MXACCESS_WRITE_SECURED_PASSWORD (docs/GatewayTesting.md) and
the WorkerLiveMxAccessSmokeTests.cs literal/comment attributing the rig's
GLAuth-bridged ArchestrA identity - both corrected to password. Also made
the WriteSecured parity test log (never assert) whether AuthenticateUser
succeeded or was rejected: MXAccess parity means a bad credential is a
legitimate failure reply, so it must not fail the test, but a stale rig
credential was silently degrading the nightly live smoke's WriteSecured leg
from an authenticated round-trip to a bare failure-reply-shape check with no
visible signal.

Verified: dotnet build src/ZB.MOM.WW.MxGateway.NonWindows.slnx succeeds
(IntegrationTests builds cleanly on macOS, net10.0, no COM/x86 dependency).

Flips Batch B (TST-31) to Done in the cycle-3 remediation tracker.
2026-08-20 09:19:59 -04:00
Joseph Doherty 93edb1b74d fix(WRK-33): observe the discarded command-task wrapper's fault
TryStartCommandTask discarded ObserveCommandTaskAsync's task without a
fault-observing continuation -- the one gap in the NEXT-04 discipline
applied everywhere else in this file. ObserveCommandTaskAsync only
swallows OperationCanceledException, so any other exception escaping
ProcessCommandAsync's own catch path would fault the discarded wrapper
and surface only as a finalizer-time unobserved-task exception. Wrap it
with the existing ObserveFaultWhenever helper, same as every other
discarded task in this file.
2026-08-20 09:17:21 -04:00
Joseph Doherty ddfd41b06d docs(archreview): cycle-3 remediation plan + tracker (batches A-G, model assignments) 2026-08-20 09:16:52 -04:00
Joseph Doherty 7c5eb062e3 docs(archreview): cycle-3 remediation plan + tracker (batches A-G, model assignments) 2026-08-20 09:16:14 -04:00
Joseph Doherty b75f8b1704 docs(archreview): cycle-3 remediation plan + tracker (batches A-G, model assignments) 2026-08-20 09:15:50 -04:00
Joseph Doherty 6c784e3db7 docs(archreview): cycle-3 remediation plan + tracker (batches A-G, model assignments) 2026-08-20 09:14:40 -04:00
Joseph Doherty f4919ba97c docs(archreview): cycle-3 architecture review (2026-08-19 @ 8b3e3b3)
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m19s
ci / java (push) Successful in 2m25s
ci / portable (push) Successful in 8m21s
Six domain reports + overall synthesis. All 47 cycle-2 Done claims verified
at HEAD with zero false Dones and zero regressions; 34 new findings
(0 Critical, 0 High, 4 Medium, 25 Low, 5 Info): GWC-31..38, WRK-32..34,
IPC-33..39, SEC-37..41, CLI-46..50, TST-31..36.
2026-08-20 09:10:12 -04:00
Joseph Doherty 8b3e3b3b45 Merge feat/sweep-reread-tag-summary: fca978d sweep re-read (17 collateral rationale deletions restored across 10 files) + DashboardGroupTagMapping summary now states the short-name GroupToTag key constraint
ci / windows-x86 (push) Successful in 1m15s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m16s
ci / portable (push) Successful in 8m12s
2026-08-18 09:39:12 -04:00
Joseph Doherty a8a8147a61 chore(plans): record sweep re-read + GroupToTag summary follow-ups as closed
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m20s
ci / java (push) Successful in 2m11s
ci / portable (push) Successful in 35m1s
2026-08-18 09:36:17 -04:00
Joseph Doherty f82dac1906 docs(src): restore substantive rationale prose the fca978d marker sweep deleted
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 37s
ci / java (push) Successful in 2m11s
ci / portable (push) Successful in 19m36s
Targeted re-read of the 203-file fca978d sweep (docs(src): add missing
XML docs and strip tracking-ID comments): a mechanical pre-pass narrowed
1,383 deletions to 68 files / 816 residual prose lines, and a judged
review of every one found 17 collateral deletions across 10 files —
rationale prose deleted alongside resolved markers with no equivalent
surviving anywhere in the tree. Restored (markers stay stripped, per the
sweep's intent):

- SessionManager: the three metrics-accounting invariants (kill-path
  gauge decrement safety, shutdown kill-fallback registry guard vs
  double bookkeeping, SessionClosed-not-SessionRemoved on failed close)
- SessionManagerTests: the matching accounting expectation note and the
  reason-string propagation pins (test summary + FakeWorkerClient.LastKillReason)
- MxAccessGatewayService.AcknowledgeAlarm: the routing remarks (GUID vs
  Provider!Group.Tag vs InvalidRequest; session-less via IGatewayAlarmService)
  — inheritdoc resolves to nothing (proto-generated base is undocumented)
- HubTokenService.Validate: why the hollow-token guard exists
  (non-empty AuthenticationType alone satisfies IsAuthenticated)
- DashboardSessionAdminService: why both broad catches exist (keep raw
  teardown exceptions out of Blazor's error boundary), Close + Kill paths
- WorkerPipeSession.RunAsync: why the factory result throws instead of
  NREing (unambiguous failure; finally-block Dispose can't no-op)
- LmxSubtagAlarmSource: Advise idempotency; Write is always unsecured
  (user id 0), never WriteSecured semantics
- WnWrapAlarmConsumer: the v1-prefix path is what WIN-911-style code uses
- DashboardSnapshotPublisherTests: what the 10ms slack absorbs
  (Task.Delay's coarse Windows timer quantum)
- DashboardBrowseAndAlarmModelTests: why the label text is pinned, not
  just the CSS class

Everything else flagged verified benign: inheritdoc replacements resolve
to equal-or-richer interface docs, or the substance survives relocated.
NonWindows slnx 0W/0E; touched gateway test classes 65/65.
2026-08-18 09:33:59 -04:00
Joseph Doherty 4b6f9077bf docs(dashboard): surface the short-name GroupToTag key constraint in the class summary
The consequence that full-DN keys can never match (the shared LDAP
provider pre-strips groups to short RDN names) was documented only in
MapGroupsToTags's lookup comment; the class-level <summary> now states
it and points there, mirroring DashboardGroupRoleMapping.
2026-08-18 09:20:46 -04:00
Joseph Doherty 21903710e7 Merge feat/followups-tickets: worker suite first fully green on windev (WorkerPipeSessionTests harness fix), check-codegen Check 4 runs on Windows (.exe banner + PS5.1 quoting), Gradle 9 checkGeneratedClean revived, push-driven alarms truncation banner, settings recent-limit rows, Server-0xx doc-resolution audit (two regressions re-fixed), ack leg answered as unavailable-by-configuration, full-DN group-key docs corrected
ci / java (push) Successful in 3m3s
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m23s
ci / portable (push) Successful in 12m13s
2026-08-18 07:33:34 -04:00
Joseph Doherty f4b974eec1 chore(plan): followups-and-tickets as-built record; prior follow-ups closed
ci / windows-x86 (push) Successful in 1m30s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 3m38s
ci / portable (push) Successful in 11m12s
2026-08-18 07:29:15 -04:00
Joseph Doherty fb68bdb699 test(dashboard): settings render test proves label-value pairing
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m30s
ci / java (push) Successful in 3m3s
ci / portable (push) Successful in 12m36s
2026-08-18 07:25:57 -04:00
Joseph Doherty 8ae0c2f3d3 fix(codegen): PS5.1-safe quoting for native-exe args in generate-proto probes
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m26s
ci / java (push) Successful in 3m29s
ci / portable (push) Successful in 11m43s
2026-08-18 07:25:42 -04:00
Joseph Doherty 753d070535 style(worker-tests): field ordering in FakeRuntimeSession
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m31s
ci / java (push) Successful in 3m32s
ci / portable (push) Successful in 23m6s
refreshStaActivityOnCapture was declared between DispatchAsync and the property that
wraps it, rather than with the other instance fields. Moved up to the field block per the
member ordering in docs/style-guides/CSharpStyleGuide.md. Declaration move only — no
behavior, no other edits.
2026-08-18 07:24:39 -04:00
Joseph Doherty db7b1db947 docs(dashboard): GroupToRole/GroupToTag keys are short names — full-DN keys never match the pre-stripped groups
Follow-up to the Server-040 comment restoration (d3ac527). The restored
comment in DashboardGroupRoleMapping records that the shared
ZB.MOM.WW.Auth.Ldap provider hands the mapper groups already stripped to
short RDN names, so a full-DN GroupToRole *key* can never match. Three
doc sites still told operators the opposite, and the tag mapper carried
the same consequence with no comment at all.

- docs/GatewayConfiguration.md: GroupToRole and GroupToTag rows both said
  keys may be 'short CN or full DN — leading-RDN match'. Corrected to
  short group names only, with the reason (pre-stripped groups) and a
  note that the mapper's full-string-then-RDN fallback only matters to
  non-library callers of the IGroupRoleMapper<string> seam.
- docs/GatewayDashboardDesign.md: the login-flow bullet said the user's
  memberOf '(or short CN)' is matched against GroupToRole; it now
  describes what actually happens — groups arrive short and are matched
  against short-name keys.
- DashboardGroupTagMapping: short comment at the lookup pointing at the
  DashboardGroupRoleMapping comment, since it reuses ExtractFirstRdnValue
  and inherits the same full-DN-key-unsupported consequence.

Both doc rows are pre-existing on main — they predate this branch and
were not introduced by the Server-040 work; only their incorrectness was
surfaced by it. Comment/doc-only; no logic changes.
2026-08-18 07:23:59 -04:00
Joseph Doherty 6b5c737b04 docs(toolchain): windev protoc-gen-go-grpc updated to pinned 1.6.2
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m24s
ci / java (push) Successful in 2m54s
ci / portable (push) Failing after 6m48s
2026-08-18 07:23:37 -04:00
Joseph Doherty 1605f542cc docs(probe): align RPC-doc hedging with the findings doc; attribute rig-state inventory 2026-08-18 07:23:33 -04:00
Joseph Doherty 5417222f44 chore(plan): tasks 1-8 completed with review outcomes; task 9 in progress
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m23s
ci / java (push) Successful in 2m38s
ci / portable (push) Successful in 15m28s
2026-08-18 07:13:46 -04:00
Joseph Doherty aaeb86b132 fix(worker-tests): decouple FakeRuntimeSession dispatch safety-net from the inspection window
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m13s
ci / java (push) Successful in 2m26s
ci / portable (push) Failing after 4m56s
The BlockDispatch branch waited 5 real seconds and then proceeded regardless — it does not
branch on the wait's result. The long-in-flight test's inspection loop is bounded by
elapsed time and a frame floor, so on a loaded box (the documented 4-5x slowdown class) it
can plausibly outrun that 5 s. When it does, the reply is emitted mid-window,
AssertNotWorkerFault waves it past, and the reply leg then waits for a reply already gone
by — failing at the 20 s cancellation with no message, on exactly the loaded-box run the
widened windows exist to survive.

The wait is a pure safety net: nothing asserts on it firing, and every test that blocks
dispatch releases it explicitly (ReleaseDispatch, or a WorkerShutdown envelope, both of
which Set the event) — none reaches the timeout on a healthy run. Named it
BlockedDispatchSafetyNet and raised it to 30 s, above any window a test opens and above the
20 s cancellation those tests arm, so a wedged test always fails on its own token with its
own message. Dispose still releases the wait, so teardown never waits on it either.

Inline rationale in the test now states the decoupling and what a close pairing would cost,
rather than asserting the window stays inside a 5 s ceiling. Nothing else changed.
2026-08-18 07:08:47 -04:00
Joseph Doherty bc22792a36 docs(probe): close superseded unblock bullet, hedge the summary, evidence gobject_id, ack-caveat in RPC docs 2026-08-18 07:03:20 -04:00
Joseph Doherty 462850afb7 fix(worker-tests): doc-comment placement + watchdog-window headroom in long-in-flight test
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 17s
ci / java (push) Successful in 2m38s
ci / portable (push) Successful in 8m34s
Three review findings on 7da52b6:

1. The AssertNotFault doc block landed between the predicate-overload ReadUntilAsync's
   doc block and the helper, so the compiler attached the merged block to the helper and
   ReadUntilAsync lost its docs entirely. Helper and its docs moved above ReadUntilAsync,
   whose docs are back where they belong.

2. The compressed watchdog windows (50 ms grace, 100 ms ceiling) reintroduced the
   load-sensitivity the fix removed, one layer down.
   ReportWatchdogFaultIfNeededAsync measures staleness AFTER the heartbeat frame is
   written and flushed over the real named pipe, so a beat whose pipe I/O outlasts the
   ceiling faults a healthy session no matter how fresh the captured activity was. At
   100 ms that is a plausible stall on a loaded box, and this is the one test asserting
   the watchdog NEVER fires. Widened to a 200 ms grace and a 1 s ceiling — still two
   orders of magnitude under the 75 s production default.

   The inspection loop is now bounded by a 2 s window (twice the ceiling, so a fake whose
   activity stopped advancing still accumulates past it and faults) with a 30-frame floor,
   rather than a fixed 30 frames that no longer outran the wider ceiling. The floor keeps
   a window that saw almost no beats from passing as a clean one. Two seconds stays well
   inside FakeRuntimeSession's 5 s dispatch-block ceiling, so the command is still in
   flight for the whole window.

3. AssertNotFault renamed AssertNotWorkerFault, matching the WorkerFault body case it
   tests.

Scenario intent unchanged: long in-flight command, pump refreshing, zero fault frames,
reply delivered. Test-only.
2026-08-18 07:01:16 -04:00
Joseph Doherty d1ae43d0d3 docs(probe): ack-leg third attempt — unavailable by configuration, GUID leg still assumed 2026-08-18 06:56:58 -04:00
Joseph Doherty f57a6ae5ff fix(dashboard): alarms feed doc row + monitor-contract comment + mid-truncation attach test
The In-process page feeds table still described the alarms page's subscription as
provider status only, contradicting the three passages updated in 7b6dfba.

The loop's catch-all comment (and the cadence bullet that repeated it) claimed the
monitor completes a subscriber's stream on restart. It does not: ClearCache pushes
snapshot_status(false) through the still-open channel, and a subscriber is only
completed-with-error on a failed TryWrite, or removed by its own disposal.

The SnapshotStatus arm's priming claim had no test behind it — every push test
attached to an untruncated feed and pushed the edge itself. ScriptedAlarmFeed now
replays an optional open sequence, and a new test attaches to a feed already
primed provider_status then snapshot_status(truncated) and asserts the banner
comes up with no edge pushed after render.
2026-08-18 06:49:20 -04:00
Joseph Doherty d3ac52758c docs(reviews): re-verify doc-only Server-0xx resolutions; re-fix regressions
Audited every Server-0xx finding whose Resolution described a
documentation-only or comment-only change, and spot-checked the doc
sub-claims of otherwise test-backed resolutions. 20 entries annotated
in place (append-only; no historical resolution text rewritten).

Two corrections had not survived and are re-applied:

Server-040: the MapGroupsToRoles lookup-precedence comment moved intact
into DashboardGroupRoleMapping (792e3f9) and was then deleted wholesale
by fca978d, a sweep meant only to strip (Server-NNN) tracking markers.
That also removed a later, substantive paragraph recording that the
shared ZB.MOM.WW.Auth.Ldap provider pre-strips groups to short RDN
names, so a full-DN GroupToRole key is unsupported. Both paragraphs
restored, minus the tracking IDs.

Server-009: the WAL / busy_timeout note vanished when the Storage
section of docs/Authentication.md was rewritten to delegate
connection-factory detail to ZB.MOM.WW.Auth.ApiKeys. The behavior is
still live in the library (confirmed against 0.2.1), so the fix is
prose-only.

Server-011/014/022/023 are annotated as moot rather than regressed:
the IAlarmRpcDispatcher trio was deleted in dc9c0c9 and no stale
'not yet wired' / 'PR A.6/A.7' prose survives in Server source.
Server-038's documented v1 ACL gap was later closed by
IDashboardSessionAcl, so its remarks are current.

Comment/doc-only; no logic changes.
2026-08-18 06:48:45 -04:00
Joseph Doherty 7da52b65b7 fix(worker-tests): model the STA pump at heartbeat capture — WorkerPipeSessionTests long-in-flight repro
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m11s
ci / java (push) Successful in 2m14s
ci / portable (push) Successful in 8m33s
RunAsync_LongInFlightCommandThatKeepsPumping_DoesNotFaultAndDeliversReply failed
deterministically on the Windows box with a StaHung fault whose command_method was
empty and whose staleness was 233 ms — i.e. a fault raised with no command in flight,
before the scenario under test began. FakeRuntimeSession stamps LastStaActivityUtc once,
at construction, and the test only started refreshing it after the blocked dispatch
signalled. Everything between those two points — handshake, STA init, the first
heartbeat — captured a snapshot already stale past the compressed 50 ms grace, with no
correlation id for the watchdog to suppress on, so the watchdog correctly reported the
fake as hung.

The harness, not the product, was wrong: StaRuntime.ThreadMain calls MarkActivity() on
every WaitForWorkOrMessages iteration, so a live worker is never captured stale, idle or
busy. Model that where it belongs — FakeRuntimeSession.RefreshStaActivityOnCapture (opt
in, default off) stamps activity at each CaptureHeartbeat and leaves the rest of the
snapshot alone — and arm it before RunAsync so the first beat is covered. The test-owned
refresh loop goes away with it; a thread-pool loop racing a compressed grace could not
have held the invariant anyway.

Scenario intent is unchanged and slightly stronger: the command still blocks in dispatch
across 30 heartbeats (~600 ms, many multiples of the 100 ms stuck ceiling), no frame may
be a fault, and the reply must still arrive. The reply leg is now fault-checked too
(previously it skipped frames blindly), and the pump keeps running across the release, as
it does in production while the reply is marshalled off the STA. Fault assertions now
report the category and diagnostic message instead of a bare body-case mismatch.

Test-only change; no product code, frame protocol, or STA rule touched.
2026-08-18 06:46:08 -04:00
Joseph Doherty 7b6dfba654 feat(dashboard): alarms page consumes snapshot_status feed frame for the truncation banner
The truncated-snapshot caveat moved from poll-only to push-driven. The page
already held an in-process alarm-feed subscription for the provider badge; it
now also handles the feed's snapshot_status frame, so a capped provider fetch is
caveated when the monitor decides it rather than up to three seconds later.

The poll's assignment stays as the reconcile baseline — both sources read the
same monitor verdict, and the frame is consumed, never synthesized page-side.
StreamAsync primes every subscriber with a snapshot_status frame at open, so a
page attaching mid-truncation needs no priming logic of its own; the loop is
renamed StatusFeedLoopAsync because it now feeds two indicators, not one.
2026-08-18 06:43:23 -04:00
Joseph Doherty 2b1efb5e50 docs(auth): make Authentication.md examples runnable as written 2026-08-18 06:40:11 -04:00
Joseph Doherty a390fe16e2 feat(dashboard): settings page shows RecentFaultLimit and RecentSessionLimit 2026-08-18 06:39:32 -04:00
Joseph Doherty c94c4d43c4 fix(codegen): normalize .exe off plugin version banners so Check 4 runs on Windows 2026-08-18 06:39:00 -04:00
Joseph Doherty df45cb4a37 fix(java-client): checkGeneratedClean via ProviderFactory.exec — Project.exec is gone in Gradle 9 2026-08-18 06:38:35 -04:00
Joseph Doherty fd941c249e docs(plan): followups-and-tickets — 10 tasks closing every recorded follow-up
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m21s
ci / java (push) Successful in 2m11s
ci / portable (push) Successful in 8m53s
2026-08-18 06:36:47 -04:00
Joseph Doherty 45058d57a5 Merge feat/followup-closeout: feed-level snapshot_status truncation frame on StreamAlarms rolled out to all five clients, ShowTagValues gating all three dashboard seams, both-direction codegen Check 3, GroupToTag/UntaggedSessionVisibility settings rows, dashboard_tags on ApiKeysPage, wnwrap probe answers (@COUNT records-in-reply, re-raise GUID replacement, ack leg re-blocked)
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m20s
ci / java (push) Successful in 2m15s
ci / portable (push) Successful in 8m8s
2026-08-18 06:26:30 -04:00
Joseph Doherty f2a422b02f docs(auth): canonical scope strings in the two remaining stale lists
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m18s
ci / java (push) Successful in 2m58s
ci / portable (push) Successful in 8m49s
Both lists predate the scope rename and would mislead anyone creating a key:
CLAUDE.md's Authentication section still named the pre-rename scopes, and
docs/Authentication.md's ops.alice example passed 'read,write', which
GatewayScopes.ValidateScopes rejects outright. Same defect family as the
Build/Test/Run sample fixed in a5f843c.

Recorded as a follow-up: code review finding Server-012 claims it fixed the two
CLAUDE.md lists on 2026-05-18, but neither correction was present — a Resolved
finding is not re-examined, so the sibling Server-0xx doc resolutions want a
spot-check for the same pattern.
2026-08-18 06:13:52 -04:00
Joseph Doherty a5f843cbfb chore(plan): close the record — prior follow-ups annotated, closeout as-built noted
The prior plan's "Follow-ups recorded, not started" block described pre-branch
behavior; every item is now closed, narrowed, or restated with its evidence, so
the block no longer misleads a reader who lands on it first. The stale Rust-guard
bullet is corrected in place rather than deleted: Check 3 always existed, and
saying so is the only way the reader learns what the real (one-directional) gap was.

Also fixes the CLAUDE.md apikey sample, which named a verb the parser has never
accepted ('create'; only 'create-key' exists, no alias), omitted the required
--key-id, and listed non-canonical scope strings that GatewayScopes now rejects
at create time — the sample could not have run.
2026-08-18 06:12:31 -04:00
Joseph Doherty c3c603f169 docs(alarms): describe the real StreamAlarms open sequence, and close TST-16
The final integration review's non-blocker reservations, all documentation
or comment truth except one test arm.

The alarm feed opens provider_status -> snapshot_status -> cached
active_alarm -> snapshot_complete, which is what GatewayAlarmMonitor has
done since the snapshot_status frame landed. Two places still described
the old order: docs/Grpc.md said provider_status arrived *after* the
initial snapshot, contradicting its own snapshot_status section two
paragraphs down, and AlarmFeedMessage's leading proto comment named
neither status frame at all. Both now state the sequence the monitor
emits, so a client author reading either one gets the frame order right.

The proto comment change flows through the generated trees (Contracts,
Go, Java) and the client descriptor set; the Rust vendored copy stays
byte-identical to canonical. Python's generator does not carry proto
comments into its output, so it has no delta.

AlarmsHubPublisherTests' valueless-payload case covered snapshot_complete
and provider_status but not snapshot_status, leaving the newest arm
unpinned against the redaction switch that must ignore it. Added.

WnWrapAlarmConsumer's ack comment led with the 2026-05-01 reading that
-55 tracks the 8-arg overload, then refuted itself six lines later with
the 2026-08-18 probe. It now leads with the observation labelled as
narrower than it reads -- mirroring the correction already in
docs/AlarmClientDiscovery.md -- so the block argues one thing: the 6-arg
call site stays for parity, and rc semantics are per the probe. A
paragraph orphaned by an earlier splice is rewrapped. Comment interior
only; the file compiles on Windows.

TST-16 gets a dated closure note rather than a rewrite: the flag it
called dead was implemented 2026-08-18. GatewayDashboardDesign's /browse
paragraph gains the failed-read carve-out GatewayConfiguration already
documented, so the two agree that a failed read keeps its - placeholder.
2026-08-18 06:09:06 -04:00
Joseph Doherty a95e510015 chore(plan): Task 8 verification-green at 90331b6 — residuals recorded 2026-08-18 05:56:45 -04:00
Joseph Doherty 47d74032e6 chore(plan): Task 7 complete — probe retry answered Q2 and the re-raise leg, ack leg re-blocked with evidence 2026-08-18 05:48:10 -04:00
Joseph Doherty 540a1188aa docs(alarms): stop describing the wnwrap ack as working, and say the re-raise replaces the record
Follow-up to 90331b6. Three comment/prose corrections, no behaviour change.

AcknowledgeByName's comment still said the 6-arg overload "works and reaches the
alarm-history path correctly", which the same commit's own findings contradict in
three other places. It now says what was observed: rc=0 means accepted, not
applied — the 2026-08-18 probe acked a live alarm six ways and the snapshot, the
OPERATOR_NAME field, and the extension's .Acked attribute all stayed put. The -55
tracks the consumer, not the overload. Subscribe's comment gets the same
treatment: "lets AlarmAckByName succeed" becomes "return rc=0".

AlarmProbeFindings.md said the re-raise arrives as "a separate record" alongside
the returned one, which reads as coexistence and is wrong. The snapshot carries
one record per tag: the cap=1024 replies bracketing the re-raise are both
elementCount=3 (one per TestMachine_00{1,2,3}) at an identical 1613 bytes, and the
old GUID is absent from the later one. The re-raise replaces the record, so a
single poll spanning it sees the Clear and the Raise together.

Worker diff verified strictly comment-only; builds x86 on windev, 0W/0E.
2026-08-18 05:46:35 -04:00
Joseph Doherty 90331b622c docs(alarms): second wnwrap probe attempt — secured-write verb unblocked the rig, answering @COUNT and the re-raise GUID leg
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m23s
ci / java (push) Successful in 2m25s
ci / portable (push) Successful in 24m49s
AuthenticateUser("Administrator", "") + WriteSecured raises the alarm UDAs that
plain Write could not touch (SecurityError detail=1008), so the 2026-08-17 blocker
was the verb, exactly as that run's own Unblocking list predicted.

Two of the three open questions are now observed rather than assumed:

- ALARM_RECORDS/@COUNT reports the records in the reply, not the total active
  count. With three alarms active it read 1 at cap 1 and 2 at cap 2. There is no
  exact truncation signal to switch to, so IsTruncatedFetch's conservative rule is
  the design rather than a placeholder — behaviour unchanged, only the comments.
- Clear-then-re-raise mints a new GUID; the ALM->RTN leg keeps its GUID
  (reconfirming the 2026-05-01 capture). ComputeTransitions already reads the
  re-raise correctly as one instance ending and another beginning.

The acknowledge leg stays unobserved for a narrower reason: every wnwrap ack
surface is inert on this rig. AlarmAckByName returns 0 from the ack-only consumer
and -55 from the SetXmlAlarmQuery-applied one, for both the 6-arg and 8-arg forms,
and neither the snapshot STATE, OPERATOR_NAME, nor the extension's own .Acked
attribute moves. That corrects AlarmClientDiscovery.md, which read the zero return
as a working ack.

Comment- and prose-only; no behaviour change. The three throwaway probes ran from
the windev CI clone and were deleted; that clone is a clean tree at ab3ff16.
2026-08-18 05:40:11 -04:00
Joseph Doherty ad30adca2f chore(plan): Task 2 complete — five-client snapshot_status rollout approved 2026-08-18 05:33:25 -04:00
Joseph Doherty 4d4f443c27 fix(clients): review nits — alphabetize the Java CLI alarm imports, cover the Rust snapshot-status render
The Java import block's AlarmFeedMessage already sorted after
AlarmProviderStatus before c748361; adding AlarmSnapshotStatus widened the
gap. Order all three alphabetically.

The Rust CLI tests the sibling ProviderStatus render path but not the new
SnapshotStatus arm, so the summary string and the JSON shape were both
uncovered. Add the matching test over alarm_feed_message_summary and
alarm_feed_message_to_json.
2026-08-18 05:33:00 -04:00
Joseph Doherty c7483615cf chore(clients): roll out feed-level snapshot_status to all five clients
Task 1 added AlarmSnapshotStatus and AlarmFeedMessage.snapshot_status = 5.
Carry it downstream from the canonical Contracts protos:

- Rust vendored protos under clients/rust/protos, refreshed byte-identical
  (build.rs falls back to them for out-of-repo tarball builds)
- client descriptor set (protoc 34.1 pin)
- Go (protoc-gen-go v1.36.11 / protoc-gen-go-grpc 1.6.2)
- Python (grpcio-tools 1.80.0 pin)
- Java (gradle generateProto)

.NET needs no regeneration: the client compiles against the Contracts
Generated/ output committed with the proto change.

The hand-written CLI feed renderers switch on the payload oneof, so codegen
alone does not carry the arm. Add snapshot-status to the .NET, Go, Rust, and
Java renderers; the .NET and Go renderers were also missing provider-status,
which has been on the wire since the provider-mode work, so add it there too.
Java's renderer is an exhaustive switch expression and did not compile until
the new case landed. The Python CLI renders generic protobuf-JSON and needs
no change.

Each client README gains a paragraph on the feed-level frame next to its
existing from_truncated_snapshot paragraph: it arrives at stream open after
provider_status and before the cached active_alarm frames, then on every
verdict change including the clearing frame a monitor restart emits, so a
live consumer can track set completeness without polling QueryActiveAlarms.

GatewayDashboardDesign: list the two new payload cases the AlarmsHub forwards,
and — separately — record the GroupToRole / GroupToTag / UntaggedSessionVisibility
rows the settings page already renders but the bullet list omitted.
2026-08-18 05:26:06 -04:00
Joseph Doherty d13144a9d7 chore(plan): Task 3 complete — ShowTagValues coverage approved across all three seams 2026-08-18 05:21:45 -04:00
Joseph Doherty 155c9d619f chore(plan): Task 6 complete — dashboard_tags display and create input approved 2026-08-18 05:21:21 -04:00
Joseph Doherty 51b0a8f55f chore(plan): Task 1 complete — feed-level snapshot_status approved 2026-08-18 05:21:11 -04:00
Joseph Doherty e50ec493cb docs(config): ShowTagValues /browse clause — redaction applies to successful reads only 2026-08-18 05:20:25 -04:00
Joseph Doherty 312e072070 docs(proto): spell out the monitor-restart clearing emission on AlarmSnapshotStatus 2026-08-18 05:19:31 -04:00
Joseph Doherty fab600d3b0 fix(dashboard): de-duplicate dashboard_tags typed into the API-key create form
The form split tags with the shared ParseList and attached the result verbatim,
so "team-a, TEAM-A" persisted as two entries and the constraints column read
dashboard_tags=[team-a, TEAM-A] — one grant reported as two on the page whose
job is to show what a key was granted. Enforcement never saw it (a session holds
its tags in a case-insensitive set), which is exactly why the display was the
only place it could surface.

De-duplicates ordinal-ignore-case at the attach point only, first spelling
winning, matching ApiKeyAdminCommandLineParser.ParseDashboardTags. ParseList is
untouched: the five glob lists are matched literally, so near-duplicates there
are not necessarily the same rule and must survive verbatim — pinned by a test.

The help text claimed to mirror the CLI flag; it now claims only the shared
separators and the dedupe, since the form still drops an empty segment silently
where the CLI hard-fails. A browser form has no exit code to fail with, so that
difference stays, and Authorization.md now records it.
2026-08-18 05:19:02 -04:00
Joseph Doherty 1ea6f60ea2 fix(dashboard): keep the '-' placeholder on failed /browse reads instead of '[redacted]' 2026-08-18 05:18:47 -04:00
Joseph Doherty bf7b44c2d7 chore(plan): Task 5 complete — settings rows approved with comparer fix 2026-08-18 05:15:37 -04:00
Joseph Doherty ff16c69d59 fix(config): propagate GroupToTag's comparer into the effective-config projection 2026-08-18 05:14:58 -04:00
Joseph Doherty c037d9960d feat(dashboard): ApiKeysPage lists and accepts dashboard_tags constraints
The constraints column enumerated only the eight positional ApiKeyConstraints
members, so a key whose sole recorded policy was a dashboard tag summarised to
an empty string and rendered as "-" — the same cell a key with no policy at
all gets. ApiKeyConstraints.IsEmpty counts DashboardTags, so that key is not
unconstrained, and the column was quietly telling operators otherwise about a
grant that decides who can watch a session's events.

The create form had no dashboard-tags input either, so tagged keys could only
be minted from the apikey create-key CLI. Adds the field beside the other
constraint lists (same ParseList separators) and attaches it through the
record's init-only member, since it postdates the eight-member constructor.

CreateModel, OpenCreateDialog and TryBuildCreateRequest widen to internal for
the new render tests: the create form is behind a click and static rendering
cannot dispatch one. That is the assembly's existing InternalsVisibleTo seam.
2026-08-17 07:17:43 -04:00
Joseph Doherty fccf75324b feat(alarms): feed-level snapshot_status truncation frame on StreamAlarms 2026-08-17 07:16:51 -04:00
Joseph Doherty 094f2ffee4 feat(dashboard): settings page shows GroupToTag and UntaggedSessionVisibility 2026-08-17 07:15:24 -04:00
Joseph Doherty eff17d177c fix(dashboard): ShowTagValues now gates the alarms hub and /browse live values 2026-08-17 07:14:40 -04:00
Joseph Doherty 222b01f488 chore(plan): Task 4 complete — Check 3 reverse sweep approved 2026-08-17 07:07:09 -04:00
Joseph Doherty 711793ffdc ci(codegen): Check 3 sweeps both directions — canonical protos missing a vendored copy now fail 2026-08-17 07:04:49 -04:00
Joseph Doherty 338bc20a0a docs(plan): follow-up closeout — feed-level truncation signal, ShowTagValues coverage completion, codegen reverse sweep, dashboard display gaps, probe retry 2026-08-17 07:01:48 -04:00
Joseph Doherty ab3ff1612c Merge feat/deferred-closeout: SEC-25 per-session dashboard event ACL on both seams, alarm-truncation degraded-status signal through proto and all five clients, frame-writer lock-parking closed, dead ReadEventsAsync chain removed, alarm probes recorded blocked with evidence
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m19s
ci / java (push) Successful in 2m8s
ci / portable (push) Successful in 7m56s
2026-08-17 05:34:24 -04:00
Joseph Doherty 870b744e9b docs(closeout): reviewer's two residual-record asks — alarms-hub redaction gap named, ConstraintText artifact recorded; all 12 tasks complete
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m18s
ci / java (push) Successful in 2m17s
ci / portable (push) Successful in 8m13s
2026-08-17 05:27:31 -04:00
Joseph Doherty b621d692d0 docs+test(closeout): final-review reservations — stale ACL prose, worker test gaps, config sample fix
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m14s
ci / java (push) Successful in 2m10s
ci / portable (push) Successful in 8m31s
2026-08-17 05:23:23 -04:00
Joseph Doherty f5a58d884b docs(plans): closeout as-built record; prior plan's out-of-scope table closed 2026-08-17 05:03:02 -04:00
Joseph Doherty d05f38b661 docs(plans): closeout tasks 1-10 complete
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m20s
ci / java (push) Successful in 2m13s
ci / portable (push) Successful in 8m2s
2026-08-17 04:54:36 -04:00
Joseph Doherty 64da630258 docs(rust-client): field-access notation fix in the truncation note (review nit) 2026-08-17 04:52:20 -04:00
Joseph Doherty 1d8a4a6442 test(dashboard)+docs: SEC-25 live-LDAP ACL coverage; design marked implemented
The per-session dashboard event ACL shipped in 693a78d + 7ec0b35 with unit
coverage over a fabricated principal. What a fabricated principal cannot show is
that the group names the shared directory actually returns -- short RDN values,
not DNs -- are the ones Dashboard:GroupToTag keys match. Two [LiveLdapFact]s
close that: gw-viewer binds for real, its GwReader membership grants team-a, and
IDashboardSessionAcl then admits a team-a-tagged session and refuses a
team-b-tagged one; multi-role takes the Administrator bypass. The mapping is
config-side only -- no GLAuth entry, group, or membership was added, and
glauth.md records that explicitly so a future reader does not go looking for a
directory change that never happened.

multi-role is a member of GwReader as well as GwAdmin, so it holds team-a too.
Its bypass is therefore asserted on team-b and on the untagged session -- the two
it would lose if the Administrator branch were ever dropped -- rather than on
team-a, which would pass either way.

One cheap hardening from a prior review: a GatewayOptionsTests case binds
Dashboard:GroupToTag through a real ConfigurationBuilder and looks the group up
mis-cased. The property initializer seeds an OrdinalIgnoreCase dictionary, but
only the binder decides whether that instance survives; if it did not, a
mis-cased group name from the directory would grant no tags and the ACL would
deny with no diagnostic.

Docs follow the shipped shape: docs/Sessions.md gains the session-tag model
(owner-key sourced, immutable, visibility-not-access), gateway.md and CLAUDE.md
gain the ACL in their dashboard-auth paragraphs, and three
GatewayDashboardDesign.md passages that still described the ACL as outstanding
now describe both gated seams and the decision order. GatewayConfiguration.md's
ShowTagValues row no longer claims the redaction is the only thing between a
Viewer and another session's values -- it is now the second of two independent
layers. gateway.md's hub-token lifetime corrected 30 minutes -> 5, matching
HubTokenService. Authentication.md disambiguates --dashboard-tags as the only
constraint flag that splits on commas. The plan doc header is Implemented; its
as-built section 12 already existed and is not duplicated.

Verified: NonWindows.slnx builds clean; GatewayOptions/DashboardSessionAcl/
EventsHub filters 37/37; the live-LDAP suite skips cleanly without the env var
and runs 7/7 green against the shared GLAuth with it.
2026-08-17 04:48:34 -04:00
Joseph Doherty d9ea8a81f1 chore(clients): regenerate for alarm truncation fields; READMEs note the degraded flag
Task 8 added ActiveAlarmSnapshot.from_truncated_snapshot = 16 and
QueryActiveAlarmsReplyPayload.snapshot_truncated = 2. Regenerate every
downstream binding from the canonical Contracts protos:

- client descriptor set (protoc 34.1 pin)
- Go (protoc-gen-go v1.36.11 / protoc-gen-go-grpc 1.6.2)
- Python (grpcio-tools 1.80.0 pin)
- Java (gradle generateProto)
- Rust vendored protos under clients/rust/protos, which build.rs falls back
  to for out-of-repo tarball builds and which must track Contracts

.NET needed no regeneration - the client compiles against the Contracts
Generated/ output already committed with the proto change. No client has a
typed wrapper model around ActiveAlarmSnapshot; all five pass the generated
type straight through, so codegen alone carries the field.

Each client README's alarm section gains a paragraph on the flag: the
snapshot set may omit actives and absence-implies-cleared inference was
suspended, so callers must not reconcile deletions from a truncated set.
Distinguished from the per-record 'degraded' subtag-fallback flag, which it
is easily confused with.
2026-08-17 04:48:20 -04:00
Joseph Doherty b9fb0dd720 fix(alarms): atomic snapshot+truncation read; direct tests for the flag plumbing (review)
Review found AlarmDispatcher.SnapshotActiveAlarms reading the snapshot and the
truncation verdict through two independent lock acquisitions, defended by a
comment claiming read-order made a race "widen only, never narrow". That claim
was false: a not-truncated -> truncated poll landing between the two reads pairs
a stale false with a capped snapshot, which is exactly the false all-clear the
feature exists to prevent. It was safe only because AlarmCommandHandler
STA-serializes consumer calls — an accident of the call graph, not an invariant.

Made the invariant structural instead of documented. IMxAccessAlarmConsumer now
exposes ONE accessor, `IReadOnlyList<MxAlarmSnapshotRecord> SnapshotActiveAlarms(
out bool truncated)`, which implementations must satisfy from a single
acquisition of the lock guarding the retained snapshot — mirroring the write
side, where FoldFetch already updates snapshot and verdict together. The
separate LastSnapshotTruncated property is gone from every layer, so there is no
second read left to pair badly. `out` over a result struct follows the file's
established idiom (FoldFetch, ParseSnapshotXml).

The same threading applies one level up: IAlarmCommandHandler.QueryActive now
carries `out bool snapshotTruncated`, so MxAccessCommandExecutor stamps the reply
payload from the value the records were stamped with rather than reading the
state a second time.

Direct tests for the three hops that were only covered end-to-end:
- AlarmDispatcherTests: truncated consumer snapshot stamps FromTruncatedSnapshot
  on every mapped record, with a complete-snapshot control, plus an assertion
  that the independent per-record Degraded flag is not dragged along.
- AlarmCommandHandlerTests: the verdict delegates through the dispatcher
  (Theory over both values), and survives a prefix filter that removes every
  record — the case the per-record flag cannot cover.
- AlarmCommandExecutorTests: the reply payload's SnapshotTruncated comes from the
  handler (Theory over both values), including the zero-record case.
The WnWrapAlarmConsumer truncation tests now assert through
SnapshotActiveAlarms(out ...) rather than an internal field, because the pairing
is the contract.

Also: GatewayAlarmMonitor's _snapshotTruncated comment now says "as of the last
full reconcile" rather than implying it tracks the current _alarms contents,
which live transitions keep moving via ApplyTransition between passes.

Detection heuristic still untouched (fetchedRecordCount >= maxAlarmsPerFetch);
no @COUNT parsing, per docs/AlarmProbeFindings.md. Still additive gateway
metadata about our fetch mechanics, not MXAccess behavior — not a parity
deviation, and no event is synthesized.

Gateway: NonWindows.slnx builds clean (0 warnings); ~Alarm filter 107/107 pass.
Worker + Worker.Tests are windev-gated; the signature change was reviewed by
inspection across all 7 IMxAccessAlarmConsumer implementers, all 3
IAlarmCommandHandler implementers, and every call site.
2026-08-17 04:39:33 -04:00
Joseph Doherty 7ec0b3594c fix(dashboard): close AttachEventsAsync re-entrancy window; pin ACL decision-table corners (SEC-25 review)
Follow-up to the per-session event ACL. Part of that change rode into 693a78d
via a concurrent agent's pathspec-less commit; this commit carries the review
fixes and uses pathspecs on the commit itself so it cannot recur in either
direction.

Gating the page's subscribe seam made AttachEvents asynchronous — it awaits the
authentication state — and that await is a suspension point the synchronous
version did not have. On a rapid A -> B navigation the suspended A continuation
resumes after B's parameter set has run to completion, re-reads the live
SessionId (now B's), and attaches B a SECOND time. The ACL is not bypassed —
the newer attach already cleared that same session — but the fields holding B's
first subscription are overwritten in place, so nothing ever disposes it: its
EventsHubViewerRegistry entry is never released, which keeps the mirror cloning
events for a session the page is no longer watching through that handle, and
its pump is never cancelled. A resource leak the ACL work introduced.

OnParametersSetAsync now claims a monotonic _attachGeneration synchronously,
before its first await, and AttachEventsAsync re-checks it after the await and
before any field write or Subscribe call. A stale attach returns rather than
detaching: it owns nothing, and tearing down there would destroy the newer
attach's subscription. DetachEventsAsync needs no such guard — it captures and
nulls the live fields synchronously before it awaits, so a resumed detach only
unwinds what it already took ownership of. Same dispatcher-owned identity idea
as the existing ReferenceEquals guards in PumpEventsAsync and
MarkDisconnectedAsync, one level up.

The interleaving is not expressible with the static HtmlRenderer idiom the other
page tests use: it renders a root component once and exposes no parameter-update
seam. The new test therefore adds a minimal Renderer subclass whose only job is
to mount a component and drive a second SetParametersAsync into it while the
first is parked on a gated AuthenticationStateProvider. That subclass is the
lone reason for a narrowly scoped BL0006 suppression, justified in place: it is
test-only scaffolding that never ships, and the cost of the warning coming true
is a compile break in one test file on an SDK bump. Confirmed non-vacuous by
mutation — with the generation check disabled the test goes red on the doubled
subscription and the two passing ACL tests stay green.

Two decision-table corners are now pinned rather than implied. Admin x
nonexistent session id resolves to ALLOW, because the admin bypass is evaluated
before the registry lookup; a plausible "look the session up first, it reads
better" refactor would flip it, so a test documents the ordering. EventsHub's
remarks said "an unknown session id is denied" without qualification, which read
as universal; they now state that the bypass is checked first and every rule
below it is a non-Admin rule.

HubTokenServiceTests gains the truly-absent-field case: a hand-built payload
JSON with no Tags key at all, protected through the same purpose, which is the
shape every in-flight token has across the deploy that introduces the field. The
existing test covered present-but-empty, which does not exercise the null
coalesce that stands between a legacy token and a crash on the hub auth path.
ProtectorPurpose became internal so the test cannot drift from the real purpose
string.

Tag-count cardinality cap considered and recorded as a deliberate non-goal.

Build 0 warnings / 0 errors; 48 filtered (ACL/hub/token/page) and 257 dashboard
tests pass.
2026-08-17 04:35:59 -04:00
Joseph Doherty 693a78db7d feat(alarms): structural degraded-status signal for truncated alarm snapshots
The truncation-cliff fix made alarm transitions truncation-safe but silent:
when GetXmlCurrentAlarms2 returns exactly maxAlmCnt records the worker
suppresses absence-implies-Clear inference and says so only in a rate-limited
stderr warning. No client and no operator could tell a complete active set
from a capped one.

Two additive proto3 booleans carry the verdict out:

- QueryActiveAlarmsReplyPayload.snapshot_truncated = 2 (worker IPC reply)
- ActiveAlarmSnapshot.from_truncated_snapshot = 16 (per record)

The per-record field is not an aesthetic choice. QueryActiveAlarms returns a
bare `stream ActiveAlarmSnapshot` with no envelope, header, or trailer, so a
per-record boolean is the only carrier that stays wire-compatible; an envelope
message would change every existing client's stream element type. The reply
payload states it too because a prefix filter can leave zero records and a
truncated fetch with nothing to report still has to say so. The flag means
"this set may be incomplete", never "this record is unreliable" — it is
independent of the subtag-fallback `degraded` field.

Detection is deliberately UNCHANGED: IsTruncatedFetch remains
`fetchedRecordCount >= maxAlarmsPerFetch`. The live probe (docs/AlarmProbeFindings.md,
ce5d8ae) could not verify whether ALARM_RECORDS/@COUNT reports the total active
count or only the records in the reply, so @COUNT is not parsed for detection;
switching to it stays blocked on probe evidence. The probe's comment
annotations in WnWrapAlarmConsumer.cs are preserved.

Reset semantics: not latched. WnWrapAlarmConsumer.FoldFetch replaces the
verdict on every poll under the same lock as the snapshot merge, so the first
sub-cap fetch clears it; GatewayAlarmMonitor.ClearCache drops it with the cache
generation it describes. A caveat that never turns off is one operators learn
to ignore.

Flow: WnWrapAlarmConsumer.LastSnapshotTruncated -> AlarmDispatcher (stamps every
record) / IAlarmCommandHandler (payload) -> MxAccessCommandExecutor reply ->
GatewayAlarmMonitor._snapshotTruncated -> IGatewayAlarmService.SnapshotTruncated
-> DashboardAlarmQueryResult -> AlarmsPage warning banner (render-side only; the
poll loop and DisposeAsync drain are untouched). The public QueryActiveAlarms
RPC forwards worker snapshots unmodified, so the per-record flag needed no
mapper change — a test pins that.

Parity: this describes OUR fetch mechanics — additive gateway metadata — not
MXAccess provider behavior. No event is synthesized and no MXAccess-observable
semantics change, so it is not a parity deviation.

Tests: worker LastSnapshotTruncated set/reset/consecutive-burst (windev-run);
gateway end-to-end truncated reply -> monitor -> public stream, with the
complete-reply control as the load-bearing assertion; AlarmsPage banner
present/absent. Docs: gateway.md alarm surface, docs/DesignDecisions.md entry.
2026-08-17 04:18:34 -04:00
Joseph Doherty b8b7b69ba0 fix(worker): observe detached drain faults per NEXT-04 discipline
Both tasks the detached-lock-wait path starts and discards now carry the
file's fault-observing continuation, factored out of ObserveAbandonedFault as
ObserveFault so the idiom has one definition. DrainDetachedAsync swallows the
drain but its _writeLock.Release() sits in a finally outside that catch, so a
Release that ever throws — a SemaphoreFullException from some future
double-release regression — had no awaiter and would have surfaced on net48 as
TaskScheduler.UnobservedTaskException at finalization instead of an
attributable failure. Same for a throw out of OnDetachedLockWaitSettled.

Review's Minor (distinguishing a cancelled from a faulted lock wait before
tombstoning) is deliberately not taken: a faulted WaitAsync is unreachable
here — nothing disposes _writeLock — so the branch would be untestable new
logic whose only effect is internal state, the caller already receiving the
fault itself from the rethrow. Recorded as a comment at the site instead.

edited on macOS, windev verification pending (plan Task 11). Re-ran the net10
scratch harness over WorkerFrameWriter and the writer suite: 0 warnings,
31/31 pass.
2026-08-17 04:07:44 -04:00
Joseph Doherty ce5d8ae7c2 docs(alarms): wnwrap live-probe findings — GUID identity, ALARM_RECORDS COUNT
Both questions stay open, and the reason is the finding: the dev rig's alarm
UDAs reject a plain MXAccess Write with SecurityError/detail=1008 from the
responding automation object, so no alarm instance can be created to follow
through an acknowledge and no population can be built to overflow a capped
fetch. The rig is otherwise live — objects deployed and on scan, wnwrap
subscribed, GetXmlCurrentAlarms2 returning well-formed XML — which is what
makes the blocker specific and the unblock (engine-side script, or
AuthenticateUser + WriteSecured, or reclassifying the UDAs) actionable.

Comment-only changes in WnWrapAlarmConsumer: scope the GUID-identity claim to
the leg live capture actually covers, and record that ALARM_RECORDS/@COUNT
exists as a candidate exact truncation signal but is deliberately not trusted
because its semantics under a capped reply are unverified. No behavior change.
2026-08-17 04:03:39 -04:00
Joseph Doherty a212e145ac feat(security): DashboardTags on API-key constraints; sessions inherit owner tags (SEC-25 groundwork)
Adds a dashboard event-visibility tag to ApiKeyConstraints, riding in the existing
constraints JSON blob so no auth-store schema migration is needed (design
docs/plans/2026-07-10-dashboard-session-acl-tst15.md sections 3/3.1, open call
settled per its own recommendation). The tag is visibility-only: no read, write,
browse, or subscribe path consults it, and HasRead/HasWriteConstraints ignore it.

GatewaySession gains an immutable, ordinal-ignore-case Tags set stamped at
construction from the owning API key, forwarded by MxAccessGatewayService.OpenSession
from the resolved ApiKeyIdentity — never from the wire request, so a client cannot
label its own session with another tenant's tag. ISessionManager gains a tag-carrying
OpenSessionAsync overload whose default implementation forwards to the tagless one, so
an implementation that does not model tags opens an untagged (least visible) session.

apikey create-key gains --dashboard-tags team-a,team-b (repeatable, trimmed,
de-duplicated; an empty segment is rejected rather than dropped) and list-keys prints
the tags column. No enforcement yet — the EventsHub ACL that consumes the tag is a
later change.
2026-08-17 03:58:14 -04:00
Joseph Doherty 9130994736 perf(worker): unpark awaited control-frame writers from the winning drain pass
WriteAsync enqueued its frame and then contended unconditionally for the write
lock, so a caller that lost the race stayed in WaitAsync until the winning
drainer released — even though that winner writes, flushes, and completes the
loser's control frame at the control-to-event class boundary, part-way through
its pass. The boundary flush made the delivery point honest; the awaited task
was still charged for the whole event backlog it had just been flushed ahead of.

WriteAsync now awaits its own frame's completion racing the lock acquisition.
Completion first: the caller returns at its frame's delivery point and the
outstanding acquisition is detached, not dropped — a continuation drains
whatever is queued and releases, so the lock is never acquired and silently
held and a frame enqueued between the previous drainer's last dequeue and its
release is still written. Lock first: drain as before. Cancellation keeps the
WRK-22 tombstone semantics exactly, and a wait cancelled after the caller has
already detached releases nothing (SemaphoreSlim hands no count to a wait it
cancels), so no count leaks and no queued frame is stranded. A token that fires
after the frame's completion won the race changes nothing — the frame was
delivered. WriteBatchAsync deliberately keeps the plain wait-then-drain shape:
its last completion resolves at the end-of-pass flush anyway.

Three tests: the latency win (a control caller returning while the winning
WriteBatchAsync event burst is demonstrably still blocked mid-pass), a
mixed-priority concurrency soak pinning exactly-once writes and a single
drainer, and the cancel-after-detach corner (a wrongly released count would
surface as the drainer's own Release throwing SemaphoreFullException).

edited on macOS, windev verification pending (plan Task 11). Verified here by
compiling and running WorkerFrameWriter plus the writer suite against net10.0
in a scratch harness: 31/31 pass, and the two behaviour-pinning tests fail
against the pre-change parked implementation.
2026-08-17 03:55:05 -04:00
Joseph Doherty c79aaaf9eb feat(dashboard): GroupToTag / UntaggedSessionVisibility config (SEC-25)
Groundwork for the per-session dashboard event ACL (docs/plans/2026-07-10-dashboard-session-acl-tst15.md 3.2): a dashboard group can now grant visibility tags, and untagged sessions default to AdminOnly. Enforcement lands with the EventsHub ACL; nothing consumes the grant yet.

GroupToTag is deliberately uncoupled from GroupToRole - a group may appear in either map, both, or neither - and is validated for shape only. Tags gate dashboard event visibility, never data access.
2026-08-17 03:46:56 -04:00
Joseph Doherty fa9eb0c0b4 refactor(sessions): remove the dead ISessionManager.ReadEventsAsync chain
ISessionManager.ReadEventsAsync had zero production call sites: the worker
event channel is drained once by GatewaySession.MapWorkerEventsAsync (the
distributor pump), and every consumer — gRPC subscribers, the dashboard
mirror, the alarm monitor — attaches to the distributor. The interface
member, SessionManager's forwarder, and GatewaySession.ReadEventsAsync are
gone; IWorkerClient/WorkerClient.ReadEventsAsync is untouched, it is the
live worker-channel claim.

No test was removed or rewired: nothing invoked the member through the
interface. Nine ISessionManager test fakes carried a required-member stub
(seven threw NotSupportedException or yielded nothing; EventStreamServiceTests
and GatewaySessionDashboardMirrorTests forwarded to the session; the two
MxAccessGatewayService fakes yielded their Events list) — all nine stubs were
deleted. The MxAccessGatewayService suites' streaming tests already run
through FakeEventStreamService, which reads the same Events list, so their
coverage is unchanged; only the now-inaccurate doc comments on Events /
LastReadEventsSessionId were reworded.

The MapWorkerEventsAsync comment no longer describes a twin to keep in step;
it now states the single-reader claim directly. docs/Sessions.md drops
ReadEventsAsync from the SessionManager member list and from the Run-state
prose. The 2026-08-15 deferred-remediation as-built note records the removal.
2026-08-17 03:46:41 -04:00
Joseph Doherty af9f185d32 docs(plans): deferred-closeout plan — SEC-25, truncation signal, lock-parking, dead chain, alarm probes 2026-08-17 03:41:59 -04:00
Joseph Doherty ac3f04f6ac Merge perf/deferred-remediation: deferred perf findings closed — in-process dashboard feeds, worker teardown/frame-writer fixes, value-cache clone removal, first green Windows secrets test
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m22s
ci / java (push) Successful in 2m37s
ci / portable (push) Successful in 9m0s
2026-08-16 04:38:53 -04:00
Joseph Doherty dcbec978ee fix(dashboard): alarm poll loop retries through faults and surfaces them; correct stale session-events empty-state copy
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 28s
ci / java (push) Successful in 2m23s
ci / portable (push) Successful in 9m26s
2026-08-16 04:32:54 -04:00
Joseph Doherty a37c304f26 chore(plans): tasks 13-14 complete; as-built note on the hub-connection-factory removal
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 25s
ci / java (push) Successful in 2m18s
ci / portable (push) Successful in 9m2s
2026-08-16 04:22:34 -04:00
Joseph Doherty 7cfb2e727d fix(dashboard): parallel bounded alarm drains, best-effort disposal catch-alls, ConfigureAwait alignment 2026-08-16 04:20:59 -04:00
Joseph Doherty 3faa272db9 fix(dashboard): bounded alarm drains, feed resubscribe, live pill, drop dead hub factory; doc corrections 2026-08-16 04:12:25 -04:00
Joseph Doherty e1ff05c605 docs: deferred-table closure, stale-comment sweep, as-built execution record 2026-08-15 21:33:28 -04:00
Joseph Doherty bb7aa6209f chore(plans): tasks 1-12 complete through review chains
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 56s
ci / java (push) Successful in 2m12s
ci / portable (push) Successful in 8m19s
2026-08-15 21:25:15 -04:00
Joseph Doherty 3ef56be2dd fix(worker): unconditional fault observation for abandoned pipe I/O; exception-total transport dispose 2026-08-15 21:21:25 -04:00
Joseph Doherty e913dab5db fix(worker): session-owned stream disposal unblocks and observes the net48 pipe read at teardown 2026-08-15 21:06:03 -04:00
Joseph Doherty aac79579ab perf(worker): control-frame completions resolve at the class-transition flush, not after the event batch
The two-class writer already got control bytes out ahead of a queued event
backlog, but a frame counts as delivered only once flushed, and the drain
deferred its single FlushAsync — and every TrySetResult — to the end of the
pass. A heartbeat, command reply, fault, or shutdown ack was therefore written
first and completed last, behind up to a full 128-frame event batch.

The drain now records each frame's priority class on PendingFrame and flushes
at every control-to-event boundary, completing and clearing the written set
there. Cost stays bounded: a pure-event pass still pays exactly one flush, a
run of control frames still pays one for the run, and only a pass that mixes
both classes pays a second — never one flush per control frame, the
syscall-per-heartbeat cost WRK-12 removed.

A boundary flush that itself fails is a new failure window and is handled like
the end-of-pass flush failure, additionally failing the event frame the drain
had already claimed off its queue and every frame still queued. Frames a
boundary flush completed leave the written set, so a later failure in the same
pass can no longer reach back and fail an already-delivered control frame.

The awaited task of a caller that lost the write-lock race is still bounded by
the winning drainer's pass — that enqueue-then-contend parking is unchanged and
now documented on WriteAsync and in docs/WorkerFrameProtocol.md.
2026-08-15 21:05:57 -04:00
Joseph Doherty 9871d4772d perf(worker): value cache borrows the write-once event's instances — three clones per OnDataChange removed
MxAccessValueCache.Set deep-copied the Value (recursive for an MxArray), the
SourceTimestamp, and the Statuses RepeatedField (container plus every
MxStatusProxy) on every OnDataChange. The aliasing audit found all three
removable: the sink enqueues the event first — which stamps
WorkerSequence/WorkerTimestamp inside the queue lock — and only then runs the
postPublish hook that reaches Set, so the event is write-once by then and the
queue's ownership invariant forbids later mutation. The producer never reuses
instances (fresh MxEvent per mapper call, fresh MxValue per convert), and the
alias already existed on the read side: MxAccessSession.SucceededRead puts the
cache's own Value/SourceTimestamp/status references on every BulkReadResult,
which the worker only serializes onto the IPC pipe.

Set and CachedValue now carry the ownership contract: the cache holds borrowed
references into an enqueued, write-once MxEvent; consumers may read and
serialize, never mutate. Mutation would corrupt the still-queued event AND
invalidate QueuedEvent.Size — the enqueue-time memoized serialized size the
byte-budgeted Drain charges — so a grown message could overshoot the negotiated
frame max and fault the session with MessageTooLarge. MxAccessEventQueue's
class remark, which claimed the cache keeps an independent snapshot, is
corrected to point at the borrow.

MxAccessWriteCompletionCache.Record keeps its parallel statuses.Clone()
deliberately, with a cross-reference explaining why: it takes a bare
RepeatedField whose provenance its signature cannot constrain, and it is on the
command-rate write path, not the streaming hot path.

Tests: Set_StoresIndependentSnapshot_UnaffectedByLaterEventMutation codified
the invariant being reversed, so it is replaced by
Set_BorrowsTheEventsOwnInstances_ByOwnershipContract (Assert.Same on Value,
SourceTimestamp, and the status row). Adds the missing cached-read-path test to
MxAccessCommandExecutorTests — nothing in the worker exercised was_cached ==
true end to end — asserting the cache hit, reference identity out to the
BulkReadResult, and that no COM call is made for the read.

Not built or tested here: these are net48/x86 worker files that cannot compile
on the macOS tree. Verification is deferred to the windev gate.
2026-08-15 21:03:35 -04:00
Joseph Doherty 53881c6220 docs(dashboard): in-process page feeds, two-tier idle gating, hubs as the external surface 2026-08-15 20:51:36 -04:00
Joseph Doherty 756296886b fix(dashboard): generation-scoped idle gate in UnsubscribeAsync — no zero-subscriber pump survives 2026-08-15 20:43:17 -04:00
Joseph Doherty b0f5941e46 fix(dashboard): generation-tagged feed subscribers survive pump teardown races; drain-timeout observability 2026-08-15 20:32:07 -04:00
Joseph Doherty 38dd7678f2 fix(dashboard): guard stale-session batches inside the renderer dispatch; register IDashboardSessionEventSubscriber 2026-08-15 20:26:42 -04:00
Joseph Doherty 10406a3541 fix(dashboard): mutate provider-status state inside the renderer dispatch 2026-08-15 20:17:07 -04:00
Joseph Doherty e245237c2b feat(dashboard): in-process snapshot feed replaces the pages' loopback /hubs/snapshot hop 2026-08-15 20:16:29 -04:00
Joseph Doherty e23f816bfb feat(dashboard): in-process session event subscription feeds the viewer registry — SessionDetailsPage drops its /hubs/events hop 2026-08-15 20:15:52 -04:00
Joseph Doherty d44fe1d6b5 feat(dashboard): AlarmsPage reads provider status from IGatewayAlarmService in-process 2026-08-15 20:10:51 -04:00
Joseph Doherty 59420d8568 refactor(sessions): _subscribers to plain Dictionary — every access is under _lifecycleLock 2026-08-15 20:07:15 -04:00
Joseph Doherty 94dbe9f1af perf(sessions): fold the ReadEventsAsync pass-through into MapWorkerEventsAsync — one fewer iterator per event 2026-08-15 20:07:11 -04:00
Joseph Doherty 935f002dbf perf(grpc): consume the subscriber channel directly in StreamEventsAsync — drops the ReadAllAsync iterator hop 2026-08-15 20:06:48 -04:00
Joseph Doherty 25f07f89dd fix(tests): clear the SQLite pool before deleting the secrets path-guard temp dir — Windows sharing violation 2026-08-15 20:04:59 -04:00
Joseph Doherty a756e47682 docs(plans): deferred-findings remediation plan — 14 tasks over the six deferred findings + the Windows secrets-test bug 2026-08-15 20:03:52 -04:00
Joseph Doherty 15f188e4f9 Merge perf/review-remediation: full 2026-08-15 perf-review remediation — pipe buffers, signal-driven drains, STA message-driven waits, viewer-gated dashboard mirror, pull-model metrics, async audit pipeline, parallel teardown, truncation-safe alarms 2026-08-15 19:25:09 -04:00
Joseph Doherty 2faf243189 chore(plans): all 25 tasks complete — final integration review: ready to merge
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m19s
ci / java (push) Successful in 2m6s
ci / portable (push) Successful in 8m57s
2026-08-15 18:07:33 -04:00
Joseph Doherty f4b065b9f6 docs(worker): reviewer follow-up comments and tests from the remediation reviews
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m18s
ci / java (push) Successful in 2m11s
ci / portable (push) Successful in 11m5s
The worker-side half of the review tail. Tests and comments only — nothing here
changes worker behavior, and none of it compiles on the macOS tree (net48/x86),
so it was reviewed line by line against the already-windev-validated files.

- MxAccessHandleRegistryTests gains the multi-candidate case behind
  MxAccessSession.TryGetCachedReadFor's fall-through: one tag under two item
  handles, the lower registered-but-unadvised and the higher advised. Asserted
  at the registry rather than the session because the session's read path needs
  a live MXAccess COM instance; what the registry owes the scan is the stable
  ascending candidate order and a per-item-handle (not per-tag) advice index,
  and both are pinned here along with the fall-through contract in prose.
- A single adversarial lifecycle test — register, advise, re-register the same
  item handle under a new tag, unadvise, unregister the server — asserting every
  index agrees after each step. The individual transitions were already covered;
  what was not was that they compose, and a stale entry in any one index
  resurrects a handle MXAccess has already retired.
- StaWaitHelperTests.WaitForSignalOrMessages_PreSignalledHandle_ReturnsImmediately
  drains pending messages first, like the other two wait tests. Without it a
  stale message can end the wait instead of the handle, failing the
  signal-consumed post-condition for an unrelated reason.
- GatewayTesting.md records the two findings from the Task 24 windev gate:
  SecretsStorePathGuardTests.CreateBuilder_AcceptsSecretsStoreOutsideContentRoot_AndCreatesIt
  fails deterministically on Windows on main too (SQLite pooling holds secrets.db
  open across the cleanup's recursive delete; pre-existing, tracked separately),
  and the StaWaitHelper timing tests' flake signature on a loaded box is a
  message wake — the helper working as designed — not a broken wait.
2026-08-15 17:56:16 -04:00
Joseph Doherty dc2df628e3 chore(followups): reviewer-recommended tests, comments, and hardening from the remediation reviews
The remediation reviews approved every task but left a tail of small notes.
This lands the gateway-side half of them.

Hardening (behavior changes, all narrow):

- BuildFilteredWriteBulkCommand's unreachable default case failed OPEN: a
  fifth bulk-write kind added upstream without a filter case here would have
  shipped the DENIED entries to the worker while reporting them denied to the
  caller. It now throws UnreachableException.
- SqliteCanonicalAuditStore.ListRecentAsync no longer throws on a row it
  cannot date. The retention sweep deliberately preserves such rows (SQLite's
  datetime() yields NULL, so the DELETE never matches), which guaranteed the
  dashboard's recent-audit view would meet one eventually and lose the whole
  page to it. The row is now reported at DateTimeOffset.MinValue with every
  other column intact, behind an optional logger.
- The audit drain loop's finally now completes the channel writer alongside
  detaching the drain, so a producer that raced past the attached check takes
  the write-through branch instead of stranding its event in a buffer nobody
  reads until shutdown. TryComplete is idempotent, so StopAsync is unaffected.

Tests:

- MapCommandReply ownership (Assert.Same on the inner reply), mirroring the
  existing MapEvent ownership test.
- Redactor key-id length boundary at exactly 64 and 65 characters, pinning
  which way it fails. Nothing validates key-id length at creation, so
  docs/Diagnostics.md's "which no issued key id does" is now stated as the
  heuristic it is.
- ApiKeyFailureLimiter.Reset with a PartitionResolution whose partition was
  evicted between the Check and the Reset: inert, and clears nobody else's
  block.
- Constraint-cache concurrency stress: the cap is enforced by the inserting
  thread, so overshoot must be transient and proportional to the in-flight
  inserters, and the cache must settle at or under the cap.
- ListRecentAsync against a raw-SQL undateable row.

Comment/doc accuracy:

- EventsHubViewerRegistry.ReleaseConnection records that it relies on
  SignalR's default sequential per-connection dispatch
  (MaximumParallelInvocationsPerClient = 1).
- A PERF(followup) note on Invoke's double session resolve and why removing it
  needs a SessionManager overload.
- SessionEventDistributor: the volatile-field comment named the pump as the
  lock-free reader, but the pump's single capture point is inside _replayLock;
  the genuinely lock-free reader is SubscriberCount. OnSubscriberOverflow's
  "cannot be observed here" now excepts the DisposeAsync abandon path. The
  churn test names its ConcurrentDictionary bucket-order assumption and that a
  violation surfaces as a read timeout, not a silent pass.
- The two "restores the sequential drain's behavior" claims (SessionManager,
  docs/Sessions.md) were wrong: the sequential drain leaked too, because
  KillWorkerAsync's entry ThrowIfCancellationRequested aborted the whole loop
  on the first session for zero kills. Reworded to "fixes a leak the
  sequential drain also had", with the sweep-bound/shutdown-unbound
  ParallelOptions asymmetry explained.
- ISessionManager.ShutdownAsync's token doc: it degrades the drain to a kill
  sweep rather than cancelling it, with the bounded overrun stated.
  SessionShutdownHostedService.StopAsync records that its cancellation-logging
  branch is now unreachable.
2026-08-15 17:54:31 -04:00
Joseph Doherty 7755745f2f chore(plans): Task 24 windev gate green — slnx+x86 builds clean, worker 499/499, gateway 1039/1040 (1 pre-existing main failure, 1 isolated-pass load flake) 2026-08-15 17:44:27 -04:00
Joseph Doherty b2d8dd70ed chore(plans): Phase B implementation complete; as-built note for Task 17
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m12s
ci / java (push) Successful in 2m26s
ci / portable (push) Successful in 7m56s
2026-08-15 17:35:05 -04:00
Joseph Doherty 58d97ad4e8 perf(worker): memoize event frame size at enqueue; drain stops sizing under the STA's lock 2026-08-15 17:27:47 -04:00
Joseph Doherty b5ea6bb461 fix(alarms): fetch/poll ceilings; truncation-semantics docs; log-format conformance 2026-08-15 17:19:41 -04:00
Joseph Doherty 7c9add3d73 fix(worker): StaWaitHelper regression tests + full-drain contract notes; consolidate wait P/Invoke 2026-08-15 17:17:10 -04:00
Joseph Doherty 896d81e286 docs(worker): record the SemaphoreSlim/STA dispatch invariant + single-waiter contract; harden fake 2026-08-15 17:13:17 -04:00
Joseph Doherty 25cbe5cd3e fix(worker): guard timestamp-format derivation against pathological culture patterns 2026-08-15 17:10:58 -04:00
Joseph Doherty 13583322b5 perf(worker): launcher-configurable event queue capacity 2026-08-15 17:10:54 -04:00
Joseph Doherty f3e1de5f37 fix(alarms): truncation-safe transitions; perf: single-pass alarm parse; configurable poll cadence 2026-08-15 17:04:06 -04:00
Joseph Doherty 94fdc18c3c perf(worker): exact-format timestamp parse and compiled status-field accessors on the event path 2026-08-15 16:59:52 -04:00
Joseph Doherty f4a6cb1db2 perf(worker): signal-driven event drain — removes the 25 ms latency floor and idle wakeups 2026-08-15 16:59:13 -04:00
Joseph Doherty dc9424d3bd perf(worker): message-driven completion waits — the STA pumps continuously while waiting 2026-08-15 16:58:54 -04:00
Joseph Doherty afec56d03b perf(worker): reverse tag index + memoized views + indexed removals in the handle registry 2026-08-15 16:58:02 -04:00
Joseph Doherty f56798aeb9 perf(worker): pooled frame buffers — brings the net48 codec up to the gateway side's GWC-30 pattern 2026-08-15 13:28:29 -04:00
Joseph Doherty 13df92fbd8 chore(plans): Phase A complete — gate green at 1015/1015 2026-08-15 13:25:05 -04:00
Joseph Doherty 95f8ba918d fix(sessions): exception-total shutdown body with non-cancellable kill fallback 2026-08-15 13:19:08 -04:00
Joseph Doherty 0bc13b5292 perf(sessions): bounded-parallel teardown in lease sweep and shutdown 2026-08-15 12:43:30 -04:00
Joseph Doherty a1a38b5538 fix(ipc): cancellation-priority timeout classification; structurally-enforced no-throw cancel send 2026-08-15 12:41:38 -04:00
Joseph Doherty 1742e38c10 fix(events): graceful unregister no longer masquerades as overflow under FailFast 2026-08-15 12:38:20 -04:00
Joseph Doherty 7b2d04605e fix(audit): write-through on completed channel, poison-batch isolation, drain-fault fallback 2026-08-15 12:37:09 -04:00
Joseph Doherty 07b83561d1 docs(events): correct the capture-under-replayLock rationale 2026-08-15 12:34:36 -04:00
Joseph Doherty f920b4cbf5 fix(dashboard): clamped CAS retry loop in snapshot hub connection counter + direct counter tests
Decrement was decrement-first with a single non-retried repair CAS. From zero,
two unmatched decrements (SignalR calls OnDisconnectedAsync for a connection
whose OnConnectedAsync faulted) capture -1 and -2; a real Increment then makes
the count -1, and the first decrementer's stale CompareExchange(0, -1) matches
and resets to zero — erasing a live connection, so the idle gate freezes an open
dashboard. The same lost race also made Decrement report 0 when it had not
written 0.

Clamping now happens inside the compare-and-swap: read, clamp, publish, retry on
loss. A lost race re-reads the fresh value instead of repairing a stale one.

The counter moves to its own file per the one-public-type-per-file convention and
gains direct tests: the zero floor under concurrent unmatched decrements, matched
pairs settling at zero, and an interleaved connect/disconnect stress round. The
stress test asserts the observable invariants only — the specific interleaving
cannot be forced through the public API (verified: the previous implementation
passes it), which its remarks now state rather than implying a reproducer. A hub
wiring test is skipped for the EventsHub reason: driving Hub.OnConnectedAsync
needs caller-clients and connection-context fakes, and the overrides are two
lines of delegation to the tested type.

Also documents that the API-key refresh's pre-gate time check races benignly.
2026-08-15 12:32:58 -04:00
Joseph Doherty 44ca7c8623 fix(dashboard): enforce/document the advised-set cap honestly; cover handle-0 and oversize-read paths 2026-08-15 12:31:36 -04:00
Joseph Doherty 7c1ea12331 perf(ipc): WaitAsync command timeouts + forward WorkerCancel so a timed-out COM call frees the STA 2026-08-15 12:28:09 -04:00
Joseph Doherty 7171892984 perf(grpc): O(1) unconstrained bulk fast path, direct filtered-command build, cache eviction, capacity hints 2026-08-15 12:24:17 -04:00
Joseph Doherty 88d38bb900 perf(auth): allocation-free token parsing; single partition-key build per RPC 2026-08-15 12:23:25 -04:00
Joseph Doherty 3ff073d1ea perf(grpc): transfer reply ownership instead of deep-cloning every worker reply 2026-08-15 12:22:59 -04:00
Joseph Doherty 8e2066b4bd docs(dashboard): restore the advised-set LRU paragraph dropped by the snapshot commit
77c5731 committed this file from a working copy that predated 75e3dc2's
advised-set section, silently deleting it. Puts the paragraph back verbatim;
no other content changes.
2026-08-15 12:22:33 -04:00
Joseph Doherty 9735ac3b7c perf(metrics): pull-model worker queue gauge (fixes last-writer-wins), Interlocked command counters 2026-08-15 12:21:51 -04:00
Joseph Doherty 77c5731b7b perf(dashboard): idle-gate the snapshot tick; cache static config; bound key-list refresh
The snapshot publisher broadcast to Clients.All on every ~1s tick forever, with
zero viewers. Each tick cost a session-registry snapshot and sort, a metrics
snapshot that copies dictionaries under the global metrics lock, a full rebuild
of EffectiveGatewayConfiguration, and a SQLite read of the API key table.

DashboardSnapshotHub now counts live connections into the singleton
DashboardSnapshotHubConnectionCounter (clamped at zero, since SignalR can call
OnDisconnectedAsync for a connection whose OnConnectedAsync faulted). The
publisher drives the snapshot enumerator by hand instead of await foreach: with
no connections it does not call MoveNextAsync at all, so the producing iterator
stays suspended at its yield and no snapshot is built — the gate removes the
build, not just the broadcast. It re-checks once a second, so the first viewer
resumes the tick within about one interval; that viewer is seeded immediately by
DashboardPageBase's synchronous GetSnapshot() and by the hub's OnConnectedAsync.

Two per-tick costs are bounded independently of the gate: the effective
configuration is startup-static (options are bound once at boot and never
reloaded), so it is built once and cached; and the API key summaries refresh at
most every 15s, since the list only changes when an operator creates, rotates,
or revokes a key. Only a successful refresh restarts the interval, so a failed
or timed-out read is still retried on the next tick with the previous summaries
left on screen.
2026-08-15 12:21:49 -04:00
Joseph Doherty e04b1c9199 perf(events): copy-on-write subscriber snapshot in fan-out pump 2026-08-15 12:21:42 -04:00
Joseph Doherty 75e3dc2794 perf(dashboard): LRU cap on the shared live-read session's advised set 2026-08-15 12:21:13 -04:00
Joseph Doherty f1e26fed4f perf(alarms): memoize CurrentAlarms projection, invalidate on mutation 2026-08-15 12:20:51 -04:00
Joseph Doherty ca34a2d65d fix(logging): fail-closed bearer redaction; hoist per-request logger creation 2026-08-15 12:17:27 -04:00
Joseph Doherty e2ac5d117a perf(audit): bounded async audit writer with batched inserts, one-time bootstrap, retention sweep 2026-08-15 12:17:22 -04:00
Joseph Doherty 6c5218913b perf(dashboard): gate event mirror on live viewers — no clone, no send for unwatched sessions
DashboardEventBroadcaster.Publish ran a deep protobuf Clone (redaction is on by
default) and a group SendAsync for every event of every session, before anything
checked whether a dashboard client was actually watching. In the steady state the
session:{id} group is empty, so that work was thrown away per event.

SignalR does not expose group membership, so EventsHub now mirrors its own
add/remove into a singleton EventsHubViewerRegistry, and OnDisconnectedAsync
releases everything a dropped connection held (SessionDetailsPage disposes the
connection rather than unsubscribing). Publish returns early when the session has
no viewers, before the redaction clone. Watched sessions behave exactly as before.

Lazy mirror-lease start/stop was deliberately not attempted — it entangles the
dashboard with SessionEventDistributor subscribe lifetime for no saving beyond
this gate; recorded in docs/GatewayDashboardDesign.md.
2026-08-15 12:07:33 -04:00
Joseph Doherty da8463534b perf(ipc): give worker pipes real OS buffers instead of zero-quota rendezvous 2026-08-15 12:02:43 -04:00
Joseph Doherty 9958f80026 docs(plans): perf review remediation plan 2026-08-15 12:01:18 -04:00
Joseph Doherty 5744aad028 test(dashboard): prove the Secrets nav gate exists, by its absence
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m15s
ci / java (push) Successful in 2m16s
ci / portable (push) Successful in 9m19s
The policy tests shipped at 1c30611 could not detect a deleted gate. They
assert that secrets:manage admits an Administrator and refuses a Viewer —
true, and library behaviour this repo did not author. The wiring is the
only thing that change introduced, and nothing covered it.

The point is sharper for repos where the link already existed before
gating, which includes this one: "an Administrator still sees it" is
identical to the pre-change behaviour, so it cannot distinguish a working
gate from an inert AuthorizeView. Only the negative observation proves a
gate is there at all.

SecretsNavRenderTests renders MainLayout through the framework's static
HtmlRenderer — no component-testing package, because the assertion is
about emitted markup rather than interactivity — and asserts:

- absent for a Viewer, and for an anonymous caller (the load-bearing pair)
- present for an Administrator (the control: without it, a rail that
  rendered nothing at all would satisfy both absence assertions and the
  suite would report a working gate over a blank page)
- the ungated API Keys sibling still present for a Viewer, so a later
  "consistency fix" that hides it fails loudly rather than silently
  removing read access

Confirmed non-vacuous by mutation rather than by argument: with the
AuthorizeView removed from the layout, both absence tests go red and all
three original policy tests stay green.

Build 0 warnings / 0 errors; suite 899/899 (895 + 4).
2026-08-13 10:02:48 -04:00
Joseph Doherty 87d575dce4 Merge feat/secrets-nav-role-gate: side rail's Secrets link gated on secrets:manage, the policy the page itself enforces
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m15s
ci / java (push) Successful in 2m22s
ci / portable (push) Successful in 7m50s
2026-08-13 09:46:14 -04:00
Joseph Doherty 1c30611b1e feat(dashboard): gate the side rail's Secrets link on secrets:manage
Family-wide nav sweep: the Secrets management page should be linked from
each app's UI, visible to Administrator-role users only.

The link already existed in MainLayout's Admin section. The gate did not:
the rail rendered every item for every visitor, including a Viewer and the
anonymous-localhost read-only identity. Not an access hole — the mounted
page carries [Authorize(Policy = "secrets:manage")], so a Viewer clicking
through was denied — but a dead link presented as a live one. There was
also no existing role-gated nav pattern to follow; the rail's only
AuthorizeView was the footer's signed-in/signed-out split.

Gated on the POLICY rather than a role literal, so nav visibility cannot
drift from what the page enforces. In this host the two are equivalent:
GatewayOptionsValidator constrains Dashboard:GroupToRole values to
Administrator or Viewer, so the shared library's other manage-granting
roles (secrets-manager, secrets-reveal) are unreachable. The policy form
stays correct if that ever relaxes, where a role literal would then hide
the link from users who can use the page.

API Keys is deliberately left ungated. It looks like the same case and is
not: ApiKeysPage renders for a Viewer with write affordances hidden, so
hiding its link would remove legitimate read access. The secrets page has
no read-only mode. The rule is "gate the link when the page denies the
role outright", not "gate everything under Admin".

Coverage: three tests pin the policy's verdict per principal
(Administrator admitted, Viewer refused, unauthenticated refused), and
/admin/secrets joins the canonical route list — it is the one nav
destination mounted from an RCL rather than declared here, so a routing
regression could remove it without touching this repo's pages. The
principal helper sets an authentication type deliberately: without one
the role assertions would pass vacuously for the wrong reason.

Not a rendering test — the suite has no component-testing harness, and
adding one to assert a single AuthorizeView would be a large dependency
for a small claim.

Build 0 warnings / 0 errors; suite 895/895.
2026-08-13 09:33:04 -04:00
Joseph Doherty 1cb14d22bf chore(deps): bump ZB.MOM.WW.Auth to 0.2.1 — AD continuation-referral fix
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m12s
ci / java (push) Successful in 3m18s
ci / portable (push) Successful in 9m59s
2026-08-13 08:30:53 -04:00
Joseph Doherty 55bca95ad2 Merge docs/deploy-provenance-rows: 08-11/08-12 wonder deploys recorded; backup dirs read as a provenance chain
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m28s
ci / java (push) Successful in 2m46s
ci / portable (push) Successful in 8m27s
2026-08-12 04:20:42 -04:00
Joseph Doherty 5fe96b6677 docs(runbook): record the 08-11/08-12 wonder deploys and the backup-chain technique
Adds the two wonder rows that were deliberately withheld on 2026-08-11
while the pre-55f2889 SHA was unsettled. It is settled: b948e69 (08-09)
and 0a9715d (08-11) were never competing claims about one binary, they
are two deploys two days apart.

What settled it is worth recording as a technique in its own right, so
it goes in as a fourth way to identify a build: each Server.bak.<ts>
holds the exe that deploy REPLACED, so a VersionInfo sweep across the
backups reconstructs a host's deploy history from the host alone — no
repo access, no deploy record. The subtlety that makes it readable is
that a backup's timestamp dates the NEXT deploy, not the build inside
it. Reading a file version is non-destructive, unlike opening a SQLite
store in a backup directory.

Also records the full garbage version stamp recovered from the 08-09
binary, because the failure mode is a false positive rather than a
blank: "0.1.2+fatal:..." reads like a version that succeeded and then
picked up noise, when the leading 0.1.2 is just the static base <Version>
every build carries. For a binary in that window the commit is not
recoverable from the binary at all, so finding nothing is the expected
result rather than evidence against a SHA established another way.

Provenance is stated per cell rather than uniformly: the worker SHAs on
the new rows are carried forward and marked unconfirmed, b948e69 rests
on PDB hash plus the contemporaneous record and never on a stamp, and
55f2889 was read from the live stamp, which is trustworthy only because
it postdates 0152180.
2026-08-12 04:20:38 -04:00
Joseph Doherty 2c0daee481 Merge chore/shared-lib-latest-pins: every ZB.MOM.WW pin to newest published; Auth 0.2.0 obliged the LdapOptions shadow mirror
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m13s
ci / java (push) Successful in 2m24s
ci / portable (push) Successful in 9m24s
2026-08-12 04:17:34 -04:00
Joseph Doherty 62394f5b85 chore(deps): move every ZB.MOM.WW pin to the newest published version
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m11s
ci / java (push) Successful in 2m5s
ci / portable (push) Successful in 9m15s
Auth 0.1.5 -> 0.2.0, Health 0.2.0 -> 0.3.0, Secrets/.Abstractions/.Ui
0.6.1 -> 0.6.2. Theme, GalaxyRepository, Audit, Configuration, Telemetry
and Telemetry.Serilog were already at the newest version on the feed.

Checked against the shared-lib source rather than the version numbers,
because these packages are versioned as a family and a bump is not by
itself evidence that the package changed:

- Auth 0.2.0 is the only one carrying content for us: LDAP backup-DC
  failover (FallbackServers, endpoint walk with sticky preference,
  boot-time entry validation). Purely additive; the default is empty,
  which leaves single-endpoint behaviour unchanged.
- Health 0.3.0 carries a breaking change, but every line of it is in
  ZB.MOM.WW.Health.Akka, which we do not reference. No commit touched
  the core ZB.MOM.WW.Health package between 0.2.0 and 0.3.0.
- Secrets 0.6.2 is a message-only change: one validator string literal
  gains mounted-volume guidance. SecretsStorePathRules is untouched.

The four non-csproj files are not a separate feature. Configuration/
LdapOptions is a deliberate shadow of the shared type and carries an
explicit warning to mirror any new upstream field, because AddZbLdapAuth
binds the whole MxGateway:Ldap section onto the shared options. So
FallbackServers is live on our config surface the moment the package
lands, and without the mirror an operator could configure a backup DC
that works but is invisible on the dashboard's Settings page. The
Settings row renders "none" when empty, since that is the answer someone
who believes a backup DC is configured actually needs.

Entry syntax is deliberately NOT re-validated here: the shared validator
already fails the boot on a malformed entry and owns the (internal)
parser, so a second copy would drift. Note both validators skip entirely
when Ldap:Enabled is false.

Verified the binder is non-strict (ErrorOnUnknownConfiguration is unused
anywhere in the tree), so the upgrade could not break startup on a
newly-recognised key either way.

Build 0 warnings / 0 errors; gateway suite 892/892, unchanged. The live
LDAP tests are opt-in and were not run, so the failover path itself is
covered only by the shared library's own tests.
2026-08-12 04:16:23 -04:00
Joseph Doherty 55f2889c24 Merge fix/secrets-prehost-content-root: run the store-path guard where the store is actually created
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m17s
ci / java (push) Successful in 3m12s
ci / portable (push) Successful in 10m36s
0.6.0 landed the rules but not the enforcement on this path: the pre-host secrets
container has no IHostEnvironment, so the library skipped the content-root check
and the migrator created the database before the real host could refuse it. The
pin to 0.6.1 alone does not close that — the call site has to pass the content
root explicitly, which is why this is a code change and not a version bump.
2026-08-12 02:13:40 -04:00
Joseph Doherty 5fdd8a570a fix(secrets): run the store-path guard in the pre-host container (0.6.1)
ci / windows-x86 (push) Successful in 1m19s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m59s
ci / portable (push) Successful in 10m48s
0.6.0 put the store-path rules in the shared library, but the guard was not
running at the moment that matters here.

CreateBuilder resolves ${secret:} references before the host exists, using a
throwaway ServiceCollection that contains no IHostEnvironment — and it runs the
store migrator, which creates the database. The library resolved the content
root from IHostEnvironment alone, so it could not distinguish "no content root"
from "no host registered" and skipped the under-content-root rule entirely. The
store was created at the rejected path; the boot then failed a moment later when
the real host validated. The leftover empty database with its -wal/-shm siblings
is exactly the artifact that made the 2026-08-09 credential loss read as "the
database is there, it's just empty".

The pin alone does not close this. An app with a correctly configured path shows
no symptom and is still unprotected, because the guard simply is not running when
the store is created. 0.6.1 adds a 4-argument AddZbSecrets overload taking the
content root explicitly, and the call site has to use it. The in-host
registration below needs nothing.

Verified by removing the fix rather than by observing a clean boot — which is how
this survived its first release. With the 3-argument overload the new test fails
by finding a created database at
src/ZB.MOM.WW.MxGateway.Server/probe-secrets-*.db: inside the source tree, since
that is what the content root resolves to under test.

Two things about the test itself, both of which it would have been easy to get
subtly wrong:

It asserts no-file-created before asserting that startup threw. "It threw" is the
weaker claim, and asserting it first masks the stronger one — the run that proved
this defect would have reported "no exception was thrown" and said nothing about
the database sitting in the source tree.

The accepting case asserts the database *is* created, not merely that nothing
threw. A not-null builder is close to a tautology once no exception escaped, and
it would still pass if the pre-host container stopped opening the store at all —
which would also quietly void the rejecting case, since that one can only observe
a file the migration would otherwise have written. The two assertions hold each
other up.

Found by HistorianGateway's adoption, which probed the rejected paths instead of
observing a successful boot.
2026-08-11 08:54:06 -04:00
Joseph Doherty 9bc70d1af3 Merge feat/session-health-check-and-store-path-guard: session health probe + store-path guard + Secrets 0.6.0
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m13s
ci / java (push) Successful in 2m23s
ci / portable (push) Successful in 17m53s
Three separable changes, each building on its own: the mxaccess-sessions health
check on the active tier, the content-root rule that closes the gap the 2026-08-09
credential-store loss went through, and the Secrets re-pin (0.2.3 -> 0.6.0) that
moves the same rules into the shared library.
2026-08-11 08:43:43 -04:00
Joseph Doherty 6ba52a68f0 build(secrets): re-pin ZB.MOM.WW.Secrets to 0.6.0
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 2m18s
ci / java (push) Successful in 2m23s
ci / portable (push) Successful in 8m14s
0.6.0 moves the store-path rules into the shared library — both the rooted check
and the content-root check — with a SecretsOptionsValidator wired into
AddZbSecrets and validated on start, so all four consuming apps get the guard
from one implementation rather than four copies. mxgw therefore adds no local
validator over the Secrets section.

This is a four-minor jump, not a re-pin: mxgw was on 0.2.3 and skips 0.3.0,
0.4.0, 0.4.1, 0.5.0 and 0.5.1 in one step. Verified rather than assumed —
diffing the 0.2.3 and 0.6.0 assemblies shows the added surface is the new path
rules and their plumbing (AddIfNotRooted, AddIfUnderContentRoot,
ComputeDefaultSqlitePath, DefaultSqlitePath, IValidateOptions, IsPathRooted,
GetFullPath) and nothing touching store or delete behaviour. Secrets.Ui is
byte-identical across the range: both assemblies are 39424 bytes and differ only
in the version stamp, because this package family shares one version across
every package even when a release touches only one of them. So the browser gate
run against the /admin/secrets delete modal on 0.2.3 still covers what ships
here.

The library default is LocalApplicationData-derived so the family's
cross-platform apps still boot locally. The gateway keeps its own
CommonApplicationData value, which always wins — see the note on
ApplyDefaultSecretsStorePath for why the difference is deliberate and why
deleting that method as a redundancy would silently move the store.
2026-08-11 08:42:28 -04:00
Joseph Doherty 882c7ca3cd fix(config): reject credential and cache paths inside the application directory
Rooted is not the same as safe, and the gap between the two cost a production
host every one of its API keys on 2026-08-09.

MxGateway:Authentication:SqlitePath was set to an absolute path inside the
directory the upgrade procedure renames to Server.bak.*. That passes the
existing rooted check cleanly. The deploy renamed the directory away, the store
went with it, and the gateway created a fresh empty one at the same path — no
error, no log line. No gRPC consumer could authenticate for two days. The deploy
itself was correct: the binaries were the point of the rename and the store was
collateral.

GatewayConfigPathRules gains AddIfUnderContentRoot, applied to the auth store
and the Galaxy snapshot. Both are written by the running process and both are
lost the same way. The rule compares resolved full paths and requires a
directory-separator boundary, so a sibling directory whose name merely starts
with the content root's ("/srv/app-data" against "/srv/app") is not treated as
inside it — on a fail-closed startup rule, that false positive would be a
gateway that refuses to boot on a legitimate path. Case sensitivity follows the
running OS rather than assuming case-insensitivity everywhere, which would
reject /srv/App as under /srv/app on Linux where they are different directories.

The rule is not exempted in Development. An environment-conditional guard is
never exercised where the mistake is made, and what failed in production was a
config that looked fine.

Secrets:SqlitePath is the same defect one layer down: it shipped as a bare
relative "mxgateway-secrets.db", which is how a stray database landed in
src/…Server/ and tripped the repository's tree-hygiene test. It is bound by the
shared ZB.MOM.WW.Secrets package, so appsettings.json now ships no value and the
default is computed from CommonApplicationData in code — the same mechanism
SEC-33 already used for the Galaxy snapshot, ten lines away, for the same reason.
Setting a default for an unset key is deliberately not the same act as
relocating a value someone configured, which these rules still refuse to do.

Note the migration edge this creates: a host relying on the old repo default now
looks somewhere new, finds nothing, and creates an empty store — this bug
re-introduced by its own fix. Deployed hosts are safe because they set the path
explicitly, in appsettings copied forward or in the service environment. The
latter is the more robust of the two, since it cannot be lost by a missed
preserve step.
2026-08-11 08:42:16 -04:00
Joseph Doherty c69a1c441b feat(diagnostics): report MXAccess session health on the active probe
Adds a `mxaccess-sessions` health check reporting how many MXAccess sessions are
healthy. Each session is one worker process holding one MXAccess COM instance —
a live connection into a Galaxy — so this answers "how many Galaxy connections
are healthy" in the vocabulary the code actually uses.

Zero sessions is Healthy, deliberately, and the rest of the design follows from
that. The gateway opens a session when a client asks and holds none otherwise,
so an idle gateway is working normally. A count threshold ("unhealthy below N")
would sit red forever on a host nothing dials yet, and a permanently red probe
is one operators stop reading — which leaves them worse off than no probe. The
check therefore grades on whether the sessions that exist are usable: nothing
faulted is Healthy, some faulted beside a ready or starting one is Degraded, and
every session faulted is Unhealthy. Counts ride along as entry data for the
family Overview dashboard.

Tagged `active` rather than `ready` for the same reason. Readiness decides
whether the process should be sent traffic, and a gateway with no sessions is
ready to serve — unlike the auth store, which every call depends on. Failing
readiness here would pull a working gateway out of rotation over a condition its
own clients create.

Reads ISessionRegistry, which already exposes Snapshot(); ISessionManager stays
the command surface and grows no enumerator.
2026-08-11 08:41:54 -04:00
Joseph Doherty 22a34f7f31 Merge docs/runbook-evidence-precision: what the deployed-build identification evidence rests on
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m17s
ci / java (push) Successful in 2m51s
ci / portable (push) Successful in 10m51s
2026-08-11 06:06:48 -04:00
Joseph Doherty f6b6184e70 docs(runbook): state what the identification evidence actually rests on
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m18s
ci / java (push) Successful in 2m55s
ci / portable (push) Successful in 10m39s
Two precision fixes to the runbook landed in fd0e88e, both making it weaker in
the sense that matters.

The PDB source-hash entry read as though hashes settled the 2026-08-09
provenance question by themselves. They did not: a hash match and a
contemporaneous deploy record written that evening independently named the same
two commits, and the agreement is what makes the result trustworthy. A hash
match alone tells you which sources a binary was built from — not that the build
was intentional, nor which host it reached. A future reader holding only one of
the two derivations should know to look for a second.

The two-swap timeline was asserted from the investigation's timestamps, which a
later reader cannot re-derive. It is also confirmable from artifacts still on
disk — windev keeps two worker backup directories from that day and wonder one,
because there were two worker operations. Records that, so the story can be
checked against the boxes rather than believed.
2026-08-11 06:06:48 -04:00
Joseph Doherty e5dbcee17c Merge docs/deployed-build-identification: runbook for mapping a running binary to a commit
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m17s
ci / java (push) Successful in 3m26s
ci / portable (push) Successful in 11m7s
2026-08-11 06:05:23 -04:00
Joseph Doherty fd0e88e74c docs(runbook): how to identify a deployed build, and why the version stamp lies
ci / windows-x86 (push) Successful in 1m19s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 3m24s
ci / portable (push) Failing after 6m14s
A 2026-08-11 investigation treated the windev production binary as having no
traceable provenance. Both pieces of evidence were normal output of our own
build and deploy procedure, and the binary was fine — but nothing in the repo
said so, so it cost a forensics pass to establish.

The version stamp is the first trap. SHA stamping landed in ec6f82b (TST-11)
and was broken on Windows until 0152180 (NEXT-09) a month later: the trailing
backslash in MSBuildThisFileDirectory escaped the Exec's closing quote, and
because the target runs ContinueOnError with ConsoleToMSBuild, git's stderr was
stamped as the revision. Every Windows build in that window reads
`0.1.2+fatal: cannot change to ...`. The point an operator needs is that this
is non-diagnostic in *both* directions — it neither incriminates a build nor
confirms one — which is not obvious from a stamp that looks like a failure.

The second is that the build directory is *supposed* to be gone: deploys build
from a detached worktree so the host's checkout stays on its own branch, and
remove it afterwards.

Records what does identify a build instead, cheapest first — including the wire
probe for the worker, since plain Write/Write2 carrying statuses[0] separates
53f69cd from b948e69 without host access or symbols — plus the deploys to date.

Also records something that went unlogged and is the reason this read as a
mystery rather than an improvement: the 2026-08-09 deploy returned the x86
worker to mainline. Production had been running dd7ca163, contained only by
origin/test/client-e2e-coverage and not an ancestor of main, so the worker in
production was not rebuildable from any mainline commit.
2026-08-11 06:05:19 -04:00
Joseph Doherty 0a9715d819 Merge feat/dashboard-ui-cleanup-sweep: admin-UI cleanup pass + ZB.MOM.WW.Theme 0.4.1
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m17s
ci / java (push) Successful in 2m28s
ci / portable (push) Successful in 9m25s
The Theme bump is separable from the sweep and lands as its own commit: 0.4.0
upstreamed the button-sizing block the sweep had installed locally, and this
app's local .btn rule was trimmed rather than deleted because it also carried
border-radius/font-weight/white-space that the kit does not ship.
2026-08-11 05:49:11 -04:00
Joseph Doherty 01033d7aaf fix(dashboard): admin-UI cleanup sweep
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 2m18s
ci / java (push) Successful in 2m24s
ci / portable (push) Successful in 8m51s
Family-wide admin-UI cleanup pass (scadaproj admin_ui_cleanup.md) applied to the
Blazor dashboard. Behaviour is unchanged throughout — no @onclick, disabled,
binding, auth gate, or arm->confirm flow was touched.

Uncontrolled error text is now truncated at the render site. Fault messages,
Galaxy load errors, and browse-tree load failures were rendered in full into
fixed-width table cells, where a long exception string blows out the column.
Each site gets DashboardDisplay.Abbreviate plus a title attribute carrying the
untruncated text, so nothing becomes unreachable. Abbreviate is length-checked
rather than a bare range slice: `value[..n]` on a shorter string throws and
takes the whole page render down with it. The two detail views whose entire
purpose is to show one fault in full — SessionDetailsPage and GalaxyPage's
Last Error — are deliberately left untruncated.

Two classes referenced from markup had no definition anywhere in the sheet.
.browse-stale-banner was inert; .tree-load-status was a real visual defect —
loading and failed-to-load rows sit among .tree-row siblings and carry the same
leading .tree-toggle-empty spacer, but that spacer only takes its width as a
flex item, so without a flex container those rows lost their indent.

Confirm/cancel pairs in ConfirmDialog and the API-key create form are now
btn-groups with role="group" and an aria-label, replacing margin-spaced loose
buttons.

Removes a paragraph on GalaxyPage naming internal RPCs (DiscoverHierarchy,
GetLastDeployTime) — implementation detail with no meaning to a dashboard
operator.

Verified in a real browser, not bUnit: full build clean, 879/879 tests, and a
live gate against a running dashboard with a genuine ~250-char SqlClient
exception as the erroring row. Results per check, including the checks that
could NOT be exercised without an x86 worker, are recorded in
docs/plans/2026-08-11-dashboard-ui-sweeps.md.

That plan doc also records a correction: this app is NOT Bootstrap-free. The
sweep brief said it was, citing the scadaproj index; libman.json pins
bootstrap 5.3.3 and App.razor:7 links it ahead of the theme. The stale claim had
already cost this app one skipped family sweep (scadaproj#2, the /admin/secrets
modal), so that modal was live-gated here too and passes.
2026-08-11 05:48:22 -04:00
Joseph Doherty dc53f04b81 build(theme): adopt ZB.MOM.WW.Theme 0.4.1 button sizing
site.css declared `font-size: 0.82rem` literally on `.btn`. Bootstrap sizes
buttons through `font-size: var(--bs-btn-font-size)`, and `.btn-sm` /
`.btn-group-sm > .btn` do nothing but redefine that variable — so a literal at
equal specificity, loaded after Bootstrap, silently flattened every small button
in the dashboard into a padding-only difference. `btn-sm` was inert app-wide.

The kit now owns the sizing: 0.4.0 shipped the four `--bs-btn-*` overrides in
layout.css, which ThemeHead emits ahead of site.css, so removing the local
literal restores `.btn-sm` without a local copy. 0.4.1 is a pin-only follow-up
(it fixes `.rail-btn-block`, which this app does not use; `theme.css` is
byte-identical and the `.btn` rule is unchanged between the two).

The rest of the local rule is kept deliberately. 0.4.0 upstreamed sizing only,
not `border-radius` / `font-weight` / `white-space`, so deleting the block
wholesale would have dropped three app-specific declarations. `border-radius`
also stays a literal rather than `--bs-btn-border-radius`, because `.btn-sm`
redefines that variable and small buttons would shrink to Bootstrap's radius.

Verified in a browser against the running dashboard: `.btn` 13.6px and `.btn-sm`
12.48px, btn-group seams intact, and `--bs-btn-font-size` now declared in
exactly two sheets (bootstrap.min.css, layout.css) instead of three.
2026-08-11 05:47:52 -04:00
Joseph Doherty 917694c33d Merge fix/test-class-async-disposal: xUnit v2 ignores IAsyncDisposable on test classes; teardown was dead code
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m12s
ci / java (push) Successful in 1m59s
ci / portable (push) Successful in 8m18s
2026-08-10 16:11:54 -04:00
Joseph Doherty 3e504ca9c4 test(sessions): switch SessionWorkerClientFactoryFakeWorkerTests to IAsyncLifetime so its teardown actually runs
xUnit v2 (2.9.3) never invokes IAsyncDisposable.DisposeAsync on a test
class, so the unobserved-fault safety net this class documents — awaiting
every scripted worker task after each test — has been dead code since it
was written. Found via dumpasync on the wedged-testhost investigation: the
NeverReadyWorkerProcessLauncher's Task.Delay(Infinite, _stop.Token) was
still pending (DelayPromiseWithCancellation, detached) minutes after its
test finished, which is only possible if DisposeAsync never ran.

Proven both ways with a throw-probe in DisposeAsync: under IAsyncDisposable
all 3 tests pass (never called); under IAsyncLifetime all 3 fail from the
probe (called after every test). Sweep confirmed this is the only test
class on IAsyncDisposable — all other implementers are helpers disposed
via await using.
2026-08-10 16:11:54 -04:00
Joseph Doherty 02baf0c27b Merge fix/testhost-exit-hang: zero-buffer Windows pipes blocked one test's writes forever; windev suite baseline is 879, same as macOS
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m17s
ci / java (push) Successful in 2m45s
ci / portable (push) Successful in 7m57s
2026-08-10 10:04:40 -04:00
Joseph Doherty bfcf82975c test(windev): buffer the test named pipes so the full-suite testhost exits
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m14s
ci / java (push) Successful in 2m19s
ci / portable (push) Successful in 8m48s
The windev full-suite wedge — every test reported, then the x64 testhost sitting
at ~0 CPU forever while `dotnet test` never returns — was one test blocked on a
pipe write, not a leaked thread or an undisposed fixture.

`dotnet-stack report` on the wedged host showed no thread running test code:
xUnit's RunTestsInAssembly was parked on WaitHandle.WaitOne() waiting for the
assembly-finished event, so the wait lived in a suspended async state machine.
`dotnet-dump analyze -c dumpasync` named the frame — WorkerClientTests
.StagingChannelOverflowFaultsWorkerWithoutWaitingForFullModeTimeout awaiting
WorkerFrameWriter.WriteAsync on a 63-byte frame, with <extra>5__15 = 6, i.e. the
seventh of the twelve events the test pushes past the client's staging bound.

That test faults the worker client on purpose, and a faulted client stops its
read loop by design. The test-side pipe came from the NamedPipeServerStream
overload without buffer arguments, which passes inBufferSize: 0 / outBufferSize: 0
to CreateNamedPipe; on Windows that reserves no buffer at all, so a write
completes only once the peer reads it. Measured on windev, that pipe absorbed
0 bytes against a non-reading peer where the same pipe declared with 64 KiB
buffers absorbed 65 520. On macOS and Linux .NET backs named pipes with Unix
domain sockets whose socket buffer swallows the writes regardless, which is why
the identical test never hung there and the bug read as environmental.

Test-owned server pipes now go through TestSupport/TestNamedPipe.CreateServer in
both test projects, declaring explicit 64 KiB buffers so those tests exercise the
gateway's own staging/queue backpressure rather than the OS pipe's flow control.

Separately, every fake-worker write in WorkerClientTests now goes through
PipePair.WriteAsync, bounded by the class's five-second TestTimeout. That is
where the severity came from: a test method that never returns keeps xUnit from
raising ITestAssemblyFinished, so one unbounded await cost the whole suite its
result. A blocked write is now a named test failure instead of a silent wedge.

The fix also retires a wrong belief the wedge had created. windev reported 855
where macOS reported 879, and that gap was recorded in GatewayTesting.md as
Unix-gated test cases; it was really the results lost when the wedged host was
torn down. The same clone now reports 879 passed, matching macOS exactly.

Product code is unaffected. SessionWorkerClientFactory.CreatePipe keeps the
unbuffered declaration deliberately: both ends run continuous read loops and every
gateway write is bounded by the worker client's _stopCts, so a stalled peer
cancels the write rather than blocking on it.

Verified on windev at this SHA: gateway suite x64 three times (879 passed,
exit 0, no surviving testhost each time) and Worker.Tests x86 twice (400 passed,
11 skipped, exit 0, clean), plus the macOS gateway suite once (879 passed).

Docs: GatewayTesting.md replaces the --blame-hang workaround section with the
root cause and corrects the baseline to 879, CLAUDE.md's Source Update Workflow
no longer tells readers the windev suite wedges, and ToolchainLinks.md records
dotnet-stack and dotnet-dump as installed on windev.
2026-08-10 10:01:57 -04:00
Joseph Doherty f78781d9ef Merge fix/windev-baseline-env-failures: the two 'environmental' windev failures were Windows-only test bugs
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m52s
ci / java (push) Successful in 2m10s
ci / portable (push) Successful in 7m51s
2026-08-10 09:13:55 -04:00
Joseph Doherty e2352d1666 test(windev): fix the two Windows-only gateway test failures dismissed as environmental
ci / java (push) Successful in 2m24s
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 3m33s
ci / portable (push) Successful in 8m13s
Both failures in the windev baseline were test bugs that reproduce on any Windows
host, not anything missing or misconfigured on windev.

SelfSignedCertificateProviderTests.GenerateCertificate_HasExpectedSansEkuAndValidity
asserted SAN content by substring-matching X509Extension.Format(false). That string
comes from the platform crypto library: Windows' CryptFormatObject renders the IPv6
loopback fully expanded (0000:0000:...:0001) where the managed formatter renders
"::1", so the loopback assertion could never hold on Windows. Decode the extension
with X509SubjectAlternativeNameExtension and compare parsed IPAddress values and DNS
names instead, which removes the platform-dependent formatting from the assertion.

SessionManagerTests.OpenSessionAsync_PipeNameIsShortAndUniquePerPidAndSession guards
the 104-byte macOS sun_path budget NEXT-01 shortened the pipe name to fit. It padded
the measured name up to a five-digit pid but never substituted that worst case
downward, so Windows' routine six-digit pids over-counted by a character against a
budget that does not constrain the host running the test. Substitute the five-digit
macOS worst case for the running pid's digit count so the check measures the name
format rather than the current pid.

EventStreamServiceTests.WaitUntilAsync now reports the unmet condition on timeout
instead of letting a bare TaskCanceledException escape. Its five-second real-clock
deadline is genuinely load-sensitive on windev (36 logical CPUs, maxParallelThreads
-1), and an opaque cancellation there is exactly what got the previous failures
filed as "environmental" and left unexplained.

Documents the windev run in docs/GatewayTesting.md: the two fixed bugs and their root
causes, the real-pipe suites whose failures are evidence of machine load rather than
of the change under test, and the full-suite testhost that completes every test and
then never exits (filtered runs exit normally; macOS exits cleanly). Corrects the
CLAUDE.md claim that the suite exits cleanly on the Windows dev box.
2026-08-10 09:05:38 -04:00
Joseph Doherty 347d59fc62 Merge fix/deflake-eventburst: flush-edge sampling replaces wall-clock window in EventBurst_DrainLoopCoalescesFlushes
ci / java (push) Successful in 2m13s
ci / nightly-windev (push) Has been skipped
ci / portable (push) Successful in 9m25s
ci / windows-x86 (push) Failing after 20m21s
2026-08-10 08:53:07 -04:00
Joseph Doherty 404c06b2fa Merge feat/tst-24-client-wire-tests: .NET/Python real-server wire tests, Python channel-loop fix, TST-05 revisit (partial)
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m45s
ci / portable (push) Successful in 11m39s
ci / windows-x86 (push) Failing after 21m2s
# Conflicts:
#	archreview/remediation/00-tracking.md
2026-08-10 08:23:39 -04:00
Joseph Doherty a8f86b5336 test(tst-24): drive the .NET and Python clients against real in-process gRPC servers
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m19s
ci / portable (push) Successful in 23m59s
ci / windows-x86 (push) Failing after 58m42s
TST-24 asked for per-client wire tests against a fake gateway. An audit first
corrected the finding's premise: Go, Rust, and Java already had them — bufconn,
a loopback tonic server, and InProcessServerBuilder respectively — each already
asserting the round trip, the server-observed bearer header, and the ReplayGap
sentinel. The two genuine gaps were .NET (every test substituted the transport
interface; the test project had no server package at all) and Python (stub
monkeypatching everywhere but one opt-in TLS test).

Both now serve mxaccess_gateway.v1.MxAccessGateway over a real transport —
Kestrel h2c and grpc.aio, each on an ephemeral loopback port — and drive the
ordinary public client API against it. Only the gateway's behaviour is canned;
the framing, serialization, metadata, and status codes are genuine. Four shapes
each: full round trip with every reply field asserted, the authorization header
as received by the server (including on the streaming RPC), the ReplayGap
sentinel surfaced as the client's typed signal, and a real PERMISSION_DENIED
mapping to the typed authorization error.

The .NET client was only ever compiled in CI, never tested, so the portable job
gains a dotnet test step.

Fixes a bug the new tests caught on their first run: Python's connect() built the
grpc.aio channel inside asyncio.to_thread, and a grpc.aio channel binds to the
event loop current on the constructing thread, so every non-stub connection
raised 'There is no current event loop in thread'. No mock-based test could see
it, and the test guarding the off-loop behaviour patched create_channel and so
asserted the bug. Split resolve_channel_security (blocking TOFU probe, off-loop)
from create_channel (on-loop); the guard tests now assert both halves.
2026-08-10 08:22:25 -04:00
Joseph Doherty 50322bacb3 test(worker): deflake EventBurst_DrainLoopCoalescesFlushes
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m13s
ci / java (push) Successful in 2m38s
ci / portable (push) Successful in 8m17s
Root cause. The test sampled its flush baseline after a bare `Task.Delay(100)`
and then charged every later flush to the 128-event burst. Two facts make that
window unsound:

1. `RunHeartbeatLoopAsync` sends its first beat *immediately* on entering the
   message loop — the far-off `HeartbeatInterval` only spaces later beats — so
   the pre-burst frames are WorkerHello, WorkerReady, and a heartbeat, not the
   two the test's comment assumed.
2. `WorkerFrameWriter.DrainQueuedFramesAsync` flushes *after* writing a drained
   batch, so the bytes reach the pipe before the flush runs. Reading a frame off
   the gateway side is therefore no evidence that its flush has been counted,
   and no sleep makes it evidence.

Under load on the shared windows-x86 runner the first heartbeat's flush was
scheduled after the 100 ms sample, so it landed inside the measured window and
the assertion saw two flushes for the burst — exactly the observed
`Expected: 1 / Actual: 2` (Gitea run 675, job 2558, and the same failure since
a346d51). Nothing about the coalescing behavior was wrong; only the test's
timing assumption.

Fix. Replace the sleep with explicit synchronization, no widened timeouts.
`FlushCountingPassthroughStream` now records the flush *shape* — the number of
stream writes coalesced into each flush — and exposes
`WaitForAllWritesFlushedAsync`, a TaskCompletionSource signal released when the
next flush drains the pending writes. The test reads the first heartbeat (the
last pre-burst frame), waits for its flush, and only then samples the baseline;
after the burst it takes the same edge before asserting. The assertion is also
sharpened from a bare count to the shape: exactly one flush beyond the baseline
*and* that flush carried all 128 event frames, so a split batch fails even if
the reader observes it mid-split.

docs/WorkerFrameProtocol.md notes the peer-visible ordering the fix turns on:
frames reach the pipe before the flush that follows them, so an observer of the
flush must wait for it rather than infer it from frames arriving.
2026-08-10 08:13:30 -04:00
Joseph Doherty 9e6f66dd8f Merge fix/tst25-nightly-issue-path: reachable run links in nightly failure issues (TST-25 Check 6 closed)
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 1m16s
ci / java (push) Successful in 2m13s
ci / portable (push) Successful in 8m46s
2026-08-10 08:11:42 -04:00
Joseph Doherty 30c92e8e59 fix(ci): give the nightly-windev issue body a reachable run link (TST-25 Check 6)
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 1m13s
ci / java (push) Successful in 2m7s
ci / portable (push) Successful in 8m30s
Exercising TST-25 acceptance Check 6 end-to-end showed the on-failure issue
step works — it has filed an issue on every red nightly since 2026-07-17
(#126-#139) — but that every one of those issues links `http://gitea:3000`,
the runner-internal origin Gitea hands the runner. That URL is correct for
the issue-creation API call (the job container resolves `gitea` only on the
docker network and has no LAN egress to the public origin) and unreachable
for anyone clicking out of the issue.

Keep `github.server_url` for the API call; add a `PUBLIC_SERVER_URL` job env
used only for the browser-facing link in the body. Verified with a
forced-failure probe run (677) whose issue (#140) carries a public link that
returns 200.

Also records Check 6 as done in scripts/ci/README.md and corrects the
2026-07-13 tracking entry that wrote the check off as abandoned.
2026-08-10 08:10:08 -04:00
Joseph Doherty d4302c6ac4 docs(tst-05): revisit under the restored windev tier — scheduled half closed, control-command coverage still open
The nightly-windev job (cycle-2 TST-25) discharges TST-05's scheduling design:
cron 0 6 * * * runs run-windev-ci.sh live, which sets
MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1, runs WorkerLiveMxAccessSmokeTests on windev,
and opens a Gitea issue on failure. That converts 'run by memory' into 'runs
nightly, reports failures'.

The finding's other half — audit the live suite for coverage of each of the eleven
late-added command kinds — is not discharged. The audit's answer is negative for
five: the suite covers all six COM commands and none of the five control commands
(Ping/GetSessionState/GetWorkerInfo/DrainEvents/ShutdownWorker), which are exactly
the kinds the finding calls masked by FakeWorkerHarness canned replies. Recorded as
Partially done with the residual test work specified, rather than claiming closure.
2026-08-10 08:08:01 -04:00
Joseph Doherty c46e5bbd15 Merge fix/next-cycle-batch: resolve NEXT-01/02/03/04/05/09 from the 2026-07-12 next-cycle candidates
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 1m18s
ci / java (push) Successful in 2m46s
ci / portable (push) Successful in 7m50s
- NEXT-01: pipe names shortened to mxgw-{pid}-{sessionUid}; macOS full gateway suite green (879/879) under default TMPDIR for the first time
- NEXT-02: .NET + Java CLIs render ReplayGap as the typed cross-CLI row; five-CLI contract converged
- NEXT-03: best-effort reconcile/live dedup on the alarm feed; at-least-once consumer contract unchanged
- NEXT-04: fault-observing continuation on frames abandoned by cancellation in the worker frame writer
- NEXT-05: lazy tombstone purge kept, decision documented in WorkerFrameProtocol.md
- NEXT-09: Windows builds stamp a real short SHA into InformationalVersion (verified 0.1.2+5fe74db on windev), with a SHA-shape guard

windev x86 Worker.Tests 399 passed / 11 skipped / 1 failed = the documented
EventBurst_DrainLoopCoalescesFlushes flake; .NET client 35/35, Java 52/52.
NEXT-08 and NEXT-10 remain open (posture/cross-repo decisions).
2026-08-10 06:11:22 -04:00
Joseph Doherty 5fe74db971 docs(tracking): strike NEXT-01/02/03/04/05/09 as resolved 2026-08-10
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 1m15s
ci / java (push) Successful in 2m9s
ci / portable (push) Successful in 8m12s
Six of the eight open next-cycle candidates are closed in this batch
(fix/next-cycle-batch): macOS pipe-path length, .NET/Java CLI ReplayGap
rendering, alarm reconcile/live dedup, frame-writer unobserved-fault
hygiene (plus the NEXT-05 lazy-purge decision), and the Windows
InformationalVersion stamp. NEXT-08 (GLAuth TLS posture) and NEXT-10
(glauth.md user-table reconciliation entangled with the OPC-UA group
taxonomy) remain open by design — both need cross-repo/posture decisions,
not gateway code.
2026-08-10 06:07:00 -04:00
Joseph Doherty 2c03e0a684 fix(alarms): dedup reconcile/live duplicate broadcasts on the alarm feed (NEXT-03)
A periodic reconcile can synthesize a repair transition whose matching live
transition is still buffered in the alarm lease; both then broadcast as
indistinguishable duplicates on StreamAlarms and the dashboard hub. Nothing
serializes the two paths, and a correct serialization needs a worker-side
high-water mark on QueryActiveAlarms (proto + worker change + a stall path),
so this closes the common case with a local best-effort dedup instead: both
paths already carry the same worker-derived identity — the worker stamps
record.TransitionTimestampUtc into both OnAlarmTransitionEvent's
transition_timestamp and ActiveAlarmSnapshot.last_transition_timestamp — so
ApplyTransition suppresses a live transition whose (timestamp, resulting
state) the cache already carries from a repair. The Clear leg has no cache
entry left to compare, so ApplyReconcile tombstones each synthesized Clear by
the instance's original_raise_timestamp for one reconcile generation; a
matching live Clear consumes the tombstone, while a new raise/clear cycle
carries a newer raise timestamp and passes. Suppression fires only on a
positive marker match — unset timestamps keep today's behavior — so the
documented at-least-once consumer contract stands (gateway.md, Sessions.md
updated in the same change).

Tests: two new regressions drive the exact race through the GWC-26 harness
(repair-then-buffered-live for Raise and for Clear, each with a genuine
follow-up transition proving no over-suppression); GatewayAlarmMonitor
suites 18/18.
2026-08-10 06:06:17 -04:00
Joseph Doherty 8624e21372 fix(clients): render ReplayGap as the typed cross-CLI row in the .NET and Java CLIs (NEXT-02)
The .NET and Java stream-events commands handed the raw ReplayGap sentinel
MxEvent to their protobuf JSON formatters (Java text mode printed
'0 MX_EVENT_FAMILY_UNSPECIFIED'), while the Go/Python/Rust CLIs already emit
the typed row (CLI-35/36). Both now branch on the sentinel: Java text mode
prints 'REPLAY_GAP requested_after=<n> oldest_available=<n>' and JSON mode a
hand-built {"replayGap":{...}} line via nextItem()/isReplayGap(); the .NET
CLI emits the same hand-built row in jsonl/text (its text mode is
JSON-per-line) and inside the --json events array. Rows are hand-built so
the cursors are JSON numbers like the other three CLIs, not the proto3 JSON
mapping's quoted uint64 strings — the CrossLanguageSmokeMatrix divergence
table collapses to a single converged contract.

Tests: .NET MxGatewayClientCliTests 35/35 (new RendersReplayGapAsTypedRow
covers jsonl + aggregate); Java gradle test 52/52 (new
streamEventsRendersReplayGapAsTypedRow covers --json + text over the
in-process harness). No generated-file churn.
2026-08-10 06:01:44 -04:00
Joseph Doherty 84dbf20a43 fix(worker): observe faults on frames abandoned by cancellation (NEXT-04, NEXT-05 decision)
A WriteAsync/WriteBatchAsync caller cancelled after the draining lock-holder
claimed its frame unwinds without awaiting that frame's completion; the same
holds for a frame already faulted by a concurrent FailAllQueued, where
TrySetCanceled loses. A later wire-write failure then lands TrySetException on
a task with no awaiter and surfaces as TaskScheduler.UnobservedTaskException.
The tombstone helpers now attach a fault-observing continuation to every frame
in the cancelled call (a cancelled task never fires OnlyOnFaulted, so
unconditional attach is safe), outside _gate because an already-faulted task
runs the continuation inline.

NEXT-05 is resolved as a documented decision, not a code change: tombstoned
entries keep their lazy DequeueNext purge — any subsequent write drains both
queues to empty and the heartbeat loop bounds residency to one interval, while
eager Queue<T> rebuilds under _gate would add ordering-invariant surface for
no gain. Rationale recorded in docs/WorkerFrameProtocol.md alongside the
WRK-22 residual-window contract.

New regression test drives the exact abandonment: gated stream holds writer A
mid-write, the queued event frame is claimed and blocked mid-write, its caller
is cancelled, the write then faults with a marker exception, and the test
asserts the marker never reaches UnobservedTaskException after a forced GC.
net48 x86 build/test runs on windev with the rest of this batch.
2026-08-10 05:57:53 -04:00
Joseph Doherty 8769ee9765 fix(sessions): keep named-pipe socket paths inside the macOS sun_path limit (NEXT-01)
The pipe name mxaccess-gateway-{pid}-session-{32hex} plus .NET's
CoreFxPipe_ prefix overflowed the 104-byte Unix-domain-socket path limit
under the default per-user macOS TMPDIR (~49 chars), so every test that
opened a real pipe threw ArgumentOutOfRangeException at pipe creation
unless TMPDIR=/tmp was exported. Rename to mxgw-{pid}-{sessionUid} (the
session guid hex without the session- prefix; worst-case 43 chars) and
shorten the three test-fixture names the same way. Uniqueness is
unchanged: gateway pid + full session guid. The worker receives the pipe
name via its launch command line, so mixed Server/Worker deploy SHAs are
unaffected. Docs updated in the same change (gateway.md,
GatewayProcessDesign, GatewayConfiguration, Sessions, CLAUDE.md); new
regression test pins the format and the length budget.

Verified: SessionManagerTests 39/39; SessionWorkerClientFactory,
GatewayEndToEndFakeWorkerSmoke, WorkerClient, and ReconnectReplay suites
33/33 under the default macOS TMPDIR — this also retires the
previously-misdiagnosed 'macOS pipe-timeout test failures': they were
this path-length throw, not a timeout-message defect.
2026-08-10 05:54:11 -04:00
Joseph Doherty 0152180929 build(tst-11): stop Windows builds stamping git stderr into InformationalVersion (NEXT-09)
MSBuildThisFileDirectory ends in a backslash, which escaped the closing quote
of the Exec command on Windows; git then failed and, because the target runs
with ContinueOnError + ConsoleToMSBuild (which mixes stderr into
ConsoleOutput), the failure text was stamped as the source revision — an
observed stamp read '0.1.2+fatal: cannot change to ...'. Append '.' to the
quoted path so the trailing separator can no longer escape the quote, and
gate SourceRevisionId on a short-SHA shape so no future git failure text can
become the revision either. Windows verification runs on windev with the
rest of this batch; macOS stamp confirmed unchanged (0.2.0+75c71ad).
2026-08-10 05:49:44 -04:00
Joseph Doherty 75c71adf45 docs(gateway): MxStatusDetail vocabulary table for statuses consumers
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 1m12s
ci / java (push) Successful in 2m2s
ci / portable (push) Successful in 8m14s
Requested by OtOpcUa after 06/S-1 closure: their OPC UA status mapping needs
the per-code detail vocabulary (a byte-truncation bug mapped refused writes
into the Good band). Source: installed toolkit interop enum via the MXAccess
analysis project.
2026-08-09 20:13:57 -04:00
Joseph Doherty 53f69cde37 Merge fix/write-completion-plain-writes: correlate OnWriteComplete onto plain Write/Write2 (06/S-1 follow-up)
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 1m25s
ci / java (push) Successful in 2m23s
ci / portable (push) Successful in 8m5s
2026-08-09 19:48:26 -04:00
Joseph Doherty 91d8715c74 docs: write-completion correlation covers all four unary write kinds
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 1m51s
ci / java (push) Successful in 2m16s
ci / portable (push) Successful in 8m26s
2026-08-09 19:47:04 -04:00
Joseph Doherty 794c44246a chore(clients): regenerate Go/Java bindings and Rust vendored proto for the plain-write statuses comments 2026-08-09 19:47:04 -04:00
Joseph Doherty b0e65d4f31 feat(worker): correlate OnWriteComplete onto plain Write/Write2 replies (06/S-1 follow-up)
OtOpcUa's dominant FreeAccess write path goes out as MX_COMMAND_KIND_WRITE,
not WriteSecured — the original 06/S-1 brief mis-scoped the correlation, so
a refused plain write was invisible on the unary reply (verified live on
windev 2026-08-09). ExecuteWrite/ExecuteWrite2 now use the same pre-call
version baseline + bounded pump-wait as the secured kinds. Bulk writes stay
fire-and-forget.
2026-08-09 19:47:00 -04:00
Joseph Doherty b948e6975e feat: correlate OnWriteComplete onto WriteSecured/WriteSecured2 unary replies (OtOpcUa 06/S-1)
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 1m9s
ci / java (push) Successful in 2m3s
ci / portable (push) Successful in 7m20s
2026-08-09 12:52:18 -04:00
Joseph Doherty 45c530da6e chore(clients): regenerate Go/Java bindings and Rust vendored proto for the statuses contract comments
ci / nightly-windev (push) Has been skipped
ci / java (push) Failing after 7s
ci / windows-x86 (push) Failing after 1m8s
ci / portable (push) Successful in 7m18s
2026-08-09 12:37:35 -04:00
Joseph Doherty c867aca36b test(worker): deterministic pump-wait ordering, env hermeticity, ResolveWriteCompletionTimeout coverage 2026-08-09 12:37:35 -04:00
Joseph Doherty 436ef69f07 fix(worker): thread the completion cache through CreateForTesting
ci / nightly-windev (push) Has been skipped
ci / java (push) Failing after 1m57s
ci / windows-x86 (push) Failing after 1m57s
ci / portable (push) Failing after 8m28s
2026-08-09 12:33:20 -04:00
Joseph Doherty 2b468bd8fc docs: write-completion correlation configuration and semantics
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 43s
ci / java (push) Failing after 1m56s
ci / portable (push) Failing after 4m24s
2026-08-09 12:28:30 -04:00
Joseph Doherty 431a096cab feat(gateway): configurable worker write-completion wait (MxGateway:Worker:WriteCompletionWaitMilliseconds) 2026-08-09 12:27:30 -04:00
Joseph Doherty b0e2b8ba74 test(worker): write-completion correlation executor coverage 2026-08-09 12:26:23 -04:00
Joseph Doherty 66fe063410 feat(worker): bounded pump-wait correlates OnWriteComplete onto secured-write replies 2026-08-09 12:24:23 -04:00
Joseph Doherty 8de23086d0 feat(worker): share the completion cache between sink and session 2026-08-09 12:23:20 -04:00
Joseph Doherty a76ecdd59c feat(worker): event sink records OnWriteComplete rows into the completion cache 2026-08-09 12:22:54 -04:00
Joseph Doherty fc23a65cca feat(worker): versioned OnWriteComplete completion cache 2026-08-09 12:21:51 -04:00
Joseph Doherty aec95b78c9 docs(proto): document the correlated write-completion statuses contract 2026-08-09 12:20:36 -04:00
Joseph Doherty f9229ee44d docs(plan): write-completion correlation implementation plan 2026-08-09 12:20:00 -04:00
Joseph Doherty 5dbe93d13e docs(design): WriteSecured completion correlation onto the unary reply (OtOpcUa 06/S-1) 2026-08-09 12:15:07 -04:00
Joseph Doherty 129e47e541 docs(tracking): close NEXT-07, file NEXT-08/09/10, record the runner token reset
NEXT-07 is struck: windev was redeployed from origin/main (a346d51) and the service is
healthy, and the root cause the row predicted is confirmed -- the 2026-06-25 build's
Auth.ApiKeys 0.1.2.0 supports auth-DB schema 2 while the database sits at schema 3, which
is the current shared-lib version, so deploying forward was the fix rather than touching
the DB. The original text stays for the triage record.

Three findings surfaced by that work, each deliberately left for the next cycle rather
than patched in passing:

- NEXT-08: the shared GLAuth offers no TLS, so SEC-06 makes GatewayConfiguration.md's
  "deployed hosts must set Ldaps or StartTls" unsatisfiable for anything genuinely
  labelled Production. windev's relabel to Staging is honest for a dev rig but defers
  the posture question rather than answering it.
- NEXT-09: Directory.Build.props:29 quotes a path ending in a backslash, so the SHA-stamp
  git invocation is malformed on Windows and ContinueOnError stamps git's stderr into
  InformationalVersion -- a Windows binary cannot be correlated to a commit, which is what
  TST-11 exists to guarantee.
- NEXT-10: glauth.md's pre-provisioned-user table contradicts both the directory and its
  own dashboard section, and was the root cause of the NEXT-06 fixture drift. Reconciling
  it sweeps the OPC-UA group taxonomy, so it is scoped out here on purpose.

The TST-30 runner work is hygiene, not closure: runner-1 now mounts its registration token
from a 0600 file like runner-2, but both still share one instance-scope token that was
world-readable for months and is provably still live. Gitea 1.26.4 cannot rotate it from
the CLI or API, so the UI reset is recorded as a pending operator action with its
follow-through (refresh the token file, shred the token-bearing compose backups).
2026-08-07 10:31:29 -04:00
Joseph Doherty 1d6858939d docs(sec-36): record the completed windev dashboard verification
SEC-36's primary check -- dashboard /login through the real DashboardAuthenticator
search bind -- was deferred because windev's gateway was crash-looping on the stale
deployment filed as NEXT-07. That host was redeployed 2026-08-07, so the check ran:
login as multi-role returns 302 with the dashboard cookie and the authenticated page
renders the admin nav, while an anonymous control still redirects to /login. The
rotated service-account credential is now proven end-to-end on the deployed host, not
only by the equivalent ldapsearch primitive, and the runbook's Correction 3 is past
tense throughout rather than describing a fault that no longer exists.

Also record why windev runs the Staging environment name. The redeploy tripped SEC-06's
Production hard-stop on Ldap:Transport=None, and windev cannot satisfy it: it binds the
shared GLAuth, which offers no TLS, and runs Dashboard:DisableLogin=true. The Production
label contradicted its own configuration, so the host was relabelled rather than the
guard weakened -- exactly the permissive-staging-rig case the SEC-35 section already
carves out.
2026-08-07 10:31:15 -04:00
Joseph Doherty de67b45d04 test(ldap): align DashboardLdapLiveTests fixtures with the shared directory (NEXT-06)
The suite's fixtures had drifted from the shared GLAuth config, so a green run
proved nothing about the service-account bind: the only success-path test used
admin/admin123, but the directory's admin carries the standard dev password, and
the "not an admin" test used a readonly user that does not exist there at all --
it passed via the user-not-found branch rather than the group-missing branch it
names.

Realign to real users from scadaproj/infra/glauth/config.toml: admin/password
(othergroups include GwAdmin, gid 5610) for the success path, and
gw-viewer/password (GwReader only, gid 5611) for the bind-succeeds-but-no-role
path. Both are published dev credentials documented in glauth.md, not secrets.

The gw-viewer test drops its old no-leak assertion on the credential literal:
the real password is the word "password", which legitimately occurs in the
generic denial text, so the check would fail for the wrong reason. The no-leak
property is still covered with a distinctive literal by the wrong-password test.
In its place the test now asserts the property this fixture is uniquely able to
prove -- an authorization failure must be reported with the same message as an
authentication failure, so it cannot be used to enumerate valid accounts.

appsettings ships Server=localhost, so document the MxGateway__Ldap__Server
override the suite needs to reach the shared GLAuth alongside the existing
MXGATEWAY_RUN_LIVE_LDAP_TESTS and ServiceAccountPassword variables.

Verified live: Failed: 0, Passed: 5 against 10.100.0.35:3893.
2026-08-07 10:03:10 -04:00
Joseph Doherty 3d991d2160 docs(tst-30): remove mislabelled macOS instance runner (id 4)
The local act_runner on this Mac registered as instance runner id 4 with
ubuntu-latest/22.04/20.04 labels, so it competed with the two docker
runners on 10.100.0.35 for Linux jobs it had no Docker daemon to run --
13 of the last 20-run window in historiangw landed on it and all but one
failed. Registration deleted; local config kept disabled for re-use with
mac-specific labels.
2026-08-07 10:02:29 -04:00
Joseph Doherty 05667169eb docs: sync runner-topology and SEC-36 rotation prose with 2026-08-07 executed state 2026-08-07 09:28:49 -04:00
Joseph Doherty 9760497d66 docs(sec-36): rotation executed 2026-08-07; runbook host-path/vd03/verification corrections; new findings (LDAP test fixtures, windev stale deploy) 2026-08-07 09:21:54 -04:00
Joseph Doherty 5b153dac74 docs(clients): record 2026-08-07 publish of 0.2.0 client family (Java 0.2.1); cargo token needs Bearer prefix 2026-08-07 09:15:47 -04:00
Joseph Doherty 41e86481e2 docs(tst-30): second runner gitea-runner-2 live; close operator action 2026-08-07 09:12:00 -04:00
Joseph Doherty a346d514dd test(contracts): scope command-reply fixture invariants past the CLI-40/41 authenticate-user malformed-reply fixtures
ci / portable (push) Successful in 14m0s
ci / java (push) Successful in 6m50s
ci / windows-x86 (push) Failing after 1m21s
ci / nightly-windev (push) Has been skipped
The blanket loop asserted HRESULT/Statuses/ReturnValue on every command_replies fixture, but the authenticate-user.* fixtures added for the malformed-reply and credential-redaction contracts deliberately omit them (NRE on ReturnValue.DataType). Keep universal Kind/ProtocolStatus invariants for all; apply the MXAccess-detail block only to fixtures that carry it. Test-only.
2026-08-07 08:48:49 -04:00
Joseph Doherty a2d3f66b8b docs(archreview): record next-cycle candidate findings + pending operator actions surfaced during remediation
ci / windows-x86 (push) Successful in 1m19s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m19s
ci / portable (push) Failing after 4m55s
2026-08-07 08:48:03 -04:00
Joseph Doherty 93d84019b9 docs(tracking): sync IPC-23 domain register to Done (doc wave landed; Grpc.md row intentionally scoped out — DrainEvents is a worker diagnostic)
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 1m19s
ci / java (push) Successful in 2m4s
ci / portable (push) Failing after 4m43s
2026-08-07 08:47:31 -04:00
Joseph Doherty 6d26ed094c docs(tracking): close old-tracker CLI-24, CLI-34 as Done (2026-07-12 review old-tracker actions; both incidentally fixed)
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m22s
ci / windows-x86 (push) Successful in 1m20s
ci / portable (push) Failing after 4m26s
2026-08-07 08:11:57 -04:00
Joseph Doherty 4201da63d2 docs(tracking): flip IPC-24/IPC-25 to Done in the Contracts&IPC domain register (missed by the codegen-wave tracker update)
ci / windows-x86 (push) Failing after 1m19s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m17s
ci / portable (push) Failing after 4m51s
2026-08-07 08:10:39 -04:00
Joseph Doherty 9c780f8164 Merge branch 'fix/cli-39-version-train'
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 1m20s
ci / java (push) Successful in 2m12s
ci / portable (push) Failing after 4m44s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
2026-08-07 08:09:39 -04:00
Joseph Doherty 440e7cf03d fix(CLI-39): bump Contracts nupkg to 0.2.0; scope pack-clients.ps1 regexes
Code review of the CLI-39 branch caught an Important gap: Contracts.csproj
was left at the already-published 0.1.2 while the .NET Client moved to
0.2.0. Invoke-PackDotnet in scripts/pack-clients.ps1 packs and publishes
both ZB.MOM.WW.MxGateway.Contracts and .Client through the same -Publish
loop, and the new collision guard runs every nupkg it finds through
Assert-GiteaPackageNotPublished. Left as-is, the next real .NET publish
would pack Contracts at 0.1.2, the guard would correctly refuse to
republish it, and the loop would abort mid-way with Client (alphabetically
first) possibly already pushed -- the two packages permanently out of
lockstep.

- src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csproj:
  <Version> 0.1.2 -> 0.2.0, matching the .NET Client (they have always
  released together).
- src/Directory.Build.props: corrected a comment that was now stale --
  it claimed the repo-wide 0.1.2 default was kept to match the Contracts
  package, which is no longer true now that Contracts.csproj overrides it.
  The <Version> value itself is unchanged; Server/Worker/Tests staying at
  0.1.2 is a separate, not-yet-made decision, out of scope for CLI-39.
- docs/ClientPackaging.md: Contracts.csproj added as a fifth manifest in
  the Versioning section, with the near-miss recorded.

Also hardened scripts/pack-clients.ps1 per the same review: the Python
(pyproject.toml) and Rust (Cargo.toml) version-extraction regexes now
scope to the [project]/[package] section header instead of matching the
first "version = ..." line anywhere in the file (Cargo.toml has an
identical second one under [workspace.package] -- matching whichever came
first was luck of ordering, not correctness). One-line comment added on
the nuget filename-parse regex.

Verified live against the real Gitea registry: Contracts and Client both
still refuse at 0.1.2 and both now pass at 0.2.0, including running the
actual Invoke-PackDotnet filename-parse-then-guard logic against two
freshly packed real .nupkg files. dotnet build of Contracts.csproj and the
client slnx both clean. No publish performed.
2026-08-07 08:07:40 -04:00
Joseph Doherty ae605d2368 Merge remote-tracking branch 'origin/fix/wrk-22-25-seam'
ci / windows-x86 (push) Failing after 1m19s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m10s
ci / portable (push) Failing after 4m27s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
#	archreview/2026-07-12/remediation/30-contracts-ipc.md
2026-08-07 08:01:10 -04:00
Joseph Doherty 9b2abef4e1 fix(CLI-39): bump client versions off published 0.1.2; guard the publish pipeline
Converges all five clients on one version after four had drifted onto the
already-published 0.1.2/0.1.1 while their APIs kept changing underneath it:

- Rust Cargo.toml [package] + [workspace.package] -> 0.2.0 (CLIENT_VERSION
  already derives from CARGO_PKG_VERSION, no separate edit).
- Python pyproject.toml + version.py -> 0.2.0; new test asserts __version__
  matches pyproject.toml (closes the CLI-26 residual drift mode).
- Go mxgateway/version.go ClientVersion -> 0.2.0.
- .NET ZB.MOM.WW.MxGateway.Client.csproj <Version> -> 0.2.0.
- Java -> 0.2.1, not 0.2.0: the live Gitea Maven feed already had 0.2.0
  published (2026-06-26), before the CLI-37/38/40/41 conformance fixes
  changed the client's observable behavior, so reusing 0.2.0 would label
  two different APIs identically. Recorded as an exception in
  docs/ClientPackaging.md's new Versioning section.

Publish-pipeline guards:

- scripts/tag-go-module.ps1 implements the CLI-21 guard: after semver
  validation it refuses to tag unless clients/go/mxgateway/version.go's
  ClientVersion already matches the requested tag version.
- scripts/pack-clients.ps1 gains a Gitea package-registry collision guard
  wired into every per-language -Publish step; it aborts if the target
  name+version already exists rather than force-overwriting. Verified live
  against the real Gitea registry (credentials already present in this
  environment) — correctly refuses on every known-published artifact and
  passes on every unpublished target.

Docs updated in the same commit: docs/ClientPackaging.md (new Versioning
section), and the five client READMEs' stale 0.1.1/0.1.2 example versions.

No .proto changes. No publish performed.
2026-08-07 07:58:49 -04:00
Joseph Doherty 815e58d28b docs(tracking): record WRK-22/24/25/27 + IPC-26 windev evidence (377 pass)
ci / java (push) Successful in 2m7s
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 1m12s
ci / portable (push) Failing after 4m12s
2026-08-07 07:52:44 -04:00
Joseph Doherty 8df35cd63a fix(WRK-22,WRK-24,WRK-25,WRK-27,IPC-26): worker write-seam hardening
ci / java (push) Successful in 2m7s
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 1m13s
ci / portable (push) Failing after 4m6s
WRK-22/IPC-26: tombstone a WriteAsync/WriteBatchAsync cancelled while
waiting for the write lock (PendingFrame.Claimed under _gate; DequeueNext
skips cancelled, claims the frame it returns) so a cancelled write never
reaches the wire unless already claimed mid-write (documented residual).

WRK-25: add WriteBatchAsync; RunEventDrainLoopAsync submits the drained
event batch through it, so a burst of N events costs one flush not N.
IPC-30 oversized-event structured fault preserved via FindOversizedEvent.

WRK-24: reject a below-1024 negotiated frame maximum at the handshake
(MinNegotiableFrameBytes, matching GatewayOptionsValidator floor).

WRK-27: alarm poll advertises StaCallInProgress on the heartbeat snapshot
so the watchdog suppresses to the ceiling, not the grace.

Docs (WorkerFrameProtocol.md, MxAccessWorkerInstanceDesign.md) and the
2026-07-12 remediation registers/change-log updated in the same commit.
2026-08-07 07:50:38 -04:00
Joseph Doherty a55956ffa5 Merge branch 'fix/tst-30-runner-docs'
ci / java (push) Successful in 2m11s
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 1m38s
ci / portable (push) Failing after 17m43s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
2026-08-07 07:50:21 -04:00
Joseph Doherty aba22358f5 Merge branch 'fix/ipc-27-descriptor-test'
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m14s
ci / java (push) Successful in 2m8s
ci / portable (push) Failing after 4m45s
2026-08-07 07:49:33 -04:00
Joseph Doherty b604fed72b docs(TST-30): document shared-runner CI bottleneck + second-runner operator runbook
Doc half of TST-30 (single shared Gitea runner is a CI throughput/availability
bottleneck): docs/GatewayTesting.md's Continuous Integration section gains a
"Runner capacity is shared and finite" subsection covering the maxParallel=1
instance-level runner shared with dohertj2/lmxopcua, the ~20-30 min queue
latency observed under cross-repo contention, and Gitea 1.26's missing run
cancel/delete API. The existing "windev tier down" degraded-mode paragraph now
also covers "runner contended" as a reason to bypass the queue via
CI_SHA=<sha> scripts/ci/run-windev-ci.sh <mode> or the manual windev worktree
flow, generalizing it per the finding's design note.

New operator runbook docs/runbooks/TST-30-second-ci-runner.md carries the
actual runner registration (option a: second act_runner instance on
10.100.0.35 with the same container.network: traefik config, recommended;
option b: dedicated labelled runner, escalation only; option c: runner on
windev, rejected) plus verification steps and the no-cancel caveat. The
optional workflow-level concurrency group is documented as unverified --
framed as "verify before relying on it" -- and left unimplemented in ci.yml,
since registering the runner and any runs-on gating is operator/infra work
outside this repo's tree.

Tracking: TST-30 -> Done (doc half; runner registration operator-pending) in
both registers + change-log row.
2026-08-07 07:47:37 -04:00
Joseph Doherty 6060d21995 fix(IPC-27): close descriptor-freshness blind spots for enums, services, and Galaxy
ClientProtoInputTests.Descriptor_ContainsEveryContractMessageAndField only
compared messages and fields, and only enumerated the gateway/worker
descriptors, so a new enum value, a new RPC, or any galaxy_repository.proto-only
change would not redden the test even though it is documented as the primary
protoc-free CI gate.

Rename to Descriptor_ContainsEveryContractSymbol and extend the reflection walk
on both sides (published protoset and in-process contract) to also collect
enums/enum values ({enumFullName}, {enumFullName}/{valueName}) and
services/methods ({serviceFullName}, {serviceFullName}/{methodName}), and add
GalaxyRepositoryReflection.Descriptor to the enumerated files. The comparison
stays a flat, order-insensitive string-set diff with no protoc dependency.

Update docs/ClientProtoGeneration.md and docs/Contracts.md prose from
"message or field" to the full symbol coverage.

Red-path proof: pointed the test at the pre-IPC-01 stale protoset and confirmed
it failed naming max_frame_bytes, several MxCommandKind/AlarmProviderMode enum
values, MxAccessGateway/StreamAlarms and GalaxyRepository/BrowseChildren, and
the galaxy_repository.v1.* surface; restored the real path and re-ran green.

Flips IPC-27 to Done in the 2026-07-12 remediation tracker and register.
2026-08-07 07:47:18 -04:00
Joseph Doherty 1c2f3a62c1 Merge branch 'fix/sec-36-ldap-secret'
ci / windows-x86 (push) Successful in 1m16s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m31s
ci / portable (push) Failing after 4m37s
2026-08-07 07:44:43 -04:00
Joseph Doherty 0646c73e48 Merge branch 'fix/ipc-24-25-codegen'
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m15s
ci / java (push) Successful in 2m5s
ci / portable (push) Failing after 4m32s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
2026-08-07 07:42:35 -04:00
Joseph Doherty eacdd2d453 fix(IPC-23,IPC-24,IPC-25,IPC-32): proto-comment regen wave + codegen-freshness guards
Proto comments (comment-only, no wire change):
- mxaccess_worker.proto GatewayHello.max_frame_bytes: every worker->gateway frame
  must serialize within the negotiated max; reply builders truncate (IPC-23).
- mxaccess_gateway.proto DrainEventsReply: count-cap + byte-cap, drain-until-empty
  caller contract (IPC-23).
- mxaccess_gateway.proto ReplayGap.oldest_available_sequence: empty-ring value is
  highest-observed+1, oldest-1 resume formula stays valid (GWC-25 deferred amendment).

Regen wave: Contracts/Generated (C# XML doc), rust vendored protos (byte-copy),
Go bindings (worker binding was genuinely stale - lacked MaxFrameBytes entirely),
Python worker _pb2 (real descriptor delta), Java aggregates (javadoc, zero
protobuf-version churn under the pinned toolchain), client descriptor set.

IPC-24: pinned Java toolchain regenerates with no gencode-version churn, so the
unconditional churn-revert step in ci.yml is a fossil - deleted it; git diff is
now a true message-level drift gate for the single-file Java aggregates.

IPC-25: pin protoc-gen-go v1.36.11 / protoc-gen-go-grpc 1.6.2 in the Go generate
script (+ fix a latent pwsh-7 parse bug); add Check 4 to check-codegen.ps1
(regenerate Go+Python bindings, fail on diff, tool-missing fails not skips); add
the pinned-generator installs to the portable CI job.

IPC-32: relabel check-codegen banners 1/4..4/4 (folded into the Check 4 edit).

Docs: ClientProtoGeneration.md, Contracts.md, GatewayTesting.md, build.gradle
checkGeneratedClean caveat. Tracking: IPC-23/24/25/32 -> Done, GWC-25 proto note
resolved, change-log 2026-08-07.
2026-08-07 07:41:18 -04:00
Joseph Doherty 8c312c717c fix(SEC-36): scrub committed dev LDAP service-account password; add user-secrets channel + rotation runbook
Repo-side half of SEC-36. The appsettings.json plaintext was already discharged
before this branch (HEAD ships the fail-closed ${secret:ldap/mxgateway/bind}
store reference), so the residual leak was the literal value in glauth.md,
docs/GatewayTesting.md, and the historical archreview SEC-06 evidence -- all
scrubbed to <service-account-password> placeholders pointing at the source of
truth scadaproj/infra/glauth/.

- csproj: add <UserSecretsId>mxaccessgw-server</UserSecretsId> (dev channel)
- GatewayOptionsValidator: blank-password message now names both channels
  (dev user-secrets, deployed MxGateway__Ldap__ServiceAccountPassword)
- test: assert the message names both channels
- docs: GatewayConfiguration.md (three channels + rotation note), glauth.md
  (placeholders + rotation-required + runbook pointer), GatewayTesting.md
- new operator runbook docs/runbooks/SEC-36-ldap-credential-rotation.md
  (live rotation + NSSM staging remain operator-pending)
- tracking: SEC-36 -> Done (repo-side) in both registers + change-log

Deviation: kept the ${secret:} reference in appsettings.json rather than
deleting it (spec step 2 assumed the stale plaintext baseline); deleting it
would regress the shipped/documented/tested secret-store channel.

git grep -i for the old value is empty across all tracked files.
2026-08-07 07:40:41 -04:00
Joseph Doherty 10534ec906 docs(TST-27,WRK-26,CLI-42,CLI-43,IPC-28): P1 doc-drift batch, discharges IPC-29
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m17s
ci / java (push) Successful in 2m10s
ci / portable (push) Failing after 3m53s
TST-27: docs/GatewayConfiguration.md's ShowTagValues row no longer says
"Reserved" — it now states what false (default) does (DashboardEventBroadcaster
blanks tag values from a deep-cloned MxEvent before the SignalR events-hub
mirror), the security relevance (no per-session hub ACL yet, so this
redaction is the only thing between a low-trust Viewer and other sessions'
tag values), and the honest scope limit (does not cover /browse).

WRK-26 (discharges IPC-29): docs/MxAccessWorkerInstanceDesign.md's "Outbound
Queues" section rewritten from the stale five-level priority list to the
two-class Control/Event scheduler actually shipped, with the collapsed-
decision rationale, and the overflow paragraph rewritten to the implemented
fail-fast. docs/WorkerFrameProtocol.md gained a "Write Scheduling And
Sequencing" section describing HEAD truthfully: WRK-23's peek-stamp-commit
sequencing is live, WRK-25's event-batch flush coalescing is not (the drain
loop still awaits each event write individually), and WRK-22's cancellation
tombstone is not yet defined (noted as pending, not documented as shipped).

CLI-42: clients/rust/README.md and docs/ClientPackaging.md document the
vendored Rust proto layout matching build.rs — repo-path-first resolution
falling back to clients/rust/protos/, the check-codegen.ps1 Check 3 refresh
rule, and why cargo package/publish run without --no-verify.

CLI-43: docs/style-guides/JavaStyleGuide.md now says Java 17 (Ignition 8.3
baseline), mirroring CLI-12's wording, matching the shipped build.gradle.

IPC-28: docs/Grpc.md's exception-mapping prose gained CommandTooLarge ->
ResourceExhausted, and the Invoke section gained the oversized-payload
sentence, cross-referencing GatewayConfiguration.md's headroom rule.

Tracking: TST-27, WRK-26, CLI-42, CLI-43, IPC-28 flipped to Done and IPC-29
marked discharged-by-WRK-26 in 00-tracking.md and the 20/30/50/60 domain
registers, with a 2026-08-07 change-log entry.

Doc-only change; no source, proto, or test edits.
2026-08-07 07:26:58 -04:00
Joseph Doherty 97f79e79ef Merge remote-tracking branch 'origin/fix/wrk-21-drain-cluster'
ci / windows-x86 (push) Successful in 1m27s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m11s
ci / portable (push) Failing after 4m7s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
#	docs/MxAccessWorkerInstanceDesign.md
2026-08-07 07:18:58 -04:00
Joseph Doherty 758277bc62 docs(tracking): record WRK-21 review follow-ups (monotonic budget, guarded fallback) with windev evidence
ci / windows-x86 (push) Successful in 1m19s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m15s
ci / portable (push) Successful in 9m30s
2026-08-07 07:13:46 -04:00
Joseph Doherty 6bc3f9b991 fix(WRK-21): make drain budget monotonic at the reserve boundary; guard the reply-too-large fallback
ci / windows-x86 (push) Successful in 1m16s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m17s
ci / portable (push) Successful in 9m31s
Review follow-ups on the WRK-21 cluster.

1. ResolveDrainReplyByteBudget was a step function, not a floor: just above the
   64 KiB reserve the budget collapsed to a few bytes (at the validator-permitted
   floor MaxMessageBytes = 1024 + 64 KiB it was exactly 1024), too small to move a
   byte-heavy event, so DrainEvents truncated on every call and the drain-until-
   empty loop never terminated. It now takes the max of (frameMax - reserve) and
   frameMax/2, so the budget is monotonic and never below half the frame max. New
   test DrainEvents_AtValidatorFloorFrameMax_MakesProgressAndTerminates drives a
   byte-heavy queue at the exact validator floor and asserts it drains to empty
   with no head ever reported oversized.

2. The reply-too-large fallback write is now itself size-guarded
   (WriteReplyTooLargeFallbackAsync, used by both the control and STA reply seams):
   at a pathologically tiny negotiated max below the gateway's floor the fallback
   could also throw MessageTooLarge and — uncaught — kill the session, defeating the
   "no diagnostics command is session-fatal" invariant. It now log-and-swallows;
   comment notes WRK-24 adds the negotiated-max lower bound that makes it unreachable.

3. Corrected the RepeatedFieldOverheadBytes doc comments: WorkerEvent.CalculateSize()
   already includes the event's tag and length prefix (the same shape the reply's
   repeated events field packs), so the 8 bytes is pure slack over an already-
   conservative estimate, not compensation for a missing wrapper.
2026-08-07 07:09:13 -04:00
Joseph Doherty 34db678635 Merge branch 'fix/cli-40-41-44'
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m19s
ci / java (push) Successful in 2m14s
ci / portable (push) Failing after 4m12s
2026-08-07 07:08:27 -04:00
Joseph Doherty 0d874f91ee fix(CLI-40): scrub the credential from the redacted error's structured reply, route MXACCESS_FAILURE to MxAccess (Rust), fix Go Subscribe terminal-error drop
Code-review follow-up on the CLI-40/41/44 branch.

ISSUE 1 (all five, critical): the message-only scrub still leaked the
server-echoed credential through the redacted error's structured reply accessor
(.NET Reply/Statuses, Java reply()/protocolStatus(), Go MxAccessError.Reply via
errors.As, Rust reply()/into_reply(), Python raw_reply). The redacted error now
carries a scrubbed clone of the reply (protocol_status.message,
diagnostic_message, statuses[].diagnostic_text), with per-language tests asserting
the reply accessor no longer contains the credential.

ISSUE 2 (Rust, critical): ensure_command_success routed MXACCESS_FAILURE to
Error::Command (unlike the other four clients), bypassing attach_secrets and
leaking via derived Debug/Display. MXACCESS_FAILURE now routes to Error::MxAccess,
fixing the cross-client inconsistency.

ISSUE 3 (Go, important): the CLI-44 terminal send was unconditionally
non-blocking, dropping a genuine terminal error under a full buffer on the
never-drop SubscribeEvents path. It is now reserved-slot-non-blocking only for the
cancel-on-overflow path and blocking for the never-drop path.

New shared fixture authenticate-user.echoed-credential-mxaccess-failure.reply.json
wired into all five suites. Minors: whitespace-secret guard on .NET/Java redact
helpers; Java preserves exception subtype on redaction; redaction-helper unit
tests (Go/Java/.NET). Docs (ClientBehaviorFixtures.md, ClientLibrariesDesign.md)
updated to make the structured-field claim true.
2026-08-07 07:04:56 -04:00
Joseph Doherty c9925688f5 docs(tracking): record the WRK-21 cluster as Done with windev evidence
ci / java (push) Successful in 2m28s
ci / windows-x86 (push) Failing after 1m28s
ci / nightly-windev (push) Has been skipped
ci / portable (push) Successful in 8m38s
Change-log row for 2026-08-07: what landed for WRK-21/WRK-28/WRK-23/IPC-30, why
IPC-23 stays In progress (proto-comment/doc wave pending), and the verification
evidence — macOS NonWindows build + validator tests, and the documented windev
path (scripts/ci/windev-worker-ci.ps1 -Mode test) at a256560: x86 Worker build
clean, Worker.Tests 367 passed / 0 failed / 11 skipped.
2026-08-07 06:54:24 -04:00
Joseph Doherty 2aac29618e Merge branch 'fix/sec-33-34'
ci / java (push) Successful in 2m16s
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m25s
ci / portable (push) Successful in 8m32s
2026-08-07 06:51:31 -04:00
Joseph Doherty a2565604df test(WRK-21): keep the drain-to-empty walk inside the pipe harness envelope
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m14s
ci / java (push) Successful in 2m12s
ci / portable (push) Successful in 8m2s
PipePair runs both ends of a duplex pipe in one process with blocking
FlushFileBuffers under every frame write, so it wedges after roughly 85 large
round trips. Drain the full 10,000 byte-heavy events to empty at the queue layer,
where the no-loss property actually lives, and keep the pipe walk at 1,000 events
(29 replies) so it still proves the split end to end. Also give the truncation
test's budget slack: item handle 0 is a proto3 default and is not serialized, so
the probe measurement is a lower bound on the fixture's per-event cost.
2026-08-07 06:50:17 -04:00
Joseph Doherty 193daa9ee8 fix(SEC-33,SEC-34): address code review — missed docs, key-id guard comment, test consolidation
Same-commit docs rule (were missed in the prior commit):
- docs/GalaxyRepository.md: SnapshotCachePath now documents the per-OS derived
  default and the GalaxyRepositoryOptionsValidator rooting/validity enforcement.
- A2-galaxyrepository-adoption-handoff.md: correct the now-inaccurate NSSM caveat
  (SnapshotCachePath override is optional, not required; blank seeds a rooted host
  default, no silent no-op) and repoint the option-validation item at the new
  GalaxyRepositoryOptionsValidator.

SEC-34 guard confirmed and documented: TryParseKeyId's '_' split cannot truncate a
key id because both — and the only — gateway key-creation paths
(ApiKeyAdminCommandLineParser.IsValidKeyId, DashboardApiKeyManagementService.ValidateKeyId)
restrict key ids to IsAsciiLetterOrDigit || '.' || '-', and key ids are never
library-generated. Added a citing comment; no behavior change.

Test consolidation: moved the three host-start SqlitePath overrides into
TestHostEnvironmentInitializer (per-process temp store, mirroring Secrets__SqlitePath)
so future host-start tests auto-cover.
2026-08-07 06:49:24 -04:00
Joseph Doherty dc7fd16dd5 fix(CLI-40,CLI-41,CLI-44): exact-secret scrub, uniform malformed-reply contract, Go terminal-error mislabel
CLI-40: port the exact-secret credential scrub to Rust/Java/.NET (Go/Python
already did it). AuthenticateUser/WriteSecured(2) helpers now redact the exact
caller-supplied secret from any surfaced error, as defense-in-depth on top of the
by-construction guarantee. Rust hand-writes a redacting Debug (derived Debug would
leak the reply); Java/.NET rebuild the same exception type with the redacted
message and do not carry the secret-bearing original forward (so ToString/stack
traces stay clean too).

CLI-41: uniform malformed-reply contract for AuthenticateUser/ArchestrAUserToId/
AddBufferedItem across all five clients — typed payload, else a present int32
return_value, else a typed malformed-reply error. Fixes Go/Java silent-0, .NET
NRE, and Rust's own internal inconsistency.

CLI-44: the Go event goroutine's Recv-error path now uses a non-blocking
sendTerminalEventResult on the reserved slot, so a genuine terminal stream error
is reported as itself instead of being mislabeled ErrSlowConsumer under overflow.

Riders from the CLI-37/38 review: (a) .NET ToDiagnosticSummary and Python
_mxaccess_message surface the raw success member (diagnostics-only parity with
Rust); (b) the status-conversion fixture carries an independent wantSuccess
boolean and the Go/.NET fixture tests assert against it instead of recomputing
the formula under test.

Shared fixtures (authenticate-user.{echoed-credential,missing-payload,
return-value-only}.reply.json) + manifest + ClientBehaviorFixtures.md +
ClientLibrariesDesign.md updated in the same change. Tracking: CLI-40/41/44 -> Done.
2026-08-07 06:42:40 -04:00
Joseph Doherty 7e7f7cad84 fix(SEC-33,SEC-34): host-meaningful path rooting; verification-cache invalidate race
SEC-33: make rooting host-meaningful and stop shipping foreign-platform literals.
- Delete IsRootedForAnyPlatform; AddIfNotRooted now uses Path.IsPathRooted (current OS).
- Promote AddIfNotRooted/AddIfInvalidPath to shared GatewayConfigPathRules so the new
  Galaxy validator reuses them and the two validators cannot drift.
- Remove Authentication:SqlitePath and Galaxy:SnapshotCachePath Windows literals from
  appsettings.json; the CommonApplicationData-derived code defaults take over. The
  Galaxy default is seeded as a configuration value before AddZbGalaxyRepository
  (SnapshotCachePath is init-only, so a PostConfigure mutation cannot compile).
- New GalaxyRepositoryOptionsValidator (ValidateOnStart) enforces a valid, host-rooted
  SnapshotCachePath when PersistSnapshot is true.
- Root-cause the stray junk-named auth DB: host start eagerly builds
  AuthSqliteConnectionFactory; under the non-rooted Windows literal on macOS SQLite
  wrote it relative to the test bin CWD. The three real-host-start tests now pin
  SqlitePath to a temp path.

SEC-34: verification cache Invalidate-vs-in-flight-repopulation race closed with a
per-key generation counter (bump-before-evict, snapshot-then-recheck). The expiry
cap (window 2) takes the documented fallback: the library verification identity
carries no ExpiresUtc, so the cache cannot cap at the key's expiry (donor-library ask).

GWC-24 rider: cap MxGateway:Events:QueueCapacity at int.MaxValue/2 so the derived
checked(2 * EventChannelCapacity) in WorkerClient cannot overflow at session creation.

SEC-35 (doc-only): note IsProduction() env-name semantics in GatewayConfiguration.md.

Docs updated same commit (GatewayConfiguration.md, Authentication.md) and tracking
registers/change-log flipped (00-tracking.md, 40-security-dashboard.md).
2026-08-07 06:36:01 -04:00
Joseph Doherty 7c2eaf09e2 test(WRK-21): size the byte-heavy drain fixture for the pipe harness
ci / java (push) Successful in 2m12s
ci / portable (push) Successful in 8m23s
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 11m46s
PipePair has no continuous read pump — the test thread drains the pipe only
while it sits in ReadUntilAsync — so multi-megabyte DrainEvents frames
interleaved with the heartbeat loop wedge both ends inside FlushFileBuffers,
each waiting for the other to read. Negotiate a 128 KiB frame maximum instead:
the 10,000 byte-heavy events still overflow it many times over, so every
assertion (bounded reply, reported truncation, no event loss across repeated
drains, surviving session) is unchanged.
2026-08-07 06:25:15 -04:00
Joseph Doherty d2bb32d97b Merge branch 'fix/gwc-28-29-30-polish'
ci / java (push) Successful in 2m44s
ci / windows-x86 (push) Successful in 1m3s
ci / nightly-windev (push) Has been skipped
ci / portable (push) Successful in 17m13s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
2026-08-07 06:20:14 -04:00
Joseph Doherty 404f7cd993 docs(archreview): close GWC-28, GWC-29, GWC-30, TST-28
Flip the four findings to Done in the 2026-07-12 tracking registers
(Gateway core + Testing) and in the per-domain registers of
10-gateway-core.md and 60-testing-docs-gaps.md; append the 2026-08-07
change-log row recording what shipped, the pre-fix red for GWC-28, and
the TST-28 mutation check.
2026-08-07 06:15:45 -04:00
Joseph Doherty eeee3e48a3 fix(GWC-30): reuse the frame reader's length-prefix scratch buffer
ReadAsync allocated a fresh 4-byte array per inbound frame; the GWC-08
pass pooled the payload buffer but left the prefix. Replaced with a
per-instance scratch field — the reader is single-consumer by
construction (one read loop per WorkerClient, handshake reads complete
before the loop starts), so a per-instance buffer is safe and the
non-reentrancy that makes it safe is now stated on the class. Pooling
four bytes via ArrayPool would cost more than the allocation it saves.

Tests: WorkerFrameProtocolTests.ReadAsync_WithMultipleFramesOnOneReader_
ParsesEveryFrame reads five frames of differing payload length through
one reader, so a stale prefix carried between calls would misparse.
2026-08-07 06:15:39 -04:00
Joseph Doherty a044f92c5d fix(GWC-29): drop the wasted request clone on the Invoke hot path
Invoke deep-cloned the whole MxCommandRequest — including its command
payload, potentially a large bulk-write graph — only to overwrite the
cloned command with commandToInvoke and discard it. MapCommand then did
the one clone actually needed. Net cost: a full wasted command deep-clone
per Invoke, worst for exactly the bulk writes that are largest.

Adds a MapCommand(MxCommand) overload (MapCommand reads nothing else off
the request) and has Invoke pass commandToInvoke directly; the request
overload delegates so other callers are untouched.

The remaining clone inside MapCommand stays and is now documented as
required rather than incidental: commandToInvoke may be the gRPC-owned
request.Command, and the caller reads it again after dispatch via
TrackCommandReply, so ownership transfer (à la GWC-07) is not safe here.
That clone is what keeps WorkerClient.CreateCommandEnvelope's no-aliasing
invariant true.

Tests: MxAccessGrpcMapperTests.MapCommandFromCommandClonesPayload
(mutating the input leaves the mapped command untouched; both overloads
produce equal results under a fixed TimeProvider).
2026-08-07 06:15:31 -04:00
Joseph Doherty f27eb28063 fix(GWC-28): stamp gateway envelope sequence at write, not construction
CreateEnvelope stamped Sequence with an interlocked increment when the
envelope was built, so two concurrent InvokeAsync callers could take 1
and 2 and then enqueue in the order 2, 1 — non-monotonic on the wire,
breaking gateway.md's "monotonic per sender" contract. Benign today
(neither side validates inbound sequence, old GWC-10 still open) but it
would fault healthy sessions the moment worker-side validation lands.

WriteLoopAsync now stamps immediately before _writer.WriteAsync. It is
the outbound channel's single consumer (SingleReader = true), so wire
order and stamp order are the same thing by construction and
_nextSequence drops to a plain ulong with no interlocking. This mirrors
the worker's WRK-04 fix, which the gateway half never received.

Also adds TST-28: a [Theory] pinning that GatewayHello.MaxFrameBytes
carries the configured worker-frame maximum (default + 2 MiB override).
The adoption half is asserted only in the Windows-only worker suite, so
a regression to sending 0 — "older gateway, use default" to the worker —
would silently downgrade the negotiated IPC-02 limit with every CI test
still green. Mutation-checked (hard-coded 0 fails both cases).

Tests: WorkerClientTests.ConcurrentInvokesEmitStrictlyIncreasingSequences
OnTheWire (32 parallel invokes; failed 3/3 pre-fix) and
.StartAsync_SendsGatewayHelloWithConfiguredMaxFrameBytes.
2026-08-07 06:15:20 -04:00
Joseph Doherty 3f854d6cbf Merge branch 'fix/sec-31-32-limiter'
ci / windows-x86 (push) Successful in 1m21s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m23s
ci / portable (push) Successful in 9m4s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
2026-08-07 06:13:14 -04:00
Joseph Doherty 5b681ee59b fix(SEC-31,SEC-32): identify a probe-slot reservation by version, not by timestamp
ReleaseProbe recognised its own reservation by comparing NextProbeAtTicks to
now + _probeIntervalTicks. RecordInto's rearm-on-trip writes that identical
expression, so a concurrent RecordFailure on the same WindowState whose `now`
lands on the claimer's tick — routine at ~1 ms clock resolution under load — was
mistaken for the caller's own claim. The release then stomped the legitimate
fresh re-arm back to the stale previousProbeAtTicks, which is already due, handing
the next arrival a free probe the re-arm had just closed.

WindowState gains a monotonic ProbeVersion bumped by every writer of
NextProbeAtTicks (TryConsumeProbe's claim and RecordInto's re-arm alike).
TryConsumeProbe returns the stamp it set as part of a ProbeClaim; ReleaseProbe
restores the previous value only while the state's version still equals that
stamp, checking and restoring in one lock(state) section and bumping the version
again on restore so no other stale release can match either.

Test: ProbeSlotRestore_DoesNotStompConcurrentRearmAtSameTick, with the clock held
still so the claim and the interleaved failure necessarily share a tick. Making it
deterministic needed a seam — the claim-to-release window is a few nanoseconds and
racing threads do not hit it (an earlier thread-based attempt passed against the
defective guard three runs out of three, and its end state was ordering-dependent
rather than correctness-dependent, so it was dropped rather than shipped as
theatre). The seam is an internal ProbeReleaseInterleaveHook, null in production,
costing one null check on the already-refused path. Verified as a genuine red
against the timestamp guard: Expected ThrottledByPeer, Actual ProbeAdmitted.
2026-08-07 06:10:14 -04:00
Joseph Doherty c836899d62 chore: untrack accidentally-committed agent worktree gitlinks; gitignore them
ci / java (push) Successful in 2m25s
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m17s
ci / portable (push) Successful in 8m41s
2026-08-07 06:09:34 -04:00
Joseph Doherty 9825c69d92 Merge branch 'fix/cli-45-credential-envvar'
ci / java (push) Successful in 2m51s
ci / windows-x86 (push) Successful in 1m21s
ci / nightly-windev (push) Has been skipped
ci / portable (push) Successful in 9m41s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
2026-08-07 06:09:03 -04:00
Joseph Doherty 9357ff2dd4 Merge branch 'fix/cli-37-38-conformance'
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m14s
ci / java (push) Successful in 2m14s
ci / portable (push) Successful in 9m13s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
2026-08-07 06:08:12 -04:00
Joseph Doherty 37cb3b0df8 fix(CLI-45): standardize the CLI credential env var and fail fast on empty passwords
All five client CLIs now share one credential contract for `authenticate-user`:
flags `--password` / `--password-env` (Go: `-password` / `-password-env`) with
default env `MXGATEWAY_VERIFY_PASSWORD`, resolution flag-then-env, and a resolved
credential that is missing *or empty* is a usage error naming the flag and the
variable. The value is never echoed and never reaches the wire.

Go and Java previously sent an empty credential when the variable was unset,
turning a misconfigured environment into a real MXAccess authentication attempt.
Go now returns the guard error before dialing; Java throws a picocli
ParameterException instead of falling back to "". Python's `--password-env`
gained the canonical default and its UsageError names the resolved variable.
Rust treats an empty flag or env value as missing, with the resolution extracted
into a testable `resolve_verify_user_password`. .NET adopts the canonical flags
and keeps `--verify-user-password`, `--verify-user-password-env`, and
MXGATEWAY_VERIFY_USER_PASSWORD as deprecated aliases for one release.

Docs same commit: CrossLanguageSmokeMatrix.md gains the credential contract and
the per-CLI subcommand-coverage table (the documented-not-fixed half of the
finding); all five READMEs name the canonical variable and the fail-fast rule,
and the .NET README carries the deprecation note. Tracking flipped to Done in
both remediation registers with a change-log row.

No .proto changed; no generated code regenerated.
2026-08-07 06:05:00 -04:00
Joseph Doherty 6092172694 Merge branch 'fix/gwc-26-27-alarm-attach'
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m17s
ci / java (push) Successful in 2m4s
ci / portable (push) Successful in 7m8s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
#	archreview/2026-07-12/remediation/10-gateway-core.md
2026-08-07 06:02:17 -04:00
Joseph Doherty d6b2f24c3f fix(CLI-37,CLI-38): make status/HRESULT reply validation conformant across all five clients
One cross-client conformance pass; also closes first-cycle CLI-08.

CLI-37: an MxStatusProxy entry is a failure iff `category !=
MX_STATUS_CATEGORY_OK`. The proto contract has always said so — `success` is
the raw 16-bit COM member carried verbatim for diagnostics, not a boolean — but
four clients branched on `success` alone and .NET required both, so the same
gateway reply produced opposite verdicts per language. An absent entry stays
success; a present entry with an UNSPECIFIED category is a failure, because the
worker always maps a category and an unmapped one is not proven OK.

CLI-38: a reply fails on HRESULT iff `hresult` is present and negative, so
positive COM success codes such as S_FALSE (1) pass. .NET/Go/Java used `!= 0`,
which errored on a parity-preserving S_FALSE that Python and Rust accepted.
This makes the existing ClientLibrariesDesign.md claim true rather than
rewriting the doc to describe the divergence.

Four shared fixtures pin both rules cross-client, and each language suite also
carries a table test for the two edges a fixture cannot express (absent entry,
UNSPECIFIED category). A Java test fake that built a status with a bare
`setSuccess(1)` and no category is fixed — under the category rule that reply
was never a success.
2026-08-07 06:00:58 -04:00
Joseph Doherty 09ccd9561f docs(GWC-26): record alarm feed repairs as at-least-once; share the channel worker fake
Code-review follow-up on fix/gwc-26-27-alarm-attach.

ApplyReconcile's snapshot-derived feed repairs are at-least-once, not
exactly-once: a reconcile reads the worker's current state while the matching
live transition may still be buffered in the monitor's lease, so both broadcast
and the duplicates are indistinguishable on the alarm feed. This pre-dates the
acked-state delta — the Raise/Clear presence repair has always had it, since
nothing serializes a reconcile pass against the in-flight live stream — so
closing it (serialization or timestamp dedup) stays out of scope for a P2 fix.
Documented instead, with the consumer contract stated explicitly (apply
transitions idempotently, never as an increment or toggle):

- ApplyReconcile gains a "Delivery semantics" comment.
- gateway.md softens the "defense in depth" prose to state the semantics.
- docs/Sessions.md carries the same caveat on the alarm-feed description.
- Tracker change-log records it as a known pre-existing characteristic and a
  candidate finding for the next review cycle.

Also hoists the ChannelWorkerClient fake — duplicated across the three alarm
test files — into TestSupport/, dropping the usings it took with it.
2026-08-07 06:00:16 -04:00
Joseph Doherty acebe18773 fix(SEC-31,SEC-32): make probe admission atomic and stop Reset clearing a shared fallback partition
Two defects found in code review of the limiter rework.

Probe admission was check-then-act across two lock scopes: Check() read
"probe due" under lock(state), released it, then re-acquired to advance
NextProbeAtTicks. A burst of requests arriving together at an interval boundary
could therefore all observe the slot as due and all be admitted, handing the
verifier the very burst the interval exists to bound. The claim is now a single
critical section (TryConsumeProbe). The two layers are still claimed one at a
time — holding two per-state locks at once would need a global lock ordering to
stay deadlock-free — so a slot claimed on the composite partition is compensated
via ReleaseProbe when the aggregate then refuses, which otherwise silently spent
the partition's next slot and pushed the legitimate holder out by a full
interval.

Reset() removed whatever partition the caller resolved to, including the
address's shared fallback partition when the caller's key id had been collapsed
into it by the per-peer cap (or when the token was junk-shaped). That bucket also
carries failures contributed by other key ids from the same address, so one
successful authentication became a reset button for an in-progress spray. Reset
now clears only a partition the caller owns (effectiveKeyId == presented key id);
the shared bucket decays by window expiry instead, and the caller still recovers
through probe admission. The key's aggregate is cleared either way, as designed.

Also applied from the review: closure-free GetOrAdd overload on _partitions, and
a remarks paragraph acknowledging the best-effort O(n) eviction scan under
sustained overflow. Threading the resolved partition key from Check through to
RecordFailure/Reset was declined: Check resolves with mint:false and RecordFailure
with mint:true, and the two can legitimately differ when a concurrent caller fills
the per-peer cap in between — reusing Check's key would record into the wrong
partition and bypass the cap, which is not worth saving one string concat.

Tests (limiter suite 11 -> 14): ProbeAdmission_UnderConcurrentArrivals_
GrantsExactlyOneSlot (200 rounds x 8 barrier-released threads at the boundary),
ProbeAdmission_WhenAggregateRefuses_ReturnsTheClaimedPeerSlot, and
Reset_WithOverCapKeyId_DoesNotClearSharedFallbackPartition. The latter two were
confirmed as genuine reds against the unfixed code; the concurrency test is a
guard — it is deterministically green on the fixed structure but did not
reproduce the original nanosecond-wide window on its own.
2026-08-07 05:57:26 -04:00
Joseph Doherty 1a75f61ebe test(GWC-26): deflake ApplyReconcileBroadcastsAcknowledgeDelta
Seeding the cache with a live Raise transition raced the first reconcile: when
the reconcile snapshot populated the cache first, the still-buffered live Raise
was applied — and broadcast — after the test's feed subscriber had registered,
so the exactly-one-transition assertion saw two. Seed through a reconcile pass
instead (forced by a provider-mode probe, as the acked step already did) so no
live transition is ever in flight. Verified: 5 consecutive full alarm-monitor
runs green, and the test still fails (timeout) with the ApplyReconcile
acked-delta branch removed.
2026-08-07 05:49:34 -04:00
Joseph Doherty cf66ebbcfb Merge branch 'fix/gwc-25-replaygap-trio'
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m10s
ci / windows-x86 (push) Successful in 1m30s
ci / portable (push) Successful in 7m39s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
#	archreview/2026-07-12/remediation/10-gateway-core.md
2026-08-07 05:49:13 -04:00
Joseph Doherty 44b8e37900 fix(GWC-25,CLI-35,CLI-36): make the empty-ring ReplayGap resumable end to end
An empty replay ring reported oldest_available_sequence = 0 even when gap was
true. Clients follow the documented after_worker_sequence = oldest - 1 formula,
so an unsigned client computed ulong.MaxValue: the follow-up resume replayed
nothing, reported no gap, and the live filter dropped every subsequent event —
a silently dead stream in the headline detach-and-resume scenario, reachable on
default config once ReplayRetentionSeconds (300) age-evicts the ring.

GWC-25: SessionEventDistributor.RegisterWithReplay's empty-ring branch now
reports _highestSequenceSeen + 1 — the next sequence that can possibly be
delivered — when gap is true, so oldest - 1 lands exactly on the highest
observed sequence and the resume delivers everything newer. Still 0 when there
is no gap, where the field is meaningless and never emitted. Nothing is lost:
the evicted interval was unrecoverable either way, and the sentinel's job is to
say "re-snapshot".

CLI-35: the Python CLI fed every stream item into MessageToDict, which raised on
the ReplayGap dataclass and aborted the command after consuming the stream. A
new _event_row helper renders a gap as {"replayGap": {...}} — the same camelCase
shape the Rust CLI emits — and leaves proto events on the existing path.

CLI-36: the Go CLI formatted result.Event on every row, but the library
deliberately clears Event on a gap, so text mode printed
"0 MX_EVENT_FAMILY_UNSPECIFIED" and JSON mode an empty object, discarding the
resume cursors. The loop now branches on result.IsReplayGap() and renders the
typed row in both modes, counting it toward -limit like any other row. The JSON
row's cursors are typed by hand rather than marshalled with protojson: the
proto3 JSON mapping renders 64-bit integers as strings ("7") while the Rust and
Python CLIs emit numbers (7), so going through protojson would have made Go the
only canonical CLI with a different value type.

Docs in the same change: docs/Sessions.md documents the empty-ring sentinel
value and that oldest - 1 is the universal resume formula in both the retained
and fully-evicted cases; docs/CrossLanguageSmokeMatrix.md gains a per-CLI
gap-rendering table covering both client findings, and records exactly what is
and is not comparable across CLIs (same keys and numeric cursors for Rust/Go/
Python; quoted cursors for .NET/Java; differing key order, whitespace, and
container), so a matrix runner compares parsed values rather than raw bytes.

Tests, all written red first and each reproducing its defect verbatim:
- SessionEventDistributorTests: RegisterWithReplayReportsNextDeliverableSequence
  WhenRingEmptiedByAge, ...WithRetentionDisabled, and
  ResumeUsingSentinelFormulaAfterEmptyRingGapDeliversLiveEvents.
- GatewayEndToEndReconnectReplayTests.ReconnectAfterFullAgeEvictionResumesWith
  SentinelFormula — fake-worker e2e resume walk on a fake clock; the fixture now
  takes a retention window and a TimeProvider.
- clients/python test_stream_events_renders_replay_gap.
- clients/go TestRunStreamEventsPrintsReplayGap.

GWC-25's ReplayGap.oldest_available_sequence proto-comment amendment is
deliberately deferred to the later codegen wave (see the tracker change log): it
is comment-only but triggers the full five-client regen fan-out.
2026-08-07 05:46:00 -04:00
Joseph Doherty 3b6a239ed6 fix(GWC-26): attach the alarm monitor's lease before SubscribeAlarms
RunMonitorAsync issued SubscribeAlarms and the first reconcile before the
internal distributor subscriber was attached (via ISessionManager
.ReadAlarmEventsAsync). The pump has been running since MarkReady started the
dashboard mirror and only fans to subscribers registered at fan-out time, so
every transition raised in that two-round-trip window bypassed the alarm feed —
and a missed Acknowledge was never repaired, because ApplyReconcile broadcast
presence deltas only.

- The monitor now takes the internal lease directly from its session BEFORE
  SubscribeAlarms and drains it after the first reconcile; window transitions
  buffer in the lease's bounded channel. Processing them after ApplyReconcile is
  order-safe (ApplyTransition handles alarms the snapshot already placed).
- ISessionManager.ReadAlarmEventsAsync removed — zero remaining callers.
- ApplyReconcile broadcasts an Acknowledge feed transition when a both-present
  alarm's state advanced to ActiveAcked. This is a feed-level repair on the
  AlarmFeedMessage/StreamAlarms surface rebuilt from the worker's own snapshot,
  not MxEvent emission, so the "never synthesize events" rule is untouched;
  the reasoning is recorded on ApplyReconcile.

The alarm-monitor test fakes now hand the monitor a real Ready GatewaySession
with a dashboard mirror, which is what makes the window reproducible.

Docs: docs/Sessions.md and gateway.md alarm-monitor ordering notes.

Refs: archreview/2026-07-12/remediation/10-gateway-core.md GWC-26
2026-08-07 05:40:33 -04:00
Joseph Doherty df710e18a9 fix(SEC-31,SEC-32): re-partition the API-key failure limiter on (peer, key id) with probe admission
The gRPC auth failure limiter partitioned on the key id parsed out of the
*unauthenticated* token and rejected with ResourceExhausted before VerifyAsync
ran. Key ids are not secret — they ride in every token and are listed on the
dashboard — so any network peer could send 10 garbage-secret requests per minute
and deny that key indefinitely: the legitimate holder's correct secret was
refused before it was ever checked, and the success-path Reset that would clear
the block sat behind the verification the block prevented (SEC-31). The tracked
map was also flushable — any `a_b_c`-shaped junk minted a fresh partition (the
`mxgw` literal was never compared), so ~4096 throwaway tokens evicted a blocked
entry and reset the window (SEC-32).

ApiKeyFailureLimiter moves from IsBlocked/RecordFailure/Reset(string peer) to a
partition-pair API: Check/RecordFailure/Reset(ApiKeyThrottlePartition) with an
ApiKeyThrottleDecision result. Two layers share one sliding window — a composite
(transport peer, key id) partition at ApiKeyFailureLimit, and a per-key-id
aggregate across all peers at the new ApiKeyFailureAggregateLimit (default 30)
that bounds a source-rotating sprayer. An over-limit state is now a valve rather
than a wall: one request per the new ApiKeyFailureProbeIntervalSeconds (default
5) is admitted through to the real verifier, so the correct secret always reaches
the constant-time compare and resets both layers. Guarantees preserved: guessing
stays bounded per window, and the failure path still spends no store read per
attempt.

SEC-32 rides the same change set: the interceptor validates token shape (literal
`mxgw` prefix, >= 3 non-empty `_` segments, key id <= 64 chars) before minting a
key-id partition, each transport peer may mint at most 32 of them before the
overflow collapses onto its fallback partition, and eviction prefers fully
expired windows and never drops an over-limit partition below a 2x transient
overshoot ceiling. Throttled attempts increment mxgateway.auth.throttled, tagged
stage=peer|aggregate only — /metrics is unauthenticated (open SEC-14), so no key
material may appear there.

Docs in the same commit: GatewayConfiguration limiter rows plus the two new keys,
the Authentication hot-path paragraph, the Authorization SEC-11 section, and the
limiter / SecurityOptions XML remarks (the old NAT rationale described the
defective keying). Tracking rows flipped to Done with a change-log entry.

Tests: new ApiKeyFailureLimiterTests (11) covering window pruning, composite vs
aggregate trip points, probe cadence, absolute-block mode, reset across both
layers, junk-spray eviction resistance, the per-peer cap, and expired-window
eviction preference; GatewayGrpcAuthorizationInterceptorTests gains the four
SEC-31 contract tests plus NonMxgwToken_FallsBackToTransportPeerPartition (20
total); GatewayOptionsValidatorTests covers both new keys including 0 as a
supported disable value (66 total).
2026-08-07 05:39:10 -04:00
Joseph Doherty 33ba612ddd fix(WRK-21,WRK-28,WRK-23,IPC-30): byte-budget the DrainEvents reply, stop size errors from killing sessions
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m8s
ci / portable (push) Successful in 7m41s
ci / windows-x86 (push) Failing after 12m32s
WRK-21 — DrainEvents was bounded by event count only, so a byte-heavy queue
(large string/array MxValues) built a reply above the negotiated frame maximum:
the writer rejected the frame, the exception unwound the session, and the events
already dequeued were destroyed. The drain is now byte-budgeted inside the queue
lock, so an event is dequeued only once it is known to fit and one that does not
stays at the head. Truncation is reported through the reply's existing
DiagnosticMessage (no contract change); callers drain until an empty reply. Both
reply-write seams — the control-command path and ProcessCommandAsync — now catch
MessageTooLarge and answer the correlation with an InvalidRequest reply instead
of unwinding or faulting the session. Satisfies IPC-23 R1-R3.

WRK-28 — the 10,000 drain ceiling moves to GatewayContractInfo
.MaxDrainEventsPerCommand, referenced by both the gateway request validator and
the worker clamp, replacing a comment-only sync contract. C# const only; no
.proto change.

WRK-23 — WorkerFrameWriter now peek-stamps, validates, then commits the sequence
counter immediately before the stream write, so a per-frame rejection leaves no
phantom gap on the wire.

IPC-30 — an oversized event frame stays session-fatal (it is undeliverable end to
end and neither dropping nor synthesizing a replacement is allowed), but the
death is structured: the event's identity and sizes are logged (never its value),
a WorkerFault with category PROTOCOL_VIOLATION and command method EventDrain is
written, then the session exits as before.

Docs updated in the same change: MxAccessWorkerInstanceDesign.md (drain byte cap,
truncation contract, oversized-head behavior, oversized-event policy, no control
reply is session-fatal on size), WorkerFrameProtocol.md (reply pre-sizing,
non-fatal reply-size rule, oversized-event policy, rejected frames do not consume
sequence numbers), gateway.md (DrainEvents two-axis bound).
2026-08-07 05:38:23 -04:00
Joseph Doherty 1a63fdd7db fix(GWC-27): gate AttachInternalEventSubscriber on session readiness
AttachInternalEventSubscriber ran EnsureDistributorCreated / Register /
StartPumpIfRequested with no state check, unlike AttachEventSubscriber. A
premature attach would start the pump against a not-yet-Ready worker; the pump
source throws SessionNotReady, PumpAsync completes every subscriber with that
error and latches the distributor, and _eventDistributorStarted is never reset —
so the session would reach Ready with permanently dead event streaming.

Mirror AttachEventSubscriber's gate: check _state/_workerClient.State under
_syncRoot and throw SessionManagerException(SessionNotReady) before the
distributor is created, keeping the distributor calls outside the lock.

Refs: archreview/2026-07-12/remediation/10-gateway-core.md GWC-27
2026-08-07 05:28:35 -04:00
427 changed files with 50798 additions and 4462 deletions
+60 -18
View File
@@ -4,16 +4,16 @@
# .NET Framework 4.8 / x86 and need MXAccess COM installed, so they build ONLY on a Windows host.
# They are out of scope for this Linux `portable` job — the `windows-x86` job below builds and
# tests them on windev (10.100.0.48) over SSH (see scripts/ci/run-windev-ci.sh), and the nightly
# `nightly-windev` job additionally runs the live-MXAccess smoke, which is scheduled-only so it
# never gates a push.
# `nightly-windev` job additionally runs the full solution build, gateway suite, and live-MXAccess
# smoke, which are scheduled-only so they never gate a push.
name: ci
on:
push:
pull_request:
schedule:
# 06:00 UTC nightly: the `nightly-windev` job re-runs the x86 Worker build + full Worker.Tests on
# windev, then the live-MXAccess smoke (MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1) and a full-slnx build.
# 06:00 UTC nightly: x86 Worker build + Worker.Tests, full-slnx build, gateway tests, then the
# live-MXAccess smoke (MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1), serialized on windev's host lock.
# Push/PR runs skip it — the job is gated `if: github.event_name == 'schedule'`.
- cron: '0 6 * * *'
@@ -23,7 +23,12 @@ jobs:
# tests, the codegen/descriptor freshness guards, and the clients that build on Linux.
runs-on: ubuntu-latest
steps:
# fetch-depth: 0 (full history + tags): the CLI-46 client-version-freshness check below
# diffs each client's source against its last publish tag (e.g. clients/go/vX.Y.Z), which
# a shallow, tagless checkout cannot resolve.
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/setup-dotnet@v4
with:
@@ -60,17 +65,47 @@ jobs:
dotnet tool install --global PowerShell
echo "$HOME/.dotnet/tools" >> "$GITHUB_PATH"
# IPC-01 / IPC-19 / IPC-20: descriptor set + Contracts/Generated must match the current protos.
# IPC-25 Check 4 regenerates the Go and Python client bindings and diffs them, so the pinned
# generators must be present. protoc 34.1 is already installed above; Go and Python are set up
# above. Pin protoc-gen-go / protoc-gen-go-grpc to match the committed header stamps and grpcio
# -tools to match the committed _pb2 stamp, or Check 4 false-fails (or masks drift) under churn.
- name: Install pinned client codegen generators (Check 4)
run: |
go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.6.2
echo "$(go env GOPATH)/bin" >> "$GITHUB_PATH"
python -m pip install 'grpcio-tools==1.80.0'
# IPC-01 / IPC-19 / IPC-20 / IPC-25: descriptor set + Contracts/Generated + Go/Python bindings
# must match the current protos.
- name: Codegen / descriptor freshness
shell: pwsh
run: ./scripts/check-codegen.ps1
# CLI-46: catches a client whose source changed since its last publish
# tag while the version constant still reads that tag's already-shipped
# number — the drift the pack-clients.ps1/tag-go-module.ps1 collision
# guards only catch at actual publish time. Needs tag history, hence
# `fetch-depth: 0` on this job's checkout above.
- name: Client version freshness
shell: pwsh
run: ./scripts/check-client-versions.ps1
- name: Code-reviews README consistency
run: python code-reviews/regen-readme.py --check
- name: Gateway fake-worker tests
run: dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj -c Release --no-build
- name: .NET client
run: dotnet build clients/dotnet/ZB.MOM.WW.MxGateway.Client.slnx -c Release
# The .NET client was previously only compiled here, so its test project — including
# MxGatewayClientWireTests, which drives the client against a real loopback gRPC
# server (TST-24) — never ran in CI. Every other client job already runs its tests.
- name: .NET client tests
run: dotnet test clients/dotnet/ZB.MOM.WW.MxGateway.Client.Tests/ZB.MOM.WW.MxGateway.Client.Tests.csproj -c Release --no-build
- name: Go client
working-directory: clients/go
run: |
@@ -93,10 +128,12 @@ jobs:
python -m pytest
java:
# Java client runs on a JDK-17 Linux runner (the macOS dev box has no JRE). The protobuf gradle
# plugin rewrites MxaccessGateway.java with spurious protobuf-runtime-version churn on every
# build; when no .proto changed, revert that one file so checkGeneratedClean / a dirty tree does
# not fail the build (repo memory project_java_generated_churn).
# Java client runs on a JDK-17 Linux runner (the macOS dev box has no JRE). The grpc/protobuf
# toolchain is fully pinned (clients/java/build.gradle: grpcVersion 1.76.0 / protobufVersion
# 4.33.1), so a regeneration is byte-identical to the committed aggregates modulo real .proto
# changes — `Verify generated tree is clean` (git diff) is the true drift gate (IPC-24). The
# single-file Java aggregates are where message-level proto drift lands, so this job now catches
# a .proto edited without regenerating and committing the Java client.
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
@@ -115,13 +152,10 @@ jobs:
- name: Gradle test
working-directory: clients/java
run: gradle test
- name: Revert spurious protobuf-version churn (no .proto changed)
# Both generated aggregates can pick up protobuf-runtime-version churn on regen; revert
# both so verify-clean still catches a real, uncommitted proto/codegen change elsewhere.
run: |
git checkout -- clients/java/src/main/generated/main/java/mxaccess_gateway/v1/MxaccessGateway.java || true
git checkout -- clients/java/src/main/generated/main/java/mxaccess_worker/v1/MxaccessWorker.java || true
- name: Verify generated tree is clean
# IPC-24: the pinned grpc/protobuf toolchain regenerates byte-identical output, so this
# git-diff gate now catches message-level proto drift in the single-file Java aggregates
# (the old unconditional churn-revert step masked exactly that class and was deleted).
run: git diff --exit-code -- clients/java/src/main/generated
windows-x86:
@@ -148,15 +182,23 @@ jobs:
run: ./scripts/ci/run-windev-ci.sh test
nightly-windev:
# TST-25: scheduled deep run — full Worker.Tests + live-MXAccess smoke + full-slnx build on windev.
# TST-35: scheduled deep run — Worker, full-slnx, gateway, and live-smoke tiers on windev.
# Gated to the schedule event via `if:` (not an unsatisfiable `runs-on`), so push/PR runs mark it
# skipped, not queued. Never gates a push. On failure it opens a Gitea issue so a red nightly is
# visible even though nobody watches the Actions page.
if: github.event_name == 'schedule'
runs-on: ubuntu-latest
env:
# `github.server_url` is the URL Gitea hands the runner — the docker-network-internal
# `http://gitea:3000`. That is the right base for the issue-creation API call below (the job
# container resolves `gitea` only on that network, and has no LAN egress to the public origin),
# but it is useless as a link a human clicks out of the issue. So browser-facing URLs in the
# issue body use the public origin instead. TST-25 acceptance Check 6 caught this: every
# nightly issue since #126 carried an unreachable `http://gitea:3000/...` run link.
PUBLIC_SERVER_URL: https://gitea.dohertylan.com
steps:
- uses: actions/checkout@v4
- name: Full Worker.Tests + live-MXAccess smoke on windev
- name: Worker + gateway tests + live-MXAccess smoke on windev
env:
# WINDEV_SSH_KEY is stored base64-encoded (single line) so Gitea's line-oriented secret
# masker redacts it in the step env echo; run-windev-ci.sh decodes it. A raw multiline PEM
@@ -173,7 +215,7 @@ jobs:
-H "Authorization: token ${{ github.token }}" \
-H "Content-Type: application/json" \
"${{ github.server_url }}/api/v1/repos/${{ github.repository }}/issues" \
-d "{\"title\":\"nightly-windev failed on ${{ github.sha }}\",\"body\":\"The scheduled windev Worker + live-MXAccess run failed: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} . A red nightly may mean the Windows tier is down rather than the change — see docs/GatewayTesting.md (Continuous Integration).\"}"
-d "{\"title\":\"nightly-windev failed on ${{ github.sha }}\",\"body\":\"The scheduled windev Worker + live-MXAccess run failed: ${PUBLIC_SERVER_URL}/${{ github.repository }}/actions/runs/${{ github.run_id }} . A red nightly may mean the Windows tier is down rather than the change — see docs/GatewayTesting.md (Continuous Integration).\"}"
# NOTE: there is intentionally no native `windows` runner job. act_runner v0.6.1 host-mode on
# Windows is broken and Windows containers are impractical for the net48/x86/MXAccess Worker, so the
+4
View File
@@ -152,3 +152,7 @@ generated-scratch/
*-docs-issues.md
*-docs-fixed.md
*-docs-final.md
# Agent worktrees (subagent isolation) — never commit
.claude/worktrees/
.worktrees/
+14 -7
View File
@@ -156,8 +156,11 @@ accept a `browseSubtreeGlobs` param, so either fix is small plumbing:
Delete mxaccessgw's own `Galaxy/GalaxyRepositoryServiceCollectionExtensions.cs` registrations.
4. **Option validation** — the shared lib **binds only, ships no validator** (deliberate). mxaccessgw
already validates Galaxy options via `Configuration/GatewayOptionsValidator.cs` — **keep that**; it
stays the owner of fail-fast validation, exactly as HistorianGateway's `ConfigPreflight` does.
stays the owner of fail-fast Galaxy validation. **Updated 2026-08-07 (SEC-33):** this is now a dedicated
`Configuration/GalaxyRepositoryOptionsValidator.cs` (registered as `IValidateOptions<GalaxyRepositoryOptions>`
with `ValidateOnStart`), which enforces a valid, host-rooted `SnapshotCachePath` when `PersistSnapshot`
is true — exactly as HistorianGateway's `ConfigPreflight` does. (The original handoff pointed at
`GatewayOptionsValidator.cs`, but that validator does not see the lib-bound `GalaxyRepositoryOptions`.)
5. **Health check** — keep mxaccessgw's existing Galaxy-SQL readiness check; read the connection
string from the same `MxGateway:Galaxy` section the lib binds (HistorianGateway does this with a raw
@@ -189,11 +192,15 @@ be **deleted**. **Keep** the mxaccessgw-specific ones that exercise behavior the
## Post-adoption notes / caveats
- **Deployment config (NSSM):** the deployed services (`MxAccessGw` on 10.100.0.48; the wonder host) read
config from **NSSM environment variables, not `appsettings.json`**. The lib's `SnapshotCachePath` default
is empty (persistence no-ops). `appsettings.json` sets `MxGateway:Galaxy:SnapshotCachePath` +
`PersistSnapshot`, but the deployments must carry `MxGateway__Galaxy__SnapshotCachePath` and
`MxGateway__Galaxy__PersistSnapshot` in their NSSM env on redeploy, or snapshot persistence silently
no-ops in production.
config from **NSSM environment variables, not `appsettings.json`**. **Updated 2026-08-07 (SEC-33):** the
lib's own `SnapshotCachePath` default is empty (would no-op persistence), but mxaccessgw no longer relies
on it. `appsettings.json` no longer sets `SnapshotCachePath` at all; instead the gateway seeds a
`CommonApplicationData`-derived default (`C:\ProgramData\MxGateway\galaxy-snapshot.json` on the Windows
hosts) when the bound value is blank, and `GalaxyRepositoryOptionsValidator` fails startup if
`PersistSnapshot` is true with a non-rooted/invalid path. So `MxGateway__Galaxy__SnapshotCachePath` in the
NSSM env is now **optional** (an override), not required — a deployment that omits it gets the rooted host
default and persistence works; it no longer silently no-ops. `MxGateway__Galaxy__PersistSnapshot` still
governs whether persistence runs at all.
- **Pre-existing NU1903 (unrelated) — ✅ RESOLVED (2026-07-18, commit `2f0cfe3`):** adding the package surfaced a transitive `SQLitePCLRaw.lib.e_sqlite3`
2.1.11 advisory (GHSA-2m69-gcr7-jv3q, at the time no upstream patch) that breaks the build under `TreatWarningsAsErrors`
— already red on `main`. Initially resolved with a targeted `NuGetAuditSuppress` in `src/Directory.Build.props`
+6 -6
View File
@@ -10,7 +10,7 @@ The architecture is a two-process design — read `gateway.md` before making str
- **Gateway** (`src/ZB.MOM.WW.MxGateway.Server`, .NET 10, x64): ASP.NET Core gRPC server. Owns the public API, sessions, auth, the Blazor dashboard, and the Galaxy Repository SQL browse RPCs. The Galaxy-browse implementation comes from the shared **`ZB.MOM.WW.GalaxyRepository`** package (`AddZbGalaxyRepository`/`MapZbGalaxyRepository`), not inline code; mxaccessgw adds `GatewayBrowseScopeProvider` (per-key browse-subtree scoping) and a host-side dashboard summary projector. See `A2-galaxyrepository-adoption-handoff.md`. **Never instantiates MXAccess COM directly.**
- **Worker** (`src/ZB.MOM.WW.MxGateway.Worker`, .NET Framework 4.8, **x86**): one process per session. Owns one MXAccess COM instance on a dedicated STA, pumps Windows messages, and converts COM events to protobuf.
- **IPC**: gateway↔worker uses one bidirectional named pipe per worker (`mxaccess-gateway-{gatewayPid}-{sessionId}`) with length-prefixed `WorkerEnvelope` protobuf frames. Gateway hosts the pipe server and launches the worker. **gRPC is not used inside the worker** — .NET Framework 4.8 doesn't have a first-class gRPC stack.
- **IPC**: gateway↔worker uses one bidirectional named pipe per worker (`mxgw-{gatewayPid}-{sessionUid}` — kept short so the macOS/Linux test matrix's Unix-domain-socket path fits the 104-byte macOS `sun_path` limit) with length-prefixed `WorkerEnvelope` protobuf frames. Gateway hosts the pipe server and launches the worker. **gRPC is not used inside the worker** — .NET Framework 4.8 doesn't have a first-class gRPC stack.
- **Contracts** (`src/ZB.MOM.WW.MxGateway.Contracts`): multi-targets `net10.0;net48` and owns the `.proto` files (`mxaccess_gateway.proto`, `mxaccess_worker.proto`, `galaxy_repository.proto`). All other projects consume the generated types from here. Do not hand-edit anything under `Generated/`. Note `galaxy_repository.proto` is intentionally kept here as the generation source for the language clients even though the gateway server consumes the wire-identical Galaxy types from the `ZB.MOM.WW.GalaxyRepository` package — it is not dead code; deleting it breaks all five clients.
The worker must do all MXAccess COM calls on its dedicated STA thread, and the STA loop must pump Windows messages (`MsgWaitForMultipleObjectsEx` + `PeekMessage`/`DispatchMessage`) so MXAccess events deliver. A plain blocking queue on an STA is not enough.
@@ -71,7 +71,7 @@ dotnet test src/ZB.MOM.WW.MxGateway.Worker.Tests/ZB.MOM.WW.MxGateway.Worker.Test
dotnet run --project src/ZB.MOM.WW.MxGateway.Server/ZB.MOM.WW.MxGateway.Server.csproj
# API-key admin CLI (same exe, "apikey" subcommand)
dotnet run --project src/ZB.MOM.WW.MxGateway.Server/ZB.MOM.WW.MxGateway.Server.csproj -- apikey create --display-name "dev" --scopes session,invoke,event,metadata,admin
dotnet run --project src/ZB.MOM.WW.MxGateway.Server/ZB.MOM.WW.MxGateway.Server.csproj -- apikey create-key --key-id dev --display-name "dev" --scopes session:open,session:close,invoke:read,invoke:write,invoke:secure,events:read,metadata:read,admin
```
Single test by name (xUnit `--filter`):
@@ -124,7 +124,7 @@ powershell -ExecutionPolicy Bypass -File scripts/run-client-e2e-tests.ps1
When source code changes, build and test the affected component before reporting work done. If the change crosses component boundaries, build each affected component — don't rely on a single top-level build:
**Run targeted tests per task, never the full suite each time.** When executing a plan task-by-task, run only the tests that exercise the code that task touched (`dotnet test --filter "FullyQualifiedName~<TestClass>"`, or the per-task test named in the plan). The full gateway suite is slow — run it at most once per phase (after a related batch of tasks lands), not after every task. This is a speed guideline, not a correctness one: the suite exits cleanly (verified on macOS and the Windows dev box — 0 surviving `testhost`/worker processes after a full run), so filtered runs are about turnaround, not about avoiding a process leak.
**Run targeted tests per task, never the full suite each time.** When executing a plan task-by-task, run only the tests that exercise the code that task touched (`dotnet test --filter "FullyQualifiedName~<TestClass>"`, or the per-task test named in the plan). The full gateway suite is slow — run it at most once per phase (after a related batch of tasks lands), not after every task. This is a speed guideline, not a correctness one: the suite exits cleanly on both macOS and windev (0 surviving `testhost`/worker processes after a full run), so filtered runs are about turnaround. The long-standing windev full-suite wedge — every test reported, then `testhost` never exiting — was a zero-buffer named pipe blocking one test's write forever; it is fixed and the `--blame-hang` workaround is no longer needed. See "Running the Gateway Suite on windev" in `docs/GatewayTesting.md`, which also documents the load-sensitivity caveat that still applies there.
| Changed area | Required verification |
|---|---|
@@ -155,11 +155,11 @@ External analysis sources referenced by design docs:
## Authentication
Gateway gRPC clients authenticate with an API key in metadata: `authorization: Bearer mxgw_<key-id>_<secret>`. Keys are stored hashed (with a peppered SHA) in a gateway-owned SQLite DB (default `C:\ProgramData\MxGateway\gateway-auth.db`). Scopes (`session`, `invoke`, `event`, `metadata`, `admin`) gate specific RPCs; missing → `Unauthenticated`, insufficient → `PermissionDenied`. The `apikey` subcommand on the server exe manages keys; see `src/ZB.MOM.WW.MxGateway.Server/Security/Authentication/`.
Gateway gRPC clients authenticate with an API key in metadata: `authorization: Bearer mxgw_<key-id>_<secret>`. Keys are stored hashed (with a peppered SHA) in a gateway-owned SQLite DB (default `C:\ProgramData\MxGateway\gateway-auth.db`). Scopes (`session:open`, `session:close`, `invoke:read`, `invoke:write`, `invoke:secure`, `events:read`, `metadata:read`, `admin`) gate specific RPCs; missing → `Unauthenticated`, insufficient → `PermissionDenied`. The `apikey` subcommand on the server exe manages keys; see `src/ZB.MOM.WW.MxGateway.Server/Security/Authentication/`.
Session event streaming is **owner-scoped**: the API key that opened a session is recorded on the session, and every `StreamEvents` attach/reattach is rejected with `PermissionDenied` unless the caller's key id matches the owner. Possessing the `event` scope and knowing a session id is not sufficient — this closes the reconnect/fan-out trust boundary (detach-grace and replay retention are on by default) so an `event`-scoped key cannot attach to another key's retained session.
Session event streaming is **owner-scoped**: the API key that opened a session is recorded on the session, and every `StreamEvents` attach/reattach is rejected with `PermissionDenied` unless the caller's key id matches the owner. Possessing the `events:read` scope and knowing a session id is not sufficient — this closes the reconnect/fan-out trust boundary (detach-grace and replay retention are on by default) so an `events:read`-scoped key cannot attach to another key's retained session.
Dashboard auth is LDAP-backed (separate from the gRPC API-key model). `/login` binds against `MxGateway:Ldap` and maps the user's LDAP groups to `Admin` or `Viewer` via `MxGateway:Dashboard:GroupToRole`, then issues an HTTP-only secure cookie named `__Host-MxGatewayDashboard` when `Dashboard:RequireHttpsCookie` is true (default) and no `Dashboard:CookieName` override is set, else the plain `MxGatewayDashboard` (the `__Host-` prefix requires a Secure cookie). SignalR hubs at `/hubs/{snapshot,alarms,events}` accept either the cookie or a 5-minute bearer minted at `/hubs/token`. `Dashboard:AllowAnonymousLocalhost` (default `true`) grants **read-only** loopback access — it satisfies the Viewer requirement but never the Admin-only requirement, so anonymous localhost can view the dashboard but not reach API-key CRUD or session Close/Kill (`Authentication:Mode=Disabled` is scoped the same way). `Dashboard:DisableLogin` (default `false`) auto-authenticates every dashboard request — including remote browsers — as `Dashboard:AutoLoginUser` (default `multi-role`) with both Admin and Viewer roles; dev/test only, never enable in production.
Dashboard auth is LDAP-backed (separate from the gRPC API-key model). `/login` binds against `MxGateway:Ldap` and maps the user's LDAP groups to `Admin` or `Viewer` via `MxGateway:Dashboard:GroupToRole`, then issues an HTTP-only secure cookie named `__Host-MxGatewayDashboard` when `Dashboard:RequireHttpsCookie` is true (default) and no `Dashboard:CookieName` override is set, else the plain `MxGatewayDashboard` (the `__Host-` prefix requires a Secure cookie). SignalR hubs at `/hubs/{snapshot,alarms,events}` accept either the cookie or a 5-minute bearer minted at `/hubs/token`; `/hubs/alarms` and the `/alarms` page additionally require a **real authenticated dashboard role** (`MxGateway.Dashboard.Alarms`, a requirement type the environment-bypass handler does not handle), because the alarm feed is gateway-central with no per-session provenance and so cannot be tag-scoped — its metadata stays galaxy-wide for authenticated Viewers, but anonymous localhost and `Authentication:Mode=Disabled` do not receive it at all. Dashboard event visibility is **tag-scoped per session** (`IDashboardSessionAcl`, gating both the events hub and the session-details page's in-process subscribe): an Administrator sees every session, while any other caller sees a session only when its tags — inherited from the owning API key's `apikey --dashboard-tags`, never from the client's request — intersect the tags their LDAP groups grant via `Dashboard:GroupToTag`. That grant is **re-resolved from the caller's LDAP group claims against current options on every subscribe** (not read from the tag claims stamped at login, which would outlive a config-revoked grant for the cookie's 8-hour sliding life); login-stamped tag claims are the fallback only for a principal carrying no group claims — i.e. the hub-token principal, whose grant the mint already re-resolved. Untagged sessions follow `Dashboard:UntaggedSessionVisibility` (default `AdminOnly`), and a principal with neither claim kind (anonymous localhost included) is an empty-grant Viewer. `Dashboard:AllowAnonymousLocalhost` (default `true`) grants **read-only** loopback access — it satisfies the Viewer requirement but never the Admin-only requirement, so anonymous localhost can view the dashboard but not reach API-key CRUD, session Close/Kill, or the alarm feed (`Authentication:Mode=Disabled` is scoped the same way). `Dashboard:DisableLogin` (default `false`) auto-authenticates every dashboard request — including remote browsers — as `Dashboard:AutoLoginUser` (default `multi-role`) with both Admin and Viewer roles; dev/test only, never enable in production.
## Process / Platform Notes
+9 -2
View File
@@ -79,6 +79,12 @@ Add one entry per finding to the `## Findings` section of the module's
- **Finding ID** — `<Module>-NNN`, numbered sequentially within the module and
never reused (e.g. `Worker-001`). IDs are permanent even after resolution.
The `GWC`/`WRK`/`IPC`/`SEC`/`CLI`/`TST` prefixes are reserved for the
`archreview/` domain registers (a separate, whole-repo review series with its
own tracking documents) — never assign one of these prefixes to a
`code-reviews/` finding, and any other work stream (perf passes, ad hoc
remediation plans, etc.) that wants a stable per-item ID must mint its own
distinct prefix rather than borrowing one of these six.
- **Severity:**
- **Critical** — data loss, security breach, crash/deadlock, or outage.
- **High** — incorrect behaviour with significant impact; no safe workaround.
@@ -127,8 +133,9 @@ finding carries an unrecognised Status value. The PowerShell wrapper
`scripts/check-code-reviews-readme.ps1` runs that check and is the intended hook
for CI or a pre-commit step.
> The repo's installed `python` is the real interpreter; the bare `python3`
> alias resolves to the Windows Store stub and fails. Use `python`.
> On Windows, the installed interpreter may be available only as `python`
> because `python3` can resolve to the Windows Store stub. macOS and Linux
> normally use `python3`; CI uses the configured `python` executable.
The per-module `findings.md` files are the source of truth; `README.md` is the
aggregated index and must always agree with them — which the script guarantees.
@@ -25,7 +25,7 @@
| TST-10 | Not started | **Confirmed open** | No `docs/Deployment.md` (`ls docs/` — no deploy doc); no deploy script under `scripts/` | Deploy knowledge still lives only in operator memory notes. |
| TST-14 | Not started | **Confirmed open** | Repo root still holds `MxAccessGateway-docs-{issues,fixed,final}.md`, `MxGatewayClient-docs-{issues,fixed}.md`, `oldtasks.md`, `stillpending.md` (root `ls`) | `oldtasks.md` did gain a legitimate new role as the epic-governance record (TST-04), so "delete when epic closes" needs re-deciding; the five untracked `*-docs-*.md` files remain pure clutter. |
| TST-15 | Not started (design done) | **Confirmed open** | `Server/Dashboard/Hubs/EventsHub.cs:42` — `TODO(per-session-acl)` still present | Design now exists: `docs/plans/2026-07-10-dashboard-session-acl-tst15.md`. Mitigation shipped meanwhile: SEC-25 value redaction in the hub mirror (see TST-27). |
| TST-16 | Not started | **Partially resolved — see TST-27** | `Server/Dashboard/Hubs/DashboardEventBroadcaster.cs:16,29` — `ShowTagValues` now gates value redaction of the SignalR mirror (SEC-25) | The flag is no longer fully dead, but it still does not gate `/browse` live-value display, and the config doc still calls it "Reserved" — fresh drift (TST-27). |
| TST-16 | Not started | **Partially resolved — see TST-27** | `Server/Dashboard/Hubs/DashboardEventBroadcaster.cs:16,29` — `ShowTagValues` now gates value redaction of the SignalR mirror (SEC-25) | The flag is no longer fully dead, but it still does not gate `/browse` live-value display, and the config doc still calls it "Reserved" — fresh drift (TST-27). **Closed 2026-08-17** (branch `feat/followup-closeout`): both residuals shipped — `AlarmsHubPublisher` redacts `current_value`/`limit_value` from a deep clone on both value-bearing `AlarmFeedMessage` arms before the `/hubs/alarms` broadcast, and `DashboardLiveDataService` substitutes `[redacted]` for `/browse` value text at the service boundary (the recommended seam), so the flag now gates every dashboard value surface. Source never mutated on either hub, so gRPC subscribers are unaffected. Tests: `AlarmsHubPublisherTests`, two `DashboardLiveDataServiceTests` cases. TST-16 → `Done` in [../remediation/00-tracking.md](../remediation/00-tracking.md#change-log). |
| TST-17 | Not started | **Confirmed open** | `Worker/MxAccess/WnWrapAlarmConsumer.cs:~264-275` — `_ = ackOperatorDomain; _ = ackOperatorFullName;` then 6-arg call; no diagnostic surfaced in the ack reply | Silent drop unchanged (comment block explains the −55 stub, but callers still can't see the degrade). |
| TST-18 | Not started | **Confirmed open** | No `*HostedService*` test files under `Tests/Gateway/Sessions/` or `Tests/Gateway/Workers/` | Sweep cores remain covered indirectly (`SessionManagerTests.cs:985,1015` now also cover faulted-reap). |
| TST-19 | Not started | **Confirmed open** | `Tests/Gateway/Workers/Fakes/FakeWorkerHarness.cs` — no lockstep note / `WorkerPipeSession` pointer (grep empty); no README in `Fakes/` | Risk is higher now that the real-worker suite has no scheduled run (TST-25). |
@@ -53,7 +53,7 @@
### TST-27 — `ShowTagValues` config doc still says "Reserved" after SEC-25 made the flag live — **Medium** · documentation currency
- **Files:** `docs/GatewayConfiguration.md:185` ("Reserved display control for tag values"); `src/ZB.MOM.WW.MxGateway.Server/Dashboard/Hubs/DashboardEventBroadcaster.cs:16-29` (flag read and enforced); `docs/GatewayDashboardDesign.md:170` (current, documents the redaction)
- **Description:** SEC-25 wired `Dashboard:ShowTagValues` into `DashboardEventBroadcaster`: when `false` (default), tag values are blanked from the deep-cloned event before it is mirrored to SignalR. The flag is therefore no longer dead — but the authoritative config reference still labels it "Reserved", so an operator consulting the options table concludes toggling it does nothing, when it actually controls whether tag values leak to every dashboard hub subscriber. This is remediation-created drift: the dashboard design doc was updated in the same change, the config doc was not. (Residual of TST-16 remains separately: the flag still gates nothing in the `/browse` live-value path.)
- **Description:** SEC-25 wired `Dashboard:ShowTagValues` into `DashboardEventBroadcaster`: when `false` (default), tag values are blanked from the deep-cloned event before it is mirrored to SignalR. The flag is therefore no longer dead — but the authoritative config reference still labels it "Reserved", so an operator consulting the options table concludes toggling it does nothing, when it actually controls whether tag values leak to every dashboard hub subscriber. This is remediation-created drift: the dashboard design doc was updated in the same change, the config doc was not. (Residual of TST-16 remains separately: the flag still gates nothing in the `/browse` live-value path.) **Closed 2026-08-17** — the config row was rewritten again when TST-16's residuals shipped: it now states the three seams the flag covers (events-hub mirror, alarms hub, `/browse`), so neither the "Reserved" wording nor the `/browse` carve-out survives. See the TST-16 row above.
- **Recommendation:** Update the `:185` row to describe the mirror-redaction behavior (and its security relevance given the missing per-session ACL, TST-15); note the `/browse` gap or close it when TST-16 is decided.
### TST-28 — Gateway-side `max_frame_bytes` handshake field has no test in the portable suite — **Low** · test coverage
File diff suppressed because one or more lines are too long
@@ -9,12 +9,12 @@ This document turns the 2026-07-12 re-review's **new** Gateway Server Core findi
| ID | Sev | Tier | Eff | Dep | Status | Title |
|----|-----|------|-----|-----|--------|-------|
| GWC-24 | Medium | P1 | M | GWC-21 (coord) | Done | Unbounded event staging channel: sustained slow drain grows memory silently and invisibly |
| GWC-25 | Medium | P0 | S | CLI-35/36 (coord) | Not started | Empty-ring ReplayGap sentinel carries `oldest_available_sequence = 0`, dead-streaming a compliant client |
| GWC-26 | Low | P2 | M | GWC-27 | Not started | Alarm monitor attaches its subscriber after SubscribeAlarms; window transitions bypass the feed, missed Acknowledge never repaired |
| GWC-27 | Low | P2 | S | GWC-26 | Not started | `AttachInternalEventSubscriber` bypasses the readiness gate; premature attach poisons the distributor permanently |
| GWC-28 | Low | P2 | S | GWC-10 (coord) | Not started | Gateway→worker envelope `sequence` stamped at creation, not at write — non-monotonic on the wire under concurrent invokes |
| GWC-29 | Low | — | S | — | Not started | `Invoke` deep-clones the entire request only to discard the cloned command |
| GWC-30 | Info | — | S | — | Not started | Frame reader allocates a fresh 4-byte length-prefix array per frame |
| GWC-25 | Medium | P0 | S | CLI-35/36 (coord) | Done | Empty-ring ReplayGap sentinel carries `oldest_available_sequence = 0`, dead-streaming a compliant client |
| GWC-26 | Low | P2 | M | GWC-27 | Done | Alarm monitor attaches its subscriber after SubscribeAlarms; window transitions bypass the feed, missed Acknowledge never repaired |
| GWC-27 | Low | P2 | S | GWC-26 | Done | `AttachInternalEventSubscriber` bypasses the readiness gate; premature attach poisons the distributor permanently |
| GWC-28 | Low | P2 | S | GWC-10 (coord) | Done | Gateway→worker envelope `sequence` stamped at creation, not at write — non-monotonic on the wire under concurrent invokes |
| GWC-29 | Low | — | S | — | Done | `Invoke` deep-clones the entire request only to discard the cloned command |
| GWC-30 | Info | — | S | — | Done | Frame reader allocates a fresh 4-byte length-prefix array per frame |
Dependency notes: GWC-26 and GWC-27 both change the internal-subscriber attach path (`GatewaySession.AttachInternalEventSubscriber` and its `SessionManager`/alarm-monitor callers) — land GWC-27's readiness gate first (or in the same commit), then GWC-26's reorder, so the reordered monitor attach is proven against the gate. GWC-24 is the direct successor of the prior cycle's GWC-04 backpressure fix and raises the value of the still-open GWC-21 (making `EventChannelFullModeTimeout` configurable); GWC-28 is the gateway half of the worker's WRK-04 fix and must be coordinated with the still-open GWC-10 if inbound sequence enforcement is ever added. GWC-25 is server-complete on its own, but the end-to-end reconnect story also needs the client-domain CLI-35/36 fixes (Python CLI crashes on the sentinel, Go CLI destroys it).
@@ -61,7 +61,7 @@ Coordinate with (do not block on) open GWC-21: if `EventChannelFullModeTimeout`
The proto comment currently states "`oldest_available_sequence` itself IS still retained", which becomes false in the empty-ring case — per the docs-with-source rule, amend the field comment in the same commit to define the empty-ring value ("when nothing is retained, this is the next sequence that can be delivered — `highest observed + 1` — and the `oldest − 1` resume formula remains valid; the interval evicted is unchanged"). This is a comment-only proto change (no descriptor delta), but the repo's codegen rules still apply — see the steps.
**Implementation.**
**Implementation.** (Code + `docs/Sessions.md` landed 2026-08-07 on `fix/gwc-25-replaygap-trio`; the deferred proto-comment amendment below **landed 2026-08-07** with the IPC-23 codegen wave on `fix/ipc-24-25-codegen` — GWC-25 is fully resolved.)
- `Sessions/SessionEventDistributor.cs:463-467`: replace `oldestAvailableSequence = 0;` with `oldestAvailableSequence = gap ? _highestSequenceSeen + 1 : 0;` plus a comment explaining the `oldest − 1` client formula this must keep valid (cite this finding).
- `src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_gateway.proto` (`ReplayGap.oldest_available_sequence`, ~line 759): append the empty-ring sentence above. Then regenerate per repo rules: delete `src/ZB.MOM.WW.MxGateway.Contracts/Generated/*.cs`, `dotnet build src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csproj`, and **commit `Generated/`** (net48 worker builds break otherwise). Sync the vendored client copies of the proto byte-identical (`clients/*/`); a comment-only edit changes no descriptor, so: Python `*_pb2*` output is unchanged (comments are not embedded — regenerate with the pinned grpcio-tools only if the files actually differ), Go/C#/Rust generated doc comments will churn — regenerate those per each client README, and revert spurious Java aggregate-file churn if no message-level delta appears (per the established Java convention).
- `docs/Sessions.md` (~lines 228-234, ReplayGap section): document the empty-ring sentinel value and that `after_worker_sequence = oldest_available_sequence − 1` is the universal resume formula in both the retained and fully-evicted cases.
@@ -16,14 +16,15 @@ members, no positional records). The worker builds and tests only on the Windows
| ID | Sev | Tier | Eff | Dep | Status | Title |
|----|-----|------|-----|-----|--------|-------|
| WRK-21 | Medium | P0 | M | IPC-23 (same defect, fix owned here); WRK-28 (same lines) | Not started | DrainEvents bound is count-based only; an oversized reply still kills the session and loses the drained events |
| WRK-22 | Low | — | S | IPC-26 (same defect, fix owned here) | Not started | Cancelled `WriteAsync` leaves its frame queued; it is still written later |
| WRK-23 | Low | — | S | WRK-21 (rejection path becomes backstop-only) | Not started | Rejected frames consume sequence numbers, producing wire gaps |
| WRK-24 | Low | — | S | — | Not started | `AdoptNegotiatedMaxMessageBytes` has no lower-bound sanity check |
| WRK-25 | Low | P2 | S | WRK-22 (both touch enqueue/dequeue) | Not started | WRK-12 flush coalescing never engages on the event hot path |
| WRK-26 | Low | P1 | S | WRK-23 (soft — sequence prose); discharges IPC-29 | Not started | Write-priority and overflow doc drift from the WRK-07 change |
| WRK-27 | Low | — | S | — | Not started | Alarm poll bypasses the watchdog's in-flight suppression (15 s vs 75 s) |
| WRK-28 | Low | — | S | WRK-21 (land in the same commit cluster) | Not started | 10,000 drain cap is a duplicated magic constant with a comment-only sync contract |
| WRK-21 | Medium | P0 | M | IPC-23 (same defect, fix owned here); WRK-28 (same lines) | Done | DrainEvents bound is count-based only; an oversized reply still kills the session and loses the drained events |
| WRK-22 | Low | — | S | IPC-26 (same defect, fix owned here) | Done | Cancelled `WriteAsync` leaves its frame queued; it is still written later |
| WRK-23 | Low | — | S | WRK-21 (rejection path becomes backstop-only) | Done | Rejected frames consume sequence numbers, producing wire gaps |
| WRK-24 | Low | — | S | — | Done | `AdoptNegotiatedMaxMessageBytes` has no lower-bound sanity check |
| WRK-25 | Low | P2 | S | WRK-22 (both touch enqueue/dequeue) | Done | WRK-12 flush coalescing never engages on the event hot path |
| WRK-26 | Low | P1 | S | WRK-23 (soft — sequence prose); discharges IPC-29 | Done | Write-priority and overflow doc drift from the WRK-07 change |
| WRK-27 | Low | — | S | — | Done | Alarm poll bypasses the watchdog's in-flight suppression (15 s vs 75 s) |
| WRK-28 | Low | — | S | WRK-21 (land in the same commit cluster) | Done | 10,000 drain cap is a duplicated magic constant with a comment-only sync contract |
| WRK-31 | — | — | — | — | Reserved (do not reuse) | Not a finding. `e913dab` (session-owned transport disposal) coined "WRK-31" ad hoc in code comments (`Ipc/WorkerPipeSession.cs:45,177`) and in `Ipc/WorkerPipeSessionTests.cs`, before the register assigned it. WRK-29/WRK-30 appear nowhere in the repo or its history but may have been burned by the same interim numbering, so they are held back defensively too — see the tracker's WRK-29/30/31 rows. Registered here per cycle-3 `archreview/2026-08-19/20-worker.md` (~line 9, "Finding-ID note"); that review's new findings start at **WRK-32** to guarantee no collision. |
---
@@ -12,16 +12,16 @@ All `path:line` citations were re-verified against the working tree at `4f5371f`
| ID | Sev | Tier | Eff | Dep | Status | Title |
|----|-----|------|-----|-----|--------|-------|
| IPC-23 | Medium | P0 | S¹ | WRK-21 | Not started | DrainEvents bound is count-based only; byte-heavy queue still builds a session-killing reply frame (contract requirements here; fix mechanics in WRK-21) |
| IPC-24 | Medium | P0 | S | — | Not started | CI's unconditional Java churn-revert masks real generated-code drift for message-level proto changes |
| IPC-25 | Medium | P0 | M | — | Not started | Committed Go/Python worker bindings are stale at HEAD; no guard covers them |
| IPC-26 | Low | P2 | S¹ | WRK-22 | Not started | Cancelled write leaves a ghost frame that is still written (contract requirement here; fix mechanics in WRK-22) |
| IPC-27 | Low | P2 | S | — | Not started | Descriptor freshness test blind to enums, enum values, services/methods, and the Galaxy contract |
| IPC-28 | Low | — | S | — | Not started | `docs/Grpc.md` omits the `CommandTooLarge` → `ResourceExhausted` mapping |
| IPC-29 | Low | — | S | — | Not started | Worker writer priority scheduling and write-time sequence stamping undocumented in the frame-protocol doc |
| IPC-30 | Low | P0 | M | WRK-21 (same file/batch) | Not started | Oversized worker→gateway event frame is session-fatal — make the death deliberate, structured, and diagnosable |
| IPC-23 | Medium | P0 | S¹ | WRK-21 | Done | DrainEvents bound is count-based only; byte-heavy queue still builds a session-killing reply frame (contract requirements here; fix mechanics in WRK-21) |
| IPC-24 | Medium | P0 | S | — | Done | CI's unconditional Java churn-revert masks real generated-code drift for message-level proto changes |
| IPC-25 | Medium | P0 | M | — | Done | Committed Go/Python worker bindings are stale at HEAD; no guard covers them |
| IPC-26 | Low | P2 | S¹ | WRK-22 | Done (mechanics landed in WRK-22) | Cancelled write leaves a ghost frame that is still written (contract requirement here; fix mechanics in WRK-22) |
| IPC-27 | Low | P2 | S | — | Done | Descriptor freshness test blind to enums, enum values, services/methods, and the Galaxy contract |
| IPC-28 | Low | — | S | — | Done | `docs/Grpc.md` omits the `CommandTooLarge` → `ResourceExhausted` mapping |
| IPC-29 | Low | — | S | — | Done (discharged by WRK-26) | Worker writer priority scheduling and write-time sequence stamping undocumented in the frame-protocol doc |
| IPC-30 | Low | P0 | M | WRK-21 (same file/batch) | Done | Oversized worker→gateway event frame is session-fatal — make the death deliberate, structured, and diagnosable |
| IPC-31 | Info | — | — | — | N/A | Gateway stamps sequence at creation, worker at write — accepted divergence; sequence is documented diagnostic-only (`gateway.md:328-330`); revisit only if sequence ever becomes load-bearing |
| IPC-32 | Info | — | S | IPC-25 | Not started | `check-codegen.ps1` check labels miscounted (folded into the IPC-25 script edit) |
| IPC-32 | Info | — | S | IPC-25 | Done | `check-codegen.ps1` check labels miscounted (folded into the IPC-25 script edit) |
¹ Effort for the work owned by *this* plan (proto comments + docs + acceptance criteria). The code mechanics are M and are tracked under WRK-21 / WRK-22 in the worker plan.
@@ -231,6 +231,8 @@ The published-side collection (`CollectPublishedSymbols`, `:61-79`) grows matchi
**Disposition: N/A — no action.** `gateway.md:328-330` already declares sequence a per-sender diagnostic aid, never validated; both validators ignore it; the divergence is harmless and now explicitly documented by the IPC-29 doc section (which states the two sides' stamping points). If sequence ever becomes load-bearing, stamp in the gateway write loop (`WorkerClient.cs:390-397`) — recorded here so the next editor finds the decision.
**Resolution: Done — superseded by GWC-28 (2026-08-20, per cycle-3 verification).** GWC-28 moved the gateway-side stamp from creation to the point of writing: `WriteLoopAsync` now stamps `Sequence` immediately before `_writer.WriteAsync` (`Server/Workers/WorkerClient.cs:465-471`), and `CreateEnvelope` leaves it unset (`:1229-1236`); `gateway.md:425-431` was rewritten to "both sides stamp at the point of writing", closing the divergence this finding recorded. Regression test `WorkerClientTests.ConcurrentInvokesEmitStrictlyIncreasingSequencesOnTheWire` covers it. Per `archreview/2026-08-19/30-contracts-ipc.md` (IPC-31 row), tracker flipped from `N/A` to `Done (superseded by GWC-28)`. See cycle-3 **IPC-39** for the one interaction the change created with the IPC-03 pre-check.
---
## IPC-32 — `check-codegen.ps1` check labels are miscounted `Info` · `—` (rides IPC-25)
@@ -10,12 +10,12 @@ Repo rules that bind every entry: docs change in the same commit as the source (
| ID | Sev | Tier | Eff | Dep | Status | Title |
|----|-----|------|-----|-----|--------|-------|
| SEC-31 | Medium | P0 | M | — | Not started | Failure limiter partitions on attacker-controlled key id and blocks before verification (lockout DoS) |
| SEC-32 | Low | P0 | S | SEC-31 | Not started | Failure-limiter LRU is flushable by junk-token spray; token prefix never validated |
| SEC-33 | Low | P1 | M | — (co-locate SEC-23) | Not started | Any-platform path-rooting acceptance re-opens SEC-01 on Unix; Galaxy `SnapshotCachePath` unvalidated |
| SEC-34 | Low | P2 | S | — | Not started | Verification cache: expiry outlives TTL; `Invalidate` races in-flight repopulation |
| SEC-35 | Info | — | S | — | N/A (doc-only note) | Production hard-stops key on the exact `Production` environment name |
| SEC-36 | Low | P1 | M | cross-repo (`scadaproj/infra/glauth`) | Not started | Committed dev LDAP service-account password: remove from repo and rotate |
| SEC-31 | Medium | P0 | M | — | Done | Failure limiter partitions on attacker-controlled key id and blocks before verification (lockout DoS) |
| SEC-32 | Low | P0 | S | SEC-31 | Done | Failure-limiter LRU is flushable by junk-token spray; token prefix never validated |
| SEC-33 | Low | P1 | M | — (co-locate SEC-23) | Done | Any-platform path-rooting acceptance re-opens SEC-01 on Unix; Galaxy `SnapshotCachePath` unvalidated |
| SEC-34 | Low | P2 | S | — | Done | Verification cache: expiry outlives TTL; `Invalidate` races in-flight repopulation |
| SEC-35 | Info | — | S | — | N/A (doc-only note discharged 2026-08-07) | Production hard-stops key on the exact `Production` environment name |
| SEC-36 | Low | P1 | M | cross-repo (`scadaproj/infra/glauth`) | Done | Committed dev LDAP service-account password: remove from repo and rotate |
---
@@ -135,6 +135,8 @@ dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --fil
```
Post-run, verify no new `C:\*` file exists under any `bin/` (manual `find src -name 'C:*'`).
**Outcome (2026-08-07 — Done).** Implemented as designed. `IsRootedForAnyPlatform` deleted; `AddIfNotRooted`/`AddIfInvalidPath` promoted to a shared `GatewayConfigPathRules` internal helper (used by both validators) and now use `Path.IsPathRooted` (current OS). Both Windows literals removed from `appsettings.json`; the Galaxy `SnapshotCachePath` default is applied gateway-side via a configuration value seeded before `AddZbGalaxyRepository` (the package's `SnapshotCachePath` is **init-only**, so a `PostConfigure` mutation does not compile — deviation from the design's "PostConfigure default"; same effect). New `GalaxyRepositoryOptionsValidator` registered with `ValidateOnStart`. **Stray-file root cause:** starting the full host eagerly constructs `AuthSqliteConnectionFactory`, which creates the auth DB path; with the shipped Windows literal that path is non-rooted on macOS, so SQLite materialized `C:\ProgramData\MxGateway\gateway-auth.db` as a junk-named relative file under the test's `bin/` CWD (invisible to the hygiene test's bin/obj filter). After the literal removal the code default resolves under an unwritable `/usr/share` on macOS, so the three tests that start the real host (`GatewayApplicationTests.Build_MapsMetricsEndpoint`, `.StartAsync_InvalidGatewayConfiguration_FailsStartup`, `GatewayTlsBootstrapTests`) now pin `SqlitePath` to a temp path. No stray file remains (`find src -name 'C:*'` empty).
---
## SEC-34 — Verification cache: expiry outlives TTL; `Invalidate` races in-flight repopulation `Low` · `P2`
@@ -165,6 +167,8 @@ Post-run, verify no new `C:\*` file exists under any `bin/` (manual `find src -n
dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter "FullyQualifiedName~CachingApiKeyVerifier"
```
**Outcome (2026-08-07 — Done).** Window 3 (Invalidate race) implemented exactly as designed: per-key generation counter, bump-before-evict in `Invalidate`, snapshot-before-inner + set-then-recheck in `VerifyAsync`, key id parsed from the token up front. Covered by `Invalidate_DuringInFlightVerification_DiscardsStaleRepopulation`. **Window 2 (expiry cap) took the documented fallback**, not the cap: the design's confirmation step failed — the library verification identity (`ZB.MOM.WW.Auth.Abstractions.ApiKeys.ApiKeyIdentity`, the type on `ApiKeyVerification.Identity`) carries **no** `ExpiresUtc` (that property is on `ApiKeyRecord`, the store row, not the returned identity), so the cache cannot cap an entry at the key's expiry. Per the design's contingency, the ≤ TTL expiry window is documented in the class remarks and `docs/Authentication.md`, with a donor-library ask (surface expiry on the verification identity). Consequently the two expiry-cap tests (`CacheEntry_DoesNotOutliveKeyExpiry`, `AlreadyExpiredIdentity_IsNotCached`) are **not** added — they cannot be written against a type with no expiry field; window 1 (CLI) accepted and documented as before.
---
## SEC-35 — Production hard-stops key on the exact `Production` environment name `Info` · `—` (N/A: doc-only)
@@ -179,6 +183,8 @@ dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --fil
**Verification.** Doc-only; no build. Cross-read against `GatewayOptionsValidator.cs:23-27`.
**Outcome (2026-08-07 — discharged).** The documentation contract landed as a rider on the SEC-33/34 commit: `docs/GatewayConfiguration.md` gained a "Production hard-stops key on the exact environment name (SEC-35)" subsection stating that both hard-stops fire only on `IHostEnvironment.IsProduction()` (unset `ASPNETCORE_ENVIRONMENT` or the exact `Production` name) and that any other name keeps the permissive dev posture. No code change, as designed.
---
## SEC-36 — Committed dev LDAP service-account password: remove from repo and rotate `Low` · `P1` · cross-repo dependency
@@ -212,3 +218,9 @@ dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --fil
dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter "FullyQualifiedName~GatewayOptionsValidator"
```
(asserts the blank-password validation still fires with the updated message). Manual: with user-secrets set on the dev box, `dotnet run --project src/ZB.MOM.WW.MxGateway.Server/...` and a dashboard `/login` as `multi-role` succeeds against the rotated GLAuth; the deployed-host login re-check from step 1 counts as the production verification. Live-LDAP integration tests (`MXGATEWAY_RUN_LIVE_LDAP_TESTS=1`) only where the GLAuth instance is reachable; otherwise document skipped per the testing matrix.
**Outcome (2026-08-07 — Done, repo-side; live rotation operator-pending).** Landed on `fix/sec-36-ldap-secret`. **The design's baseline had already shifted:** at HEAD `appsettings.json` no longer commits the literal — it ships `"ServiceAccountPassword": "${secret:ldap/mxgateway/bind}"`, a fail-closed encrypted-store reference (documented `GatewayConfiguration.md:252`, tested by `PreHostSecretExpansionTests`) introduced by the Secrets-store adoption after this remediation was written. **Deviation from Implementation step 2:** the `${secret:}` reference was **kept, not deleted** — deleting it regresses the shipped/documented/tested store channel and the committed-plaintext finding is already resolved for `appsettings.json`. The load-bearing residual — the literal value still present in `glauth.md`'s samples (`:33,65,103,136,245`), `docs/GatewayTesting.md`, and the historical `archreview/*` SEC-06 evidence — was scrubbed to `<service-account-password>` placeholders, each with a pointer to the source of truth `scadaproj/infra/glauth/` and a rotation-required note. Steps 3–6 implemented as designed: `<UserSecretsId>mxaccessgw-server</UserSecretsId>` added (step 3); the `ValidateLdap` blank-password message now names both channels — dev `dotnet user-secrets set "MxGateway:Ldap:ServiceAccountPassword" <value>` and deployed `MxGateway__Ldap__ServiceAccountPassword` — plus a note on the `${secret:}` store default (step 4), asserted by the extended `Validate_Fails_WhenLdapEnabledAndServiceAccountPasswordBlank`; docs updated same commit (step 5); `git grep -i` for the old value is empty across tracked files (step 6). The cross-repo **step 1 (rotate GLAuth on `10.100.0.35`, pre-stage the NSSM env var on `10.100.0.48` and on `wonder-app-vd03` only if `Ldap.Enabled`, verify dashboard login)** is the operator's to execute, captured in the new runbook `docs/runbooks/SEC-36-ldap-credential-rotation.md`. Verification (macOS): `dotnet build …Server` 0 warnings/0 errors; `dotnet test --filter ~GatewayOptionsValidator` green.
**Outcome (2026-08-07 — operator half executed; finding now fully `Done`).** The cross-repo step 1 left open above was executed per `docs/runbooks/SEC-36-ldap-credential-rotation.md`. A new service-account password was generated, the `serviceaccount` `passsha256` in `scadaproj/infra/glauth/config.toml` replaced, and the shared GLAuth recreated on `10.100.0.35` from its actual compose directory — the **load-bearing** half of the finding is now discharged: the value disclosed by this repo's git history (live in the directory since 2026-06-04; not reproduced here) no longer binds `dc=zb,dc=local`. The new value exists only in the three channels the design named — the GLAuth `passsha256` (committed in `scadaproj`, commit `aada53b`), the NSSM service environment on `10.100.0.48`, and each dev box's user-secrets — and in no file of this repo. The retired plaintext was additionally scrubbed from the `scadaproj` glauth comments (`config.toml`, `docker-compose.yml`, `README.md`) and from the docker host's live `docker-compose.yml`; the host's `*.bak-sec36` rollback copies deliberately retain it. **Three runbook facts were wrong and are corrected in a dated block at its top.** (1) Its step 3 said `cd ~/Desktop/scadaproj/infra/glauth` on the docker host; no such path exists there — the stack runs from `/home/dohertj2/zb-glauth` (container `zb-shared-glauth`), fed by the `scp` deploy documented in `scadaproj/infra/glauth/README.md`. (2) `wonder-app-vd03` is **out of scope on documentary evidence**, not merely unchecked: its gateway binds the ScadaBridge/ScadaLink local GLAuth under `dc=scadalink`/`dc=scadabridge`, a different directory that never held this credential (the host is also unreachable from the dev network); no env var was staged there. (3) Its "3-fail / 10-minute per-IP lockout" caution is **inert for this instance** — `config.toml:14` sets `LimitFailedBinds = false`. **One Done criterion is met with a caveat:** the new value **is** staged on windev (`10.100.0.48`, 10th `AppEnvironmentExtra` entry on the `MxAccessGw` NSSM service), but the runbook's primary check — dashboard `/login` as `multi-role` — **could not run**, because windev's gateway is crash-looping on an unrelated pre-existing fault: the deployed Server binary (2026-06-25) predates the 2026-07-15 auth-DB migration, so it opens a schema-version-3 database it supports only at version 2 and aborts at startup (~10k Hosting-failed events/day since at least 08-06). That is a stale-deployment problem, filed as a next-cycle candidate finding, not a rotation defect. **Verified instead by the equivalent primitive:** a direct `ldapsearch` bind as `cn=serviceaccount,dc=zb,dc=local` with the new value against `10.100.0.35:3893` succeeded and returned the `multi-role` entry — the same search bind the dashboard performs. Also surfaced and filed for next cycle: `DashboardLdapLiveTests` fixture drift leaves the suite with **no positive-proof coverage** of the service-account bind, so it could not have substituted for the dashboard check either. Tracking: both registers' SEC-36 rows, the pending-operator-actions list in `90-candidate-findings-next-cycle.md`, and the `00-tracking.md` progress log.
**Addendum (2026-08-07, later the same day — the caveat is closed).** windev was repaired under NEXT-07 (fresh publish of `origin/main` `a346d51`), and the deferred dashboard check then ran on that host: with `Dashboard:DisableLogin=false` supplied as a process-env-only override on a foreground run, `GET /login` returned 200 with an antiforgery token, `POST /auth/login` as `multi-role`/`password` returned 302 to `/` with a `MxGatewayDashboard` cookie, the authenticated `GET /` rendered the admin nav, and an anonymous control redirected to `/login?ReturnUrl=%2F`. The rotated credential is therefore proven through the real `DashboardAuthenticator` search-bind path on the deployed host, not only by the `ldapsearch` primitive. As deployed windev keeps `DisableLogin=true`, so routine operation there does not exercise LDAP; the standing regression proof is the realigned `DashboardLdapLiveTests` (NEXT-06, commit `de67b45`), 5/5 green against the shared GLAuth. `docs/runbooks/SEC-36-ldap-credential-rotation.md` Correction 3 carries the same record.
+11 -11
View File
@@ -16,17 +16,17 @@ Operating constraints carried from prior work:
| ID | Sev | Tier | Eff | Dep | Status | Title |
|----|-----|------|-----|-----|--------|-------|
| CLI-35 | Medium | P0 | S | — | Not started | Python CLI `stream-events` crashes on a ReplayGap |
| CLI-36 | Medium | P0 | S | — | Not started | Go CLI `stream-events` silently destroys the ReplayGap signal |
| CLI-37 | Medium | P1 | M | CLI-38 | Not started | Status-array validation must branch on `category` per the proto contract (4-vs-1 divergence) |
| CLI-38 | Medium | P1 | S | — | Not started | Align .NET/Go/Java on `hresult < 0` — lands prior CLI-08 and cures the design-doc drift |
| CLI-39 | Medium | P1 | S | CLI-35..38, CLI-45 | Not started | Bump client versions off the already-published 0.1.2 before the next publish; add registry-collision guard |
| CLI-40 | Low | — | M | — | Not started | Port the exact-secret credential scrub to Rust/Java/.NET |
| CLI-41 | Low | — | M | — | Not started | Uniform malformed-reply contract for AuthenticateUser/ArchestrAUserToId/AddBufferedItem |
| CLI-42 | Low | P1 | S | — | Not started | Document the vendored Rust proto layout (CLI-02's missing doc half) |
| CLI-43 | Low | — | S | — | Not started | Java style guide still prescribes "Java 21 preferred" |
| CLI-44 | Low | — | S | — | Not started | Go event goroutine can mislabel a genuine terminal error as `ErrSlowConsumer` |
| CLI-45 | Low | P1 | M | — | Not started | Standardize CLI credential env-var name and fail fast on missing/empty passwords |
| CLI-35 | Medium | P0 | S | — | Done | Python CLI `stream-events` crashes on a ReplayGap |
| CLI-36 | Medium | P0 | S | — | Done | Go CLI `stream-events` silently destroys the ReplayGap signal |
| CLI-37 | Medium | P1 | M | CLI-38 | Done | Status-array validation must branch on `category` per the proto contract (4-vs-1 divergence) |
| CLI-38 | Medium | P1 | S | — | Done | Align .NET/Go/Java on `hresult < 0` — lands prior CLI-08 and cures the design-doc drift |
| CLI-39 | Medium | P1 | S | CLI-35..38, CLI-45 | Done | Bump client versions off the already-published 0.1.2 before the next publish; add registry-collision guard |
| CLI-40 | Low | — | M | — | Done | Port the exact-secret credential scrub to Rust/Java/.NET |
| CLI-41 | Low | — | M | — | Done | Uniform malformed-reply contract for AuthenticateUser/ArchestrAUserToId/AddBufferedItem |
| CLI-42 | Low | P1 | S | — | Done | Document the vendored Rust proto layout (CLI-02's missing doc half) |
| CLI-43 | Low | — | S | — | Done | Java style guide still prescribes "Java 21 preferred" |
| CLI-44 | Low | — | S | — | Done | Go event goroutine can mislabel a genuine terminal error as `ErrSlowConsumer` |
| CLI-45 | Low | P1 | M | — | Done | Standardize CLI credential env-var name and fail fast on missing/empty passwords |
Cross-domain dependencies: **CLI-35/CLI-36 pair with GWC-25** (gateway emits `oldest_available_sequence = 0` on an empty replay ring — the server-side half of the same reconnect story; the CLI fixes here are independently landable but the end-to-end resume walk in the smoke matrix needs both). **CLI-39 pairs with the publishing process** (`scripts/pack-clients.ps1`, `scripts/tag-go-module.ps1`, Gitea package registry).
@@ -12,10 +12,10 @@ Prior-cycle open findings (TST-05..24 where still open) are tracked in the prior
|----|-----|------|-----|-----|--------|-------|
| TST-25 | High | P1 | M | — (unlocks TST-05, TST-24) | Done | Windows/x86 test tier has zero automation — restore via SSH-driven windev CI job |
| TST-26 | Medium | P1 (folded into TST-25) | S | TST-25 | Done | docs/GatewayTesting.md, check-codegen.ps1, and ci.yml comments describe removed CI jobs |
| TST-27 | Medium | P1 (doc batch) | S | — | Not started | `ShowTagValues` config row still says "Reserved" after SEC-25 made the flag live |
| TST-28 | Low | P2 | S | relates IPC-02 | Not started | Gateway-side `max_frame_bytes` handshake field untested in the CI-run suite |
| TST-27 | Medium | P1 (doc batch) | S | — | Done | `ShowTagValues` config row still says "Reserved" after SEC-25 made the flag live |
| TST-28 | Low | P2 | S | relates IPC-02 | Done | Gateway-side `max_frame_bytes` handshake field untested in the CI-run suite |
| TST-29 | Low | P2 | S | — | Done | Retire `oldtasks.md` after folding the Phase-5 governance record into DesignDecisions.md; delete root docs-review artifacts |
| TST-30 | Low | P2 | M | — | Not started | Single shared Gitea runner is a CI throughput/availability bottleneck (cross-repo contention, no run cancel/delete) |
| TST-30 | Low | P2 | M | — | Done | Single shared Gitea runner is a CI throughput/availability bottleneck (cross-repo contention, no run cancel/delete) |
---
@@ -167,11 +167,15 @@ Independent of the runner count, document the **no-cancel** reality (Gitea 1.26
**Verification.** Push two branches back-to-back and confirm their runs execute concurrently (not serially) once a second runner exists; `GET /repos/dohertj2/mxaccessgw/actions/runners` (or the instance runner list) shows ≥2 runners online; `docs/GatewayTesting.md` describes the shared-runner/no-cancel reality and the bypass. Re-run the TST-25 acceptance push and confirm queue depth is materially lower under a concurrent `lmxopcua` run.
**Outcome (2026-08-07 — Done, doc half; runner registration operator-pending).** Landed on `fix/tst-30-runner-docs`. Implementation step 2 shipped: `docs/GatewayTesting.md`'s Continuous Integration section gained a "Runner capacity is shared and finite" subsection stating the `maxParallel=1` co-located runner is shared with `dohertj2/lmxopcua` at the instance level (not repo-scoped), the ~20–30 minute queue latency observed under cross-repo contention, and the Gitea 1.26 no-cancel/no-delete API reality; the existing "windev tier down" degraded-mode paragraph now also covers "runner contended" as a reason to use the bypass, generalized per this finding's design note. New operator runbook `docs/runbooks/TST-30-second-ci-runner.md` carries **step 1** (register a second `act_runner` on `10.100.0.35`, option (a) recommended, same `container.network: traefik` config; option (b) dedicated labelled runner as an escalation; option (c) windev-hosted runner rejected) with the verification checklist (concurrent back-to-back pushes, `GET /repos/dohertj2/mxaccessgw/actions/runners` ≥ 2) and a note that the no-cancel reality persists regardless of runner count. **Step 3 (optional workflow-level `concurrency` group)** is documented in the runbook as unverified — explicitly framed as "verify this Gitea deployment honors it before relying on it" — and left unimplemented in `ci.yml`, since it is a `ci.yml` change out of scope for this doc-only pass. **The actual runner registration (step 1) is infrastructure work outside this repo's tree and remains the operator's to execute**, tracked in the runbook. Verification performed: `grep -n 'maxParallel\|shared\|cancel' docs/GatewayTesting.md` shows the new prose; runbook file exists at the path above; no build required (doc-only change).
**Outcome (2026-08-07 — operator half executed; finding now fully `Done`).** The runner registration left open above was executed per `docs/runbooks/TST-30-second-ci-runner.md` option (a). A second instance-level `act_runner` container, `gitea-runner-2` (runner id 5, capacity 2, labels `ubuntu-latest`/`ubuntu-22.04`), now runs on `10.100.0.35` from the `/opt/gitea` compose stack with the same `container.network: traefik` setting as the original; its registration token is mounted from a `0600` file rather than inlined in compose. The existing `gitea-runner` (id 1, capacity 4) was **not** modified — capacity went 4 → 6 by addition, so the change is reversible by removing one container. Concurrency verified live by pushing HEAD (`a346d51`) to two scratch branches, `scratch/tst30-a` (run 661) and `scratch/tst30-b` (run 662), while an unrelated run (660) was already in flight: at 13:07:53Z jobs from **three** runs were `in_progress` simultaneously — run 660 `portable` and run 662 `portable`/`java` on runner 1, run 661 `portable`/`java` on `gitea-runner-2` — which the pre-change single-runner topology could not have produced. `gitea:3000` resolution holds on the new instance: run 661's `portable` job (task 1145, scheduled on `gitea-runner-2`) logged `git remote add origin http://gitea:3000/dohertj2/mxaccessgw` followed by a successful `fetch … From http://gitea:3000/dohertj2/mxaccessgw`, and its job container's workspace was confirmed checked out at `a346d514dd24e775640e5667aa7cd8e561fec68a`. **Runbook correction:** its verification checklist said `GET /repos/dohertj2/mxaccessgw/actions/runners` should show ≥2 — that endpoint still returns `total_count: 0` because both runners are registered at the **instance** level, exactly as this finding documented; the correct check is `GET /api/v1/admin/actions/runners`, which lists ids 1, 4 (an unrelated local macOS runner), and 5. Recorded as a dated "Executed" note at the top of the runbook. The no-cancel reality is unchanged and the `run-windev-ci.sh` bypass remains valid, so `docs/GatewayTesting.md`'s prose needed no edit.
---
## Cross-domain dependencies
- **TST-25 → old TST-05 / old TST-24:** the SSH-driven nightly is where the scheduled live-MXAccess smoke lands (closes old TST-05), and a working CI Windows tier unblocks wiring client wire-behavior tests into CI (old TST-24).
- **TST-25 → old TST-05 / old TST-24:** the SSH-driven nightly is where the scheduled live-MXAccess smoke lands (closes old TST-05), and a working CI Windows tier unblocks wiring client wire-behavior tests into CI (old TST-24). *Followed up 2026-08-10:* old TST-05 is `Partially done` — the nightly closes the scheduling half, but the live suite still covers none of the five worker **control** commands; old TST-24 is `Done`, and its client wire tests turned out to need no Windows tier at all (they run in the `portable` job). See the first-cycle tracker.
- **TST-25 ↔ IPC-24/IPC-25:** the nightly windev job is also the natural home for any Windows-side codegen verification the contracts/IPC remediation adds; coordinate job naming so both plans extend the same `windows-x86`/nightly jobs rather than adding parallel ones.
- **TST-26 ⊂ TST-25:** same commit, by rule.
- **TST-27:** ships in the cycle's P1 doc-drift batch alongside WRK-26 and CLI-42 (roadmap item 8); its `/browse` residual stays with TST-16 (prior cycle).
@@ -0,0 +1,25 @@
# Candidate Findings for the Next Review Cycle (surfaced during 2026-07-12 remediation)
These were discovered while remediating the 2026-07-12 backlog but were **out of scope** for it — each is either pre-existing, by-design residual, or a new observation. They are recorded here so the next review cycle can triage them. None blocks the 2026-07-12 cycle, which is complete. Rows struck through have since been fixed ahead of that cycle; the original finding text is kept so the triage record stays readable.
| ID (proposed) | Area | Severity (est.) | Summary |
|---|---|---|---|
| ~~NEXT-01~~ | Testing / macOS | Low | **Resolved 2026-08-10** — the pipe name is now `mxgw-{pid}-{sessionUid}` (session guid hex, worst-case 43 chars), which fits the 104-byte `sun_path` budget under the default macOS `TMPDIR`; the three test-fixture pipe names were shortened the same way, and a `SessionManagerTests` regression pins the format and length budget. All previously failing suites (SessionWorkerClientFactory, e2e fake-worker smoke, WorkerClient, reconnect-replay) pass 33/33 under the default `TMPDIR` — this also retired the separately-remembered "macOS pipe-timeout test failures", which were this throw misread. Docs updated (gateway.md, GatewayProcessDesign, GatewayConfiguration, Sessions, CLAUDE.md). Original finding: Fake-worker/e2e gateway tests fail on macOS under the default `TMPDIR` because the `CoreFxPipe_mxaccess-gateway-{pid}-{sessionId}` path exceeds the 104-char Unix-domain-socket `sun_path` limit under `/var/folders/…/T/`. Workaround today is `TMPDIR=/tmp`. Fix options: shorten the pipe name, or document the `TMPDIR=/tmp` requirement in `docs/GatewayTesting.md`. Surfaced independently by multiple remediation agents. |
| ~~NEXT-02~~ | Clients (.NET, Java) | Low | **Resolved 2026-08-10** — both CLIs now branch on the sentinel and emit the typed cross-CLI row with numeric cursors (Java text mode prints `REPLAY_GAP requested_after=<n> oldest_available=<n>`; .NET emits the `{"replayGap":{…}}` row in jsonl/text and inside the `--json` events array). CrossLanguageSmokeMatrix.md's divergence table collapsed to one converged contract. New CLI regressions in both languages (.NET 35/35, Java 52/52). Original finding: The .NET and Java CLIs render the raw `ReplayGap` sentinel `MxEvent` on `stream-events` instead of a typed gap row — Java text mode prints `0 MX_EVENT_FAMILY_UNSPECIFIED`. Same defect class as CLI-36 (Go) / CLI-35 (Python), which were fixed this cycle; the .NET/Java halves were out of scope. The cross-language smoke matrix now records this divergence honestly. |
| ~~NEXT-03~~ | Gateway alarms | Low | **Resolved 2026-08-10** — best-effort dedup in `GatewayAlarmMonitor`: a buffered live transition whose worker timestamp + resulting state the cache already carries from a repair is suppressed, and reconcile Clear repairs tombstone the instance by `original_raise_timestamp` for one reconcile generation so the buffered live Clear dedups too. Positive-match only (unset timestamps never suppress), so the documented at-least-once consumer contract stands; serialization was rejected as the larger change that still needs a worker-side high-water mark to be correct. Two new race-driving regressions; alarm suites 18/18. Original finding: `GatewayAlarmMonitor.ApplyReconcile` feed-repair broadcasts (the new acked-delta from GWC-26 **and** the pre-existing Raise/Clear repair) are **at-least-once, not exactly-once**: a periodic reconcile can synthesize a transition whose matching live transition is still buffered in the alarm lease, so both broadcast as indistinguishable duplicates on the alarm feed (StreamAlarms + dashboard hub). Pre-existing (the Raise/Clear repair always had it); GWC-26 documented the at-least-once contract rather than closing the race. Closing it needs reconcile/live serialization or a monotonic dedup marker. |
| ~~NEXT-04~~ | Worker frame writer | Low | **Resolved 2026-08-10** — the tombstone helpers now attach a fault-observing continuation to every frame of a cancelled call (a cancelled task never fires `OnlyOnFaulted`, so unconditional attach is safe; covers both the claimed-mid-write frame and the already-faulted-by-`FailAllQueued` frame where `TrySetCanceled` loses). New regression drives the exact abandonment and asserts a marker exception never reaches `TaskScheduler.UnobservedTaskException`. Original finding: WRK-22/WRK-25 cancellation path: a frame `Claimed` by a concurrent lock-holder just before its caller's cancellation races in is never awaited by that caller; if the write then faults, `TrySetException` lands on a `Task` nobody observes (unobserved-task-exception). By-design residual, non-crash (no `UnobservedTaskException` handler registered), pre-existing to single-frame WRK-22 and amplified per-batch by WRK-25. Hygiene fix: attach a fault-observing continuation to abandoned/tombstoned frame completions. |
| ~~NEXT-05~~ | Worker frame writer | Info | **Resolved 2026-08-10 as a documented decision** — the lazy `DequeueNext` purge stays: any subsequent write drains both queues to empty and the heartbeat loop bounds tombstone residency to one interval, while eager `Queue<T>` rebuilds under `_gate` would add ordering-invariant surface next to the WRK-22 interlock for no real gain. Rationale recorded in `docs/WorkerFrameProtocol.md`. Original finding: A batch whose remaining frames are tombstoned by cancellation leaves dead `PendingFrame` entries in `_eventFrames`/`_controlFrames` until a future `DequeueNext` pops and skips them. Same pre-existing behavior as single-frame WRK-22, amplified per-batch; in practice heartbeats purge them promptly, so not a real leak. |
| ~~NEXT-06~~ | Testing / live LDAP | Medium | **Resolved 2026-08-07** — fixtures realigned to the shared directory (`admin`/`password` for the GwAdmin success path, `gw-viewer`/`password` for the bind-succeeds-but-no-role path); verified `Failed: 0, Passed: 5` live against the shared GLAuth at `10.100.0.35:3893`, so the success-path assertion (GwAdmin group claim + Admin role claim) now fails if the service-account credential is wrong. Original finding: `DashboardLdapLiveTests` fixtures have drifted from the shared GLAuth directory, leaving the suite with **no positive-proof coverage of the service-account bind**. Its only success-path test, `AuthenticateAsync_AdminInGwAdminGroup_Succeeds`, binds `admin`/`admin123`, but the directory's `admin` user carries the standard dev password (`scadaproj/infra/glauth/config.toml`), so that assertion cannot pass. `AuthenticateAsync_ReadOnlyUserMissingGwAdminGroup_Fails` binds fixture user `readonly`, which **does not exist** in the GLAuth config at all — it passes for the wrong reason (user-not-found rather than the group-missing branch it names; the `readonly` name is in fact barred by the README's user/group case-collision rule). The three remaining tests are negative assertions that pass whether or not the service account can bind. Net effect: a green `DashboardLdapLiveTests` run proves nothing about the bind credential — surfaced during SEC-36, where the suite was considered as a substitute for the deferred dashboard-login check and rejected. Fix: realign the fixtures to real directory users (e.g. `multi-role`/`gw-viewer`) or add the missing users to the GLAuth config, and add one test that fails when the service-account credential is wrong. |
| ~~NEXT-07~~ | Deployment / windev | High | **Resolved 2026-08-07** — a fresh portable framework-dependent publish of `origin/main` (`a346d51`) was built in a clean clone at `C:\build\mxgw-redeploy`, deployed to `C:\publish\mxaccessgw\Server-20260807`, and the `MxAccessGw` NSSM service repointed at it; the service now holds a stable PID with both `5120`/`5130` listening, a worker spawned, the Galaxy snapshot restored (129 objects / 56,731 attributes) and a clean event log. Root cause confirmed as the version skew this row predicted: the deployed 2026-06-25 build carried `ZB.MOM.WW.Auth.ApiKeys` 0.1.2.0, which supports auth-DB schema 2, against a `gateway-auth.db` stamped at schema 3 on 2026-07-15 by an ephemeral run of newer code — schema 3 is the current shared-lib version (`SqliteAuthSchema.CurrentVersion=3` in Auth 0.1.5), so the redeploy is the forward fix and the DB was left alone. Rollback artifacts kept: `C:\ProgramData\MxGateway\gateway-auth.db.bak-next07` (with `-wal`/`-shm`) and the previous `C:\publish\mxaccessgw\Server` directory. Two side effects worth recording: the old deploy's `appsettings.json` held the LDAP bind password in **plaintext on disk**, while the new one keeps the repo's `${secret:ldap/mxgateway/bind}` token with the NSSM environment supplying the value, so no plaintext LDAP secret remains on that host; and the redeploy tripped the SEC-06 `Ldap:Transport=None` production hard-stop (`GatewayOptionsValidator.cs:178`), resolved by relabelling the host — windev runs `Dashboard:DisableLogin=true`, which this repo's own docs mark dev/test-only, so its `Production` label contradicted its configuration and `DOTNET_ENVIRONMENT` was changed to `Staging` (that one NSSM environment entry only; the other nine preserved byte-identical). SEC-06 is untouched for genuinely production hosts — see NEXT-08 for the posture problem that relabelling defers. Original finding: The `10.100.0.48` (windev) gateway deployment is **stale and crash-looping**, and has been since at least 2026-08-06 (~10k Hosting-failed events/day). The deployed Server binary dates to 2026-06-25 and predates the auth-DB migration of 2026-07-15: it opens a schema-version-3 `gateway-auth.db` that it supports only at version 2 and aborts at startup, so the `MxAccessGw` service never reaches a listening state. Not a code defect in the current tree — a deploy-drift/operations gap — but it means the repo's only deployed host has been dark for over a day and any host-level verification (including SEC-36's dashboard-login check) is blocked until it is repaired. Fix: deploy a current Server build to windev, or restore/downgrade the auth DB to schema 2 if the old binary must stand. Worth asking separately why a service in a permanent restart loop raised no alert. Discovered during SEC-36. |
| NEXT-08 | Security / LDAP posture | Medium | **The shared GLAuth offers no TLS, so SEC-06 makes it undeployable from a `Production`-labelled host.** `GatewayOptionsValidator` (`src/ZB.MOM.WW.MxGateway.Server/Configuration/GatewayOptionsValidator.cs:178`) refuses to start when `Ldap:Transport=None` in the `Production` environment, and `docs/GatewayConfiguration.md`'s `Transport` row states "Deployed hosts must set `Ldaps` or `StartTls`" — but the shared instance at `10.100.0.35:3893` has `[ldaps] enabled=false`, port `3894` closed, and answers StartTLS with `protocolError`, so neither value can work against it. That instruction is currently unsatisfiable for every host that authenticates there. windev sidestepped it on 2026-08-07 by moving to the `Staging` environment name (NEXT-07), which is honest for a dev/test rig but is not available to a real production host. Resolution needs either LDAPS/StartTLS on the shared GLAuth (certificate plus a trust story on each gateway host) or an explicit written posture decision that production gateways bind a different, TLS-capable directory. Surfaced during the NEXT-07 redeploy. |
| ~~NEXT-09~~ | Build / versioning | Low | **Resolved 2026-08-10** — the Exec path now appends `.` so the trailing backslash can no longer escape the closing quote, and `SourceRevisionId` is additionally gated on a short-SHA regex so no future git failure text can be stamped either. macOS stamp verified unchanged; Windows stamp verified on windev with this batch. Original finding: **Windows builds stamp git's error text into `InformationalVersion`.** `src/Directory.Build.props:29` runs `git -C "$(MSBuildThisFileDirectory)" …`; MSBuild's directory property ends in a backslash, which escapes the closing quote, so the command is malformed on Windows. The target carries `ContinueOnError`, so the failure is silent and git's stderr is captured as the revision — an observed stamp reads `0.1.2+fatal: cannot change to …`. Any Windows build without a preset `SourceRevisionId` therefore ships a binary that cannot be correlated back to a commit, defeating the point of TST-11. Not reproducible on macOS/Linux, where the separator is `/`. Fix sketch: append `.` to the path or trim the trailing separator before quoting. Surfaced while identifying the deployed binary during NEXT-07. |
| NEXT-10 | Docs / glauth | Medium | **`glauth.md`'s "Pre-provisioned users" table contradicts both the directory and the rest of its own file.** It documents `readonly`/`readonly123` and `admin`/`admin123`, neither of which matches `scadaproj/infra/glauth/config.toml` (`readonly` does not exist there; `admin` carries the standard dev password), and lists the `ReadOnly` gid as `5501` against an actual `5601`. Its dashboard section, by contrast, is correct — so the file is internally inconsistent and a reader cannot tell which half to trust. This table was the **root cause of the NEXT-06 fixture drift**, and it has propagated further: `docs/GatewayTesting.md`'s `MXGATEWAY_LIVE_MXACCESS_WRITE_SECURED_PASSWORD` default and the matching literal in `WorkerLiveMxAccessSmokeTests` both take `admin123` from it. Deliberately **not** fixed in the 2026-08-07 pass: the table is entangled with the OPC-UA group taxonomy (gids, role mapping, and the sister-repo consumers of the same directory), so reconciling it means sweeping that taxonomy as one unit rather than patching two rows. |
## Operator actions still pending (from this cycle's runbooks)
These are **live-infrastructure actions the operator must execute** — the repo-side work is complete and merged:
- ~~**SEC-36** — rotate the dev LDAP service-account credential per `docs/runbooks/SEC-36-ldap-credential-rotation.md` (generate new secret in `scadaproj/infra/glauth`, pre-stage the NSSM env var on deployed hosts, rotate GLAuth on `10.100.0.35`, verify dashboard login). The committed literal is gone from the working tree but remains recoverable from git history until rotation completes — **rotation is the load-bearing half.**~~ **Executed 2026-08-07**: the `serviceaccount` `passsha256` was replaced in `scadaproj/infra/glauth/config.toml` (commit `aada53b`) and the shared GLAuth recreated on `10.100.0.35`, so the literal recoverable from this repo's history no longer binds `dc=zb,dc=local`. The new value lives only in the GLAuth hash, windev's NSSM environment, and dev user-secrets. `wonder-app-vd03` was out of scope (it binds a different, `dc=scadalink`/`dc=scadabridge` directory). **Caveat, since closed:** windev's dashboard `/login` verification was deferred while that host's gateway was crash-looping on the unrelated stale-deployment fault filed as NEXT-07, so the bind was verified directly by `ldapsearch` as `cn=serviceaccount,dc=zb,dc=local` instead. After the 2026-08-07 redeploy the real check ran on windev (login as `multi-role` → 302 + dashboard cookie, anonymous control → `/login`), so the rotated credential is now proven through the `DashboardAuthenticator` path itself; see `docs/runbooks/SEC-36-ldap-credential-rotation.md` Correction 3. SEC-36 is fully `Done`.
- ~~**TST-30** — register a second Gitea `act_runner` on `10.100.0.35` per `docs/runbooks/TST-30-second-ci-runner.md` to relieve the single-shared-runner bottleneck.~~ **Executed 2026-08-07**: `gitea-runner-2` (id 5, capacity 2) is online on `10.100.0.35` via the `/opt/gitea` compose stack, same `container.network: traefik`, token from a `0600` file mount; the existing runner (id 1, capacity 4) was untouched. Concurrency verified — jobs from three runs ran simultaneously across both runners, and a `gitea-runner-2` job cloned successfully from `http://gitea:3000`. TST-30 is now fully `Done`.
- **TST-30 follow-up — reset the Gitea instance runner registration token.** Runner-1's compose block was moved to the same `0600` file-mount pattern as runner-2 on 2026-08-07 (compose and both backups now `0600 root:root`, runner-1 recreated with its identity intact), but hygiene alone does not retire the token: both runners share **one instance-scope registration token** that was world-readable for roughly five months and is still live — a probe registered runner id 6 with it, then deleted it. Gitea 1.26.4 exposes no rotation via CLI or API (both paths are get-or-create and hand back the same value), so the reset must be done in the admin web UI ("Reset registration token"). Afterwards, refresh `/opt/gitea/runner_token` on `10.100.0.35` and shred the two token-bearing compose backups — they are the last copies of the old value. See `docs/runbooks/TST-30-second-ci-runner.md`.
- **TST-25 follow-ups** — old **TST-05** (scheduled live-MXAccess smoke) is now covered by the `nightly-windev` job; old **TST-24** (client wire tests in CI) is unblocked by the working Windows tier.
+117
View File
@@ -0,0 +1,117 @@
# MxAccessGateway — Architecture Review, Cycle 3
Date: 2026-08-19. Branch: `main` at `8b3e3b3` (clean; ~288 commits since the cycle-2 baseline `4f5371f`). Working tree: macOS, **static review only** — no builds, no test runs, no source changes, no remote hosts (one sanctioned read-only script, `regen-readme.py --check`, was executed by the testing reviewer).
## Scope and method
Third run of the review series ([cycle 1](../00-overall.md) 2026-07-08, [cycle 2](../2026-07-12/00-overall.md) 2026-07-12). Same shape: six domain reviews, each with three passes — (1) verify every cycle-2 finding's tracked status (`../2026-07-12/remediation/00-tracking.md`, plus the NEXT-xx candidate rows in `90-candidate-findings-next-cycle.md`) against code read at HEAD, (2) deep-review the remediation code itself and everything else landed in the period, (3) a fresh sweep. Every claim in the domain reports carries a `file:line` citation read at HEAD.
| Report | Domain |
|---|---|
| [10-gateway-core.md](10-gateway-core.md) | Gateway server: sessions, worker lifecycle, IPC client, gRPC streaming, alarm monitor, GalaxyRepository package adoption |
| [20-worker.md](20-worker.md) | Worker process: STA pump, COM lifetime, pipe session, event queue, write-completion correlation, subtag failover |
| [30-contracts-ipc.md](30-contracts-ipc.md) | Protos, frame protocol (both sides), codegen/descriptor pipeline, launch-environment contract |
| [40-security-dashboard.md](40-security-dashboard.md) | API-key auth + failure limiter, LDAP dashboard, tag ACL, secrets store, SignalR hubs, redaction |
| [50-clients.md](50-clients.md) | .NET, Go, Java, Python, Rust clients; publish pipeline; cross-client parity |
| [60-testing-docs-gaps.md](60-testing-docs-gaps.md) | Test architecture, CI (incl. the windev tier), documentation currency, repo hygiene |
The period under review is the largest yet: the full cycle-2 remediation train (47 findings), the SEC-25 tag-ACL epic, the secrets-store adoption, the alarm truncation / `snapshot_status` rollout, the OnWriteComplete correlation feature, a worker perf wave, the 0.2.0/0.2.1 client publish, the windev CI tier, and the fca978d marker-sweep restoration.
## Overall verdict
**The cycle-2 remediation held completely.** All 47 cycle-2 findings tracked `Done` were verified present, sound, and tested at HEAD — **zero false `Done` claims, zero regressions in previously verified fixes, across all six domains.** Four gateway/IPC items carry a caveat in the precise sense of cycle 2 ("fix is correct for the filed defect; a residual failure mode one layer out survives"), each filed as a new Low (GWC-32/34/35, IPC-33). The worker, security, clients, and testing domains reported no caveats at all.
The new top line is **0 Critical, 0 High, 4 Medium, 25 Low, 5 Info** across 34 new findings — down from 0/1/14/28/4 (47) in cycle 2 and 1/12/54/39 in cycle 1. For the first time in the series, **none of the new findings is a correctness defect in the core data path that a client can trigger in default configuration.** The four Mediums are: one deployment-posture gap outside this repo's control (SEC-37, GLAuth has no TLS), one operator action outstanding from a tracked runbook (TST-32, runner-token reset), one version-hygiene repeat (CLI-46, constants equal to already-published artifacts — for the second cycle running), and one stale fixture doc with live-test fallout (TST-31, `glauth.md` credential table).
The residual risk profile is now dominated by **edges of new machinery and process residue**, not product defects: the recency-based write-completion correlation (WRK-32), the fail-open browse scope under a missing ambient identity (GWC-38), the un-tag-scoped alarms hub (SEC-40), the undeclared launch-env-var contract (IPC-36), and a client family that keeps shipping post-publish changes under the published version number (CLI-46).
## Prior-finding verification
| Domain | Cycle-2 Done claims | Verified | Caveats (filed as new) | Cycle-1 leftovers |
|---|:-:|:-:|---|---|
| Gateway core | 7 (GWC-24..30) | 7/7 | GWC-24→**GWC-32**/33, GWC-26→**GWC-35**, GWC-27→**GWC-34** | 13 re-confirmed open, none regressed; GWC-16 recommended Won't-fix (documented deferral) |
| Worker | 8 (WRK-21..28) + NEXT-04/05 | 8/8 + 2/2 | none — WRK-21 fix exceeded its design (enqueue-time sizing) | WRK-01/04/06 re-verified intact |
| Contracts & IPC | 8 (IPC-23..32) | 8/8 | IPC-23→**IPC-33** | IPC-08 substantially resolved by `7c1ea12` (recommend closing; residual = IPC-34); IPC-10/11/12 open; IPC-31 superseded by GWC-28 (flip to Resolved) |
| Security & dashboard | 6 (SEC-31..36) | 6/6 | none; SEC-36 includes the executed operator rotation | SEC-19/21/26 resolved in the period; SEC-13 impact reduced; SEC-24 narrowed; 13/14/15(accepted)/16/17/18/23/27 open |
| Clients | 11 (CLI-35..45) + NEXT-02 | 11/11 + 1/1 | none — including the halves prior cycles skipped (CLI-21 tag guard, CLI-02/42 docs, CLI-26 drift test) | CLI-05/08 partly, CLI-09/13/17/32 divergence table updated |
| Testing, docs & gaps | 6 (TST-25..30) | 6/6 | TST-30 carries one tracked operator residual (→ **TST-32**) | TST-06 materially narrowed (tracker stale); TST-07/09/14 partially addressed; TST-17 row needs refresh; TST-05/10/18–22 open |
**Tracker housekeeping recommended:** close GWC-16 (Won't-fix, recorded), IPC-08 (resolved, residual → IPC-34), IPC-31 (superseded by GWC-28); refresh TST-06 and TST-17 rows; note that `e913dab` burned **WRK-31** (and possibly 29/30) as interim IDs and the 2026-08-15 perf pass stamped unregistered **GWC-3x** labels into code comments (GWC-31) — the register must stay the sole ID authority.
## Severity roll-up (new findings only)
| Domain | Critical | High | Medium | Low | Info | IDs |
|---|:-:|:-:|:-:|:-:|:-:|---|
| Gateway core | — | — | — | 6 | 2 | GWC-31..38 |
| Worker | — | — | — | 3 | — | WRK-32..34 |
| Contracts & IPC | — | — | — | 5 | 2 | IPC-33..39 |
| Security & dashboard | — | — | 1 | 3 | 1 | SEC-37..41 |
| Clients | — | — | 1 | 4 | — | CLI-46..50 |
| Testing, docs & gaps | — | — | 2 | 4 | — | TST-31..36 |
| **Total** | **0** | **0** | **4** | **25** | **5** | **34** |
(Cycle 2: 0 Critical, 1 High, 14 Medium, 28 Low, 4 Info. Cycle 1: 1 Critical, 12 High, 54 Medium, 39 Low.)
## Headline findings
### Medium
- **SEC-37 — the Production plaintext-LDAP hard-stop is unsatisfiable.** The shared GLAuth (`10.100.0.35:3893`) offers no TLS, so every host in the family that authenticates against it cannot run with `DOTNET_ENVIRONMENT=Production` — which is why the deployed service runs as `Staging`. Promotion of NEXT-08; the fix is family-wide (GLAuth TLS in `scadaproj/infra/glauth/`), not in this repo. ([40](40-security-dashboard.md))
- **CLI-46 — version constants again equal the already-published 0.2.0/0.2.1** while post-publish changes shipped — including the fix for the published-broken Python `connect()` found by TST-24. Same shape as cycle-2's CLI-39; the publish-pipeline guards (registry-collision hard-stop) will now *block* the next publish rather than ship a duplicate, but the bump discipline still isn't happening at change time. ([50](50-clients.md))
- **TST-31 — `glauth.md`'s user table contradicts the directory**, and the wrong credential propagated into the live-LDAP test surface (promotes NEXT-10). ([60](60-testing-docs-gaps.md))
- **TST-32 — the Gitea runner registration token (world-readable ~5 months, proven live) still awaits its web-UI reset** — the one operator residual of the otherwise-complete TST-30. ([60](60-testing-docs-gaps.md))
### Low — items worth a second look
- **WRK-32 — write-completion correlation is recency-based per handle**: a late `OnWriteComplete` from an earlier write can be reported as the current write's outcome. Parity-clean otherwise (no synthesized events), but the docs should state the attribution rule. ([20](20-worker.md))
- **GWC-38 — `GatewayBrowseScopeProvider` fails open** to the full Galaxy hierarchy when the ambient identity is absent, resting a scoping control on an undeclared interceptor-ordering invariant. ([10](10-gateway-core.md))
- **SEC-40 / SEC-41 — tag ACL edges**: the session-less alarms hub broadcasts galaxy-wide alarm metadata to every Viewer including empty-grant and anonymous-localhost principals; revoked `GroupToTag` grants survive on cookie principals for up to the 8 h sliding life. ([40](40-security-dashboard.md))
- **SEC-38 — limiter aggregate sidestep**: pre-filling the per-peer key-id cap (SEC-32) routes later failures to the fallback partition, which doesn't feed the cross-peer aggregate, restoring the `limit × addresses` guessing bound. ([40](40-security-dashboard.md))
- **IPC-33 — DrainEvents byte budget is sized against the pipe max, not gRPC `MaxSendMessageSize`**; with a legal larger `Worker:MaxMessageBytes` the reply crosses the pipe and fails at the gRPC send with events already dequeued — the cycle-2 "no event loss" invariant violated one boundary later. ([30](30-contracts-ipc.md), [20](20-worker.md))
- **IPC-34 — `WorkerCancel` fires only on the gateway's own timeout**; a client deadline/disconnect leaves the command on the STA queue (the IPC-08 residual). ([30](30-contracts-ipc.md))
- **IPC-36 — six launch env-vars form an undeclared second gateway↔worker contract**, four new since baseline, duplicated as literals on both sides, two documented. ([30](30-contracts-ipc.md))
- **GWC-37 — the subtag watch-list is resolved once per monitor lifecycle**; a Galaxy-Repository outage at startup empties the failover's coverage for the session's life. ([10](10-gateway-core.md))
- **GWC-32 / GWC-34 — dispose races**: `WorkerClient.DisposeAsync` never enters a terminal state (the GWC-24 overflow fault can misfire); `EnsureDistributorCreated` recreates a pump-less distributor after teardown and a racing attach hangs silently. ([10](10-gateway-core.md))
- **CLI-47/48/50 — alarm-feed CLI divergence**: `snapshot_status` rendering differs per language, the new renderer arms are untested in three CLIs (Go has no `stream-alarms` test at all), and the Python CLI's default `--max-messages 1` returns only the provider-status preamble so the README example fetches no alarm data. ([50](50-clients.md))
- **TST-35 — the Windows-host run of the *gateway* (net10) suite is still manual-only**, and its failure class recurred three times in August; TST-34 — the code-reviews README check is wired into no CI job. ([60](60-testing-docs-gaps.md))
## Cross-cutting themes
1. **"One layer out" is now the only residual pattern in the core.** Every caveated fix this cycle (GWC-24/26/27 → GWC-32/34/35, IPC-23 → IPC-33) is the cycle-2 pattern: the invariant holds at the boundary that was filed and leaks at the next one (pipe max vs gRPC max; shutdown state vs dispose state; unacked→acked vs acked→unacked). The fix shape remains the same advice as last cycle: specify against the invariant, not the symptom — but the blast radius has shrunk from "session-fatal" to "misdiagnosed fault / silent hang in a race".
2. **The tag-ACL epic is sound at its two designed seams and leaky at the one it didn't name.** Events hub and session-details subscribe are pinned with decision-table tests and live-LDAP coverage; the alarms hub (SEC-40) was never in scope, the cookie lifetime (SEC-41) wasn't either, and the browse scope (GWC-38) fails open. A one-page "which surfaces are tag-scoped and what principal each trusts" table would close all three or document them as accepted.
3. **Version bumps are still not part of the change, for the second cycle** (CLI-39 → CLI-46). The pipeline guards now make this a blocked publish instead of a duplicate publish, which is the right failure — but the process fix is a "did any `clients/**` source change since the last publish tag" check at PR time, not at pack time.
4. **Second contracts exist outside the proto.** The launch env-vars (IPC-36) and the GalaxyRepository wire-identity (IPC-37, still unguarded though now testable) are both gateway↔peer agreements with no schema, no test, and partial docs. The codegen guard is now excellent for everything *in* the protos; these are what's left.
5. **ID discipline slipped in the period.** Interim WRK-31 and unregistered GWC-3x labels in code comments (GWC-31), plus stale tracker rows (TST-06/17, IPC-08/31, GWC-16) — cheap to fix, and the register is the only thing that makes cycle-over-cycle verification possible.
6. **Documentation currency is high and the one big wound healed.** The fca978d sweep's collateral damage was systematically restored (816 lines judged, 17 restorations), every currency spot-check matched code, and the remaining doc findings (TST-31/33/36, GWC-36, IPC-38) are inherited debris rather than new drift.
## Prioritized roadmap
**P0 — small, do now**
1. CLI-46: bump all five client versions past 0.2.0/0.2.1 before anything else touches `clients/`; add the PR-time "source changed since publish tag" check.
2. TST-32: reset the Gitea runner registration token (web UI); TST-31: fix `glauth.md` and the propagated live-test credential.
3. Tracker housekeeping: close GWC-16/IPC-08/IPC-31, refresh TST-06/TST-17, register WRK-31 and retire the GWC-3x comment labels (GWC-31).
**P1 — hardening, each a contained change**
4. Tag-ACL completeness: SEC-40 (scope the alarms hub or document it as galaxy-wide-by-design), SEC-41 (re-resolve tag claims on restart / shorten validity), GWC-38 (fail closed when identity is absent and assert interceptor order).
5. IPC-33 + IPC-39: budget DrainEvents against `min(pipe max, gRPC send max)`; fold the sequence-stamp window into the pre-check.
6. IPC-34: forward `WorkerCancel` on caller cancellation, not only gateway timeout; IPC-35: surface handshake `WorkerFault`s structurally.
7. WRK-32: document (or tighten) the recency-based completion attribution; WRK-33: give the discarded command-task observer its fault-observing continuation.
8. GWC-32/34: terminal-state on dispose; refuse `EnsureDistributorCreated` after disposal. GWC-37: re-resolve the subtag watch-list on a timer or on Galaxy-Repository recovery.
9. SEC-37: family-wide — add TLS to the shared GLAuth (`scadaproj/infra/glauth/`), then move the deployed service to `Production`.
**P2 — the rest**
10. Client CLI parity batch (CLI-47/48/49/50), IPC-36 launch-env contract (single shared constants class + doc table), IPC-37 wire-identity test, TST-34/35 CI wiring, doc debris (TST-33/36, GWC-36, IPC-38, SEC-39 comment), alongside the unchanged cycle-1 backlog.
## What is demonstrably good
Everything cycle 2 asked to preserve is intact: STA pump and COM teardown discipline, write-time sequence stamping (now peek-stamp-commit on the worker, GWC-28 on the gateway), negotiated frame maxima with startup cross-validation, additive-only proto evolution with every generated tree regenerated in the same commit as the proto change, fail-closed auth with constant-time compare, owner-scoped `StreamEvents`, and the layered fake-worker test architecture.
New this period and worth protecting under change:
- **The windev CI tier** — per-push x86 worker build + Worker.Tests and a nightly live-MXAccess run, hardened by its own acceptance checks (base64-stored key, `MXGW_CI_LOCK_HELD` handoff, SHA shape validation, fail-red on unreachable host), and the full-suite testhost hang root-caused to a zero-buffer pipe and structurally fixed (windev baseline = macOS = 879).
- **Byte-budgeted DrainEvents with enqueue-time sizing** — the cycle-2 headline defect fixed beyond its design; the IPC-33 residual is a configuration edge, not the default path.
- **The failure limiter's composite partition + aggregate + probe admission** — the cycle-2 lockout-DoS is closed; SEC-38/39 are edge refinements.
- **Fixtures-first client conformance** — ten shared command-reply fixtures consumed by all five suites, so HRESULT sign, category-verdict, scrub, and malformed-reply rules are pinned in bytes; plus real-wire tests in all five clients (which immediately found the published-broken Python `connect()`).
- **Three-seam value redaction** (events mirror, alarms hub, `/browse`), clone-only, with the config doc telling the truth about all three.
- **The codegen guard** — descriptor set, `Contracts/Generated`, Rust vendored protos in both directions, Go/Python bindings with pinned generators, Java aggregates via the java job's diff gate, and Check 4 now Windows-runnable.
- **Per-correlation `CommandTooLarge` → `ResourceExhausted`** remains the model for fault isolation the residual IPC-33/34/35 items should copy.
+144
View File
@@ -0,0 +1,144 @@
# Gateway Server Core — Cycle-3 Review
- **Date:** 2026-08-19
- **Commit:** `8b3e3b3` (`main`, clean)
- **Baseline:** `4f5371f` (2026-07-12 cycle-2 review; ~288 commits since, ~99 of them touching `src/ZB.MOM.WW.MxGateway.Server`)
- **Method:** static review on macOS; every cited file read at HEAD. No build, no test, no source change.
- **Scope:** gateway server process — `Sessions/`, `Workers/`, `Grpc/` (`MxAccessGatewayService`, `EventStreamService`, mapper, validator), `Alarms/` (monitor, watch-list resolver, subtag-fallback gateway half), the session/alarm-feed plumbing the dashboard consumes (`DashboardLiveDataService` session use, `AlarmsHubPublisher` feed consumption), and the `ZB.MOM.WW.GalaxyRepository` package adoption (`AddZbGalaxyRepository`/`MapZbGalaxyRepository` wiring, `GatewayBrowseScopeProvider`, `DashboardGalaxySummaryProjector`). Excludes `Security/` internals and dashboard UI (security-dashboard reviewer), except the browse-scope provider assigned here.
- **Prior reports:** `archreview/2026-07-12/10-gateway-core.md`; remediation status source `archreview/2026-07-12/remediation/00-tracking.md` (+ `10-gateway-core.md`); first-cycle backlog `archreview/remediation/00-tracking.md`.
## Prior-finding verification
### Cycle-2 findings (GWC-24..30, all tracked Done)
Every Done claim was verified against code read at HEAD. **All 7 hold; 3 are verified with caveats** — in each case the fix is present and sound for the reported defect, but a residual failure mode one layer out survives and is filed as a new finding.
| ID | Claimed | Verified? | Evidence (HEAD) | Notes |
|----|---------|-----------|-----------------|-------|
| GWC-24 | Done | ✅ holds, **with caveat** | `Workers/WorkerClient.cs:107-119` (staging channel bounded at `checked(2 * EventChannelCapacity)`, Wait+TryWrite idiom documented), `:646-676` (`StageWorkerEvent` faults `ProtocolViolation` on a rejected `TryWrite`, message names the bound, the depth, and the actionable fix), `:653-658`/`:350-359` (single `_eventQueueDepth` counter incremented at staging, decremented at consumer read — total undelivered depth), `:120-121` + `Metrics/GatewayMetrics.cs:524` (pull-model gauge registration), `:44-48` (field comment states bound + fault + gauge coverage); startup overflow guard `Configuration/GatewayOptionsValidator.cs:343-357` (`QueueCapacity ≤ int.MaxValue / 2` so the 2× derivation cannot overflow); regression tests `Tests/Gateway/Workers/WorkerClientTests.cs:781` (`StagingChannelOverflowFaultsWorkerWithoutWaitingForFullModeTimeout`), `:865` (`WorkerEventQueueDepthGaugeCountsStagedEvents`) | Implemented exactly per the remediation design, including the shutdown-drop guard (`IsTerminalState()`), and the design's extra validator guard. Caveat: the shutdown-drop guard does not cover `DisposeAsync`, which completes the channels **without** entering a terminal state — a dispose-race `TryWrite` failure is then misdiagnosed as the backpressure fault → **GWC-32**. A faulted-but-retained client also keeps contributing stale depth to the gauge until disposal → **GWC-33** (Info). |
| GWC-25 | Done | ✅ holds | `Sessions/SessionEventDistributor.cs:519-535` (empty-ring branch: `oldestAvailableSequence = gap ? _highestSequenceSeen + 1 : 0`, with a full explanatory comment citing the finding), out-param contract updated `:430-440`; emitted verbatim `Grpc/EventStreamService.cs:139-145`, `:227-239`; proto comment amended `Contracts/Protos/mxaccess_gateway.proto:797-798` ("When nothing is retained … `highest observed + 1` … resume formula remains valid"); `docs/Sessions.md:262` (dedicated empty-ring paragraph); tests `Tests/Gateway/Sessions/SessionEventDistributorTests.cs:935` (`ResumeUsingSentinelFormulaAfterEmptyRingGapDeliversLiveEvents`) and `Tests/Gateway/GatewayEndToEndReconnectReplayTests.cs:246` (`ReconnectAfterFullAgeEvictionResumesWithSentinelFormula`) | Sound end to end. The `afterSequence == ulong.MaxValue` wrap guards in both branches (`:521`, `:540`) are preserved, so the old dead-stream input is also inert. Proto + docs + server + tests all landed as one story; the client halves (CLI-35/36, NEXT-02) are closed in the tracking record. |
| GWC-26 | Done | ✅ holds, **with caveat** | `Alarms/GatewayAlarmMonitor.cs:246-258` (internal lease attached **before** `SubscribeAlarmsAsync` + first `ReconcileAsync`, with the window rationale in the comment), `:272-281` (drain the same lease after reconcile; order-safety noted), `ApplyReconcile` acked-delta repair `:674-684` (both-present, `existing.CurrentState != incoming.CurrentState && incoming == ActiveAcked` → `Acknowledge` feed transition); `ISessionManager.ReadAlarmEventsAsync` removed (grep: absent from `Sessions/ISessionManager.cs` and `SessionManager.cs`); repair-scope + at-least-once contract documented `:607-633`; regression tests `Tests/Alarms/GatewayAlarmMonitorAttachOrderTests.cs` | Implemented per design, including the method-removal half and the "never synthesize MxEvents" boundary note (`:610-614`: repairs are `AlarmFeedMessage`s, never `MxEvent`s). Caveat: the state-delta repair is one-directional — only unacked→acked is repaired; an acked alarm silently replaced by a new **unacked** instance is absorbed without a feed transition → **GWC-35**. |
| GWC-27 | Done | ✅ holds, **with caveat** | `Sessions/GatewaySession.cs:590-617` (`AttachInternalEventSubscriber` readiness gate under `_syncRoot` before `EnsureDistributorCreated`, same exception code/message shape as `AttachEventSubscriber:955-961`; the poison mechanism is spelled out in the comment); test `Tests/Gateway/Sessions/GatewaySessionTests.cs:683` (`AttachInternalEventSubscriberBeforeReadyThrowsAndDoesNotPoisonDistributor`) | The premature-attach poison path is closed. Caveat: the disposal-side twin remains — `EnsureDistributorCreated` recreates a never-started distributor after `DisposeAsync` nulls `_eventDistributor`, silently hanging a racing attach → **GWC-34**. |
| GWC-28 | Done | ✅ holds | `Workers/WorkerClient.cs:463-472` (`WriteLoopAsync` stamps `envelope.Sequence = unchecked(++_nextSequence)` immediately before `_writer.WriteAsync`, single-consumer rationale in comment), `:49-51` (field is a plain `ulong` touched only by the write loop, no interlock), `CreateEnvelope:1225-1241` (sequence deliberately unset, comment cites GWC-28/WRK-04); test `Tests/Gateway/Workers/WorkerClientTests.cs:228` (`ConcurrentInvokesEmitStrictlyIncreasingSequencesOnTheWire`) | Exactly the WRK-04-mirror design. All outbound envelopes flow through `_outboundEnvelopes` (hello `:171`, commands `:251`, cancel `:849/:858`, shutdown `:383`), so coverage is total. Note the `InvokeAsync` size pre-check comment (`:236-241`) correctly explains why the computed size cannot be reused as the writer's `knownSize` post-GWC-28 — the interaction was thought through. |
| GWC-29 | Done | ✅ holds | `Grpc/MxAccessGrpcMapper.cs:43-55` (new `MapCommand(MxCommand)` overload; the one required clone kept with the aliasing rationale), `:27-33` (request overload delegates); `Grpc/MxAccessGatewayService.cs:131-136` (`mapper.MapCommand(commandToInvoke)` — the request clone and `invokeRequest` local are gone; `TrackCommandReply` still reads `commandToInvoke` at `:147`, which is why the inner clone must stay and does) | Sound. `WorkerClient.CreateCommandEnvelope`'s no-aliasing invariant comment (`:1194-1198`) remains accurate. |
| GWC-30 | Done | ✅ holds | `Workers/WorkerFrameReader.cs:22-24` (per-reader `_lengthPrefix` scratch buffer), `:46` (used per frame), `:11-16` (class remark documents the non-reentrancy contract) | As designed. (Unrelatedly, the *label* "GWC-30" has since been reused by an off-register perf pass for a different change — see **GWC-31**.) |
### Cycle-1 findings still open at cycle 2 — re-checked at HEAD
All were re-read at HEAD. **None regressed; none silently fixed; two materially changed shape** (GWC-16 is now an explicitly documented deferral; GWC-20's client-side half improved).
| ID | Cycle-2 status | HEAD status | Evidence |
|----|----------------|-------------|----------|
| GWC-05 | Open | **Still open** | `Sessions/SessionWorkerClientFactory.cs:175-185` — pipe created with explicit 128 KiB buffers (the zero-quota fix, `da84635`) but still no ACL and no `PipeOptions.CurrentUserOnly`. The local pipe-squatting exposure and the `gateway.md` pipe-security contract gap are unchanged. |
| GWC-09 | Open | **Still open** | `Workers/WorkerProcessStartedProbe.cs:5-19` (no-op `HasExited` probe throwing `WorkerProcessLaunchException`); `Workers/WorkerProcessLauncher.cs:331-339` (`ShouldRetryStartupProbe` still excludes `WorkerProcessLaunchException`); `Configuration/WorkerOptions.cs:19-22` (both dead retry options remain). The retry pipeline still can never retry. |
| GWC-10 | Open | **Still open** | `Workers/WorkerEnvelopeValidator.cs:15-39` — version/session/body only; no inbound sequence tracking anywhere in `WorkerClient`. (The outbound half was fixed by GWC-28; enforcement remains unimplemented on both sides, consistent with IPC-31's recorded decision.) |
| GWC-11 | Open | **Still open** | `Sessions/GatewaySession.cs:23` (`_workerClient` not volatile); lock-free reads at `:308` (`WorkerProcessId`), `:313` (`WorkerClient`), `:1579` (`CloseAsync`), `:1732`, `:1765`, `:1877` (`DisposeAsync`). |
| GWC-12 | Open | **Still open** | `Workers/WorkerClient.cs:419-426` — plain `if (_disposed) return; _disposed = true;`, no interlock. (Now compounds with new GWC-32: dispose also never enters a terminal state.) |
| GWC-13 | Open | **Still open** | `Sessions/GatewaySession.cs:1956-2024` — 25 ms poll loop, routed through the injected `TimeProvider` (unchanged since cycle 2). |
| GWC-16 | Open | **Still open — now a recorded deferral; recommend closing as Won't-fix** | `Grpc/MxAccessGatewayService.cs:111-115` — a `PERF(followup)` comment now documents the double session lookup as measured-negligible and deliberately not collapsed (the fix would duplicate `InvokeAsync`'s fault mapping at a second entry point). The rationale is sound; the tracker row should be flipped to `Won't fix` citing this comment rather than left `Not started`. |
| GWC-17 | Open | **Still open** | `Sessions/SparseArrayExpander.cs:306` (`RpcException` factory below the gRPC layer), `:75` (GWC-03's cap rejection still flows through the same leak); invocation comment at `Sessions/GatewaySession.cs:1103-1111` still documents the layering leak explicitly. |
| GWC-18 | Open | **Still open** | `Sessions/GatewaySession.cs:14` — no `IAsyncDisposable`; `DisposeAsync` at `:1787`. |
| GWC-19 | Open | **Still open** | `Sessions/GatewaySession.cs:1730-1734` — `KillWorker(string)`; grep across `src/` + `clients/` finds zero callers (one doc-comment mention in `SessionManagerTests.cs:1433`). |
| GWC-20 | Open | **Still open (client half improved)** | `Configuration/WorkerOptions.cs:52` — the config key is still `HeartbeatIntervalSeconds` ("interval … for worker heartbeats") while it is bound to the **check** interval at `Sessions/SessionWorkerClientFactory.cs:98`; the `WorkerClientOptions` field was renamed to `HeartbeatCheckInterval` (`Workers/WorkerClientOptions.cs:40`), which fixes the client-side naming but leaves the config-key conflation. The heartbeat loop still uses raw `Task.Delay` without `TimeProvider` (`Workers/WorkerClient.cs:534`). |
| GWC-21 | Open | **Still open** | `Sessions/SessionWorkerClientFactory.cs:95-101` populates 4 of 6 `WorkerClientOptions`; `EventChannelFullModeTimeout` / `HeartbeatStuckCeiling` remain fixed defaults (`Workers/WorkerClientOptions.cs:13`, `:23`). Post-GWC-24 the staging bound is *derived* from `EventChannelCapacity` (which is configurable), so the exposure gap is narrower than at cycle 2 but the two knobs are still hard-coded. |
| GWC-22 | Open | **Still open** | `Grpc/EventStreamService.cs:217` — `metrics.StreamDisconnected("Detached")` hardcoded in the `finally`; overflow/fault exits are not differentiated. |
| GWC-23 | N/A | Unchanged | Decision stands; multi-subscriber mode remains config-gated and the validator comment survives. |
### NEXT rows touching this domain
- **NEXT-01** (macOS `sun_path` pipe names) — **verified resolved**: `Sessions/SessionManager.cs:532-536` builds `mxgw-{Environment.ProcessId}-{sessionUid}` with the length-budget rationale in the comment; `CLAUDE.md`/`gateway.md` describe the same format.
- **NEXT-03** (reconcile/live duplicate broadcasts) — **verified resolved as designed (best-effort, positive-match-only)**: `Alarms/GatewayAlarmMonitor.cs:483-489` (`IsDuplicateOfCachedState`, timestamp+state marker), `:496-507` (`IsDuplicateOfReconcileClear`, tombstone consumed on match), `:645-664` (tombstones recorded per reconcile generation, cleared at `:649`); at-least-once contract restated at `:616-625`; race-driving regressions in `Tests/Alarms/GatewayAlarmMonitorAttachOrderTests.cs:159`, `:218`. The one-generation tombstone lifetime and the unset-timestamp passthrough match the recorded design; consumers must (and are documented to) treat transitions idempotently.
- NEXT-08 / NEXT-10 (GLAuth TLS posture, glauth.md table) are security/docs-domain rows, not promoted here.
### Also verified in passing (other-domain fixes landing in gateway-core files)
- **WRK-21/WRK-28/IPC-23 gateway half** — `Grpc/MxAccessGrpcRequestValidator.cs:74-88` rejects `DrainEvents.max_events` above the now-shared `GatewayContractInfo.MaxDrainEventsPerCommand` constant, with the byte-cap division of labor documented in the comment.
- **Zero-quota pipe fix** (`da84635`) — `SessionWorkerClientFactory.cs:15-25`, `:175-185`: explicit 128 KiB in/out buffers with the windev-wedge rationale; this was the root cause of the historical full-suite testhost hang.
- **Dead `ISessionManager.ReadEventsAsync` chain removed** (`fa9eb0c`) and `StreamEventsAsync` now consumes the subscriber channel directly (`Grpc/EventStreamService.cs:110-204`, `935f002`) — the WaitToReadAsync/TryRead loop preserves the fault-surface semantics (`:158-204`) including the worker-fault → `MarkFaulted` + metric path.
- **Reply ownership transfer** (`3ff073d`) — `Grpc/MxAccessGrpcMapper.cs:62-85`: single-consumer argument for moving `reply.Reply` is documented and holds (one frame → one `PendingCommand` awaiter → one mapper call; `MergeDeniedInto` splices in place on the owned instance).
- **GalaxyRepository adoption** — `GatewayApplication.cs:151-176`: scope provider registered *before* `AddZbGalaxyRepository` so the library's `TryAddSingleton` null default cannot win (order-sensitive, correctly commented); snapshot-path default seeded pre-bind with `GalaxyRepositoryOptionsValidator` + `ValidateOnStart` (SEC-33); `MapZbGalaxyRepository` at `:351`. `A2-galaxyrepository-adoption-handoff.md`'s caveats are all marked resolved and match the code. One hardening gap filed as **GWC-38**.
- **Exception-total shutdown drain** (`95f8ba9`/`0bc13b5`) — `Sessions/SessionManager.cs:354-427`: bounded-parallel (4) close with an exception-total body, kill fallback on `CancellationToken.None`, and the deliberate token asymmetry against `CloseExpiredLeasesAsync` documented at length; `ISessionManager.ShutdownAsync`'s doc (`ISessionManager.cs:100-110`) states the bounded overrun honestly. Sweep selection stays sequential with the TOCTOU re-check inside `TryBeginCloseIfExpired` (`GatewaySession.cs:1671-1695`) — the cycle-2 close-gate invariants are preserved under the parallelization.
## New findings
IDs continue from the register. **Note on ID allocation:** the codebase already contains comment references to "GWC-31" (and a re-used "GWC-30") coined by an off-register remediation pass — those labels are *not* register entries (see GWC-31 below, which claims the ID and records the collision). Register IDs assigned here are authoritative.
### GWC-31 — The 2026-08-15 perf remediation stamped unregistered "GWC-3x" IDs into code comments, colliding with this register
- **Severity:** Low
- **Category:** Documentation & comments
- **Location:** `Workers/WorkerClient.cs:44` ("GWC-30" = pull-model queue-depth gauge), `:236` ("PERF(GWC-31)"), `:253` ("GWC-31: one pooled timer…"); `Metrics/GatewayMetrics.cs:46`, `:213`, `:284`, `:463`, `:524` ("GWC-30" = gauge pull model); `Tests/Gateway/Workers/WorkerClientTests.cs:163`, `:179` ("GWC-31" = WorkerCancel forwarding); analogous worker-side citations (`Worker/Ipc/WorkerFrameReader.cs`/`WorkerFrameWriter.cs`, per `docs/plans/2026-08-15-perf-review-remediation.md:544-548`)
- **Description:** The 2026-08-15 performance review (whose findings live only in `docs/plans/2026-08-15-*.md`, not in `archreview/`) labeled its changes with GWC-series IDs that were never added to any register. "GWC-30" now means two different things — the archreview register says *frame-reader 4-byte prefix allocation*; the code comments say *pull-model worker queue-depth gauge* — and "GWC-31" exists only in comments. Finding IDs are the repo's cross-reference currency ("cite it in commits, branches, and PRs" — remediation tracker); a reader following a code comment to the register lands on the wrong finding, and future review cycles (this one included) must dodge already-burned numbers. REVIEW-PROCESS.md's "never reused" rule is violated in spirit.
- **Recommendation:** Sweep the perf-pass citations to a distinct prefix (e.g. `PERF-nn` keyed to `docs/plans/2026-08-15-perf-review-remediation.md` task numbers) or append the perf findings to the register under fresh, unclaimed IDs and fix the comments to match. Add a line to REVIEW-PROCESS.md stating that GWC/WRK/IPC/SEC/CLI/TST prefixes are reserved for the archreview registers.
### GWC-32 — `WorkerClient.DisposeAsync` never enters a terminal state, so the GWC-24 staging-overflow fault can misfire during a dispose race and the "already terminal" shutdown-drop claim is not guaranteed
- **Severity:** Low
- **Category:** Concurrency & thread safety (residual of GWC-24)
- **Location:** `Workers/WorkerClient.cs:419-434` (`DisposeAsync` sets `_disposed`, cancels, completes all three channels — but `_state` is untouched), `:661-676` (`StageWorkerEvent` fallback: `TryWrite` false → `if (IsTerminalState()) return;` → otherwise `ProtocolViolation` fault), `:632-644` (XML doc: a completed staging channel is "safely dropped because the client is already terminal")
- **Description:** `MarkClosed` and `SetFaulted` both flip `_state` to a terminal value *before* completing the channels, so the `IsTerminalState()` guard in `StageWorkerEvent` correctly classifies a completed-channel `TryWrite` miss as shutdown. `DisposeAsync` is the third channel-completing path and does not: a client disposed while still `Ready` keeps reporting `IsTerminalState() == false`. If the read loop is inside `DispatchEnvelope` when `DisposeAsync` completes `_eventStaging` (the `_stopCts.Cancel()` only interrupts the *next* `ReadAsync`), the in-flight `StageWorkerEvent` takes the overflow arm: a spurious `ProtocolViolation` session fault with the misleading "consumer is draining slower than the worker produces" message, a `worker-event-staging` overflow metric, a redundant `KillOwnedProcess`, and a `WorkerStopped("ProtocolViolation")` metric — all during an intentional teardown. Reachability is narrow: in the normal teardown order (`CloseAsync`/`Kill` → `DisposeAsync`) the state is already terminal; the exposed path is disposing a still-Ready client, which `SessionManager.OpenSessionAsync`'s failure arm does (`Sessions/SessionManager.cs:133-140` — `MarkFaulted` faults the *session*, not the worker client) and `SessionWorkerClientFactory`'s catch avoids only because it calls `Kill` first (`SessionWorkerClientFactory.cs:120-132`). Impact is corrupted diagnostics/metrics, not data loss — but the fault fires on exactly the code path an operator investigates after a failed open, pointing them at backpressure that never happened. It also makes the `StageWorkerEvent` XML doc's "already terminal" claim false on one of the three completion paths.
- **Recommendation:** Treat disposal as terminal in the guard — `if (_disposed || IsTerminalState()) return;` — or have `DisposeAsync` transition `_state` to `Closed` under `_syncRoot` before completing the channels (which would also give GWC-12's double-dispose interlock a natural home). Update the `StageWorkerEvent` doc to name all three completion paths.
### GWC-33 — A faulted-but-retained worker client keeps contributing its stale undelivered-event depth to the pull-model gauge until session disposal
- **Severity:** Info
- **Category:** Performance & resource management (observability)
- **Location:** `Workers/WorkerClient.cs:120-121` (gauge source registered at construction), `:426-429` (unregistered only in `DisposeAsync`), `:1011-1039` (`SetFaulted` completes channels but leaves staged+queued events uncounted-down); retention window: `Configuration/SessionOptions.cs` `FaultedGraceSeconds` + `Sessions/SessionManager.cs:283-289` (faulted sessions swept only when reapable)
- **Description:** The GWC-24 gauge counts staged + queued events and is decremented only on consumer read. When a client faults, its channels are completed with the fault and never drained, so `_eventQueueDepth` freezes at whatever was undelivered. The registration is removed only in `DisposeAsync`; with a positive `FaultedGraceSeconds` (deliberately configurable so operators can observe faults before the reap) the dead client's frozen depth inflates the aggregate `worker event queue depth` gauge for the whole grace window. With the default `FaultedGraceSeconds = 0` the window is one sweep interval, so the default posture is fine — recorded so the next metrics pass knows it was seen.
- **Recommendation:** Optional: dispose the gauge registration inside `SetFaulted`/`MarkClosed` (the comment at `:427-428` already articulates the principle: "the sum must not keep counting a client that is going away").
### GWC-34 — `EnsureDistributorCreated` recreates a never-started distributor after `DisposeAsync`; an attach that races a session teardown registers on a pump-less distributor and hangs silently
- **Severity:** Low
- **Category:** Concurrency & thread safety (residual of GWC-27 — the disposal-side twin)
- **Location:** `Sessions/GatewaySession.cs:1865-1875` (`DisposeAsync` nulls `_eventDistributor`; `_eventDistributorStarted` is never reset — `:41`, `:554-559`), `:535-563` (`EnsureDistributorCreated`: `_eventDistributor is null` → constructs a fresh distributor; `startNow` is false because the started flag is already latched), `:946-998` (`AttachEventSubscriber`: readiness gate under `_syncRoot`, then the distributor calls run *outside* the lock)
- **Description:** GWC-27 closed the premature-attach poison; the mirror-image hole remains open on the teardown side. An `AttachEventSubscriber` (or `AttachEventSubscriberWithReplay` / `AttachInternalEventSubscriber`) that passes its readiness gate while the session is still `Ready`, then loses the CPU until `CloseAsync` + `DisposeAsync` complete, reaches `EnsureDistributorCreated` after `_eventDistributor` was nulled: a **new** distributor is constructed, `startNow` comes back `false` (the latch is one-way), the subscriber registers on a distributor whose pump will never start and whose channel will never be completed — `WaitToReadAsync` in `EventStreamService.cs:167` then blocks until the client's own deadline/cancel. No error, no log, no fault; the replacement distributor is also never disposed. Contrast the adjacent race that *is* handled: if disposal instead wins between `EnsureDistributorCreated` returning the **old** distributor and `Register`, the old distributor's disposed check throws `ObjectDisposedException` loudly and the attach rolls back (`RegisterSubscriber`, `SessionEventDistributor.cs:361-386`). Reachability is small (the attach must stall across an entire multi-second close + dispose between two adjacent statements), but the failure mode is the worst kind — silent, per-stream, and invisible to metrics (the backlog registration reports 0 forever). This is the same "one distributor per session, never reset" lifecycle assumption GWC-27 documented, violated from the other end.
- **Recommendation:** Make the latch symmetric with the null-out: in `DisposeAsync`, set a `_eventDistributorDisposed` flag (or reuse a session-terminal check) under `_syncRoot` when nulling `_eventDistributor`, and have `EnsureDistributorCreated` throw `SessionManagerException(SessionNotReady)` instead of re-creating when it is set. One flag, one throw — the attach paths' existing catch/rollback already handles the exception.
### GWC-35 — `ApplyReconcile`'s state-delta repair is one-directional: an acked alarm replaced by a new unacked instance (or any acked→active regression) is absorbed silently by the snapshot replace
- **Severity:** Low
- **Category:** Correctness & logic (alarm feed completeness; residual of GWC-26)
- **Location:** `Alarms/GatewayAlarmMonitor.cs:666-691` (`ApplyReconcile` both-present loop: broadcasts only when `incoming.Value.CurrentState == AlarmConditionState.ActiveAcked`), snapshot replace at `:687-691`
- **Description:** The GWC-26 repair covers three legs — missing→present (`Raise`), present→missing (`Clear`), and unacked→acked (`Acknowledge`). The fourth leg is absent: an alarm the cache holds as `ActiveAcked` that the worker snapshot now reports as `Active` (the acked instance cleared and re-raised as a new unacked instance while the live `Clear`+`Raise` transitions were lost — e.g. the internal alarm lease overflowed and disconnected, or worker-side event loss) is absorbed by the `_alarms` replace with no feed transition. `CurrentAlarms`/`QueryActiveAlarms` are correct (they read the replaced cache), but live `StreamAlarms`/dashboard subscribers keep rendering the alarm as acknowledged until its next genuine transition — for a re-raised alarm, precisely the state an operator must not misread. Reachability is much lower than the pre-GWC-26 window (a lease overflow restarts the whole monitor lifecycle, which re-snapshots every subscriber), so this is a completeness gap in the defense-in-depth rather than a live bug.
- **Recommendation:** Generalize the both-present delta to any `CurrentState` change: broadcast `Acknowledge` for `→ ActiveAcked` and `Raise` (with the incoming snapshot's new instance identity) for `→ Active`. The existing `TransitionFromSnapshot` already carries the timestamps the NEXT-03 dedup needs to suppress the live duplicates.
### GWC-36 — `EventSubscriberReplayAttachment.OldestAvailableSequence` doc still describes the pre-GWC-25 empty-ring semantics
- **Severity:** Info
- **Category:** Documentation & comments
- **Location:** `Sessions/EventSubscriberReplayAttachment.cs:26-30` ("The oldest worker sequence still retained and replayable; `0` when nothing is retained.")
- **Description:** GWC-25 changed the empty-ring-with-gap value to `highest observed + 1`; the distributor's out-param doc (`SessionEventDistributor.cs:430-440`), the proto, and `docs/Sessions.md` were all updated, but this record's XML doc was missed. "`0` when nothing is retained" is now exactly the value the fix exists to avoid — a reader implementing against this doc re-introduces the dead-stream formula.
- **Recommendation:** Mirror the distributor's out-param wording (retained-oldest, or `highest observed + 1` when the ring is empty; `0` only when `Gap` is false).
### GWC-37 — The alarm subtag watch-list is resolved once per monitor lifecycle: a Galaxy-Repository outage at startup silently empties the subtag failover's coverage for the session's entire lifetime
- **Severity:** Low
- **Category:** Error handling & resilience (alarm subtag fallback)
- **Location:** `Alarms/GatewayAlarmMonitor.cs:330-357` (`SubscribeAlarmsAsync` resolves the watch list once and sends it in the `SubscribeAlarms` command); `Alarms/AlarmWatchListResolver.cs:54-83` (GR discovery failure → warn and continue with a config-only, typically empty set), `:141-145` (zero-target warning); no re-resolve anywhere — `ApplyProviderModeChangeAsync` (`GatewayAlarmMonitor.cs:513-566`) triggers only a reconcile on failover
- **Description:** The subtag fallback's coverage is fixed at `SubscribeAlarms` time: the resolved `AlarmSubtagTarget` list travels once to the worker and is never refreshed while the monitor session lives. Discovery failure is deliberately non-fatal (`ResolveAsync` catches, warns, and returns whatever config contributes) — reasonable for the resolver, but the monitor then subscribes with an empty watch list and runs indefinitely. In `Auto` mode the alarmmgr primary works fine, so nothing looks wrong until the moment a failover to subtag fires — at which point the degraded provider covers zero alarms, the feed goes quiet, and the only evidence is a single `LogWarning` emitted possibly weeks earlier at startup. The same staleness applies to Galaxy objects added after monitor start (a documented design property), but the outage-at-startup case differs in kind: the operator configured discovery, it silently produced nothing, and the safety net the feature exists to provide is empty. The monitor's supervisor restarts (and thus re-resolves) only when the session faults — a healthy alarmmgr session never does.
- **Recommendation:** Re-resolve and re-subscribe on the failover edge (`ApplyProviderModeChangeAsync` when `toMode == Subtag`), or treat "GR discovery failed AND the effective mode can need the watch list (Auto/ForceSubtag) AND zero targets resolved" as a lifecycle failure so the supervisor's backoff loop retries resolution instead of latching the empty set. At minimum, surface the empty-watch-list condition on the provider status (`Degraded`/`Reason`) rather than only in a startup log line.
### GWC-38 — `GatewayBrowseScopeProvider` fails open to the full Galaxy hierarchy when the ambient identity is absent, resting the browse-scoping control on an undeclared interceptor-ordering invariant
- **Severity:** Low
- **Category:** Security (defense-in-depth; GalaxyRepository adoption)
- **Location:** `Security/Authorization/GatewayBrowseScopeProvider.cs:14-25` (`identityAccessor.Current?.EffectiveConstraints ?? ApiKeyConstraints.Empty` → empty `BrowseSubtrees` → library treats as "no scoping"); registration-order dependency documented at `GatewayApplication.cs:151-154`
- **Description:** The per-key browse-subtree scoping — the one piece of authorization mxaccessgw adds on top of the shared GalaxyRepository package — resolves to *unscoped* (full hierarchy) whenever `IGatewayRequestIdentityAccessor.Current` is null. The code's own comment says this is "safe only because the global interceptor has already authenticated and authorized the request first. Do not reorder the interceptor without revisiting this." That is a correct description of today's wiring and an honest warning, but it makes the control's fail state *open*: any future path that reaches the library's galaxy service without the interceptor populating the ambient identity (a new endpoint mapping, an interceptor scoped per-service instead of globally, a library upgrade changing pipeline order) silently grants every scoped key the full hierarchy, with nothing failing and nothing logged. Scoping is a constraint-narrowing control, so its absence is invisible in normal operation. (Whether identity-absent is legitimately reachable under `Authentication:Mode=Disabled` is the security reviewer's domain; this finding is about the fail-open shape itself.)
- **Recommendation:** Distinguish "authenticated with no BrowseSubtrees constraint" (legitimately unscoped) from "no ambient identity" (invariant broken): when `Current` is null *and* API-key authentication is enabled, throw `RpcException(PermissionDenied)` (or log-and-deny) instead of returning unscoped. One branch converts a silent authorization bypass into a loud misconfiguration signal.
## Severity roll-up (new findings)
| Severity | Count | IDs |
|----------|:-:|-----|
| Critical | 0 | — |
| High | 0 | — |
| Medium | 0 | — |
| Low | 6 | GWC-31, GWC-32, GWC-34, GWC-35, GWC-37, GWC-38 |
| Info | 2 | GWC-33, GWC-36 |
Prior-finding verification: 7/7 cycle-2 Done claims hold (3 with caveats filed as GWC-32/34/35); 13 still-open cycle-1 findings re-confirmed open with none regressed; GWC-16 recommended for closure as a recorded Won't-fix.
## Still good / preserve under change
The cycle-2 remediation wave and the subsequent perf pass landed without breaking the core invariants, and several of them got structurally *stronger*. The backpressure topology is now bounded end to end again — worker queue → pipe → bounded staging (2×) → bounded consumer channel → bounded per-subscriber channels — with a loud fail-fast fault at every seam and a single observable depth gauge across the first two gateway stages; the GWC-24 implementation even added the startup overflow guard the design didn't ask for. The replay→live handoff argument is intact after the GWC-25 change, and the empty-ring sentinel arithmetic is documented in four places that all agree (distributor, proto, docs/Sessions.md, e2e test). The GWC-28 write-loop stamping is the textbook fix, and the `InvokeAsync` size-pre-check comment proves the interaction with it was reasoned about rather than stumbled past. The alarm monitor's attach-before-subscribe plus the NEXT-03 positive-match dedup give the alarm feed a defensible, *stated* delivery contract (at-least-once, idempotent-state), and the reconcile repairs are correctly quarantined to the feed layer — the "never synthesize events" rule for `MxEvent`s is respected everywhere, with the ReplayGap sentinel still the sole, documented control signal. Session state remains single-lock disciplined through the new detach-grace/faulted-grace/close-gate arms; the parallelized sweep and shutdown preserved the TOCTOU re-check and made the shutdown body exception-total with an honestly documented token asymmetry. The GalaxyRepository adoption is clean consumer wiring: scope-provider-before-package registration order is commented, the snapshot path is validated at startup, and the host-side projector is a pure function memoizable on the cache sequence. Comment quality across the new code is exceptional — most of this cycle's findings were *found via* comments that honestly documented their own residual assumptions ("safe only because…", "already terminal") — preserve that habit; it is what keeps one-layer-out defects findable. `GatewaySession` has grown again (2,173 lines) and the long-standing split-out recommendation stands.
+108
View File
@@ -0,0 +1,108 @@
# Worker Process — Architecture Review (cycle 3)
## Scope & method
Date 2026-08-19 · commit `8b3e3b3` (`main`, clean) · macOS static review — the worker targets .NET Framework 4.8 x86 and was **not built or tested** here; every claim cites file:line evidence read directly from the working tree at HEAD. Scope: `src/ZB.MOM.WW.MxGateway.Worker` (STA runtime/pump, `MxAccessSession`/`MxAccessStaSession` COM lifetime, `Ipc/` pipe session + frame reader/writer, event queue and DrainEvents, value/write-completion caches, alarm consumers incl. the subtag failover, watchdog/heartbeat, program lifecycle/exit codes) and `src/ZB.MOM.WW.MxGateway.Worker.Tests`. This is a verification pass over the 2026-07-12 review (`archreview/2026-07-12/20-worker.md`, remediation IDs WRK-21..28 in `archreview/2026-07-12/remediation/20-worker.md` / `00-tracking.md`, plus NEXT-04/NEXT-05 from `remediation/90-candidate-findings-next-cycle.md`), a re-check of the still-open cycle-1 findings, and a deep review of everything landed since `4f5371f` (~50 worker commits, 53 files, +9,014/−427).
Changed-in-scope since `4f5371f`: the WRK-21/28/23/IPC-30 drain cluster (`33ba612`, `6bc3f9b`), the WRK-22/24/25/27/IPC-26 write-seam batch (`8df35cd`), the OnWriteComplete correlation feature (`fc23a65`..`b0e65d4`), the perf wave (pooled frame buffers `f56798a`, reverse tag index `afec56d`, message-driven completion waits `dc9424d`, signal-driven event drain `f4b065b`, exact-format timestamp parse `94fdc18`/`25cbe5c`, enqueue-time frame sizing `58d97ad`, value-cache borrowing `9871d47`, class-transition flush `aac7957`, drain-pass unparking `9130994`), the NEXT-04/NEXT-05 abandonment-observation batch (`84dbf20`, `3ef56be`, `b8b7b69`), session-owned transport disposal (`e913dab`, labeled WRK-31 in code), the alarm-side truncation/failover hardening (`f3e1de5`, `b5ea6bb`, `693a78d`, `b9fb0dd`), and the WorkerPipeSessionTests harness rework (`7da52b6`, `462850a`, `aaeb86b`, `753d070`).
**Finding-ID note:** the tracker instruction says new IDs continue at WRK-29, but `e913dab` already consumed **WRK-31** as an interim ID (cited in code at `Ipc/WorkerPipeSession.cs:45,177` and in `Ipc/WorkerPipeSessionTests.cs`); WRK-29/WRK-30 appear nowhere in the repo or its history but may have been burned in the same interim numbering. To guarantee no reuse, this review's new findings start at **WRK-32**.
## Executive summary
- All eight cycle-2 Done claims (WRK-21..28) are present, sound, and tested at HEAD; none is a false Done and none carries a functional caveat. The WRK-21 fix went beyond the design: the drain sizes events at enqueue time (`58d97ad`) so the byte-budget walk no longer holds the queue lock the STA needs, and `6bc3f9b` made the budget monotonic at the reserve boundary so drain-until-empty terminates even at the 1024-byte negotiated floor.
- NEXT-04 (fault-observing continuations on abandoned/tombstoned frames) and NEXT-05 (lazy tombstone purge as a documented decision) are both verified — the observer discipline is applied uniformly in the writer (`ObserveAbandonedFault`/`ObserveFault`), and `e913dab`/`3ef56be` extended it to the abandoned net48 pipe read and overrun background tasks, with a GC-forcing `UnobservedTaskException` regression test.
- The two riskiest perf changes checked out. The value-cache borrow (`9871d47`) is safe because the post-publish hook runs only after `Enqueue` and every consumer of the borrowed instances is serialize-only (verified through `MxAccessSession.SucceededRead` and the write-completion cache, which deliberately still clones); the class-transition flush (`aac7957`) and drain-pass unparking (`9130994`) preserve the "written and flushed before completion" contract, are lost-wakeup-free (a walked-away lock acquisition is detached, drains, and releases — never dropped), and cannot double-release the semaphore.
- One cycle-1 finding was fixed incidentally: **WRK-13** (unpooled writer buffers) is resolved by `f56798a` — the writer now rents one combined prefix+payload buffer from `ArrayPool` (`Ipc/WorkerFrameWriter.cs:675-687`). The old tracker should be updated. WRK-18's stale-doc half is also discharged (the design doc now describes the implemented fail-fast and honestly names the generic exit code); its dedicated-exit-code half stays open.
- The remaining cycle-1 opens (WRK-02/03/05/08/09/10/14/16/17/18/19 and the WRK-02/03 slice of WRK-20) are all still present. WRK-05 is materially narrowed: in auto/failover mode `FailoverAlarmConsumer.RunPrimary` absorbs primary COM failures (`MxAccess/FailoverAlarmConsumer.cs:284-291`), but the default alarmmgr-only path is still one-poll-exception-fatal (`MxAccess/MxAccessStaSession.cs:392-405`).
- Fresh sweep found no Medium or higher. Three new Lows: recency-based OnWriteComplete correlation can attribute a stale completion from an earlier write to the current write's reply (WRK-32); the discarded command-task observer is the one discarded task in the worker without a fault-observing continuation (WRK-33); the StaHung fault writes the in-flight correlation id into `WorkerFault.CommandMethod` (WRK-34, pre-existing since 2026-04-26).
## Prior-finding verification — cycle 2 (all tracked Done) and NEXT items
| ID | Claimed status | Verified? | Evidence (file:line at HEAD) | Notes |
|----|----------------|-----------|------------------------------|-------|
| WRK-21 (= IPC-23) | Done | **Yes** | Byte-budgeted `Drain(uint, int)` dequeues only events known to fit, leaves a non-fitting head queued, reports `OversizedHeadSequence` (`MxAccess/MxAccessEventQueue.cs:389-429`); per-event size memoized at enqueue outside the drain's lock hold (`:248-281`, `:502-509`, slack constants `:73-81`); reply builder consumes it with truncation `DiagnosticMessage` (`Ipc/WorkerPipeSession.cs:978-1020`); budget = `max(MaxMessageBytes − 64 KiB, MaxMessageBytes/2)` so it stays monotonic at the reserve boundary (`:1034-1039`, `6bc3f9b`); shared result type is net48-safe (`MxAccess/WorkerEventDrainResult.cs`). Backstops: control-reply write catches `MessageTooLarge` and answers with a small `InvalidRequest` reply (`Ipc/WorkerPipeSession.cs:847-864`), STA command replies get the same catch (`:1136-1147`), and the fallback write itself is size-guarded so a degenerate negotiated max cannot make the backstop fatal (`:876-900`). Tests: `WorkerPipeSessionTests.DrainEvents_ByteHeavyQueue_ReplyIsBoundedAndSessionSurvives` (:571), `DrainEvents_RepeatedCalls_RecoverAllEventsWithoutLoss` (:630), `ControlReplyTooLarge_WritesErrorReplyInsteadOfDying` (:770), `CommandReplyTooLarge_WritesErrorReplyInsteadOfFaulting` (:815), plus `MxAccessEventQueueTests` byte-budget cases. | Sound, and stronger than designed (no sizing walk under the STA-shared lock). No diagnostics command is session-fatal on size anymore. |
| WRK-22 (= IPC-26) | Done | **Yes** | `PendingFrame.Claimed` interlock documented and mutated only under `_gate` (`Ipc/WorkerFrameWriter.cs:54-62`); cancelled waiter tombstones only unclaimed frames (`:169-183`, `:308-338`); `DequeueNext` skips cancelled frames and claims atomically in the same critical section (`:584-618`); contract documented in the method doc (`:105-112`) and `docs/WorkerFrameProtocol.md:203-211`. Tests `WorkerFrameProtocolTests.WriteAsync_CancelledWhileWaitingForLock_FrameIsNeverWritten` (:695), `WriteAsync_CancelledEventFrame_DoesNotTrailShutdownAck` (:800). | Sound. The claimed-mid-write residual window is documented as by-design. |
| WRK-23 | Done | **Yes** | Peek-stamp-commit: candidate stamped before the size checks (sequence participates in `CalculateSize`), counter committed only once the frame is certain to be written, so a per-frame rejection burns no number (`Ipc/WorkerFrameWriter.cs:648-666`; comment `:72-76`). | Wire sequence stays contiguous across rejections; the gap-free concurrent-sequence test's guarantee is now genuinely true. |
| WRK-24 | Done | **Yes** | `MinNegotiableFrameBytes = 1024` matching the gateway validation floor, rejected at the handshake with `InvalidConfiguration` (`Ipc/WorkerFrameProtocolOptions.cs:30`, `:144-150`); range documented in the property doc and `docs/WorkerFrameProtocol.md`. | Also closes the degenerate config that could have made WRK-21's fallback swallow reachable (`Ipc/WorkerPipeSession.cs:866-875` documents that chain). |
| WRK-25 | Done | **Yes** | `WriteBatchAsync` enqueues the whole batch under one `_gate` acquisition, drains under one lock, awaits every completion so no per-frame rejection faults unobserved, rethrows the first failure in batch order (`Ipc/WorkerFrameWriter.cs:236-306`); the event drain loop submits drained batches through it (`Ipc/WorkerPipeSession.cs:615-630`). Tests `WriteBatchAsync_FlushesOnceAndPreservesOrder` (:839), `WriteBatchAsync_ControlFrameQueuedDuringBatch_JumpsRemainingEvents` (:876), `WorkerPipeSessionTests.EventBurst_DrainLoopCoalescesFlushes` (deflaked `50322ba`). | The WRK-12 coalescing now engages on the exact path it was built for: an uninterrupted N-event burst pays one flush. `aac7957` later refined the mixed-class case (see deep review). |
| WRK-26 (discharges IPC-29) | Done | **Yes** | `docs/MxAccessWorkerInstanceDesign.md:717-751` — Outbound Queues rewritten to the two-class scheduler, explicitly records the accepted collapse of the five-level order and why, and rewrites the overflow paragraph to the implemented fail-fast (naming the still-generic `UnexpectedFailure` exit code honestly — WRK-18's exit-code half remains tracked); `docs/WorkerFrameProtocol.md:79+` "Write Scheduling And Sequencing" covers priority classes, write-time stamping, per-frame vs stream-failure semantics, flush coalescing, and cancellation. | No stale five-level prose remains. |
| WRK-27 | Done | **Yes** | `volatile bool staAlarmPollInProgress` flipped on the STA exactly around `PollOnce` (`MxAccess/MxAccessStaSession.cs:64-70`, `:352-365`); surfaced via `WorkerRuntimeHeartbeatSnapshot.StaCallInProgress` (`MxAccess/WorkerRuntimeHeartbeatSnapshot.cs`, ctor default keeps net48-safe plain ctor); watchdog suppression honors it alongside the in-flight correlation id up to the ceiling (`Ipc/WorkerPipeSession.cs:1359-1370`). Test `WorkerPipeSessionTests.Watchdog_StaCallInProgress_SuppressedUntilCeiling` (:1260). | A slow-but-healthy poll now gets the 75 s ceiling, not the 15 s grace; a poll blocking past the ceiling still faults, per the ceiling's contract. |
| WRK-28 | Done | **Yes** | `GatewayContractInfo.MaxDrainEventsPerCommand = 10_000` in Contracts (`src/ZB.MOM.WW.MxGateway.Contracts/GatewayContractInfo.cs:33`), consumed by the gateway validator (`src/ZB.MOM.WW.MxGateway.Server/Grpc/MxAccessGrpcRequestValidator.cs:82-86`) and the worker clamp (`Ipc/WorkerPipeSession.cs:992-994`). | Single shared constant; the desync failure mode is removed, not merely detected. |
| NEXT-04 | Resolved 2026-08-10 | **Yes** | Unconditional fault-observing continuation on every frame a cancelled caller abandons — `TombstoneIfUnclaimed`/`TombstoneUnclaimed` → `ObserveAbandonedFault` → `ObserveFault` (`Ipc/WorkerFrameWriter.cs:308-372`); a cancelled task never fires `OnlyOnFaulted`, so the unconditional attach is safe, and it runs outside `_gate` because an already-faulted task continues inline. Also applied to the detached lock wait (`:389-426`) and, session-side, to overrun background tasks and the abandoned pipe read (`Ipc/WorkerPipeSession.cs:521-576`). Regression test drives the exact abandonment and asserts a marker exception never reaches `TaskScheduler.UnobservedTaskException` (`WorkerFrameProtocolTests:733-785`). | Verified; the discipline is now near-uniform (the one residual gap is WRK-33). |
| NEXT-05 | Resolved as documented decision | **Yes** | `DequeueNext` still lazily skips tombstones (`Ipc/WorkerFrameWriter.cs:588-618`); the rationale — any subsequent write drains both queues, the heartbeat bounds tombstone residency to one interval, and eager `Queue<T>` rebuilds would add ordering-invariant surface next to the WRK-22 interlock — is recorded in `docs/WorkerFrameProtocol.md:218-221`. | Decision recorded where the review asked for it. |
Done claims that failed verification: **none**. Regressions in previously-verified fixes: **none** — WRK-01 (pump refresh, `Sta/StaRuntime.cs:95-100`), WRK-04/GWC-28 (write-time sequence stamping, now peek-stamp-commit), WRK-06 (converter cache, extended by `94fdc18`'s compiled accessors), WRK-07 (two-class scheduler), WRK-11 (ownership transfer, now load-bearing for both the value-cache borrow and enqueue-time sizing), WRK-12 (flush coalescing, finally effective via WRK-25), and WRK-15 (doc names) all remain intact at HEAD.
## Prior-finding verification — still-open cycle-1 items
| ID | Status at HEAD | Evidence | Notes |
|----|----------------|----------|-------|
| WRK-02 | **Still open** | `Sta/StaRuntime.cs:265-269` (loop exception stored in write-only `startupException` after startup; only `startedEvent.Set()`); `InvokeAsync` (`:161-189`) checks disposal and shutdown but not thread death — work enqueued after `ThreadMain` unwinds is consumed by nobody and its task hangs forever (`CancelQueuedCommands` in the `finally` at `:272` only drains what was queued at death). | Highest-value remaining worker item, unchanged three cycles running. |
| WRK-03 | **Still open** | `Ipc/WorkerPipeSession.cs:1226-1229` — `TryStartCommandTask` silently `return`s when `_acceptingCommands` is false: no reply, no log. | The gateway's correlation wait times out blind. |
| WRK-05 | **Still open, narrowed** | `MxAccess/MxAccessStaSession.cs:392-405` — any exception from one poll records a fault and stops the loop; the drain loop turns it into session termination (`Ipc/WorkerPipeSession.cs:589-598`). Narrowed: in auto mode the `FailoverAlarmConsumer` absorbs primary COM failures inside `PollOnce` (`MxAccess/FailoverAlarmConsumer.cs:284-291`, catches all but OOM and counts toward failover), so a transient alarmmgr E_FAIL no longer escapes. The default alarmmgr-only mode (`Ipc/WorkerPipeSession.cs:78`, and subtag-source failures in standby mode) is still one-failure-fatal. | Worth re-scoping to "non-failover alarm modes" if kept. |
| WRK-08 | **Still open** | `Ipc/WorkerPipeSession.cs:487-491` — the message-loop `finally` cancels the drain loop after the shutdown ack; events already drained from the queue (`:600`) whose batch write is cancelled are tombstoned and lost, and the residual queue is never shipped. `docs/MxAccessWorkerInstanceDesign.md` still documents no discard (`grep residual/unshipped` finds nothing). | Same scale as before the batch change (the old loop also drained 128 ahead of writing). Documentation half remains the cheap part. |
| WRK-09 | **Still open** | No `AppDomain.UnhandledException`/`TaskScheduler.UnobservedTaskException` handler anywhere under `src/ZB.MOM.WW.MxGateway.Worker/` (grep at HEAD; the only mentions are comments noting the absence). | The NEXT-04 observer network reduces the practical exposure but is not a process-level backstop. |
| WRK-10 | **Still open** | `WorkerApplication.cs:112-139` — all three catch blocks log `exception_type` only; the redactor (`Bootstrap/WorkerLogRedactor.cs`) is available and unused here. | Still a cheap diagnostics win. |
| WRK-13 | **Resolved (side effect of `f56798a`)** | `Ipc/WorkerFrameWriter.cs:675-687` — single `ArrayPool<byte>.Shared.Rent` for prefix+payload, one stream write, returned in `finally`; matches the reader's pooling. | Flip to Resolved in `archreview/remediation/00-tracking.md` with commit `f56798a`. |
| WRK-14 | **Still open** | `Ipc/WorkerPipeSession.cs:33-62` `_camelCase` vs `MxAccess/MxAccessStaSession.cs:50-71` and `Sta/StaRuntime.cs:10-24` bare `camelCase`. | Cosmetic; new files followed their host file's convention, so the split persists. |
| WRK-16 | **Still open** | `Ipc/WorkerPipeSession.cs:1449-1544` — seven `CreateEnvelope`/`CreateBaseEnvelope` pairs; `Ipc/WorkerPipeClient.cs` still has 7 public constructors. | |
| WRK-17 | **Still open** | `WorkerApplication.cs:110-119` maps every `WorkerFrameProtocolException` (including `EndOfStream`) to exit 6 `ProtocolViolation`. | |
| WRK-18 | **Half discharged** | Doc half done: `docs/MxAccessWorkerInstanceDesign.md:743-751` now describes the implemented fail-fast and names the generic `UnexpectedFailure` exit honestly. Code half open: a drain-loop fault still exits 1 via the generic catch (`WorkerApplication.cs:131-139`); no dedicated overflow/event-fault exit code. | |
| WRK-19 | **Still open** | `Sta/StaCommandDispatcher.cs` — no logger field, no start/end/failure log anywhere in the file. | |
| WRK-20 | **Residual only** | The WRK-21..28 batch added its full test matrix (byte-budget drain, size backstops, cancellation/tombstoning, batch flush, floor handshake, watchdog suppression), plus teardown `UnobservedTaskException` tests (`WorkerPipeSessionTests:2328-2434`). Still missing: STA-thread-death (WRK-02) and command-refused-during-shutdown (WRK-03) tests, because their fixes are open. | |
## Deep review of the new code
**Frame-writer unparking (`9130994`) and class-transition flush (`aac7957`).** `WriteAsync` now races its frame's completion against the lock acquisition (`Ipc/WorkerFrameWriter.cs:148-198`): a caller whose frame is delivered by another drainer returns at the boundary flush instead of at the winner's release, and the acquisition it walks away from is handed to `DetachLockWait` (`:389-399`), whose continuation either observes a cancelled wait (no count taken, nothing to release) or drains-and-releases (`:409-452`). I probed this seam hard: acquisition and cancellation are mutually exclusive in `SemaphoreSlim`, so no count can leak and no double-release path exists (`DrainDetachedAsync` releases in a `finally` that runs exactly once per successful acquisition); a frame enqueued between a drainer's last `DequeueNext` and its release is picked up by the detached pass, so no lost wakeup; every discarded task in the chain carries the NEXT-04 observer. The drain flushes and completes at each control→event boundary (`:484-511`), so a heartbeat/reply/fault's delivery point is never charged for the event backlog behind it; frames completed at a boundary leave `written`, so a later flush failure in the same pass cannot reach back and fail an already-delivered frame (`:543-557`) — the failure-attribution edges are all correct. Cost claims in the comments are accurate: a pure-event pass still pays one flush; only a genuinely mixed pass pays one extra per class transition. Residual (accepted, documented at `Ipc/WorkerPipeSession.cs:186-191`): a detached acquisition can schedule an empty drain pass after session teardown — harmless because the queues are empty and an empty pass writes nothing.
**Value-cache borrowing (`9871d47`).** The hunt for aliasing bugs came up clean. The borrow is sound on three legs, each verified: (1) ordering — `MxAccessBaseEventSink.EnqueueEvent` runs the `postPublish` cache write only after `eventQueue.Enqueue` returns (`MxAccess/MxAccessBaseEventSink.cs:248-295`), so the event is already write-once (WRK-11 ownership plus the sequence/timestamp stamps under the queue lock); (2) consumers — `MxAccessSession.SucceededRead` copies the borrowed `Statuses` element references into a fresh `RepeatedField` on the reply and reference-shares `Value`/`SourceTimestamp` (`MxAccess/MxAccessSession.cs:1066-1097`), and everything downstream only serializes (the worker↔gateway process boundary prevents any other consumer from aliasing); (3) the size-memoization coupling — a post-enqueue mutation would now also invalidate `QueuedEvent.Size`, and the ownership contract text in both classes names that hazard explicitly (`MxAccess/MxAccessValueCache.cs:76-108`, `MxAccess/MxAccessEventQueue.cs:15-32`). The write-completion cache deliberately still clones and documents why the asymmetry is not an oversight (`MxAccess/MxAccessWriteCompletionCache.cs:77-89`). Protobuf reads (`CalculateSize`/`WriteTo`) from multiple threads on a frozen message are safe, and every cross-thread hand-off passes through a lock.
**Enqueue-time frame sizing (`58d97ad`).** `CalculateWorkerEventSize` reconstructs the generated `CalculateSize` arithmetic for exactly the two stamped fields (`MxAccess/MxAccessEventQueue.cs:493-509`); I checked the tag math — `worker_sequence` (field 9, varint) and `worker_timestamp` (field 10, length-delimited) are both single-byte tags, the `WorkerEvent.event` wrapper (field 1) matches the `DrainEventsReply.events` packing shape exactly, and the +8 `RepeatedFieldOverheadBytes` slack absorbs the documented renumbering hazard (`:66-81`). Clearing the two fields before measurement makes the pre-lock measurement exact even against a caller that pre-filled them. The wake signal (`SignalWake`, `:471-491`) is a level, not a count; I walked the check-then-release race against a concurrent consumer and every interleaving either sees the event in its re-drain or leaves a pending permit — no lost wake, and the 25 ms fallback tick bounds any hypothetical future unsignalled mutation.
**Session-owned transport disposal and abandonment observation (`e913dab`, `3ef56be`, WRK-31).** The session now owns the pipe stream and disposes it as the last teardown step — the only thing that unblocks a net48 pipe read parked in the kernel — then awaits the abandoned read's resulting fault (`Ipc/WorkerPipeSession.cs:143-290`). The `_pendingReadTask` bookkeeping preserves the reader's single-consumer invariant through teardown (set at issue, cleared before the loop's own await, read only after the loop's task completes — the await supplies the happens-before), and two `Debug.Assert`s pin it (`:447-449`, `:470-472`). `ObserveBackgroundTaskStopAsync` correctly separates the logging budget from observation: an overrun task gets `ObserveFaultWhenever` before the method returns (`:521-558`), so the 1 s budget is a diagnostics decision, not a correctness one. `DisposeTransportStream`'s total catch is well argued (nothing thrown while closing a handle may displace the session's terminal exception, `:212-250`), and `WorkerPipeClient` keeps its `using` as an explicitly-documented idempotent backstop (`Ipc/WorkerPipeClient.cs:153-165`).
**Oversized-event death (IPC-30) through the batch path.** A `MessageTooLarge` surfacing from the batch's awaited completions is mapped back to the first envelope in batch order that overshoots the negotiated max, logged with identity-but-never-value fields, written as a structured `ProtocolViolation` fault, and then thrown (`Ipc/WorkerPipeSession.cs:626-725`). `FindOversizedEvent`'s recomputation is consistent with what the writer rejected (the rejected envelope retains its tentative sequence stamp, so `CalculateSize` agrees), and the unreachable-fallback comment is honest. Other per-frame rejection codes still unwind the session as worker bugs — the deliberate IPC-30 scope.
**Write-completion correlation (`fc23a65`..`b0e65d4`).** Parity-clean in the load-bearing respects: no synthesized events (the completion rides the unary reply's `statuses` only), acceptance-only `Hresult`/`ProtocolStatus` either way, expiry yields empty statuses ("unconfirmed, not failed"), zero timeout restores pure fire-and-forget, and the pump-wait keeps the STA dispatching for the whole wait via the shared `MsgWaitForMultipleObjectsEx` declaration (`Sta/StaNativeMethods.cs`, single-copy rationale documented). `StaWaitHelper` handles the failed-wait degrade (one-shot trace + blind sleep, `Sta/StaWaitHelper.cs:104-112`) and documents the `MWMO_INPUTAVAILABLE` drain-to-empty contract that would otherwise busy-spin. The one soft spot is attribution across successive writes to the same handle — WRK-32 below.
**Alarm subtag failover and truncation.** The production wiring works despite the `standbyFactory: null` default: the `comFactory` threaded through `MxAccessStaSession.StartAsync` lets `AlarmCommandHandler.BuildStandby` construct the `SubtagAlarmConsumer` over `LmxSubtagAlarmSource` (`MxAccess/AlarmCommandHandler.cs:206-248`); mode selection (alarmmgr / subtag / auto-failover) matches the documented matrix, provider-mode changes are surfaced as queue events with handler detach on unsubscribe/dispose (`:159-183`, `:258-267`). WnWrap truncation handling is careful where it matters: a truncated fetch merges into the retained snapshot instead of clearing it (`ApplySnapshotUpdate`), so Clear transitions are never synthesized from records that merely fell off a truncated fetch, and the truncation verdict rides out of the same locked fold that produced the transitions (`FoldFetch`), with a rate-limited warning. STA affinity for the wnwrap COM object is asserted at every consumer entry (`MxAccess/MxAccessStaSession.cs:410-440`), and the affinity-violation exception is deliberately routed to the fault path rather than the graceful-stop path (`:378-405`).
**Handle-registry reverse index (`afec56d`).** Every mutation maintains all four secondary indexes and the three memoized views; I walked the consistency edges — re-registration with a changed definition drops the stale index entry first, per-server teardown removes exactly the owned sets, advice de-indexing trims empty containers, and the sorted-insertion keeps `GetItemHandlesForDefinition` in the same order the old linear scan visited (`MxAccess/MxAccessHandleRegistry.cs`). The registry is STA-confined by documented contract, so the lock-free indexes are sound. The returned live-view caveat is documented at the API.
**Timestamp fast path (`94fdc18`, `25cbe5c`).** The per-culture derived formats are validated by round-tripping a probe instant before they ever reach the event path, so a pathological culture pattern (quoted `ss`, existing fraction) degrades to no-fast-path instead of a throwing `TryParseExact`; the fast path is provably a shortcut, never a different answer, because the formats are derived from the same culture the general parse would use (`MxAccess/MxAccessEventMapper.cs:373-509`).
**Test harness rework.** `FakeRuntimeSession` now mirrors production's coalesced wake signal (never-disposed `SemaphoreSlim`, same rationale comment), models STA-pump-at-heartbeat (`refreshStaActivityOnCapture`), and its 30 s dispatch safety net is decoupled from scenario timing with the reasoning written down (`TestSupport/FakeRuntimeSession.cs:23-51`). The teardown tests arm `TaskScheduler.UnobservedTaskException` and force GC — the strongest possible assertion shape for the NEXT-04/WRK-31 class of fix. Test counts in scope: 40 facts/theories in `WorkerPipeSessionTests`, 31 in `WorkerFrameProtocolTests`, 23 in `MxAccessEventQueueTests`, plus new suites for the completion cache, wait helper, registry, and WnWrap XML parsing.
## New findings
**WRK-32 — Low (error handling / documentation).** Write-completion correlation is recency-based per handle; a late `OnWriteComplete` from an earlier write can be reported as the current write's outcome.
Evidence: the baseline is `CurrentVersion` captured before the COM call (`MxAccess/MxAccessCommandExecutor.cs:447-450`, `:522-528`), and `AwaitWriteCompletion` copies onto the reply *any* completion whose version exceeds it (`:1005-1027`; version test only, `MxAccess/MxAccessWriteCompletionCache.cs:162-169`). MXAccess's `OnWriteComplete` carries no correlation token, only the handle pair — so if write A to a tag times out unconfirmed (its callback still undelivered) and the client immediately writes B to the same tag, A's stale callback dispatching during B's pump-wait bumps the version past B's baseline and A's status rows ride B's reply. The executor doc comment claims "the OnWriteComplete row *matching* a WriteSecured/WriteSecured2 call" (`:996-1003`), and neither `docs/MxAccessWorkerInstanceDesign.md` nor the proto comments mention the ambiguity (grep for misattribution/stale/earlier-write finds nothing). Impact is bounded — the statuses are an enrichment field, `Hresult`/`ProtocolStatus` stay acceptance-only, and back-to-back same-tag writes with a timed-out predecessor is the only exposed sequence — and the ambiguity is inherent to the COM surface (a native client correlating by handle has the same problem, so mechanical parity is arguably preserved).
Recommendation: document the recency-based contract at the executor, in the design doc's write-completion section, and in the reply-field proto comment; additionally, run one `pumpStep()` immediately before capturing the baseline so an already-queued stale callback is folded into the baseline instead of the wait window. Full closure is impossible without an MXAccess-side token — say so where consumers will read it.
**WRK-33 — Low (concurrency / NEXT-04 discipline gap).** The discarded command-task observer is the one discarded task in the worker without a fault-observing continuation.
Evidence: `TryStartCommandTask` discards its wrapper — `_ = ObserveCommandTaskAsync(commandTask);` (`Ipc/WorkerPipeSession.cs:1235`) — and `ObserveCommandTaskAsync` catches only `OperationCanceledException` (`:1238-1254`). `ProcessCommandAsync` handles its own failures, but its catch path ends in `TryWriteFaultAsync`, whose filter is exactly `IOException`/`ObjectDisposedException`/`WorkerFrameProtocolException` (`:1440-1447`); any other exception escaping that write — or a future edit anywhere in the catch path — propagates out of `ProcessCommandAsync`, faults the discarded wrapper, and surfaces only as a finalizer-time unobserved-task exception in a process that installs no handler (WRK-09). Every other discarded task in this codebase now carries an observer (`ObserveFaultWhenever` lives 660 lines up in this same file); the gap is unlikely to fire but contradicts the discipline the NEXT-04/WRK-31 batch established and tested.
Recommendation: `ObserveFaultWhenever(ObserveCommandTaskAsync(commandTask));`, or broaden `ObserveCommandTaskAsync` to catch-all-and-log (the redaction-safe fields are already at hand in `LogCommandResultDropped`'s shape).
**WRK-34 — Low (documentation / diagnostics).** The StaHung fault writes the in-flight correlation id into `WorkerFault.CommandMethod`.
Evidence: `ReportWatchdogFaultIfNeededAsync` builds its fault with `CreateFault(WorkerFaultCategory.StaHung, snapshot.CurrentCommandCorrelationId, $"STA activity is stale by {staleFor}.")` (`Ipc/WorkerPipeSession.cs:1384-1389`), whose second parameter is `commandMethod` (`:1712-1728`) — so the wire fault's `command_method` field carries a correlation id (or empty), a correlation id never lands in a correlation-shaped field, and a gateway/dashboard consumer reading `CommandMethod` as a method name gets an opaque id string. Pre-existing since `4a3560c` (2026-04-26), never previously flagged; noted now because the WRK-27 work touched this exact branch and the field's semantics matter more once polls also suppress here.
Recommendation: set `CommandMethod` to a stable label (empty, or `"Watchdog"`), and carry the stuck command's correlation id in the fault's diagnostic message or the envelope's `CorrelationId` where the gateway already correlates.
## Severity roll-up (new findings)
| Severity | Count | IDs |
|----------|-------|-----|
| High | 0 | — |
| Medium | 0 | — |
| Low | 3 | WRK-32, WRK-33, WRK-34 |
## Still good / preserved
Every structural strength recorded in the prior two reviews is intact at HEAD, and several were deepened. The STA core still runs the canonical `MsgWaitForMultipleObjectsEx` + `PeekMessage`/`DispatchMessage` loop with the wake event and 50 ms idle pump (`Sta/StaRuntime.cs:255-261`), and the subtle Win32 wait contract now has a single shared declaration with the drift hazard written down (`Sta/StaNativeMethods.cs`). All MXAccess COM calls remain on the dedicated STA (commands via the dispatcher, alarm polls via `InvokeAsync` with an affinity assertion, the consumer factory on-thread at `MxAccess/MxAccessStaSession.cs:282-296`); COM lifetime discipline (`FinalReleaseComObject`, sink-detach-before-release, UnAdvise → RemoveItem → Unregister, alarm handler disposed on the STA before data-side teardown) is unchanged. Nothing synthesizes events — the completion-correlation feature explicitly kept `OperationComplete` untouchable and the truncation work explicitly refuses to fabricate Clear transitions. Fail-fast backpressure stands (no drop policy; overflow self-records the first fault; the dispatcher's 128-entry bound rejects loudly). Credentials and values stay out of logs (the oversized-event fault logs sizes and identity only, `AuthenticateUser` passes the password straight through, the DrainEvents truncation message names sequences not values). net48 constraints are respected in every new type (plain ctors, get-only properties, `readonly struct`, no init/records — several carry the CS0518 rationale in their remarks). The IPC layer's failure-attribution story is now the best part of the worker: written-and-flushed-before-completion, per-frame vs stream-failure separation, boundary-flush completion, tombstone/claim interlock, and a fault observer on effectively every abandoned task, each with a regression test that asserts the invariant rather than the implementation. Comment quality across the new code is exceptional — ownership contracts, memory-model reasoning, and rejected alternatives are recorded at the exact seams where the next editor would otherwise re-litigate them.
## Verification needed on Windows (windev)
- This review is static: the x86 worker was not built and `Worker.Tests` were not run here. The tracking log and repo memory record the worker suite fully green on windev after the harness fixes (`7da52b6`/`462850a`/`aaeb86b`), and the WRK-21 cluster carries windev evidence (`c992568`); any fix for WRK-32..34 needs an x86 build plus filtered `MxAccessCommandExecutorTests`/`WorkerPipeSessionTests` runs there.
- WRK-32's pre-baseline pump mitigation should get a windev test that records a stale completion into the fake's message path before the second write's baseline and asserts it does not ride the second reply.
- WRK-33 is assertable with the existing `UnobservedTaskException`-arming test shape (`WorkerPipeSessionTests:2328-2434`): drive an exception through `TryWriteFaultAsync`'s filter gap with a throwing fake writer and assert nothing reaches the finalizer.
- Live MXAccess only: the WRK-05 narrowing (failover absorbing a transient alarmmgr E_FAIL without a session fault) deserves a live-rig confirmation next time the opt-in alarm smoke runs; the rig's object-driven alarms make this drivable only from the simulator side.
+143
View File
@@ -0,0 +1,143 @@
# Contracts & IPC Protocol — Cycle-3 Review
- **Date:** 2026-08-19
- **Commit:** `8b3e3b3` (`main`, clean)
- **Baseline:** `4f5371f` (2026-07-12 cycle-2 review; ~288 commits since, ~55 of them touching the contracts, frame-protocol, codegen, or IPC-doc paths in scope)
- **Method:** static review on macOS; every cited file read at HEAD. No build, no test, no source change. `git diff 4f5371f..HEAD -- '*.proto'` walked line by line; generated trees grep-checked for every new symbol; the GalaxyRepository package proto compared against the sibling `scadaproj` checkout.
- **Scope:** `src/ZB.MOM.WW.MxGateway.Contracts` (three protos, `Generated/` hygiene, `net10.0;net48` multi-targeting, `galaxy_repository.proto` retention vs the `ZB.MOM.WW.GalaxyRepository` package), the frame protocol on both sides (`Worker/Ipc/*`, `Server/Workers/WorkerFrame*`, `WorkerClient` envelope/sequence/size handling, hello/handshake, `max_frame_bytes` negotiation, DrainEvents byte bound), the codegen/descriptor pipeline (`scripts/check-codegen.ps1` Checks 1–4, `scripts/publish-client-proto-inputs.ps1`, `clients/proto/*`, vendored Rust protos, Go/Python/Java binding freshness guards, Gradle `checkGeneratedClean`), proto-evolution hygiene, and currency of `docs/WorkerFrameProtocol.md`, `gateway.md` (IPC envelope section), `docs/Grpc.md`, `docs/ClientProtoGeneration.md`, `docs/Contracts.md`. Worker-internal mechanics of the drain/write seam are owned by `20-worker.md` and are cross-referenced, not re-reviewed.
- **Prior reports:** `archreview/2026-07-12/30-contracts-ipc.md` (cycle 2); `archreview/30-contracts-ipc.md` (cycle 1); remediation status sources `archreview/2026-07-12/remediation/00-tracking.md` + `30-contracts-ipc.md`, `90-candidate-findings-next-cycle.md`; first-cycle backlog `archreview/remediation/00-tracking.md`.
## Proto evolution since baseline — verified additive-only
`git diff 4f5371f..HEAD -- src/ZB.MOM.WW.MxGateway.Contracts/Protos/` contains exactly four wire-affecting lines, all additive with next-free tags:
| Change | Location | Tag check |
|---|---|---|
| `bool snapshot_truncated = 2;` on `QueryActiveAlarmsReplyPayload` | `mxaccess_gateway.proto:735` | message previously ended at field 1 |
| `bool from_truncated_snapshot = 16;` on `ActiveAlarmSnapshot` | `:951` | previous highest was `source_provider = 15` |
| `AlarmSnapshotStatus snapshot_status = 5;` oneof arm on `AlarmFeedMessage` | `:1028` | previous highest arm was `provider_status = 4` |
| new `message AlarmSnapshotStatus { bool truncated = 1; }` | `:1048-1054` | new type |
Everything else in the diff is comment-only (write-completion `statuses` contract at `:538-551`, DrainEvents truncation at `:702-705`, `ReplayGap` empty-ring sentinel at `:797-801`, `AlarmFeedMessage` open sequence at `:1006-1012`, and the `GatewayHello.max_frame_bytes` byte-cap sentence at `mxaccess_worker.proto:50-52`). No renumbering, no repurposing; `reserved 1` / `reserved "session_id"` intact on the two retired fields (`mxaccess_gateway.proto:964-965`, `:978-979`); wire-compat policy headers present in all three files; `gatewayProtocolVersion: 3` / `workerProtocolVersion: 1` in `clients/proto/proto-inputs.json:4-5` match `GatewayContractInfo.cs:12,15` (additive changes correctly did not bump either). `galaxy_repository.proto` is unchanged and still wire-identical to the package source (see IPC-37).
Generated-tree freshness for the new symbols, grep-verified at HEAD: `Contracts/Generated/MxaccessGateway.cs` (`AlarmSnapshotStatus` ×16, `from_truncated_snapshot` ×2), Go `mxaccess_gateway.pb.go` (×15 / ×3), Python `mxaccess_gateway_pb2.py` (descriptor bytes carry `\x66rom_truncated_snapshot`, `AlarmSnapshotStatus`), Java `MxaccessGateway.java` (×106 / ×7), the committed protoset (×2 / ×2), and the Rust vendored copies `cmp`-identical for all three files. Protos, protoset, and every binding set were last touched in the same commit (`c3c603f`, 2026-08-18) — the regen-everything-together discipline held, including for a comment-only change.
## Prior-finding verification
### Cycle-2 findings (IPC-23..32)
Every tracked status was verified against code read at HEAD. **All eight Done claims hold; one carries a residual filed as a new finding (IPC-23 → IPC-33).**
| ID | Claimed | Verified? | Evidence (HEAD) | Notes |
|----|---------|-----------|-----------------|-------|
| IPC-23 | Done (mechanics in WRK-21) | ✅ holds, **with caveat** | Contract half: `GatewayHello.max_frame_bytes` comment extended (`mxaccess_worker.proto:50-52`); `DrainEventsReply` truncation comment (`mxaccess_gateway.proto:702-705`); `docs/WorkerFrameProtocol.md:43-51` states the byte-cap rule and drain-until-empty contract; `gateway.md:565-572` documents both axes. Mechanics: byte-budgeted `Drain(uint, int)` leaves a non-fitting head queued and reports `OversizedHeadSequence` (`Worker/MxAccess/MxAccessEventQueue.cs:389-429`, per-event size memoized at enqueue `:493-509`); reply builder `CreateDrainEventsReply` consumes it with a truncation `DiagnosticMessage` (`Worker/Ipc/WorkerPipeSession.cs:978-1020`); budget = `max(Max − 64 KiB, Max/2)` (`:1034-1039`); shared count ceiling `GatewayContractInfo.MaxDrainEventsPerCommand` (`Contracts/GatewayContractInfo.cs:33`) consumed by both the gateway validator (`Grpc/MxAccessGrpcRequestValidator.cs:81-86`) and the worker clamp (`:992-994`); reply-too-large backstop answers `InvalidRequest` instead of dying (`:847-864`). R1–R3 satisfied. | Caveat: R1 bounds the reply against the **pipe** frame max; the same `MxCommandReply` then crosses the public gRPC boundary whose send cap is `MaxGrpcMessageBytes` = pipe max − 64 KiB, and the budget's own 64 KiB headroom consumes exactly that margin → **IPC-33**. The tracker's rationale for leaving `docs/Grpc.md` without a DrainEvents row ("a worker IPC diagnostic, not a public RPC") is inaccurate — `DrainEvents` is a public `MxCommandKind` (`mxaccess_gateway.proto:136`) reachable through `Invoke` → **IPC-38**. |
| IPC-24 | Done | ✅ holds | Churn-revert step deleted; `java` job runs `gradle test` then `git diff --exit-code -- clients/java/src/main/generated` (`.gitea/workflows/ci.yml:135-142` (test step `:135-137`, diff gate `:138-142`), comment at `:113-119` records the rationale); toolchain pinned `grpcVersion 1.76.0` / `protobufVersion 4.33.1` (`clients/java/build.gradle:8,11`); `checkGeneratedClean` revived for Gradle 9 via `providers.exec` evaluated lazily in `doLast` (`clients/java/zb-mom-ww-mxgateway-client/build.gradle:71-92`, `df45cb4`); docs updated (`docs/ClientProtoGeneration.md:97,100-106`). Java aggregate at HEAD carries every new symbol (see above). | The lazy `ProviderFactory.exec` shape is correct: the provider is first realized in `doLast`, after the `generateProto` dependency ran, so the git probe sees the regenerated tree. |
| IPC-25 | Done | ✅ holds | Go/Python bindings regenerated and fresh (grep above; `max_frame_bytes` now in `mxaccess_worker.pb.go` ×2 and `mxaccess_worker_pb2.py` ×1); Check 4 regenerates both via the per-client scripts and fails on any `git status --porcelain` diff, with tool-missing → fail not skip (`scripts/check-codegen.ps1:109-129`, header `:19-24`); Go script asserts `protoc-gen-go v1.36.11` / `protoc-gen-go-grpc 1.6.2` exactly and warns-only on protoc (`clients/go/generate-proto.ps1:13-15,63-78`), with the Windows `.exe` banner normalized (`:22-28`, `c94c4d4`); Python script asserts grpcio-tools 1.80.0 with PS5.1-safe quoting (`clients/python/generate-proto.ps1:36-49`, `8ae0c2f`); CI installs the pinned generators before the check (`ci.yml:63-72`); pins table in `docs/ClientProtoGeneration.md:90-98`. | Windows-side fixes landed after the cycle-2 design (Check 4 now runs on windev too). Backslash-joined paths in both scripts are normalized by pwsh on Unix (verified locally with `Test-Path`). |
| IPC-26 | Done (mechanics in WRK-22) | ✅ holds | Contract requirement ("cancelled means never written") implemented by claim/tombstone interlock: `TombstoneIfUnclaimed` (`Worker/Ipc/WorkerFrameWriter.cs:181,308-338`), `DequeueNext` skips tombstones (`:588`), NEXT-04 fault observer on every abandoned frame (`:350`); documented with the claimed-mid-write residual at `docs/WorkerFrameProtocol.md:202-224`. Full mechanics verification in `20-worker.md` (WRK-22, NEXT-04/05). | Doc landed in the same batch as the code, as the plan required. |
| IPC-27 | Done | ✅ holds | `ClientProtoInputTests.Descriptor_ContainsEveryContractSymbol` collects messages, fields, nested/top-level enums and values, services and methods from the protoset (`Tests/Contracts/ClientProtoInputTests.cs:37-62,108-146`) and walks all three reflection descriptors including `GalaxyRepositoryReflection.Descriptor` (`:65-99`); doc prose updated (`docs/ClientProtoGeneration.md:76-82`). | Presence-only (no field numbers / types), per the recorded design; the protoc-pinned script check remains the byte-level gate. |
| IPC-28 | Done | ✅ holds | `docs/Grpc.md:285` lists `CommandTooLarge → ResourceExhausted`; Invoke-section paragraph at `:85` explains the per-correlation failure and the headroom rule; matches `Grpc/MxAccessGatewayService.cs:1056`. | |
| IPC-29 | Done (discharged by WRK-26) | ✅ holds | `docs/WorkerFrameProtocol.md:79-224` "Write Scheduling And Sequencing": two-class scheduler, peek-stamp-commit write-time stamping (`:60-64`, `:98-107`), per-frame vs stream failure (`:109-120`), flush coalescing and boundary flush (`:122-172`), unparking (`:174-200`), cancellation tombstones (`:202-224`); plus a new "Pipe Buffers" section (`:226-235`, `da84635`). Each claim cross-checked against `Worker/Ipc/WorkerFrameWriter.cs` anchors cited in `20-worker.md`. | The doc now slightly *exceeds* the code in one place: `mxaccess_worker.proto:50-52` says "reply builders truncate to fit" immediately after listing events — events do not truncate (IPC-30 policy). Readable as intended; no action. |
| IPC-30 | Done | ✅ holds | Batch-path `MessageTooLarge` is mapped to the first overshooting envelope (`Worker/Ipc/WorkerPipeSession.cs:633-648`, `FindOversizedEvent :648-660`), logged identity-only, written as a `ProtocolViolation` fault with `CommandMethod = "EventDrain"` (`:713`), then thrown; policy documented at `docs/WorkerFrameProtocol.md:53-58`. | Gateway side needed no change: `DispatchEnvelope` already routes `WorkerFault` to `SetFaulted(WorkerFaulted, …)` (`Server/Workers/WorkerClient.cs:614-618`). |
| IPC-31 | N/A (accepted divergence) | **Superseded — resolved by GWC-28** | The gateway now stamps `Sequence` in `WriteLoopAsync` immediately before the write (`Server/Workers/WorkerClient.cs:465-471`), `CreateEnvelope` leaves it unset (`:1229-1236`); `gateway.md:425-431` rewritten to "both sides stamp at the point of writing". Regression test `WorkerClientTests.ConcurrentInvokesEmitStrictlyIncreasingSequencesOnTheWire`. | The two sides now implement the same semantics; flip the IPC-31 row from N/A to Resolved-by-GWC-28 in the tracker. See IPC-39 for the one interaction the change created with the IPC-03 pre-check. |
| IPC-32 | Done | ✅ holds | Banners read `Check 1/4` … `Check 4/4` (`scripts/check-codegen.ps1:40,52,78,110`); header comment lists four checks (`:4-24`). | |
**Done claims that do not fully hold:** none outright. IPC-23 holds for the defect as filed but leaves the gRPC-boundary residual (IPC-33).
### Cycle-1 findings still open at cycle 2 — re-checked at HEAD
| ID | Cycle-2 status | HEAD status | Evidence |
|----|----------------|-------------|----------|
| IPC-08 | Open (`WorkerCancel` defined, handled, never sent) | **Substantially resolved by `7c1ea12`/`a1a38b5` — recommend closing with a residual** | `InvokeAsync` forwards a `WorkerCancel` when the gateway's own command timeout fires and it owned the removal (`Server/Workers/WorkerClient.cs:269-291`); `TrySendCancelForTimedOutCommand` is structurally no-throw (`:835-870`, doc `:806-833`); worker routes it to `CancelCommand` (`Worker/Ipc/WorkerPipeSession.cs:755-756` → `MxAccess/MxAccessStaSession.cs:529-532` → `Sta/StaCommandDispatcher.CancelQueuedCommand :151`); test `WorkerClientTests.InvokeAsync_WhenCommandTimesOut_SendsWorkerCancelForThatCorrelation` (`:185`); `docs/GatewayProcessDesign.md:596-602` documents it. Residual: the caller-cancellation path sends nothing → **IPC-34**. |
| IPC-10 | Open (softened) | **Still open, unchanged** | `Server/Workers/WorkerEnvelopeValidator.cs:15-39` and `Worker/Ipc/WorkerEnvelopeValidator.cs` check version/session/body only; `mxaccess_worker.proto:23` `uint64 sequence = 3;` still has no comment. `gateway.md:425-431` continues to state honestly that sequence is a diagnostic aid, and now that both sides stamp at write (GWC-28/WRK-04) the property an inbound check would rely on actually holds. Acceptable state. |
| IPC-11 | Open | **Still open, unchanged** | Strict equality on both sides: `Worker/Ipc/WorkerPipeSession.cs:370-375` (`ProtocolVersionMismatch`), `Server/Workers/WorkerClient.cs:913-918`; no min/max-range note on `supported_protocol_version` (`mxaccess_worker.proto:43`). No action was ever required. |
| IPC-12 | Open | **Still open, surface unchanged** | `Server/Workers/WorkerFrameWriter.cs:8-11` class doc still has no single-writer remark and the class has no lock; the invariant is now stated at the *call site* instead (`WorkerClient.cs:465-470`: "This loop is the channel's single consumer (SingleReader = true)"), and every outbound envelope — hello `:171`, commands `:251`, cancels `:848-858`, shutdown — funnels through `_outboundEnvelopes`. The gateway reader, by contrast, got its contract written into the class remark when GWC-30 made it stateful (`WorkerFrameReader.cs:11-16`); the writer deserves the same one-paragraph note. |
| IPC-16 / IPC-22 | N/A | Unchanged | Correlation id still carried on envelope + inner reply (`gateway.md:432-434` documents envelope as authoritative); status-code-plus-prose error model unchanged. |
| IPC-18 | Preserve | **Preserved** | See "Proto evolution" above — `Generated/`, all five client trees, the protoset, and the Rust vendored copies are in sync at HEAD; `galaxy_repository.proto` retention comment intact (`ZB.MOM.WW.MxGateway.Contracts.csproj:36-41`). |
### NEXT rows touching this domain
- **NEXT-04 / NEXT-05** (frame-writer abandonment observation; lazy tombstone purge as a documented decision) — both verified in `20-worker.md`; the contract-level statement lives at `docs/WorkerFrameProtocol.md:213-224`. Nothing further from this domain.
- **NEXT-01** (pipe name length) — verified in `10-gateway-core.md`; the frame protocol is unaffected (the name is a transport address, not a wire field).
### Also verified in passing
- **TST-28** — the gateway half of the `max_frame_bytes` handshake is pinned on the CI-run suite: `WorkerClientTests.StartAsync_SendsGatewayHelloWithConfiguredMaxFrameBytes` (`:46-57`), hello built at `WorkerClient.cs:1172-1184` from `FrameOptions.MaxMessageBytes`.
- **Negotiation floor/ceiling agreement** — worker accepts `[1024, 256 MiB]` (`Worker/Ipc/WorkerFrameProtocolOptions.cs:19,30,137-161`); the gateway validator bounds `Worker:MaxMessageBytes` to the identical range (`Configuration/GatewayOptionsValidator.cs:10-11,290-293`) and enforces pipe ≥ gRPC + 64 KiB with `long` arithmetic (`:614-628`). The worker can never reject a value the gateway's validator admits.
- **Public gRPC caps are actually applied** — `MaxReceiveMessageSize` **and** `MaxSendMessageSize` are both set from `Protocol:MaxGrpcMessageBytes` (`Security/Authorization/GrpcAuthorizationServiceCollectionExtensions.cs:36-37`); default 16 MiB (`Configuration/ProtocolOptions.cs:16`). Load-bearing for IPC-33.
- **Check 3 both directions** (`711793f`) — a canonical proto without a vendored Rust copy now fails too (`scripts/check-codegen.ps1:95-103`), closing the `build.rs`-prefers-canonical blind spot its comment describes.
- **Contracts package version** bumped to 0.2.0 with the CLI-39 release train (`ZB.MOM.WW.MxGateway.Contracts.csproj:10`); `Grpc.Tools` 2.80.0 / `Google.Protobuf` 3.34.1 pins unchanged.
## New findings
IDs continue from the register (cycle 2 ended at IPC-32). No `IPC-33+` label exists anywhere in the repo or its history (grep), so there is no collision of the GWC-31 kind here.
### IPC-33 — The DrainEvents byte budget is sized against the pipe frame maximum, not the public gRPC send cap; the reply can cross the pipe and then fail at the gRPC boundary with the drained events already consumed
- **Severity:** Low
- **Files:** `src/ZB.MOM.WW.MxGateway.Worker/Ipc/WorkerPipeSession.cs:31` (`DrainReplyFrameHeadroomBytes = 64 KiB`), `:1034-1039` (`ResolveDrainReplyByteBudget` = `max(MaxMessageBytes − 64 KiB, MaxMessageBytes / 2)`); `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessEventQueue.cs:389-429` (events dequeued as they are packed); `src/ZB.MOM.WW.MxGateway.Server/Configuration/WorkerOptions.cs:57-66` (pipe default = gRPC 16 MiB + 64 KiB reserve); `src/ZB.MOM.WW.MxGateway.Server/Configuration/GatewayOptionsValidator.cs:614-628` (pipe ≥ gRPC + reserve, no upper coupling); `src/ZB.MOM.WW.MxGateway.Server/Security/Authorization/GrpcAuthorizationServiceCollectionExtensions.cs:37` (`MaxSendMessageSize = MaxGrpcMessageBytes`); `src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_gateway.proto:20` (`Invoke` returns the bare `MxCommandReply`).
- **Description:** WRK-21/IPC-23 made the reply fit the *pipe* (R1), but the same `MxCommandReply` is then returned verbatim through `Invoke`, where the gateway's gRPC `MaxSendMessageSize` applies. The two limits are coupled only from below: the validator requires pipe ≥ gRPC + 64 KiB, and the drain budget subtracts the same 64 KiB — so at default configuration the event bytes are budgeted to exactly `MaxGrpcMessageBytes` (16 MiB), leaving the `MxCommandReply` wrapper (34-byte correlation id, protocol status, kind, the `drain_events` length prefix, and — precisely in the truncated case — the `DiagnosticMessage`) with zero margin. The per-event `RepeatedFieldOverheadBytes` slack (+8 per event) absorbs this whenever the packed batch has more than ~30 events, so the default-config overshoot needs a backlog of ≥ 16 MiB composed of a few very large (≥ ~500 KiB) array events — narrow but real. The coupling is entirely absent for the configurations the validator explicitly permits: `Worker:MaxMessageBytes` may be anything up to 256 MiB while `Protocol:MaxGrpcMessageBytes` stays 16 MiB, and then every `DrainEvents` against a backlog above 16 MiB builds a reply of up to `pipeMax − 64 KiB`, which the worker writes, the pipe carries, the gateway parses — and the gRPC layer refuses to send (`ResourceExhausted: Sending message exceeds the maximum configured message size`). At that point the events are already dequeued from the worker queue (`Drain` removes what it packs), so up to 10 000 events are lost to both the diagnostic caller and the event stream, and the caller sees only a failed `Invoke`. This is the IPC-23 invariant ("no diagnostics command may kill a session") satisfied, and its sibling ("no event loss on truncation", R2) violated one boundary later. The same asymmetry exists for single events in the 64 KiB window between the gRPC cap and the pipe cap (an event the pipe accepts but `StreamEvents` cannot send), but a single ≥ 16 MiB event is pathological and already fails loudly on the stream; it is noted, not filed.
- **Recommendation:** Bound the drain against the *smaller* of the two boundaries. Cheapest correct fix without a wire change: the gateway knows both caps, so either (a) add an upper coupling in `GatewayOptionsValidator` (`Worker:MaxMessageBytes ≤ MaxGrpcMessageBytes + 2 × reserve`, or simply document and validate the equality the default already has) and change the worker budget to `MaxMessageBytes − 2 × reserve` so the reply wrapper has a full reserve below the gRPC cap; or (b) convey `MaxGrpcMessageBytes` in `GatewayHello` (additive `uint32`, same zero-means-unknown convention as `max_frame_bytes`) and budget against `min(pipe − reserve, grpc − reserve)`. Add a gateway fake-worker test that drains a reply packed to the budget through the real gRPC pipeline (the TST-24 wire-test harness can host it) so the boundary is exercised end to end.
### IPC-34 — `WorkerCancel` is forwarded only when the gateway's own command timeout fires; a caller-cancelled `Invoke` (client deadline, disconnect) leaves the command queued on the worker's STA
- **Severity:** Low
- **Files:** `src/ZB.MOM.WW.MxGateway.Server/Workers/WorkerClient.cs:269-291` (timeout arm → `TrySendCancelForTimedOutCommand`), `:294-305` (`OperationCanceledException` arm: `RemovePendingCommandAsFailed(GatewayShutdown, "Command wait was canceled.")`, no cancel sent), `:307-318` (timer-won-but-token-cancelled arm, same); `src/ZB.MOM.WW.MxGateway.Server/Grpc/MxAccessGatewayService.cs:138` (`context.CancellationToken` flows into `InvokeAsync`); `src/ZB.MOM.WW.MxGateway.Server/Sessions/GatewaySession.cs:1100`; worker side `src/ZB.MOM.WW.MxGateway.Worker/Sta/StaCommandDispatcher.cs:151-185` (`CancelQueuedCommand` drops only not-yet-started entries); `docs/GatewayProcessDesign.md:596-602`; `gateway.md:913-919`.
- **Description:** The IPC-08 closure's own rationale — "the gateway has stopped waiting, but the worker has not stopped working … a queued-but-not-started command is dropped instead of occupying the STA behind a caller that is already gone" (`WorkerClient.cs:279-284`) — applies identically when the caller's token fires, and that is the *common* abandonment path: gRPC clients routinely set per-call deadlines shorter than the gateway's `CommandTimeout`, and a client disconnect cancels `context.CancellationToken`. Both land in the `OperationCanceledException` arm, which removes the pending entry and returns, so the correlation stays on the single STA queue and executes when its turn comes. Under a burst of deadline-cancelled calls (a client retry storm against a slow provider) the STA works through dead commands while live callers queue behind them — exactly the backlog the cancel was added to shorten. The gateway-side design text still frames cancellation as "stop waiting" only (`gateway.md:915`), and `docs/GatewayProcessDesign.md:596` says "on timeout", so the asymmetry is documented-by-omission rather than decided. Secondary: the arm labels caller cancellation `WorkerClientErrorCode.GatewayShutdown` (pre-existing; maps to `Cancelled` at the gRPC layer per `docs/Grpc.md:285`, so the client sees the right status, but metrics/logs record a shutdown that did not happen).
- **Recommendation:** Call the same no-throw `TrySendCancelForTimedOutCommand` (renamed to `TrySendCancelForAbandonedCommand`, reason "caller cancelled") from the caller-cancellation arms, gated on the same `removed` ownership check so a reply that won the race is never chased. Skip it only when `_stopCts` is the source of the cancellation (the channel is being completed anyway). Add a `WorkerClientTests` twin of the timeout test driven by the caller token. Separately, introduce a `CallerCancelled` error code (or reuse `CommandTimeout` semantics) so the metric label stops saying `GatewayShutdown`; update the `docs/Grpc.md:285` mapping line in the same change.
### IPC-35 — A `WorkerFault` the worker emits during the startup handshake is flattened by the gateway into a generic `ProtocolViolation`, discarding the structured fault category and diagnostic message
- **Severity:** Low
- **Files:** `src/ZB.MOM.WW.MxGateway.Server/Workers/WorkerClient.cs:894-907` (`ReadHandshakeEnvelopeAsync`: any body other than the expected one → `WorkerClientException(ProtocolViolation, "Worker handshake expected {expected} but received {actual}")`), `:164-185` (`StartAsync` expects `WorkerHello` then `WorkerReady`); worker side `src/ZB.MOM.WW.MxGateway.Worker/Ipc/WorkerPipeSession.cs:335-357` (`CompleteStartupHandshakeAsync` writes a `WorkerFault` for every `WorkerFrameProtocolException` and for `MxAccessCreationException` — COM creation failure, the most likely real-world startup fault — before rethrowing), `:1392-1447` (`TryWriteFaultAsync` overloads); `src/ZB.MOM.WW.MxGateway.Server/Sessions/SessionManager.cs:132-160` (the exception message is what reaches `MarkFaulted`, the warning log, and the `SessionManagerException` inner).
- **Description:** The worker deliberately reports startup failures as structured frames — `MxAccessCreationException.From(...)` carries the COM HRESULT/ProgID context and `WorkerFaultCategory` — so the gateway can tell "MXAccess is not installed / not registered / access denied" apart from a pipe-level protocol problem. The gateway's handshake reader does not look inside: it sees `BodyCase == WorkerFault` where it expected `WorkerReady`, throws a `ProtocolViolation` whose message names only the body type, and `SessionManager.OpenSessionAsync` surfaces that text to the operator log and to the client's `OpenFailed`. The post-handshake path does this correctly (`DispatchEnvelope` → `SetFaulted(WorkerFaulted, CreateWorkerFaultMessage(fault))`, `WorkerClient.cs:614-618`), so the information loss is confined to the window in which it matters most — first session open on a freshly deployed host, where the answer is almost always an MXAccess installation or permission problem, and the operator is pointed at "protocol violation" instead. The `FakeWorkerHarness` only ever scripts well-formed handshakes (`Tests/Gateway/Workers/Fakes/FakeWorkerHarness.cs:140-200`), so nothing pins the behaviour either way.
- **Recommendation:** In `ReadHandshakeEnvelopeAsync`, special-case `BodyCase == WorkerFault`: throw `WorkerClientException(WorkerFaulted, CreateWorkerFaultMessage(envelope.WorkerFault))` (the same formatter the steady-state path uses) so category and diagnostic reach the log and the `OpenFailed` detail. Keep `ProtocolViolation` for every other unexpected body. Add a `WorkerClientTests` case where the fake writes a `WorkerFault{Category = MxaccessCreationFailed}` in place of `WorkerReady` and asserts the category text surfaces.
### IPC-36 — The worker launch environment has become a second, undeclared gateway↔worker contract: six variable names duplicated as independent literals on both sides, four of them new since baseline, two documented
- **Severity:** Low
- **Files:** gateway literals `src/ZB.MOM.WW.MxGateway.Server/Workers/WorkerProcessLauncher.cs:18-52` (`MXGATEWAY_WORKER_NONCE`, `…_PIPE_CONNECT_ATTEMPT_TIMEOUT_MS`, `…_WRITE_COMPLETION_WAIT_MS`, `MXGATEWAY_ALARM_POLL_INTERVAL_MS`, `MXGATEWAY_ALARM_MAX_ALARMS_PER_FETCH`, `MXGATEWAY_EVENT_QUEUE_CAPACITY`), set at `:207-217`; worker literals `src/ZB.MOM.WW.MxGateway.Worker/Bootstrap/WorkerOptions.cs`, `Ipc/WorkerPipeClient.cs`, `MxAccess/MxAccessStaSession.cs:14-24`, `MxAccess/MxAccessEventQueue.cs`, `MxAccess/WnWrapAlarmConsumer.cs` (one copy each); the one precedent that *is* shared: `src/ZB.MOM.WW.MxGateway.Contracts/GatewayContractInfo.cs:35-43` (`LiveMxAccessOptInVariableName`, shared "so any future opt-in tweak does not silently leave one project behind"); docs `docs/WorkerProcessLauncher.md:25-27,47-51` (nonce + connect timeout only), `docs/MxAccessWorkerInstanceDesign.md:93-98` ("Expected protected environment values": nonce + log context only); public proto comment `src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_gateway.proto:540` names `MXGATEWAY_WORKER_WRITE_COMPLETION_WAIT_MS`.
- **Description:** Since `4f5371f`, four runtime knobs were plumbed to the worker through `ProcessStartInfo.Environment` rather than through `GatewayHello` — write-completion wait, alarm poll cadence, alarm fetch cap, event-queue capacity. Environment conveyance is a reasonable choice (the values are needed before or independent of the handshake, and the worker's fallback-to-default on a missing variable keeps old/new pairs working), but it makes the launch environment a wire contract in everything but name, and it is treated with none of the discipline the proto contract gets: each name exists as two unrelated string literals in two projects (a typo on either side silently reverts that knob to its default — the worker's documented "missing or unparseable → default" fallback turns a contract break into a no-op), the Contracts project already demonstrates the right home for exactly this kind of name and was not used, and the documentation that should enumerate the contract lists two of the six. The public `mxaccess_gateway.proto` comment additionally exposes the internal env-var name to every language-client consumer, where the operator-facing key is `MxGateway:Worker:WriteCompletionWaitMilliseconds` (the `WorkerOptions.cs:28-34` and launcher `:25-27` doc comments are also stale — they still say "WriteSecured/WriteSecured2" although `b0e65d4` extended the wait to plain `Write`/`Write2`).
- **Recommendation:** Move the six names into a `WorkerLaunchEnvironment` static class in `ZB.MOM.WW.MxGateway.Contracts` (net48-safe `const string`s, like `GatewayContractInfo`) and reference it from both sides; add a one-table "Launch environment" section to `docs/WorkerProcessLauncher.md` (variable, source config key, worker default on absence) and point `docs/MxAccessWorkerInstanceDesign.md:93-98` at it; replace the env-var name in the proto comment with the config key; refresh the two "WriteSecured-only" comments. A small test asserting the launcher sets every constant in the class keeps the table and the code from drifting.
### IPC-37 — The `galaxy_repository.proto` wire-identity with the `ZB.MOM.WW.GalaxyRepository` package is still unguarded, although both descriptors are now loadable in one test process
- **Severity:** Low
- **Files:** `src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csproj:36-41` (retention rationale: the proto is the codegen source for all five clients and must stay wire-identical to the package); `src/ZB.MOM.WW.MxGateway.Server/ZB.MOM.WW.MxGateway.Server.csproj:23` (`ZB.MOM.WW.GalaxyRepository` 0.2.0 — the server serves the package's implementation); `src/ZB.MOM.WW.MxGateway.Tests/Contracts/ClientProtoInputTests.cs:65-70` (walks `GalaxyRepositoryReflection.Descriptor` from *Contracts* only); package proto `~/Desktop/scadaproj/ZB.MOM.WW.GalaxyRepository/src/ZB.MOM.WW.GalaxyRepository/Protos/galaxy_repository.proto` (last changed `5572edd`, 2026-06-23).
- **Description:** Verified at HEAD: the two files differ only in `option csharp_namespace` (comment-stripped `diff`), so the invariant holds today. Nothing keeps it true. The package version moves with the whole `ZB.MOM.WW.*` family (repo memory: a version jump is not evidence the package changed), so a routine `PackageReference` bump can bring a wire change in `galaxy_repository.v1` — a new field, a renamed RPC, a different streaming shape — that the server starts serving while the five clients keep generating from the retained, now-stale copy. Check 1–4 and the descriptor test would all stay green, because they compare the clients against the *retained* proto, never the retained proto against the *served* one. Cycle 1 recorded this as an optional test (`archreview/remediation/30-contracts-ipc.md:340-344`) when it would have required a cross-repo path; that is no longer the case — the test project transitively loads the package through the server reference (`Tests/Configuration/GalaxyRepositoryOptionsValidatorTests.cs` and four other test files already `using ZB.MOM.WW.GalaxyRepository`), so `ZB.MOM.WW.GalaxyRepository.Grpc.GalaxyRepositoryReflection.Descriptor` and `ZB.MOM.WW.MxGateway.Contracts.Proto.Galaxy.GalaxyRepositoryReflection.Descriptor` are both in-process.
- **Recommendation:** Add `GalaxyContractParityTests`: take `FileDescriptor.ToProto()` from both, clear `SourceCodeInfo` and `Options.CsharpNamespace` (and `Name` if the paths differ), and assert byte-equality of the serialized `FileDescriptorProto`. One test, no protoc, runs in the portable CI job, and turns a silent package-bump drift into a red build with the exact differing symbol named. Record the new guard in the csproj retention comment and `docs/Contracts.md`.
### IPC-38 — `docs/Grpc.md` has no coverage of `DrainEvents` although it is a public `MxCommandKind` reachable through `Invoke`; the tracker's reason for omitting it is inaccurate
- **Severity:** Info
- **Files:** `docs/Grpc.md` (grep for `DrainEvents` → 0 hits); `src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_gateway.proto:136` (`DrainEventsCommand drain_events = 103` in the public `MxCommand` oneof), `:520-523`, `:701-706`; `src/ZB.MOM.WW.MxGateway.Server/Grpc/MxAccessGrpcRequestValidator.cs:74-88` (public-boundary validation of `max_events`); `archreview/2026-07-12/remediation/00-tracking.md:87` ("`docs/Grpc.md` intentionally carries no DrainEvents row since it scopes to the seven public gRPC RPCs and DrainEvents is a worker IPC diagnostic"); `gateway.md:565-572` (does document it).
- **Description:** `DrainEvents` is not an IPC-only diagnostic: any client with `invoke:read` can issue it through `Invoke`, the public validator has a rule for it, and the cycle-2 plan (IPC-23 step 5) asked for a rules-table note because the reply is now truncated on two axes with a drain-until-empty contract that callers must know. The tracker closed that step on a premise the proto contradicts. `gateway.md` carries the prose, so the information exists; `docs/Grpc.md` is the document the client READMEs and the rules table point readers at for per-command behaviour, and it is silent.
- **Recommendation:** Add a `DrainEvents` row to the rules table in `docs/Grpc.md` (count cap, byte cap, `DiagnosticMessage` truncation marker, drain-until-empty, and that drained events are removed from the session's event stream), cross-linking `gateway.md:565`. Correct the tracker note.
### IPC-39 — The IPC-03 `CommandTooLarge` pre-check now measures the envelope before GWC-28 stamps the sequence, leaving an ≤ 11-byte window in which the writer's `MessageTooLarge` is still session-fatal; the explanatory comment has the direction of conservatism backwards for the pre-check itself
- **Severity:** Info
- **Files:** `src/ZB.MOM.WW.MxGateway.Server/Workers/WorkerClient.cs:236-249` (pre-check: `CalculateSize()` on an envelope whose `Sequence` is still 0 → the `sequence` field is not encoded), `:465-471` (`WriteLoopAsync` stamps `Sequence` → up to 1 tag + 10 varint bytes added), `src/ZB.MOM.WW.MxGateway.Server/Workers/WorkerFrameWriter.cs:42-55` (post-stamp `CalculateSize()` > max → `MessageTooLarge`), `:478-483` (write-loop exception → `SetFaulted(WriteFailed)`, session-fatal).
- **Description:** The comment at `:236-241` correctly explains why the pre-check size cannot be reused as the writer's `knownSize` ("a lower bound … conservative in the right direction") — true for that purpose, but the same fact inverts for the pre-check's own purpose: a lower bound admits envelopes the writer will reject. An envelope in `(MaxMessageBytes − 11, MaxMessageBytes]` passes `:243` and faults the session at the write. Reachability is nil under the shipped validator: every caller-controlled command arrives through gRPC capped at `MaxGrpcMessageBytes`, which is ≥ 64 KiB below the pipe max, and gateway-authored commands are tiny — so this is a correctness-of-argument note, not a live defect. It is worth recording because the headroom rule is the only thing making it unreachable, and the comment currently reads as if the pre-check were exact.
- **Recommendation:** Either compare against `MaxMessageBytes − MaxSequenceEncodingBytes` (11) in the pre-check, or reword the comment to say the pre-check is a lower bound that relies on the 64 KiB reserve for its safety margin. No test needed beyond the existing `CommandTooLarge` case.
## Severity roll-up (new findings)
| Severity | Count | IDs |
|----------|:-:|-----|
| Critical | 0 | — |
| High | 0 | — |
| Medium | 0 | — |
| Low | 5 | IPC-33, IPC-34, IPC-35, IPC-36, IPC-37 |
| Info | 2 | IPC-38, IPC-39 |
Prior-finding verification: 8/8 cycle-2 Done claims hold (IPC-23 with the IPC-33 residual); IPC-31 superseded by GWC-28 (recommend flipping to Resolved); cycle-1 IPC-08 substantially resolved by `7c1ea12` (recommend closing, residual tracked as IPC-34); IPC-10/11/12 confirmed still open, none regressed.
## Still good / preserve under change
The contract boundary is in the best shape it has been across three cycles. Proto evolution since baseline was purely additive with correct next-free tags and reserved ranges intact, and — for the first time — every generated artifact (C# `Generated/`, Go, Python, Java, Rust vendored copies, the protoset) moved in the *same commit* as the proto, including for a comment-only edit; the freshness net that makes this checkable is now genuinely closed (Checks 1–4 + the reflection test + the Java `git diff` gate, all in CI, all tool-missing-fails-not-skips, all with their generator pins asserted rather than assumed). The frame protocol core is unchanged where it should be — length validated before allocation on both readers, exact-read loops with typed `EndOfStream`, pooled buffers with sound lifetimes on all four codec halves, single-serialization single-write framing — and materially stronger where it changed: both sides now stamp `sequence` at the point of writing so the documented "monotonic per sender" contract is finally true on both ends, the negotiated `max_frame_bytes` handshake has matching `[1024, 256 MiB]` bounds on both sides so a legal gateway configuration can never be refused by the worker, the DrainEvents reply is bounded on count *and* bytes with no event loss at the pipe, oversized events die structured and named, and cancelled writes are guaranteed never to reach the wire. `WorkerCancel` is no longer a dead arm. `docs/WorkerFrameProtocol.md` is now a faithful, claim-by-claim description of the shipped writer, and `gateway.md`'s envelope section points at the proto as the single source of truth instead of carrying a drifting copy. Preserve in particular: the per-client generator pins and the "tool missing = check fails" rule (a skipped guard is the silent-drift hole the whole P0 codegen batch existed to close); the pipe-over-gRPC 64 KiB reserve invariant in the validator (IPC-33 and IPC-39 both rest on it); the habit of landing proto-comment changes with the full regen wave; and the csproj retention comment on `galaxy_repository.proto` — until IPC-37's test exists, that comment is the only thing standing between a package bump and a five-client wire drift.
## Verification needed on Windows (windev)
- This review is static. The worker-side halves cited here (byte-budgeted drain, IPC-30 structured death, tombstones, the floor/ceiling handshake rejection) are exercised only by `Worker.Tests -p:Platform=x86`; the tracker and repo memory record the suite fully green on windev after the harness fixes, which this review did not re-run.
- IPC-33's end-to-end shape (reply built at budget → `Invoke` send failure) cannot be reproduced with the fake worker alone on macOS if the fix lands worker-side; the validator/gateway-side variant (option a) is portable and should get a `GatewayOptionsTests` case plus a TST-24-style wire test.
- IPC-35's regression test is portable (fake worker writes a handshake-phase `WorkerFault`); no windev run needed.
@@ -0,0 +1,110 @@
# Security & Dashboard — Re-review (Cycle 3)
**Date:** 2026-08-19 · **Commit:** `8b3e3b3` (`main`, clean) · **Baseline:** `4f5371f` (2026-07-12 cycle) · **Method:** static review on the macOS working tree (no builds, no runtime probes, no source modifications). Every claim cites a `path:line` read at HEAD. Prior findings are the `SEC-NN` IDs from `archreview/remediation/40-security-dashboard.md` (cycle 1) and `archreview/2026-07-12/40-security-dashboard.md` / `archreview/2026-07-12/remediation/40-security-dashboard.md` (cycle 2); new findings continue from SEC-37. This domain absorbed the most new machinery since the baseline (~55 of ~247 commits touched Security/Dashboard/Configuration): the SEC-31/32 failure-limiter re-partition, SEC-33/34/36 remediations, the SEC-25 tag-ACL epic (`DashboardTags` constraints, session tag inheritance, `GroupToTag`/`UntaggedSessionVisibility`, events-hub + in-process ACL), the `ShowTagValues` three-seam closure, the secrets-store adoption (`${secret:}` expansion, `/admin/secrets`), the bounded async audit writer with retention, and the snapshot-feed/alarm-page reliability rework.
## Prior-finding verification
### Cycle-2 findings (SEC-31…SEC-36)
| ID | Claimed | Verified? | Evidence (file:line at HEAD) | Notes |
|----|---------|-----------|------------------------------|-------|
| SEC-31 | Done | **Yes** | Composite `(transport peer, key id)` partition + cross-peer aggregate + probe admission, exactly per the remediation design: `Security/Authorization/ApiKeyFailureLimiter.cs:14-27` (two-layer remarks), `:143-204` (`Check` with probe claim/compensation), `:206-235` (`RecordFailure` into both layers), `:237-276` (`Reset` reachable-while-throttled); interceptor threads the partition and reuses the check-time resolution (`GatewayGrpcAuthorizationInterceptor.cs:83-115`, `:133-137`); new knobs validated (`SecurityOptions.ApiKeyFailureAggregateLimit`/`ApiKeyFailureProbeIntervalSeconds`, wired at `ApiKeyFailureLimiter.cs:74-83`); throttle metric tagged by stage only (`Metrics/GatewayMetrics.cs:350-356`, with an explicit SEC-14/SEC-20 cardinality comment). `ApiKeyFailureLimiterTests.cs` exists. The lockout inversion is structurally closed: a correct secret always reaches the verifier via a probe slot and resets both layers. | Probe-claim compensation is version-stamped (`:341-383`), closing the same-tick stomp the follow-up commits (`5b681ee`, `acebe18`) addressed. Two new residual corners found in the rework — see **SEC-38** (aggregate-layer bypass via the per-peer key-id cap) and **SEC-39** (null==null fallback-partition reset). Neither reopens the SEC-31 lockout: they weaken bounds, they do not deny the legitimate holder. |
| SEC-32 | Done | **Yes** | Token shape checked before a key-id partition is minted: `GatewayGrpcAuthorizationInterceptor.cs:149-193` (`TryResolveKeyId` — `mxgw` must be the entire first segment, key id ≤ 64 chars, non-empty third segment; internal for direct test pinning). Per-peer key-id cap `MaxKeyIdPartitionsPerPeer = 32` with overflow collapsing to the address's fallback partition (`ApiKeyFailureLimiter.cs:48-53`, `:393-444`). Eviction never removes load-bearing state below a 2× ceiling and prefers expired windows (`:479-548`). | The junk-token flush is closed as designed. The cap's overflow-collapse rule has a side effect the design did not anticipate — see **SEC-38**. |
| SEC-33 | Done | **Yes** | `IsRootedForAnyPlatform` is gone; rooting is host-meaningful `Path.IsPathRooted` in the shared `Configuration/GatewayConfigPathRules.cs:14-39` (used by both validators, with the SEC-01/SEC-33 mechanism named in the doc-comment). `appsettings.json` no longer carries either Windows literal (Authentication section `:20-24` has no `SqlitePath`; Galaxy section `:79-84` has no `SnapshotCachePath`). Gateway-side Galaxy default seeded pre-bind (`GatewayApplication.cs:156-168` — the init-only-property deviation is documented in place) and validated by the new `Configuration/GalaxyRepositoryOptionsValidator.cs:44-72` registered with `ValidateOnStart` (`GatewayApplication.cs:172-176`). No stray `C:*` artifact under `src/` (`find`-verified). | Went beyond the finding: `882c7ca` added `AddIfUnderContentRoot` (`GatewayConfigPathRules.cs:93-124`) after a real 2026-08-09 production key-loss incident (store inside the renamed app directory), and the secrets store got the same absolute default treatment (`GatewayApplication.cs:201-239`). |
| SEC-34 | Done | **Yes** | Invalidate race closed with the per-key generation counter: bump-before-evict (`Security/Authentication/CachingApiKeyVerifier.cs:181-199`), snapshot-before-inner + set-then-recheck with self-evict (`:128-141`, `:147-178`); `TryParseKeyId` correctness argument (full key id, `_` forbidden by both key-creation paths) documented at `:205-218`. Expiry window took the documented fallback per the recorded outcome: the library verification identity carries no `ExpiresUtc`, so the ≤ TTL expiry window is documented in the remarks (`:49-60`) with the donor-library ask. | Matches the remediation outcome exactly, including the deviation. Cache-key design unchanged (SHA-256 of the token, success-only). |
| SEC-35 | N/A (doc-only) | **Yes** | `docs/GatewayConfiguration.md:282` — the "hard-stops key on the exact environment name" contract is documented (both guards fire only on `IHostEnvironment.IsProduction()`). Validator unchanged: `GatewayOptionsValidator.cs:38`, `:211-214` (LDAP), `:387-391` (DisableLogin). | Note the consequence is now live: windev runs as `Staging` (NEXT-07 relabel), so neither guard applies there — that is the posture gap promoted as **SEC-37**. |
| SEC-36 | Done | **Yes (repo side + rotation executed)** | `appsettings.json:33` ships `"ServiceAccountPassword": "${secret:ldap/mxgateway/bind}"` — a fail-closed encrypted-store reference expanded pre-host (`GatewayApplication.cs:75-106`), not a literal. `git grep` for the retired value returns nothing in tracked files. `glauth.md` samples use placeholders; `docs/runbooks/SEC-36-ldap-credential-rotation.md` records the 2026-08-07 rotation (GLAuth `passsha256` replaced in `scadaproj` commit `aada53b`, instance recreated, dashboard `/login` verified through the real `DashboardAuthenticator` path post-NEXT-07). `<UserSecretsId>mxaccessgw-server</UserSecretsId>` present in the csproj; the blank-password validator message names both channels. | The load-bearing half (rotation) is done: the historically recoverable value no longer binds `dc=zb,dc=local`. Fully closed. |
### Cycle-1 residuals (still-open SEC-13…SEC-27 era)
| ID | Prior status | At HEAD | Evidence | Notes |
|----|--------------|---------|----------|-------|
| SEC-13 | Open | **Open, impact reduced** | `Diagnostics/GatewayLogRedactor.cs:11-17` still lists only `AuthenticateUser`/`WriteSecured`/`WriteSecured2` — the `WriteSecuredBulk`/`WriteSecured2Bulk` command kinds (`Grpc/MxAccessGatewayService.cs:364-381`) are still missing. However the redactor is no longer latent: `GatewayLogRedactorSeam` is registered as the telemetry `ILogRedactor` (`GatewayApplication.cs:139`) and masks **every** non-null `CommandValue` because the seam exposes no value-logging opt-in (`Diagnostics/GatewayLogRedactorSeam.cs:38-56`; honestly documented at `docs/Diagnostics.md:120`). The Bulk omission only bites if a future opt-in knob passes `valueLoggingEnabled: true`. | Fix the two missing names when (if) the opt-in knob lands; until then fail-closed. |
| SEC-14 | Open | **Confirmed open** | `GatewayApplication.cs:347-348` — `MapZbHealth()`/`MapZbMetrics()` with no `RequireAuthorization`. | Mitigating discipline held: the new throttle counter deliberately carries no key/peer tags because "/metrics is not authenticated (open SEC-14)" (`GatewayMetrics.cs:352-354`). Note `/health/active` has off-repo consumers (family Overview dashboard), so any auth change needs cross-repo coordination. |
| SEC-15 | Open (accepted) | **Confirmed open** | GET `/logout` still skips antiforgery, intentional (`Dashboard/DashboardEndpointRouteBuilderExtensions.cs:92-99`). | Accepted risk; unchanged. |
| SEC-16 | Open | **Confirmed open** | `Program.cs:37-39` still maps `command.Pepper` → `MxGateway:ApiKeyPepper` (pepper on the CLI command line). | |
| SEC-17 | Open | **Confirmed open** | `GatewayGrpcAuthorizationInterceptor.cs:27-54` overrides only unary + server-streaming. All current RPCs remain those shapes. | |
| SEC-18 | Open | **Confirmed open** | `Dashboard/DashboardApiKeyManagementService.cs:22,289` — `"pepper unavailable"` message-substring match. | |
| SEC-19 | Open | **Resolved** (this period) | `Security/Audit/SqliteCanonicalAuditStore.cs:23-31` — the `CREATE TABLE IF NOT EXISTS` is latched: `AuditDrainService` runs `EnsureInitializedAsync` once at startup and later operations skip the DDL round-trip (self-bootstrap kept for the CLI/DI-free paths; the latch is deliberately racy-idempotent). Landed with the `e2ac5d1`/`7b2d046` audit rework (bounded async writer, batched inserts). | |
| SEC-21 | Open | **Resolved** (this period) | `Dashboard/DashboardSnapshotService.cs:25,38,42,287-310` — API-key summaries refresh at most every 15 s behind a zero-timeout gate with a 2 s store timeout, and the snapshot tick itself is idle-gated (`77c5731`, `DashboardSnapshotFeed` idle gate). The per-tick unconditional store read is gone. | |
| SEC-23 | Open | **Confirmed open** | `GatewayOptionsValidator.ValidateDashboard` (`:380-455`) gained GroupToRole/GroupToTag/UntaggedSessionVisibility shape checks and the Production `DisableLogin` stop, but still ignores `AutoLoginUser`, `RequireHttpsCookie`, and `CookieName` — an explicit `CookieName` starting `__Host-` with `RequireHttpsCookie=false` is still applied unconditionally (`DashboardServiceCollectionExtensions.cs:154-158`) with no validator diagnosis. | The runtime guard against *deriving* `__Host-` without Secure is correct (`:159-162`); only the explicit-override consistency check is missing. |
| SEC-24 | Open | **Confirmed open (narrowed)** | `Configuration/GatewayConfigurationProvider.cs:58-74` / `EffectiveDashboardConfiguration.cs:3-12` — the projection gained `ShowTagValues`, `GroupToRole`, `GroupToTag` (with comparer propagation, `ff16c69`), and `UntaggedSessionVisibility`, and the settings page renders them (`SettingsPage.razor:53-104`); it still omits `DisableLogin`/`AutoLoginUser`/`RequireHttpsCookie`/`CookieName` and the `MxGateway:Security` limiter keys the SEC-31 design asked to absorb here. Secrets stay redacted (`:20,30`). | |
| SEC-26 | Open | **Resolved** (this period) | Retention sweep exists: `SqliteCanonicalAuditStore.cs:172` (`DELETE FROM audit_event` with a documented NULL-timestamp keep rule at `:251-253`), driven by `SecurityOptions.AuditRetentionDays` (default 90, `SecurityOptions.cs:96-99`) from the audit drain service. | |
| SEC-27 | Open | **Confirmed open** | `DashboardSnapshotService.cs:17,114` — `GatewayStatus` is still the hardcoded `"Healthy"` constant. | |
### SEC-25 epic verification (the tag-ACL machinery is real and fail-closed)
Not a table row above because cycle 2 closed SEC-25 as "mitigation, ACL deferred"; the deferred ACL has now shipped and was reviewed as new code:
- **Tags come from the owning key, never the request:** `Grpc/MxAccessGatewayService.cs:36-46` (owner identity from `identityAccessor`, `owner?.EffectiveConstraints.DashboardTags` into `OpenSessionAsync`); `Sessions/GatewaySession.cs:207-212,270` (immutable ordinal-ignore-case `FrozenSet`); `Security/Authentication/ApiKeyConstraints.cs:35-44` (defensive copy on init; rides the constraints blob to avoid a schema migration; explicitly visibility-only, never data-access).
- **ACL decision table:** `Dashboard/DashboardSessionAcl.cs:50-84` — Admin (authenticated) bypass; phantom session denied; untagged session follows `UntaggedSessionVisibility` (default `AdminOnly`, fail-closed); otherwise ordinal-ignore-case tag intersection over `DashboardTagClaimType` claims; `null` principal denied outright, even under `AllViewers` (`IDashboardSessionAcl.cs:25-32`). Anonymous localhost is an empty-grant Viewer.
- **Both subscribe seams gated identically:** `Dashboard/Hubs/EventsHub.cs:59-79` (ACL before group join, registry-before-group ordering, opaque `HubException`), and the session-details in-process seam `SessionDetailsPage.razor:321-361` (ACL before any field write or `Subscribe`). The `7ec0b35` re-entrancy fix is sound: the attach generation is claimed synchronously before the first await (`:217-233`), a stale continuation returns without attaching or detaching (`:340-343`), and the pump's `ReferenceEquals` batch guard drops cross-session batches on the dispatcher (`:383-407`).
- **Claims plumbing:** tags stamped at login (`DashboardAuthenticator.cs:114-158`) and re-resolved from group claims at hub-token mint so a token bounds grant staleness to 5 minutes (`HubTokenService.cs:80-98`); token `Validate` still rejects identity-less payloads (`:118-126`) and rehydrates tag claims for the hub-path ACL (`:139-145`).
- **Owner-scoped gRPC attach (the threat-model anchor) holds:** `Grpc/EventStreamService.cs:52-63` — reattach is `PermissionDenied` unless the caller's key id ordinal-equals the session owner; null owner matches only a null caller.
- **Redaction (second, independent layer) covers all three value seams:** events-hub mirror deep-clone (`Dashboard/Hubs/DashboardEventBroadcaster.cs:91,247-259` — top-level `value` plus alarm `CurrentValue`/`LimitValue`, source never mutated), alarms hub deep-clone on both value-bearing arms (`AlarmsHubPublisher.cs:59,103-126` — the cycle-2 closeout's named "alarms-hub redaction gap" residual, closed by `eff17d1`/TST-16 on 2026-08-17), and `/browse` live values at the service boundary with the `'-'`-placeholder-on-failed-reads fix (`DashboardLiveDataService.cs:132-140`; `1ea6f60`). `docs/GatewayConfiguration.md:195` documents all three seams and the "redaction applies to successful reads only" rule accurately.
- **Tests exist for the corners:** `Tests/Gateway/Dashboard/DashboardSessionAclTests.cs`, `EventsHubTests.cs`, `SessionDetailsPageEventAclTests.cs`, `EventsHubViewerRegistryTests.cs`, `HubTokenServiceTests.cs`, `AlarmsHubPublisherTests.cs`, `DashboardGroupTagMappingTests.cs`; live-LDAP ACL proof via `IntegrationTests/DashboardLdapLiveTests.cs:186-243` (real bind → real group claims → GroupToTag → ACL admit/deny, plus Admin bypass).
- **Docs honesty:** the short-name `GroupToRole`/`GroupToTag` key constraint (full-DN keys can never match the pre-stripped groups) is stated at the mapping site (`DashboardGroupTagMapping.cs:8-13,41-47`) and in the class summaries (`db7b1db`, `4b6f907`).
## New findings
### SEC-37 · Medium (Security / deployment posture) — The shared GLAuth offers no TLS, so the Production plaintext-LDAP hard-stop is unsatisfiable for every host that authenticates against it (promotion of NEXT-08)
**Files:** `src/ZB.MOM.WW.MxGateway.Server/Configuration/GatewayOptionsValidator.cs:208-214`; `docs/GatewayConfiguration.md:264-265`; `glauth.md:30,357-360`; `archreview/2026-07-12/remediation/90-candidate-findings-next-cycle.md` (NEXT-08).
The SEC-06 guard refuses to start a `Production`-environment host with `Ldap:Transport=None`, and the config doc instructs "Deployed hosts must set `Ldaps` or `StartTls`" — but the only shared directory (`10.100.0.35:3893`) has LDAPS disabled, port 3894 closed, and answers StartTLS with `protocolError` (per the NEXT-07/08 record), and `glauth.md:30` documents "LDAPS | disabled in dev". Neither permitted value can work, so the instruction is unsatisfiable: every deployed gateway that binds this directory must either run plaintext (which `glauth.md:358-360` itself warns sends passwords cleartext) or avoid the `Production` label. That is exactly what happened — windev was relabelled `Staging` on 2026-08-07 to boot past the guard. The net posture is that the hard-stop protects no real deployment: hosts that would trip it are renamed around it, and the SEC-35 documentation note now formalizes the escape hatch. The dashboard admin surface (API-key CRUD, session Kill, secrets UI) sits behind credentials that transit this plaintext bind.
**Recommendation:** resolve the posture, in one of two written directions: (a) enable LDAPS (or StartTLS) on the shared GLAuth — certificate issuance plus a trust story on each gateway host, source of truth `scadaproj/infra/glauth/` — and flip the deployed hosts' `Transport`; or (b) record an explicit decision that production gateways bind a different, TLS-capable directory and that the shared GLAuth is dev/staging-only, and say so in `docs/GatewayConfiguration.md`'s `Transport` row and `glauth.md`. Until one lands, every "must use Ldaps/StartTls" sentence is aspirational and should be labeled as such. (Cross-repo: coordinate with the family per the umbrella-index rule.)
### SEC-38 · Low (Security) — The failure limiter's cross-peer aggregate layer can be sidestepped by pre-filling the per-peer key-id cap, restoring the `limit × addresses` guessing bound the layer was built to remove
**Files:** `src/ZB.MOM.WW.MxGateway.Server/Security/Authorization/ApiKeyFailureLimiter.cs:225-231` (aggregate fed only when `effectiveKeyId` is non-null), `:393-444` (`ResolvePartitionKey` returns the `(peer, null)` fallback once the address has minted `MaxKeyIdPartitionsPerPeer = 32` key ids), `:153-166` (`Check` consults the aggregate only via `effectiveKeyId`).
The SEC-32 cap collapses the 33rd-and-later key id from one address onto that address's fallback partition, and — per the comment's cardinality rationale — such failures deliberately do not feed the per-key-id aggregate. The side effect: an attacker who first mints 32 junk `mxgw`-shaped key-id partitions from an address (32 requests) makes every subsequent guess against the *victim's real key id* from that address resolve to `(peer, null)` — so those failures neither feed the victim's aggregate (`RecordFailure`, `:227`) nor consult it (`Check`, `:158-163`). A source-rotating attacker (the IPv6-/64 scenario the aggregate was designed for, per the SEC-31 remediation design's G1 analysis) thereby recovers `ApiKeyFailureLimit` guesses per window per address with the aggregate fully bypassed, at ~32 extra requests per address — instead of the designed bound of `ApiKeyFailureAggregateLimit` total followed by ~1 probe per interval globally. Severity is Low, not Medium: gateway secrets are library-generated high-entropy values, so online guessing remains computationally hopeless and the limiter is defense-in-depth plus store-read cost shielding (which still holds per-address); the finding is that the *documented* distributed-attack bound (`ApiKeyFailureLimiter.cs:17-19`, `docs/Authentication.md` limiter section) is not the actual bound.
**Recommendation:** feed (and consult) the aggregate for any *validly shaped* key id regardless of whether it earned a composite partition — the cardinality concern the comment cites is already handled by `_aggregates`' own capped eviction (`:234`), whose policy protects over-limit entries and preferentially evicts single-failure junk. Alternatively, count cap-overflow failures into the aggregate of the presented key id only when that aggregate already exists (zero new cardinality, closes the consult-bypass for hot keys). Update the class remarks and `docs/Authentication.md` either way.
### SEC-39 · Info (Security) — `Reset` treats "no key id" as owning the shared fallback partition: a verified token that fails the interceptor's key-id parse clears the address's fallback state the guard comment promises to preserve
**Files:** `src/ZB.MOM.WW.MxGateway.Server/Security/Authorization/ApiKeyFailureLimiter.cs:252-276`; `src/ZB.MOM.WW.MxGateway.Server/Security/Authorization/GatewayGrpcAuthorizationInterceptor.cs:149-193`.
`Reset`'s ownership check is `string.Equals(effectiveKeyId, partition.KeyId)` (`:264`), which is true when both are null — and a partition with `KeyId = null` resolves to the transport-peer fallback key (`:398-403`), so the reset then `RemovePartition(peer)`s the *shared* fallback bucket, the exact wipe the comment above it (`:259-263`) explains must not happen because that bucket holds junk-spray failures from the same address. A null `KeyId` on a *successful* verification requires a token the library verifier accepts but `TryResolveKeyId` rejects — the identifiable shapes are a key id longer than 64 chars (`:178`) and a secret whose first character is `_` (`:187`, matching the old `Split`-based rejection); whether the shared verifier accepts either depends on library parsing not readable in this repo. Practical impact is minimal (the attacker must hold a *valid* key of an unusual shape merely to clear their own address's junk-spray throttle), hence Info — but the null==null branch contradicts the stated invariant and costs one line to close.
**Recommendation:** make the fallback non-resettable explicitly: `if (effectiveKeyId is not null && string.Equals(effectiveKeyId, partition.KeyId, StringComparison.Ordinal))`, and pin with a test (successful reset with a null-key-id partition leaves the fallback partition tracked).
### SEC-40 · Low (Security) — Alarm visibility is not tag-scoped: the session-less alarms hub (and alarms page) broadcasts galaxy-wide alarm metadata to every Viewer, including empty-grant Viewers and anonymous localhost
**Files:** `src/ZB.MOM.WW.MxGateway.Server/Dashboard/Hubs/AlarmsHub.cs:12-25` (`[Authorize(HubClientsPolicy)]`, auto-join `__alarms__` on connect); `AlarmsHubPublisher.cs:48-63` (unfiltered `StreamAsync(alarmFilterPrefix: null)` re-broadcast to all clients); `Dashboard/DashboardAuthorizationHandler.cs:39-53` (loopback/disabled bypasses satisfy the Viewer requirement the hub policy uses); `docs/GatewayDashboardDesign.md:277` ("all clients receive every message").
The SEC-25 ACL scopes *session event* visibility by tag intersection, and CLAUDE.md/docs describe dashboard event visibility as tag-scoped — but the alarm feed is a parallel, session-less surface with no per-caller scoping at all: any principal satisfying `HubClientsPolicy` (any Viewer role, the `Authentication:Mode=Disabled` bypass, or anonymous localhost under the default `AllowAnonymousLocalhost=true`) receives every `AlarmFeedMessage` the central monitor emits. Values are stripped by default (`ShowTagValues`, verified above), but the surviving metadata — alarm reference (a full tag path), state, severity, category, area, and acknowledging-operator usernames — discloses process structure and operator activity for the whole subscription expression to callers the tag model would deny every single session. A Viewer with an empty tag grant, whom the ACL was explicitly designed to show nothing, still sees the full plant alarm picture. The alarm monitor being gateway-central (not per-session) means tag scoping does not map mechanically, so this may well be a deliberate posture — but no design doc states it against the ACL: `GatewayDashboardDesign.md:277` describes the fan-out without a scoping rationale, and the ACL plan (`docs/plans/2026-07-10-dashboard-session-acl-tst15.md`) does not carve alarms out.
**Recommendation:** decide and document. If deliberate, add one paragraph to the ACL section of `docs/GatewayDashboardDesign.md` (and the `GroupToTag` row of `docs/GatewayConfiguration.md`) stating that alarm metadata is intentionally visible to every dashboard Viewer and why (central monitor, no per-session provenance). If not, the cheap tightening is a dedicated policy on `AlarmsHub` (e.g. require a real role claim, excluding the anonymous-loopback bypass) or an area/tag-prefix filter derived from the caller's grant.
### SEC-41 · Low (Security) — Tag grants revoked by a `GroupToTag` config change survive on cookie principals across the restart that applies the change: the ACL trusts login-stamped tag claims for up to the 8-hour sliding cookie life, while the hub-token path re-resolves at mint
**Files:** `src/ZB.MOM.WW.MxGateway.Server/Dashboard/DashboardAuthenticator.cs:145-149` (tags stamped as claims at login); `Dashboard/DashboardSessionAcl.cs:75-81` (ACL reads the *stamped* tag claims); `Dashboard/HubTokenService.cs:84-90` (token mint *re-resolves* tags from group claims against current options — the asymmetry); `Dashboard/DashboardServiceCollectionExtensions.cs:114-130` (`ZbCookieDefaults.Apply(..., idleTimeout: 8h)` — sliding, so an active session renews indefinitely).
`GatewayOptions` binds once at startup (no reload — noted at `DashboardSnapshotService.cs:45-49`), so changing `GroupToTag` requires a gateway restart. A restart tears down circuits and hub connections, but **not** the auth cookie: data-protection keys persist, and a browser returning after the restart re-authenticates from the cookie, whose `DashboardTagClaimType` claims were resolved under the *old* map at login. `DashboardSessionAcl` consults exactly those claims, so a user whose group's tag grant was removed from config keeps observing the corresponding sessions' event streams — hub joins and the session-details in-process seam alike — until they log out or the sliding 8-hour window finally lapses, renewable indefinitely by activity. The hub-token seam does not have this gap for the *grant* (mint re-resolves from group claims, bounding staleness to 5 minutes, exactly as its comment at `:41-47` claims) — but that comment's promise that "revoking a GroupToTag entry takes effect ... within one lifetime" holds only for token-authenticated connections, and the cookie-authenticated in-process seam it shares an ACL with is the weaker path. LDAP group-membership revocations have the same 8-hour cookie bound on both paths (standard cookie-auth behavior, but worth stating for a *security grant*).
**Recommendation:** have the ACL (or a claims transformation) re-resolve tags from the principal's `LdapGroupClaimType` claims against current options — the same one-call `DashboardGroupTagMapping.MapGroupsToTags` the token mint already uses — instead of trusting login-stamped tag claims; the group claims are already on the cookie. That makes a config-revoked grant take effect at the next subscribe after restart, symmetrically with the token path. At minimum, document the revocation bounds (restart + re-login for cookies, 5 minutes for tokens, never for an already-open subscription) in `docs/GatewayDashboardDesign.md`'s ACL section.
## Severity roll-up (new findings)
| Severity | Count | IDs |
|----------|-------|-----|
| Medium | 1 | SEC-37 |
| Low | 3 | SEC-38, SEC-40, SEC-41 |
| Info | 1 | SEC-39 |
Prior-backlog movement this period: SEC-19, SEC-21, SEC-26 resolved; SEC-31…SEC-36 all verified Done (SEC-36 including the operator rotation); still open from cycle 1: SEC-13 (impact reduced — redactor now wired fail-closed), SEC-14, SEC-15 (accepted), SEC-16, SEC-17, SEC-18, SEC-23, SEC-24 (narrowed), SEC-27.
## Still good / preserved
The load-bearing controls all survive at HEAD, several strengthened:
- **Fail-closed authorization core:** unrecognized request → `admin` scope (`GatewayGrpcScopeResolver`); opaque auth failures that never disclose the failing stage (`GatewayGrpcAuthorizationInterceptor.cs:96-109`); constant-time compare stays inside the shared verifier on every cache miss with only SHA-256 digests in memory (`CachingApiKeyVerifier.cs:36-39,284-286`); the throttle message reveals nothing about secret validity.
- **The SEC-25 ACL is genuinely fail-closed on every branch** (unknown session, empty grant, untagged-session default `AdminOnly`, null principal), enforced identically on both subscribe seams, with the decision-table corners and the live-LDAP group-shape proof pinned by tests; session tags are immutable and inherited only from the owning key's constraints, never the client request; the gRPC event-stream attach stays owner-scoped (`EventStreamService.cs:52-63`).
- **Read-only environment bypasses stay read-only:** the `Disabled`-mode and anonymous-loopback bypasses satisfy only requirements containing Viewer, never `AdminOnly` (`DashboardAuthorizationHandler.cs:37-53`); API-key CRUD and session Close/Kill keep their service-layer Admin re-checks (`DashboardApiKeyAuthorization.cs:10-18`, `DashboardSessionAdminService`) beneath the policy layer; the secrets UI mounts behind its own `secrets:manage` policy (`GatewayApplication.cs:147-150`), which the dashboard bypass handler cannot satisfy (different requirement type).
- **Value redaction is now a complete, tested, three-seam system** (events mirror, alarms hub, `/browse`), clone-only so gRPC clients and the replay ring are untouched, with the config doc telling the truth about all three seams and the failed-read placeholder rule.
- **Secret handling improved materially:** the committed LDAP literal is gone and rotated; config secrets ride fail-fast `${secret:}` references expanded pre-host; the effective-config projection redacts pepper name and LDAP password; hub tokens remain 5-minute, data-protected, identity-required; no new call site logging credentials, tag values, or grant contents was found in the changed files (the ACL logs decisions, never tag values — `DashboardAuthenticator.cs:145-146`).
- **Credential-store paths are now defended in depth:** host-meaningful rooting, the under-content-root rule born of a real outage, and absolute defaults for all three stores (auth DB, secrets store, Galaxy snapshot) via one shared rule set (`GatewayConfigPathRules`).
- **Dashboard hygiene held:** local Bootstrap only (no UI component libraries in any new page); antiforgery on POST login/logout; `SanitizeReturnUrl` still blocks open redirects (`DashboardEndpointRouteBuilderExtensions.cs:238-249`); the login limiter stays remote-IP-partitioned; the viewer registry / snapshot feed / broadcaster concurrency reworks (`EventsHubViewerRegistry.cs`, `DashboardSnapshotFeed.cs`, `DashboardEventBroadcaster.cs`) are careful, generation-guarded code with the subtle orderings documented in place and covered by dedicated tests.
+104
View File
@@ -0,0 +1,104 @@
# Language Clients — Architecture Review (Cycle 3)
**Date:** 2026-08-19 · **Commit:** `8b3e3b3` (main) · **Baseline:** `4f5371f` (2026-07-12 cycle) · **Method:** static review on macOS (no builds, no tests run)
Scope: `clients/dotnet` (lib + CLI + tests), `clients/go` (mxgw-go), `clients/java`, `clients/python`, `clients/rust` (lib + CLI + build.rs + vendored protos), cross-client parity, `docs/CrossLanguageSmokeMatrix.md`, `docs/ClientLibrariesDesign.md`, `docs/ClientPackaging.md`, `docs/ClientBehaviorFixtures.md`, client READMEs, `scripts/pack-clients.ps1` / `scripts/tag-go-module.ps1`. Prior findings tracked as CLI-01..34 (`archreview/remediation/00-tracking.md`) and CLI-35..45 (`archreview/2026-07-12/remediation/50-clients.md` + `00-tracking.md`), plus NEXT-02 (`archreview/2026-07-12/remediation/90-candidate-findings-next-cycle.md`).
Since 4f5371f the client family absorbed: the full CLI-35..45 remediation train, NEXT-02, the 0.2.0/0.2.1 family publish (2026-08-07, `5b153da`), TST-24 real-wire tests for .NET and Python (`a8f86b5`), alarm truncation-flag regen (`d9ea8a8`), the feed-level `snapshot_status` rollout (`c748361` + `4d4f443`), StreamAlarms open-sequence doc truth (`c3c603f`), and Gradle-9 survival for `checkGeneratedClean` (`df45cb4`).
## Prior-finding verification
### 2026-07-12 cycle (CLI-35..45) and NEXT-02
| ID | Claimed status | Verified? | Evidence (file:line at HEAD) | Notes |
|----|----------------|-----------|------------------------------|-------|
| CLI-35 | Done | **Yes** | `clients/python/src/zb_mom_ww_mxgateway_cli/commands.py:24` (imports `ReplayGap`), `:1660-1666` (`_event_row` branches on `isinstance(item, ReplayGap)` and emits `{"replayGap": {"requestedAfterSequence": …, "oldestAvailableSequence": …}}`); regression `clients/python/tests/test_cli.py:935` (`test_stream_events_renders_replay_gap`, asserts exit 0, the gap row at index 0, the normal event row after, `:971-974`) | Exactly the designed fix, camelCase row matching the cross-CLI shape. |
| CLI-36 | Done | **Yes** | `clients/go/cmd/mxgw-go/main.go:1012-1029` (`if result.IsReplayGap()` → JSON `replayGapCursors` struct `:70`, text `REPLAY_GAP requested_after=%d oldest_available=%d`); test `clients/go/cmd/mxgw-go/main_test.go:734` (`TestRunStreamEventsPrintsReplayGap` over a real fake gateway `:698-713`, both output modes) | Gap counted toward `-limit` as designed. |
| NEXT-02 | Resolved 2026-08-10 | **Yes** | .NET `clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli/MxGatewayClientCli.cs:1425-1429` (stream loop branch), `:1845-1846` (`EventToJsonElement` inside `--json` aggregate), `:1858-1873` (`FormatReplayGapRow` — hand-built so cursors are JSON numbers, not proto-JSON uint64 strings); Java `clients/java/zb-mom-ww-mxgateway-cli/.../MxGatewayCli.java:1665-1674` (`replayGap` JSON row + `REPLAY_GAP` text line); tests `MxGatewayCliTests.java:909-937`, `clients/dotnet/ZB.MOM.WW.MxGateway.Client.Tests/MxGatewayClientCliTests.cs` (grep `FormatReplayGapRow`/`replayGap` hits); `docs/CrossLanguageSmokeMatrix.md:43-67` now records one converged contract with a per-CLI table | All five CLIs emit the same two camelCase keys with numeric cursors; the matrix doc explains *why* the row is hand-typed (proto3 JSON stringifies uint64). Genuinely converged. |
| CLI-37 | Done | **Yes — all five conformant on `category`** | .NET `MxStatusProxyExtensions.cs:15-20` (`Category is MxStatusCategory.Ok` only); Go `clients/go/mxgateway/status.go:15-17` (`nil ∥ category == OK`, doc `:7-14` states the diagnostics-only rule); Java `MxStatuses.java:33-35` (+ `success()` Javadoc `:55-57` marks it diagnostic); Python `errors.py:144-152`; Rust `error.rs:407-413` (`status.category != Ok as i32`, doc `:383-388` covers the unspecified-category-fails edge); shared fixtures `clients/proto/fixtures/behavior/command-replies/write.status-category-error-success-set.reply.json` + `write.status-category-ok-success-zero.reply.json`, registered in the manifest, documented in `docs/ClientBehaviorFixtures.md` (Reply Validation Conformance table), and consumed by all five suites (`MxCommandReplyExtensionsTests.cs`, `protofixtures_test.go`, `MxGatewayFixtureTests.java`, `test_errors.py`, `client_behavior.rs`) | The 4-vs-1 divergence is gone; the edge decisions (absent entry = success, present-unspecified = failure) are uniform and documented. |
| CLI-38 | Done | **Yes — all five on `hresult < 0`** | .NET `MxCommandReplyExtensions.cs:38` (`HasHresult && Hresult < 0`); Go `errors.go:299` (`GetHresult() < 0`); Java `MxGatewayErrors.java:52` (`getHresult() < 0`); Python `errors.py:137`; Rust `error.rs:407`; fixtures `write.hresult-s-false.reply.json` (S_FALSE=1 → success) + `write.hresult-e-fail.reply.json` (E_FAIL → failure) wired into all five suites | Lands prior-cycle CLI-08 (cycle-1 tracker row updated to Done via CLI-38 — verified). `docs/ClientLibrariesDesign.md:112-125` is now *true*, and states both rules explicitly. |
| CLI-39 | Done | **Yes (bump + both guards); pattern has already recurred → CLI-46** | Versions: `clients/rust/Cargo.toml:3`/`:28` = 0.2.0, `clients/python/pyproject.toml:9` + `version.py:3` = 0.2.0 (+ new drift test `test_cli.py:17` `test_version_matches_pyproject_toml`, closing the CLI-26 residual), `clients/go/mxgateway/version.go` = 0.2.0, `.../ZB.MOM.WW.MxGateway.Client.csproj:22` = 0.2.0, Contracts `.csproj:10` = 0.2.0 (`440e7cf`), Java `build.gradle` = 0.2.1. Guards: `scripts/tag-go-module.ps1:40-53` (tag fails on `ClientVersion` mismatch — the CLI-21 guard finally implemented), `scripts/pack-clients.ps1:91-156` (`Test-GiteaPackageExists` + hard abort "never force-overwrites"). Publish record: `docs/ClientPackaging.md:102-113` (2026-08-07: nuget/pypi/cargo 0.2.0, maven 0.2.1 with the rationale for the Java exception, Go tag `clients/go/v0.2.0` — tag exists in this repo) | Executed to the letter, including the doc's "bump before every publish" rule and the collision-guard rationale. However the *same hazard is live again at HEAD*: post-publish library changes (incl. a shipped Python `connect()` fix) sit at the already-published 0.2.0/0.2.1 — see **CLI-46**. |
| CLI-40 | Done | **Yes** | Java `MxGatewaySecrets.java:62-78` (`redactExact`); .NET `MxGatewaySecretRedaction.cs` (`Redact`/`RedactReply:55-68`/`RedactStatus:81-89`/`RedactStatuses` — scrubs message *and* the structured reply accessors, the `0d874f9` follow-up); Rust `error.rs:208-231` (`MxAccessError.secrets` + `with_secrets`, hand-written `Debug` prints only the count `:202-204`); fixtures `authenticate-user.echoed-credential.reply.json` + `.echoed-credential-mxaccess-failure.reply.json` in the manifest, consumed by all five suites (`MxGatewaySessionReplyContractTests.cs`, `command_reply_fixtures_test.go`, `MxGatewayCredentialReplyTests.java`, `test_malformed_reply.py`/`test_errors.py`, `client_behavior.rs`) | The echoed-credential fixture is exactly the test the prior review said was missing ("the fakes don't echo"). |
| CLI-41 | Done | **Yes** | Go `errors.go:164-176` (`MalformedReplyError`); Python `errors.py:56`; Java `MxGatewayMalformedReplyException.java` + throw sites `MxGatewaySession.java:889/:917/:948` (redaction rethrow preserves the type `:1143-1144`); .NET `MxGatewayMalformedReplyException.cs`; Rust `session.rs:1024-1042` (payload → present `return_value` → `Error::MalformedReply`, now uniform across its extractors); fixtures `authenticate-user.missing-payload.reply.json` + `authenticate-user.return-value-only.reply.json`; contract documented in `docs/ClientLibrariesDesign.md` ("Malformed-reply extraction" paragraph) | Silent-0 user ids and the .NET NRE are gone; the payload → return_value → typed-error ladder is uniform. |
| CLI-42 | Done | **Yes** | `clients/rust/README.md:24-36` (repo-path-first / vendored-fallback resolution, same-commit refresh rule, both-direction drift guard); `docs/ClientPackaging.md:199-213` (same, plus why `cargo package` runs *with* verification); `711793f` extended check-codegen Check 3 to sweep both directions (canonical proto missing a vendored copy now fails) | Vendored copies verified byte-identical to Contracts at HEAD (all three protos, diff clean). |
| CLI-43 | Done | **Yes** | `docs/style-guides/JavaStyleGuide.md:8` — "Target Java 17 (the Ignition 8.3 baseline; the client build enforces …)" | `grep "Java 21"` in style-guides: clean. |
| CLI-44 | Done | **Yes** | `clients/go/mxgateway/session.go:1077` (Recv-error path uses `sendTerminalEventResult`), `:1096-1111` (bypasses the overflow substitution via the reserved slot) | Root cause of a genuine stream error is no longer relabeled `ErrSlowConsumer`. |
| CLI-45 | Done | **Yes** | Canonical env `MXGATEWAY_VERIFY_PASSWORD` in all five: Go `main.go:452` (default) + `:483-491` (fail-fast before dialing, message names flag+var only); Java `MxGatewayCli.java:194` + `:1166-1179` (`ParameterException` on null/blank); Rust `main.rs:151` (clap default) + test `:2644-2660`; Python `commands.py:38` (`DEFAULT_VERIFY_PASSWORD_ENV`); .NET `MxGatewayClientCli.cs:353-359` (canonical default + `MXGATEWAY_VERIFY_USER_PASSWORD`/`--verify-user-password*` kept as deprecated aliases, resolution order documented `:385-386` and in help `:2455-2456`). Tests: Go `main_test.go:605` (`RejectsEmptyPassword`) + `:632` (`ReadsPasswordFromCanonicalEnv`). Contract documented at `docs/CrossLanguageSmokeMatrix.md:90-119`, incl. the parity rationale (libraries still transmit whatever they are given) and the .NET deprecation window | One exported variable now drives all five CLIs; empty credential is a usage error everywhere. |
**Done claims that don't fully hold:** none. Every 2026-07-12 remediation claim checked out at HEAD with the designed shape (fixtures-first conformance, docs in the same commit, guards actually implemented — including the CLI-21 tag guard skipped last cycle). This is the cleanest verification pass of the three cycles.
### Cycle-1 residue (CLI-05..33, previously Open)
Re-verified briefly at HEAD; all unchanged unless noted:
- **CLI-08 → Done** via CLI-38 (see above); cycle-1 tracker row correctly updated.
- Still open, unchanged: **CLI-05** (`MxGatewaySession.cs:19` internal ctor, no .NET `AttachSession`), **CLI-06** (`:1501-1505` `DisposeAsync` → unbounded `CloseAsync`), **CLI-07** (`MxGatewayClient.cs:311`), **CLI-09** (no typed auth errors in Go `errors.go`), **CLI-10** (`client.go:64/:68` `WithBlock`/`DialContext`), **CLI-11** (library has `RequireCertificateValidation` (`client.go:239`) but `mxgw-go` still exposes no flag), **CLI-13** (`MxGatewayClient.java:248` hardcoded 16), **CLI-14** (no TOFU runtime warning in `options.py`), **CLI-17/20** (TLS posture divergence: .NET accept-all `MxGatewayClient.cs:369`, Go skip-verify, Java `InsecureTrustManagerFactory` `MxGatewayClient.java:391`, Python TOFU, Rust strict), **CLI-19** (duplicate `InternalsVisibleTo`), **CLI-22** (`session.go:1176-1181` empty correlation id on rand failure), **CLI-23** (nil-vs-empty asymmetry — now at least documented in the `WriteBulk` doc comment `session.go:431-432`), **CLI-25** (`close()` doesn't await termination `MxGatewayClient.java:347-351`), **CLI-27** (`session.py` close still lock-free), **CLI-28** (bottom-of-file import `session.py`), **CLI-31** (`mxgw-cli/src/main.rs` now **2,991** lines — grew again; the other CLIs sit at 1,668–2,478), **CLI-32** (no .NET/Java client-side bulk cap), **CLI-33** (no backpressure section in `docs/ClientBehaviorFixtures.md`).
## New findings
### CLI-46 — Version constants again equal the already-published 0.2.0/0.2.1 while post-publish changes shipped — including the fix for a published-broken Python `connect()`
**Severity:** Medium · **Category:** packaging / release hygiene
**Files:** `clients/python/pyproject.toml:9` + `clients/python/src/zb_mom_ww_mxgateway/version.py:3` (0.2.0 = the pypi artifact published 2026-08-07, `docs/ClientPackaging.md:102-113`); `clients/python/src/zb_mom_ww_mxgateway/client.py:61-72` (the fixed `connect()`); post-publish library commits `a8f86b5` (2026-08-10), `d9ea8a8`, `c748361` (2026-08-17/18); guard `scripts/pack-clients.ps1:145-156`
The published pypi 0.2.0 Python client cannot open a real connection at all: its `connect()` built the `grpc.aio` channel inside `asyncio.to_thread`, and a `grpc.aio` channel binds to the event loop current on the constructing thread, so every non-stub connection raised "There is no current event loop in thread" (TST-24's first real-wire run caught it; fixed in-tree by `a8f86b5` — `resolve_channel_security` runs off-loop, `create_channel` on-loop). That fix has sat unpublished for nine days while the version constant still reads 0.2.0. Since the publish, all five libraries also gained wire-surface additions at the published numbers (alarm truncation fields `d9ea8a8`, feed-level `snapshot_status` bindings `c748361`; Java at its published 0.2.1). This is precisely the CLI-39 pattern one cycle later — now *mitigated* (the pack-script collision guard refuses a same-version publish, and the bump-together deferral is deliberate) but the live registry keeps serving a broken Python client, and nothing in the tree records that 0.2.0-as-published ≠ 0.2.0-as-source.
**Recommendation:** cut the next release train soon (0.2.1 four clients / 0.2.2 Java, or 0.3.0 across the board), prioritizing the Python publish; until then, note the known-broken `connect()` against pypi 0.2.0 in `clients/python/README.md`/`docs/ClientPackaging.md` (or yank the pypi artifact). Consider bumping version constants in the same commit as the first post-publish library change rather than at publish time — the collision guard then enforces nothing because nothing needs catching.
### CLI-47 — `snapshot-status`/`provider-status` rendering diverges: .NET text mode hides the flag on the frames that matter most; Rust's JSON is the lone always-explicit shape
**Severity:** Low · **Category:** cross-client parity / CLI
**Files:** .NET `MxGatewayClientCli.cs:15` (`JsonFormatter.Default` — omits default-valued proto3 fields) + `:1563-1566` (text arms format the payload through it); Go `main.go:1112-1114` (`snapshot-status truncated=%t`); Java `MxGatewayCli.java:2309-2312` (`truncated=%b`) + `protoJson` `:2362-2368` (default printer); Rust `main.rs:2262-2264` (summary) + `:2314-2318` (hand-built JSON, always includes `"truncated"`); Python CLI renders generic `MessageToDict`
The c748361 rollout achieved surface parity (all five render the new arm) but not rendering parity. The `snapshot_status` clearing frame — `truncated = false`, emitted on monitor restart, explicitly called out by the commit and `docs/Grpc.md` as a frame consumers must see — renders in the .NET CLI's text mode as `snapshot-status {}` (proto3 JSON omits the false bool), while Go/Java/Rust print `truncated=false` explicitly. The same applies to a healthy `provider-status` (`degraded=false` invisible in .NET text). In JSON mode the split flips: Rust's hand-built row always carries `"truncated"`, while .NET/Go/Java/Python protobuf formatters emit `{"snapshotStatus": {}}` for the clearing frame. Semantically recoverable (absent = false), but the same class of divergence NEXT-02 just cleaned up for `ReplayGap`, where the cross-CLI row was deliberately hand-specified.
**Recommendation:** hand-format the .NET text arms like the other three (`snapshot-status truncated={bool}`, `provider-status mode=… degraded=… reason=…`); decide one JSON shape (either Rust adopts formatter-omission or the matrix documents Rust's explicitness the way `CrossLanguageSmokeMatrix.md:54-67` documents the ReplayGap row's deliberate choices).
### CLI-48 — The new alarm-feed renderer arms are untested in three of the four CLIs that changed; the Go CLI has no `stream-alarms` test at all
**Severity:** Low · **Category:** testing coverage
**Files:** `clients/go/cmd/mxgw-go/main_test.go` (test inventory `:16-734` — no stream-alarms/`formatAlarmFeedMessage` test of any kind); .NET `MxGatewayClientCliTests.cs:671-706` (`stream-alarms` test feeds only `ActiveAlarm` + `SnapshotComplete`); Java `MxGatewayCliTests.java:653-687` (asserts `active-alarm`/`snapshot-complete`/`transition` arms only); Rust `main.rs:2986` (the one snapshot-status render test, added by `4d4f443`)
The review-nits commit covered only the Rust render; the .NET/Go/Java `provider-status` and `snapshot-status` arms shipped with zero assertions. The .NET `{}`-rendering divergence (CLI-47) is exactly what a text-mode assertion on a `truncated=false` frame would have caught — cycle 2's lesson ("each new epic achieves surface parity but re-introduces behavioral divergence") repeating at the test layer.
**Recommendation:** one fake-fed `stream-alarms` test per CLI whose feed includes a `provider_status` and a `snapshot_status` frame (including the false/clearing shape), asserting the rendered row in both output modes; for Go this also creates the missing `stream-alarms` scaffold.
### CLI-49 — Go library exposes no aliases for `AlarmProviderStatus`/`AlarmSnapshotStatus`, whose generated types live in an `internal` package; the `AlarmFeedMessage` alias doc names three of five payload arms
**Severity:** Low · **Category:** code organization & conventions / documentation
**Files:** `clients/go/mxgateway/types.go:129-137` (alias block: `AlarmFeedMessage`, `ActiveAlarmSnapshot`, `OnAlarmTransitionEvent` — but not the two status payloads; `ReplayGap` got its alias at `:38` when it shipped); generated types under `clients/go/internal/generated` (Go `internal` visibility)
Because the generated package is `internal`, a Go consumer cannot name `*generated.AlarmSnapshotStatus`: they can call `message.GetSnapshotStatus().GetTruncated()` on an inferred value but cannot declare a variable, struct field, or function parameter of the type — unlike every other StreamAlarms payload, which `types.go` deliberately aliases for exactly this reason. (`AlarmProviderStatus` has had the same gap since provider-mode shipped.) The `AlarmFeedMessage` alias comment (`types.go:130-132`) still describes the oneof as "an active-alarm snapshot row, a snapshot-complete sentinel, or a transition" — stale by two arms.
**Recommendation:** add `AlarmProviderStatus = pb.AlarmProviderStatus` and `AlarmSnapshotStatus = pb.AlarmSnapshotStatus` aliases and refresh the `AlarmFeedMessage` comment to name all five arms.
### CLI-50 — Python CLI `stream-alarms` default (`--max-messages 1`, collect-then-print) now returns only the provider-status preamble — the README's own example fetches no alarm data
**Severity:** Low · **Category:** cross-client parity / CLI usability
**Files:** `clients/python/src/zb_mom_ww_mxgateway_cli/commands.py:514` (`--max-messages` default 1), `:1539-1551` (`_collect_alarm_messages` — bounded collect, printed once at the end), `:1122-1134`; `clients/python/README.md:451` (`mxgw-py stream-alarms --max-messages 1 --json`); feed order per `docs/Grpc.md`/`c3c603f`: `provider_status` → `snapshot_status` → cached `active_alarm` → `snapshot_complete`
The other four CLIs stream rows as they arrive and default to unbounded (`--limit 0` / `--max-events 0`). Python is the outlier twice over: it buffers then prints (so it cannot tail a live feed at all), and its default of 1 — reasonable when the first frame was an alarm — now returns exactly one `provider_status` frame, since the feed gained a two-frame status preamble. The README example therefore demonstrates a command that yields no alarm rows. Pre-existing shape, but the preamble growth this cycle changed what the default actually returns.
**Recommendation:** raise the default (or document that N status frames precede the data) and update the README example; longer term consider printing rows incrementally like the other four CLIs.
## Severity roll-up (new findings)
| Severity | Count | IDs |
|---|---|---|
| High | 0 | — |
| Medium | 1 | CLI-46 |
| Low | 4 | CLI-47, CLI-48, CLI-49, CLI-50 |
## Cross-client parity note (updated)
The 2026-07-12 divergence table has largely collapsed. Converged this cycle, with shared fixtures pinning the bytes: **HRESULT `< 0`** (CLI-38), **status verdict = `category != OK`** (CLI-37), **exact-secret scrub incl. echoed diagnostics** (CLI-40), **malformed-reply ladder** (CLI-41), **CLI ReplayGap row** (CLI-35/36 + NEXT-02, all five, same keys, numeric cursors), **credential env-var contract** (CLI-45). What remains divergent:
| Behavior | .NET | Go | Java | Python | Rust |
|---|---|---|---|---|---|
| Version vs published artifact (CLI-46) | 0.2.0 = shipped | 0.2.0 = shipped | 0.2.1 = shipped | 0.2.0 = shipped (**broken `connect()` on pypi**) | 0.2.0 = shipped |
| CLI text render of snapshot/provider status (CLI-47) | proto-JSON, flag invisible when false | explicit `truncated=…` | explicit | n/a (JSON only) | explicit ✓ |
| CLI JSON shape of `snapshot_status` (CLI-47) | omits false | omits false | omits false | omits false | always explicit |
| New renderer arms tested (CLI-48) | no | no (no stream-alarms test) | no | n/a (generic dump) | yes ✓ |
| `stream-alarms` consumption model (CLI-50) | stream, unbounded default | stream, unbounded | stream, unbounded | **collect, default 1** | stream, unbounded |
| Session re-attach (CLI-05) | **No** | Yes | Yes | Yes | Yes |
| Client-side bulk cap (CLI-32) | No | Yes | No | Yes | Yes |
| Event backpressure (CLI-13/33) | unbuffered | 16+1, loud, true root cause ✓ | 16, cancel+error | unbuffered | unbuffered |
| TLS default (CLI-17) | accept-all | accept-all | accept-all | TOFU pin | strict pin |
| Typed auth errors (CLI-09) | Yes | **No** | Yes | Yes | Yes |
| Real-wire test harness | Kestrel h2c ✓ (TST-24) | bufconn ✓ | InProcessServer ✓ | grpc.aio loopback ✓ (TST-24) | tonic loopback ✓ |
CLI subcommand coverage remains uneven by design and is now honestly specified (`docs/CrossLanguageSmokeMatrix.md:163-186`: .NET all nine, Rust + unregister/credential pair, Go/Python/Java credential pair only) — verified accurate against all five dispatch tables at HEAD, including the note that every gap is CLI-surface-only (all five libraries carry all typed helpers, confirmed for the Rust bulk family at `session.rs:605-654`).
## What is good
- **The remediation train executed at design fidelity.** All eleven 2026-07-12 findings landed exactly as their remediation designs specified — including the parts prior cycles skipped (the CLI-21 tag guard, the CLI-02/42 doc half, the CLI-26 pyproject-drift test). The conformance findings (CLI-37/38/40/41) all went fixtures-first: ten shared command-reply fixtures under `clients/proto/fixtures/behavior/command-replies/`, each consumed by all five suites, so the canonical rules are pinned in bytes, not prose.
- **The publish pipeline is now guarded end to end**: version-vs-tag check in `tag-go-module.ps1`, registry-collision hard-stop in `pack-clients.ps1`, publish coordinates and the Java 0.2.1 exception recorded in `docs/ClientPackaging.md` with the reasoning ("these are the numbers a future release bumps off").
- **TST-24 paid for itself immediately**: the first real-wire run of the Python client found `connect()` could never work outside stubs — a defect every mock-based test had actively asserted as correct. All five clients now have genuine-transport tests (framing, metadata, status codes, ReplayGap, PERMISSION_DENIED mapping), and CI runs all five suites plus a Java generated-tree drift check (`.gitea/workflows/ci.yml:84-142`).
- **Generated-code hygiene is the best it has been**: vendored Rust protos byte-identical to Contracts (verified, all three files); check-codegen Check 3 sweeps both directions; the Java toolchain pin eliminated the spurious ~64k-line protobuf churn (per `build.gradle:61-70`, IPC-24) and `checkGeneratedClean` survived Gradle 9 (`df45cb4`); Python regen stays on the pinned grpcio-tools 1.80.0 / protobuf 6.31.1 baseline (`pyproject.toml:41-44`, generated header confirms 6.31.1); even a proto-comment-only change (`c3c603f`) regenerated every downstream tree in the same commit.
- **Parity discipline held through the alarm epic**: no client wraps or reinterprets `AlarmFeedMessage`/`ActiveAlarmSnapshot` — the truncation and status flags pass through the generated types untouched in all five, and every README explains the `from_truncated_snapshot`/`snapshot_status` semantics with the same don't-reconcile-deletions guidance, explicitly disambiguating them from the per-record `degraded` flag.
- **The smoke-matrix doc is a model of honest divergence-recording**: where the CLIs still differ (subcommand coverage, JSON formatting quirks per language), the doc specifies the delta instead of leaving it to be discovered — the exact practice this review series asked for two cycles ago.
@@ -0,0 +1,104 @@
# Testing, Documentation & Underdeveloped Areas — Re-Review (cycle 3)
- **Date:** 2026-08-19
- **Commit:** `8b3e3b3` (`main`, "Merge feat/sweep-reread-tag-summary…")
- **Baseline:** prior report `archreview/2026-07-12/60-testing-docs-gaps.md` @ `4f5371f` (2026-07-12); ~288 commits since.
- **Method:** macOS static review — reading only, no builds, no test runs, no remote hosts. One read-only script executed (`python3 code-reviews/regen-readme.py --check`, sanctioned by REVIEW-PROCESS.md). All paths relative to `/Users/dohertj2/Desktop/MxAccessGateway`.
## Prior-finding verification
### Cycle-2 findings (TST-25..30)
| ID | Claimed status | Verified? | Evidence (file:line) | Notes |
|----|----------------|-----------|----------------------|-------|
| TST-25 | Done | **Yes** | `.gitea/workflows/ci.yml:144-165` (`windows-x86` per-push job, Linux runner → SSH to windev, exit-code propagation) and `:167-201` (`nightly-windev`, `if: github.event_name == 'schedule'`, `run-windev-ci.sh live`, on-failure Gitea-issue step); `scripts/ci/run-windev-ci.sh` (SHA-shape validation `:46-49`, base64 key decode `:70-78`, bootstrap holds the worktree lock `:91-119`); `scripts/ci/windev-worker-ci.ps1` (lock reuse via `MXGW_CI_LOCK_HELD` `:77-94`, re-checkout under lock + HEAD verification `:99-111`, cumulative build/test/live modes `:119-139`) | The whole design shipped as specified, including the acceptance-check hardening: the CI key is stored base64 single-line so Gitea's line-oriented masker redacts it (`ci.yml:156-159`), the bootstrap fetch/checkout races were serialized under the mkdir lock, and the nightly failure issue carries a browser-reachable link via `PUBLIC_SERVER_URL` (`ci.yml:174-181`, Check 6 closed 2026-08-10 per `docs/runbooks/TST-30-second-ci-runner.md`). Ground truth from the operator confirms the tier runs green. |
| TST-26 | Done | **Yes** | `docs/GatewayTesting.md:624-657` describes exactly the four jobs that exist (`portable`, `java`, `windows-x86`, `nightly-windev`) incl. the "red can mean the tier is down" triage (`:649-652`); `scripts/check-codegen.ps1:26-27` now attributes the x86 guard to "the SSH-driven `windows-x86` CI job"; `ci.yml:14-18` cron comment matches what the schedule runs | Guard attribution is correct both ways: primary = Check 2 regeneration diff in `portable`, secondary = the windows-x86 net48 compile (`GatewayTesting.md:699-703`). No named job is missing and no phantom job survives. |
| TST-27 | Done | **Yes** | `docs/GatewayConfiguration.md:195` — the `ShowTagValues` row now describes all three seams (events-hub mirror, alarms hub, `/browse` live values incl. the successful-read-only nuance from `e50ec49`), the clone-don't-mutate guarantee, and the ACL layering | "Reserved" is gone; the row was rewritten a second time when TST-16's residuals shipped (2026-08-17), so it documents strictly more than the finding asked. |
| TST-28 | Done | **Yes** | `src/ZB.MOM.WW.MxGateway.Tests/Gateway/Workers/WorkerClientTests.cs:46` — `StartAsync_SendsGatewayHelloWithConfiguredMaxFrameBytes(int maxMessageBytes)` theory, in the CI-run portable suite | Exactly the designed test (default + override), pinning the gateway half of the IPC-02 negotiation on Linux CI. |
| TST-29 | Done | **Yes** | `oldtasks.md` absent from the tree; the five `*-docs-*.md` root artifacts absent; `docs/DesignDecisions.md:264` "Session-Resilience Epic Scope" carries the Phase-5 deferred-not-planned record and `:295` the "`EnableOrphanReattach` does not exist and must not be referenced" rule; CLAUDE.md's invariant paragraph points at DesignDecisions | Governance record migrated intact before deletion, exactly per the design. Residual: `stillpending.md` (also a TST-14 root artifact) still contradicts the migrated record — see new finding TST-33. |
| TST-30 | Done | **Yes, with one pending operator action** | `docs/runbooks/TST-30-second-ci-runner.md:1-13` (Executed 2026-08-07: `gitea-runner-2`, id 5, capacity 2, `container.network: traefik`, token from a `0600` file mount; concurrency verified across three simultaneous runs) plus the honest correction that the per-repo runners endpoint legitimately returns 0 (instance-level registration); `docs/GatewayTesting.md:659-685` documents the shared-capacity/no-cancel reality and the `run-windev-ci.sh` bypass | The runner exists and the docs match the executed state. The **follow-up it surfaced is still open**: the shared instance registration token (world-readable ~5 months, proven still live by a probe registration) needs a web-UI reset — promoted to new finding TST-32 so it stops living only inside a closed finding's runbook. |
**Done claims that failed verification: none.** All six hold with file:line evidence; TST-30 carries the one explicitly-tracked operator residual (TST-32 below).
### Cycle-1 leftovers still tracked open (spot-verified at HEAD)
| ID | Tracker status | State at HEAD |
|----|----------------|---------------|
| TST-05 | Partially done | **Residual confirmed open.** The live suite is 8 `[LiveMxAccessFact]`s (`WorkerLiveMxAccessSmokeTests.cs:34-920`) with zero references to `Ping`/`GetSessionState`/`GetWorkerInfo`/`DrainEvents`/`ShutdownWorker`; the gap is honestly documented at `docs/GatewayTesting.md:90-95`. The scheduling half stays closed (nightly-windev runs the suite). The two control-command live facts specified in `archreview/remediation/60-testing-docs-gaps.md` remain unauthored (windev-only work). |
| TST-06 | Not started | **Materially narrowed, tracker stale.** `Tests/Gateway/Dashboard/DashboardLiveDataServiceTests.cs`, `EventsHubTests.cs`, `EventsHubViewerRegistryTests.cs`, `AlarmsHubPublisherTests.cs`, `SessionDetailsPageEventAclTests.cs` all exist now (mostly landed with SEC-25/TST-16 closeouts and the in-process dashboard-feed work). The cycle-1 row deserves a re-audit-and-close pass rather than "Not started". |
| TST-07 | Not started | Partially addressed: the `WorkerClientTests` 150 ms sleep-then-negative-assert is gone; `SessionManagerTests.cs:491` still has a real-clock `Task.Delay(50)`. |
| TST-09 | Not started | Partially addressed: `GatewayApplication.cs:117-124` now registers `AuthStoreHealthCheck` **and** `SessionHealthCheck` (the `/health/active` probe with off-repo consumers). Still no Galaxy SQL / LDAP / worker-executable checks. |
| TST-10 | Not started | Still no `docs/Deployment.md`; partial mitigation via `docs/runbooks/IdentifyingADeployedBuild.md` and the SEC-36 rotation runbook, but deploy/upgrade mechanics (dated `Server-YYYYMMDD` dirs, NSSM repoint, env-var config) still live in operator memory notes, and the 2026-08-09 credential-store-renamed-away outage (`docs/GatewayConfiguration.md:100`) shows the cost. |
| TST-14 | Not started | Mostly discharged by TST-29 (oldtasks + the five untracked artifacts gone). Remaining root-file triage: `stillpending.md` (now the stale one — see TST-33) and `A2-galaxyrepository-adoption-handoff.md` (carries a current "ADOPTED" banner; acceptable as a handoff record). |
| TST-17 | Not started | **Materially changed — row needs refresh by the worker domain.** `WnWrapAlarmConsumer.cs:309-327` now passes `szDomainName`/`szOprFullName` into the COM call (the `_ = ackOperatorDomain` discard is gone), and the 2026-08 probe cycle recorded the vendor 8-arg leg as "unavailable by configuration" with evidence (`docs/AlarmProbeFindings.md`, commits `d1ae43d`, `540a118`). Whether a caller-visible degrade signal exists is the worker reviewer's call, but "silent drop, unchanged" is no longer an accurate row. |
| TST-18 | Not started | Confirmed open — no `*HostedService*` tests under `src/ZB.MOM.WW.MxGateway.Tests`. |
| TST-19 | Not started | Confirmed open — `FakeWorkerHarness.cs` still has no lockstep note / `WorkerPipeSession` pointer. Mitigation since: the harness's canned control replies are no longer the *only* automation (windows-x86 runs the real worker suite per push), but the drift risk the finding names is intact. |
| TST-20 | Not started | Confirmed open — `run-client-e2e-tests.ps1:48` `DrainEveryTags = 15` workaround intact. |
| TST-21 | Not started | Confirmed open — `Server/appsettings.json:10` daily rolling file sink, no size cap / retained-count. |
| TST-22 | Not started | Confirmed open — the shape block (`docs/GatewayConfiguration.md:12-88`) is maintained (gained `GroupToTag`, `UntaggedSessionVisibility`, `RecentFaultLimit`/`RecentSessionLimit`) but still omits documented keys (`DisableLogin`, `AutoLoginUser`, `CookieName`, `Tls`, `Ldap`, `Security`, `Alarms:Fallback`, `Worker:WriteCompletionWaitMilliseconds`, …). The option **tables** below it are complete and current — the omissions are only in the illustrative block. |
## New-code / new-docs review since `4f5371f`
**CI workflow evolution — verified sound.** `ci.yml` at HEAD is a four-job pipeline whose comments are accurate down to the failure semantics. Details worth recording: the portable job now runs the .NET client *tests* (TST-24 wire tests included, `ci.yml:83-90`), installs pwsh and the pinned Check-4 generators explicitly (`:58-72`), and pins protoc 34.1; the java job's `git diff --exit-code` is the true drift gate after IPC-24 deleted the churn-revert; `run-windev-ci.sh` validates `CI_SHA` shape before it reaches a remote shell (`:46-49`), never echoes the key, and pins host keys from the committed `windev.known_hosts`; `windev-worker-ci.ps1` re-checks-out under the lock, verifies `HEAD == Sha`, `git clean -fdx -e ci.log`, and is deliberately PS5.1-safe. The nightly's issue-on-failure path uses the runner-resolvable `github.server_url` for the API call but `PUBLIC_SERVER_URL` for the human link — the exact defect Check 6 caught, fixed in `30c92e8`.
**check-codegen.ps1 —** now four checks with the Check 3 reverse sweep (`:98-103`, canonical proto missing a vendored copy fails — the drift class no in-repo build can surface, `711793f`) and Check 4 Windows support (`.exe` banner normalization in the per-client `generate-proto.ps1` probes, `c94c4d4`; PS5.1-safe native-arg quoting, `8ae0c2f`). The header comment (`:2-27`) is accurate.
**fca978d marker sweep → f82dac1 restoration — assessed complete on sampled evidence.** The sweep (2026-07-07, 203 files) deleted ~1,100 prose-bearing comment lines alongside the tracking markers; the 2026-08-18 re-read narrowed those to 816 residual lines, judged each, and restored 17 collateral deletions across 10 files (`f82dac1`, merged `8b3e3b3`). Spot-checks of four substantive deletions **outside** the restored set all found surviving equivalents at HEAD: the `SessionEventDistributor` mode-vs-count FailFast race rationale survives in `docs/Sessions.md:206` (including the "fixed mode rather than a live subscriber-count snapshot" decision) and `docs/GatewayConfiguration.md:159`; the `MxAccessStaSession` off-STA-deadlock affinity-guard rationale survives in `AlarmCommandHandler.cs:68-80`; the subtag source's own-COM-object isolation rationale survives in `LmxSubtagAlarmSource.cs` class remarks; the watchdog `HeartbeatStuckCeiling` rationale survives in `WorkerPipeSessionOptions.cs:35+`. No residual collateral damage found. The companion `d3ac527` audit (re-verifying 20 doc-only `Server-0xx` resolutions, re-fixing the two the sweep had silently reverted — Server-040 among them) closes the loop on the sweep's other damage class. This is a model recovery process.
**Doc currency at HEAD — high.** Sampled claims all match code: `docs/Sessions.md:17` pipe-name prose matches `SessionManager.cs:536` (`mxgw-{pid}-{uid}`, with the sun_path rationale); `docs/Sessions.md:248` owner-scoped attach matches `EventStreamService`; `docs/GatewayConfiguration.md` documents the new `WriteCompletionWaitMilliseconds` (`:121`, with the OtOpcUa 2 s budget interaction), `EventQueueCapacity` (`:122`), alarm poll/fetch ceilings incl. truncation semantics (`:435-436`), and the `Fallback` block (`:448-451`); `docs/GatewayTesting.md` is the standout — the windev sections (`:488-622`) record the 879 baseline, the two "environmental" failures that were test bugs, the load-sensitivity list, and a first-class post-mortem of the zero-buffer-pipe testhost hang with the structural fixes (`TestNamedPipe.CreateServer`, bounded `PipePair.WriteAsync`). `docs/Authentication.md` was made runnable-as-written (`2b2748b`); `docs/AlarmProbeFindings.md` states the probe questions and their evidence honestly (ack leg unavailable-by-configuration, GUID leg still assumed). `python3 code-reviews/regen-readme.py --check` passes at HEAD ("up to date and consistent").
**New-feature test placement — correct layers.** Alarm truncation / `snapshot_status`: proto-level flag plumbed with direct tests (`b9fb0dd` atomic snapshot+truncation read), gateway feed frame (`AlarmTruncationSignalTests.cs`), dashboard banner (`AlarmsPageTruncationBannerTests.cs` incl. the mid-truncation attach case, `f57a6ae`), and all five clients regenerated with render coverage (`4d4f443`). SEC-25 tag ACL: `DashboardSessionAclTests`, both subscribe seams covered (`EventsHubTests`, `SessionDetailsPageEventAclTests`), decision-table corners pinned (`7ec0b35`), plus live-LDAP ACL coverage (`1d8a4a6`). Drain cluster (WRK-21/IPC-23/IPC-30): worker-side byte-budget tests run per-push via windows-x86; the gateway side keeps the drain-to-empty walk inside the pipe-harness envelope (`a256560`). Write-completion correlation: worker executor tests (`b0e2b8b`, `c867aca`) plus gateway-side config plumb tests (`GatewayOptionsTests`, `WorkerProcessLauncherTests`). In-process wire tests (TST-24) exist for all five clients and run in CI (`docs/GatewayTesting.md:280-321`).
## New findings
### TST-31 — `glauth.md`'s pre-provisioned-users table contradicts the directory, and its wrong credential propagated into the live-test surface — **Medium** · documentation currency / test fixtures (promotes NEXT-10)
- **Files:** `glauth.md:55` (ReadOnly gid `5501` vs actual `5601`), `:65-78` (pre-provisioned users table: `readonly`/`readonly123` does not exist in `scadaproj/infra/glauth/config.toml`; `admin` carries the standard dev password, not `admin123`), `:124` (`Password: admin123` in a worked example); propagation: `docs/GatewayTesting.md:119` (`MXGATEWAY_LIVE_MXACCESS_WRITE_SECURED_PASSWORD` default `admin123`) and `src/ZB.MOM.WW.MxGateway.IntegrationTests/WorkerLiveMxAccessSmokeTests.cs:1432` (`?? "admin123"`, with `:517` attributing the identity to the GLAuth-bridged ArchestrA `admin`/`admin123`).
- **Description:** Carried over from `archreview/2026-07-12/remediation/90-candidate-findings-next-cycle.md` NEXT-10, which was deliberately deferred pending the OPC-UA group-taxonomy sweep and is still fully present at HEAD. The table was already the proven root cause of one defect (the NEXT-06 live-LDAP fixture drift, fixed 2026-08-07 — the *fixtures* were realigned but the table that misled them was not). The remaining propagation has a quiet coverage cost: the nightly live smoke's `WriteSecured` parity test authenticates with the defaulted `admin`/`admin123`, and because `AuthenticateUser` legitimately returns a real MXAccess *failure* reply on bad credentials (parity, not `InvalidRequest`), a wrong default does not fail the test — it silently degrades the WriteSecured leg from "authenticated round-trip" to "failure-reply shape check" every night. A reader also cannot tell which half of `glauth.md` to trust, since its dashboard section is correct.
- **Recommendation:** Reconcile the table against `scadaproj/infra/glauth/config.toml` as one unit with the OPC-UA group taxonomy (the reason it was deferred), or — if that sweep stays parked — fix the two provably-wrong rows and the gid now and leave the taxonomy note. In the same change, re-derive the `MXGATEWAY_LIVE_MXACCESS_WRITE_SECURED_PASSWORD` default and the `WorkerLiveMxAccessSmokeTests` literal from whatever the rig's ArchestrA-bridged identity actually is, and make the live test log (not assert) whether `AuthenticateUser` succeeded, so a credential regression in the nightly is at least visible.
### TST-32 — The shared Gitea runner registration token (world-readable for ~5 months, proven live) still awaits its web-UI reset — **Medium** · CI/ops security
- **Files:** `docs/runbooks/TST-30-second-ci-runner.md:33-42` ("Follow-up 2026-08-07 — token hygiene"); `archreview/2026-07-12/remediation/90-candidate-findings-next-cycle.md:24` (the pending-operator-action row).
- **Description:** Both CI runners share one **instance-scope** registration token that sat world-readable in `/opt/gitea/docker-compose.yml` for roughly five months and is **still valid** — a probe registered runner id 6 with it and then deleted it. Host hygiene is done (token now a `0600` file mount, compose + backups locked down), but hygiene does not retire the credential: anyone who read it can register a rogue runner that receives this repo's jobs, including the `WINDEV_SSH_KEY` secret that grants shell on windev. Gitea 1.26.4 exposes no rotation via CLI or API (both are get-or-create), so the reset must be done in the admin web UI, then `/opt/gitea/runner_token` refreshed and the two token-bearing compose backups shredded. This has been a documented "pending operator action" since 2026-08-07 with no tracked finding forcing it to a close — precisely the state in which such actions rot.
- **Recommendation:** Execute the reset per the runbook (minutes of work; re-registration of the two runners with the new token is the only follow-on). Track it here rather than only in a closed finding's runbook, and after the reset append the dated "Executed" note the runbook's own convention expects.
### TST-33 — `stillpending.md` has drifted into self-contradiction and now overstates what is pending — **Low** · repo hygiene / documentation currency
- **Files:** `stillpending.md:54` (reconnect "not safe to rely on end-to-end yet" — Tasks 13–15 shipped and were verified as cycle-2 TST-01/TST-02 with all-five-client consumers); `:56` (Phase 5 orphan reattach "**planned** (epic Phase 5, not started)" — directly contradicts the governance record TST-29 migrated to `docs/DesignDecisions.md:264-295`, which says **deferred, not planned**, and CLAUDE.md's invariant); `:111` ("WorkerLiveMxAccessSmokeTests.cs (6 `[LiveMxAccessFact]`)" — there are 8); `:118` ("Cross-language client↔gateway wire behavior — no per-client integration unit tests… All client wire behavior is unverified in default unit runs" — false since TST-24 closed 2026-08-10: all five clients have real-transport wire tests running in CI); `:150` (§8: per-session-ACL "still scheduled… not yet started") vs `:65` (§2: the same item "RESOLVED (2026-08-17)") — the file contradicts itself.
- **Description:** The file is not a frozen snapshot — it has been annotated through 2026-08-17 — so readers reasonably treat it as live, and it is one of the two remaining root-level triage artifacts from TST-14. Its most dangerous staleness is `:56`: it re-asserts the "planned Phase 5 / `EnableOrphanReattach`" framing that TST-04/TST-29 spent two cycles extinguishing, in the exact file a future contributor greps when asking "what's still pending".
- **Recommendation:** One currency pass: fix the five spots above (§2 reconnect → shipped; §2 Phase 5 → deferred-not-planned, pointing at DesignDecisions; §5 counts and the wire-test bullet; §8 ACL row → resolved, matching §2). Then either commit to keeping it current or stamp it "historical snapshot — see archreview trackers for live status" at the top, which is the cheaper honest option.
### TST-34 — The code-reviews README consistency check is "the intended hook for CI" but is wired into no CI job — **Low** · CI
- **Files:** `REVIEW-PROCESS.md:124-131` ("`regen-readme.py --check` exits non-zero if README.md is stale… The PowerShell wrapper `scripts/check-code-reviews-readme.ps1` runs that check and is the intended hook for CI or a pre-commit step"); `.gitea/workflows/ci.yml` (no reference to either script in any job); also `REVIEW-PROCESS.md:130-131` (the "bare `python3` alias resolves to the Windows Store stub — use `python`" note is Windows-only advice presented unconditionally; on the macOS tree `python` may be absent and `python3` is correct).
- **Description:** The check passes at HEAD (verified this review), and the repo's process doc names CI as its intended home, but 13 months of CI evolution never picked it up — so a stale `code-reviews/README.md` or an inconsistent open-findings count merges silently until the next human runs the script. The portable job already installs Python 3.12, so the cost is one step.
- **Recommendation:** Add a `Code-reviews README consistency` step to the `portable` job (`python code-reviews/regen-readme.py --check`), and qualify the interpreter note in REVIEW-PROCESS.md as Windows-specific (macOS/Linux: `python3`).
### TST-35 — The Windows-host run of the gateway (net10) suite is still manual-only, and its failure class recurred three times in August — **Low** · CI/coverage
- **Files:** `docs/GatewayTesting.md:488-498` ("The gateway suite… is not part of the CI Windows tier — `windows-x86` and `nightly-windev` run only the x86 Worker build and `Worker.Tests`. It is still run on windev by hand…"); evidence the class is real: `e2352d1` (two Windows-only gateway test bugs long dismissed as environmental), `bfcf829`/`02baf0c` (zero-buffer Windows pipe wedged the whole suite's testhost), `25f07f8` (SQLite pool sharing-violation on Windows-only cleanup).
- **Description:** TST-25 restored the Worker tier, but the *gateway* suite's Windows-specific divergences (platform crypto formatting, Windows pipe buffer semantics, file-sharing rules) surface only when someone remembers the manual windev run. Every one of the three August finds was a real defect in test code that CI could have caught nightly; the suite's idle wall time on windev is ~31 s, and the nightly job already builds the full slnx there — the marginal cost of running the suite is small. Severity Low, not Medium: the same 879 tests run on Linux per push, the manual procedure is well documented with a current baseline, and known load-sensitivity (the suite flakes when run concurrently with the x86 tier, `GatewayTesting.md:530-555`) means naive stacking into the nightly needs care.
- **Recommendation:** Extend `windev-worker-ci.ps1`'s `live` mode with a `dotnet test src/ZB.MOM.WW.MxGateway.Tests` step **after** the Worker tier finishes (the lock already serializes the box, which addresses the documented load sensitivity), and record the 879 expectation in the script log. If nightly wall time becomes a concern, a weekly cadence still beats "when someone remembers".
### TST-36 — Docs and CLAUDE.md refer to the "`event` scope"; the canonical scope string is `events:read` — **Low** · documentation
- **Files:** `CLAUDE.md:160` (twice: "Possessing the `event` scope…", "an `event`-scoped key"); `docs/Sessions.md:248` (twice); `docs/DesignDecisions.md:93`; `docs/Grpc.md:103` (mitigated — it parenthesizes `GatewayScopes.EventsRead`); canonical constants in `src/ZB.MOM.WW.MxGateway.Server/Security/Authorization/GatewayScopes.cs:10` (`"events:read"`).
- **Description:** `f2a422b` ("canonical scope strings in the two remaining stale lists") fixed the scope *lists* but left the `event`-scope shorthand in running prose. There is no `event` scope: an operator who takes the prose literally and runs `apikey create-key --scopes event` mints a key that cannot stream. CLAUDE.md even uses the correct `events:read` forty lines earlier (`:158`), so the same file teaches both spellings.
- **Recommendation:** Mechanical sweep: replace "`event` scope" with "`events:read` scope" in the four prose sites (Grpc.md's parenthesized form is acceptable as-is). One grep verifies completion: `git grep -n '\`event\` scope'` over `CLAUDE.md docs/` returns nothing.
## Severity roll-up (new findings)
| Severity | Count | IDs |
|---|---|---|
| High | 0 | — |
| Medium | 2 | TST-31, TST-32 |
| Low | 4 | TST-33, TST-34, TST-35, TST-36 |
## Still good / preserved
- **The windev CI tier is the best piece of infrastructure this domain has reviewed in three cycles.** It shipped exactly per design and then got hardened by its own acceptance checks: the cleartext-key leak was found and fixed by storing the key base64 (with the masking rationale documented in three places), the bootstrap lock race was closed with an explicit `MXGW_CI_LOCK_HELD` handoff, the SHA is shape-validated before it reaches a remote shell, an unreachable windev fails red instead of wedging, and the nightly's failure path was itself tested to failure (Check 6) and fixed. `scripts/ci/README.md` records the bring-up, the acceptance checks with dated outcomes, and the degraded mode.
- **The codegen guard now covers every generated surface in both directions:** descriptor set, `Contracts/Generated`, Rust vendored protos (including the reverse sweep for never-vendored canonical protos — drift no in-repo build can surface), Go/Python bindings with pinned generators that fail rather than skip when missing, and the Java aggregates via the java job's git-diff gate. Check 4 was additionally made Windows-runnable, so windev regen workflows use the same guard.
- **The prior cycle's High (TST-25) is not just closed — its unlocks were followed through:** old TST-24 closed with real-transport wire tests in all five clients running in CI, old TST-05's scheduling half closed by the nightly, the EventBurst flake root-caused and deflaked (flush-edge sampling), the two "windev-environmental" failures exposed as test bugs, and the months-old full-suite testhost hang root-caused to a zero-buffer Windows pipe with structural fixes (`TestNamedPipe.CreateServer`, bounded `PipePair.WriteAsync`) plus a first-class post-mortem in `docs/GatewayTesting.md` — the windev baseline now equals macOS at 879.
- **Documentation discipline visibly recovered from its one big self-inflicted wound.** The fca978d sweep's collateral damage was found by a systematic re-read (816 lines judged, 17 restorations), the doc-only `Server-0xx` resolutions were re-audited with the two regressions re-fixed (`d3ac527`), and every doc-currency spot-check this review ran (Sessions, GatewayConfiguration, GatewayTesting, Authentication, alarm probes, pipe naming, write-completion) matched code at HEAD. The two currency findings this cycle (TST-31, TST-33) are inherited debris, not new drift from the ~288-commit period.
- **New features landed with tests at the right layers**: alarm truncation is asserted from the worker flag through the proto, the gateway feed frame, the dashboard banner (including mid-truncation attach), and all five client renders; the SEC-25 tag ACL is pinned at both subscribe seams with decision-table corners and live-LDAP coverage; `regen-readme.py --check` passes; `code-reviews/README.md` is consistent with the per-module findings at HEAD.
@@ -0,0 +1,238 @@
# MxAccessGateway — Remediation Plan & Tracking (2026-08-19 review)
Source review: [`../00-overall.md`](../00-overall.md) (HEAD `8b3e3b3`, committed as `f4919ba`). Generated 2026-08-20.
Tracks the 34 cycle-3 IDs (GWC-31+, WRK-32+, IPC-33+, SEC-37+, CLI-46+, TST-31+) plus the tracker-housekeeping actions the overall asked for. Cycle-1/2 trackers remain the record for their own IDs.
**Status legend:** `Not started` · `In progress` · `Done` · `Blocked (external)` · `Deferred (P2)` · `N/A`.
## Execution model
This plan is executed as **seven parallel batches**, each by one subagent working in an isolated git worktree on its own `fix/c3-<batch>` branch, merged back into `main` by the coordinator in the order listed (lowest conflict surface first). Every batch must: build the affected component(s) on macOS (`NonWindows.slnx` — the x86 worker cannot build here), run only the filtered tests it touches, update docs in the same commit, and cite finding IDs in commit messages (`fix(IPC-34): …`). Worker-side changes (batches E, F) are verified afterwards on windev via `windev-worker-ci.ps1 -Mode test`.
**Model assignment rationale:** `opus` for batches that change concurrency/authorization logic or require reasoning across several interacting files with regression tests; `sonnet` for contained, well-specified edits with a clear recipe (version bumps, doc reconciliation, a two-line worker fix); `haiku` for none — every batch either touches code invariants or needs judgement about stale prose, where the cost of a wrong mechanical sweep exceeds the saving.
| Batch | Branch | Findings | Model | Why this model | Status |
|---|---|---|---|---|---|
| A | `fix/c3-client-versions` | CLI-46 | sonnet | Mechanical five-client bump + README note + one script guard; recipe is fully specified | Done |
| B | `fix/c3-glauth-doc` | TST-31 | sonnet | Reconcile a table against one TOML source of truth, re-derive one default, change an assert to a log | Done |
| C | `fix/c3-tracker-hygiene` | GWC-31 + housekeeping (close GWC-16/IPC-08/IPC-31, refresh TST-06/TST-17, register WRK-31) | sonnet | Cross-file comment/ID sweep needs care about what each ID *means*, but no logic changes | Done (`3619152`) |
| D | `fix/c3-dashboard-acl` | SEC-40, SEC-41, GWC-38 | opus | Authorization semantics, fail-closed decisions, claims re-resolution; each needs a regression test | **Done** (`b177636` GWC-38, `2ef7b93` SEC-41+SEC-40, `8257a03` comment fixup; branch not merged/pushed) |
| E | `fix/c3-ipc-boundaries` | IPC-33, IPC-39, IPC-34, IPC-35 | opus | Changes the WorkerClient command lifecycle, the options validator, and the worker drain budget; race-sensitive | **Done** (`577a0db` IPC-33, `a71663a` IPC-39, `b27035b` IPC-34, `ab41362` IPC-35; branch `fix/c3-ipc-boundaries`, not merged/pushed) — worker half verified on windev (`f2ab9cc`, Worker.Tests 524/524) |
| F | `fix/c3-worker-residuals` | WRK-32, WRK-33 (+WRK-34 opportunistic) | sonnet | Small, precisely located worker edits + doc prose; cannot be built on macOS so the agent's job is a careful, minimal diff | Done (windev x86 build + Worker.Tests 524/524 green on `f2ab9cc`) |
| G | `fix/c3-session-lifecycle` | GWC-32, GWC-34, GWC-37 | opus | Distributor/dispose lifecycle races and an alarm-monitor re-resolve path; needs lock-discipline reasoning | Done (`058d8d5`, `282a0e7`, `1932217`) |
**Out of scope for this execution (recorded, not dropped):**
| ID | Status | Why |
|---|---|---|
| TST-32 | Blocked (external) | Gitea runner registration-token reset is a web-UI action on the Gitea admin page; no repo change. Operator to-do. |
| SEC-37 | Blocked (external) | Family-wide: TLS on the shared GLAuth lives in `scadaproj/infra/glauth/` (separate repo + docker host). Plan separately; after it lands, move the deployed service to `Production`. |
| CLI-47/48/49/50, IPC-36/37, TST-35 | Not started | Activated by the 2026-08-25 P2 continuation below. |
| GWC-33/35, SEC-38/39 | Done | Closed by continuation batch I; regression counts and commits are recorded below. |
| GWC-36, IPC-38, TST-33/34/36 | Done | Closed by continuation batch H; verification and commit are recorded below. |
| WRK-34 | Done | Completed opportunistically in batch F (`3679e59`): watchdog faults use `CommandMethod=Watchdog` and carry the stuck correlation ID in the diagnostic message. Tracker correction recorded by the continuation audit. |
## Batch specifications
### A — CLI-46 client release-train bump (sonnet)
- Bump .NET/Go/Python/Rust clients `0.2.0 → 0.2.1`, Java `0.2.1 → 0.2.2` in every version-bearing file (`pyproject.toml` + `version.py`, `Cargo.toml` workspace + any `version =` consumers, `Directory.Build.props`/csproj `<Version>`, Java `build.gradle`/`gradle.properties`, Go module is tag-driven — update any `Version` const if one exists). Follow `docs/ClientPackaging.md` for the authoritative file list.
- `clients/python/README.md` + `docs/ClientPackaging.md`: note that published pypi 0.2.0 has a broken `connect()` (event-loop binding, fixed by `a8f86b5`) and is superseded by 0.2.1.
- Add a PR-time check to `scripts/check-codegen.ps1` (or a new `scripts/check-client-versions.ps1` wired where the other checks run): fail if any `clients/<lang>` source differs from its last publish tag while the version constant still equals that tag's version. Document in `docs/ClientPackaging.md`.
- Verify: `dotnet build clients/dotnet/...slnx`; `cargo check --workspace`; `python -m pytest` (clients/python); `JAVA_HOME=/opt/homebrew/opt/openjdk@17 gradle test` then **revert** spurious generated-proto churn; `go build ./... && go test ./...`.
### B — TST-31 glauth.md + live-test credential (sonnet)
- Reconcile `glauth.md` against `~/Desktop/scadaproj/infra/glauth/config.toml`: ReadOnly gid, the pre-provisioned users table (drop `readonly/readonly123`, fix `admin` password), the worked example at `:124`. Keep the OPC-UA taxonomy note.
- Re-derive `MXGATEWAY_LIVE_MXACCESS_WRITE_SECURED_PASSWORD` default in `docs/GatewayTesting.md` and the `?? "admin123"` literal in `WorkerLiveMxAccessSmokeTests.cs` from the config; make the test **log** (not assert) whether `AuthenticateUser` succeeded.
- Verify: `dotnet build src/ZB.MOM.WW.MxGateway.NonWindows.slnx` (IntegrationTests compiles on mac? if excluded, note it).
### C — GWC-31 + tracker housekeeping (sonnet)
- Sweep code comments citing unregistered `GWC-30`/`GWC-31` (perf pass) in `WorkerClient.cs`, `GatewayMetrics.cs`, `WorkerClientTests.cs`, worker `WorkerFrameReader/Writer.cs` to `PERF-nn` keyed to task numbers in `docs/plans/2026-08-15-perf-review-remediation.md`; add a PERF-ID table to that plan. Do not touch the genuine GWC-30 (frame-reader prefix) citations if any exist.
- `archreview/REVIEW-PROCESS.md` (or wherever the ID rule lives): state that GWC/WRK/IPC/SEC/CLI/TST prefixes are reserved for archreview registers.
- Cycle-2 tracker (`archreview/2026-07-12/remediation/00-tracking.md` + domain docs): GWC-16 → Won't fix (rationale from cycle-3 10-gateway-core.md), IPC-08 → Done (`7c1ea12`, residual IPC-34), IPC-31 → Done (superseded by GWC-28), refresh TST-06/TST-17 rows, register WRK-31 (interim ID burned by `e913dab`) so it is never reused.
- Verify: `dotnet build src/ZB.MOM.WW.MxGateway.NonWindows.slnx`; `dotnet test --filter FullyQualifiedName~WorkerClientTests`.
### D — SEC-40, SEC-41, GWC-38 (opus)
- SEC-41: `DashboardSessionAcl` re-resolves tags from `LdapGroupClaimType` claims via `DashboardGroupTagMapping.MapGroupsToTags` against current options (as `HubTokenService` does) instead of trusting login-stamped tag claims; keep stamped claims only as fallback when no group claims exist (document). Test: principal with stale tag claim + group whose grant was removed → denied.
- SEC-40: decide **tighten**: `AlarmsHub` gets a dedicated policy requiring a real role claim (anonymous-loopback/Disabled bypass excluded) — and document in `docs/GatewayDashboardDesign.md` ACL section + `docs/GatewayConfiguration.md` GroupToTag row that alarm metadata remains galaxy-wide for authenticated Viewers (central monitor, no per-session provenance). Test for the policy.
- GWC-38: `GatewayBrowseScopeProvider` throws `RpcException(PermissionDenied)` (and logs) when `Current` is null **and** API-key auth is enabled; unchanged when auth mode is Disabled. Test both branches.
- Verify: build Server; `dotnet test --filter "FullyQualifiedName~DashboardSessionAcl|FullyQualifiedName~AlarmsHub|FullyQualifiedName~BrowseScope"` (adjust to real class names).
**Outcome (done, branch `fix/c3-dashboard-acl`, commits `b177636`, `2ef7b93`, `8257a03`; not merged, not pushed).** Decisions worth carrying forward:
- **SEC-41 fallback:** the ACL re-resolves from `mxgateway:ldap_group` claims when the principal has any, and falls back to the stamped `zb:dashboardtag` claims only when it has none. That is exactly the hub-token principal — `HubTokenService.Validate` rehydrates roles and tags but **not** groups, so re-resolution already happened at mint. The fallback cannot re-open the cookie gap, because `DashboardAuthenticator` stamps both claim kinds from one group list.
- **SEC-40 requirement type:** new `AuthenticatedDashboardRoleRequirement` + `AuthenticatedDashboardRoleHandler` and a `MxGateway.Dashboard.Alarms` policy. `DashboardAuthorizationHandler` is `AuthorizationHandler<DashboardAuthorizationRequirement>`, so it is never invoked for the new type — the same structural device the Secrets UI's `secrets:manage` already relies on. Both hub schemes are retained on the policy.
- **SEC-40 scope answer:** the `/alarms` Blazor page carries the SAME policy (it reads the central monitor in process — gating only the hub left the disclosure one navigation away), and the side-rail Alarms link is wrapped in an `AuthorizeView` on that policy. `AlarmsHubPublisher` is unchanged and stays unfiltered by design: audience is decided at hub admission and the feed carries no per-session provenance.
- **GWC-38:** `PermissionDenied` + warning only when `Authentication:Mode=ApiKey`; `Mode=Disabled` still returns unscoped. `GalaxyRepositoryHostWiringTests` phase 1 was changed to push an *unconstrained identity* rather than none — that is production's real unscoped case.
- New/changed tests: `AlarmFeedAuthorizationTests` (new, 7), `DashboardSessionAclTests` (+5), `GatewayBrowseScopeProviderTests` (+2), `SecretsNavRenderTests` (+2, and it must register the new policy — an unregistered policy name *throws* inside `AuthorizeView` rather than denying). Filtered run over the touched classes: 92 passed / 0 failed; `NonWindows.slnx` builds zero-warning.
### E — IPC-33, IPC-39, IPC-34, IPC-35 (opus)
- IPC-33 option (a): `GatewayOptionsValidator` adds the upper coupling `Worker:MaxMessageBytes ≤ Protocol:MaxGrpcMessageBytes + 2×reserve`; worker `ResolveDrainReplyByteBudget` becomes `MaxMessageBytes − 2×DrainReplyFrameHeadroomBytes` (floor `MaxMessageBytes/2`); update `docs/GatewayConfiguration.md`, `docs/WorkerFrameProtocol.md`. Test: `GatewayOptionsTests` case for the new bound.
- IPC-39: pre-check compares against `MaxMessageBytes − MaxSequenceEncodingBytes (11)`; fix the comment direction.
- IPC-34: caller-cancellation arms call `TrySendCancelForAbandonedCommand` (renamed) gated on `removed`, skipped when `_stopCts` is the source; new `WorkerClientErrorCode.CallerCancelled` replacing the `GatewayShutdown` mislabel, mapped to `Cancelled` in the gRPC layer; update `docs/Grpc.md` mapping line, `gateway.md:913-919`, `docs/GatewayProcessDesign.md:596-602`. Test: twin of the timeout test driven by the caller token.
- IPC-35: `ReadHandshakeEnvelopeAsync` special-cases `WorkerFault` → `WorkerClientException(WorkerFaulted, CreateWorkerFaultMessage(...))`. Test: fake writes `WorkerFault{MxaccessCreationFailed}` instead of `WorkerReady`, category text surfaces.
- Verify: build Server; `dotnet test --filter "FullyQualifiedName~WorkerClientTests|FullyQualifiedName~GatewayOptions"`. Worker half → windev afterwards.
### F — WRK-32, WRK-33 (sonnet)
- WRK-33: `ObserveFaultWhenever(ObserveCommandTaskAsync(commandTask));` at `WorkerPipeSession.cs:1235` (or catch-all-and-log in the observer).
- WRK-32: one `pumpStep()` before capturing the `CurrentVersion` baseline in `MxAccessCommandExecutor.cs`; rewrite the executor doc comment (`:996-1003`) to state the recency-based contract; add the same to `docs/MxAccessWorkerInstanceDesign.md` write-completion section and the reply-field comment in `mxaccess_gateway.proto` (**comment-only proto edit → regenerate all generated artifacts in the same commit**, per the repo rule; if regen tooling is unavailable on macOS, skip the proto comment and say so).
- WRK-34 if trivial: `CreateFault(StaHung, "Watchdog", …)` with the correlation id moved into the message.
- Cannot build on macOS: keep diff minimal, net48-safe (no init-only/positional records). Verified on windev afterwards.
### G — GWC-32, GWC-34, GWC-37 (opus)
- GWC-32: `DisposeAsync` transitions `_state` to `Closed` under `_syncRoot` before completing channels (or `if (_disposed || IsTerminalState()) return;` in `StageWorkerEvent`); update the XML doc to name all three completion paths. Test: dispose a Ready client mid-dispatch → no `ProtocolViolation` fault/metric.
- GWC-34: `_eventDistributorDisposed` flag set under `_syncRoot` when nulling; `EnsureDistributorCreated` throws `SessionManagerException(SessionNotReady)` when set. Test: attach after dispose → throws, no silent hang.
- GWC-37: on `ApplyProviderModeChangeAsync(toMode == Subtag)` re-resolve the watch list and re-subscribe; additionally surface "zero targets resolved while mode can need them" on the provider status (`Degraded` + reason). Update `docs/` alarm-subtag section. Test with the fake worker: GR discovery fails at start, succeeds at failover → watch list non-empty after failover.
- Verify: build Server; `dotnet test --filter "FullyQualifiedName~WorkerClientTests|FullyQualifiedName~GatewaySession|FullyQualifiedName~AlarmMonitor|FullyQualifiedName~AlarmWatchList"`.
- **Outcome (Done):** `058d8d5` GWC-32, `282a0e7` GWC-34, `1932217` GWC-37 on `fix/c3-session-lifecycle`. Filtered run 176/176 green; `NonWindows.slnx` clean.
- GWC-37 **does not re-send** the watch-list on the failover edge. The worker's `AlarmCommandHandler.Subscribe` rejects a second `SubscribeAlarms` while subscribed, and the only expressible replacement (`UnsubscribeAlarms` + `SubscribeAlarms`) rebuilds `FailoverAlarmConsumer` — restarting it on the primary that just failed and discarding the warm standby — so it would flap the feed rather than repair it. The re-resolve happens and any drift is reported on the same status frame as the failover; a monitor restart is the supported way to apply a new list. Adding a watch-list-update worker command is a possible future item, not done here.
- GWC-37 changes `IAlarmWatchListResolver.ResolveAsync` to return `AlarmWatchListResolution`, and gives `Degraded && Mode==ALARMMGR` a real meaning (empty/undiscoverable standby) with a new dashboard label `Alarm Manager (no subtag standby)`.
- GWC-32's dispose path now also records `WorkerStopped("dispose")` via the existing once-latch, closing a `workers_running` gauge leak on the dispose-a-Ready-client path. GWC-12's double-dispose interlock got a natural home (the `_disposed` claim moved under `_syncRoot`) but is **not** claimed as fixed.
## Merge order
C is merged **last** (its comment sweep touches the same files E and G edit). Order: A, B, F, D, G, E, C. After merge: full `NonWindows` build + one full gateway test run; then windev worker CI for E+F.
## Integration gate (2026-08-20)
All seven branches merged into `main` (A `ee68cc3`, B `f59c34a`, C `f2ab9cc`, D `9f0c3fd`, E `cfbc995`, F `aa07497`, G `c6650c1`). Merged tree: `NonWindows.slnx` 0 warnings; full gateway suite **1183/1183** passed; windev `windev-worker-ci.ps1 -Sha f2ab9cc -Mode test`: x86 Worker build succeeded, `Worker.Tests` **524 passed / 11 skipped / 0 failed**. Follow-ups surfaced by the batches: a worker command to refresh a live alarm subscription's watch-list (would let GWC-37's re-resolve be applied, not just reported); WRK-32's proto reply-field comment deferred until the next proto regen wave; the five clients are bumped but not yet published.
## P2 continuation (planned 2026-08-25)
Current-state audit: refreshed `origin/main` and re-checked every deferred finding and integration-gate follow-up. Sixteen of the seventeen P2 findings remain open; WRK-34 was already completed by `3679e59`. All three follow-ups remain open. Exact target client coordinates are still absent from Gitea (HTTP 404), and `clients/go/v0.2.1` does not exist.
Approved design: [`../../../docs/plans/2026-08-25-cycle-3-p2-followups-design.md`](../../../docs/plans/2026-08-25-cycle-3-p2-followups-design.md) (`30548da`).
Execution plan: [`../../../docs/plans/2026-08-25-cycle-3-p2-followups.md`](../../../docs/plans/2026-08-25-cycle-3-p2-followups.md).
The continuation runs serially in the isolated `.worktrees/c3-p2-followups` worktree on `codex/c3-p2-followups`, with a tracker update at every batch boundary. Codex equivalents are selected from the implementation risk: `gpt-5.6-terra`/medium for contained documentation, CI, and client work; `gpt-5.6-sol`/high for concurrency, authorization, protocol, and live alarm lifecycle changes.
| Batch | Findings / follow-ups | Codex model | Status | Exit gate |
|---|---|---|---|---|
| H — tracker and currency | WRK-34 record, GWC-36, IPC-38, TST-33/34/36 | `gpt-5.6-terra` medium | Done | Currency greps clean; README check in portable CI; NonWindows build clean |
| I — gateway correctness/security | GWC-33/35, SEC-38/39 | `gpt-5.6-sol` high | Done | Focused regression tests green; NonWindows build clean |
| J — secondary contracts | IPC-36/37 | `gpt-5.6-sol` high | Done | Shared launch constants and descriptor parity tests green |
| K — client alarm parity | CLI-47/48/49/50 | `gpt-5.6-terra` high | Done | All five client suites green |
| L — live watch-list/proto wave | watch-list refresh follow-up + WRK-32 proto comment | `gpt-5.6-sol` high | Done | Codegen clean; portable alarm/contract tests green; windev Worker.Tests green |
| M — Windows gateway CI | TST-35 | `gpt-5.6-terra` medium | Done | windev `live` mode proves Worker, gateway, full-slnx, and live smoke tiers |
| N — integration and release | publish five clients + Contracts | `gpt-5.6-sol` high | In progress | Full local/windev gates; merged `main`; registry coordinates and Go tag verified |
### Continuation tasklist
#### H — tracker and currency
- [x] Correct WRK-34 from Deferred to Done with commit `3679e59`.
- [x] GWC-36: correct `EventSubscriberReplayAttachment.OldestAvailableSequence` semantics.
- [x] IPC-38: add public `DrainEvents` behavior to `docs/Grpc.md` and correct the cycle-2 omission rationale.
- [x] TST-33: reconcile the five stale/self-contradictory `stillpending.md` claims and mark its live-status boundary.
- [x] TST-34: run `code-reviews/regen-readme.py --check` in portable CI and qualify the interpreter note.
- [x] TST-36: replace prose ``event`` scope references with canonical ``events:read``.
- [x] Record batch-H commit and verification evidence here.
Batch-H verification (2026-08-25): `python3 code-reviews/regen-readme.py --check`
reported the generated README current and consistent; the TST-33 and live-document
TST-36 stale-pattern greps returned no matches; `docs/Grpc.md` contains the public
`DrainEvents` contract; and `dotnet build src/ZB.MOM.WW.MxGateway.NonWindows.slnx`
succeeded with **0 warnings / 0 errors** in 1.59 s. Batch commit: `92aae28`.
#### I — gateway correctness and security
- [x] GWC-33: unregister terminal clients from the event-queue-depth gauge without shortening fault retention.
- [x] GWC-35: repair acknowledged-to-active snapshot deltas with a Raise transition and instance identity.
- [x] SEC-38: feed/consult the aggregate for valid key IDs that overflow the peer partition cap.
- [x] SEC-39: prove a null effective key ID cannot reset the shared fallback partition.
- [x] Run focused tests and clean NonWindows build; record counts and commits here.
Batch-I verification (2026-08-25): the combined `WorkerClientTests`,
`GatewayAlarmMonitorAttachOrderTests`, and `ApiKeyFailureLimiterTests` filter passed
**55/55** tests in 12 s; `dotnet build src/ZB.MOM.WW.MxGateway.NonWindows.slnx`
succeeded with **0 warnings / 0 errors** in 3.42 s. Finding commits: GWC-33
`b0156c6`, GWC-35 `7807201`, and SEC-38/SEC-39 `1875426`.
#### J — secondary contracts
- [x] IPC-36: centralize all six worker-launch environment names in Contracts and consume them on both sides.
- [x] IPC-36: document variable/config/default mapping and test launcher completeness.
- [x] IPC-37: add normalized retained-vs-served Galaxy descriptor parity test.
- [x] Run focused tests and clean NonWindows build; record counts and commits here.
Batch-J verification (2026-08-25): the combined `WorkerProcessLauncherTests` and
`GalaxyContractParityTests` filter passed **10/10** tests in 1 s; the descriptor
mutation proof identified the changed protobuf message; and `dotnet build
src/ZB.MOM.WW.MxGateway.NonWindows.slnx` succeeded with **0 warnings / 0
errors** in 2.20 s. IPC-36/IPC-37 commit: `d6c5a11`.
#### K — client alarm-feed parity
- [x] CLI-47: make .NET text status flags explicit and document deliberate JSON differences.
- [x] CLI-48: test provider/snapshot status arms in .NET, Go, and Java; add Go stream-alarm scaffold.
- [x] CLI-49: export Go aliases for `AlarmProviderStatus` and `AlarmSnapshotStatus`; fix five-arm documentation.
- [x] CLI-50: stream Python alarm rows incrementally with an unbounded default and bounded opt-in.
- [x] Run .NET, Go, Python, Rust, and Java client suites; record counts and commits here.
Batch-K verification (2026-08-25): `dotnet test
clients/dotnet/ZB.MOM.WW.MxGateway.Client.slnx` passed **134/134** portable
tests (1 live test skipped); `go test ./...` passed both tested packages;
`python -m pytest` passed **169/169** portable tests (1 opt-in TLS test
skipped); `cargo test --workspace` passed **108/108** tests; and
`JAVA_HOME=/opt/homebrew/opt/openjdk@17 gradle test` passed **132/132** tests.
CLI-47/CLI-48/CLI-49/CLI-50 commit: `5cdb3da`.
#### L — live watch-list refresh and protobuf wave
- [x] Add explicit fallback-enabled subscription semantics for an initially empty discovered list.
- [x] Add and round-trip `UpdateAlarmWatchListCommand` and its `MxCommandKind`.
- [x] Route the command through the worker executor and alarm handler on the STA.
- [x] Build/subscribe/attach a replacement standby before swap; preserve the old standby on failure.
- [x] Preserve current primary/subtag mode during a successful replacement.
- [x] Re-resolve on the existing gateway reconciliation cadence and send only material changes/recovery.
- [x] Refresh immediately at failover edge, update acknowledged fingerprint only after worker OK, and reconcile after success.
- [x] Expand WRK-32's public reply-field comment to the sequential stale-callback/recency case.
- [x] Replace IPC-36's internal env-var name in the proto comment with the public configuration key.
- [x] Regenerate Contracts and all client generated trees once; run codegen and all affected suites.
- [x] Verify x86 Worker build and Worker.Tests on windev; record SHA/counts here.
Batch-L verification (2026-08-25): `pwsh scripts/check-codegen.ps1` passed all
four descriptor/C#/Rust/Go/Python freshness checks, and Java
`checkGeneratedClean` passed. The exact portable alarm/contract/descriptor
filter passed **77/77**; `NonWindows.slnx` built with **0 warnings / 0 errors**;
the .NET, Go, Python, Rust, and Java client suites were green (respectively
**134 passed / 1 live skip**, both Go packages, **169 passed / 1 opt-in TLS
skip**, **108 passed**, and **132 passed**). Windev `test` mode at
`986c6c0ce4f02c62b4f59ec44780de5af6d291b8` built the x86/net48 Worker with
**0 warnings / 0 errors** and passed Worker.Tests **529 passed / 11 opt-in
skips / 0 failed**. Feature/proto commit: `484028e`; Batch-J x86 compile
follow-up: `986c6c0`.
#### M — Windows gateway CI
- [x] TST-35: add the serialized net10 gateway suite to windev `live` mode.
- [x] Update `docs/GatewayTesting.md` and `scripts/ci/README.md` with order/failure semantics.
- [x] Execute `live` mode on windev and record Worker, gateway, full-slnx, and live-smoke evidence here.
Batch-M verification (2026-08-25): windev `live` mode at exact commit
`cc143b52cf04be0b2f08350922a987b2e250bc1d` completed the serialized sequence in
**4m45s** with remote exit code 0. The x86/net48 Worker build succeeded with **0
warnings / 0 errors** in 22.59 s; Worker.Tests passed **529**, skipped **11**
opt-in probes, and failed **0**; the full Windows `slnx` build succeeded with **0
warnings / 0 errors** in 43.24 s; the net10 gateway suite passed **1193/1193**
in 53 s; and the live-MXAccess smoke passed **8/8** in 1m46s. TST-35 commit:
`cc143b5`.
#### N — integration and release
- [x] Run `git diff --check`, codegen, client-version, README, NonWindows build, and full gateway suite gates.
- [x] Run all five client suites on the final candidate.
- [ ] Run final windev `live` gate at the exact candidate SHA.
- [ ] Merge/push the verified continuation to `main`; wait for green CI.
- [ ] Re-query all package coordinates and Go tag immediately before publish; abort on collision.
- [ ] Publish .NET Client + Contracts 0.2.1, Python 0.2.1, Rust 0.2.1, Java 0.2.2.
- [ ] Create/push `clients/go/v0.2.1` at the exact release commit.
- [ ] Verify every registry coordinate and Go module/tag, update `docs/ClientPackaging.md`, and record immutable release evidence here.
Batch-N portable verification (2026-08-25): `git diff --check`, codegen freshness,
client-version freshness, and the generated README check all passed without changing
the worktree. `NonWindows.slnx` built with **0 warnings / 0 errors**; the gateway
suite passed **1193/1193**, and the portable integration suite passed **5** with
**19** live-only skips. The final .NET, Go, Python, Rust, and Java client gates were
green (respectively **134 passed / 1 live skip**, all Go packages built/tested with
clean `gofmt`, **169 passed / 1 opt-in TLS skip** from a clean virtual environment,
**108 passed** plus clean `cargo fmt`/Clippy, and **132 passed / 0 failed** with the
generated Java tree unchanged).
+1 -1
View File
@@ -49,7 +49,7 @@ Impact: logout (`Dashboard/DashboardEndpointRouteBuilderExtensions.cs:136-155`)
Recommendation: keep the lifetime short (or shorten to ~5 minutes given the factory refreshes per reconnect, `docs/GatewayDashboardDesign.md:497-499`), and confirm no request-path logging captures query strings (Serilog request logging is not currently enabled; keep it that way or scrub `access_token`).
**SEC-6 · Medium — LDAP is plaintext-by-default with a committed service-account password.**
Evidence: `src/ZB.MOM.WW.MxGateway.Server/Configuration/LdapOptions.cs:49-61` (defaults `Transport=None`, `AllowInsecure=true`, `ServiceAccountPassword = "serviceaccount123"`), `appsettings.json:21-33` (same values checked into the repo), `glauth.md:30,327` (dev LDAPS disabled; "binding sends passwords cleartext on the wire").
Evidence: `src/ZB.MOM.WW.MxGateway.Server/Configuration/LdapOptions.cs:49-61` (defaults `Transport=None`, `AllowInsecure=true`, `ServiceAccountPassword = "<service-account-password>"` — value redacted per SEC-36), `appsettings.json:21-33` (same values checked into the repo), `glauth.md:30,327` (dev LDAPS disabled; "binding sends passwords cleartext on the wire").
Impact: every dashboard login sends the operator's password in cleartext to `10.100.0.35:3893`, and the LDAP service-account credential is in source control. This is a documented dev posture (the shadow-options rationale at `LdapOptions.cs:20-28` is explicit that the shared library is secure-by-default), and the validator does enforce the `Transport=None ⇒ AllowInsecure` consistency rule (`GatewayOptionsValidator.cs:82-85`) — but nothing distinguishes dev from prod at runtime.
Recommendation: for production deployment docs, require `Transport=Ldaps`/`StartTls` + `AllowInsecure=false` and move `ServiceAccountPassword` to env-var/secret configuration; consider an `IsProduction` startup check mirroring SEC-4. LDAP injection risk is delegated to the shared `ZB.MOM.WW.Auth.Ldap` provider (bind-then-search per `Dashboard/DashboardAuthenticator.cs:41-47`); its escaping cannot be verified from this repo — flag for review in the donor repo.
+17 -11
View File
@@ -74,7 +74,7 @@ Full design + implementation for each row lives in the linked domain doc under i
| GWC-13 | Low | — | M | — | Not started | Worker-ready wait is a 25 ms poll loop |
| GWC-14 | Low | P2 | M | — | Done | Replay ring is a `LinkedList` with a node alloc per event |
| GWC-15 | Low | P2 | S | — | Done | Per-event gauge reads `ChannelReader.Count` every event |
| GWC-16 | Low | — | S | — | Not started | `Invoke` resolves the session twice per command |
| GWC-16 | Low | — | S | — | Won't fix | `Invoke` resolves the session twice per command — measured negligible, deliberately not collapsed (see domain doc) |
| GWC-17 | Low | — | M | — | Not started | Sessions layer throws `Grpc.Core.RpcException` (layering leak) |
| GWC-18 | Low | — | S | — | Not started | `GatewaySession` implements `DisposeAsync` without `IAsyncDisposable` |
| GWC-19 | Low | — | S | — | Not started | Dead `GatewaySession.KillWorker(string)` + stale doc |
@@ -119,7 +119,7 @@ Full design + implementation for each row lives in the linked domain doc under i
| IPC-05 | Medium | P2 | S | — | Done | Redundant deep copies on command and event hot paths |
| IPC-06 | Medium | P2 | S | — | Done | `gateway.md` Worker Envelope sketch no longer matches the contract |
| IPC-07 | Medium | P2 | S | — | Done | `docs/Grpc.md` says six RPCs; there are seven |
| IPC-08 | Medium | — | M | — | Not started | `WorkerCancel` defined and handled but never sent |
| IPC-08 | Medium | — | M | — | Done | `WorkerCancel` defined and handled but never sent — resolved by `7c1ea12`/`a1a38b5`; residual (caller-cancellation leg) tracked as cycle-3 IPC-34 |
| IPC-09 | Medium | P1 | M | IPC-01 | Done | Known codegen fragilities have no in-repo guards |
| IPC-10 | Low | — | S | — | Not started | Envelope `sequence` is write-only; monotonicity never validated |
| IPC-11 | Low | — | S | — | Not started | No protocol version negotiation despite `supported_protocol_version` name |
@@ -181,7 +181,7 @@ Full design + implementation for each row lives in the linked domain doc under i
| CLI-05 | Medium | — | S | — | Not started | .NET session cannot be re-attached to an existing session id |
| CLI-06 | Medium | — | S | — | Not started | .NET `DisposeAsync` blocks/throws on unreachable gateway |
| CLI-07 | Medium | — | S | — | Not started | .NET retry budget self-defeats on `DeadlineExceeded` |
| CLI-08 | Medium | — | S | CLI-03 | Not started | .NET/Go/Java treat any nonzero HRESULT as failure (should be `< 0`) |
| CLI-08 | Medium | — | S | CLI-03 | Done | .NET/Go/Java treat any nonzero HRESULT as failure (should be `< 0`) — landed via 2026-07-12 [CLI-38](../2026-07-12/remediation/50-clients.md#cli-38--align-netgojava-on-hresult--0-lands-prior-cli-08-cures-the-doc-drift---medium--p1) |
| CLI-09 | Medium | — | M | — | Not started | Go has no typed auth-error mapping (Unauthenticated vs PermissionDenied) |
| CLI-10 | Medium | — | M | — | Not started | Go uses deprecated `grpc.DialContext` + `grpc.WithBlock()` |
| CLI-11 | Medium | — | S | — | Not started | Go CLI cannot opt into strict TLS validation |
@@ -197,7 +197,7 @@ Full design + implementation for each row lives in the linked domain doc under i
| CLI-21 | Low | P2 | S | — | Done | Go `ClientVersion = "0.1.0-dev"` stale vs tagged releases |
| CLI-22 | Low | — | S | — | Not started | Go `newCorrelationID` swallows `crypto/rand` error → empty id |
| CLI-23 | Low | — | S | — | Not started | Go nil-vs-empty bulk short-circuit asymmetry |
| CLI-24 | Low | — | S | — | Not started | Java `MxEventStream` single-consumer constraint undocumented |
| CLI-24 | Low | — | S | — | Done | Java `MxEventStream` single-consumer constraint undocumented (closed 2026-08-07 per 2026-07-12 review old-tracker action; documented at MxEventStream.java:25 "Single consumer") |
| CLI-25 | Low | — | S | — | Not started | Java `close()` does not await channel termination |
| CLI-26 | Low | P2 | S | — | Done | Python `version.py` (0.1.0) ≠ `pyproject.toml` (0.1.2) |
| CLI-27 | Low | — | S | — | Not started | Python `Session.close()` not concurrency-safe; synthesizes reply |
@@ -207,7 +207,7 @@ Full design + implementation for each row lives in the linked domain doc under i
| CLI-31 | Low | — | M | — | Not started | Rust CLI is a single 2,699-line `main.rs` |
| CLI-32 | Low | — | S | — | Not started | Client-side bulk caps differ (.NET/Java unbounded) |
| CLI-33 | Low | — | S | CLI-01,13 | Not started | Per-language event backpressure semantics undocumented |
| CLI-34 | Low | — | S | — | Not started | Python `build/`/`.pytest_cache/` present on disk (untracked) |
| CLI-34 | Low | — | S | — | Done | Python `build/`/`.pytest_cache/` present on disk (untracked) (closed 2026-08-07 per 2026-07-12 review old-tracker action; both gitignored in clients/python/.gitignore) |
### Testing, docs & gaps — [60-testing-docs-gaps.md](60-testing-docs-gaps.md)
@@ -217,8 +217,8 @@ Full design + implementation for each row lives in the linked domain doc under i
| TST-02 | High | P0 | M | TST-04 | Done | Reconnect owner re-validation not implemented |
| TST-03 | High | P1 | M | — | Done | No CI exists |
| TST-04 | High | P2 | L | — | Done | Session-resilience epic 16/28 tasks unfinished |
| TST-05 | Medium | P1 | S | TST-03 | Not started | Real-worker control/COM paths verified opt-in only |
| TST-06 | Medium | — | M | — | Not started | Dashboard live-data path untested |
| TST-05 | Medium | P1 | S | TST-03 | Partially done | Real-worker control/COM paths verified opt-in only. **Scheduling half closed 2026-08-10** by the TST-25 `nightly-windev` job (`.gitea/workflows/ci.yml`, cron `0 6 * * *` → `scripts/ci/run-windev-ci.sh live`, which sets `MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1`, runs `WorkerLiveMxAccessSmokeTests`, and opens a Gitea issue on failure) — "opt-in, run by memory" is now "runs nightly, reports failures". **Residual: the coverage-audit half.** The live suite's 8 facts cover all six late-added COM commands but none of the five control commands (`Ping`, `GetSessionState`, `GetWorkerInfo`, `DrainEvents`, `ShutdownWorker`), which real workers answer in `Worker/Ipc/WorkerPipeSession.cs` yet are still only exercised through `FakeWorkerHarness` canned replies — precisely the masking the finding named |
| TST-06 | Medium | — | M | — | Materially narrowed | Dashboard live-data path untested — `DashboardLiveDataServiceTests`, `EventsHubTests`, `EventsHubViewerRegistryTests`, `AlarmsHubPublisherTests`, `SessionDetailsPageEventAclTests` now exist (landed with SEC-25/TST-16); needs a re-audit-and-close pass, not re-implementation |
| TST-07 | Medium | — | S | — | Not started | Real-clock sleeps with negative assertions are latent flakes |
| TST-08 | Medium | P1 | M | — | Done | Full-suite orphaned testhost processes (does not reproduce; doc de-stale) |
| TST-09 | Medium | — | M | — | Not started | Health checks cover only the auth store |
@@ -227,16 +227,16 @@ Full design + implementation for each row lives in the linked domain doc under i
| TST-12 | Medium | P0 | S | — | Done | CLAUDE.md misstates default retention behaviour |
| TST-13 | Medium | P2 | S | — | Done | gateway.md carries stale design-era sketches |
| TST-14 | Medium | P2 | S | — | Not started | Repo-root working artifacts need triage |
| TST-15 | Medium | P2 | M | TST-04 | Not started | Dashboard EventsHub has no per-session ACL |
| TST-16 | Medium | — | S | — | Not started | `Dashboard:ShowTagValues` is a dead flag |
| TST-17 | Medium | — | S | — | Not started | Vendor-gated alarm parity residuals silently lossy |
| TST-15 | Medium | P2 | M | TST-04 | Done | Dashboard EventsHub has no per-session ACL |
| TST-16 | Medium | P1 | S | — | Done | `Dashboard:ShowTagValues` is a dead flag |
| TST-17 | Medium | — | S | — | Materially changed | Vendor-gated alarm parity residuals silently lossy — `WnWrapAlarmConsumer.cs:309-327` now passes `szDomainName`/`szOprFullName`; the vendor 8-arg leg is recorded "unavailable by configuration" with evidence (`docs/AlarmProbeFindings.md`, `d1ae43d`, `540a118`); "silent drop, unchanged" is no longer accurate — needs a worker-domain refresh pass |
| TST-18 | Low | — | S | — | Not started | Hosted-service wrappers untested |
| TST-19 | Low | — | S | — | Not started | Keep FakeWorkerHarness canned replies in lockstep |
| TST-20 | Low | — | S | — | Not started | E2E script papers over a real advise-without-consumer sharp edge |
| TST-21 | Low | — | S | — | Not started | Log rotation configured but minimal |
| TST-22 | Low | — | S | — | Not started | Config-shape JSON block omits documented keys |
| TST-23 | Low | P2 | S | — | Done | Bidirectional `Session` RPC never built |
| TST-24 | Low | P2 | M | TST-03 | Not started | Client wire behaviour has no automated verification. **Gate cleared:** TST-03 CI is Done (live and green 2026-07-10; Windows/x86 tier green 2026-07-13 via the TST-25/TST-26 SSH-driven windev job), so TST-24 is unblocked — deferred by choice now, not CI-gated |
| TST-24 | Low | P2 | M | TST-03 | Done | Client wire behaviour has no automated verification — closed 2026-08-10. Go/Rust/Java already had real-server wire tests (the finding's premise was stale); the genuine gaps were .NET (transport-interface fake everywhere, no server package) and Python (stub monkeypatch everywhere but one opt-in TLS test). Added `MxGatewayClientWireTests` + `WireFakeGatewayServer` (Kestrel h2c) and `tests/test_wire_fake_gateway.py` (`grpc.aio` loopback), plus a `dotnet test` step for the .NET client in the `portable` CI job. Caught a real bug: Python `connect()` built the `grpc.aio` channel inside `asyncio.to_thread` and failed for every non-stub connection |
## Cross-cutting clusters
@@ -253,6 +253,12 @@ Findings the review flagged as one coordinated design pass — sequence them tog
| Date | Change |
|---|---|
| 2026-08-17 | **TST-16 → `Done`: `Dashboard:ShowTagValues` now gates every dashboard value surface** (branch `feat/followup-closeout`). The flag was live on one seam only (the events-hub mirror, shipped with SEC-25); the two recorded residuals are closed, so it is no longer a partly-dead flag. **Alarms hub:** `AlarmsHubPublisher` took `IOptions<GatewayOptions>` and now clears `current_value`/`limit_value` from a **deep clone** of each `AlarmFeedMessage` on both value-bearing payload arms (`transition`, `active_alarm`) before the `/hubs/alarms` broadcast. Clone-only, matching `DashboardEventBroadcaster.RedactValues`: the source message fans out from one feed to gRPC `StreamAlarms` subscribers and the alarms page, none of which this dashboard-display flag governs. The redaction switches on the value-bearing arms and passes everything else through untouched and uncloned, so a payload arm added later (the `snapshot_status` arm landing alongside this change) is forwarded unbroken rather than silently dropped. **`/browse`:** the gate lives at the service boundary per TST-16's own recommendation — `DashboardLiveDataService` substitutes `DashboardTagValue.RedactedValueText` (`[redacted]`) for the formatted value text, so the value never enters the render tree and `BrowsePage.razor` needed no change; data type, quality, source timestamp, and read errors stay real, keeping the panel diagnostic. A visible marker, not a blank, so a suppressed value is distinguishable from a tag that read back empty. Tests: new `AlarmsHubPublisherTests` (4 — both arms redacted with metadata intact, source not mutated, `true` passes values through as the same instance, valueless arms forwarded as the same instance) and two `DashboardLiveDataServiceTests` cases (`false` → `[redacted]`, `true` → formatted); `DashboardEventBroadcasterTests` re-run as regression. Docs: `docs/GatewayConfiguration.md`'s `ShowTagValues` row rewritten from "gates only the events hub mirror, two still-open residuals" to the three seams it now covers, with the "no gRPC client is affected" scope statement; `docs/GatewayDashboardDesign.md` gained the redaction note on the `AlarmsHub` row and had its Browse-panel paragraph corrected (it claimed the panel "always shows values regardless of `Dashboard:ShowTagValues`"). |
| 2026-08-17 | **Alarm-snapshot truncation now has a structural degraded-status signal** (branch `feat/deferred-closeout`, commits `693a78d` + `b9fb0dd`). No review ID — this is branch work outside the 153-finding register, recorded here so the tracker is not silent on a shipped change to the alarm surface. Before it, a capped `GetXmlCurrentAlarms2` fetch suppressed absence-implies-Clear inference and said so only in a rate-limited worker stderr warning, so no client and no operator could tell a complete active set from a capped one. Two additive proto3 booleans carry the verdict out — `QueryActiveAlarmsReplyPayload.snapshot_truncated = 2` and `ActiveAlarmSnapshot.from_truncated_snapshot = 16` (per record, because `QueryActiveAlarms` is a bare `stream ActiveAlarmSnapshot` with no envelope; the reply payload states it too, since a prefix filter can leave zero records and a capped fetch with nothing to report still has to say so). Flow: `WnWrapAlarmConsumer` → `AlarmDispatcher` / `IAlarmCommandHandler` → `MxAccessCommandExecutor` reply → `GatewayAlarmMonitor` → `IGatewayAlarmService.SnapshotTruncated` → `AlarmsPage` banner. `b9fb0dd` then made the pairing structural after review: `IMxAccessAlarmConsumer` and `IAlarmCommandHandler` expose one accessor (`SnapshotActiveAlarms(out bool truncated)` / `QueryActive(..., out bool snapshotTruncated)`) satisfied from a single lock acquisition, so the snapshot and its verdict can no longer be read across a poll; the separate `LastSnapshotTruncated` property is gone from every layer. Detection is deliberately unchanged (`fetchedRecordCount >= maxAlarmsPerFetch`); switching to `ALARM_RECORDS/@COUNT` stays blocked on probe evidence (`docs/AlarmProbeFindings.md`). Not latched, and dropped with the cache generation by `ClearCache`. Additive gateway metadata about our fetch mechanics, not MXAccess behaviour — no synthesized event, so not a parity deviation. Docs: `gateway.md` alarm surface, `docs/DesignDecisions.md`. |
| 2026-08-17 | **TST-15 → `Done` (discharges the ACL half of SEC-25): per-session dashboard event ACL shipped** (branch `feat/deferred-closeout`, commits `693a78d` + `7ec0b35`). Implements `docs/plans/2026-07-10-dashboard-session-acl-tst15.md`, whose header is now `Implemented` with as-built notes in its §12. `IDashboardSessionAcl.CanViewSession` is the single decision **both** subscribe seams consult — `EventsHub.SubscribeSession` (denial is a `HubException`; the caller is neither joined to the group nor registered in `EventsHubViewerRegistry`, so the mirror stays off) and `SessionDetailsPage`'s in-process subscribe (inline denial, no subscription) — so neither path is the weaker one and the `TODO(per-session-acl)` is gone. Decision order, fail-closed on every branch: authenticated Administrator → allow (evaluated **before** the registry lookup, so Admin × unknown-session allows — pinned by a test because reordering the two checks is a plausible refactor); session not found → deny; untagged session → `Dashboard:UntaggedSessionVisibility` (`AdminOnly` default); else allow iff `session.Tags ∩ zb:dashboardtag` claims, ordinal-ignore-case. Session tags are inherited from the owning API key's `dashboard_tags` constraint (`apikey --dashboard-tags`, already in the `ApiKeyConstraints` JSON blob — no SQLite migration) and never from the client's wire request. Viewer grants come from `Dashboard:GroupToTag` over the user's LDAP groups, stamped at cookie login (`DashboardAuthenticator.CreatePrincipal`) and **re-resolved, not copied**, at hub-token mint (`HubTokenService.Issue`), so the 5-minute token lifetime bounds a stale grant. Anonymous localhost is an empty-grant Viewer; `Dashboard:DisableLogin` auto-login carries both roles and so takes the admin bypass unchanged. Tests: `DashboardSessionAclTests` (decision table, every branch asserted in its denying direction too), `EventsHubTests`, `DashboardAuthenticatorTests`, `HubTokenServiceTests`, a `GatewayOptionsTests` case proving `Dashboard:GroupToTag` keeps its ordinal-ignore-case lookup through configuration binding, and two `[LiveLdapFact]`s in `DashboardLdapLiveTests` that drive a real bind against the shared GLAuth (`gw-viewer` → `team-a` grant admits the `team-a` session and refuses the `team-b` one; `multi-role` bypasses on the sessions its own grant does not cover). The live pair needed **no GLAuth change** — the tag layer is config-side, keyed on the existing `GwAdmin`/`GwReader` groups (recorded in `glauth.md`). Docs: `docs/Sessions.md` (session-tag model), `gateway.md` + CLAUDE.md dashboard-auth paragraphs, `docs/GatewayDashboardDesign.md` (three passages that described the ACL as outstanding), `docs/GatewayConfiguration.md` (`ShowTagValues` row: redaction is now the second of two layers, not the only one), `docs/Authentication.md` (`--dashboard-tags` is the only *constraint* flag that splits on commas). |
| 2026-08-10 | **TST-25 acceptance Check 6 (forced-failure nightly issue) → Done.** The 2026-07-13 record wrote this check off as "abandoned to shared-runner congestion"; that was wrong on both counts. The 2026-07-13 probe *did* land (issue #125, `[CHECK6 PROBE]`, run 375), and since 2026-07-17 the `nightly-windev` `if: failure()` step has filed an issue on **every** red nightly — #126–#139, all authored by the `gitea-actions` bot. Traced run 672 (schedule, main, red) line by line: main step fails → `exitcode '1': failure` → the `if: failure()` step runs → `POST /api/v1/repos/dohertj2/mxaccessgw/issues` with the built-in token masked to `***` → issue #139 created at the matching timestamp. Re-confirmed by a fresh forced-failure probe on the throwaway branch `test/tst25-check6-nightly-issue` (temporary `tst25-check6-probe.yml` reproducing the job shape with `exit 1` for the live step; run 677 → issue #140). Branch deleted, issues #125 and #140 closed with explanatory comments. **One real defect found and fixed** (`fix/tst25-nightly-issue-path`, not merged): `${{ github.server_url }}` is the runner-internal `http://gitea:3000`, so every filed issue's run link was unreachable from a browser. The API call must keep using it (the job container resolves `gitea` only on the docker network and has no LAN egress to the public origin), so the fix adds a `PUBLIC_SERVER_URL: https://gitea.dohertylan.com` job env used **only** for the browser-facing link in the issue body; the probe validated the fixed template (#140 carries a `https://gitea.dohertylan.com/...` link that returns 200). **Separately observed, not fixed:** the nightly has been red continuously since at least 2026-07-17 (run 672: `x86 Worker.Tests failed with exit code 1`, 1 failed / 398 passed / 11 skipped — the known `EventBurst_DrainLoopCoalescesFlushes` class of flake), and the step de-duplicates nothing, so 14 issues are open, seven of them (#132–#138) for the identical SHA `47c0b64`. Worth a follow-up: fix the red nightly, and consider having the step reuse an open issue with the same title instead of filing a new one. |
| 2026-08-10 | **TST-24 → `Done`: per-client wire tests land for the two clients that lacked them** (branch `feat/tst-24-client-wire-tests`). Audit first corrected the finding's premise: **Go, Rust, and Java already had real-server wire tests** — `newBufconnClient`/`fakeGatewayServer` over `grpc/test/bufconn`, `spawn_fake_gateway` over a loopback `TcpListener` with tonic's `Server`, and `InProcessGateway`/`TestGatewayService` over `InProcessServerBuilder` — each already asserting the round trip, the server-observed `authorization` bearer header, and the `ReplayGap` sentinel. The real gaps were **.NET** (every test substituted `FakeGatewayTransport` for `IMxGatewayClientTransport`, and the test project had no server package) and **Python** (stub monkeypatching everywhere except one opt-in TLS test serving only `OpenSession`). Added `WireFakeGatewayServer` + `MxGatewayClientWireTests` (Kestrel h2c on `127.0.0.1:0` serving `MxAccessGatewayBase`; new `Grpc.AspNetCore.Server` 2.76.0 + `Microsoft.AspNetCore.App` refs on the test project) and `clients/python/tests/test_wire_fake_gateway.py` (`grpc.aio` server on `127.0.0.1:0`, no new deps). Four shapes each: full round trip with every reply field asserted, the bearer header **as received by the server** on the streaming RPC too, the `ReplayGap` sentinel surfaced as the client's typed signal, and a genuine `PERMISSION_DENIED` mapping to the typed authorization error. CI: the `portable` job only *built* the .NET client, so a `dotnet test` step was added. **The new tests immediately caught a shipped bug** — Python `GatewayClient.connect()`/`GalaxyRepositoryClient.connect()` constructed the `grpc.aio` channel inside `asyncio.to_thread`, which raises `RuntimeError: There is no current event loop in thread 'asyncio_0'` because a `grpc.aio` channel binds to the loop current on the constructing thread; every non-stub connection failed, and the one test guarding the off-loop behaviour (Client.Python-028) monkeypatched `create_channel` and so asserted the bug. Fixed by splitting `resolve_channel_security` (blocking TOFU probe, off-loop) from `create_channel` (on-loop), with the `-028` tests retargeted to assert both halves. Verified: .NET 133 passed/1 skipped (pre-existing live-gateway skip), Python 168 passed/1 skipped plus 6/6 opt-in TLS. Docs: `docs/GatewayTesting.md` § Client Wire Tests, `clients/dotnet/README.md`, `clients/python/README.md`. |
| 2026-08-10 | **TST-05 revisited under the restored Windows tier → `Partially done`** (branch `feat/tst-24-client-wire-tests`, doc/tracker-only). The finding's **scheduling** half is closed: cycle-2 TST-25's `nightly-windev` job (cron `0 6 * * *`) runs `scripts/ci/run-windev-ci.sh live` → `windev-worker-ci.ps1 -Mode live`, which sets `MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1`, runs `WorkerLiveMxAccessSmokeTests` on windev after the x86 build/Worker.Tests/full-slnx steps, and files a Gitea issue when red. The **coverage-audit** half is *not* closed, and the audit the design asked for now has a negative answer: the suite's eight `[LiveMxAccessFact]`s cover all six late-added COM commands (`Suspend`, `Activate`, `AuthenticateUser`, `ArchestrAUserToId`, `AddBufferedItem`, `SetBufferedUpdateInterval`) but zero of the five control commands — `MxCommandKind.{Ping,GetSessionState,GetWorkerInfo,DrainEvents,ShutdownWorker}` appear nowhere in `WorkerLiveMxAccessSmokeTests.cs`, so the exact paths the Finding calls masked are still only proven against `FakeWorkerHarness` canned replies while the real implementations live in `Worker/Ipc/WorkerPipeSession.cs`. Residual work (two `[LiveMxAccessFact]`s, windev-only to author and verify) is specified in [60-testing-docs-gaps.md](60-testing-docs-gaps.md#tst-05--real-worker-controlcom-paths-verified-opt-in-only---medium--p1). |
| 2026-07-10 | **TST-15 design fleshed out** (still `Not started` — design only, not implementation): `docs/plans/2026-07-10-dashboard-session-acl-tst15.md`. Resolves the crux the deferral left open — the dashboard is LDAP-identity (Admin/Viewer) while sessions are API-key-owned (`OwnerKeyId`), two disjoint identity domains — via a **session tag** sourced from the owning API key (rides in the existing `ApiKeyConstraints` JSON blob, no SQLite migration). Admin-sees-all; Viewer may `SubscribeSession` iff `session.Tags ∩ viewer.GrantedTags ≠ ∅` (new `Dashboard:GroupToTag` map → hub-token tag claims); untagged sessions Admin-only by default (`Dashboard:UntaggedSessionVisibility`). Includes the enforcement path (`HubTokenPayload.Tags` + `IDashboardSessionAcl` gate at `SubscribeSession`), task breakdown (epic Tasks 16–19), test plan incl. live-LDAP, and rejected alternatives (client-supplied tag; group→key-id map). Tracker + `60-testing-docs-gaps.md` TST-15 section point at the doc. **TST-03 investigated:** the CI never ran because the repo had **zero registered Gitea Actions runners** (Actions is enabled; runs are created on push/PR/nightly but fail instantly with nothing to execute them). A Mac runner proved the pipeline executes but cannot clone — this Gitea hands runners the internal `http://gitea:3000` URL, reachable only by a runner co-located on the gitea Docker network. Fix = run a co-located runner on the Gitea host (recipe prepared, `scratchpad/gitea-runner/setup-gitea-host-runner.sh`); pending host access. TST-03 stays `In review`. |
| 2026-07-09 | **P2 Epic wrap — user decision: DEFER TST-15 + TST-24, close the epic.** Epic bucket result: 5 of 7 findings `Done` (CLI-15, CLI-04, CLI-30, TST-01, TST-04); **TST-15** and **TST-24** intentionally deferred to a follow-up (kept `Not started`, not `Won't fix` — they are gated, not rejected). **TST-15** (dashboard EventsHub per-session ACL) is epic Phase 4 — a real feature needing a new session-"tag" mechanism + dashboard group→tag config, not a mechanical fix; the `EventsHub` `TODO(per-session-acl)` stays, and the already-shipped **SEC-25** mitigation (tag *values* redacted from the dashboard mirror by default) means no sensitive payload leaks through the hub today regardless of the missing ACL — so deferring carries no value-leak risk. **TST-24** (per-client wire tests) depends on **TST-03** (CI), which is `In review` (YAML authored, never run on a Gitea runner) — no point wiring client tests into a pipeline that isn't live yet. Net P2: 35/38 `Done`; remaining = TST-15 (deferred feature), TST-24 (deferred, CI-gated), TST-14 (user deletes their own untracked gitignored `*-docs-*.md` files). |
| 2026-07-09 | P2 Epic — **Java client completes CLI-15 + CLI-04 locally** (commit `1cc0fa4`); **CLI-15, CLI-04, CLI-30, TST-01 all → `Done` (5/5 clients + server e2e)**. Java CLI-15: `MxEventStreamItem` record + `MxEventStream.nextItem()` (`isReplayGap()`/`replayGap()`/`event()`); existing `Iterator<MxEvent>` path unchanged, sentinel never swallowed. Java CLI-04: Phase 1 `adviseSupervisory`/`writeSecured`/`writeSecured2`/`authenticateUser`/`archestrAUserToId` + Phase 2 `addBufferedItem`/`setBufferedUpdateInterval`/`suspend`/`activate` (unregister already present) on `MxGatewaySession`, each through `invokeCommand` → `ensureProtocolSuccess`+`ensureMxAccessSuccess`; credentials scrubbed via `MxGatewaySecrets.redactCredentials` (tests assert absent from message/toString/CLI). `gradle test` 106/0 (58 client + 48 cli), no generated churn. Built locally with `JAVA_HOME=/opt/homebrew/opt/openjdk@17` — Java toolchain now works on the Mac (see prior note). Shared docs `ClientLibrariesDesign.md` + CLAUDE.md updated to "all five clients". **TST-01 → Done** (server e2e `fed0685` + all 5 client `ReplayGap` consumers). This closes session-resilience epic Phase 3 fully. |
@@ -330,6 +330,8 @@ Open question the review cannot settle: whether a future probe (COM-init readine
**Verification.** `dotnet build src/ZB.MOM.WW.MxGateway.Server`; `dotnet test src/ZB.MOM.WW.MxGateway.Tests --filter "FullyQualifiedName~SessionManager"`.
**Resolution: Won't fix (2026-08-20, closed by cycle-3 verification).** `Grpc/MxAccessGatewayService.cs:111-115` carries a `PERF(followup)` comment documenting the double session lookup as measured-negligible and deliberately not collapsed — the fix would duplicate `InvokeAsync`'s fault mapping at a second entry point, trading a negligible per-command cost for a maintenance liability. Per the cycle-3 gateway-core review (`archreview/2026-08-19/10-gateway-core.md`, GWC-16 row), the rationale is sound; tracker flipped from `Not started` to `Won't fix`.
---
## GWC-17 — Sessions layer throws `Grpc.Core.RpcException` `Low` · `—`
@@ -175,6 +175,8 @@ The frame protocol and proto evolution hygiene are sound; the remediable risk cl
**Verification.** `dotnet build src/ZB.MOM.WW.MxGateway.Server` + `dotnet test ...Tests --filter FullyQualifiedName~WorkerClient`; worker side `-p:Platform=x86`.
**Resolution: Done (2026-08-20, closed by cycle-3 verification, residual filed).** Wired up substantially as designed: `InvokeAsync` forwards a `WorkerCancel` when the gateway's own command timeout fires and it owned the removal (`Server/Workers/WorkerClient.cs:269-291`), `TrySendCancelForTimedOutCommand` is structurally no-throw (`:835-870`), the worker routes it to `CancelCommand` (`Worker/Ipc/WorkerPipeSession.cs:755-756` → `MxAccess/MxAccessStaSession.cs:529-532` → `Sta/StaCommandDispatcher.CancelQueuedCommand:151`), regression test `WorkerClientTests.InvokeAsync_WhenCommandTimesOut_SendsWorkerCancelForThatCorrelation` exists, and `docs/GatewayProcessDesign.md:596-602` documents it. Landed in `7c1ea12`/`a1a38b5`. **Residual, not covered by this closure:** the caller-cancellation leg (context deadline / client disconnect, as opposed to the gateway's own `CommandTimeout`) sends nothing — filed as cycle-3 **IPC-34**. Per `archreview/2026-08-19/30-contracts-ipc.md` (IPC-08 row), tracker flipped from `Not started` to `Done` with the residual cross-referenced.
---
## IPC-09 — Known codegen fragilities have no in-repo guards `Medium` · `P1`
@@ -132,7 +132,7 @@ This document turns every finding in the Security/Dashboard/Observability review
## SEC-06 — LDAP plaintext-by-default with a committed service password `Medium` · `P1`
**Finding.** *(review SEC-6)* `Configuration/LdapOptions.cs:49-61` defaults `Transport=None`, `AllowInsecure=true`, `ServiceAccountPassword="serviceaccount123"`; `appsettings.json:21-33` ships the same. `glauth.md:30,327` confirms dev LDAPS is disabled and binds send cleartext. The validator enforces the `Transport=None ⇒ AllowInsecure` consistency rule (`GatewayOptionsValidator.cs:82-85`) but nothing distinguishes dev from prod.
**Finding.** *(review SEC-6)* `Configuration/LdapOptions.cs:49-61` defaults `Transport=None`, `AllowInsecure=true`, `ServiceAccountPassword="<service-account-password>"` (value redacted per SEC-36); `appsettings.json:21-33` ships the same. `glauth.md:30,327` confirms dev LDAPS is disabled and binds send cleartext. The validator enforces the `Transport=None ⇒ AllowInsecure` consistency rule (`GatewayOptionsValidator.cs:82-85`) but nothing distinguishes dev from prod.
**Impact.** Every dashboard login sends the operator's password cleartext to `10.100.0.35:3893`, and a service-account credential is in source control.
@@ -140,7 +140,7 @@ This document turns every finding in the Security/Dashboard/Observability review
**Implementation.**
- `Configuration/GatewayOptionsValidator.cs`: in `ValidateLdap`, when Production and `Transport == None`, emit an error (co-locate with SEC-04's env plumbing).
- Deployment: keep `serviceaccount123` only for local GLAuth dev; document env-var override (`MxGateway__Ldap__ServiceAccountPassword`) for the NSSM-wrapped hosts; rotate the dev credential's reuse.
- Deployment: keep the dev service-account password only for local GLAuth dev; document env-var override (`MxGateway__Ldap__ServiceAccountPassword`) for the NSSM-wrapped hosts; rotate the dev credential's reuse.
- Docs: `docs/GatewayConfiguration.md` Ldap section and a production hardening note referencing `glauth.md`.
- Tests: `GatewayOptionsValidatorTests` — `Transport=None` + Production → invalid.
@@ -457,6 +457,8 @@ This document turns every finding in the Security/Dashboard/Observability review
- Tests: broadcaster test asserting values redacted when `ShowTagValues=false`.
- Docs: `docs/GatewayDashboardDesign.md` — clarify the current v1 posture.
**Update 2026-08-17 — the deferred half landed.** The scoping mechanism this finding waited on shipped as TST-15 (`693a78d` + `7ec0b35`): `IDashboardSessionAcl` gates `SubscribeSession` *and* the session-details page's in-process subscribe, so the `TODO(per-session-acl)` is gone and the redaction is no longer the only thing between a low-trust Viewer and another session's events. See the TST-15 section in [60-testing-docs-gaps.md](60-testing-docs-gaps.md#tst-15--dashboard-eventshub-has-no-per-session-acl) and the 2026-08-17 change-log row in [00-tracking.md](00-tracking.md#change-log). Redaction stays — the two layers are independent: the ACL decides who may subscribe, `ShowTagValues` decides what a permitted subscriber sees.
**Verification.** `dotnet build src/ZB.MOM.WW.MxGateway.Server` and the events-hub/broadcaster test filter.
---
+35 -3
View File
@@ -159,6 +159,14 @@ This finding is the umbrella; TST-01/02/15 are its actionable slices. The remedi
## TST-05 — Real-worker control/COM paths verified opt-in only `Medium` · `P1`
> **Status revisit 2026-08-10 (unlocked by TST-25): `Partially done` — one half closed, one half open.**
>
> **Closed — the scheduled cadence.** The `nightly-windev` job in `.gitea/workflows/ci.yml` (cron `0 6 * * *`, gated `if: github.event_name == 'schedule'`) runs `scripts/ci/run-windev-ci.sh live`, which drives `scripts/ci/windev-worker-ci.ps1 -Mode live` on windev: x86 Worker build → full `Worker.Tests` → full-slnx build → `MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1 dotnet test … --filter FullyQualifiedName~WorkerLiveMxAccessSmokeTests`. A red nightly opens a Gitea issue, so nobody has to watch the Actions page. That is exactly this finding's **Design** paragraph, and it is the `live-mxaccess` job the design referred to (renamed; the removed job it originally pointed at is gone — see cycle-2 TST-25/TST-26).
>
> **Open — the coverage audit.** The design also required auditing `WorkerLiveMxAccessSmokeTests.cs` for coverage of *each* of the eleven late-added command kinds and adding missing `[LiveMxAccessFact]` cases. That audit now has an answer, and it is negative for five of the eleven. The suite's eight facts reach all six late-added **COM** commands (`Suspend`, `Activate`, `AuthenticateUser`, `ArchestrAUserToId`, `AddBufferedItem`, `SetBufferedUpdateInterval` — the `NewComCommands_RoundTripWithRealReplies` and `BufferedItem_*` facts). None of them sends any of the five **control** commands: `MxCommandKind.{Ping,GetSessionState,GetWorkerInfo,DrainEvents,ShutdownWorker}` do not appear anywhere in the file. Those are the very kinds the Finding below names as masked. The real worker answers them off-STA in `src/ZB.MOM.WW.MxGateway.Worker/Ipc/WorkerPipeSession.cs` (dispatch switch at `:574`+), so the nightly exercises that code path only incidentally, never by assertion — a regression in `CreatePingReply`/`CreateSessionStateReply`/`CreateWorkerInfoReply`/the drain snapshot/the shutdown-after-reply ordering still ships green through both CI and the nightly.
>
> **Residual work to close TST-05 fully** (small, Windows-only): add one `[LiveMxAccessFact]` to `WorkerLiveMxAccessSmokeTests` that, against a live worker, invokes `Ping` → `GetSessionState` → `GetWorkerInfo` → `DrainEvents` and asserts each returns a non-`INVALID_REQUEST` reply carrying real worker state (e.g. `worker_process_id` matching the launched process), plus a separate fact for `ShutdownWorker` asserting the OK reply arrives *before* the worker exits and the session is then faulted/closed. `ShutdownWorker` needs `admin` scope and terminates the worker, so it must be the last fact in its own fixture. Not done here because it can only be authored and verified on windev with MXAccess installed; this revisit is doc/tracker-only.
**Finding.** All eleven late-added command kinds are unit-tested against fakes and live-verified once on the dev rig (`stillpending.md` §1.1), but the default suite exercises `Ping`/`GetWorkerInfo`/`DrainEvents`/`ShutdownWorker` only through `FakeWorkerHarness.RespondToControlCommandAsync` (verify current line range in `src/ZB.MOM.WW.MxGateway.Tests/Gateway/Workers/Fakes/FakeWorkerHarness.cs`), which returns canned replies.
**Impact.** A worker-side regression in these paths is invisible until someone sets `MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1`.
@@ -183,6 +191,8 @@ This finding is the umbrella; TST-01/02/15 are its actionable slices. The remedi
**Verification.** `dotnet test ... --filter FullyQualifiedName~DashboardLiveDataServiceTests`.
**Status refresh (2026-08-20, per cycle-3 verification).** Materially narrowed — the tracker row was stale. `Tests/Gateway/Dashboard/DashboardLiveDataServiceTests.cs`, `EventsHubTests.cs`, `EventsHubViewerRegistryTests.cs`, `AlarmsHubPublisherTests.cs`, and `SessionDetailsPageEventAclTests.cs` all exist now (mostly landed with the SEC-25/TST-16 closeouts and the in-process dashboard-feed work). Per `archreview/2026-08-19/60-testing-docs-gaps.md` (TST-06 row), this needs a re-audit-and-close pass against the current coverage rather than fresh implementation — left open pending that audit, tracker status changed from `Not started` to `Materially narrowed` to reflect the actual state.
---
## TST-07 — Real-clock sleeps with negative assertions are latent flakes `Medium` · `—`
@@ -327,11 +337,11 @@ If TST-02's interim mitigation (flip retention off) is chosen instead of impleme
**Impact.** Acceptable for a single-tenant dashboard; wrong the moment `GroupToRole` admits low-trust viewers. It is the dashboard-side twin of the gRPC owner-revalidation gap (TST-02).
**Design.** Fully fleshed out in `docs/plans/2026-07-10-dashboard-session-acl-tst15.md` (epic Phase 4, Tasks 16–19, TST-04). In brief: the dashboard authenticates LDAP users (Admin/Viewer) while sessions are API-key-owned (`OwnerKeyId`) — two disjoint identity domains — so the ACL needs a bridge: a **session tag** sourced from the owning API key (riding in the existing `ApiKeyConstraints` JSON blob, no SQLite migration). Admin sees all; a Viewer may `SubscribeSession` iff `session.Tags ∩ viewer.GrantedTags ≠ ∅` (granted via a new `Dashboard:GroupToTag` map, carried into the hub token as tag claims); untagged sessions are Admin-only by default. The Viewer-default decision (admin-sees-all vs strict) is settled there. Until Phase 4 lands, keep the TODO (it correctly documents the accepted single-tenant assumption); do not silently remove it.
**Design.** Fully fleshed out in `docs/plans/2026-07-10-dashboard-session-acl-tst15.md` (epic Phase 4, Tasks 16–19, TST-04). In brief: the dashboard authenticates LDAP users (Admin/Viewer) while sessions are API-key-owned (`OwnerKeyId`) — two disjoint identity domains — so the ACL needs a bridge: a **session tag** sourced from the owning API key (riding in the existing `ApiKeyConstraints` JSON blob, no SQLite migration). Admin sees all; a Viewer may `SubscribeSession` iff `session.Tags ∩ viewer.GrantedTags ≠ ∅` (granted via a new `Dashboard:GroupToTag` map, carried into the hub token as tag claims); untagged sessions are Admin-only by default. The Viewer-default decision (admin-sees-all vs strict) is settled there.
**Implementation.** `Dashboard/Hubs/EventsHub.cs` (ACL check on group join), hub-token minting to carry the session tag, `Configuration/DashboardOptions.cs` for any group-to-tag config (Task 17). Tests: `...Tests/Gateway/Dashboard/` hub ACL cases incl. live-LDAP users (Task 19). Docs: `docs/Sessions.md`/`gateway.md` dashboard section document the ACL model; CLAUDE.md dashboard-auth paragraph.
**Implementation.** Shipped 2026-08-17 on `feat/deferred-closeout` (`693a78d` + `7ec0b35`); the `TODO(per-session-acl)` is gone. `Dashboard/IDashboardSessionAcl.cs` + `Dashboard/DashboardSessionAcl.cs` hold the single decision, consulted by `Dashboard/Hubs/EventsHub.cs` (`SubscribeSession` → `HubException` on denial, no group join and no viewer registration) and by `Dashboard/Components/Pages/SessionDetailsPage.razor`'s in-process subscribe — the design's one correction, since the page path was not a hub client and would otherwise have been the unguarded seam. Tags ride from the owning key via `ISessionManager.OpenSessionAsync`'s tagged overload into the immutable `GatewaySession.Tags`; grants are stamped by `DashboardAuthenticator.CreatePrincipal` and re-resolved at `HubTokenService.Issue`. Config: `Dashboard:GroupToTag` and `Dashboard:UntaggedSessionVisibility` on `Configuration/DashboardOptions.cs`. Tests: `Tests/Gateway/Dashboard/DashboardSessionAclTests.cs`, `EventsHubTests.cs`, a `Configuration/GatewayOptionsTests.cs` binding case for the `GroupToTag` comparer, and two `[LiveLdapFact]`s in `IntegrationTests/DashboardLdapLiveTests.cs`. Docs: `docs/Sessions.md`, `gateway.md`, CLAUDE.md, `docs/GatewayDashboardDesign.md`, `docs/GatewayConfiguration.md`, `glauth.md`.
**Verification.** `dotnet test ... --filter FullyQualifiedName~EventsHub`; `dotnet build src/ZB.MOM.WW.MxGateway.Server`.
**Verification.** `dotnet build src/ZB.MOM.WW.MxGateway.NonWindows.slnx`; `dotnet test ... --filter FullyQualifiedName~DashboardSessionAclTests`, `~EventsHubTests`, `~GatewayOptionsTests`; live-LDAP pair run green against the shared GLAuth with `MXGATEWAY_RUN_LIVE_LDAP_TESTS=1` (and skipping cleanly without it).
---
@@ -347,6 +357,8 @@ If TST-02's interim mitigation (flip retention off) is chosen instead of impleme
**Verification.** `dotnet build src/ZB.MOM.WW.MxGateway.Server`; test toggling the flag suppresses/shows values; docs match.
**Closed 2026-08-18 — implemented (the Design's recommended path).** The flag is no longer dead: it now gates value display on all three seams that carry a tag value. `Dashboard/DashboardLiveDataService.cs` substitutes `[redacted]` at the service boundary for each successfully read `/browse` tag (a failed read keeps its `-` placeholder, so the error row and the redaction cannot contradict each other); `Dashboard/Hubs/AlarmsHubPublisher.cs` clears `current_value`/`limit_value` from a **deep-cloned** `AlarmFeedMessage` before broadcasting to `/hubs/alarms`, and `DashboardEventBroadcaster` does the same for the events-hub mirror — clones, because the sources fan out to gRPC `StreamAlarms`/`StreamEvents` and the replay ring, so no gRPC client is affected by this dashboard-display flag. The flag is **kept**, default still `false`. The stale prose the Finding flagged is gone: `docs/GatewayConfiguration.md` and `docs/GatewayDashboardDesign.md` now describe the real behaviour. Status tracked in `00-tracking.md` (already **Done**).
---
## TST-17 — Vendor-gated alarm parity residuals silently lossy `Medium` · `—`
@@ -361,6 +373,8 @@ If TST-02's interim mitigation (flip retention off) is chosen instead of impleme
**Verification.** `dotnet build src/ZB.MOM.WW.MxGateway.Worker -p:Platform=x86`; `dotnet test src/ZB.MOM.WW.MxGateway.Worker.Tests/... -p:Platform=x86` (Windows).
**Status refresh (2026-08-20, per cycle-3 verification).** Materially changed — the tracker row needs a refresh from the worker domain. `WnWrapAlarmConsumer.cs:309-327` now passes `szDomainName`/`szOprFullName` into the COM call (the `_ = ackOperatorDomain` discard is gone), and the 2026-08 probe cycle recorded the vendor 8-arg leg as "unavailable by configuration" with evidence (`docs/AlarmProbeFindings.md`, commits `d1ae43d`, `540a118`). Per `archreview/2026-08-19/60-testing-docs-gaps.md` (TST-17 row), "silent drop, unchanged" is no longer an accurate description of the row; whether a caller-visible degrade signal now exists is the worker reviewer's call to close out. Tracker status changed from `Not started` to `Materially changed`.
---
## TST-18 — Hosted-service wrappers untested `Low` · `—`
@@ -449,6 +463,24 @@ If TST-02's interim mitigation (flip retention off) is chosen instead of impleme
## TST-24 — Client wire behaviour has no automated verification `Low` · `—`
> **Resolution 2026-08-10 (branch `feat/tst-24-client-wire-tests`): `Done`.** All five clients now drive their public API against a fake gateway served over a real gRPC transport, in the client's own default suite, and all five run in CI.
>
> **Corrected premise.** The Finding's "no in-process gateway integration tests" was already stale when it was re-verified: **Go, Rust, and Java had real-server wire tests**, not mocks — Go's `newBufconnClient`/`fakeGatewayServer` (`clients/go/mxgateway/client_session_test.go`) over `grpc/test/bufconn`, Rust's `spawn_fake_gateway` (`clients/rust/tests/client_behavior.rs`) over a loopback `TcpListener` with tonic's `Server`, and Java's `InProcessGateway`/`TestGatewayService` (`MxGatewayClientSessionTests.java`) over `InProcessServerBuilder`. Each already asserted the round trip, the `authorization` bearer header as *observed by the server*, and the `ReplayGap` sentinel. The cycle-2 re-verification cited `clients/python/tests/test_replay_gap.py` as evidence for "the other four clients still unit-test against mocks"; that generalized from Python to Go/Rust/Java incorrectly. `InProcessGatewayHarness` (the "template" the Impact paragraph names) is in fact the *thinner* of the Java harnesses — it serves only `streamEvents`/`closeSession` for the CLI tests.
>
> **Real gap, and what was built.** Two clients genuinely had none. **.NET** substituted `FakeGatewayTransport` for `IMxGatewayClientTransport` in every test, so not even the generated stub ran, and its test project had no server package at all. **Python** monkeypatched `MxAccessGatewayStub` everywhere except one opt-in TLS test that served only `OpenSession`. Both now have the pattern:
> - `clients/dotnet/ZB.MOM.WW.MxGateway.Client.Tests/WireFakeGatewayServer.cs` + `MxGatewayClientWireTests.cs` — Kestrel h2c on `127.0.0.1:0` serving `MxAccessGateway.MxAccessGatewayBase`; needed new `Grpc.AspNetCore.Server` + `Microsoft.AspNetCore.App` references on the test project.
> - `clients/python/tests/test_wire_fake_gateway.py` — a `grpc.aio` server on `127.0.0.1:0` serving `MxAccessGatewayServicer`; no new dependencies (`grpcio` is a runtime dep).
>
> Each covers the four shapes the Design asked for: round trip (`OpenSession` → `Invoke`/`Register` → `StreamEvents` → `CloseSession` with every reply field asserted), the bearer header as received by the server on the streaming RPC as well as the unary ones, the `ReplayGap` sentinel surfaced as the client's typed signal (TST-01), and a real `PERMISSION_DENIED` mapping to the typed authorization error.
>
> **CI.** The `portable` job previously only *built* the .NET client; a `dotnet test` step was added, so its wire tests actually run. Go/Rust/Python already ran their suites there and Java in the `java` job.
>
> **Bug this immediately caught** — the justification for the whole finding. `GatewayClient.connect()` / `GalaxyRepositoryClient.connect()` in the Python client were **broken for every real (non-stub) connection**: they built the `grpc.aio` channel inside `asyncio.to_thread`, and a `grpc.aio` channel binds to the event loop current on the constructing thread, so the worker thread raised `RuntimeError: There is no current event loop in thread 'asyncio_0'`. No mock-based test could see it — the one test asserting the off-loop behaviour (`Client.Python-028`) monkeypatched `create_channel` and therefore asserted the bug. Fixed by splitting `resolve_channel_security` (blocking TOFU probe, runs off-loop) from `create_channel` (must run on the loop thread), keeping the Client.Python-028 guarantee; the two `-028` tests were retargeted to assert both halves.
>
> **Deliberately out of scope.** Only the four session RPCs are served — the alarm feed (`StreamAlarms`, `QueryActiveAlarms`, `AcknowledgeAlarm`) and Galaxy browse are not, matching the Design's "full parity is out of scope". Java's `InProcessGatewayHarness` still lacks `openSession`/`invoke`; the client-module tests cover those shapes, so it was left alone.
>
> **Docs.** `docs/GatewayTesting.md` § Client Wire Tests (the cross-client pattern + per-client harness table), `clients/dotnet/README.md`, `clients/python/README.md`.
**Finding.** All five clients have unit tests (13/8/3/13/7 files for dotnet/go/rust/python/java) but no in-process or containerized gateway integration tests; the only cross-language verification is the operator-run `scripts/run-client-e2e-tests.ps1`. `CrossLanguageSmokeMatrixTests` checks shapes only.
**Impact.** Low-to-moderate: a gateway contract change can pass every default suite and break all five clients (partly mitigated by shared-proto codegen). The Java CLI already proves the cheap pattern — `InProcessGatewayHarness` (`stillpending.md` §8).
+70 -2
View File
@@ -23,6 +23,20 @@ dotnet build clients/dotnet/ZB.MOM.WW.MxGateway.Client.slnx
dotnet test clients/dotnet/ZB.MOM.WW.MxGateway.Client.slnx --no-build
```
Most tests substitute `FakeGatewayTransport` for `IMxGatewayClientTransport`, so
they never touch the wire. `MxGatewayClientWireTests` is the exception: it drives
the ordinary public API against `WireFakeGatewayServer`, a real gRPC server
(Kestrel h2c on an ephemeral loopback port) serving
`MxAccessGateway.MxAccessGatewayBase`. Only the gateway's behaviour is canned —
the HTTP/2 framing, protobuf serialization, `authorization` metadata, and gRPC
status codes are genuine, so it catches decode and metadata breaks a transport
fake cannot see. No MXAccess or worker is involved; it runs in the default suite.
See `docs/GatewayTesting.md` (Client Wire Tests) for the cross-client pattern.
```powershell
dotnet test clients/dotnet/ZB.MOM.WW.MxGateway.Client.Tests/ZB.MOM.WW.MxGateway.Client.Tests.csproj --filter FullyQualifiedName~MxGatewayClientWireTests
```
## Packaging
Create local library and CLI artifacts from the repository root:
@@ -135,6 +149,28 @@ token and pass through the `MxGateway:Alarms` configuration on the
server — when alarms are disabled, the gateway returns an empty list / empty
stream rather than failing.
`ActiveAlarmSnapshot.FromTruncatedSnapshot` marks a record that came from a
provider fetch which hit the per-fetch cap: the snapshot set may omit active
alarms, and the gateway suspended its absence-implies-cleared inference for that
poll. Treat the set as possibly incomplete rather than reconciling deletions
from it. It is set-level degraded status, not a comment on the record's own
fidelity, and is distinct from `Degraded` (the subtag fallback provider).
`StreamAlarmsAsync` also carries that completeness verdict at feed level, as an
`AlarmFeedMessage.PayloadOneofCase.SnapshotStatus` frame whose
`SnapshotStatus.Truncated` is true while the monitor's cached set derives from a
truncated fetch. One arrives at stream open (after the `ProviderStatus` frame,
before the cached `ActiveAlarm` frames) so a late joiner learns the current
verdict, then one on every verdict change — including the clearing frame sent
when the gateway's alarm monitor restarts and drops a truncated verdict. Track
it if you need set completeness on a live feed without polling
`QueryActiveAlarmsAsync`.
The CLI's text rows always spell out `degraded=false` and `truncated=false` so
healthy and clearing frames remain visible. Its protobuf-JSON modes retain the
standard proto3 omission rule, so those same false-valued payloads appear as
`{"providerStatus": {}}` and `{"snapshotStatus": {}}`; absent means false.
`MxGatewaySession.CloseAsync` is explicit and idempotent. Repeated calls return
the first `CloseSessionReply` instead of sending another close request.
@@ -163,6 +199,12 @@ can keep the full `MxCommandReply`, HRESULT, and status array when MXAccess
itself rejects a command. `MxAccessException.Reply` contains the raw generated
reply.
`EnsureMxAccessSuccess()` follows COM semantics: only a **negative** HRESULT is
a failure, so positive success codes such as `S_FALSE` (1) pass. A status entry
fails only when `Category` is not `MxStatusCategory.Ok` — `MxStatusProxy.Success`
mirrors the raw COM member for diagnostics and never decides the verdict, which
is why `IsSuccess()` branches on the category alone.
## Write Semantics And Common Pitfalls
These are MXAccess parity behaviors that surprise new callers. The gateway
@@ -248,7 +290,7 @@ dotnet run --project clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli -- advise --s
dotnet run --project clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli -- write --session-id <id> --server-handle 1 --item-handle 1 --type int32 --value 123 --json
dotnet run --project clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli -- write2 --session-id <id> --server-handle 1 --item-handle 1 --type int32 --value 123 --timestamp 2026-01-01T00:00:00Z --json
dotnet run --project clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli -- stream-events --session-id <id> --max-events 1 --json
dotnet run --project clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli -- stream-alarms --filter-prefix Area001 --max-events 1 --json
dotnet run --project clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli -- stream-alarms --filter-prefix Area001 --max-events 4 --json
dotnet run --project clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli -- acknowledge-alarm --reference "\\Galaxy\Area001.Pump001.PumpFault" --comment "ack from cli" --operator operator1 --json
dotnet run --project clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli -- smoke --endpoint http://localhost:5000 --api-key-env MXGATEWAY_API_KEY --item Area001.Pump001.Speed --json
```
@@ -258,6 +300,32 @@ optionally writes a value when `--type` and `--value` are supplied, reads a
bounded event stream, and closes the session in a `finally` block. CLI error
output redacts API keys supplied through `--api-key`.
### `authenticate-user` credentials
```powershell
$env:MXGATEWAY_VERIFY_PASSWORD = "<verify-user password>"
dotnet run --project clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli -- authenticate-user --session-id <id> --server-handle 1 --verify-user operator --json
```
The credential comes from `--password` or, preferably, the environment variable
named by `--password-env` (default `MXGATEWAY_VERIFY_PASSWORD`) so it stays out
of shell history and the process table. It is never echoed to stdout or stderr,
and error output routes it through the same redaction seam as the API key. A
missing or empty resolved credential is a usage error naming the option and the
variable: the CLI fails before the invoke rather than authenticating with an
empty password.
`MXGATEWAY_VERIFY_PASSWORD` is the canonical variable across all five client CLIs
— see [Cross-Language Smoke Matrix](../../docs/CrossLanguageSmokeMatrix.md).
**Deprecated names.** This CLI previously used `--verify-user-password`,
`--verify-user-password-env`, and `MXGATEWAY_VERIFY_USER_PASSWORD`. All three
still resolve, for one release only, so existing scripts keep working; migrate to
the canonical names above. The full resolution order is `--password`,
`--verify-user-password`, the variable named by `--password-env` (or the
deprecated `--verify-user-password-env`, default `MXGATEWAY_VERIFY_PASSWORD`),
then `MXGATEWAY_VERIFY_USER_PASSWORD`.
## Galaxy Repository Browse
`GalaxyRepositoryClient` is a separate read-only wrapper around the
@@ -455,7 +523,7 @@ dotnet nuget add source https://gitea.dohertylan.com/api/packages/dohertj2/nuget
Then add the package to your project:
````bash
dotnet add package ZB.MOM.WW.MxGateway.Client --version 0.1.1
dotnet add package ZB.MOM.WW.MxGateway.Client --version 0.2.0
````
The `ZB.MOM.WW.MxGateway.Contracts` package is pulled in transitively.
@@ -346,31 +346,78 @@ public static class MxGatewayClientCli
}
/// <summary>
/// Resolves the effective MXAccess verify-user credential from
/// <c>--verify-user-password</c> or, failing that, the
/// <c>--verify-user-password-env</c>-named environment variable (default
/// <c>MXGATEWAY_VERIFY_USER_PASSWORD</c>). The credential is never echoed;
/// this resolver exists so the error-redaction catch block can strip it
/// from any surfaced error (CLI-04), mirroring <see cref="TryResolveApiKey"/>.
/// Canonical CLI credential environment variable, shared by every official
/// client CLI (CLI-45) so one exported variable drives the same operator
/// workflow in all five languages.
/// </summary>
private const string DefaultVerifyPasswordEnvironmentName = "MXGATEWAY_VERIFY_PASSWORD";
/// <summary>
/// Pre-CLI-45 environment variable, still honoured as a deprecated fallback
/// for one release so existing scripts keep working.
/// </summary>
private const string LegacyVerifyPasswordEnvironmentName = "MXGATEWAY_VERIFY_USER_PASSWORD";
/// <summary>
/// Resolves the name of the environment variable holding the verify-user
/// credential: <c>--password-env</c>, then the deprecated
/// <c>--verify-user-password-env</c> alias, then
/// <c>MXGATEWAY_VERIFY_PASSWORD</c>.
/// </summary>
private static string ResolveVerifyPasswordEnvironmentName(CliArguments arguments)
{
string? environmentName = arguments.GetOptional("password-env");
if (!string.IsNullOrEmpty(environmentName))
{
return environmentName;
}
environmentName = arguments.GetOptional("verify-user-password-env");
return string.IsNullOrEmpty(environmentName)
? DefaultVerifyPasswordEnvironmentName
: environmentName;
}
/// <summary>
/// Resolves the effective MXAccess verify-user credential in the CLI-45
/// order: <c>--password</c>, the deprecated <c>--verify-user-password</c>
/// alias, the environment variable named by <c>--password-env</c> (default
/// <c>MXGATEWAY_VERIFY_PASSWORD</c>), then the deprecated
/// <c>MXGATEWAY_VERIFY_USER_PASSWORD</c>. An empty value from any source is
/// treated as absent. The credential is never echoed; this resolver exists so
/// the error-redaction catch block can strip it from any surfaced error
/// (CLI-04), mirroring <see cref="TryResolveApiKey" />.
/// </summary>
private static string? TryResolveVerifyUserPassword(CliArguments arguments)
{
string? password = arguments.GetOptional("verify-user-password");
string? password = arguments.GetOptional("password");
if (!string.IsNullOrEmpty(password))
{
return password;
}
string passwordEnvironmentName = arguments.GetOptional("verify-user-password-env")
?? "MXGATEWAY_VERIFY_USER_PASSWORD";
password = arguments.GetOptional("verify-user-password");
if (!string.IsNullOrEmpty(password))
{
return password;
}
return Environment.GetEnvironmentVariable(passwordEnvironmentName);
password = Environment.GetEnvironmentVariable(ResolveVerifyPasswordEnvironmentName(arguments));
if (!string.IsNullOrEmpty(password))
{
return password;
}
password = Environment.GetEnvironmentVariable(LegacyVerifyPasswordEnvironmentName);
return string.IsNullOrEmpty(password) ? null : password;
}
/// <summary>
/// Resolves the verify-user credential for <c>authenticate-user</c>, throwing
/// a redaction-safe error when neither the flag nor the env var is set. The
/// thrown message names only the option/env var, never the value.
/// a redaction-safe error when no source yields a non-empty value. Failing
/// fast keeps a misconfigured environment from becoming a real MXAccess
/// authentication attempt with an empty credential (CLI-45); the thrown
/// message names only the option/env var, never the value.
/// </summary>
private static string ResolveVerifyUserPassword(CliArguments arguments)
{
@@ -380,11 +427,10 @@ public static class MxGatewayClientCli
return password;
}
string passwordEnvironmentName = arguments.GetOptional("verify-user-password-env")
?? "MXGATEWAY_VERIFY_USER_PASSWORD";
throw new ArgumentException(
$"Verify-user password is required. Pass --verify-user-password or set {passwordEnvironmentName}.");
"Verify-user password is required. Pass --password or set "
+ $"{ResolveVerifyPasswordEnvironmentName(arguments)} (deprecated aliases: "
+ $"--verify-user-password, --verify-user-password-env, {LegacyVerifyPasswordEnvironmentName}).");
}
private static CancellationTokenSource CreateCancellation(CliArguments arguments, string command)
@@ -710,8 +756,10 @@ public static class MxGatewayClientCli
TextWriter output,
CancellationToken cancellationToken)
{
// The credential is resolved from --verify-user-password or its env var and
// is never echoed. On any surfaced error the RunCoreAsync catch block routes
// The credential is resolved from --password or its env var (default
// MXGATEWAY_VERIFY_PASSWORD) and is never echoed; a missing or empty value
// fails fast before the invoke rather than reaching the wire (CLI-45).
// On any surfaced error the RunCoreAsync catch block routes
// it through MxGatewayCliSecretRedactor so it cannot reach stderr (CLI-04).
return InvokeAndWriteAsync(
arguments,
@@ -1370,14 +1418,16 @@ public static class MxGatewayClientCli
.WithCancellation(cancellationToken)
.ConfigureAwait(false))
{
if (jsonLines)
{
output.WriteLine(ProtobufJsonFormatter.Format(gatewayEvent));
}
else if (json)
if (json && !jsonLines)
{
events.Add(gatewayEvent);
}
else if (gatewayEvent.ReplayGap is { } replayGap)
{
// Render the ReplayGap sentinel as the typed cross-CLI row instead of the raw
// sentinel MxEvent (NEXT-02, mirroring the Go/Python/Rust CLIs).
output.WriteLine(FormatReplayGapRow(replayGap));
}
else
{
output.WriteLine(ProtobufJsonFormatter.Format(gatewayEvent));
@@ -1496,8 +1546,9 @@ public static class MxGatewayClientCli
/// <summary>
/// Renders one <see cref="AlarmFeedMessage"/> for the human-readable
/// (non-JSON) stream-alarms output, distinguishing the <c>payload</c> oneof
/// arms: a snapshot active alarm, the snapshot-complete sentinel, or a live
/// transition.
/// arms: a snapshot active alarm, the snapshot-complete sentinel, a live
/// transition, the provider-mode status, or the feed-level
/// snapshot-completeness status.
/// </summary>
private static string FormatAlarmFeedMessage(AlarmFeedMessage feedMessage)
{
@@ -1509,6 +1560,10 @@ public static class MxGatewayClientCli
$"snapshot-complete {feedMessage.SnapshotComplete}",
AlarmFeedMessage.PayloadOneofCase.Transition =>
$"transition {ProtobufJsonFormatter.Format(feedMessage.Transition)}",
AlarmFeedMessage.PayloadOneofCase.ProviderStatus =>
$"provider-status mode={feedMessage.ProviderStatus.Mode} degraded={feedMessage.ProviderStatus.Degraded.ToString().ToLowerInvariant()} reason=\"{feedMessage.ProviderStatus.Reason}\"",
AlarmFeedMessage.PayloadOneofCase.SnapshotStatus =>
$"snapshot-status truncated={feedMessage.SnapshotStatus.Truncated.ToString().ToLowerInvariant()}",
_ => $"unknown-payload {feedMessage.PayloadCase}",
};
}
@@ -1787,7 +1842,31 @@ public static class MxGatewayClientCli
private static JsonElement EventToJsonElement(MxEvent gatewayEvent)
{
return JsonDocument.Parse(ProtobufJsonFormatter.Format(gatewayEvent)).RootElement.Clone();
string row = gatewayEvent.ReplayGap is { } replayGap
? FormatReplayGapRow(replayGap)
: ProtobufJsonFormatter.Format(gatewayEvent);
return JsonDocument.Parse(row).RootElement.Clone();
}
/// <summary>
/// Formats the typed ReplayGap row shared by the CLIs (NEXT-02). Hand-built so the
/// cursors are JSON numbers like the Go/Python/Rust rows, not the protobuf JSON
/// formatter's quoted uint64 strings.
/// </summary>
/// <param name="replayGap">Replay gap sentinel payload.</param>
/// <returns>A single-line JSON row describing the gap.</returns>
private static string FormatReplayGapRow(ReplayGap replayGap)
{
return JsonSerializer.Serialize(
new
{
replayGap = new
{
requestedAfterSequence = replayGap.RequestedAfterSequence,
oldestAvailableSequence = replayGap.OldestAvailableSequence,
},
},
JsonOptions);
}
private static MxValue ParseValue(CliArguments arguments)
@@ -2372,7 +2451,9 @@ public static class MxGatewayClientCli
writer.WriteLine("mxgw-dotnet activate --session-id <id> --server-handle <n> --item-handle <n> [--json]");
writer.WriteLine("mxgw-dotnet write-secured --session-id <id> --server-handle <n> --item-handle <n> --type <type> --value <value> --current-user-id <n> [--verifier-user-id <n>] [--json]");
writer.WriteLine("mxgw-dotnet write-secured2 --session-id <id> --server-handle <n> --item-handle <n> --type <type> --value <value> --current-user-id <n> [--verifier-user-id <n>] [--timestamp <iso>] [--json]");
writer.WriteLine("mxgw-dotnet authenticate-user --session-id <id> --server-handle <n> --verify-user <user> (--verify-user-password <pw> | --verify-user-password-env <ENVVAR>) [--json]");
writer.WriteLine("mxgw-dotnet authenticate-user --session-id <id> --server-handle <n> --verify-user <user> [--password <pw>] [--password-env <ENVVAR>] [--json]");
writer.WriteLine(" credential: --password, else the --password-env variable (default MXGATEWAY_VERIFY_PASSWORD); required and never empty.");
writer.WriteLine(" deprecated aliases: --verify-user-password, --verify-user-password-env, MXGATEWAY_VERIFY_USER_PASSWORD.");
writer.WriteLine("mxgw-dotnet archestra-user-to-id --session-id <id> --server-handle <n> --user-guid <guid> [--json]");
writer.WriteLine("mxgw-dotnet subscribe-bulk --session-id <id> --server-handle <n> --items <ref,ref> [--json]");
writer.WriteLine("mxgw-dotnet unsubscribe-bulk --session-id <id> --server-handle <n> --item-handles <n,n> [--json]");
@@ -32,6 +32,57 @@ public sealed class MxCommandReplyExtensionsTests
Assert.Contains("0x80040200", exception.Message);
}
/// <summary>Verifies that a non-OK status category fails even when the raw success member is set.</summary>
[Fact]
public void EnsureMxAccessSuccess_WithNonOkCategoryAndSuccessSet_Throws()
{
MxCommandReply reply = ReadReplyFixture(
"write.status-category-error-success-set.reply.json");
reply.EnsureProtocolSuccess();
MxAccessException exception = Assert.Throws<MxAccessException>(
reply.EnsureMxAccessSuccess);
Assert.Equal(1, Assert.Single(exception.Statuses).Success);
Assert.Contains("CommunicationError", exception.Message, StringComparison.Ordinal);
}
/// <summary>Verifies that an Ok status category succeeds even when the raw success member is zero.</summary>
[Fact]
public void EnsureMxAccessSuccess_WithOkCategoryAndZeroSuccess_ReturnsReply()
{
MxCommandReply reply = ReadReplyFixture(
"write.status-category-ok-success-zero.reply.json");
Assert.Equal(0, Assert.Single(reply.Statuses).Success);
Assert.Same(reply, reply.EnsureProtocolSuccess());
Assert.Same(reply, reply.EnsureMxAccessSuccess());
}
/// <summary>Verifies that a positive HResult (S_FALSE) is a COM success code, not a failure.</summary>
[Fact]
public void EnsureMxAccessSuccess_WithPositiveHResult_ReturnsReply()
{
MxCommandReply reply = ReadReplyFixture("write.hresult-s-false.reply.json");
Assert.Equal(1, reply.Hresult);
Assert.Same(reply, reply.EnsureProtocolSuccess());
Assert.Same(reply, reply.EnsureMxAccessSuccess());
}
/// <summary>Verifies that a negative HResult fails even when every status entry is Ok.</summary>
[Fact]
public void EnsureMxAccessSuccess_WithNegativeHResult_Throws()
{
MxCommandReply reply = ReadReplyFixture("write.hresult-e-fail.reply.json");
reply.EnsureProtocolSuccess();
MxAccessException exception = Assert.Throws<MxAccessException>(
reply.EnsureMxAccessSuccess);
Assert.Equal(-2147467259, exception.HResultCode);
}
/// <summary>Verifies that session-not-found protocol failures throw the correct gateway exception.</summary>
[Fact]
public void EnsureProtocolSuccess_WithSessionFailure_ThrowsSessionException()
@@ -1,3 +1,4 @@
using System.Text.Json;
using Google.Protobuf.WellKnownTypes;
using ZB.MOM.WW.MxGateway.Client.Cli;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
@@ -235,7 +236,7 @@ public sealed class MxGatewayClientCliTests
"--session-id", "session-fixture",
"--server-handle", "12",
"--verify-user", "operator",
"--verify-user-password", password,
"--password", password,
],
output,
error,
@@ -246,6 +247,235 @@ public sealed class MxGatewayClientCliTests
Assert.Contains("[redacted]", error.ToString());
}
/// <summary>
/// CLI-45: <c>--password</c> is the primary credential flag, matching the other
/// four CLIs. The credential reaches the wire but never stdout/stderr.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task RunAsync_AuthenticateUser_AcceptsCanonicalPasswordFlag()
{
const string password = "canonical-flag-credential";
using var output = new StringWriter();
using var error = new StringWriter();
FakeCliClient fakeClient = new();
fakeClient.InvokeReplies.Enqueue(new MxCommandReply
{
SessionId = "session-fixture",
Kind = MxCommandKind.AuthenticateUser,
ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok },
AuthenticateUser = new AuthenticateUserReply { UserId = 11 },
});
int exitCode = await MxGatewayClientCli.RunAsync(
[
"authenticate-user",
"--endpoint", "http://localhost:5000",
"--api-key", "test-api-key",
"--session-id", "session-fixture",
"--server-handle", "12",
"--verify-user", "operator",
"--password", password,
"--json",
],
output,
error,
_ => fakeClient);
Assert.Equal(0, exitCode);
MxCommandRequest request = Assert.Single(fakeClient.InvokeRequests);
Assert.Equal(password, request.Command.AuthenticateUser.VerifyUserPassword);
Assert.DoesNotContain(password, output.ToString(), StringComparison.Ordinal);
Assert.DoesNotContain(password, error.ToString(), StringComparison.Ordinal);
}
/// <summary>
/// CLI-45: the credential is read from the environment variable named by
/// <c>--password-env</c>, whose default is the canonical
/// <c>MXGATEWAY_VERIFY_PASSWORD</c> shared by all five CLIs.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Theory]
[InlineData(null, "MXGATEWAY_VERIFY_PASSWORD")]
[InlineData("MXGW_TEST_CLI45_ENV", "MXGW_TEST_CLI45_ENV")]
public async Task RunAsync_AuthenticateUser_ReadsCredentialFromNamedEnvironmentVariable(
string? passwordEnvArgument,
string environmentName)
{
const string password = "env-sourced-credential";
using EnvironmentVariableScope scope = new(environmentName, password);
using var output = new StringWriter();
using var error = new StringWriter();
FakeCliClient fakeClient = new();
fakeClient.InvokeReplies.Enqueue(new MxCommandReply
{
SessionId = "session-fixture",
Kind = MxCommandKind.AuthenticateUser,
ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok },
AuthenticateUser = new AuthenticateUserReply { UserId = 12 },
});
List<string> args =
[
"authenticate-user",
"--endpoint", "http://localhost:5000",
"--api-key", "test-api-key",
"--session-id", "session-fixture",
"--server-handle", "12",
"--verify-user", "operator",
"--json",
];
if (passwordEnvArgument is not null)
{
args.Add("--password-env");
args.Add(passwordEnvArgument);
}
int exitCode = await MxGatewayClientCli.RunAsync([.. args], output, error, _ => fakeClient);
Assert.Equal(0, exitCode);
MxCommandRequest request = Assert.Single(fakeClient.InvokeRequests);
Assert.Equal(password, request.Command.AuthenticateUser.VerifyUserPassword);
Assert.DoesNotContain(password, output.ToString(), StringComparison.Ordinal);
}
/// <summary>
/// CLI-45: the pre-rename names stay usable for one release — the deprecated
/// <c>--verify-user-password</c> flag and the deprecated
/// <c>MXGATEWAY_VERIFY_USER_PASSWORD</c> environment variable both still resolve.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task RunAsync_AuthenticateUser_HonoursDeprecatedAliases()
{
using var flagOutput = new StringWriter();
using var flagError = new StringWriter();
FakeCliClient flagClient = new();
flagClient.InvokeReplies.Enqueue(new MxCommandReply
{
SessionId = "session-fixture",
Kind = MxCommandKind.AuthenticateUser,
ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok },
AuthenticateUser = new AuthenticateUserReply { UserId = 13 },
});
int flagExitCode = await MxGatewayClientCli.RunAsync(
[
"authenticate-user",
"--endpoint", "http://localhost:5000",
"--api-key", "test-api-key",
"--session-id", "session-fixture",
"--server-handle", "12",
"--verify-user", "operator",
"--verify-user-password", "legacy-flag-credential",
"--json",
],
flagOutput,
flagError,
_ => flagClient);
Assert.Equal(0, flagExitCode);
Assert.Equal(
"legacy-flag-credential",
Assert.Single(flagClient.InvokeRequests).Command.AuthenticateUser.VerifyUserPassword);
using EnvironmentVariableScope canonical = new("MXGATEWAY_VERIFY_PASSWORD", null);
using EnvironmentVariableScope legacy = new("MXGATEWAY_VERIFY_USER_PASSWORD", "legacy-env-credential");
using var envOutput = new StringWriter();
using var envError = new StringWriter();
FakeCliClient envClient = new();
envClient.InvokeReplies.Enqueue(new MxCommandReply
{
SessionId = "session-fixture",
Kind = MxCommandKind.AuthenticateUser,
ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok },
AuthenticateUser = new AuthenticateUserReply { UserId = 14 },
});
int envExitCode = await MxGatewayClientCli.RunAsync(
[
"authenticate-user",
"--endpoint", "http://localhost:5000",
"--api-key", "test-api-key",
"--session-id", "session-fixture",
"--server-handle", "12",
"--verify-user", "operator",
"--json",
],
envOutput,
envError,
_ => envClient);
Assert.Equal(0, envExitCode);
Assert.Equal(
"legacy-env-credential",
Assert.Single(envClient.InvokeRequests).Command.AuthenticateUser.VerifyUserPassword);
}
/// <summary>
/// CLI-45: a missing or empty credential fails fast before the invoke — the CLI
/// never sends a fabricated empty password to the wire. The error names the flag
/// and the environment variable, never a value.
/// </summary>
/// <param name="explicitEmptyFlag">Whether to pass an explicit empty --password.</param>
/// <returns>A task that represents the asynchronous operation.</returns>
[Theory]
[InlineData(false)]
[InlineData(true)]
public async Task RunAsync_AuthenticateUser_FailsFastOnMissingOrEmptyCredential(bool explicitEmptyFlag)
{
using EnvironmentVariableScope canonical = new("MXGATEWAY_VERIFY_PASSWORD", explicitEmptyFlag ? string.Empty : null);
using EnvironmentVariableScope legacy = new("MXGATEWAY_VERIFY_USER_PASSWORD", null);
using var output = new StringWriter();
using var error = new StringWriter();
FakeCliClient fakeClient = new();
List<string> args =
[
"authenticate-user",
"--endpoint", "http://localhost:5000",
"--api-key", "test-api-key",
"--session-id", "session-fixture",
"--server-handle", "12",
"--verify-user", "operator",
];
if (explicitEmptyFlag)
{
args.Add("--password");
args.Add(string.Empty);
}
int exitCode = await MxGatewayClientCli.RunAsync([.. args], output, error, _ => fakeClient);
Assert.Equal(1, exitCode);
Assert.Empty(fakeClient.InvokeRequests);
Assert.Contains("--password", error.ToString(), StringComparison.Ordinal);
Assert.Contains("MXGATEWAY_VERIFY_PASSWORD", error.ToString(), StringComparison.Ordinal);
}
/// <summary>
/// Sets an environment variable for the duration of a test and restores the
/// previous value on dispose, so credential-resolution tests do not depend on
/// (or leak into) the ambient environment.
/// </summary>
private sealed class EnvironmentVariableScope : IDisposable
{
private readonly string _name;
private readonly string? _original;
public EnvironmentVariableScope(string name, string? value)
{
_name = name;
_original = Environment.GetEnvironmentVariable(name);
Environment.SetEnvironmentVariable(name, value);
}
public void Dispose()
{
Environment.SetEnvironmentVariable(_name, _original);
}
}
/// <summary>Verifies that error output redacts sensitive API key values.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
@@ -356,6 +586,84 @@ public sealed class MxGatewayClientCliTests
Assert.DoesNotContain("ON_WRITE_COMPLETE", output.ToString());
}
/// <summary>
/// Verifies stream-events renders the ReplayGap sentinel as the typed cross-CLI row —
/// numeric cursors under a replayGap key — instead of the raw sentinel MxEvent (NEXT-02).
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task RunAsync_StreamEvents_RendersReplayGapAsTypedRow()
{
using var output = new StringWriter();
using var error = new StringWriter();
FakeCliClient fakeClient = new();
fakeClient.Events.Add(new MxEvent
{
ReplayGap = new ReplayGap
{
RequestedAfterSequence = 7,
OldestAvailableSequence = 42,
},
});
fakeClient.Events.Add(new MxEvent
{
SessionId = "session-fixture",
Family = MxEventFamily.OnDataChange,
WorkerSequence = 43,
});
int exitCode = await MxGatewayClientCli.RunAsync(
[
"stream-events",
"--endpoint",
"http://localhost:5000",
"--api-key",
"test-api-key",
"--session-id",
"session-fixture",
"--max-events",
"2",
],
output,
error,
_ => fakeClient);
Assert.Equal(0, exitCode);
string[] rows = output.ToString().Split(Environment.NewLine, StringSplitOptions.RemoveEmptyEntries);
Assert.Equal(2, rows.Length);
using JsonDocument gapRow = JsonDocument.Parse(rows[0]);
JsonElement gap = gapRow.RootElement.GetProperty("replayGap");
Assert.Equal(7UL, gap.GetProperty("requestedAfterSequence").GetUInt64());
Assert.Equal(42UL, gap.GetProperty("oldestAvailableSequence").GetUInt64());
Assert.Equal(JsonValueKind.Number, gap.GetProperty("requestedAfterSequence").ValueKind);
Assert.DoesNotContain("MX_EVENT_FAMILY_UNSPECIFIED", rows[0], StringComparison.Ordinal);
Assert.Contains("workerSequence", rows[1], StringComparison.Ordinal);
// The aggregate --json shape carries the same typed row inside the events array.
using var aggregateOutput = new StringWriter();
int aggregateExit = await MxGatewayClientCli.RunAsync(
[
"stream-events",
"--endpoint",
"http://localhost:5000",
"--api-key",
"test-api-key",
"--session-id",
"session-fixture",
"--max-events",
"2",
"--json",
],
aggregateOutput,
error,
_ => fakeClient);
Assert.Equal(0, aggregateExit);
using JsonDocument aggregate = JsonDocument.Parse(aggregateOutput.ToString());
JsonElement firstRow = aggregate.RootElement.GetProperty("events")[0];
Assert.Equal(42UL, firstRow.GetProperty("replayGap").GetProperty("oldestAvailableSequence").GetUInt64());
}
/// <summary>Verifies that stream-alarms with --max-events stops output and distinguishes payload cases.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
@@ -397,6 +705,50 @@ public sealed class MxGatewayClientCliTests
Assert.Equal(string.Empty, error.ToString());
}
/// <summary>Verifies that false alarm-feed status flags remain explicit in text while protobuf JSON omits defaults.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task RunAsync_StreamAlarms_StatusFramesDistinguishTextFromProtobufJson()
{
FakeCliClient fakeClient = new();
fakeClient.AlarmFeedMessages.Add(new AlarmFeedMessage
{
ProviderStatus = new AlarmProviderStatus { Degraded = false },
});
fakeClient.AlarmFeedMessages.Add(new AlarmFeedMessage
{
SnapshotStatus = new AlarmSnapshotStatus { Truncated = false },
});
using var textOutput = new StringWriter();
using var textError = new StringWriter();
int textExit = await MxGatewayClientCli.RunAsync(
["stream-alarms", "--endpoint", "http://localhost:5000", "--api-key", "test-api-key", "--max-events", "2"],
textOutput,
textError,
_ => fakeClient);
Assert.Equal(0, textExit);
Assert.Equal(
["provider-status mode=Unspecified degraded=false reason=\"\"", "snapshot-status truncated=false"],
textOutput.ToString().Split(Environment.NewLine, StringSplitOptions.RemoveEmptyEntries));
Assert.Equal(string.Empty, textError.ToString());
using var jsonOutput = new StringWriter();
using var jsonError = new StringWriter();
int jsonExit = await MxGatewayClientCli.RunAsync(
["stream-alarms", "--endpoint", "http://localhost:5000", "--api-key", "test-api-key", "--max-events", "2", "--jsonl"],
jsonOutput,
jsonError,
_ => fakeClient);
Assert.Equal(0, jsonExit);
Assert.Equal(
["{ \"providerStatus\": { } }", "{ \"snapshotStatus\": { } }"],
jsonOutput.ToString().Split(Environment.NewLine, StringSplitOptions.RemoveEmptyEntries));
Assert.Equal(string.Empty, jsonError.ToString());
}
/// <summary>Verifies that acknowledge-alarm builds a request and prints the JSON reply.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
@@ -0,0 +1,162 @@
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Client.Tests;
/// <summary>
/// Drives the public client API against <see cref="WireFakeGatewayServer"/> — a real
/// gRPC server on loopback — so the transport, protobuf serialization, call metadata,
/// and gRPC status mapping are all exercised. Every other test in this project
/// substitutes <see cref="FakeGatewayTransport"/> and therefore proves nothing about
/// what actually crosses the wire.
/// </summary>
public sealed class MxGatewayClientWireTests
{
private const string ApiKey = "mxgw_wiretest_secret";
/// <summary>
/// Verifies the full session happy path decodes real wire bytes end to end.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task SessionRoundTrip_OverRealTransport_DecodesEveryReplyField()
{
await using WireFakeGatewayServer server = await WireFakeGatewayServer.StartAsync();
await using MxGatewayClient client = server.CreateClient(ApiKey);
MxGatewaySession session = await client.OpenSessionAsync(
new OpenSessionRequest { ClientSessionName = "wire-test" });
Assert.Equal(WireFakeGatewayServer.FakeGatewayService.SessionId, session.SessionId);
Assert.Equal("fake-backend", session.OpenSessionReply.BackendName);
Assert.Equal(1234, session.OpenSessionReply.WorkerProcessId);
Assert.Equal(3u, session.OpenSessionReply.GatewayProtocolVersion);
Assert.Equal(["events", "invoke"], session.OpenSessionReply.Capabilities);
int serverHandle = await session.RegisterAsync("wire-test-client");
Assert.Equal(WireFakeGatewayServer.FakeGatewayService.ServerHandle, serverHandle);
MxCommandRequest? invoke = server.Service.InvokeRequest;
Assert.NotNull(invoke);
Assert.Equal(MxCommandKind.Register, invoke.Command.Kind);
Assert.Equal("wire-test-client", invoke.Command.Register.ClientName);
List<MxEvent> events = await CollectAsync(client.StreamEventsAsync(
new StreamEventsRequest { SessionId = session.SessionId }));
MxEvent single = Assert.Single(events);
Assert.Equal(MxEventFamily.OnDataChange, single.Family);
Assert.Equal(WireFakeGatewayServer.FakeGatewayService.ServerHandle, single.ServerHandle);
Assert.Equal(WireFakeGatewayServer.FakeGatewayService.ItemHandle, single.ItemHandle);
Assert.Equal(17, single.Value.Int32Value);
Assert.Equal(192, single.Quality);
Assert.Equal(9ul, single.WorkerSequence);
Assert.Equal(MxEvent.BodyOneofCase.OnDataChange, single.BodyCase);
CloseSessionReply closeReply = await session.CloseAsync();
Assert.Equal(SessionState.Closed, closeReply.FinalState);
Assert.Equal(
WireFakeGatewayServer.FakeGatewayService.SessionId,
server.Service.CloseSessionRequest?.SessionId);
}
/// <summary>
/// Verifies the API key reaches the server as a bearer header on unary and
/// streaming calls alike. A transport fake can only assert what the client passes;
/// this asserts what the server receives.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task ApiKey_ReachesTheServerAsBearerMetadata_OnEveryRpc()
{
await using WireFakeGatewayServer server = await WireFakeGatewayServer.StartAsync();
await using MxGatewayClient client = server.CreateClient(ApiKey);
MxGatewaySession session = await client.OpenSessionAsync(
new OpenSessionRequest { ClientSessionName = "wire-test" });
await session.RegisterAsync("wire-test-client");
await CollectAsync(client.StreamEventsAsync(
new StreamEventsRequest { SessionId = session.SessionId }));
await session.CloseAsync();
string expected = $"Bearer {ApiKey}";
Assert.Equal(
new Dictionary<string, string>
{
["OpenSession"] = expected,
["Invoke"] = expected,
["StreamEvents"] = expected,
["CloseSession"] = expected,
},
server.Service.AuthorizationByMethod);
}
/// <summary>
/// Verifies the gateway's replay-gap sentinel survives serialization and is
/// surfaced as a typed, non-terminal stream item.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task ReplayGapSentinel_SurvivesTheWire_AsTypedStreamItem()
{
await using WireFakeGatewayServer server = await WireFakeGatewayServer.StartAsync(service =>
service.ReplayGap = new ReplayGap
{
RequestedAfterSequence = 3,
OldestAvailableSequence = 8,
});
await using MxGatewayClient client = server.CreateClient(ApiKey);
MxGatewaySession session = await client.OpenSessionAsync(
new OpenSessionRequest { ClientSessionName = "wire-test" });
List<MxEventStreamItem> items = [];
IAsyncEnumerable<MxEvent> stream = client.StreamEventsAsync(new StreamEventsRequest
{
SessionId = session.SessionId,
AfterWorkerSequence = 3,
});
await foreach (MxEventStreamItem item in stream.AsStreamItemsAsync())
{
items.Add(item);
}
Assert.Equal(2, items.Count);
Assert.True(items[0].IsReplayGap);
Assert.Equal(3ul, items[0].ReplayGap!.RequestedAfterSequence);
Assert.Equal(8ul, items[0].ReplayGap!.OldestAvailableSequence);
Assert.False(items[1].IsReplayGap);
Assert.Equal(MxEventFamily.OnDataChange, items[1].Event.Family);
Assert.Equal(3ul, server.Service.StreamEventsRequest?.AfterWorkerSequence);
}
/// <summary>
/// Verifies a genuine <c>PERMISSION_DENIED</c> status maps to the typed client
/// exception rather than a bare RpcException.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task PermissionDeniedStatus_MapsToAuthorizationException()
{
await using WireFakeGatewayServer server = await WireFakeGatewayServer.StartAsync(
service => service.DenyInvoke = true);
await using MxGatewayClient client = server.CreateClient(ApiKey);
MxGatewaySession session = await client.OpenSessionAsync(
new OpenSessionRequest { ClientSessionName = "wire-test" });
await Assert.ThrowsAsync<MxGatewayAuthorizationException>(
() => session.RegisterAsync("wire-test-client"));
}
private static async Task<List<MxEvent>> CollectAsync(IAsyncEnumerable<MxEvent> stream)
{
List<MxEvent> events = [];
await foreach (MxEvent gatewayEvent in stream)
{
events.Add(gatewayEvent);
}
return events;
}
}
@@ -0,0 +1,96 @@
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Client.Tests;
/// <summary>
/// Unit tests for <see cref="MxGatewaySecretRedaction"/> — the exact-substring scrub applied to
/// diagnostic text and rebuilt exceptions before they leave the client on a failure path.
/// </summary>
public sealed class MxGatewaySecretRedactionTests
{
[Fact]
public void Redact_ReplacesEveryOccurrenceOfSecret()
{
string result = MxGatewaySecretRedaction.Redact(
"pw=hunter2 retry pw=hunter2 again hunter2",
"hunter2");
Assert.DoesNotContain("hunter2", result, StringComparison.Ordinal);
Assert.Equal("pw=<redacted> retry pw=<redacted> again <redacted>", result);
}
[Fact]
public void Redact_ScrubsBothSecretsWhenOneIsSubstringOfTheOther()
{
// "secret" is a substring of "secretPassword"; both must be fully scrubbed regardless of
// supplied order — no residual leak of either verbatim value.
string result = MxGatewaySecretRedaction.Redact(
"a=secretPassword b=secret",
"secret",
"secretPassword");
Assert.DoesNotContain("secretPassword", result, StringComparison.Ordinal);
Assert.DoesNotContain("secret", result, StringComparison.Ordinal);
}
[Fact]
public void Redact_WithNullSecretsArray_ReturnsMessageUnchanged()
{
const string message = "nothing to scrub here";
string result = MxGatewaySecretRedaction.Redact(message, null!);
Assert.Equal(message, result);
}
[Fact]
public void Redact_WithEmptySecretsArray_ReturnsMessageUnchanged()
{
const string message = "nothing to scrub here";
string result = MxGatewaySecretRedaction.Redact(message);
Assert.Equal(message, result);
}
[Fact]
public void Redact_IgnoresWhitespaceOnlySecret()
{
// A whitespace-only secret must not over-redact the internal spaces of the message.
const string message = "user operator logged in";
string result = MxGatewaySecretRedaction.Redact(message, " ");
Assert.Equal(message, result);
}
[Fact]
public void Redacted_PreservesConcreteSubtypeAndDoesNotChainSecretBearingOriginal()
{
const string secret = "hunter2";
Exception transportCause = new InvalidOperationException("transport reset");
MxGatewaySessionException original = new(
$"session rejected credential '{secret}'",
"session-1",
"correlation-1",
new ProtocolStatus { Code = ProtocolStatusCode.SessionNotReady, Message = $"echoed '{secret}'" },
hResult: -1,
statuses: [new MxStatusProxy { DiagnosticText = $"denied '{secret}'" }],
innerException: transportCause);
MxGatewayException redacted = MxGatewaySecretRedaction.Redacted(original, secret);
// Concrete runtime type is preserved.
Assert.IsType<MxGatewaySessionException>(redacted);
// The secret is gone from the message and every structured accessor.
Assert.DoesNotContain(secret, redacted.Message, StringComparison.Ordinal);
Assert.DoesNotContain(secret, redacted.ToString(), StringComparison.Ordinal);
Assert.DoesNotContain(secret, redacted.ProtocolStatus!.Message, StringComparison.Ordinal);
Assert.All(redacted.Statuses, status =>
Assert.DoesNotContain(secret, status.DiagnosticText, StringComparison.Ordinal));
Assert.Contains("<redacted>", redacted.Message, StringComparison.Ordinal);
// The secret-bearing original is NOT chained; the original's transport cause is carried.
Assert.NotSame(original, redacted.InnerException);
Assert.Same(transportCause, redacted.InnerException);
}
}
@@ -0,0 +1,165 @@
using Google.Protobuf;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Client.Tests;
/// <summary>
/// Tests for the credential-scrub (CLI-40) and malformed-reply (CLI-41) contracts on the
/// credential and id-returning session helpers, driven from shared behavior fixtures.
/// </summary>
public sealed class MxGatewaySessionReplyContractTests
{
/// <summary>
/// CLI-40: when MXAccess echoes the submitted credential back in its failure diagnostic,
/// the surfaced exception message must scrub it to the library redaction marker.
/// </summary>
[Fact]
public async Task AuthenticateUserAsync_RedactsEchoedCredentialInFailureMessage()
{
const string password = "sup3rSecretVerify9f3a2b";
FakeGatewayTransport transport = CreateTransport();
transport.AddInvokeReply(ReadReplyFixture("authenticate-user.echoed-credential.reply.json"));
await using MxGatewayClient client = CreateClient(transport);
MxGatewaySession session = await client.OpenSessionAsync();
MxAccessException exception = await Assert.ThrowsAsync<MxAccessException>(
async () => await session.AuthenticateUserAsync(12, "operator", password));
Assert.DoesNotContain(password, exception.Message, StringComparison.Ordinal);
Assert.Contains("<redacted>", exception.Message, StringComparison.Ordinal);
// ToString() is what logging frameworks emit; the secret-bearing original must not be
// chained as an inner exception where it would re-surface the credential verbatim.
Assert.DoesNotContain(password, exception.ToString(), StringComparison.Ordinal);
}
/// <summary>
/// CLI-40: the redacted exception must not leak the echoed credential through any structured
/// accessor either — <see cref="MxAccessException.Reply"/> (protocol message, diagnostic
/// message, and each MXSTATUS_PROXY diagnostic text) and <see cref="MxGatewayException.Statuses"/>
/// all carry the server-echoed credential verbatim before the fix. Both the OK+negative-HRESULT
/// and the MXACCESS_FAILURE reply route to <see cref="MxAccessException"/>, so both must scrub.
/// </summary>
/// <param name="fixture">The echoed-credential reply fixture to drive.</param>
[Theory]
[InlineData("authenticate-user.echoed-credential.reply.json")]
[InlineData("authenticate-user.echoed-credential-mxaccess-failure.reply.json")]
public async Task AuthenticateUserAsync_RedactsEchoedCredentialInStructuredAccessors(string fixture)
{
const string password = "sup3rSecretVerify9f3a2b";
FakeGatewayTransport transport = CreateTransport();
transport.AddInvokeReply(ReadReplyFixture(fixture));
await using MxGatewayClient client = CreateClient(transport);
MxGatewaySession session = await client.OpenSessionAsync();
MxAccessException exception = await Assert.ThrowsAsync<MxAccessException>(
async () => await session.AuthenticateUserAsync(12, "operator", password));
Assert.DoesNotContain(password, exception.Message, StringComparison.Ordinal);
Assert.Contains("<redacted>", exception.Message, StringComparison.Ordinal);
Assert.DoesNotContain(password, exception.ToString(), StringComparison.Ordinal);
Assert.DoesNotContain(password, exception.Reply.ProtocolStatus.Message, StringComparison.Ordinal);
Assert.DoesNotContain(password, exception.Reply.DiagnosticMessage, StringComparison.Ordinal);
foreach (MxStatusProxy status in exception.Reply.Statuses)
{
Assert.DoesNotContain(password, status.DiagnosticText, StringComparison.Ordinal);
}
foreach (MxStatusProxy status in exception.Statuses)
{
Assert.DoesNotContain(password, status.DiagnosticText, StringComparison.Ordinal);
}
}
/// <summary>
/// CLI-41: an OK reply that carries neither the typed AuthenticateUser payload nor an
/// int32 return_value is a malformed reply, surfaced as a typed exception rather than an NRE.
/// </summary>
[Fact]
public async Task AuthenticateUserAsync_MissingPayloadAndReturnValue_ThrowsMalformedReply()
{
FakeGatewayTransport transport = CreateTransport();
transport.AddInvokeReply(ReadReplyFixture("authenticate-user.missing-payload.reply.json"));
await using MxGatewayClient client = CreateClient(transport);
MxGatewaySession session = await client.OpenSessionAsync();
await Assert.ThrowsAsync<MxGatewayMalformedReplyException>(
async () => await session.AuthenticateUserAsync(12, "operator", "pw"));
}
/// <summary>
/// CLI-41: an OK reply that omits the typed payload but carries an int32 return_value
/// resolves to that return value.
/// </summary>
[Fact]
public async Task AuthenticateUserAsync_ReturnValueOnly_ResolvesReturnValue()
{
FakeGatewayTransport transport = CreateTransport();
transport.AddInvokeReply(ReadReplyFixture("authenticate-user.return-value-only.reply.json"));
await using MxGatewayClient client = CreateClient(transport);
MxGatewaySession session = await client.OpenSessionAsync();
int userId = await session.AuthenticateUserAsync(12, "operator", "pw");
Assert.Equal(7, userId);
}
/// <summary>
/// CLI-41: the AddBufferedItem fallback shares the malformed-reply contract — an OK reply
/// with neither a typed item handle nor an int32 return_value throws the typed exception.
/// </summary>
[Fact]
public async Task AddBufferedItemAsync_MissingPayloadAndReturnValue_ThrowsMalformedReply()
{
FakeGatewayTransport transport = CreateTransport();
transport.AddInvokeReply(new MxCommandReply
{
SessionId = "session-fixture",
Kind = MxCommandKind.AddBufferedItem,
ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok },
});
await using MxGatewayClient client = CreateClient(transport);
MxGatewaySession session = await client.OpenSessionAsync();
await Assert.ThrowsAsync<MxGatewayMalformedReplyException>(
async () => await session.AddBufferedItemAsync(12, "Area001.Pump001.Speed", "runtime"));
}
private static MxGatewayClient CreateClient(FakeGatewayTransport transport)
{
return new MxGatewayClient(transport.Options, transport);
}
private static FakeGatewayTransport CreateTransport()
{
return new FakeGatewayTransport(new MxGatewayClientOptions
{
Endpoint = new Uri("http://localhost:5000"),
ApiKey = "test-api-key",
});
}
private static MxCommandReply ReadReplyFixture(string fileName)
{
DirectoryInfo directory = new(AppContext.BaseDirectory);
while (directory is not null)
{
string path = Path.Combine(
directory.FullName,
"clients",
"proto",
"fixtures",
"behavior",
"command-replies",
fileName);
if (File.Exists(path))
{
return JsonParser.Default.Parse<MxCommandReply>(File.ReadAllText(path));
}
directory = directory.Parent!;
}
throw new FileNotFoundException(fileName);
}
}
@@ -19,9 +19,9 @@ public sealed class MxStatusProxyExtensionsTests
{
MxStatusProxy status = JsonParser.Default.Parse<MxStatusProxy>(
testCase.GetProperty("status").GetRawText());
int success = testCase.GetProperty("status").GetProperty("success").GetInt32();
Assert.Equal(success != 0 && status.Category is MxStatusCategory.Ok, status.IsSuccess());
bool wantSuccess = testCase.GetProperty("wantSuccess").GetBoolean();
Assert.Equal(wantSuccess, status.IsSuccess());
Assert.Equal(
testCase.GetProperty("status").GetProperty("rawCategory").GetInt32(),
status.RawCategory);
@@ -31,6 +31,22 @@ public sealed class MxStatusProxyExtensionsTests
}
}
/// <summary>Verifies that the raw success member never overrides the authoritative category.</summary>
[Theory]
[InlineData(MxStatusCategory.Ok, 0, true)]
[InlineData(MxStatusCategory.Ok, 1, true)]
[InlineData(MxStatusCategory.CommunicationError, 1, false)]
[InlineData(MxStatusCategory.Unspecified, 1, false)]
public void IsSuccess_BranchesOnCategoryOnly(
MxStatusCategory category,
int success,
bool expected)
{
MxStatusProxy status = new() { Category = category, Success = success };
Assert.Equal(expected, status.IsSuccess());
}
private static string ReadFixture(string category, string fileName)
{
DirectoryInfo directory = new(AppContext.BaseDirectory);
@@ -0,0 +1,264 @@
using System.Collections.Concurrent;
using System.Net;
using Grpc.Core;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Hosting.Server;
using Microsoft.AspNetCore.Hosting.Server.Features;
using Microsoft.AspNetCore.Server.Kestrel.Core;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Client.Tests;
/// <summary>
/// Hosts the real <c>mxaccess_gateway.v1.MxAccessGateway</c> service on a loopback
/// Kestrel endpoint so client tests exercise genuine HTTP/2 framing, protobuf
/// serialization, call metadata, and gRPC status propagation.
/// </summary>
/// <remarks>
/// <para>
/// This is the counterpart of <see cref="FakeGatewayTransport"/>: that fake replaces
/// <c>IMxGatewayClientTransport</c>, so nothing below the client wrapper runs. This one
/// replaces only the gateway's <em>behaviour</em> — every byte between the client and
/// the service is the real wire format. Contract breaks that a transport fake cannot
/// see (a field the client never decodes, metadata it does not actually send, a status
/// code it maps differently once it arrives as a real <see cref="RpcException"/>) fail
/// here.
/// </para>
/// <para>
/// Plaintext h2c is used deliberately: TLS is covered by
/// <c>MxGatewayClientTlsHandlerTests</c>, and h2c keeps the harness certificate-free so
/// it runs identically on every CI host. See <c>docs/GatewayTesting.md</c>
/// (Client Wire Tests) for the shared pattern and its Python counterpart.
/// </para>
/// </remarks>
internal sealed class WireFakeGatewayServer : IAsyncDisposable
{
private readonly WebApplication _app;
private WireFakeGatewayServer(WebApplication app, FakeGatewayService service, int port)
{
_app = app;
Service = service;
Endpoint = new Uri($"http://127.0.0.1:{port}");
}
/// <summary>
/// Gets the canned service backing the endpoint; tests read its recorded requests.
/// </summary>
public FakeGatewayService Service { get; }
/// <summary>
/// Gets the h2c endpoint to point <see cref="MxGatewayClientOptions.Endpoint"/> at.
/// </summary>
public Uri Endpoint { get; }
/// <summary>
/// Starts a server on an ephemeral loopback port.
/// </summary>
/// <param name="configure">Optional configuration of the canned service.</param>
/// <returns>The started server.</returns>
public static async Task<WireFakeGatewayServer> StartAsync(Action<FakeGatewayService>? configure = null)
{
FakeGatewayService service = new();
configure?.Invoke(service);
WebApplicationBuilder builder = WebApplication.CreateBuilder();
builder.Logging.ClearProviders();
builder.WebHost.ConfigureKestrel(options =>
// Port 0 lets the OS pick; HTTP/2 without TLS (h2c) is what the client's
// plain http:// endpoint negotiates via RequestVersionExact.
options.Listen(IPAddress.Loopback, 0, listen => listen.Protocols = HttpProtocols.Http2));
builder.Services.AddGrpc();
builder.Services.AddSingleton(service);
WebApplication app = builder.Build();
app.MapGrpcService<FakeGatewayService>();
await app.StartAsync().ConfigureAwait(false);
return new WireFakeGatewayServer(app, service, ResolvePort(app));
}
/// <summary>
/// Creates a client bound to this server's endpoint.
/// </summary>
/// <param name="apiKey">API key the client should present.</param>
/// <returns>A client that talks to this server over h2c.</returns>
public MxGatewayClient CreateClient(string apiKey) =>
MxGatewayClient.Create(new MxGatewayClientOptions
{
Endpoint = Endpoint,
ApiKey = apiKey,
UseTls = false,
DefaultCallTimeout = TimeSpan.FromSeconds(30),
});
/// <inheritdoc />
public async ValueTask DisposeAsync()
{
await _app.StopAsync().ConfigureAwait(false);
await _app.DisposeAsync().ConfigureAwait(false);
}
private static int ResolvePort(WebApplication app)
{
IServerAddressesFeature? addresses = app.Services
.GetRequiredService<IServer>()
.Features
.Get<IServerAddressesFeature>();
string address = addresses?.Addresses.FirstOrDefault()
?? throw new InvalidOperationException("Kestrel did not report a bound address.");
return new Uri(address).Port;
}
/// <summary>
/// Canned gateway answering the four session RPCs with gateway-shaped replies.
/// </summary>
internal sealed class FakeGatewayService : MxAccessGateway.MxAccessGatewayBase
{
/// <summary>The session id every reply carries.</summary>
public const string SessionId = "wire-session-1";
/// <summary>The server handle the canned Register reply returns.</summary>
public const int ServerHandle = 4242;
/// <summary>The item handle the canned data-change event carries.</summary>
public const int ItemHandle = 77;
/// <summary>
/// Gets the <c>authorization</c> header value observed per RPC name.
/// </summary>
public ConcurrentDictionary<string, string> AuthorizationByMethod { get; } = new();
/// <summary>
/// Gets or sets a value indicating whether <c>Invoke</c> fails with
/// <see cref="StatusCode.PermissionDenied"/> instead of replying.
/// </summary>
public bool DenyInvoke { get; set; }
/// <summary>
/// Gets or sets the replay-gap sentinel emitted at the head of the event stream.
/// </summary>
public ReplayGap? ReplayGap { get; set; }
/// <summary>
/// Gets the last <c>Invoke</c> request the client sent, as decoded from the wire.
/// </summary>
public MxCommandRequest? InvokeRequest { get; private set; }
/// <summary>
/// Gets the last <c>StreamEvents</c> request the client sent.
/// </summary>
public StreamEventsRequest? StreamEventsRequest { get; private set; }
/// <summary>
/// Gets the last <c>CloseSession</c> request the client sent.
/// </summary>
public CloseSessionRequest? CloseSessionRequest { get; private set; }
/// <inheritdoc />
public override Task<OpenSessionReply> OpenSession(
OpenSessionRequest request,
ServerCallContext context)
{
Record(context);
return Task.FromResult(new OpenSessionReply
{
SessionId = SessionId,
BackendName = "fake-backend",
WorkerProcessId = 1234,
WorkerProtocolVersion = 1,
GatewayProtocolVersion = 3,
Capabilities = { "events", "invoke" },
ProtocolStatus = Ok(),
});
}
/// <inheritdoc />
public override Task<MxCommandReply> Invoke(MxCommandRequest request, ServerCallContext context)
{
Record(context);
InvokeRequest = request;
if (DenyInvoke)
{
throw new RpcException(new Status(StatusCode.PermissionDenied, "invoke scope required"));
}
return Task.FromResult(new MxCommandReply
{
SessionId = request.SessionId,
CorrelationId = request.ClientCorrelationId,
Kind = request.Command.Kind,
ProtocolStatus = Ok(),
Hresult = 0,
Register = new RegisterReply { ServerHandle = ServerHandle },
});
}
/// <inheritdoc />
public override async Task StreamEvents(
StreamEventsRequest request,
IServerStreamWriter<MxEvent> responseStream,
ServerCallContext context)
{
Record(context);
StreamEventsRequest = request;
if (ReplayGap is not null)
{
// The sentinel shape the gateway emits: family unspecified, body unset,
// only replay_gap populated.
await responseStream.WriteAsync(new MxEvent
{
SessionId = request.SessionId,
ReplayGap = ReplayGap,
}).ConfigureAwait(false);
}
await responseStream.WriteAsync(new MxEvent
{
SessionId = request.SessionId,
Family = MxEventFamily.OnDataChange,
ServerHandle = ServerHandle,
ItemHandle = ItemHandle,
Value = new MxValue { Int32Value = 17 },
Quality = 192,
WorkerSequence = 9,
OnDataChange = new OnDataChangeEvent(),
}).ConfigureAwait(false);
}
/// <inheritdoc />
public override Task<CloseSessionReply> CloseSession(
CloseSessionRequest request,
ServerCallContext context)
{
Record(context);
CloseSessionRequest = request;
return Task.FromResult(new CloseSessionReply
{
SessionId = request.SessionId,
FinalState = SessionState.Closed,
ProtocolStatus = Ok(),
});
}
private static ProtocolStatus Ok() => new() { Code = ProtocolStatusCode.Ok };
private void Record(ServerCallContext context)
{
string? authorization = context.RequestHeaders.GetValue("authorization");
if (authorization is not null)
{
// context.Method is the fully-qualified "/package.Service/Method";
// key on the bare method name so assertions stay readable.
AuthorizationByMethod[context.Method[(context.Method.LastIndexOf('/') + 1)..]] =
authorization;
}
}
}
}
@@ -12,6 +12,15 @@
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.14.1" />
<PackageReference Include="xunit" Version="2.9.3" />
<PackageReference Include="xunit.runner.visualstudio" Version="3.1.4" />
<!-- Wire tests only (WireFakeGatewayServer): hosts the real MxAccessGateway service
on loopback Kestrel so the client is driven over genuine HTTP/2 + protobuf rather
than a substituted transport. Version tracks the gateway server's Grpc.AspNetCore
(src/ZB.MOM.WW.MxGateway.Server) and the client's Grpc.Net.Client, both 2.76.0. -->
<PackageReference Include="Grpc.AspNetCore.Server" Version="2.76.0" />
</ItemGroup>
<ItemGroup>
<FrameworkReference Include="Microsoft.AspNetCore.App" />
</ItemGroup>
<ItemGroup>
@@ -23,7 +23,11 @@ public static class MxCommandReplyExtensions
throw CreateProtocolException(reply, code);
}
/// <summary>Validates that the reply indicates MXAccess success (no HResult or status failures), throwing MxAccessException if not.</summary>
/// <summary>
/// Validates that the reply indicates MXAccess success, throwing MxAccessException if not.
/// Following COM semantics, only a negative HResult is a failure — positive success codes
/// such as <c>S_FALSE</c> pass — and a status entry fails only when its category is not Ok.
/// </summary>
/// <param name="reply">The command reply to check.</param>
/// <returns>The same reply, for chaining.</returns>
public static MxCommandReply EnsureMxAccessSuccess(this MxCommandReply reply)
@@ -31,7 +35,7 @@ public static class MxCommandReplyExtensions
ArgumentNullException.ThrowIfNull(reply);
bool mxAccessFailure = reply.ProtocolStatus?.Code is ProtocolStatusCode.MxaccessFailure;
bool hResultFailure = reply.HasHresult && reply.Hresult != 0;
bool hResultFailure = reply.HasHresult && reply.Hresult < 0;
bool statusFailure = reply.Statuses.Any(status => !status.IsSuccess());
if (!mxAccessFailure && !hResultFailure && !statusFailure)
@@ -0,0 +1,46 @@
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Client;
/// <summary>
/// Exception thrown when the gateway returns a protocol-OK reply that carries neither the
/// expected typed payload nor an int32 <c>return_value</c>, so the client cannot resolve the
/// operation result. This replaces the historical <see cref="NullReferenceException"/> that a
/// blind <c>reply.ReturnValue.Int32Value</c> fallback would throw.
/// </summary>
public sealed class MxGatewayMalformedReplyException : MxGatewayException
{
/// <summary>Initializes a new instance with the given message.</summary>
/// <param name="message">The error message describing the malformed reply.</param>
public MxGatewayMalformedReplyException(string message)
: base(message)
{
}
/// <summary>Initializes a new instance with full diagnostic context.</summary>
/// <param name="message">The error message describing the malformed reply.</param>
/// <param name="sessionId">The session ID, if available.</param>
/// <param name="correlationId">The correlation ID for tracing, if available.</param>
/// <param name="protocolStatus">The protocol status details, if available.</param>
/// <param name="hResult">The HResult code, if available.</param>
/// <param name="statuses">The MXAccess statuses, if available.</param>
/// <param name="innerException">The underlying exception, if any.</param>
public MxGatewayMalformedReplyException(
string message,
string? sessionId = null,
string? correlationId = null,
ProtocolStatus? protocolStatus = null,
int? hResult = null,
IReadOnlyList<MxStatusProxy>? statuses = null,
Exception? innerException = null)
: base(
message,
sessionId,
correlationId,
protocolStatus,
hResult,
statuses ?? [],
innerException)
{
}
}
@@ -0,0 +1,229 @@
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Client;
/// <summary>
/// Scrubs exact secret substrings out of diagnostic text before it leaves the client on an
/// exception path. MXAccess can echo a submitted credential or secured value back inside a
/// failure diagnostic (protocol message, MXSTATUS_PROXY diagnostic text, HRESULT description);
/// this helper replaces any such verbatim occurrence with <c>&lt;redacted&gt;</c> so the raw
/// request payload never reaches a caught exception's message. The marker matches the Go, Rust,
/// and Java clients.
/// </summary>
internal static class MxGatewaySecretRedaction
{
private const string Marker = "<redacted>";
/// <summary>
/// Replaces every usable secret in <paramref name="secrets"/> with the redaction marker
/// (ordinal comparison). Returns the message unchanged when it is null or empty, or when no
/// usable secret is supplied. A secret that is null, empty, or whitespace-only is ignored so
/// it cannot over-redact ordinary separator characters in the message.
/// </summary>
/// <param name="message">The diagnostic message to scrub.</param>
/// <param name="secrets">The secret values to remove from the message.</param>
/// <returns>The scrubbed message.</returns>
internal static string Redact(string message, params string?[] secrets)
{
if (string.IsNullOrEmpty(message) || secrets is null)
{
return message;
}
string result = message;
foreach (string? secret in secrets)
{
if (!string.IsNullOrWhiteSpace(secret))
{
result = result.Replace(secret, Marker, StringComparison.Ordinal);
}
}
return result;
}
/// <summary>
/// Returns a scrubbed clone of <paramref name="reply"/>: the protocol-status message, the
/// reply-level diagnostic message, and each MXSTATUS_PROXY diagnostic text have every verbatim
/// secret replaced with the redaction marker. The original is left untouched. MXAccess can echo
/// a submitted credential into any of these fields, so a redacted exception must carry the
/// scrubbed reply rather than the secret-bearing original.
/// </summary>
/// <param name="reply">The reply to clone and scrub.</param>
/// <param name="secrets">The secret values to remove.</param>
/// <returns>A scrubbed clone of the reply.</returns>
internal static MxCommandReply RedactReply(MxCommandReply reply, params string?[] secrets)
{
ArgumentNullException.ThrowIfNull(reply);
MxCommandReply clone = reply.Clone();
if (clone.ProtocolStatus is not null)
{
clone.ProtocolStatus.Message = Redact(clone.ProtocolStatus.Message, secrets);
}
clone.DiagnosticMessage = Redact(clone.DiagnosticMessage, secrets);
foreach (MxStatusProxy status in clone.Statuses)
{
status.DiagnosticText = Redact(status.DiagnosticText, secrets);
}
return clone;
}
/// <summary>
/// Returns a scrubbed clone of <paramref name="status"/> (its message with every verbatim
/// secret removed), or <see langword="null"/> when the input is null.
/// </summary>
/// <param name="status">The protocol status to clone and scrub.</param>
/// <param name="secrets">The secret values to remove.</param>
/// <returns>A scrubbed clone, or <see langword="null"/>.</returns>
internal static ProtocolStatus? RedactStatus(ProtocolStatus? status, params string?[] secrets)
{
if (status is null)
{
return null;
}
ProtocolStatus clone = status.Clone();
clone.Message = Redact(clone.Message, secrets);
return clone;
}
/// <summary>
/// Returns a list of scrubbed clones of <paramref name="statuses"/> — each MXSTATUS_PROXY's
/// diagnostic text has every verbatim secret removed. The originals are left untouched.
/// </summary>
/// <param name="statuses">The statuses to clone and scrub.</param>
/// <param name="secrets">The secret values to remove.</param>
/// <returns>A list of scrubbed clones.</returns>
internal static IReadOnlyList<MxStatusProxy> RedactStatuses(
IReadOnlyList<MxStatusProxy> statuses,
params string?[] secrets)
{
if (statuses is null || statuses.Count is 0)
{
return statuses ?? [];
}
MxStatusProxy[] result = new MxStatusProxy[statuses.Count];
for (int i = 0; i < statuses.Count; i++)
{
MxStatusProxy clone = statuses[i].Clone();
clone.DiagnosticText = Redact(clone.DiagnosticText, secrets);
result[i] = clone;
}
return result;
}
/// <summary>
/// Returns an exception equivalent to <paramref name="ex"/> but with any verbatim secret
/// scrubbed from its message. When nothing changes, the original exception is returned
/// unchanged; otherwise a new exception of the same concrete runtime type is built and the
/// original reply/status context is preserved. The secret-bearing original is deliberately
/// <b>not</b> chained as the inner exception — doing so would let its unredacted message
/// re-surface through <see cref="Exception.ToString"/> (which logging frameworks call). The
/// original's own inner cause (a transport error, never the request payload) is carried
/// forward instead.
/// </summary>
/// <param name="ex">The exception to redact.</param>
/// <param name="secrets">The secret values to remove from the message.</param>
/// <returns>The redacted exception, or the original when no change was needed.</returns>
internal static MxGatewayException Redacted(MxGatewayException ex, params string?[] secrets)
{
ArgumentNullException.ThrowIfNull(ex);
string redacted = Redact(ex.Message, secrets);
bool messageChanged = !string.Equals(redacted, ex.Message, StringComparison.Ordinal);
Exception? cause = ex.InnerException;
// MxAccessException derives its structured fields from the raw reply, so scrubbing must
// clone and redact that reply — the message alone changing is not enough, because the reply
// can carry the echoed secret even when the message does not.
if (ex is MxAccessException access)
{
if (!messageChanged && !ReplyContainsSecret(access.Reply, secrets))
{
return ex;
}
return new MxAccessException(redacted, RedactReply(access.Reply, secrets), cause);
}
// Other subtypes carry the secret through ProtocolStatus.Message and Statuses[].DiagnosticText.
if (!messageChanged
&& !ContainsSecret(ex.ProtocolStatus?.Message, secrets)
&& !StatusesContainSecret(ex.Statuses, secrets))
{
return ex;
}
ProtocolStatus? status = RedactStatus(ex.ProtocolStatus, secrets);
IReadOnlyList<MxStatusProxy> statuses = RedactStatuses(ex.Statuses, secrets);
return ex switch
{
MxGatewaySessionException => new MxGatewaySessionException(
redacted, ex.SessionId, ex.CorrelationId, status, ex.HResultCode, statuses, cause),
MxGatewayWorkerException => new MxGatewayWorkerException(
redacted, ex.SessionId, ex.CorrelationId, status, ex.HResultCode, statuses, cause),
MxGatewayAuthenticationException => new MxGatewayAuthenticationException(
redacted, ex.SessionId, ex.CorrelationId, status, ex.HResultCode, statuses, cause),
MxGatewayAuthorizationException => new MxGatewayAuthorizationException(
redacted, ex.SessionId, ex.CorrelationId, status, ex.HResultCode, statuses, cause),
MxGatewayMalformedReplyException => new MxGatewayMalformedReplyException(
redacted, ex.SessionId, ex.CorrelationId, status, ex.HResultCode, statuses, cause),
MxGatewayCommandException => new MxGatewayCommandException(
redacted, ex.SessionId, ex.CorrelationId, status, ex.HResultCode, statuses, cause),
_ => new MxGatewayException(redacted, cause),
};
}
private static bool ContainsSecret(string? text, string?[] secrets)
{
if (string.IsNullOrEmpty(text) || secrets is null)
{
return false;
}
foreach (string? secret in secrets)
{
if (!string.IsNullOrWhiteSpace(secret) && text.Contains(secret, StringComparison.Ordinal))
{
return true;
}
}
return false;
}
private static bool StatusesContainSecret(IReadOnlyList<MxStatusProxy> statuses, string?[] secrets)
{
if (statuses is null)
{
return false;
}
foreach (MxStatusProxy status in statuses)
{
if (ContainsSecret(status.DiagnosticText, secrets))
{
return true;
}
}
return false;
}
private static bool ReplyContainsSecret(MxCommandReply reply, string?[] secrets)
{
if (reply is null)
{
return false;
}
return ContainsSecret(reply.ProtocolStatus?.Message, secrets)
|| ContainsSecret(reply.DiagnosticMessage, secrets)
|| StatusesContainSecret(reply.Statuses, secrets);
}
}
@@ -945,7 +945,7 @@ public sealed class MxGatewaySession : IAsyncDisposable
cancellationToken)
.ConfigureAwait(false);
reply.EnsureProtocolSuccess().EnsureMxAccessSuccess();
return reply.AddBufferedItem?.ItemHandle ?? reply.ReturnValue.Int32Value;
return ResolveInt32Result(reply.AddBufferedItem?.ItemHandle, reply, "AddBufferedItem");
}
/// <summary>
@@ -1141,7 +1141,14 @@ public sealed class MxGatewaySession : IAsyncDisposable
verifierUserId,
cancellationToken)
.ConfigureAwait(false);
reply.EnsureProtocolSuccess().EnsureMxAccessSuccess();
try
{
reply.EnsureProtocolSuccess().EnsureMxAccessSuccess();
}
catch (MxGatewayException ex)
{
throw MxGatewaySecretRedaction.Redacted(ex, ExtractSecretString(value));
}
}
/// <summary>
@@ -1215,7 +1222,14 @@ public sealed class MxGatewaySession : IAsyncDisposable
verifierUserId,
cancellationToken)
.ConfigureAwait(false);
reply.EnsureProtocolSuccess().EnsureMxAccessSuccess();
try
{
reply.EnsureProtocolSuccess().EnsureMxAccessSuccess();
}
catch (MxGatewayException ex)
{
throw MxGatewaySecretRedaction.Redacted(ex, ExtractSecretString(value));
}
}
/// <summary>
@@ -1285,8 +1299,15 @@ public sealed class MxGatewaySession : IAsyncDisposable
verifyUserPassword,
cancellationToken)
.ConfigureAwait(false);
reply.EnsureProtocolSuccess().EnsureMxAccessSuccess();
return reply.AuthenticateUser?.UserId ?? reply.ReturnValue.Int32Value;
try
{
reply.EnsureProtocolSuccess().EnsureMxAccessSuccess();
return ResolveInt32Result(reply.AuthenticateUser?.UserId, reply, "AuthenticateUser");
}
catch (MxGatewayException ex)
{
throw MxGatewaySecretRedaction.Redacted(ex, verifyUserPassword);
}
}
/// <summary>
@@ -1337,7 +1358,7 @@ public sealed class MxGatewaySession : IAsyncDisposable
MxCommandReply reply = await ArchestraUserToIdRawAsync(serverHandle, userIdGuid, cancellationToken)
.ConfigureAwait(false);
reply.EnsureProtocolSuccess().EnsureMxAccessSuccess();
return reply.ArchestraUserToId?.UserId ?? reply.ReturnValue.Int32Value;
return ResolveInt32Result(reply.ArchestraUserToId?.UserId, reply, "ArchestrAUserToId");
}
/// <summary>
@@ -1367,6 +1388,51 @@ public sealed class MxGatewaySession : IAsyncDisposable
cancellationToken);
}
/// <summary>
/// Resolves the int32 result of an OK command reply: the typed payload value when present,
/// otherwise an int32 <c>return_value</c> when the reply carries one. A reply that provides
/// neither is malformed and surfaces as <see cref="MxGatewayMalformedReplyException"/>
/// rather than the historical <see cref="NullReferenceException"/>.
/// </summary>
/// <param name="typedValue">The typed payload value, or <see langword="null"/> when absent.</param>
/// <param name="reply">The OK command reply.</param>
/// <param name="operation">The MXAccess operation name, for the diagnostic message.</param>
/// <returns>The resolved int32 result.</returns>
private static int ResolveInt32Result(int? typedValue, MxCommandReply reply, string operation)
{
if (typedValue.HasValue)
{
return typedValue.Value;
}
if (reply.ReturnValue is not null
&& reply.ReturnValue.KindCase == MxValue.KindOneofCase.Int32Value)
{
return reply.ReturnValue.Int32Value;
}
throw new MxGatewayMalformedReplyException(
$"{operation} returned a malformed reply: OK reply carried neither the typed payload nor an int32 return_value",
reply.SessionId,
reply.CorrelationId,
reply.ProtocolStatus,
reply.HasHresult ? reply.Hresult : null,
reply.Statuses.ToArray());
}
/// <summary>
/// Extracts the raw string form of a credential-bearing <see cref="MxValue"/> for redaction,
/// or <see langword="null"/> when the value does not carry a string.
/// </summary>
/// <param name="value">The value written by a secured write.</param>
/// <returns>The string payload, or <see langword="null"/>.</returns>
private static string? ExtractSecretString(MxValue value)
{
return value.KindCase == MxValue.KindOneofCase.StringValue
? value.StringValue
: null;
}
/// <summary>
/// Invokes an MXAccess command on this session.
/// </summary>
@@ -5,15 +5,18 @@ namespace ZB.MOM.WW.MxGateway.Client;
/// <summary>Extension methods for MxStatusProxy values.</summary>
public static class MxStatusProxyExtensions
{
/// <summary>Returns whether the status indicates success (success flag set and category is Ok).</summary>
/// <summary>
/// Returns whether the status indicates success, which the wire contract defines as
/// <see cref="MxStatusCategory.Ok"/>. The raw <c>Success</c> member is a verbatim COM
/// diagnostic, not a boolean, so it never participates in the verdict.
/// </summary>
/// <param name="status">The status to check.</param>
/// <returns><see langword="true"/> if the status indicates success; otherwise <see langword="false"/>.</returns>
public static bool IsSuccess(this MxStatusProxy status)
{
ArgumentNullException.ThrowIfNull(status);
return status.Success != 0
&& status.Category is MxStatusCategory.Ok;
return status.Category is MxStatusCategory.Ok;
}
/// <summary>Returns a formatted summary of the status for diagnostic output.</summary>
@@ -27,6 +30,6 @@ public static class MxStatusProxyExtensions
? "no diagnostic text"
: status.DiagnosticText;
return $"{status.Category} by {status.DetectedBy}; detail={status.Detail}; {diagnosticText}";
return $"success={status.Success}; {status.Category} by {status.DetectedBy}; detail={status.Detail}; {diagnosticText}";
}
}
@@ -19,7 +19,7 @@
<PropertyGroup>
<IsPackable>true</IsPackable>
<PackageId>ZB.MOM.WW.MxGateway.Client</PackageId>
<Version>0.1.2</Version>
<Version>0.2.1</Version>
<Description>.NET 10 gRPC client for the MxAccessGateway service. Provides typed wrappers, retry, and a lazy-browse walker over the Galaxy Repository hierarchy.</Description>
<PackageReadmeFile>README.md</PackageReadmeFile>
<!-- Only the shipped library generates XML docs (matching src/Contracts). The Cli and
+39 -5
View File
@@ -94,6 +94,12 @@ goroutine cleanup. Raw protobuf messages remain available through the
`errors.As` for `GatewayError`, `CommandError`, and `MxAccessError`; command
errors preserve the raw reply.
`EnsureMxAccessSuccess` follows COM semantics: only a **negative** HRESULT is a
failure, so positive success codes such as `S_FALSE` (1) pass. `StatusSucceeded`
judges each `MXSTATUS_PROXY` entry by its category — an entry fails when
`Category` is not `MX_STATUS_CATEGORY_OK`, and the raw `Success` member is a
diagnostic that never decides the verdict. A nil entry is success.
### Reconnect-replay gap
Each `EventResult` carries exactly one of `Event`, `ReplayGap`, or `Err`. When
@@ -139,6 +145,28 @@ call returns a `StreamAlarmsClient`; cancel its context to terminate the
stream. All three pass straight through to the gateway's central alarm
monitor.
`ActiveAlarmSnapshot.GetFromTruncatedSnapshot()` reports that the record came
from a provider fetch which hit the per-fetch cap: the snapshot set may omit
active alarms, and the gateway suspended its absence-implies-cleared inference
for that poll. Treat the set as possibly incomplete rather than reconciling
deletions from it. It is set-level degraded status, not a comment on the
record's own fidelity, and is distinct from `Degraded` (the subtag fallback
provider).
`StreamAlarms` also carries that completeness verdict at feed level, as a frame
whose `GetSnapshotStatus()` is non-nil and whose `GetTruncated()` is true while
the monitor's cached set derives from a truncated fetch. One arrives at stream
open (after the `GetProviderStatus()` frame, before the cached
`GetActiveAlarm()` frames) so a late joiner learns the current verdict, then one
on every verdict change — including the clearing frame sent when the gateway's
alarm monitor restarts and drops a truncated verdict. Track it if you need set
completeness on a live feed without polling `QueryActiveAlarms`.
The CLI's text rows always spell out `degraded=false` and `truncated=false`.
Its protobuf-JSON mode retains the standard proto3 omission rule, so those
false-valued payloads appear as `{"providerStatus":{}}` and
`{"snapshotStatus":{}}`; absent means false.
## Write Semantics And Common Pitfalls
These are MXAccess parity behaviors that surprise new callers. The gateway
@@ -177,7 +205,11 @@ parity holds: a `WriteSecured` issued without a matching prior `AuthenticateUser
and supervisory advise fails natively, and that failure is surfaced unchanged
rather than pre-empted. The CLI exposes `authenticate-user` (credential via
`-password-env`, default `MXGATEWAY_VERIFY_PASSWORD`, or `-password`) and
`write-secured`.
`write-secured`. The credential is required: a missing or empty resolved value is
a usage error naming the flag and the variable, so the CLI fails before dialing
instead of authenticating with an empty password. `MXGATEWAY_VERIFY_PASSWORD` is
the canonical variable across all five client CLIs — see
[Cross-Language Smoke Matrix](../../docs/CrossLanguageSmokeMatrix.md).
### Array writes replace the whole array
@@ -412,7 +444,7 @@ go run ./cmd/mxgw-go write -session-id <id> -server-handle 1 -item-handle 1 -typ
go run ./cmd/mxgw-go write-bulk -session-id <id> -server-handle 1 -item-handles 1,2 -values 10,20 -type int32 -plaintext -json
go run ./cmd/mxgw-go read-bulk -session-id <id> -item-handles 1,2 -plaintext -json
go run ./cmd/mxgw-go stream-events -session-id <id> -plaintext -json
go run ./cmd/mxgw-go stream-alarms -plaintext -json
go run ./cmd/mxgw-go stream-alarms -plaintext -limit 4 -json
go run ./cmd/mxgw-go smoke -item Area001.Tag.Value -plaintext -json
go run ./cmd/mxgw-go galaxy-test-connection -plaintext -json
go run ./cmd/mxgw-go galaxy-last-deploy -plaintext -json
@@ -461,7 +493,7 @@ go run ./cmd/mxgw-go smoke -endpoint $env:MXGATEWAY_ENDPOINT -plaintext -api-key
The module is resolved directly from the git repo — no package registry:
````bash
go get gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go@v0.1.1
go get gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go@v0.2.0
````
Then import:
@@ -484,11 +516,13 @@ Go modules in monorepo subdirectories use prefixed tags. To tag a release
from this repo:
````bash
pwsh scripts/tag-go-module.ps1 -Version v0.1.1 -Push
pwsh scripts/tag-go-module.ps1 -Version v0.2.0 -Push
````
The script validates semver, refuses to tag with uncommitted tracked
changes, creates an annotated tag `clients/go/v0.1.1`, and (with `-Push`)
changes, verifies `clients/go/mxgateway/version.go`'s `ClientVersion`
matches the requested tag version (failing the tag otherwise — CLI-21/CLI-39),
creates an annotated tag `clients/go/v0.2.0`, and (with `-Push`)
pushes it to origin.
## Related Documentation
+69 -5
View File
@@ -57,6 +57,25 @@ type commandReplyOutput struct {
Reply json.RawMessage `json:"reply"`
}
// replayGapRow is the JSON row stream-events emits for a reconnect-replay gap:
// {"replayGap":{"requestedAfterSequence":N,"oldestAvailableSequence":N}}.
//
// The cursors are typed by hand rather than marshalled with protojson on
// purpose. The proto3 JSON mapping renders 64-bit integers as JSON *strings*
// ("7"), but the Rust and Python CLIs emit JSON *numbers* (7) for this row —
// routing through protojson would silently make Go the odd one out and break
// the cross-language smoke matrix's row comparison. encoding/json renders
// uint64 as a number, which is the canonical rendering here.
type replayGapRow struct {
ReplayGap replayGapCursors `json:"replayGap"`
}
// replayGapCursors is the nested cursor object of replayGapRow.
type replayGapCursors struct {
RequestedAfterSequence uint64 `json:"requestedAfterSequence"`
OldestAvailableSequence uint64 `json:"oldestAvailableSequence"`
}
func main() {
if err := runWithIO(context.Background(), os.Args[1:], os.Stdout, os.Stderr); err != nil {
fmt.Fprintln(os.Stderr, err)
@@ -427,6 +446,11 @@ func runWriteSecured(ctx context.Context, args []string, stdout, stderr io.Write
return writeCommandOutput(stdout, *jsonOutput, "write-secured", options, reply, err)
}
// defaultVerifyPasswordEnv is the canonical CLI credential environment variable,
// shared by every official client CLI (CLI-45) so one exported variable drives
// the same operator workflow in all five languages.
const defaultVerifyPasswordEnv = "MXGATEWAY_VERIFY_PASSWORD"
func runAuthenticateUser(ctx context.Context, args []string, stdout, stderr io.Writer) error {
flags := flag.NewFlagSet("authenticate-user", flag.ContinueOnError)
flags.SetOutput(stderr)
@@ -439,7 +463,7 @@ func runAuthenticateUser(ctx context.Context, args []string, stdout, stderr io.W
// prefer the environment variable so it stays out of shell history and the
// process table. The -password flag remains for non-interactive scripting.
password := flags.String("password", "", "verify-user password (prefer -password-env)")
passwordEnv := flags.String("password-env", "MXGATEWAY_VERIFY_PASSWORD", "environment variable containing the verify-user password")
passwordEnv := flags.String("password-env", defaultVerifyPasswordEnv, "environment variable containing the verify-user password")
if err := flags.Parse(args); err != nil {
return err
@@ -452,8 +476,18 @@ func runAuthenticateUser(ctx context.Context, args []string, stdout, stderr io.W
}
resolvedPassword := *password
if resolvedPassword == "" && *passwordEnv != "" {
resolvedPassword = os.Getenv(*passwordEnv)
envName := *passwordEnv
if envName == "" {
envName = defaultVerifyPasswordEnv
}
if resolvedPassword == "" {
resolvedPassword = os.Getenv(envName)
}
// Fail fast rather than dialing: an unset or empty variable must not become a
// real MXAccess authentication attempt with an empty credential. The message
// names only the flag and the variable — never the resolved value.
if resolvedPassword == "" {
return fmt.Errorf("a password is required via -password or the %s environment variable", envName)
}
client, options, err := dialForCommand(ctx, common)
@@ -970,7 +1004,31 @@ func runStreamEvents(ctx context.Context, args []string, stdout, stderr io.Write
if result.Err != nil {
return result.Err
}
if *jsonOutput {
// A reconnect-replay gap is a typed signal, not an event: the library
// clears Event on it, so formatting Event here would print a meaningless
// zero row and discard the resume cursors the operator needs. Render it
// as its own row (matching the Rust CLI) and count it toward -limit like
// any other emitted row.
if result.IsReplayGap() {
if *jsonOutput {
row, err := json.Marshal(replayGapRow{
ReplayGap: replayGapCursors{
RequestedAfterSequence: result.ReplayGap.GetRequestedAfterSequence(),
OldestAvailableSequence: result.ReplayGap.GetOldestAvailableSequence(),
},
})
if err != nil {
return err
}
fmt.Fprintln(stdout, string(row))
} else {
fmt.Fprintf(
stdout,
"REPLAY_GAP requested_after=%d oldest_available=%d\n",
result.ReplayGap.GetRequestedAfterSequence(),
result.ReplayGap.GetOldestAvailableSequence())
}
} else if *jsonOutput {
fmt.Fprintln(stdout, string(mustMarshalProto(result.Event)))
} else {
fmt.Fprintf(stdout, "%d %s\n", result.Event.GetWorkerSequence(), result.Event.GetFamily())
@@ -1039,7 +1097,8 @@ func runStreamAlarms(ctx context.Context, args []string, stdout, stderr io.Write
// formatAlarmFeedMessage renders one AlarmFeedMessage in the CLI's plain-text
// output style, distinguishing the active-alarm snapshot, snapshot-complete
// sentinel, and transition cases of the message's payload oneof.
// sentinel, transition, provider-status, and snapshot-status cases of the
// message's payload oneof.
func formatAlarmFeedMessage(message *mxgateway.AlarmFeedMessage) string {
switch {
case message.GetActiveAlarm() != nil:
@@ -1050,6 +1109,11 @@ func formatAlarmFeedMessage(message *mxgateway.AlarmFeedMessage) string {
case message.GetTransition() != nil:
transition := message.GetTransition()
return fmt.Sprintf("transition %s kind=%s severity=%d", transition.GetAlarmFullReference(), transition.GetTransitionKind(), transition.GetSeverity())
case message.GetProviderStatus() != nil:
status := message.GetProviderStatus()
return fmt.Sprintf("provider-status mode=%s degraded=%t reason=%q", status.GetMode(), status.GetDegraded(), status.GetReason())
case message.GetSnapshotStatus() != nil:
return fmt.Sprintf("snapshot-status truncated=%t", message.GetSnapshotStatus().GetTruncated())
default:
return "unknown"
}
+242
View File
@@ -283,6 +283,68 @@ type pingFakeGateway struct {
pb.UnimplementedMxAccessGatewayServer
}
// alarmStatusFakeGateway emits both status arms with their proto3 boolean
// defaults so the CLI contract covers values that protobuf JSON omits.
type alarmStatusFakeGateway struct {
pb.UnimplementedMxAccessGatewayServer
}
func (g *alarmStatusFakeGateway) StreamAlarms(
_ *pb.StreamAlarmsRequest,
stream grpc.ServerStreamingServer[pb.AlarmFeedMessage],
) error {
if err := stream.Send(&pb.AlarmFeedMessage{
Payload: &pb.AlarmFeedMessage_ProviderStatus{
ProviderStatus: &pb.AlarmProviderStatus{Degraded: false},
},
}); err != nil {
return err
}
return stream.Send(&pb.AlarmFeedMessage{
Payload: &pb.AlarmFeedMessage_SnapshotStatus{
SnapshotStatus: &pb.AlarmSnapshotStatus{Truncated: false},
},
})
}
func startAlarmStatusGateway(t *testing.T) string {
t.Helper()
listener, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatalf("listen: %v", err)
}
server := grpc.NewServer()
pb.RegisterMxAccessGatewayServer(server, &alarmStatusFakeGateway{})
go func() { _ = server.Serve(listener) }()
t.Cleanup(func() {
server.Stop()
_ = listener.Close()
})
return listener.Addr().String()
}
func TestRunStreamAlarmsPrintsStatusArmsInTextAndProtobufJSON(t *testing.T) {
endpoint := startAlarmStatusGateway(t)
baseArgs := []string{"stream-alarms", "-endpoint", endpoint, "-plaintext", "-api-key", "test", "-limit", "2"}
var textOut, textErr bytes.Buffer
if err := runWithIO(t.Context(), baseArgs, &textOut, &textErr); err != nil {
t.Fatalf("text stream-alarms error = %v; stderr = %s", err, textErr.String())
}
if got, want := strings.TrimSpace(textOut.String()), "provider-status mode=ALARM_PROVIDER_MODE_UNSPECIFIED degraded=false reason=\"\"\nsnapshot-status truncated=false"; got != want {
t.Fatalf("text stream-alarms output = %q, want %q", got, want)
}
var jsonOut, jsonErr bytes.Buffer
jsonArgs := append(append([]string{}, baseArgs...), "-json")
if err := runWithIO(t.Context(), jsonArgs, &jsonOut, &jsonErr); err != nil {
t.Fatalf("JSON stream-alarms error = %v; stderr = %s", err, jsonErr.String())
}
if got, want := strings.TrimSpace(jsonOut.String()), "{\"providerStatus\":{}}\n{\"snapshotStatus\":{}}"; got != want {
t.Fatalf("JSON stream-alarms output = %q, want protobuf JSON %q", got, want)
}
}
func (g *pingFakeGateway) Invoke(_ context.Context, req *pb.MxCommandRequest) (*pb.MxCommandReply, error) {
echo := "pong:" + req.GetCommand().GetPing().GetMessage()
return &pb.MxCommandReply{
@@ -598,6 +660,69 @@ func TestRunAuthenticateUserRequiresVerifyUser(t *testing.T) {
}
}
// TestRunAuthenticateUserRejectsEmptyPassword pins the CLI-45 fail-fast contract:
// an unresolved credential must abort before dialing rather than authenticating
// with an empty password, and the usage error must name both -password and the
// canonical environment variable without echoing any value.
func TestRunAuthenticateUserRejectsEmptyPassword(t *testing.T) {
t.Setenv("MXGATEWAY_VERIFY_PASSWORD", "")
var stdout, stderr bytes.Buffer
err := runWithIO(t.Context(), []string{
"authenticate-user",
"-session-id", "s1",
"-verify-user", "operator",
"-plaintext",
"-api-key", "test",
}, &stdout, &stderr)
if err == nil {
t.Fatal("authenticate-user without a credential must fail before dialing")
}
if !strings.Contains(err.Error(), "-password") {
t.Fatalf("error must name the -password flag: %v", err)
}
if !strings.Contains(err.Error(), "MXGATEWAY_VERIFY_PASSWORD") {
t.Fatalf("error must name the canonical environment variable: %v", err)
}
}
// TestRunAuthenticateUserReadsPasswordFromCanonicalEnv pins that the default
// -password-env is MXGATEWAY_VERIFY_PASSWORD: with it set the credential guard
// passes and the command proceeds past it to the dial, which fails against an
// unused port under a short context — proving the guard was cleared without
// needing a live gateway.
func TestRunAuthenticateUserReadsPasswordFromCanonicalEnv(t *testing.T) {
t.Setenv("MXGATEWAY_VERIFY_PASSWORD", "env-sourced-credential")
ctx, cancel := context.WithTimeout(t.Context(), 200*time.Millisecond)
defer cancel()
var stdout, stderr bytes.Buffer
err := runWithIO(ctx, []string{
"authenticate-user",
"-session-id", "s1",
"-verify-user", "operator",
"-endpoint", "127.0.0.1:1",
"-plaintext",
"-api-key", "test",
"-call-timeout", "1s",
}, &stdout, &stderr)
if err == nil {
t.Fatal("expected the dial/RPC to fail against an unused port")
}
if strings.Contains(err.Error(), "flag provided but not defined") {
t.Fatalf("test invoked an unknown flag, so it never reached the guard: %v", err)
}
if strings.Contains(err.Error(), "a password is required") {
t.Fatalf("credential guard must be satisfied from %s: %v", "MXGATEWAY_VERIFY_PASSWORD", err)
}
if strings.Contains(err.Error(), "env-sourced-credential") ||
strings.Contains(stdout.String(), "env-sourced-credential") ||
strings.Contains(stderr.String(), "env-sourced-credential") {
t.Fatal("the resolved credential must never be echoed")
}
}
// TestRunWriteBulkVariantRejectsMismatchedHandlesAndValues pins the len-mismatch
// guard so a write-bulk with unequal item-handles / values counts fails fast
// before any dial.
@@ -617,3 +742,120 @@ func TestRunWriteBulkVariantRejectsMismatchedHandlesAndValues(t *testing.T) {
t.Fatalf("write-bulk mismatched handles/values error = %v", err)
}
}
// replayGapFakeGateway streams the gateway's reconnect-replay sentinel (an MxEvent
// carrying replay_gap, family UNSPECIFIED, body unset) followed by one normal data
// event — exactly what a resume whose cursor predates the retained replay ring sees.
type replayGapFakeGateway struct {
pb.UnimplementedMxAccessGatewayServer
}
func (g *replayGapFakeGateway) StreamEvents(
req *pb.StreamEventsRequest,
stream grpc.ServerStreamingServer[pb.MxEvent],
) error {
sentinel := &pb.MxEvent{
SessionId: req.GetSessionId(),
Family: pb.MxEventFamily_MX_EVENT_FAMILY_UNSPECIFIED,
ReplayGap: &pb.ReplayGap{
RequestedAfterSequence: 7,
OldestAvailableSequence: 42,
},
}
if err := stream.Send(sentinel); err != nil {
return err
}
return stream.Send(&pb.MxEvent{
SessionId: req.GetSessionId(),
Family: pb.MxEventFamily_MX_EVENT_FAMILY_ON_DATA_CHANGE,
WorkerSequence: 43,
})
}
func startReplayGapGateway(t *testing.T) string {
t.Helper()
listener, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatalf("listen: %v", err)
}
server := grpc.NewServer()
pb.RegisterMxAccessGatewayServer(server, &replayGapFakeGateway{})
go func() { _ = server.Serve(listener) }()
t.Cleanup(func() {
server.Stop()
_ = listener.Close()
})
return listener.Addr().String()
}
// TestRunStreamEventsPrintsReplayGap pins CLI-36: the CLI must render the typed
// ReplayGap signal in both output modes instead of formatting the library's
// cleared Event field (which printed "0 MX_EVENT_FAMILY_UNSPECIFIED" in text mode
// and an empty object in JSON mode, destroying the resume cursors).
func TestRunStreamEventsPrintsReplayGap(t *testing.T) {
endpoint := startReplayGapGateway(t)
baseArgs := []string{
"stream-events",
"-endpoint", endpoint,
"-plaintext",
"-api-key", "test",
"-session-id", "gap-session",
"-after-worker-sequence", "7",
"-limit", "2",
}
var stdout, stderr bytes.Buffer
if err := runWithIO(t.Context(), baseArgs, &stdout, &stderr); err != nil {
t.Fatalf("runWithIO() error = %v; stderr = %s", err, stderr.String())
}
text := stdout.String()
if !strings.Contains(text, "REPLAY_GAP requested_after=7 oldest_available=42") {
t.Fatalf("stream-events text output missing typed gap row: %q", text)
}
if strings.Contains(text, "0 MX_EVENT_FAMILY_UNSPECIFIED") {
t.Fatalf("stream-events text output destroyed the gap into a zero row: %q", text)
}
if !strings.Contains(text, "43 MX_EVENT_FAMILY_ON_DATA_CHANGE") {
t.Fatalf("stream-events text output dropped the normal event: %q", text)
}
stdout.Reset()
stderr.Reset()
if err := runWithIO(t.Context(), append(baseArgs, "-json"), &stdout, &stderr); err != nil {
t.Fatalf("runWithIO(-json) error = %v; stderr = %s", err, stderr.String())
}
lines := strings.Split(strings.TrimSpace(stdout.String()), "\n")
if len(lines) != 2 {
t.Fatalf("stream-events -json emitted %d rows, want 2: %q", len(lines), stdout.String())
}
// The cursors must decode as JSON numbers, not the strings the proto3 JSON
// mapping would produce for 64-bit fields: the Rust and Python CLIs emit
// numbers, and the cross-language matrix compares these rows across clients.
var gapRow struct {
ReplayGap *struct {
RequestedAfterSequence uint64 `json:"requestedAfterSequence"`
OldestAvailableSequence uint64 `json:"oldestAvailableSequence"`
} `json:"replayGap"`
}
if err := json.Unmarshal([]byte(lines[0]), &gapRow); err != nil {
t.Fatalf("parse gap row: %v\nrow: %s", err, lines[0])
}
if gapRow.ReplayGap == nil {
t.Fatalf("stream-events -json first row is not a replayGap row: %s", lines[0])
}
if gapRow.ReplayGap.RequestedAfterSequence != 7 || gapRow.ReplayGap.OldestAvailableSequence != 42 {
t.Fatalf("stream-events -json gap cursors = %+v, want 7/42", *gapRow.ReplayGap)
}
// Belt and braces on the value type: a protojson-rendered `"7"` already
// fails the decode above (encoding/json rejects a JSON string for an
// untagged uint64 field), but assert the raw bytes so a regression names
// the real problem instead of surfacing as an opaque unmarshal error.
if !strings.Contains(lines[0], `"requestedAfterSequence":7`) ||
!strings.Contains(lines[0], `"oldestAvailableSequence":42`) {
t.Fatalf("stream-events -json gap cursors must be JSON numbers, got: %s", lines[0])
}
}
+40 -2
View File
@@ -1,11 +1,32 @@
Set-StrictMode -Version Latest
$ErrorActionPreference = 'Stop'
# Pinned generator baseline. The committed Go bindings stamp these plugin versions in their
# headers (protoc-gen-go v1.36.11 / protoc-gen-go-grpc v1.6.2). Plugin-version drift rewrites
# those header stamps, so a regeneration on an off-pin machine would churn the tree and make
# check-codegen Check 4 false-fail (or mask real drift under churn). Assert the exact versions
# so a regen is deterministic. protoc itself is warn-only (source_code_info is normalized out of
# the committed bindings), matching publish-client-proto-inputs.ps1. On Windows a plugin reports
# its argv[0] name, so the banner carries a trailing `.exe` (e.g. "protoc-gen-go.exe v1.36.11");
# Get-NormalizedToolVersion strips that suffix before the pin compare so Check 4 runs the same on
# Windows as everywhere else.
$PinnedProtocGenGoVersion = 'protoc-gen-go v1.36.11'
$PinnedProtocGenGoGrpcVersion = 'protoc-gen-go-grpc 1.6.2'
$PinnedProtocVersion = 'libprotoc 34.1'
$repoRoot = Resolve-Path (Join-Path $PSScriptRoot '..\..')
$protoRoot = Join-Path $repoRoot 'src\ZB.MOM.WW.MxGateway.Contracts\Protos'
$outputRoot = Join-Path $PSScriptRoot 'internal\generated'
$modulePath = 'gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go/internal/generated'
function Get-NormalizedToolVersion {
# On Windows a plugin reports its argv[0] name, so the banner carries an `.exe`
# suffix ("protoc-gen-go.exe v1.36.11"). Strip it so the pin compare is
# host-independent; the version part must still match exactly.
param([string]$RawBanner)
return ($RawBanner -replace '\.exe(?=\s)', '')
}
function Resolve-Tool {
# Resolve a codegen tool from PATH first (portable), then the documented Windows fallbacks,
# instead of the previous hard-coded per-machine paths. See docs/ToolchainLinks.md.
@@ -36,8 +57,25 @@ $wingetProtoc = if ($env:LOCALAPPDATA) {
$goBin = if ($env:USERPROFILE) { Join-Path $env:USERPROFILE 'go\bin' } elseif ($env:HOME) { Join-Path $env:HOME 'go/bin' } else { $null }
$protoc = Resolve-Tool -Names @('protoc', 'protoc.exe') -FallbackPaths @($wingetProtoc)
$protocGenGo = Resolve-Tool -Names @('protoc-gen-go', 'protoc-gen-go.exe') -FallbackPaths @((if ($goBin) { Join-Path $goBin 'protoc-gen-go.exe' }), (if ($goBin) { Join-Path $goBin 'protoc-gen-go' }))
$protocGenGoGrpc = Resolve-Tool -Names @('protoc-gen-go-grpc', 'protoc-gen-go-grpc.exe') -FallbackPaths @((if ($goBin) { Join-Path $goBin 'protoc-gen-go-grpc.exe' }), (if ($goBin) { Join-Path $goBin 'protoc-gen-go-grpc' }))
$protocGenGo = Resolve-Tool -Names @('protoc-gen-go', 'protoc-gen-go.exe') -FallbackPaths @(($(if ($goBin) { Join-Path $goBin 'protoc-gen-go.exe' })), ($(if ($goBin) { Join-Path $goBin 'protoc-gen-go' })))
$protocGenGoGrpc = Resolve-Tool -Names @('protoc-gen-go-grpc', 'protoc-gen-go-grpc.exe') -FallbackPaths @(($(if ($goBin) { Join-Path $goBin 'protoc-gen-go-grpc.exe' })), ($(if ($goBin) { Join-Path $goBin 'protoc-gen-go-grpc' })))
# Assert the pinned plugin versions before generating so Check 4 cannot false-fail (or mask drift)
# on an off-pin machine. protoc is warn-only.
$protocGenGoVersion = Get-NormalizedToolVersion (& $protocGenGo --version 2>&1 | Out-String).Trim()
if ($protocGenGoVersion -ne $PinnedProtocGenGoVersion) {
throw "protoc-gen-go reports '$protocGenGoVersion', but regeneration is pinned to '$PinnedProtocGenGoVersion'. " +
"Install the pin: go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11"
}
$protocGenGoGrpcVersion = Get-NormalizedToolVersion (& $protocGenGoGrpc --version 2>&1 | Out-String).Trim()
if ($protocGenGoGrpcVersion -ne $PinnedProtocGenGoGrpcVersion) {
throw "protoc-gen-go-grpc reports '$protocGenGoGrpcVersion', but regeneration is pinned to '$PinnedProtocGenGoGrpcVersion'. " +
"Install the pin: go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.6.2"
}
$protocVersion = Get-NormalizedToolVersion (& $protoc --version 2>&1 | Out-String).Trim()
if ($protocVersion -ne $PinnedProtocVersion) {
Write-Warning "protoc reports '$protocVersion', pin is '$PinnedProtocVersion'. Descriptor comments are normalized out of the committed Go bindings, so patch drift is tolerated; keep CI on the pin."
}
# protoc discovers the plugins on PATH; prepend the directories the resolved plugins live in.
$env:Path = (Split-Path $protocGenGo -Parent) + [System.IO.Path]::PathSeparator + (Split-Path $protocGenGoGrpc -Parent) + [System.IO.Path]::PathSeparator + $env:Path
File diff suppressed because it is too large Load Diff
@@ -431,8 +431,17 @@ type GatewayHello struct {
SupportedProtocolVersion uint32 `protobuf:"varint,1,opt,name=supported_protocol_version,json=supportedProtocolVersion,proto3" json:"supported_protocol_version,omitempty"`
Nonce string `protobuf:"bytes,2,opt,name=nonce,proto3" json:"nonce,omitempty"`
GatewayVersion string `protobuf:"bytes,3,opt,name=gateway_version,json=gatewayVersion,proto3" json:"gateway_version,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
// Maximum worker-frame payload size, in bytes, negotiated by the gateway from its
// configured pipe limit. The worker adopts this as its frame-protocol MaxMessageBytes
// instead of a hard-coded default; 0 (an older gateway that never set the field) means
// "use the worker's built-in default". Sits above the public gRPC cap by an
// envelope-overhead margin so an accepted gRPC payload always fits one worker frame.
// Every worker->gateway frame — events, heartbeats, faults, and control replies
// including DrainEvents — must serialize within this limit; reply builders truncate
// to fit rather than emit an oversized frame.
MaxFrameBytes uint32 `protobuf:"varint,4,opt,name=max_frame_bytes,json=maxFrameBytes,proto3" json:"max_frame_bytes,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *GatewayHello) Reset() {
@@ -486,6 +495,13 @@ func (x *GatewayHello) GetGatewayVersion() string {
return ""
}
func (x *GatewayHello) GetMaxFrameBytes() uint32 {
if x != nil {
return x.MaxFrameBytes
}
return 0
}
type WorkerHello struct {
state protoimpl.MessageState `protogen:"open.v1"`
ProtocolVersion uint32 `protobuf:"varint,1,opt,name=protocol_version,json=protocolVersion,proto3" json:"protocol_version,omitempty"`
@@ -1109,11 +1125,12 @@ const file_mxaccess_worker_proto_rawDesc = "" +
"\fworker_event\x18\x12 \x01(\v2\x1f.mxaccess_worker.v1.WorkerEventH\x00R\vworkerEvent\x12P\n" +
"\x10worker_heartbeat\x18\x13 \x01(\v2#.mxaccess_worker.v1.WorkerHeartbeatH\x00R\x0fworkerHeartbeat\x12D\n" +
"\fworker_fault\x18\x14 \x01(\v2\x1f.mxaccess_worker.v1.WorkerFaultH\x00R\vworkerFaultB\x06\n" +
"\x04body\"\x8b\x01\n" +
"\x04body\"\xb3\x01\n" +
"\fGatewayHello\x12<\n" +
"\x1asupported_protocol_version\x18\x01 \x01(\rR\x18supportedProtocolVersion\x12\x14\n" +
"\x05nonce\x18\x02 \x01(\tR\x05nonce\x12'\n" +
"\x0fgateway_version\x18\x03 \x01(\tR\x0egatewayVersion\"\xa1\x01\n" +
"\x0fgateway_version\x18\x03 \x01(\tR\x0egatewayVersion\x12&\n" +
"\x0fmax_frame_bytes\x18\x04 \x01(\rR\rmaxFrameBytes\"\xa1\x01\n" +
"\vWorkerHello\x12)\n" +
"\x10protocol_version\x18\x01 \x01(\rR\x0fprotocolVersion\x12\x14\n" +
"\x05nonce\x18\x02 \x01(\tR\x05nonce\x12*\n" +
+148 -9
View File
@@ -10,7 +10,9 @@ import (
pb "gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go/internal/generated"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/metadata"
"google.golang.org/grpc/status"
"google.golang.org/grpc/test/bufconn"
)
@@ -200,6 +202,136 @@ func TestEventsSlowConsumerYieldsErrSlowConsumerBeforeClose(t *testing.T) {
}
}
func TestEventsFullBufferTerminalErrorKeepsRootCause(t *testing.T) {
fake := &fakeGatewayServer{
streamStarted: make(chan struct{}),
streamDone: make(chan struct{}),
streamEventCount: eventBufferSize,
streamTerminalErr: status.Error(codes.Internal, "boom"),
}
client, cleanup := newBufconnClient(t, fake)
defer cleanup()
session := NewSessionForID(client, "session-1")
events, err := session.EventsAfter(context.Background(), 0)
if err != nil {
t.Fatalf("EventsAfter() error = %v", err)
}
<-fake.streamStarted
// Do not drain until the stream has fully ended: the server sends exactly
// eventBufferSize events (filling the data slots) and then returns a genuine
// terminal gRPC error. The client must report that error as itself, using the
// reserved slot, rather than mislabeling it as ErrSlowConsumer.
select {
case <-fake.streamDone:
case <-time.After(2 * time.Second):
t.Fatal("event stream did not stop after terminal error")
}
// streamDone fires when the server returns; the client's producer goroutine
// still needs a moment to drain the gRPC stream, fill all data slots, and
// enqueue the terminal result. Let it settle before draining so the buffer is
// genuinely full when the terminal error is processed (which is what makes the
// mislabel bug observable).
time.Sleep(250 * time.Millisecond)
var last EventResult
gotResult := false
for {
select {
case res, ok := <-events:
if !ok {
if !gotResult {
t.Fatal("events channel closed without yielding any result")
}
var gwErr *GatewayError
if !errors.As(last.Err, &gwErr) {
t.Fatalf("final event result err is %T, want *GatewayError", last.Err)
}
if code := status.Code(last.Err); code != codes.Internal {
t.Fatalf("final event result gRPC code = %s, want %s", code, codes.Internal)
}
if errors.Is(last.Err, ErrSlowConsumer) {
t.Fatalf("final event result err = %v, must not be mislabeled as ErrSlowConsumer", last.Err)
}
return
}
last = res
gotResult = true
case <-time.After(2 * time.Second):
t.Fatal("events channel did not close after terminal error")
}
}
}
// TestSubscribeEventsFullBufferDeliversTerminalError is the CLI-44 regression for
// the never-drop Subscribe path. SubscribeEvents/SubscribeEventsAfter use
// cancelWhenResultBufferFull=false, so ordinary sends are blocking and uncapped and
// can fill every slot in the results channel — including the reserved terminal slot.
// A genuine terminal Recv error must still be delivered as the final result, never
// silently dropped. The server sends eventBufferSize+eventBufferReservedSlots events
// (filling every slot) and then returns a genuine gRPC error; with an unconditional
// non-blocking terminal send the error is dropped, so this fails red until the send
// path blocks for the never-drop mode.
func TestSubscribeEventsFullBufferDeliversTerminalError(t *testing.T) {
fake := &fakeGatewayServer{
streamStarted: make(chan struct{}),
streamDone: make(chan struct{}),
streamEventCount: eventBufferSize + eventBufferReservedSlots,
streamTerminalErr: status.Error(codes.Internal, "boom"),
}
client, cleanup := newBufconnClient(t, fake)
defer cleanup()
session := NewSessionForID(client, "session-1")
subscription, err := session.SubscribeEvents(context.Background())
if err != nil {
t.Fatalf("SubscribeEvents() error = %v", err)
}
defer subscription.Close()
<-fake.streamStarted
// Wait for the server to finish sending every event and return the terminal
// error, so the producer goroutine has filled every buffered slot before the
// terminal result is processed. That is what makes the dropped-terminal bug
// observable: with the buffer full, an unconditional non-blocking send discards
// the terminal error.
select {
case <-fake.streamDone:
case <-time.After(2 * time.Second):
t.Fatal("event stream did not stop after terminal error")
}
time.Sleep(250 * time.Millisecond)
// Drain fully. Every data event, then the terminal gRPC error as the final
// result, must arrive; the channel must not close without yielding it.
events := subscription.Events()
var last EventResult
gotResult := false
for {
select {
case res, ok := <-events:
if !ok {
if !gotResult {
t.Fatal("events channel closed without yielding any result")
}
var gwErr *GatewayError
if !errors.As(last.Err, &gwErr) {
t.Fatalf("final event result err is %T (%v), want the terminal *GatewayError; it was dropped", last.Err, last.Err)
}
if code := status.Code(last.Err); code != codes.Internal {
t.Fatalf("final event result gRPC code = %s, want %s", code, codes.Internal)
}
return
}
last = res
gotResult = true
case <-time.After(2 * time.Second):
t.Fatal("events channel did not close after terminal error")
}
}
}
func TestEventsSurfacesReplayGapSentinelAsTypedSignal(t *testing.T) {
fake := &fakeGatewayServer{
streamStarted: make(chan struct{}),
@@ -694,15 +826,16 @@ func newBufconnClient(t *testing.T, fake *fakeGatewayServer) (*Client, func()) {
type fakeGatewayServer struct {
pb.UnimplementedMxAccessGatewayServer
openReply *pb.OpenSessionReply
openAuth string
streamAuth string
streamStarted chan struct{}
streamDone chan struct{}
streamEventCount int
streamReplayGap *pb.ReplayGap
invokeReply *pb.MxCommandReply
invokeRequest *pb.MxCommandRequest
openReply *pb.OpenSessionReply
openAuth string
streamAuth string
streamStarted chan struct{}
streamDone chan struct{}
streamEventCount int
streamReplayGap *pb.ReplayGap
streamTerminalErr error
invokeReply *pb.MxCommandReply
invokeRequest *pb.MxCommandRequest
}
func (s *fakeGatewayServer) OpenSession(ctx context.Context, req *pb.OpenSessionRequest) (*pb.OpenSessionReply, error) {
@@ -772,6 +905,12 @@ func (s *fakeGatewayServer) StreamEvents(req *pb.StreamEventsRequest, stream grp
return err
}
}
if s.streamTerminalErr != nil {
// Return a genuine terminal stream error immediately after sending the
// events, without waiting on the client to cancel. This exercises the
// Recv-error path while the client's result buffer is still full.
return s.streamTerminalErr
}
<-stream.Context().Done()
return io.EOF
}
@@ -0,0 +1,197 @@
package mxgateway
import (
"context"
"errors"
"os"
"path/filepath"
"strings"
"testing"
pb "gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go/internal/generated"
"google.golang.org/protobuf/encoding/protojson"
)
// loadCommandReplyFixture parses a shared command-reply fixture into an
// MxCommandReply so the Go client can be driven through the same wire shapes the
// other language clients exercise.
func loadCommandReplyFixture(t *testing.T, name string) *pb.MxCommandReply {
t.Helper()
path := filepath.Join("..", "..", "proto", "fixtures", "behavior", "command-replies", name)
data, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read fixture %s: %v", name, err)
}
var reply pb.MxCommandReply
if err := protojson.Unmarshal(data, &reply); err != nil {
t.Fatalf("parse fixture %s: %v", name, err)
}
return &reply
}
func TestAuthenticateUserMissingPayloadReturnsMalformedReplyError(t *testing.T) {
fake := &fakeGatewayServer{
invokeReply: loadCommandReplyFixture(t, "authenticate-user.missing-payload.reply.json"),
}
client, cleanup := newBufconnClient(t, fake)
defer cleanup()
session := NewSessionForID(client, "session-1")
_, err := session.AuthenticateUser(context.Background(), 12, "operator", "secret")
var malformed *MalformedReplyError
if !errors.As(err, &malformed) {
t.Fatalf("AuthenticateUser() error = %v (%T), want *MalformedReplyError", err, err)
}
if malformed.Op != "authenticate user" {
t.Fatalf("MalformedReplyError.Op = %q, want %q", malformed.Op, "authenticate user")
}
}
func TestAuthenticateUserReturnValueOnlyUsesInt32ReturnValue(t *testing.T) {
fake := &fakeGatewayServer{
invokeReply: loadCommandReplyFixture(t, "authenticate-user.return-value-only.reply.json"),
}
client, cleanup := newBufconnClient(t, fake)
defer cleanup()
session := NewSessionForID(client, "session-1")
userID, err := session.AuthenticateUser(context.Background(), 12, "operator", "secret")
if err != nil {
t.Fatalf("AuthenticateUser() error = %v", err)
}
if userID != 7 {
t.Fatalf("AuthenticateUser() = %d, want 7", userID)
}
}
// AddBufferedItem shares the prefer-payload / int32-return-value / malformed
// fallback code path; cover both branches for one of the siblings.
func TestAddBufferedItemFallbackHonoursReturnValueAndReportsMalformed(t *testing.T) {
t.Run("return-value-only", func(t *testing.T) {
fake := &fakeGatewayServer{
invokeReply: loadCommandReplyFixture(t, "authenticate-user.return-value-only.reply.json"),
}
client, cleanup := newBufconnClient(t, fake)
defer cleanup()
session := NewSessionForID(client, "session-1")
itemHandle, err := session.AddBufferedItem(context.Background(), 12, "Area001.Pump001.Speed", "runtime")
if err != nil {
t.Fatalf("AddBufferedItem() error = %v", err)
}
if itemHandle != 7 {
t.Fatalf("AddBufferedItem() = %d, want 7", itemHandle)
}
})
t.Run("missing-payload", func(t *testing.T) {
fake := &fakeGatewayServer{
invokeReply: loadCommandReplyFixture(t, "authenticate-user.missing-payload.reply.json"),
}
client, cleanup := newBufconnClient(t, fake)
defer cleanup()
session := NewSessionForID(client, "session-1")
_, err := session.AddBufferedItem(context.Background(), 12, "Area001.Pump001.Speed", "runtime")
var malformed *MalformedReplyError
if !errors.As(err, &malformed) {
t.Fatalf("AddBufferedItem() error = %v (%T), want *MalformedReplyError", err, err)
}
if malformed.Op != "add buffered item" {
t.Fatalf("MalformedReplyError.Op = %q, want %q", malformed.Op, "add buffered item")
}
})
}
// TestAuthenticateUserScrubsEchoedCredentialFromError is the CLI-40 regression:
// a gateway diagnostic that echoes the raw credential back must never reach the
// caller's surfaced error text.
func TestAuthenticateUserScrubsEchoedCredentialFromError(t *testing.T) {
const credential = "sup3rSecretVerify9f3a2b"
fake := &fakeGatewayServer{
invokeReply: loadCommandReplyFixture(t, "authenticate-user.echoed-credential.reply.json"),
}
client, cleanup := newBufconnClient(t, fake)
defer cleanup()
session := NewSessionForID(client, "session-1")
_, err := session.AuthenticateUser(context.Background(), 12, "operator", credential)
if err == nil {
t.Fatal("AuthenticateUser() error = nil, want an MXAccess failure")
}
message := err.Error()
if strings.Contains(message, credential) {
t.Fatalf("surfaced error leaked the credential: %q", message)
}
if !strings.Contains(message, "<redacted>") {
t.Fatalf("surfaced error missing redaction marker: %q", message)
}
}
// TestAuthenticateUserScrubsEchoedCredentialFromStructuredReply is the CLI-40
// follow-up: redacting only the rendered Error() string is not enough. The typed
// *MxAccessError still carries the raw command reply, whose ProtocolStatus.Message,
// DiagnosticMessage, and Statuses[].DiagnosticText echo the credential verbatim. A
// logger dumping structured fields would reintroduce the leak, so the reply the
// typed error carries must be a scrubbed clone. Both the OK+negative-HRESULT and the
// MXACCESS_FAILURE fixtures route to *MxAccessError (via EnsureProtocolSuccess), so
// both must be scrubbed identically.
func TestAuthenticateUserScrubsEchoedCredentialFromStructuredReply(t *testing.T) {
const credential = "sup3rSecretVerify9f3a2b"
fixtures := []string{
"authenticate-user.echoed-credential.reply.json",
"authenticate-user.echoed-credential-mxaccess-failure.reply.json",
}
for _, fixture := range fixtures {
t.Run(fixture, func(t *testing.T) {
fake := &fakeGatewayServer{
invokeReply: loadCommandReplyFixture(t, fixture),
}
client, cleanup := newBufconnClient(t, fake)
defer cleanup()
session := NewSessionForID(client, "session-1")
_, err := session.AuthenticateUser(context.Background(), 12, "operator", credential)
if err == nil {
t.Fatal("AuthenticateUser() error = nil, want an MXAccess failure")
}
var mxErr *MxAccessError
if !errors.As(err, &mxErr) {
t.Fatalf("AuthenticateUser() error = %v (%T), want *MxAccessError", err, err)
}
reply := mxErr.Reply
if reply == nil {
t.Fatal("MxAccessError.Reply is nil, want the scrubbed command reply")
}
if got := reply.GetProtocolStatus().GetMessage(); strings.Contains(got, credential) {
t.Fatalf("MxAccessError.Reply.ProtocolStatus.Message leaked the credential: %q", got)
}
if got := reply.GetDiagnosticMessage(); strings.Contains(got, credential) {
t.Fatalf("MxAccessError.Reply.DiagnosticMessage leaked the credential: %q", got)
}
for i, status := range reply.GetStatuses() {
if got := status.GetDiagnosticText(); strings.Contains(got, credential) {
t.Fatalf("MxAccessError.Reply.Statuses[%d].DiagnosticText leaked the credential: %q", i, got)
}
}
// The wrapped CommandError's status/reply must be scrubbed too.
if mxErr.Command != nil {
if got := mxErr.Command.Status.GetMessage(); strings.Contains(got, credential) {
t.Fatalf("MxAccessError.Command.Status.Message leaked the credential: %q", got)
}
if cmdReply := mxErr.Command.Reply; cmdReply != nil {
if got := cmdReply.GetDiagnosticMessage(); strings.Contains(got, credential) {
t.Fatalf("MxAccessError.Command.Reply.DiagnosticMessage leaked the credential: %q", got)
}
}
}
if got := err.Error(); strings.Contains(got, credential) {
t.Fatalf("rendered error leaked the credential: %q", got)
}
})
}
}
+5 -4
View File
@@ -51,8 +51,9 @@ func TestStatusConversionFixtures(t *testing.T) {
var fixture struct {
Cases []struct {
ID string `json:"id"`
Status json.RawMessage `json:"status"`
ID string `json:"id"`
WantSuccess bool `json:"wantSuccess"`
Status json.RawMessage `json:"status"`
} `json:"cases"`
}
if err := json.Unmarshal(data, &fixture); err != nil {
@@ -65,8 +66,8 @@ func TestStatusConversionFixtures(t *testing.T) {
if err := protojson.Unmarshal(tc.Status, &status); err != nil {
t.Fatalf("parse status: %v", err)
}
if got, want := StatusSucceeded(&status), status.GetSuccess() != 0; got != want {
t.Fatalf("StatusSucceeded() = %v, want %v", got, want)
if got := StatusSucceeded(&status); got != tc.WantSuccess {
t.Fatalf("StatusSucceeded() = %v, want %v", got, tc.WantSuccess)
}
})
}
+119 -7
View File
@@ -6,6 +6,7 @@ import (
"strings"
pb "gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go/internal/generated"
"google.golang.org/protobuf/proto"
)
// redactedSecretMarker is the placeholder substituted for credential material in
@@ -49,20 +50,108 @@ func (e *secretRedactingError) Unwrap() error {
return e.err
}
// redactSecrets wraps err so any occurrence of a non-empty secret in the surfaced
// message is redacted, while errors.As / errors.Is still reach the wrapped typed
// error. It returns nil unchanged and skips wrapping when no non-empty secret is
// supplied, so non-secret-bearing calls keep their original error verbatim.
// scrubReplyStrings returns a clone of reply with every non-empty secret replaced
// by redactedSecretMarker in the free-text fields a gateway diagnostic could echo a
// credential into: ProtocolStatus.Message, DiagnosticMessage, and each
// Statuses[].DiagnosticText. It clones with proto.Clone so the caller's original
// reply is never mutated. A nil reply, or an empty/whitespace-only secret set, is a
// no-op (nil in, nil out; a clone otherwise).
func scrubReplyStrings(reply *pb.MxCommandReply, secrets []string) *pb.MxCommandReply {
if reply == nil {
return nil
}
clone, ok := proto.Clone(reply).(*pb.MxCommandReply)
if !ok {
return reply
}
for _, secret := range secrets {
if secret == "" {
continue
}
if clone.GetProtocolStatus() != nil {
clone.ProtocolStatus.Message = strings.ReplaceAll(clone.GetProtocolStatus().GetMessage(), secret, redactedSecretMarker)
}
clone.DiagnosticMessage = strings.ReplaceAll(clone.GetDiagnosticMessage(), secret, redactedSecretMarker)
for _, status := range clone.GetStatuses() {
status.DiagnosticText = strings.ReplaceAll(status.GetDiagnosticText(), secret, redactedSecretMarker)
}
}
return clone
}
// scrubProtocolStatusMessage returns a clone of status with every non-empty secret
// redacted from its Message, leaving the original untouched.
func scrubProtocolStatusMessage(status *ProtocolStatus, secrets []string) *ProtocolStatus {
if status == nil {
return nil
}
clone, ok := proto.Clone(status).(*ProtocolStatus)
if !ok {
return status
}
for _, secret := range secrets {
if secret != "" {
clone.Message = strings.ReplaceAll(clone.GetMessage(), secret, redactedSecretMarker)
}
}
return clone
}
// redactSecrets scrubs a non-empty secret set from the error it surfaces. When the
// wrapped error is a typed *MxAccessError or *CommandError it is rebuilt carrying
// scrubbed clones of its reply and protocol status, so a caller logging the typed
// error's structured fields cannot reintroduce the credential the rendered message
// hides. The rebuilt (or original, for other error types) value is then wrapped in
// secretRedactingError as a belt-and-suspenders scrub of any remaining rendered
// text. errors.As / errors.Is still reach the typed error through the wrapper. It
// returns nil unchanged and skips all work when no non-empty secret is supplied, so
// non-secret-bearing calls keep their original error verbatim.
func redactSecrets(err error, secrets ...string) error {
if err == nil {
return nil
}
hasSecret := false
for _, secret := range secrets {
if secret != "" {
return &secretRedactingError{err: err, secrets: secrets}
hasSecret = true
break
}
}
return err
if !hasSecret {
return err
}
rebuilt := rebuildScrubbedError(err, secrets)
return &secretRedactingError{err: rebuilt, secrets: secrets}
}
// rebuildScrubbedError rebuilds the typed error carrying scrubbed clones of any
// command reply / protocol status it holds, so credential text never survives in the
// error's structured fields. Non-reply-bearing error types are returned unchanged.
func rebuildScrubbedError(err error, secrets []string) error {
switch typed := err.(type) {
case *MxAccessError:
return &MxAccessError{
Command: scrubCommandError(typed.Command, secrets),
Reply: scrubReplyStrings(typed.Reply, secrets),
}
case *CommandError:
return scrubCommandError(typed, secrets)
default:
return err
}
}
// scrubCommandError rebuilds a CommandError with a scrubbed Status and Reply.
func scrubCommandError(cmd *CommandError, secrets []string) *CommandError {
if cmd == nil {
return nil
}
return &CommandError{
Op: cmd.Op,
Status: scrubProtocolStatusMessage(cmd.Status, secrets),
Reply: scrubReplyStrings(cmd.Reply, secrets),
}
}
// ErrSlowConsumer is the terminal error sent on the Events/EventsAfter
@@ -72,6 +161,25 @@ func redactSecrets(err error, secrets ...string) error {
// dropping events. Match it with errors.Is.
var ErrSlowConsumer = errors.New("mxgateway: event consumer fell behind; stream terminated")
// MalformedReplyError reports an OK command reply that carried neither the
// typed payload the operation expected nor a usable int32 return_value, so the
// client cannot produce a result. It gives every affected helper one uniform,
// inspectable failure instead of silently returning a zero value.
type MalformedReplyError struct {
// Op names the operation whose reply was malformed.
Op string
// Detail explains what the reply was missing.
Detail string
}
// Error returns the formatted malformed-reply message.
func (e *MalformedReplyError) Error() string {
if e == nil {
return ""
}
return fmt.Sprintf("mxgateway: %s returned a malformed reply: %s", e.Op, e.Detail)
}
// GatewayError wraps transport-level gRPC failures.
type GatewayError struct {
// Op names the operation that failed (for example "dial" or "invoke").
@@ -180,11 +288,15 @@ func EnsureProtocolSuccess(op string, status *ProtocolStatus, reply *MxCommandRe
// EnsureMxAccessSuccess returns a typed MxAccessError for failing HRESULTs or
// MXSTATUS_PROXY entries.
//
// Following COM semantics, only a negative HRESULT is a failure — positive
// success codes such as S_FALSE (1) pass. Status entries are judged by
// StatusSucceeded, which branches on the authoritative category.
func EnsureMxAccessSuccess(op string, reply *MxCommandReply) error {
if reply == nil {
return nil
}
if reply.Hresult != nil && reply.GetHresult() != 0 {
if reply.Hresult != nil && reply.GetHresult() < 0 {
return &MxAccessError{Reply: reply}
}
for _, status := range reply.GetStatuses() {
@@ -0,0 +1,115 @@
package mxgateway
import (
"errors"
"strings"
"testing"
pb "gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go/internal/generated"
)
// TestScrubReplyStringsRedactsEveryOccurrence covers the multi-occurrence case:
// one secret appearing across ProtocolStatus.Message, DiagnosticMessage, and every
// Statuses[].DiagnosticText must be fully redacted with no residue.
func TestScrubReplyStringsRedactsEveryOccurrence(t *testing.T) {
const secret = "hunter2"
reply := &pb.MxCommandReply{
ProtocolStatus: &pb.ProtocolStatus{Message: "rejected hunter2 then hunter2 again"},
DiagnosticMessage: "echoed hunter2 back",
Statuses: []*pb.MxStatusProxy{
{DiagnosticText: "first hunter2"},
{DiagnosticText: "second hunter2 and hunter2"},
},
}
scrubbed := scrubReplyStrings(reply, []string{secret})
if strings.Contains(scrubbed.GetProtocolStatus().GetMessage(), secret) {
t.Fatalf("ProtocolStatus.Message still contains the secret: %q", scrubbed.GetProtocolStatus().GetMessage())
}
if strings.Contains(scrubbed.GetDiagnosticMessage(), secret) {
t.Fatalf("DiagnosticMessage still contains the secret: %q", scrubbed.GetDiagnosticMessage())
}
for i, status := range scrubbed.GetStatuses() {
if strings.Contains(status.GetDiagnosticText(), secret) {
t.Fatalf("Statuses[%d].DiagnosticText still contains the secret: %q", i, status.GetDiagnosticText())
}
}
if !strings.Contains(scrubbed.GetProtocolStatus().GetMessage(), redactedSecretMarker) {
t.Fatalf("ProtocolStatus.Message missing redaction marker: %q", scrubbed.GetProtocolStatus().GetMessage())
}
// The original reply must be untouched (scrubReplyStrings clones).
if !strings.Contains(reply.GetDiagnosticMessage(), secret) {
t.Fatal("scrubReplyStrings mutated the original reply instead of cloning it")
}
}
// TestScrubReplyStringsRedactsOverlappingSecrets covers two secrets where one is a
// substring of the other: both must be fully redacted, with no partial leak of the
// longer secret's non-shared remainder.
func TestScrubReplyStringsRedactsOverlappingSecrets(t *testing.T) {
const shortSecret = "pass"
const longSecret = "password123"
reply := &pb.MxCommandReply{
DiagnosticMessage: "value was password123 and also pass",
}
scrubbed := scrubReplyStrings(reply, []string{longSecret, shortSecret})
got := scrubbed.GetDiagnosticMessage()
if strings.Contains(got, shortSecret) {
t.Fatalf("scrubbed message still contains a secret substring %q: %q", shortSecret, got)
}
if strings.Contains(got, longSecret) {
t.Fatalf("scrubbed message still contains %q: %q", longSecret, got)
}
// "123" is the longer secret's remainder past the shared "pass" prefix; it must
// not survive as a partial leak.
if strings.Contains(got, "123") {
t.Fatalf("scrubbed message leaked the longer secret's remainder: %q", got)
}
}
// TestRedactSecretsEmptyOrNilLeavesErrorUnchanged confirms the no-secret paths keep
// the original typed error verbatim (no wrapping, no scrubbed clone).
func TestRedactSecretsEmptyOrNilLeavesErrorUnchanged(t *testing.T) {
base := &MxAccessError{Reply: &pb.MxCommandReply{DiagnosticMessage: "boom"}}
if got := redactSecrets(base); got != error(base) {
t.Fatalf("redactSecrets with no secrets = %v, want the original error unchanged", got)
}
if got := redactSecrets(base, ""); got != error(base) {
t.Fatalf("redactSecrets with only an empty secret = %v, want the original error unchanged", got)
}
if got := redactSecrets(nil, "secret"); got != nil {
t.Fatalf("redactSecrets(nil, ...) = %v, want nil", got)
}
}
// TestRedactSecretsRebuildsTypedCommandError confirms a *CommandError (non-MXAccess
// path) is rebuilt with a scrubbed Status and Reply, and errors.As still reaches it.
func TestRedactSecretsRebuildsTypedCommandError(t *testing.T) {
const secret = "topSecretValue"
base := &CommandError{
Op: "write secured",
Status: &pb.ProtocolStatus{Message: "rejected topSecretValue"},
Reply: &pb.MxCommandReply{DiagnosticMessage: "echoed topSecretValue"},
}
redacted := redactSecrets(base, secret)
var cmdErr *CommandError
if !errors.As(redacted, &cmdErr) {
t.Fatalf("redactSecrets result %T does not unwrap to *CommandError", redacted)
}
if strings.Contains(cmdErr.Status.GetMessage(), secret) {
t.Fatalf("CommandError.Status.Message leaked the secret: %q", cmdErr.Status.GetMessage())
}
if strings.Contains(cmdErr.Reply.GetDiagnosticMessage(), secret) {
t.Fatalf("CommandError.Reply.DiagnosticMessage leaked the secret: %q", cmdErr.Reply.GetDiagnosticMessage())
}
if strings.Contains(redacted.Error(), secret) {
t.Fatalf("rendered error leaked the secret: %q", redacted.Error())
}
}
@@ -48,6 +48,89 @@ func TestGeneratedGoldenFixturesParse(t *testing.T) {
}
}
// TestCommandReplyValidationFixtures locks the shared reply-validation rules to
// the behavior fixtures: a status entry fails iff its category is not OK (the
// raw success member is diagnostics only), and an HRESULT fails iff it is
// present and negative (S_FALSE and other positive COM success codes pass).
func TestCommandReplyValidationFixtures(t *testing.T) {
tests := []struct {
fixture string
wantFailure bool
}{
{fixture: "register.ok.reply.json", wantFailure: false},
{fixture: "write.mxaccess-failure.reply.json", wantFailure: true},
{fixture: "write.status-category-error-success-set.reply.json", wantFailure: true},
{fixture: "write.status-category-ok-success-zero.reply.json", wantFailure: false},
{fixture: "write.hresult-s-false.reply.json", wantFailure: false},
{fixture: "write.hresult-e-fail.reply.json", wantFailure: true},
}
for _, tt := range tests {
t.Run(tt.fixture, func(t *testing.T) {
data, err := os.ReadFile(filepath.Join(
"..", "..", "proto", "fixtures", "behavior", "command-replies", tt.fixture))
if err != nil {
t.Fatalf("read fixture: %v", err)
}
var reply pb.MxCommandReply
if err := protojson.Unmarshal(data, &reply); err != nil {
t.Fatalf("parse fixture: %v", err)
}
err = EnsureMxAccessSuccess("invoke", &reply)
if got := err != nil; got != tt.wantFailure {
t.Fatalf("EnsureMxAccessSuccess() failed = %v (err %v), want %v", got, err, tt.wantFailure)
}
})
}
}
// TestStatusSucceededBranchesOnCategory pins the per-entry rule directly,
// including the two edges the fixtures cannot express: a nil entry is success
// and a present entry with an unspecified category is a failure.
func TestStatusSucceededBranchesOnCategory(t *testing.T) {
tests := []struct {
name string
status *MxStatusProxy
want bool
}{
{name: "nil entry", status: nil, want: true},
{
name: "ok category with zero success",
status: &pb.MxStatusProxy{
Success: 0,
Category: pb.MxStatusCategory_MX_STATUS_CATEGORY_OK,
},
want: true,
},
{
name: "error category with success set",
status: &pb.MxStatusProxy{
Success: 1,
Category: pb.MxStatusCategory_MX_STATUS_CATEGORY_COMMUNICATION_ERROR,
},
want: false,
},
{
name: "unspecified category with success set",
status: &pb.MxStatusProxy{
Success: 1,
Category: pb.MxStatusCategory_MX_STATUS_CATEGORY_UNSPECIFIED,
},
want: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := StatusSucceeded(tt.status); got != tt.want {
t.Fatalf("StatusSucceeded() = %v, want %v", got, tt.want)
}
})
}
}
func TestOpenSessionFixtureProtocolVersions(t *testing.T) {
data, err := os.ReadFile(filepath.Join("..", "..", "proto", "fixtures", "golden", "open-session-reply.ok.json"))
if err != nil {
+64 -14
View File
@@ -812,7 +812,13 @@ func (s *Session) AuthenticateUser(ctx context.Context, serverHandle int32, veri
if reply.GetAuthenticateUser() != nil {
return reply.GetAuthenticateUser().GetUserId(), nil
}
return reply.GetReturnValue().GetInt32Value(), nil
if x, ok := reply.GetReturnValue().GetKind().(*pb.MxValue_Int32Value); ok {
return x.Int32Value, nil
}
return 0, &MalformedReplyError{
Op: "authenticate user",
Detail: "reply carried neither an AuthenticateUser payload nor an int32 return_value",
}
}
// AuthenticateUserRaw invokes MXAccess AuthenticateUser and returns the raw
@@ -847,7 +853,13 @@ func (s *Session) ArchestrAUserToId(ctx context.Context, serverHandle int32, use
if reply.GetArchestraUserToId() != nil {
return reply.GetArchestraUserToId().GetUserId(), nil
}
return reply.GetReturnValue().GetInt32Value(), nil
if x, ok := reply.GetReturnValue().GetKind().(*pb.MxValue_Int32Value); ok {
return x.Int32Value, nil
}
return 0, &MalformedReplyError{
Op: "archestra user to id",
Detail: "reply carried neither an ArchestrAUserToId payload nor an int32 return_value",
}
}
// ArchestrAUserToIdRaw invokes MXAccess ArchestrAUserToId and returns the raw reply.
@@ -876,7 +888,13 @@ func (s *Session) AddBufferedItem(ctx context.Context, serverHandle int32, itemD
if reply.GetAddBufferedItem() != nil {
return reply.GetAddBufferedItem().GetItemHandle(), nil
}
return reply.GetReturnValue().GetInt32Value(), nil
if x, ok := reply.GetReturnValue().GetKind().(*pb.MxValue_Int32Value); ok {
return x.Int32Value, nil
}
return 0, &MalformedReplyError{
Op: "add buffered item",
Detail: "reply carried neither an AddBufferedItem payload nor an int32 return_value",
}
}
// AddBufferedItemRaw invokes MXAccess AddBufferedItem and returns the raw reply.
@@ -981,10 +999,12 @@ func stringSecrets(values ...*MxValue) []string {
// context cancellation stops Recv, or a terminal error is sent.
//
// The returned channel is buffered. If the consumer falls behind and the buffer
// overflows, the stream is terminated and a final EventResult carrying a
// GatewayError that wraps ErrSlowConsumer is delivered before the channel
// closes. Callers must match it with errors.Is(res.Err, ErrSlowConsumer) to
// distinguish a slow-consumer drop from a graceful server end. Use
// overflows with data, the stream is terminated and a final EventResult carrying
// a GatewayError that wraps ErrSlowConsumer is delivered before the channel
// closes; match it with errors.Is(res.Err, ErrSlowConsumer) to distinguish a
// slow-consumer drop from a graceful server end. A genuine stream error is
// reported as itself even under overflow — it is never relabeled as
// ErrSlowConsumer, so the underlying gRPC status stays inspectable. Use
// SubscribeEvents for a blocking, backpressured stream that never drops.
func (s *Session) Events(ctx context.Context) (<-chan EventResult, error) {
return s.EventsAfter(ctx, 0)
@@ -994,7 +1014,9 @@ func (s *Session) Events(ctx context.Context) (<-chan EventResult, error) {
//
// Like Events, the returned channel is buffered and terminates with a final
// EventResult wrapping ErrSlowConsumer (matchable via errors.Is) if the consumer
// falls behind and the buffer overflows, rather than silently closing.
// falls behind and the buffer overflows with data, rather than silently closing.
// A genuine stream error is reported as itself even under overflow, never
// relabeled as ErrSlowConsumer.
func (s *Session) EventsAfter(ctx context.Context, afterWorkerSequence uint64) (<-chan EventResult, error) {
subscription, err := s.subscribeEventsAfter(ctx, afterWorkerSequence, true)
if err != nil {
@@ -1048,12 +1070,11 @@ func (s *Session) subscribeEventsAfter(ctx context.Context, afterWorkerSequence
if err == io.EOF || status.Code(err) == codes.Canceled || streamCtx.Err() != nil {
return
}
sendEventResult(
streamCtx,
results,
EventResult{Err: &GatewayError{Op: "stream events", Err: err}},
cancelWhenResultBufferFull,
cancel)
// A genuine terminal stream error must be reported as itself, even
// when the data slots are full. Routing it through sendEventResult
// would let the overflow branch substitute ErrSlowConsumer and lose
// the real gRPC status, so send it directly, bypassing that branch.
sendTerminalEventResult(streamCtx, results, EventResult{Err: &GatewayError{Op: "stream events", Err: err}}, cancelWhenResultBufferFull)
return
}
}()
@@ -1072,6 +1093,35 @@ func ensureBulkSize(name string, length int) error {
return nil
}
// sendTerminalEventResult enqueues a terminal EventResult, bypassing
// sendEventResult's overflow branch so a genuine stream error is reported verbatim
// rather than relabeled as ErrSlowConsumer. How it sends depends on the mode:
//
// - cancelWhenBufferFull=true (Events/EventsAfter): ordinary data sends are capped
// at eventBufferSize, leaving eventBufferReservedSlots free, so a non-blocking
// send always lands the terminal result. Because this goroutine is the sole
// producer, at most one terminal send ever races for the reserved slot, so the
// select default only fires when the reserve is already spent — never dropping a
// first terminal error.
// - cancelWhenBufferFull=false (SubscribeEvents/SubscribeEventsAfter, never-drop):
// ordinary data sends are uncapped and blocking, so every slot including the
// reserve can hold data. A non-blocking send would then hit the full buffer and
// silently drop the terminal error, breaking the never-drop contract; instead
// block until the consumer drains a slot (or the stream context is cancelled).
func sendTerminalEventResult(ctx context.Context, results chan<- EventResult, result EventResult, cancelWhenBufferFull bool) {
if cancelWhenBufferFull {
select {
case results <- result:
default:
}
return
}
select {
case results <- result:
case <-ctx.Done():
}
}
func sendEventResult(
ctx context.Context,
results chan<- EventResult,
+12 -1
View File
@@ -1,6 +1,17 @@
package mxgateway
import (
pb "gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go/internal/generated"
)
// StatusSucceeded reports whether an MXSTATUS_PROXY entry represents success.
//
// The wire contract makes Category authoritative: an entry succeeds only when
// its category is MX_STATUS_CATEGORY_OK. The Success member mirrors the raw
// 16-bit COM value verbatim for diagnostics and is not a boolean, so it takes
// no part in the verdict. A nil entry is success (nothing was reported); a
// present entry with an unspecified category is a failure, because the worker
// always maps a category and an unmapped one is not proven OK.
func StatusSucceeded(status *MxStatusProxy) bool {
return status == nil || status.GetSuccess() != 0
return status == nil || status.GetCategory() == pb.MxStatusCategory_MX_STATUS_CATEGORY_OK
}
+7 -2
View File
@@ -127,9 +127,14 @@ type (
QueryActiveAlarmsRequest = pb.QueryActiveAlarmsRequest
// StreamAlarmsRequest is the gateway StreamAlarms request message.
StreamAlarmsRequest = pb.StreamAlarmsRequest
// AlarmFeedMessage is one message on the StreamAlarms feed — an
// active-alarm snapshot row, a snapshot-complete sentinel, or a transition.
// AlarmFeedMessage is one message on the StreamAlarms feed — an active-alarm
// snapshot row, snapshot-complete sentinel, transition, provider status, or
// snapshot-completeness status.
AlarmFeedMessage = pb.AlarmFeedMessage
// AlarmProviderStatus reports the alarm provider mode and degradation state.
AlarmProviderStatus = pb.AlarmProviderStatus
// AlarmSnapshotStatus reports whether the cached alarm snapshot is truncated.
AlarmSnapshotStatus = pb.AlarmSnapshotStatus
// ActiveAlarmSnapshot is one row in a ConditionRefresh stream.
ActiveAlarmSnapshot = pb.ActiveAlarmSnapshot
// OnAlarmTransitionEvent is the body carried by alarm-transition MxEvents.
+17
View File
@@ -0,0 +1,17 @@
package mxgateway_test
import (
"testing"
"gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go/mxgateway"
)
// TestAlarmFeedStatusAliasesArePublic is a compile-time consumer contract for
// the two status payloads carried by AlarmFeedMessage.
func TestAlarmFeedStatusAliasesArePublic(t *testing.T) {
t.Helper()
var provider mxgateway.AlarmProviderStatus
var snapshot mxgateway.AlarmSnapshotStatus
_ = provider
_ = snapshot
}
+1 -1
View File
@@ -3,7 +3,7 @@ package mxgateway
const (
// ClientVersion is the released semantic version of this Go client module.
// Keep it in sync with the module tag applied by scripts/tag-go-module.ps1.
ClientVersion = "0.1.2"
ClientVersion = "0.2.1"
// GatewayProtocolVersion matches GatewayContractInfo.GatewayProtocolVersion
// in the shared .NET contracts.
+38 -5
View File
@@ -117,6 +117,29 @@ yields alarm-feed messages from the gateway's central monitor), and
`acknowledgeAlarm` (ack by full alarm reference with an optional comment and
ack target). Close the subscription to cancel the underlying gRPC stream.
`ActiveAlarmSnapshot.getFromTruncatedSnapshot()` reports that the record came
from a provider fetch which hit the per-fetch cap: the snapshot set may omit
active alarms, and the gateway suspended its absence-implies-cleared inference
for that poll. Treat the set as possibly incomplete rather than reconciling
deletions from it. It is set-level degraded status, not a comment on the
record's own fidelity, and is distinct from `getDegraded()` (the subtag
fallback provider).
`streamAlarms` also carries that completeness verdict at feed level, as a
message whose `getPayloadCase()` is `SNAPSHOT_STATUS` and whose
`getSnapshotStatus().getTruncated()` is true while the monitor's cached set
derives from a truncated fetch. One arrives at stream open (after the
`PROVIDER_STATUS` frame, before the cached `ACTIVE_ALARM` frames) so a late
joiner learns the current verdict, then one on every verdict change — including
the clearing frame sent when the gateway's alarm monitor restarts and drops a
truncated verdict. Track it if you need set completeness on a live feed without
polling `queryActiveAlarms`.
The CLI's text rows always spell out `degraded=false` and `truncated=false`.
Its protobuf-JSON mode retains the standard proto3 omission rule, so those
false-valued payloads appear as `{"providerStatus":{}}` and
`{"snapshotStatus":{}}`; absent means false.
## Write Semantics And Common Pitfalls
These are MXAccess parity behaviors that surprise new callers. The gateway
@@ -139,7 +162,12 @@ commands, so you do not need to build raw `MxCommand` messages:
All of them run the same MXAccess reply validation as the bulk helpers (protocol
status plus HRESULT/`MxStatusProxy` check) via the shared `invoke` path, so an
MXAccess COM-side failure surfaces as `MxAccessException`.
MXAccess COM-side failure surfaces as `MxAccessException`. That validation
follows COM semantics: only a **negative** HRESULT is a failure, so positive
success codes such as `S_FALSE` (1) pass. `MxStatuses.succeeded` judges each
entry by its category — an entry fails when its category is not
`MX_STATUS_CATEGORY_OK`, and the raw `success` member is a diagnostic that never
decides the verdict. A `null` entry is success.
**Secret redaction.** Credentials passed to `authenticateUser` (and the
credential-sensitive values passed to `writeSecured`/`writeSecured2`) travel
@@ -167,8 +195,13 @@ session.write(serverHandle, itemHandle, value, userId);
native failure is surfaced, not papered over.
The CLI exposes `advise-supervisory`, `write-secured`, and `authenticate-user`
(credential via `--password` or `--password-env`, never echoed), and `write` /
`write2` take `--user-id`.
(credential via `--password` or the variable named by `--password-env`, default
`MXGATEWAY_VERIFY_PASSWORD`, never echoed), and `write` / `write2` take
`--user-id`. The credential is required: a missing or empty resolved value is a
picocli usage error naming the option and the variable, so the CLI fails before
connecting instead of authenticating with an empty password.
`MXGATEWAY_VERIFY_PASSWORD` is the canonical variable across all five client
CLIs — see [Cross-Language Smoke Matrix](../../docs/CrossLanguageSmokeMatrix.md).
### Array writes replace the whole array
@@ -383,7 +416,7 @@ gradle :zb-mom-ww-mxgateway-cli:run --args="advise-supervisory --endpoint localh
gradle :zb-mom-ww-mxgateway-cli:run --args="authenticate-user --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --session-id <id> --server-handle 1 --verify-user operator --password-env MXGATEWAY_VERIFY_PASSWORD --json"
gradle :zb-mom-ww-mxgateway-cli:run --args="write-secured --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --session-id <id> --server-handle 1 --item-handle 1 --type int32 --value 123 --current-user-id 100 --verifier-user-id 100 --json"
gradle :zb-mom-ww-mxgateway-cli:run --args="stream-events --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --session-id <id> --limit 1 --json"
gradle :zb-mom-ww-mxgateway-cli:run --args="stream-alarms --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --filter-prefix Galaxy --limit 1 --json"
gradle :zb-mom-ww-mxgateway-cli:run --args="stream-alarms --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --filter-prefix Galaxy --limit 4 --json"
gradle :zb-mom-ww-mxgateway-cli:run --args="acknowledge-alarm --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --reference \"\\Galaxy\Area001.Pump001.PumpFault\" --json"
gradle :zb-mom-ww-mxgateway-cli:run --args="smoke --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --item TestObject.TestInt --json"
```
@@ -455,7 +488,7 @@ repositories {
}
dependencies {
implementation 'com.zb.mom.ww.mxgateway:zb-mom-ww-mxgateway-client:0.1.2'
implementation 'com.zb.mom.ww.mxgateway:zb-mom-ww-mxgateway-client:0.2.2'
}
````
+11 -1
View File
@@ -13,7 +13,17 @@ ext {
subprojects {
group = 'com.zb.mom.ww.mxgateway'
version = '0.2.0'
// 0.2.0 was already published to the Gitea Maven feed on 2026-06-26,
// before the CLI-37/38/40/41 conformance fixes changed the client's
// observable behavior (status.category-based validation, hresult < 0,
// exact-secret redaction, typed malformed-reply errors). Bumped to 0.2.1
// so the published coordinate matched the conformant behavior the other
// four clients shipped at 0.2.0 for the first time. See CLI-39 and the
// "Versioning" section of docs/ClientPackaging.md.
// 0.2.1 was itself published (Maven, 2026-08-07); this is the next
// release-train bump (CLI-46), one ahead of the other four clients'
// 0.2.1 because the Java coordinate has always trailed by one number.
version = '0.2.2'
pluginManager.withPlugin('java') {
java {
File diff suppressed because it is too large Load Diff
@@ -3797,6 +3797,9 @@ public final class MxaccessWorker extends com.google.protobuf.GeneratedFile {
* instead of a hard-coded default; 0 (an older gateway that never set the field) means
* "use the worker's built-in default". Sits above the public gRPC cap by an
* envelope-overhead margin so an accepted gRPC payload always fits one worker frame.
* Every worker-&gt;gateway frame — events, heartbeats, faults, and control replies
* including DrainEvents — must serialize within this limit; reply builders truncate
* to fit rather than emit an oversized frame.
* </pre>
*
* <code>uint32 max_frame_bytes = 4;</code>
@@ -3941,6 +3944,9 @@ public final class MxaccessWorker extends com.google.protobuf.GeneratedFile {
* instead of a hard-coded default; 0 (an older gateway that never set the field) means
* "use the worker's built-in default". Sits above the public gRPC cap by an
* envelope-overhead margin so an accepted gRPC payload always fits one worker frame.
* Every worker-&gt;gateway frame — events, heartbeats, faults, and control replies
* including DrainEvents — must serialize within this limit; reply builders truncate
* to fit rather than emit an oversized frame.
* </pre>
*
* <code>uint32 max_frame_bytes = 4;</code>
@@ -4499,6 +4505,9 @@ public final class MxaccessWorker extends com.google.protobuf.GeneratedFile {
* instead of a hard-coded default; 0 (an older gateway that never set the field) means
* "use the worker's built-in default". Sits above the public gRPC cap by an
* envelope-overhead margin so an accepted gRPC payload always fits one worker frame.
* Every worker-&gt;gateway frame — events, heartbeats, faults, and control replies
* including DrainEvents — must serialize within this limit; reply builders truncate
* to fit rather than emit an oversized frame.
* </pre>
*
* <code>uint32 max_frame_bytes = 4;</code>
@@ -4515,6 +4524,9 @@ public final class MxaccessWorker extends com.google.protobuf.GeneratedFile {
* instead of a hard-coded default; 0 (an older gateway that never set the field) means
* "use the worker's built-in default". Sits above the public gRPC cap by an
* envelope-overhead margin so an accepted gRPC payload always fits one worker frame.
* Every worker-&gt;gateway frame — events, heartbeats, faults, and control replies
* including DrainEvents — must serialize within this limit; reply builders truncate
* to fit rather than emit an oversized frame.
* </pre>
*
* <code>uint32 max_frame_bytes = 4;</code>
@@ -4535,6 +4547,9 @@ public final class MxaccessWorker extends com.google.protobuf.GeneratedFile {
* instead of a hard-coded default; 0 (an older gateway that never set the field) means
* "use the worker's built-in default". Sits above the public gRPC cap by an
* envelope-overhead margin so an accepted gRPC payload always fits one worker frame.
* Every worker-&gt;gateway frame — events, heartbeats, faults, and control replies
* including DrainEvents — must serialize within this limit; reply builders truncate
* to fit rather than emit an oversized frame.
* </pre>
*
* <code>uint32 max_frame_bytes = 4;</code>
@@ -5,6 +5,7 @@ import com.zb.mom.ww.mxgateway.client.DeployEventStream;
import com.zb.mom.ww.mxgateway.client.GalaxyRepositoryClient;
import com.zb.mom.ww.mxgateway.client.LazyBrowseNode;
import com.zb.mom.ww.mxgateway.client.MxEventStream;
import com.zb.mom.ww.mxgateway.client.MxEventStreamItem;
import com.zb.mom.ww.mxgateway.client.MxGatewayAlarmFeedSubscription;
import com.zb.mom.ww.mxgateway.client.MxGatewayClient;
import com.zb.mom.ww.mxgateway.client.MxGatewayClientOptions;
@@ -46,8 +47,9 @@ import java.util.function.Consumer;
import mxaccess_gateway.v1.MxaccessGateway.AcknowledgeAlarmReply;
import mxaccess_gateway.v1.MxaccessGateway.AcknowledgeAlarmRequest;
import mxaccess_gateway.v1.MxaccessGateway.ActiveAlarmSnapshot;
import mxaccess_gateway.v1.MxaccessGateway.AlarmProviderStatus;
import mxaccess_gateway.v1.MxaccessGateway.AlarmFeedMessage;
import mxaccess_gateway.v1.MxaccessGateway.AlarmProviderStatus;
import mxaccess_gateway.v1.MxaccessGateway.AlarmSnapshotStatus;
import mxaccess_gateway.v1.MxaccessGateway.BulkReadResult;
import mxaccess_gateway.v1.MxaccessGateway.BulkWriteResult;
import mxaccess_gateway.v1.MxaccessGateway.CloseSessionRequest;
@@ -59,6 +61,7 @@ import mxaccess_gateway.v1.MxaccessGateway.MxValue;
import mxaccess_gateway.v1.MxaccessGateway.OnAlarmTransitionEvent;
import mxaccess_gateway.v1.MxaccessGateway.OpenSessionRequest;
import mxaccess_gateway.v1.MxaccessGateway.PingCommand;
import mxaccess_gateway.v1.MxaccessGateway.ReplayGap;
import mxaccess_gateway.v1.MxaccessGateway.StreamAlarmsRequest;
import mxaccess_gateway.v1.MxaccessGateway.SubscribeResult;
import mxaccess_gateway.v1.MxaccessGateway.Write2BulkEntry;
@@ -70,6 +73,7 @@ import picocli.CommandLine.Command;
import picocli.CommandLine.Mixin;
import picocli.CommandLine.Model.CommandSpec;
import picocli.CommandLine.Option;
import picocli.CommandLine.ParameterException;
import picocli.CommandLine.Spec;
/**
@@ -182,6 +186,13 @@ public final class MxGatewayCli implements Callable<Integer> {
/** Sentinel written to stdout after every command result in batch mode. */
static final String BATCH_EOR = "__MXGW_BATCH_EOR__";
/**
* Canonical CLI credential environment variable, shared by every official
* client CLI (CLI-45) so one exported variable drives the same operator
* workflow in all five languages.
*/
static final String DEFAULT_VERIFY_PASSWORD_ENV = "MXGATEWAY_VERIFY_PASSWORD";
/** Sentinel queued by {@code stream-alarms} to mark a clean end of the alarm feed. */
private static final Object ALARM_FEED_END = new Object();
@@ -1139,7 +1150,7 @@ public final class MxGatewayCli implements Callable<Integer> {
@Option(
names = "--password-env",
defaultValue = "MXGATEWAY_VERIFY_PASSWORD",
defaultValue = DEFAULT_VERIFY_PASSWORD_ENV,
description = "Environment variable holding the password when --password is omitted.")
String passwordEnv;
@@ -1151,11 +1162,20 @@ public final class MxGatewayCli implements Callable<Integer> {
public Integer call() {
// Resolve the credential from the flag or environment. It flows only
// into the request; it is never written to output, logs, or errors.
String environmentName =
passwordEnv == null || passwordEnv.isBlank() ? DEFAULT_VERIFY_PASSWORD_ENV : passwordEnv;
String resolvedPassword = password == null || password.isBlank()
? System.getenv(passwordEnv)
? System.getenv(environmentName)
: password;
if (resolvedPassword == null) {
resolvedPassword = "";
if (resolvedPassword == null || resolvedPassword.isBlank()) {
// Fail fast instead of dialing: a misconfigured environment must not
// become a real MXAccess authentication attempt with an empty
// credential (CLI-45). The message names the option and the variable
// only — never the value.
throw new ParameterException(
common.spec.commandLine(),
"a password is required via --password or the " + environmentName
+ " environment variable");
}
try (MxGatewayCliClient client = clientFactory.connect(common.resolved())) {
int userId = client.session(sessionId)
@@ -1637,11 +1657,30 @@ public final class MxGatewayCli implements Callable<Integer> {
MxEventStream events = client.session(sessionId).streamEventsAfter(afterWorkerSequence)) {
int count = 0;
while (events.hasNext()) {
MxEvent event = events.next();
if (json) {
client.out().println(protoJson(event));
MxEventStreamItem item = events.nextItem();
if (item.isReplayGap()) {
// Render the ReplayGap sentinel as the typed cross-CLI row (NEXT-02,
// mirroring the Go/Python/Rust/.NET CLIs) instead of the raw sentinel
// event, whose text form printed "0 MX_EVENT_FAMILY_UNSPECIFIED".
ReplayGap gap = item.replayGap();
if (json) {
client.out().printf(
"{\"replayGap\":{\"requestedAfterSequence\":%s,\"oldestAvailableSequence\":%s}}%n",
Long.toUnsignedString(gap.getRequestedAfterSequence()),
Long.toUnsignedString(gap.getOldestAvailableSequence()));
} else {
client.out().printf(
"REPLAY_GAP requested_after=%s oldest_available=%s%n",
Long.toUnsignedString(gap.getRequestedAfterSequence()),
Long.toUnsignedString(gap.getOldestAvailableSequence()));
}
} else {
client.out().printf("%d %s%n", event.getWorkerSequence(), event.getFamily());
MxEvent event = item.event();
if (json) {
client.out().println(protoJson(event));
} else {
client.out().printf("%d %s%n", event.getWorkerSequence(), event.getFamily());
}
}
count++;
if (limit > 0 && count >= limit) {
@@ -2244,7 +2283,8 @@ public final class MxGatewayCli implements Callable<Integer> {
/**
* Renders one {@link AlarmFeedMessage} in the CLI's plain-text output
* style, distinguishing the active-alarm snapshot, snapshot-complete
* sentinel, and transition cases of the message's {@code payload} oneof.
* sentinel, transition, provider-status, and snapshot-status cases of the
* message's {@code payload} oneof.
*/
private static String formatAlarmFeedMessage(AlarmFeedMessage message) {
return switch (message.getPayloadCase()) {
@@ -2269,6 +2309,10 @@ public final class MxGatewayCli implements Callable<Integer> {
"provider-status mode=%s degraded=%b reason=%s",
status.getMode().name(), status.getDegraded(), status.getReason());
}
case SNAPSHOT_STATUS -> {
AlarmSnapshotStatus status = message.getSnapshotStatus();
yield String.format("snapshot-status truncated=%b", status.getTruncated());
}
case PAYLOAD_NOT_SET -> "unknown";
};
}
@@ -2,7 +2,10 @@ package com.zb.mom.ww.mxgateway.cli;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotEquals;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.Assumptions.assumeTrue;
import com.zb.mom.ww.mxgateway.client.MxGatewayAlarmFeedSubscription;
import com.zb.mom.ww.mxgateway.client.MxGatewayClientOptions;
@@ -24,6 +27,8 @@ import mxaccess_gateway.v1.MxaccessGateway.ActiveAlarmSnapshot;
import mxaccess_gateway.v1.MxaccessGateway.AddItemReply;
import mxaccess_gateway.v1.MxaccessGateway.AlarmConditionState;
import mxaccess_gateway.v1.MxaccessGateway.AlarmFeedMessage;
import mxaccess_gateway.v1.MxaccessGateway.AlarmProviderStatus;
import mxaccess_gateway.v1.MxaccessGateway.AlarmSnapshotStatus;
import mxaccess_gateway.v1.MxaccessGateway.AlarmTransitionKind;
import mxaccess_gateway.v1.MxaccessGateway.BulkReadResult;
import mxaccess_gateway.v1.MxaccessGateway.BulkWriteResult;
@@ -40,6 +45,7 @@ import mxaccess_gateway.v1.MxaccessGateway.OpenSessionRequest;
import mxaccess_gateway.v1.MxaccessGateway.ProtocolStatus;
import mxaccess_gateway.v1.MxaccessGateway.ProtocolStatusCode;
import mxaccess_gateway.v1.MxaccessGateway.RegisterReply;
import mxaccess_gateway.v1.MxaccessGateway.ReplayGap;
import mxaccess_gateway.v1.MxaccessGateway.SessionState;
import mxaccess_gateway.v1.MxaccessGateway.StreamAlarmsRequest;
import mxaccess_gateway.v1.MxaccessGateway.SubscribeResult;
@@ -56,7 +62,7 @@ final class MxGatewayCliTests {
assertEquals(0, run.exitCode());
assertEquals("", run.errors());
assertTrue(run.output().contains("mxgateway-java 0.2.0"));
assertTrue(run.output().contains("mxgateway-java 0.2.2"));
assertTrue(run.output().contains("gatewayProtocolVersion=3"));
assertTrue(run.output().contains("workerProtocolVersion=1"));
}
@@ -86,7 +92,7 @@ final class MxGatewayCliTests {
CliRun run = execute(new FakeClientFactory(), "version", "--json");
assertEquals(0, run.exitCode());
assertTrue(run.output().contains("\"clientVersion\":\"0.2.0\""));
assertTrue(run.output().contains("\"clientVersion\":\"0.2.2\""));
assertTrue(run.output().contains("\"gatewayProtocolVersion\":3"));
}
@@ -211,6 +217,73 @@ final class MxGatewayCliTests {
assertFalse(run.errors().contains("super-secret-pw"), "password must never be echoed to stderr");
}
/**
* CLI-45: an unresolved credential must abort with a picocli usage error
* before the CLI dials, instead of authenticating with an empty password.
* The message names the option and the variable, never a value.
*/
@Test
void authenticateUserRejectsMissingCredentialWithUsageError() {
FakeClientFactory factory = new FakeClientFactory();
CliRun run = execute(
factory,
"authenticate-user",
"--session-id", "session-cli",
"--server-handle", "3",
"--verify-user", "operator",
"--password-env", "MXGW_CLI45_ABSENT_PASSWORD_VAR",
"--json");
assertNotEquals(0, run.exitCode(), "a missing credential must fail");
assertTrue(run.errors().contains("--password"), run.errors());
assertTrue(run.errors().contains("MXGW_CLI45_ABSENT_PASSWORD_VAR"), run.errors());
assertNull(factory.client, "the CLI must not connect without a credential");
}
/**
* CLI-45: a blank {@code --password} is treated as missing — the CLI never
* sends a fabricated empty credential to the wire.
*/
@Test
void authenticateUserRejectsBlankPasswordValue() {
FakeClientFactory factory = new FakeClientFactory();
CliRun run = execute(
factory,
"authenticate-user",
"--session-id", "session-cli",
"--server-handle", "3",
"--verify-user", "operator",
"--password", "",
"--password-env", "MXGW_CLI45_ABSENT_PASSWORD_VAR",
"--json");
assertNotEquals(0, run.exitCode(), "a blank credential must fail");
assertNull(factory.client, "the CLI must not connect without a credential");
}
/**
* CLI-45: {@code --password-env} defaults to the canonical
* {@code MXGATEWAY_VERIFY_PASSWORD}, so the usage error names it when no
* explicit variable is given. Skipped if the canonical variable happens to be
* exported in the running environment (which would satisfy the credential).
*/
@Test
void authenticateUserDefaultsToCanonicalPasswordEnvName() {
assumeTrue(System.getenv(MxGatewayCli.DEFAULT_VERIFY_PASSWORD_ENV) == null);
FakeClientFactory factory = new FakeClientFactory();
CliRun run = execute(
factory,
"authenticate-user",
"--session-id", "session-cli",
"--server-handle", "3",
"--verify-user", "operator",
"--json");
assertNotEquals(0, run.exitCode());
assertTrue(run.errors().contains("MXGATEWAY_VERIFY_PASSWORD"), run.errors());
}
// ---- ping subcommand (D4) ----
@Test
@@ -614,6 +687,21 @@ final class MxGatewayCliTests {
assertTrue(run.output().contains("\"snapshotComplete\""), run.output());
}
@Test
void streamAlarmsCommandPrintsStatusArmsInTextAndProtobufJson() {
FakeClientFactory factory = new FakeClientFactory();
CliRun text = execute(factory, "stream-alarms");
List<String> textRows = text.output().lines().filter(line -> !line.isBlank()).toList();
assertEquals("provider-status mode=ALARM_PROVIDER_MODE_UNSPECIFIED degraded=false reason=", textRows.get(3));
assertEquals("snapshot-status truncated=false", textRows.get(4));
CliRun json = execute(factory, "stream-alarms", "--json");
List<String> jsonRows = json.output().lines().filter(line -> !line.isBlank()).toList();
assertEquals("{\"providerStatus\":{}}", jsonRows.get(3));
assertEquals("{\"snapshotStatus\":{}}", jsonRows.get(4));
}
@Test
void acknowledgeAlarmCommandForwardsOptionsAndPrintsReply() {
FakeClientFactory factory = new FakeClientFactory();
@@ -832,6 +920,59 @@ final class MxGatewayCliTests {
}
}
@Test
void streamEventsRendersReplayGapAsTypedRow() {
// NEXT-02: the ReplayGap sentinel must render as the typed cross-CLI
// row (numeric cursors under a replayGap key in --json, a REPLAY_GAP
// line in text mode), never as the raw sentinel event — text mode
// used to print "0 MX_EVENT_FAMILY_UNSPECIFIED".
MxEvent gap = MxEvent.newBuilder()
.setReplayGap(ReplayGap.newBuilder()
.setRequestedAfterSequence(7L)
.setOldestAvailableSequence(42L)
.build())
.build();
MxEvent dataChange = MxEvent.newBuilder()
.setFamily(MxEventFamily.MX_EVENT_FAMILY_ON_DATA_CHANGE)
.setSessionId("session-cli")
.setWorkerSequence(43L)
.build();
try (InProcessGatewayHarness harness = new InProcessGatewayHarness()) {
harness.setScriptedEvents(List.of(gap, dataChange));
CliRun jsonRun = execute(
new HarnessClientFactory(harness),
"stream-events",
"--session-id",
"session-cli",
"--json");
assertEquals(0, jsonRun.exitCode(), "errors:\n" + jsonRun.errors());
String jsonOut = jsonRun.output();
assertTrue(
jsonOut.contains(
"{\"replayGap\":{\"requestedAfterSequence\":7,\"oldestAvailableSequence\":42}}"),
jsonOut);
assertTrue(jsonOut.contains("\"family\":\"MX_EVENT_FAMILY_ON_DATA_CHANGE\""), jsonOut);
assertFalse(jsonOut.contains("MX_EVENT_FAMILY_UNSPECIFIED"), jsonOut);
}
try (InProcessGatewayHarness harness = new InProcessGatewayHarness()) {
harness.setScriptedEvents(List.of(gap, dataChange));
CliRun textRun = execute(
new HarnessClientFactory(harness),
"stream-events",
"--session-id",
"session-cli");
assertEquals(0, textRun.exitCode(), "errors:\n" + textRun.errors());
String textOut = textRun.output();
assertTrue(textOut.contains("REPLAY_GAP requested_after=7 oldest_available=42"), textOut);
assertFalse(textOut.contains("MX_EVENT_FAMILY_UNSPECIFIED"), textOut);
assertTrue(textOut.contains("43 MX_EVENT_FAMILY_ON_DATA_CHANGE"), textOut);
}
}
// ---- galaxy-discover / galaxy-watch over the in-process harness (Task 6) ----
@Test
@@ -1285,6 +1426,12 @@ final class MxGatewayCliTests {
.setTransitionKind(AlarmTransitionKind.ALARM_TRANSITION_KIND_ACKNOWLEDGE)
.setSeverity(700))
.build());
observer.onNext(AlarmFeedMessage.newBuilder()
.setProviderStatus(AlarmProviderStatus.newBuilder().setDegraded(false))
.build());
observer.onNext(AlarmFeedMessage.newBuilder()
.setSnapshotStatus(AlarmSnapshotStatus.newBuilder().setTruncated(false))
.build());
observer.onCompleted();
return new MxGatewayAlarmFeedSubscription();
}
@@ -63,29 +63,32 @@ protobuf {
// or a plugin/protobuf version bump, silently drifts the committed output. checkGeneratedClean
// fails when the regenerated tree differs from what is committed.
//
// Caveat (repo memory project_java_generated_churn): the protobuf gradle plugin also rewrites
// MxaccessGateway.java with a spurious protobuf-runtime-version delta on every build even when no
// .proto changed. CI reverts that one file (git checkout) before invoking this task; locally, do the
// same when you did not touch a .proto. See docs/GatewayTesting.md "Continuous Integration".
// The grpc/protobuf toolchain is pinned (build.gradle: grpcVersion / protobufVersion), so a
// regeneration is byte-identical to the committed single-file aggregates modulo real .proto
// changes — no spurious protobuf-runtime-version churn (IPC-24 verified this and deleted the old
// unconditional CI churn-revert step, which masked message-level drift). Regenerate and commit
// after any .proto change. See docs/GatewayTesting.md "Continuous Integration".
tasks.register('checkGeneratedClean') {
group = 'verification'
description = 'Fails if the committed generated Java tree differs from a fresh regeneration.'
dependsOn 'generateProto'
def generatedDir = 'clients/java/src/main/generated'
def repoRoot = rootProject.projectDir.parentFile.parentFile
// Project.exec was removed in Gradle 9; ProviderFactory.exec runs the git-status probe
// lazily (captured at configuration time, evaluated in doLast) and works on both the
// installed Gradle and Gradle 9.
def gitStatus = providers.exec {
workingDir(repoRoot)
commandLine('git', 'status', '--porcelain', '--', generatedDir)
ignoreExitValue = true
}
doLast {
def generatedDir = 'clients/java/src/main/generated'
def stdout = new ByteArrayOutputStream()
def result = exec {
workingDir = rootProject.projectDir.parentFile.parentFile
commandLine 'git', 'status', '--porcelain', '--', generatedDir
standardOutput = stdout
ignoreExitValue = true
}
def dirty = stdout.toString().trim()
def dirty = gitStatus.standardOutput.asText.get().trim()
if (!dirty.isEmpty()) {
throw new GradleException(
"Generated Java is stale or churned:\n${dirty}\n" +
"Regenerate and commit after a .proto change, or 'git checkout' the spurious " +
"MxaccessGateway.java protobuf-version churn when no .proto changed.")
"Generated Java is stale:\n${dirty}\n" +
"Regenerate and commit the Java client after a .proto change " +
"(gradle :zb-mom-ww-mxgateway-client:generateProto).")
}
}
}
@@ -29,4 +29,18 @@ public final class MxAccessException extends MxGatewayCommandException {
public MxAccessException(String operation, MxCommandReply reply) {
super(operation, reply == null ? null : reply.getProtocolStatus(), reply);
}
/**
* Creates a new MXAccess exception with an already-built, verbatim message.
* Used to re-surface an MXAccess failure with a redacted message while
* preserving the original protocol status and reply.
*
* @param message the exact message to surface (already formatted/redacted)
* @param protocolStatus protocol status reported by the gateway
* @param reply raw command reply containing the MXAccess failure detail
* @param cause underlying error, or {@code null}
*/
public MxAccessException(String message, ProtocolStatus protocolStatus, MxCommandReply reply, Throwable cause) {
super(message, protocolStatus, reply, cause);
}
}
@@ -9,7 +9,7 @@ package com.zb.mom.ww.mxgateway.client;
public final class MxGatewayClientVersion {
private static final int GATEWAY_PROTOCOL_VERSION = 3;
private static final int WORKER_PROTOCOL_VERSION = 1;
private static final String CLIENT_VERSION = "0.2.0";
private static final String CLIENT_VERSION = "0.2.2";
private MxGatewayClientVersion() {
}
@@ -25,6 +25,23 @@ public class MxGatewayCommandException extends MxGatewayException {
this.reply = reply;
}
/**
* Creates a new command exception with an already-built, verbatim message.
* Used to re-surface a failure with a redacted message while preserving the
* original protocol status and reply.
*
* @param message the exact message to surface (already formatted/redacted)
* @param protocolStatus protocol status returned by the gateway
* @param reply raw command reply, or {@code null} when none was produced
* @param cause underlying error, or {@code null}
*/
protected MxGatewayCommandException(
String message, ProtocolStatus protocolStatus, MxCommandReply reply, Throwable cause) {
super(message, cause);
this.protocolStatus = protocolStatus;
this.reply = reply;
}
/**
* Returns the gateway protocol status that triggered this exception.
*
@@ -47,7 +47,9 @@ final class MxGatewayErrors {
if (reply == null) {
return;
}
if (reply.hasHresult() && reply.getHresult() != 0) {
// COM semantics: only a negative HRESULT is a failure. Positive success
// codes such as S_FALSE (1) pass.
if (reply.hasHresult() && reply.getHresult() < 0) {
throw new MxAccessException(operation, reply);
}
for (var status : reply.getStatusesList()) {
@@ -0,0 +1,32 @@
package com.zb.mom.ww.mxgateway.client;
/**
* Thrown when the gateway returns a protocol-OK command reply that carries
* neither the expected typed payload nor a usable {@code return_value}.
*
* <p>A successful reply for a value-returning command (for example
* {@code AuthenticateUser}, {@code ArchestrAUserToId}, or {@code AddBufferedItem})
* must supply either the command's typed payload or an int32 {@code return_value}.
* A reply that satisfies neither is malformed, and the client surfaces this
* distinct failure rather than silently returning a default {@code 0}.
*/
public final class MxGatewayMalformedReplyException extends MxGatewayException {
/**
* Creates a new malformed-reply exception with the supplied message.
*
* @param message human-readable description of the malformed reply
*/
public MxGatewayMalformedReplyException(String message) {
super(message);
}
/**
* Creates a new malformed-reply exception with the supplied message and cause.
*
* @param message human-readable description of the malformed reply
* @param cause underlying error that triggered the failure
*/
public MxGatewayMalformedReplyException(String message, Throwable cause) {
super(message, cause);
}
}
@@ -54,4 +54,34 @@ public final class MxGatewaySecrets {
}
return String.join(" ", parts);
}
/**
* Replaces every occurrence of each supplied secret with the redaction
* marker {@code "<redacted>"}. Unlike {@link #redactCredentials(String)},
* which scrubs by pattern, this performs an exact-substring scrub of the
* caller-known secrets — used to strip a credential the gateway echoed back
* into a free-form failure message.
*
* @param message the message to scrub, may be {@code null}
* @param secrets the exact secret substrings to remove; {@code null}, empty,
* and blank (whitespace-only) entries and a {@code null} array are ignored
* @return {@code message} unchanged when it is {@code null} or no non-blank
* secret is supplied, otherwise the message with every secret occurrence
* replaced by {@code "<redacted>"}
*/
public static String redactExact(String message, String... secrets) {
if (message == null || secrets == null) {
return message;
}
String result = message;
for (String secret : secrets) {
if (secret == null || secret.isBlank()) {
// A blank "secret" would over-redact real whitespace; skip it.
continue;
}
result = result.replace(secret, "<redacted>");
}
return result;
}
}
@@ -31,6 +31,7 @@ import mxaccess_gateway.v1.MxaccessGateway.MxSparseElement;
import mxaccess_gateway.v1.MxaccessGateway.MxStatusProxy;
import mxaccess_gateway.v1.MxaccessGateway.MxValue;
import mxaccess_gateway.v1.MxaccessGateway.OpenSessionReply;
import mxaccess_gateway.v1.MxaccessGateway.ProtocolStatus;
import mxaccess_gateway.v1.MxaccessGateway.ReadBulkCommand;
import mxaccess_gateway.v1.MxaccessGateway.RegisterCommand;
import mxaccess_gateway.v1.MxaccessGateway.RemoveItemBulkCommand;
@@ -782,15 +783,17 @@ public final class MxGatewaySession implements AutoCloseable {
*/
public MxCommandReply writeSecuredRaw(
int serverHandle, int itemHandle, int currentUserId, int verifierUserId, MxValue value) {
return invokeCommand(MxCommand.newBuilder()
.setKind(MxCommandKind.MX_COMMAND_KIND_WRITE_SECURED)
.setWriteSecured(WriteSecuredCommand.newBuilder()
.setServerHandle(serverHandle)
.setItemHandle(itemHandle)
.setCurrentUserId(currentUserId)
.setVerifierUserId(verifierUserId)
.setValue(value))
.build());
return invokeCommandRedacted(
MxCommand.newBuilder()
.setKind(MxCommandKind.MX_COMMAND_KIND_WRITE_SECURED)
.setWriteSecured(WriteSecuredCommand.newBuilder()
.setServerHandle(serverHandle)
.setItemHandle(itemHandle)
.setCurrentUserId(currentUserId)
.setVerifierUserId(verifierUserId)
.setValue(value))
.build(),
secretStringOf(value));
}
/**
@@ -837,16 +840,18 @@ public final class MxGatewaySession implements AutoCloseable {
int verifierUserId,
MxValue value,
MxValue timestampValue) {
return invokeCommand(MxCommand.newBuilder()
.setKind(MxCommandKind.MX_COMMAND_KIND_WRITE_SECURED2)
.setWriteSecured2(WriteSecured2Command.newBuilder()
.setServerHandle(serverHandle)
.setItemHandle(itemHandle)
.setCurrentUserId(currentUserId)
.setVerifierUserId(verifierUserId)
.setValue(value)
.setTimestampValue(timestampValue))
.build());
return invokeCommandRedacted(
MxCommand.newBuilder()
.setKind(MxCommandKind.MX_COMMAND_KIND_WRITE_SECURED2)
.setWriteSecured2(WriteSecured2Command.newBuilder()
.setServerHandle(serverHandle)
.setItemHandle(itemHandle)
.setCurrentUserId(currentUserId)
.setVerifierUserId(verifierUserId)
.setValue(value)
.setTimestampValue(timestampValue))
.build(),
secretStringOf(value));
}
/**
@@ -866,17 +871,24 @@ public final class MxGatewaySession implements AutoCloseable {
* @throws MxAccessException when MXAccess rejects the credential
*/
public int authenticateUser(int serverHandle, String verifyUser, String verifyUserPassword) {
MxCommandReply reply = invokeCommand(MxCommand.newBuilder()
.setKind(MxCommandKind.MX_COMMAND_KIND_AUTHENTICATE_USER)
.setAuthenticateUser(AuthenticateUserCommand.newBuilder()
.setServerHandle(serverHandle)
.setVerifyUser(verifyUser)
.setVerifyUserPassword(verifyUserPassword))
.build());
MxCommandReply reply = invokeCommandRedacted(
MxCommand.newBuilder()
.setKind(MxCommandKind.MX_COMMAND_KIND_AUTHENTICATE_USER)
.setAuthenticateUser(AuthenticateUserCommand.newBuilder()
.setServerHandle(serverHandle)
.setVerifyUser(verifyUser)
.setVerifyUserPassword(verifyUserPassword))
.build(),
verifyUserPassword);
if (reply.hasAuthenticateUser()) {
return reply.getAuthenticateUser().getUserId();
}
return reply.getReturnValue().getInt32Value();
if (reply.hasReturnValue() && reply.getReturnValue().getKindCase() == MxValue.KindCase.INT32_VALUE) {
return reply.getReturnValue().getInt32Value();
}
throw new MxGatewayMalformedReplyException(
"AuthenticateUser returned a malformed reply: OK reply carried neither "
+ "the typed payload nor an int32 return_value");
}
/**
@@ -899,7 +911,12 @@ public final class MxGatewaySession implements AutoCloseable {
if (reply.hasArchestraUserToId()) {
return reply.getArchestraUserToId().getUserId();
}
return reply.getReturnValue().getInt32Value();
if (reply.hasReturnValue() && reply.getReturnValue().getKindCase() == MxValue.KindCase.INT32_VALUE) {
return reply.getReturnValue().getInt32Value();
}
throw new MxGatewayMalformedReplyException(
"ArchestrAUserToId returned a malformed reply: OK reply carried neither "
+ "the typed payload nor an int32 return_value");
}
/**
@@ -925,7 +942,12 @@ public final class MxGatewaySession implements AutoCloseable {
if (reply.hasAddBufferedItem()) {
return reply.getAddBufferedItem().getItemHandle();
}
return reply.getReturnValue().getInt32Value();
if (reply.hasReturnValue() && reply.getReturnValue().getKindCase() == MxValue.KindCase.INT32_VALUE) {
return reply.getReturnValue().getInt32Value();
}
throw new MxGatewayMalformedReplyException(
"AddBufferedItem returned a malformed reply: OK reply carried neither "
+ "the typed payload nor an int32 return_value");
}
/**
@@ -1027,6 +1049,162 @@ public final class MxGatewaySession implements AutoCloseable {
.build());
}
/**
* Invokes a credential-bearing command, scrubbing any exact secret the
* gateway may have echoed back into a surfaced failure message. The secret
* lives only in the request, but a non-parity gateway or provider can copy
* it into a diagnostic; this guarantees it never survives in the exception
* text a caller might log.
*
* <p>On failure both the exception's message <em>and</em> its structured
* context (the {@link ProtocolStatus} and {@link MxCommandReply} a caller can
* inspect and log) are scrubbed with {@link MxGatewaySecrets#redactExact}: the
* gateway echoes the credential into {@code protocolStatus.message},
* {@code reply.diagnosticMessage}, and each {@code statuses[i].diagnosticText}.
* If nothing carried the secret (the common case) the original exception is
* rethrown untouched. Otherwise it is re-thrown as the same concrete type
* carrying the redacted message and scrubbed context; the secret-bearing
* original is not chained as a cause, so it cannot leak through a printed
* stack trace.
*/
private MxCommandReply invokeCommandRedacted(MxCommand command, String... secrets) {
try {
return invokeCommand(command);
} catch (MxGatewayException ex) {
String original = ex.getMessage();
String redactedMessage = MxGatewaySecrets.redactExact(original, secrets);
boolean messageChanged = redactedMessage != null && !redactedMessage.equals(original);
ProtocolStatus status = protocolStatusOf(ex);
ProtocolStatus scrubbedStatus = scrubProtocolStatus(status, secrets);
boolean statusChanged = status != null && !status.equals(scrubbedStatus);
MxCommandReply reply = replyOf(ex);
MxCommandReply scrubbedReply = scrubReply(reply, secrets);
boolean replyChanged = reply != null && !reply.equals(scrubbedReply);
if (!messageChanged && !statusChanged && !replyChanged) {
throw ex;
}
String message = messageChanged ? redactedMessage : original;
throw rebuildRedacted(ex, message, scrubbedStatus, scrubbedReply);
}
}
/**
* Extracts the {@link ProtocolStatus} an exception carries, if any, so it can
* be scrubbed and re-attached to the rebuilt exception.
*/
private static ProtocolStatus protocolStatusOf(MxGatewayException ex) {
if (ex instanceof MxGatewayCommandException command) {
return command.protocolStatus();
}
if (ex instanceof MxGatewaySessionException session) {
return session.protocolStatus();
}
if (ex instanceof MxGatewayWorkerException worker) {
return worker.protocolStatus();
}
return null;
}
/**
* Extracts the raw {@link MxCommandReply} an exception carries, if any.
*/
private static MxCommandReply replyOf(MxGatewayException ex) {
if (ex instanceof MxGatewayCommandException command) {
return command.reply();
}
return null;
}
/**
* Rebuilds a gateway exception of the same concrete type with a redacted
* message and already-scrubbed context, mirroring the .NET client's
* type-switch. Only a truly-unknown subtype collapses to the base
* {@link MxGatewayException}. The original (secret-bearing) exception is
* deliberately not chained as a cause.
*/
private static MxGatewayException rebuildRedacted(
MxGatewayException ex, String message, ProtocolStatus status, MxCommandReply reply) {
if (ex instanceof MxAccessException) {
return new MxAccessException(message, status, reply, null);
}
if (ex instanceof MxGatewayCommandException) {
return new MxGatewayCommandException(message, status, reply, null);
}
if (ex instanceof MxGatewaySessionException) {
return new MxGatewaySessionException(message, status, null);
}
if (ex instanceof MxGatewayWorkerException) {
return new MxGatewayWorkerException(message, status, null);
}
if (ex instanceof MxGatewayMalformedReplyException) {
return new MxGatewayMalformedReplyException(message);
}
if (ex instanceof MxGatewayAuthenticationException) {
return new MxGatewayAuthenticationException(message, null);
}
if (ex instanceof MxGatewayAuthorizationException) {
return new MxGatewayAuthorizationException(message, null);
}
return new MxGatewayException(message);
}
/**
* Produces a scrubbed clone of a command reply, removing any exact secret the
* gateway echoed into {@code protocolStatus.message},
* {@code diagnosticMessage}, or a status's {@code diagnosticText}.
*
* @param reply the reply to scrub, or {@code null}
* @param secrets the exact secrets to strip
* @return {@code null} when {@code reply} is {@code null}, otherwise a clone
* with every echoed secret replaced by the redaction marker
*/
private static MxCommandReply scrubReply(MxCommandReply reply, String... secrets) {
if (reply == null) {
return null;
}
MxCommandReply.Builder builder = reply.toBuilder();
if (builder.hasProtocolStatus()) {
builder.setProtocolStatus(scrubProtocolStatus(builder.getProtocolStatus(), secrets));
}
builder.setDiagnosticMessage(MxGatewaySecrets.redactExact(builder.getDiagnosticMessage(), secrets));
for (int index = 0; index < builder.getStatusesCount(); index++) {
MxStatusProxy.Builder status = builder.getStatuses(index).toBuilder();
status.setDiagnosticText(MxGatewaySecrets.redactExact(status.getDiagnosticText(), secrets));
builder.setStatuses(index, status);
}
return builder.build();
}
/**
* Produces a scrubbed clone of a protocol status, removing any exact secret
* the gateway echoed into its free-form {@code message}.
*/
private static ProtocolStatus scrubProtocolStatus(ProtocolStatus status, String... secrets) {
if (status == null) {
return null;
}
return status.toBuilder()
.setMessage(MxGatewaySecrets.redactExact(status.getMessage(), secrets))
.build();
}
/**
* Extracts the string payload of a secured-write value so it can be scrubbed
* from an echoed failure message. Only string-kind values carry a
* credential-shaped secret worth redacting; other kinds return {@code null}
* (ignored by {@link MxGatewaySecrets#redactExact}).
*/
private static String secretStringOf(MxValue value) {
if (value != null && value.getKindCase() == MxValue.KindCase.STRING_VALUE) {
return value.getStringValue();
}
return null;
}
private static String newCorrelationId() {
byte[] bytes = new byte[16];
RANDOM.nextBytes(bytes);
@@ -20,6 +20,20 @@ public final class MxGatewaySessionException extends MxGatewayException {
this.protocolStatus = protocolStatus;
}
/**
* Creates a new session exception with an already-built, verbatim message.
* Used to re-surface a failure with a redacted message while preserving the
* (already scrubbed) protocol status.
*
* @param message the exact message to surface (already formatted/redacted)
* @param protocolStatus protocol status returned by the gateway
* @param cause underlying error, or {@code null}
*/
protected MxGatewaySessionException(String message, ProtocolStatus protocolStatus, Throwable cause) {
super(message, cause);
this.protocolStatus = protocolStatus;
}
/**
* Returns the gateway protocol status that triggered this exception.
*
@@ -20,6 +20,20 @@ public final class MxGatewayWorkerException extends MxGatewayException {
this.protocolStatus = protocolStatus;
}
/**
* Creates a new worker exception with an already-built, verbatim message.
* Used to re-surface a failure with a redacted message while preserving the
* (already scrubbed) protocol status.
*
* @param message the exact message to surface (already formatted/redacted)
* @param protocolStatus protocol status returned by the gateway
* @param cause underlying error, or {@code null}
*/
protected MxGatewayWorkerException(String message, ProtocolStatus protocolStatus, Throwable cause) {
super(message, cause);
this.protocolStatus = protocolStatus;
}
/**
* Returns the gateway protocol status that triggered this exception.
*
@@ -8,8 +8,11 @@ import mxaccess_gateway.v1.MxaccessGateway.MxStatusSource;
* Helpers for inspecting {@link MxStatusProxy} values returned by the gateway.
*
* <p>An {@code MxStatusProxy} mirrors the MXAccess COM {@code MXSTATUS_PROXY}
* struct. The success flag uses the MXAccess convention where any non-zero
* value indicates success.
* struct. Per the wire contract, {@code category} is the authoritative verdict:
* an entry succeeds only when its category is
* {@code MX_STATUS_CATEGORY_OK}. The {@code success} member carries the raw
* 16-bit COM value verbatim for diagnostics and is not a boolean, so it never
* decides success or failure.
*/
public final class MxStatuses {
private MxStatuses() {
@@ -18,12 +21,17 @@ public final class MxStatuses {
/**
* Returns whether the supplied status proxy reports success.
*
* <p>A {@code null} status is success because nothing was reported. A
* present entry whose category is {@code MX_STATUS_CATEGORY_UNSPECIFIED}
* is a failure: the worker always maps a category, so an unmapped one is
* not proven OK.
*
* @param status the status proxy, may be {@code null}
* @return {@code true} if {@code status} is {@code null} or its success
* flag is non-zero, {@code false} otherwise
* @return {@code true} if {@code status} is {@code null} or its category is
* {@code MX_STATUS_CATEGORY_OK}, {@code false} otherwise
*/
public static boolean succeeded(MxStatusProxy status) {
return status == null || status.getSuccess() != 0;
return status == null || status.getCategory() == MxStatusCategory.MX_STATUS_CATEGORY_OK;
}
/**
@@ -44,9 +52,11 @@ public final class MxStatuses {
*/
public record MxStatusView(MxStatusProxy raw) {
/**
* Returns the raw success flag (non-zero indicates success).
* Returns the raw {@code success} member exactly as MXAccess reported
* it. This is a diagnostic value, not a verdict — use
* {@link MxStatuses#succeeded(MxStatusProxy)} to decide success.
*
* @return the success flag value
* @return the raw success member
*/
public int success() {
return raw.getSuccess();
@@ -701,14 +701,17 @@ final class MxGatewayClientSessionTests {
.setSessionId(request.getSessionId())
.setKind(request.getCommand().getKind())
.setProtocolStatus(ok());
// `category` is the authoritative success indicator, so the fake
// must set it — a bare non-zero `success` is not a success.
var okStatus = mxaccess_gateway.v1.MxaccessGateway.MxStatusProxy.newBuilder()
.setSuccess(1)
.setCategory(mxaccess_gateway.v1.MxaccessGateway.MxStatusCategory.MX_STATUS_CATEGORY_OK);
if (request.getCommand().getKind() == MxCommandKind.MX_COMMAND_KIND_SUSPEND) {
reply.setSuspend(mxaccess_gateway.v1.MxaccessGateway.SuspendReply.newBuilder()
.setStatus(mxaccess_gateway.v1.MxaccessGateway.MxStatusProxy.newBuilder()
.setSuccess(1)));
.setStatus(okStatus));
} else if (request.getCommand().getKind() == MxCommandKind.MX_COMMAND_KIND_ACTIVATE) {
reply.setActivate(mxaccess_gateway.v1.MxaccessGateway.ActivateReply.newBuilder()
.setStatus(mxaccess_gateway.v1.MxaccessGateway.MxStatusProxy.newBuilder()
.setSuccess(1)));
.setStatus(okStatus));
}
responseObserver.onNext(reply.build());
responseObserver.onCompleted();
@@ -0,0 +1,201 @@
package com.zb.mom.ww.mxgateway.client;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import com.google.protobuf.util.JsonFormat;
import io.grpc.ManagedChannel;
import io.grpc.Server;
import io.grpc.inprocess.InProcessChannelBuilder;
import io.grpc.inprocess.InProcessServerBuilder;
import io.grpc.stub.StreamObserver;
import java.nio.file.Files;
import java.nio.file.Path;
import java.time.Duration;
import java.util.UUID;
import mxaccess_gateway.v1.MxAccessGatewayGrpc;
import mxaccess_gateway.v1.MxaccessGateway.MxCommandReply;
import mxaccess_gateway.v1.MxaccessGateway.MxCommandRequest;
import mxaccess_gateway.v1.MxaccessGateway.MxValue;
import mxaccess_gateway.v1.MxaccessGateway.ProtocolStatus;
import mxaccess_gateway.v1.MxaccessGateway.ProtocolStatusCode;
import org.junit.jupiter.api.Test;
final class MxGatewayCredentialReplyTests {
private static final String CREDENTIAL = "sup3rSecretVerify9f3a2b";
@Test
void authenticateUserRedactsEchoedCredentialFromReplyDrivenError() throws Exception {
assertCredentialFullyRedacted(
"authenticate-user.echoed-credential.reply.json", "auth-echo-session");
}
@Test
void authenticateUserRedactsEchoedCredentialFromMxAccessFailureReply() throws Exception {
assertCredentialFullyRedacted(
"authenticate-user.echoed-credential-mxaccess-failure.reply.json",
"auth-echo-failure-session");
}
private static void assertCredentialFullyRedacted(String fixture, String sessionId) throws Exception {
MxCommandReply reply = loadReply(fixture);
try (InProcessGateway gateway = InProcessGateway.startReturning(reply);
MxGatewayClient client = gateway.client()) {
MxGatewaySession session = MxGatewaySession.forSessionId(client, sessionId);
MxAccessException error = assertThrows(
MxAccessException.class,
() -> session.authenticateUser(12, "operator", CREDENTIAL));
assertFalse(error.getMessage().contains(CREDENTIAL),
"credential echoed by the gateway must not survive in the surfaced message");
assertTrue(error.getMessage().contains("<redacted>"),
"the echoed credential must be replaced with the redaction marker");
// The rebuilt exception must not re-expose the credential through the
// structured reply/protocolStatus a caller can inspect and log.
MxCommandReply surfaced = error.reply();
assertNotNull(surfaced, "the redacted exception must preserve a reply for inspection");
assertFalse(surfaced.getProtocolStatus().getMessage().contains(CREDENTIAL),
"reply protocol status message must not leak the echoed credential");
assertFalse(surfaced.getDiagnosticMessage().contains(CREDENTIAL),
"reply diagnostic message must not leak the echoed credential");
for (int index = 0; index < surfaced.getStatusesCount(); index++) {
assertFalse(surfaced.getStatusesList().get(index).getDiagnosticText().contains(CREDENTIAL),
"reply status diagnostic text must not leak the echoed credential");
}
assertNotNull(error.protocolStatus(), "the redacted exception must preserve a protocol status");
assertFalse(error.protocolStatus().getMessage().contains(CREDENTIAL),
"exception protocol status must not leak the echoed credential");
}
}
@Test
void authenticateUserMissingPayloadThrowsMalformedReply() throws Exception {
MxCommandReply reply = loadReply("authenticate-user.missing-payload.reply.json");
try (InProcessGateway gateway = InProcessGateway.startReturning(reply);
MxGatewayClient client = gateway.client()) {
MxGatewaySession session = MxGatewaySession.forSessionId(client, "auth-missing-session");
assertThrows(
MxGatewayMalformedReplyException.class,
() -> session.authenticateUser(3, "operator", "pw"));
}
}
@Test
void authenticateUserReturnValueOnlyReplyReturnsInt32Fallback() throws Exception {
MxCommandReply reply = loadReply("authenticate-user.return-value-only.reply.json");
try (InProcessGateway gateway = InProcessGateway.startReturning(reply);
MxGatewayClient client = gateway.client()) {
MxGatewaySession session = MxGatewaySession.forSessionId(client, "auth-return-session");
assertEquals(7, session.authenticateUser(3, "operator", "pw"));
}
}
@Test
void addBufferedItemReturnValueOnlyReplyReturnsInt32Fallback() throws Exception {
MxCommandReply reply = MxCommandReply.newBuilder()
.setProtocolStatus(ok())
.setReturnValue(MxValue.newBuilder().setInt32Value(55))
.build();
try (InProcessGateway gateway = InProcessGateway.startReturning(reply);
MxGatewayClient client = gateway.client()) {
MxGatewaySession session = MxGatewaySession.forSessionId(client, "buffered-return-session");
assertEquals(55, session.addBufferedItem(3, "Tank01.Level", "galaxy"));
}
}
@Test
void addBufferedItemMissingPayloadThrowsMalformedReply() throws Exception {
MxCommandReply reply = MxCommandReply.newBuilder().setProtocolStatus(ok()).build();
try (InProcessGateway gateway = InProcessGateway.startReturning(reply);
MxGatewayClient client = gateway.client()) {
MxGatewaySession session = MxGatewaySession.forSessionId(client, "buffered-malformed-session");
assertThrows(
MxGatewayMalformedReplyException.class,
() -> session.addBufferedItem(3, "Tank01.Level", "galaxy"));
}
}
private static ProtocolStatus ok() {
return ProtocolStatus.newBuilder()
.setCode(ProtocolStatusCode.PROTOCOL_STATUS_CODE_OK)
.build();
}
private static MxCommandReply loadReply(String fixture) throws Exception {
MxCommandReply.Builder builder = MxCommandReply.newBuilder();
JsonFormat.parser().merge(
Files.readString(fixtureRoot().resolve("command-replies/" + fixture)),
builder);
return builder.build();
}
private static Path fixtureRoot() {
Path current = Path.of(System.getProperty("user.dir")).toAbsolutePath();
for (Path path = current; path != null; path = path.getParent()) {
Path candidate = path.resolve("clients/proto/fixtures/behavior");
if (Files.exists(candidate)) {
return candidate;
}
candidate = path.resolve("../proto/fixtures/behavior").normalize();
if (Files.exists(candidate)) {
return candidate;
}
}
throw new IllegalStateException("could not locate behavior fixtures from " + current);
}
private record InProcessGateway(Server server, ManagedChannel channel) implements AutoCloseable {
static InProcessGateway startReturning(MxCommandReply reply) throws Exception {
String serverName = "mxgw-java-cred-" + UUID.randomUUID();
MxAccessGatewayGrpc.MxAccessGatewayImplBase service =
new MxAccessGatewayGrpc.MxAccessGatewayImplBase() {
@Override
public void invoke(
MxCommandRequest request, StreamObserver<MxCommandReply> responseObserver) {
responseObserver.onNext(reply);
responseObserver.onCompleted();
}
};
Server server = InProcessServerBuilder.forName(serverName)
.directExecutor()
.addService(service)
.build()
.start();
ManagedChannel channel = InProcessChannelBuilder.forName(serverName)
.directExecutor()
.build();
return new InProcessGateway(server, channel);
}
MxGatewayClient client() {
return new MxGatewayClient(
channel,
MxGatewayClientOptions.builder()
.endpoint("in-process")
.apiKey("")
.plaintext(true)
.callTimeout(Duration.ofSeconds(5))
.build());
}
@Override
public void close() {
channel.shutdownNow();
server.shutdownNow();
}
}
}
@@ -4,6 +4,7 @@ import static org.junit.jupiter.api.Assertions.assertArrayEquals;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertInstanceOf;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import com.google.gson.JsonArray;
@@ -20,6 +21,8 @@ import mxaccess_gateway.v1.MxaccessGateway.MxStatusProxy;
import mxaccess_gateway.v1.MxaccessGateway.MxValue;
import mxaccess_gateway.v1.MxaccessGateway.ProtocolStatusCode;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
final class MxGatewayFixtureTests {
@Test
@@ -89,6 +92,50 @@ final class MxGatewayFixtureTests {
throw new AssertionError("expected MxAccessException");
}
@ParameterizedTest
@CsvSource({
"register.ok.reply.json,false",
"write.status-category-error-success-set.reply.json,true",
"write.status-category-ok-success-zero.reply.json,false",
"write.hresult-s-false.reply.json,false",
"write.hresult-e-fail.reply.json,true",
})
void replyValidationFixturesBranchOnCategoryAndNegativeHresult(String fixture, boolean expectFailure)
throws Exception {
MxCommandReply.Builder builder = MxCommandReply.newBuilder();
JsonFormat.parser().merge(
Files.readString(fixtureRoot().resolve("command-replies/" + fixture)),
builder);
MxCommandReply reply = builder.build();
if (expectFailure) {
assertThrows(MxAccessException.class, () -> MxGatewayErrors.ensureMxAccessSuccess("write", reply));
} else {
MxGatewayErrors.ensureMxAccessSuccess("write", reply);
}
}
@ParameterizedTest
@CsvSource({
"MX_STATUS_CATEGORY_OK,0,true",
"MX_STATUS_CATEGORY_OK,1,true",
"MX_STATUS_CATEGORY_COMMUNICATION_ERROR,1,false",
"MX_STATUS_CATEGORY_UNSPECIFIED,1,false",
})
void statusEntryVerdictIgnoresTheRawSuccessMember(String category, int success, boolean expectSucceeded) {
MxStatusProxy status = MxStatusProxy.newBuilder()
.setCategory(MxStatusCategory.valueOf(category))
.setSuccess(success)
.build();
assertEquals(expectSucceeded, MxStatuses.succeeded(status));
}
@Test
void absentStatusEntryIsSuccess() {
assertTrue(MxStatuses.succeeded(null));
}
@Test
void grpcAuthErrorsAreClassifiedAndRedacted() {
RuntimeException authError = MxGatewayErrors.fromGrpc(
@@ -0,0 +1,50 @@
package com.zb.mom.ww.mxgateway.client;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNull;
import org.junit.jupiter.api.Test;
final class MxGatewaySecretsTests {
@Test
void redactExactReplacesEveryOccurrenceOfASecret() {
String message = "verify s3cr3t, retry s3cr3t, done s3cr3t";
String result = MxGatewaySecrets.redactExact(message, "s3cr3t");
assertFalse(result.contains("s3cr3t"), "no occurrence of the secret may survive");
assertEquals("verify <redacted>, retry <redacted>, done <redacted>", result);
}
@Test
void redactExactFullyRedactsOverlappingSecretsWhenOneIsASubstringOfTheOther() {
String message = "password=hunter2 token=hunter2extra";
String result = MxGatewaySecrets.redactExact(message, "hunter2extra", "hunter2");
assertFalse(result.contains("hunter2"), "both the secret and its superstring must be fully redacted");
assertEquals("password=<redacted> token=<redacted>", result);
}
@Test
void redactExactWithNoSecretsReturnsMessageUnchanged() {
String message = "nothing to scrub here";
assertEquals(message, MxGatewaySecrets.redactExact(message));
}
@Test
void redactExactToleratesNullMessage() {
assertNull(MxGatewaySecrets.redactExact(null, "secret"));
}
@Test
void redactExactIgnoresBlankSecretSoRealSpacesAreNotOverRedacted() {
String message = "keep these spaces intact";
String result = MxGatewaySecrets.redactExact(message, " ", "");
assertEquals(message, result);
}
}
@@ -0,0 +1,22 @@
{
"sessionId": "session-fixture",
"correlationId": "gateway-correlation-authenticate-echoed-mxaccess-failure",
"kind": "MX_COMMAND_KIND_AUTHENTICATE_USER",
"protocolStatus": {
"code": "PROTOCOL_STATUS_CODE_MXACCESS_FAILURE",
"message": "MXAccess AuthenticateUser rejected credential 'sup3rSecretVerify9f3a2b'."
},
"hresult": -2147024891,
"statuses": [
{
"success": 0,
"category": "MX_STATUS_CATEGORY_SECURITY_ERROR",
"detectedBy": "MX_STATUS_SOURCE_RESPONDING_NMX",
"detail": 5,
"rawCategory": 8,
"rawDetectedBy": 5,
"diagnosticText": "Authentication failed for password 'sup3rSecretVerify9f3a2b'."
}
],
"diagnosticMessage": "MXAccess echoed the credential 'sup3rSecretVerify9f3a2b' back in its failure diagnostic."
}
@@ -0,0 +1,22 @@
{
"sessionId": "session-fixture",
"correlationId": "gateway-correlation-authenticate-echoed",
"kind": "MX_COMMAND_KIND_AUTHENTICATE_USER",
"protocolStatus": {
"code": "PROTOCOL_STATUS_CODE_OK",
"message": "MXAccess AuthenticateUser rejected credential 'sup3rSecretVerify9f3a2b'."
},
"hresult": -2147024891,
"statuses": [
{
"success": 0,
"category": "MX_STATUS_CATEGORY_SECURITY_ERROR",
"detectedBy": "MX_STATUS_SOURCE_RESPONDING_NMX",
"detail": 5,
"rawCategory": 8,
"rawDetectedBy": 5,
"diagnosticText": "Authentication failed for password 'sup3rSecretVerify9f3a2b'."
}
],
"diagnosticMessage": "MXAccess echoed the credential 'sup3rSecretVerify9f3a2b' back in its failure diagnostic."
}
@@ -0,0 +1,10 @@
{
"sessionId": "session-fixture",
"correlationId": "gateway-correlation-authenticate-missing-payload",
"kind": "MX_COMMAND_KIND_AUTHENTICATE_USER",
"protocolStatus": {
"code": "PROTOCOL_STATUS_CODE_OK",
"message": "AuthenticateUser reached MXAccess."
},
"diagnosticMessage": "Malformed: the OK reply carried neither an AuthenticateUser payload nor a return_value."
}
@@ -0,0 +1,15 @@
{
"sessionId": "session-fixture",
"correlationId": "gateway-correlation-authenticate-return-value-only",
"kind": "MX_COMMAND_KIND_AUTHENTICATE_USER",
"protocolStatus": {
"code": "PROTOCOL_STATUS_CODE_OK",
"message": "AuthenticateUser reached MXAccess."
},
"returnValue": {
"dataType": "MX_DATA_TYPE_INTEGER",
"variantType": "VT_I4",
"int32Value": 7
},
"diagnosticMessage": "Legacy worker populated only return_value; the typed AuthenticateUser payload is absent."
}
@@ -0,0 +1,29 @@
{
"sessionId": "session-fixture",
"correlationId": "gateway-correlation-write-e-fail",
"kind": "MX_COMMAND_KIND_WRITE",
"protocolStatus": {
"code": "PROTOCOL_STATUS_CODE_OK",
"message": "Write reached MXAccess."
},
"hresult": -2147467259,
"returnValue": {
"dataType": "MX_DATA_TYPE_NO_DATA",
"variantType": "VT_EMPTY",
"isNull": true,
"rawDiagnostic": "MXAccess returned no value for the failed write.",
"rawDataType": 2
},
"statuses": [
{
"success": 1,
"category": "MX_STATUS_CATEGORY_OK",
"detectedBy": "MX_STATUS_SOURCE_RESPONDING_LMX",
"detail": 0,
"rawCategory": 0,
"rawDetectedBy": 3,
"diagnosticText": "OK"
}
],
"diagnosticMessage": "COM semantics: a negative HRESULT (E_FAIL, 0x80004005) is a failure even when every status entry is OK."
}
@@ -0,0 +1,29 @@
{
"sessionId": "session-fixture",
"correlationId": "gateway-correlation-write-s-false",
"kind": "MX_COMMAND_KIND_WRITE",
"protocolStatus": {
"code": "PROTOCOL_STATUS_CODE_OK",
"message": "Write completed with S_FALSE."
},
"hresult": 1,
"returnValue": {
"dataType": "MX_DATA_TYPE_NO_DATA",
"variantType": "VT_EMPTY",
"isNull": true,
"rawDiagnostic": "MXAccess returned no value for the write.",
"rawDataType": 2
},
"statuses": [
{
"success": 1,
"category": "MX_STATUS_CATEGORY_OK",
"detectedBy": "MX_STATUS_SOURCE_RESPONDING_LMX",
"detail": 0,
"rawCategory": 0,
"rawDetectedBy": 3,
"diagnosticText": "OK"
}
],
"diagnosticMessage": "COM semantics: a positive HRESULT such as S_FALSE (1) is a success code, not a failure."
}
@@ -0,0 +1,29 @@
{
"sessionId": "session-fixture",
"correlationId": "gateway-correlation-write-category-error",
"kind": "MX_COMMAND_KIND_WRITE",
"protocolStatus": {
"code": "PROTOCOL_STATUS_CODE_OK",
"message": "Write reached MXAccess."
},
"hresult": 0,
"returnValue": {
"dataType": "MX_DATA_TYPE_NO_DATA",
"variantType": "VT_EMPTY",
"isNull": true,
"rawDiagnostic": "MXAccess returned no value for the write.",
"rawDataType": 2
},
"statuses": [
{
"success": 1,
"category": "MX_STATUS_CATEGORY_COMMUNICATION_ERROR",
"detectedBy": "MX_STATUS_SOURCE_RESPONDING_LMX",
"detail": 77,
"rawCategory": 5,
"rawDetectedBy": 3,
"diagnosticText": "Responding LMX lost communication mid-write."
}
],
"diagnosticMessage": "Category is authoritative: a non-OK category is a failure even when the raw success member is non-zero."
}
@@ -0,0 +1,29 @@
{
"sessionId": "session-fixture",
"correlationId": "gateway-correlation-write-category-ok",
"kind": "MX_COMMAND_KIND_WRITE",
"protocolStatus": {
"code": "PROTOCOL_STATUS_CODE_OK",
"message": "Write completed."
},
"hresult": 0,
"returnValue": {
"dataType": "MX_DATA_TYPE_NO_DATA",
"variantType": "VT_EMPTY",
"isNull": true,
"rawDiagnostic": "MXAccess returned no value for the write.",
"rawDataType": 2
},
"statuses": [
{
"success": 0,
"category": "MX_STATUS_CATEGORY_OK",
"detectedBy": "MX_STATUS_SOURCE_RESPONDING_LMX",
"detail": 0,
"rawCategory": 0,
"rawDetectedBy": 3,
"diagnosticText": "OK, reported with a zero raw success member."
}
],
"diagnosticMessage": "Category is authoritative: MX_STATUS_CATEGORY_OK is success even when the raw success member is zero."
}
@@ -20,6 +20,62 @@
"path": "command-replies/write.mxaccess-failure.reply.json",
"expectation": "MXAccess failures are data-bearing replies with HRESULT and status details, not transport failures."
},
{
"id": "command-reply.write.status-category-error-success-set",
"category": "command_replies",
"messageType": "mxaccess_gateway.v1.MxCommandReply",
"path": "command-replies/write.status-category-error-success-set.reply.json",
"expectation": "A status entry fails when its category is not MX_STATUS_CATEGORY_OK, even though the raw success member is non-zero."
},
{
"id": "command-reply.write.status-category-ok-success-zero",
"category": "command_replies",
"messageType": "mxaccess_gateway.v1.MxCommandReply",
"path": "command-replies/write.status-category-ok-success-zero.reply.json",
"expectation": "A status entry succeeds when its category is MX_STATUS_CATEGORY_OK, even though the raw success member is zero."
},
{
"id": "command-reply.write.hresult-s-false",
"category": "command_replies",
"messageType": "mxaccess_gateway.v1.MxCommandReply",
"path": "command-replies/write.hresult-s-false.reply.json",
"expectation": "A positive HRESULT such as S_FALSE (1) is a COM success code and does not fail the reply."
},
{
"id": "command-reply.write.hresult-e-fail",
"category": "command_replies",
"messageType": "mxaccess_gateway.v1.MxCommandReply",
"path": "command-replies/write.hresult-e-fail.reply.json",
"expectation": "A negative HRESULT fails the reply even when every status entry reports MX_STATUS_CATEGORY_OK."
},
{
"id": "command-reply.authenticate-user.echoed-credential",
"category": "command_replies",
"messageType": "mxaccess_gateway.v1.MxCommandReply",
"path": "command-replies/authenticate-user.echoed-credential.reply.json",
"expectation": "When a gateway/MXAccess diagnostic echoes the caller's credential back (OK envelope, negative HRESULT), the surfaced error redacts the exact secret from both the rendered message and the structured reply accessors."
},
{
"id": "command-reply.authenticate-user.echoed-credential-mxaccess-failure",
"category": "command_replies",
"messageType": "mxaccess_gateway.v1.MxCommandReply",
"path": "command-replies/authenticate-user.echoed-credential-mxaccess-failure.reply.json",
"expectation": "The same echoed-credential redaction holds when the reply is coded PROTOCOL_STATUS_CODE_MXACCESS_FAILURE, which every client routes to its MXAccess error type."
},
{
"id": "command-reply.authenticate-user.missing-payload",
"category": "command_replies",
"messageType": "mxaccess_gateway.v1.MxCommandReply",
"path": "command-replies/authenticate-user.missing-payload.reply.json",
"expectation": "An OK reply with neither the typed AuthenticateUser payload nor a return_value raises a typed malformed-reply error, never a proto3 default 0 and never an NRE."
},
{
"id": "command-reply.authenticate-user.return-value-only",
"category": "command_replies",
"messageType": "mxaccess_gateway.v1.MxCommandReply",
"path": "command-replies/authenticate-user.return-value-only.reply.json",
"expectation": "An OK reply missing the typed AuthenticateUser payload but carrying an int32 return_value falls back to the return_value (legacy-worker compatibility)."
},
{
"id": "event-stream.session-ordered",
"category": "event_streams",
@@ -3,6 +3,7 @@
"cases": [
{
"id": "ok.responding-lmx",
"wantSuccess": true,
"status": {
"success": 1,
"category": "MX_STATUS_CATEGORY_OK",
@@ -15,6 +16,7 @@
},
{
"id": "security-error.requesting-lmx",
"wantSuccess": false,
"status": {
"success": 0,
"category": "MX_STATUS_CATEGORY_SECURITY_ERROR",
@@ -27,6 +29,7 @@
},
{
"id": "raw-unknown-category",
"wantSuccess": false,
"status": {
"success": 0,
"category": "MX_STATUS_CATEGORY_UNKNOWN",
+64 -2
View File
@@ -47,6 +47,19 @@ The tests import the generated gateway and worker stubs, run fake async gateway
stubs, verify API key metadata, exercise stream cancellation, load shared value
and command fixtures, and check deterministic CLI output.
`tests/test_wire_fake_gateway.py` is the one suite that does **not** substitute a
stub: it serves a canned `MxAccessGatewayServicer` from a real `grpc.aio` server
on an ephemeral loopback port and drives the ordinary `GatewayClient` API against
it. Only the gateway's behaviour is canned — the HTTP/2 framing, protobuf
serialization, `authorization` metadata, and gRPC status codes are genuine, so it
catches decode and metadata breaks a stub fake cannot see. No MXAccess, no worker,
no TLS, so it runs in the default suite. See `docs/GatewayTesting.md`
(Client Wire Tests) for the cross-client pattern.
```powershell
python -m pytest tests/test_wire_fake_gateway.py
```
## Packaging
Install the package in editable mode for local development:
@@ -102,6 +115,31 @@ messages from the gateway's central monitor), and
and ack target). Cancel the surrounding task or `aclose()` the iterator to
terminate the stream.
`ActiveAlarmSnapshot.from_truncated_snapshot` reports that the record came from
a provider fetch which hit the per-fetch cap: the snapshot set may omit active
alarms, and the gateway suspended its absence-implies-cleared inference for that
poll. Treat the set as possibly incomplete rather than reconciling deletions
from it. It is set-level degraded status, not a comment on the record's own
fidelity, and is distinct from `degraded` (the subtag fallback provider).
`stream_alarms` also carries that completeness verdict at feed level, as a
message whose `WhichOneof("payload")` is `snapshot_status` and whose
`snapshot_status.truncated` is true while the monitor's cached set derives from
a truncated fetch. One arrives at stream open (after the `provider_status`
frame, before the cached `active_alarm` frames) so a late joiner learns the
current verdict, then one on every verdict change — including the clearing frame
sent when the gateway's alarm monitor restarts and drops a truncated verdict.
Track it if you need set completeness on a live feed without polling
`query_active_alarms`.
`mxgw-py stream-alarms` writes one JSON row as each feed message arrives. The
default `--max-messages 0` is unbounded; use a positive value for a finite
sample, remembering that the provider-status and snapshot-status preamble both
count toward it. `--timeout` ends the command after an idle wait and the stream
is closed in every exit path. Protobuf JSON omits false-valued scalar defaults,
so healthy/clearing frames appear as `{"providerStatus": {}}` and
`{"snapshotStatus": {}}`; absent means false.
Canceling a Python task cancels the client-side gRPC call or stream wait. It
does not abort an in-flight MXAccess COM call inside the worker process.
@@ -187,7 +225,13 @@ await session.write_secured(
```
The CLI mirrors these as `authenticate-user` (credential via `--password` or,
preferably, `--password-env`) and `write-secured`.
preferably, the variable named by `--password-env`, default
`MXGATEWAY_VERIFY_PASSWORD`) and `write-secured`. The credential is required: a
missing or empty resolved value raises a `UsageError` naming the option and the
variable, so the CLI fails before connecting instead of authenticating with an
empty password. `MXGATEWAY_VERIFY_PASSWORD` is the canonical variable across all
five client CLIs — see
[Cross-Language Smoke Matrix](../../docs/CrossLanguageSmokeMatrix.md).
### Array writes replace the whole array
@@ -392,6 +436,15 @@ point: the `require_certificate_validation=True` keyword on
`--require-certificate-validation` CLI flag. See
[Gateway Configuration](../../docs/GatewayConfiguration.md#automatic-self-signed-certificate).
Channel construction is split in two: `resolve_channel_security(options)` performs
the blocking part (the trust-on-first-use certificate probe) and
`create_channel(options, security=...)` builds the channel. The async `connect`
classmethods run the first off the event loop and the second on it, because a
`grpc.aio` channel binds to the event loop current on the constructing thread —
building it inside `asyncio.to_thread` raises
`RuntimeError: There is no current event loop in thread 'asyncio_N'`. Callers that
build their own channel should keep `create_channel` on the loop thread.
## CLI
The CLI emits deterministic JSON for automation:
@@ -403,7 +456,7 @@ mxgw-py register --session-id <id> --client-name python-client --json
mxgw-py add-item --session-id <id> --server-handle 1 --item Object.Attribute --json
mxgw-py advise --session-id <id> --server-handle 1 --item-handle 2 --json
mxgw-py stream-events --session-id <id> --max-events 1 --json
mxgw-py stream-alarms --max-messages 1 --json
mxgw-py stream-alarms --max-messages 4 --json
mxgw-py acknowledge-alarm --reference "\\Galaxy\Area001.Pump001.PumpFault" --json
mxgw-py write --session-id <id> --server-handle 1 --item-handle 2 --type int32 --value 123 --json
```
@@ -439,6 +492,15 @@ mxgw-py smoke --endpoint $env:MXGATEWAY_ENDPOINT --plaintext --api-key-env MXGAT
## Installing from the Gitea PyPI Feed
**The published `0.2.0` artifact cannot connect.** Its `connect()` built the
`grpc.aio` channel inside `asyncio.to_thread`, and a `grpc.aio` channel binds
to the event loop current on the *constructing* thread — so every
non-stub connection raised `RuntimeError: There is no current event loop in
thread`. Fixed in-tree by commit `a8f86b5` (`resolve_channel_security` now
runs off-loop, `create_channel` runs on-loop). The fix is superseded by
`0.2.1`; do not install `0.2.0` from the feed below — use `0.2.1` or later,
or install from source at a commit at or after `a8f86b5`.
The client publishes to the internal Gitea PyPI feed:
````bash
+5 -1
View File
@@ -36,7 +36,11 @@ function Resolve-Python {
function Assert-GrpcioToolsVersion {
param([string]$Python)
$version = (& $Python -c 'import grpc_tools; from importlib.metadata import version; print(version("grpcio-tools"))').Trim()
# Windows PowerShell 5.1 strips embedded double quotes when passing an argument to a native
# exe (pwsh 7 does not), so a Python literal quoted with " " here would arrive at Python as
# print(version(grpcio-tools)) -> NameError. Use single quotes for the Python string literal
# inside this double-quoted PowerShell string so the quoting survives on both hosts.
$version = (& $Python -c "import grpc_tools; from importlib.metadata import version; print(version('grpcio-tools'))").Trim()
if ($version -ne $PinnedGrpcioToolsVersion) {
throw "grpcio-tools $version is installed, but regeneration is pinned to $PinnedGrpcioToolsVersion. " +
"Install the pin (python -m pip install 'grpcio-tools==$PinnedGrpcioToolsVersion') before regenerating, " +
+1 -1
View File
@@ -6,7 +6,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "zb-mom-ww-mxaccess-gateway-client"
version = "0.1.2"
version = "0.2.1"
description = "Async Python client for MXAccess Gateway."
readme = "README.md"
requires-python = ">=3.12"
@@ -11,6 +11,7 @@ from .generated.galaxy_repository_pb2 import (
)
from .events import ReplayGap
from .errors import (
MalformedReplyError,
MxAccessError,
MxGatewayAuthenticationError,
MxGatewayAuthorizationError,
@@ -35,6 +36,7 @@ __all__ = [
"GalaxyRepositoryClient",
"GatewayClient",
"LazyBrowseNode",
"MalformedReplyError",
"MxAccessError",
"MxGatewayAuthenticationError",
"MxGatewayAuthorizationError",
@@ -12,7 +12,7 @@ from .auth import merge_metadata
from .errors import ensure_protocol_success, map_rpc_error
from .generated import mxaccess_gateway_pb2 as pb
from .generated import mxaccess_gateway_pb2_grpc as pb_grpc
from .options import ClientOptions, create_channel
from .options import ClientOptions, create_channel, resolve_channel_security
class GatewayClient:
@@ -58,9 +58,13 @@ class GatewayClient:
if stub is not None:
return cls(options=resolved, stub=stub)
# create_channel may perform a blocking TLS certificate probe (TOFU
# default); run it off the event loop so connect never freezes the loop.
channel = await asyncio.to_thread(create_channel, resolved)
# Resolving security may perform a blocking TLS certificate probe (TOFU
# default); run that off the event loop so connect never freezes it. The
# channel itself must be built on the loop thread — a grpc.aio channel
# binds to the loop current on the constructing thread, and a worker
# thread has none.
security = await asyncio.to_thread(resolve_channel_security, resolved)
channel = create_channel(resolved, security=security)
return cls(
options=resolved,
stub=pb_grpc.MxAccessGatewayStub(channel),
@@ -53,6 +53,10 @@ class MxAccessError(MxGatewayCommandError):
"""MXAccess HRESULT or status failure."""
class MalformedReplyError(MxGatewayError):
"""Raised when an OK reply lacks the expected typed payload and any usable return_value fallback."""
def map_rpc_error(operation: str, error: grpc.RpcError) -> MxGatewayTransportError:
"""Map a generated gRPC exception to the client exception hierarchy."""
@@ -137,8 +141,10 @@ def ensure_mxaccess_success(operation: str, reply: pb.MxCommandReply) -> pb.MxCo
raw_reply=reply,
)
# `category` is the authoritative verdict per the wire contract; `success`
# is the raw COM member carried verbatim for diagnostics only.
for mx_status in reply.statuses:
if mx_status.success == 0:
if mx_status.category != pb.MX_STATUS_CATEGORY_OK:
raise MxAccessError(
_mxaccess_message(operation, reply),
protocol_status=status,
@@ -151,8 +157,18 @@ def ensure_mxaccess_success(operation: str, reply: pb.MxCommandReply) -> pb.MxCo
def _mxaccess_message(operation: str, reply: pb.MxCommandReply) -> str:
status_text = reply.protocol_status.message or "MXAccess command failed"
hresult = reply.hresult if reply.HasField("hresult") else None
return (
message = (
f"{operation} failed: {status_text}; "
f"session={reply.session_id}; correlation={reply.correlation_id}; "
f"hresult={hresult}; statuses={len(reply.statuses)}"
)
# Append a per-status breakdown that carries the raw `success` COM member
# verbatim for diagnostic parity with the other clients. `category` remains
# the authoritative verdict; `success` is diagnostics only.
for status in reply.statuses:
category = pb.MxStatusCategory.Name(status.category)
message += (
f" [success={status.success}, category={category}, "
f"detail={status.detail}, {status.diagnostic_text}]"
)
return message
@@ -21,7 +21,12 @@ from .auth import merge_metadata
from .errors import MxGatewayError, map_rpc_error
from .generated import galaxy_repository_pb2 as galaxy_pb
from .generated import galaxy_repository_pb2_grpc as galaxy_pb_grpc
from .options import BrowseChildrenOptions, ClientOptions, create_channel
from .options import (
BrowseChildrenOptions,
ClientOptions,
create_channel,
resolve_channel_security,
)
_DISCOVER_HIERARCHY_PAGE_SIZE = 5000
_BROWSE_CHILDREN_PAGE_SIZE = 500
@@ -70,9 +75,13 @@ class GalaxyRepositoryClient:
if stub is not None:
return cls(options=resolved, stub=stub)
# create_channel may perform a blocking TLS certificate probe (TOFU
# default); run it off the event loop so connect never freezes the loop.
channel = await asyncio.to_thread(create_channel, resolved)
# Resolving security may perform a blocking TLS certificate probe (TOFU
# default); run that off the event loop so connect never freezes it. The
# channel itself must be built on the loop thread — a grpc.aio channel
# binds to the loop current on the constructing thread, and a worker
# thread has none.
security = await asyncio.to_thread(resolve_channel_security, resolved)
channel = create_channel(resolved, security=security)
return cls(
options=resolved,
stub=galaxy_pb_grpc.GalaxyRepositoryStub(channel),

Some files were not shown because too many files have changed in this diff Show More