Compare commits

..

62 Commits

Author SHA1 Message Date
Joseph Doherty a346d514dd test(contracts): scope command-reply fixture invariants past the CLI-40/41 authenticate-user malformed-reply fixtures
ci / portable (push) Successful in 14m0s
ci / java (push) Successful in 6m50s
ci / windows-x86 (push) Failing after 1m21s
ci / nightly-windev (push) Has been skipped
The blanket loop asserted HRESULT/Statuses/ReturnValue on every command_replies fixture, but the authenticate-user.* fixtures added for the malformed-reply and credential-redaction contracts deliberately omit them (NRE on ReturnValue.DataType). Keep universal Kind/ProtocolStatus invariants for all; apply the MXAccess-detail block only to fixtures that carry it. Test-only.
2026-08-07 08:48:49 -04:00
Joseph Doherty a2d3f66b8b docs(archreview): record next-cycle candidate findings + pending operator actions surfaced during remediation
ci / windows-x86 (push) Successful in 1m19s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m19s
ci / portable (push) Failing after 4m55s
2026-08-07 08:48:03 -04:00
Joseph Doherty 93d84019b9 docs(tracking): sync IPC-23 domain register to Done (doc wave landed; Grpc.md row intentionally scoped out — DrainEvents is a worker diagnostic)
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 1m19s
ci / java (push) Successful in 2m4s
ci / portable (push) Failing after 4m43s
2026-08-07 08:47:31 -04:00
Joseph Doherty 6d26ed094c docs(tracking): close old-tracker CLI-24, CLI-34 as Done (2026-07-12 review old-tracker actions; both incidentally fixed)
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m22s
ci / windows-x86 (push) Successful in 1m20s
ci / portable (push) Failing after 4m26s
2026-08-07 08:11:57 -04:00
Joseph Doherty 4201da63d2 docs(tracking): flip IPC-24/IPC-25 to Done in the Contracts&IPC domain register (missed by the codegen-wave tracker update)
ci / windows-x86 (push) Failing after 1m19s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m17s
ci / portable (push) Failing after 4m51s
2026-08-07 08:10:39 -04:00
Joseph Doherty 9c780f8164 Merge branch 'fix/cli-39-version-train'
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 1m20s
ci / java (push) Successful in 2m12s
ci / portable (push) Failing after 4m44s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
2026-08-07 08:09:39 -04:00
Joseph Doherty 440e7cf03d fix(CLI-39): bump Contracts nupkg to 0.2.0; scope pack-clients.ps1 regexes
Code review of the CLI-39 branch caught an Important gap: Contracts.csproj
was left at the already-published 0.1.2 while the .NET Client moved to
0.2.0. Invoke-PackDotnet in scripts/pack-clients.ps1 packs and publishes
both ZB.MOM.WW.MxGateway.Contracts and .Client through the same -Publish
loop, and the new collision guard runs every nupkg it finds through
Assert-GiteaPackageNotPublished. Left as-is, the next real .NET publish
would pack Contracts at 0.1.2, the guard would correctly refuse to
republish it, and the loop would abort mid-way with Client (alphabetically
first) possibly already pushed -- the two packages permanently out of
lockstep.

- src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csproj:
  <Version> 0.1.2 -> 0.2.0, matching the .NET Client (they have always
  released together).
- src/Directory.Build.props: corrected a comment that was now stale --
  it claimed the repo-wide 0.1.2 default was kept to match the Contracts
  package, which is no longer true now that Contracts.csproj overrides it.
  The <Version> value itself is unchanged; Server/Worker/Tests staying at
  0.1.2 is a separate, not-yet-made decision, out of scope for CLI-39.
- docs/ClientPackaging.md: Contracts.csproj added as a fifth manifest in
  the Versioning section, with the near-miss recorded.

Also hardened scripts/pack-clients.ps1 per the same review: the Python
(pyproject.toml) and Rust (Cargo.toml) version-extraction regexes now
scope to the [project]/[package] section header instead of matching the
first "version = ..." line anywhere in the file (Cargo.toml has an
identical second one under [workspace.package] -- matching whichever came
first was luck of ordering, not correctness). One-line comment added on
the nuget filename-parse regex.

Verified live against the real Gitea registry: Contracts and Client both
still refuse at 0.1.2 and both now pass at 0.2.0, including running the
actual Invoke-PackDotnet filename-parse-then-guard logic against two
freshly packed real .nupkg files. dotnet build of Contracts.csproj and the
client slnx both clean. No publish performed.
2026-08-07 08:07:40 -04:00
Joseph Doherty ae605d2368 Merge remote-tracking branch 'origin/fix/wrk-22-25-seam'
ci / windows-x86 (push) Failing after 1m19s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m10s
ci / portable (push) Failing after 4m27s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
#	archreview/2026-07-12/remediation/30-contracts-ipc.md
2026-08-07 08:01:10 -04:00
Joseph Doherty 9b2abef4e1 fix(CLI-39): bump client versions off published 0.1.2; guard the publish pipeline
Converges all five clients on one version after four had drifted onto the
already-published 0.1.2/0.1.1 while their APIs kept changing underneath it:

- Rust Cargo.toml [package] + [workspace.package] -> 0.2.0 (CLIENT_VERSION
  already derives from CARGO_PKG_VERSION, no separate edit).
- Python pyproject.toml + version.py -> 0.2.0; new test asserts __version__
  matches pyproject.toml (closes the CLI-26 residual drift mode).
- Go mxgateway/version.go ClientVersion -> 0.2.0.
- .NET ZB.MOM.WW.MxGateway.Client.csproj <Version> -> 0.2.0.
- Java -> 0.2.1, not 0.2.0: the live Gitea Maven feed already had 0.2.0
  published (2026-06-26), before the CLI-37/38/40/41 conformance fixes
  changed the client's observable behavior, so reusing 0.2.0 would label
  two different APIs identically. Recorded as an exception in
  docs/ClientPackaging.md's new Versioning section.

Publish-pipeline guards:

- scripts/tag-go-module.ps1 implements the CLI-21 guard: after semver
  validation it refuses to tag unless clients/go/mxgateway/version.go's
  ClientVersion already matches the requested tag version.
- scripts/pack-clients.ps1 gains a Gitea package-registry collision guard
  wired into every per-language -Publish step; it aborts if the target
  name+version already exists rather than force-overwriting. Verified live
  against the real Gitea registry (credentials already present in this
  environment) — correctly refuses on every known-published artifact and
  passes on every unpublished target.

Docs updated in the same commit: docs/ClientPackaging.md (new Versioning
section), and the five client READMEs' stale 0.1.1/0.1.2 example versions.

No .proto changes. No publish performed.
2026-08-07 07:58:49 -04:00
Joseph Doherty 815e58d28b docs(tracking): record WRK-22/24/25/27 + IPC-26 windev evidence (377 pass)
ci / java (push) Successful in 2m7s
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 1m12s
ci / portable (push) Failing after 4m12s
2026-08-07 07:52:44 -04:00
Joseph Doherty 8df35cd63a fix(WRK-22,WRK-24,WRK-25,WRK-27,IPC-26): worker write-seam hardening
ci / java (push) Successful in 2m7s
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 1m13s
ci / portable (push) Failing after 4m6s
WRK-22/IPC-26: tombstone a WriteAsync/WriteBatchAsync cancelled while
waiting for the write lock (PendingFrame.Claimed under _gate; DequeueNext
skips cancelled, claims the frame it returns) so a cancelled write never
reaches the wire unless already claimed mid-write (documented residual).

WRK-25: add WriteBatchAsync; RunEventDrainLoopAsync submits the drained
event batch through it, so a burst of N events costs one flush not N.
IPC-30 oversized-event structured fault preserved via FindOversizedEvent.

WRK-24: reject a below-1024 negotiated frame maximum at the handshake
(MinNegotiableFrameBytes, matching GatewayOptionsValidator floor).

WRK-27: alarm poll advertises StaCallInProgress on the heartbeat snapshot
so the watchdog suppresses to the ceiling, not the grace.

Docs (WorkerFrameProtocol.md, MxAccessWorkerInstanceDesign.md) and the
2026-07-12 remediation registers/change-log updated in the same commit.
2026-08-07 07:50:38 -04:00
Joseph Doherty a55956ffa5 Merge branch 'fix/tst-30-runner-docs'
ci / java (push) Successful in 2m11s
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 1m38s
ci / portable (push) Failing after 17m43s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
2026-08-07 07:50:21 -04:00
Joseph Doherty aba22358f5 Merge branch 'fix/ipc-27-descriptor-test'
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m14s
ci / java (push) Successful in 2m8s
ci / portable (push) Failing after 4m45s
2026-08-07 07:49:33 -04:00
Joseph Doherty b604fed72b docs(TST-30): document shared-runner CI bottleneck + second-runner operator runbook
Doc half of TST-30 (single shared Gitea runner is a CI throughput/availability
bottleneck): docs/GatewayTesting.md's Continuous Integration section gains a
"Runner capacity is shared and finite" subsection covering the maxParallel=1
instance-level runner shared with dohertj2/lmxopcua, the ~20-30 min queue
latency observed under cross-repo contention, and Gitea 1.26's missing run
cancel/delete API. The existing "windev tier down" degraded-mode paragraph now
also covers "runner contended" as a reason to bypass the queue via
CI_SHA=<sha> scripts/ci/run-windev-ci.sh <mode> or the manual windev worktree
flow, generalizing it per the finding's design note.

New operator runbook docs/runbooks/TST-30-second-ci-runner.md carries the
actual runner registration (option a: second act_runner instance on
10.100.0.35 with the same container.network: traefik config, recommended;
option b: dedicated labelled runner, escalation only; option c: runner on
windev, rejected) plus verification steps and the no-cancel caveat. The
optional workflow-level concurrency group is documented as unverified --
framed as "verify before relying on it" -- and left unimplemented in ci.yml,
since registering the runner and any runs-on gating is operator/infra work
outside this repo's tree.

Tracking: TST-30 -> Done (doc half; runner registration operator-pending) in
both registers + change-log row.
2026-08-07 07:47:37 -04:00
Joseph Doherty 6060d21995 fix(IPC-27): close descriptor-freshness blind spots for enums, services, and Galaxy
ClientProtoInputTests.Descriptor_ContainsEveryContractMessageAndField only
compared messages and fields, and only enumerated the gateway/worker
descriptors, so a new enum value, a new RPC, or any galaxy_repository.proto-only
change would not redden the test even though it is documented as the primary
protoc-free CI gate.

Rename to Descriptor_ContainsEveryContractSymbol and extend the reflection walk
on both sides (published protoset and in-process contract) to also collect
enums/enum values ({enumFullName}, {enumFullName}/{valueName}) and
services/methods ({serviceFullName}, {serviceFullName}/{methodName}), and add
GalaxyRepositoryReflection.Descriptor to the enumerated files. The comparison
stays a flat, order-insensitive string-set diff with no protoc dependency.

Update docs/ClientProtoGeneration.md and docs/Contracts.md prose from
"message or field" to the full symbol coverage.

Red-path proof: pointed the test at the pre-IPC-01 stale protoset and confirmed
it failed naming max_frame_bytes, several MxCommandKind/AlarmProviderMode enum
values, MxAccessGateway/StreamAlarms and GalaxyRepository/BrowseChildren, and
the galaxy_repository.v1.* surface; restored the real path and re-ran green.

Flips IPC-27 to Done in the 2026-07-12 remediation tracker and register.
2026-08-07 07:47:18 -04:00
Joseph Doherty 1c2f3a62c1 Merge branch 'fix/sec-36-ldap-secret'
ci / windows-x86 (push) Successful in 1m16s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m31s
ci / portable (push) Failing after 4m37s
2026-08-07 07:44:43 -04:00
Joseph Doherty 0646c73e48 Merge branch 'fix/ipc-24-25-codegen'
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m15s
ci / java (push) Successful in 2m5s
ci / portable (push) Failing after 4m32s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
2026-08-07 07:42:35 -04:00
Joseph Doherty eacdd2d453 fix(IPC-23,IPC-24,IPC-25,IPC-32): proto-comment regen wave + codegen-freshness guards
Proto comments (comment-only, no wire change):
- mxaccess_worker.proto GatewayHello.max_frame_bytes: every worker->gateway frame
  must serialize within the negotiated max; reply builders truncate (IPC-23).
- mxaccess_gateway.proto DrainEventsReply: count-cap + byte-cap, drain-until-empty
  caller contract (IPC-23).
- mxaccess_gateway.proto ReplayGap.oldest_available_sequence: empty-ring value is
  highest-observed+1, oldest-1 resume formula stays valid (GWC-25 deferred amendment).

Regen wave: Contracts/Generated (C# XML doc), rust vendored protos (byte-copy),
Go bindings (worker binding was genuinely stale - lacked MaxFrameBytes entirely),
Python worker _pb2 (real descriptor delta), Java aggregates (javadoc, zero
protobuf-version churn under the pinned toolchain), client descriptor set.

IPC-24: pinned Java toolchain regenerates with no gencode-version churn, so the
unconditional churn-revert step in ci.yml is a fossil - deleted it; git diff is
now a true message-level drift gate for the single-file Java aggregates.

IPC-25: pin protoc-gen-go v1.36.11 / protoc-gen-go-grpc 1.6.2 in the Go generate
script (+ fix a latent pwsh-7 parse bug); add Check 4 to check-codegen.ps1
(regenerate Go+Python bindings, fail on diff, tool-missing fails not skips); add
the pinned-generator installs to the portable CI job.

IPC-32: relabel check-codegen banners 1/4..4/4 (folded into the Check 4 edit).

Docs: ClientProtoGeneration.md, Contracts.md, GatewayTesting.md, build.gradle
checkGeneratedClean caveat. Tracking: IPC-23/24/25/32 -> Done, GWC-25 proto note
resolved, change-log 2026-08-07.
2026-08-07 07:41:18 -04:00
Joseph Doherty 8c312c717c fix(SEC-36): scrub committed dev LDAP service-account password; add user-secrets channel + rotation runbook
Repo-side half of SEC-36. The appsettings.json plaintext was already discharged
before this branch (HEAD ships the fail-closed ${secret:ldap/mxgateway/bind}
store reference), so the residual leak was the literal value in glauth.md,
docs/GatewayTesting.md, and the historical archreview SEC-06 evidence -- all
scrubbed to <service-account-password> placeholders pointing at the source of
truth scadaproj/infra/glauth/.

- csproj: add <UserSecretsId>mxaccessgw-server</UserSecretsId> (dev channel)
- GatewayOptionsValidator: blank-password message now names both channels
  (dev user-secrets, deployed MxGateway__Ldap__ServiceAccountPassword)
- test: assert the message names both channels
- docs: GatewayConfiguration.md (three channels + rotation note), glauth.md
  (placeholders + rotation-required + runbook pointer), GatewayTesting.md
- new operator runbook docs/runbooks/SEC-36-ldap-credential-rotation.md
  (live rotation + NSSM staging remain operator-pending)
- tracking: SEC-36 -> Done (repo-side) in both registers + change-log

Deviation: kept the ${secret:} reference in appsettings.json rather than
deleting it (spec step 2 assumed the stale plaintext baseline); deleting it
would regress the shipped/documented/tested secret-store channel.

git grep -i for the old value is empty across all tracked files.
2026-08-07 07:40:41 -04:00
Joseph Doherty 10534ec906 docs(TST-27,WRK-26,CLI-42,CLI-43,IPC-28): P1 doc-drift batch, discharges IPC-29
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m17s
ci / java (push) Successful in 2m10s
ci / portable (push) Failing after 3m53s
TST-27: docs/GatewayConfiguration.md's ShowTagValues row no longer says
"Reserved" — it now states what false (default) does (DashboardEventBroadcaster
blanks tag values from a deep-cloned MxEvent before the SignalR events-hub
mirror), the security relevance (no per-session hub ACL yet, so this
redaction is the only thing between a low-trust Viewer and other sessions'
tag values), and the honest scope limit (does not cover /browse).

WRK-26 (discharges IPC-29): docs/MxAccessWorkerInstanceDesign.md's "Outbound
Queues" section rewritten from the stale five-level priority list to the
two-class Control/Event scheduler actually shipped, with the collapsed-
decision rationale, and the overflow paragraph rewritten to the implemented
fail-fast. docs/WorkerFrameProtocol.md gained a "Write Scheduling And
Sequencing" section describing HEAD truthfully: WRK-23's peek-stamp-commit
sequencing is live, WRK-25's event-batch flush coalescing is not (the drain
loop still awaits each event write individually), and WRK-22's cancellation
tombstone is not yet defined (noted as pending, not documented as shipped).

CLI-42: clients/rust/README.md and docs/ClientPackaging.md document the
vendored Rust proto layout matching build.rs — repo-path-first resolution
falling back to clients/rust/protos/, the check-codegen.ps1 Check 3 refresh
rule, and why cargo package/publish run without --no-verify.

CLI-43: docs/style-guides/JavaStyleGuide.md now says Java 17 (Ignition 8.3
baseline), mirroring CLI-12's wording, matching the shipped build.gradle.

IPC-28: docs/Grpc.md's exception-mapping prose gained CommandTooLarge ->
ResourceExhausted, and the Invoke section gained the oversized-payload
sentence, cross-referencing GatewayConfiguration.md's headroom rule.

Tracking: TST-27, WRK-26, CLI-42, CLI-43, IPC-28 flipped to Done and IPC-29
marked discharged-by-WRK-26 in 00-tracking.md and the 20/30/50/60 domain
registers, with a 2026-08-07 change-log entry.

Doc-only change; no source, proto, or test edits.
2026-08-07 07:26:58 -04:00
Joseph Doherty 97f79e79ef Merge remote-tracking branch 'origin/fix/wrk-21-drain-cluster'
ci / windows-x86 (push) Successful in 1m27s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m11s
ci / portable (push) Failing after 4m7s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
#	docs/MxAccessWorkerInstanceDesign.md
2026-08-07 07:18:58 -04:00
Joseph Doherty 758277bc62 docs(tracking): record WRK-21 review follow-ups (monotonic budget, guarded fallback) with windev evidence
ci / windows-x86 (push) Successful in 1m19s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m15s
ci / portable (push) Successful in 9m30s
2026-08-07 07:13:46 -04:00
Joseph Doherty 6bc3f9b991 fix(WRK-21): make drain budget monotonic at the reserve boundary; guard the reply-too-large fallback
ci / windows-x86 (push) Successful in 1m16s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m17s
ci / portable (push) Successful in 9m31s
Review follow-ups on the WRK-21 cluster.

1. ResolveDrainReplyByteBudget was a step function, not a floor: just above the
   64 KiB reserve the budget collapsed to a few bytes (at the validator-permitted
   floor MaxMessageBytes = 1024 + 64 KiB it was exactly 1024), too small to move a
   byte-heavy event, so DrainEvents truncated on every call and the drain-until-
   empty loop never terminated. It now takes the max of (frameMax - reserve) and
   frameMax/2, so the budget is monotonic and never below half the frame max. New
   test DrainEvents_AtValidatorFloorFrameMax_MakesProgressAndTerminates drives a
   byte-heavy queue at the exact validator floor and asserts it drains to empty
   with no head ever reported oversized.

2. The reply-too-large fallback write is now itself size-guarded
   (WriteReplyTooLargeFallbackAsync, used by both the control and STA reply seams):
   at a pathologically tiny negotiated max below the gateway's floor the fallback
   could also throw MessageTooLarge and — uncaught — kill the session, defeating the
   "no diagnostics command is session-fatal" invariant. It now log-and-swallows;
   comment notes WRK-24 adds the negotiated-max lower bound that makes it unreachable.

3. Corrected the RepeatedFieldOverheadBytes doc comments: WorkerEvent.CalculateSize()
   already includes the event's tag and length prefix (the same shape the reply's
   repeated events field packs), so the 8 bytes is pure slack over an already-
   conservative estimate, not compensation for a missing wrapper.
2026-08-07 07:09:13 -04:00
Joseph Doherty 34db678635 Merge branch 'fix/cli-40-41-44'
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m19s
ci / java (push) Successful in 2m14s
ci / portable (push) Failing after 4m12s
2026-08-07 07:08:27 -04:00
Joseph Doherty 0d874f91ee fix(CLI-40): scrub the credential from the redacted error's structured reply, route MXACCESS_FAILURE to MxAccess (Rust), fix Go Subscribe terminal-error drop
Code-review follow-up on the CLI-40/41/44 branch.

ISSUE 1 (all five, critical): the message-only scrub still leaked the
server-echoed credential through the redacted error's structured reply accessor
(.NET Reply/Statuses, Java reply()/protocolStatus(), Go MxAccessError.Reply via
errors.As, Rust reply()/into_reply(), Python raw_reply). The redacted error now
carries a scrubbed clone of the reply (protocol_status.message,
diagnostic_message, statuses[].diagnostic_text), with per-language tests asserting
the reply accessor no longer contains the credential.

ISSUE 2 (Rust, critical): ensure_command_success routed MXACCESS_FAILURE to
Error::Command (unlike the other four clients), bypassing attach_secrets and
leaking via derived Debug/Display. MXACCESS_FAILURE now routes to Error::MxAccess,
fixing the cross-client inconsistency.

ISSUE 3 (Go, important): the CLI-44 terminal send was unconditionally
non-blocking, dropping a genuine terminal error under a full buffer on the
never-drop SubscribeEvents path. It is now reserved-slot-non-blocking only for the
cancel-on-overflow path and blocking for the never-drop path.

New shared fixture authenticate-user.echoed-credential-mxaccess-failure.reply.json
wired into all five suites. Minors: whitespace-secret guard on .NET/Java redact
helpers; Java preserves exception subtype on redaction; redaction-helper unit
tests (Go/Java/.NET). Docs (ClientBehaviorFixtures.md, ClientLibrariesDesign.md)
updated to make the structured-field claim true.
2026-08-07 07:04:56 -04:00
Joseph Doherty c9925688f5 docs(tracking): record the WRK-21 cluster as Done with windev evidence
ci / java (push) Successful in 2m28s
ci / windows-x86 (push) Failing after 1m28s
ci / nightly-windev (push) Has been skipped
ci / portable (push) Successful in 8m38s
Change-log row for 2026-08-07: what landed for WRK-21/WRK-28/WRK-23/IPC-30, why
IPC-23 stays In progress (proto-comment/doc wave pending), and the verification
evidence — macOS NonWindows build + validator tests, and the documented windev
path (scripts/ci/windev-worker-ci.ps1 -Mode test) at a256560: x86 Worker build
clean, Worker.Tests 367 passed / 0 failed / 11 skipped.
2026-08-07 06:54:24 -04:00
Joseph Doherty 2aac29618e Merge branch 'fix/sec-33-34'
ci / java (push) Successful in 2m16s
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m25s
ci / portable (push) Successful in 8m32s
2026-08-07 06:51:31 -04:00
Joseph Doherty a2565604df test(WRK-21): keep the drain-to-empty walk inside the pipe harness envelope
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m14s
ci / java (push) Successful in 2m12s
ci / portable (push) Successful in 8m2s
PipePair runs both ends of a duplex pipe in one process with blocking
FlushFileBuffers under every frame write, so it wedges after roughly 85 large
round trips. Drain the full 10,000 byte-heavy events to empty at the queue layer,
where the no-loss property actually lives, and keep the pipe walk at 1,000 events
(29 replies) so it still proves the split end to end. Also give the truncation
test's budget slack: item handle 0 is a proto3 default and is not serialized, so
the probe measurement is a lower bound on the fixture's per-event cost.
2026-08-07 06:50:17 -04:00
Joseph Doherty 193daa9ee8 fix(SEC-33,SEC-34): address code review — missed docs, key-id guard comment, test consolidation
Same-commit docs rule (were missed in the prior commit):
- docs/GalaxyRepository.md: SnapshotCachePath now documents the per-OS derived
  default and the GalaxyRepositoryOptionsValidator rooting/validity enforcement.
- A2-galaxyrepository-adoption-handoff.md: correct the now-inaccurate NSSM caveat
  (SnapshotCachePath override is optional, not required; blank seeds a rooted host
  default, no silent no-op) and repoint the option-validation item at the new
  GalaxyRepositoryOptionsValidator.

SEC-34 guard confirmed and documented: TryParseKeyId's '_' split cannot truncate a
key id because both — and the only — gateway key-creation paths
(ApiKeyAdminCommandLineParser.IsValidKeyId, DashboardApiKeyManagementService.ValidateKeyId)
restrict key ids to IsAsciiLetterOrDigit || '.' || '-', and key ids are never
library-generated. Added a citing comment; no behavior change.

Test consolidation: moved the three host-start SqlitePath overrides into
TestHostEnvironmentInitializer (per-process temp store, mirroring Secrets__SqlitePath)
so future host-start tests auto-cover.
2026-08-07 06:49:24 -04:00
Joseph Doherty dc7fd16dd5 fix(CLI-40,CLI-41,CLI-44): exact-secret scrub, uniform malformed-reply contract, Go terminal-error mislabel
CLI-40: port the exact-secret credential scrub to Rust/Java/.NET (Go/Python
already did it). AuthenticateUser/WriteSecured(2) helpers now redact the exact
caller-supplied secret from any surfaced error, as defense-in-depth on top of the
by-construction guarantee. Rust hand-writes a redacting Debug (derived Debug would
leak the reply); Java/.NET rebuild the same exception type with the redacted
message and do not carry the secret-bearing original forward (so ToString/stack
traces stay clean too).

CLI-41: uniform malformed-reply contract for AuthenticateUser/ArchestrAUserToId/
AddBufferedItem across all five clients — typed payload, else a present int32
return_value, else a typed malformed-reply error. Fixes Go/Java silent-0, .NET
NRE, and Rust's own internal inconsistency.

CLI-44: the Go event goroutine's Recv-error path now uses a non-blocking
sendTerminalEventResult on the reserved slot, so a genuine terminal stream error
is reported as itself instead of being mislabeled ErrSlowConsumer under overflow.

Riders from the CLI-37/38 review: (a) .NET ToDiagnosticSummary and Python
_mxaccess_message surface the raw success member (diagnostics-only parity with
Rust); (b) the status-conversion fixture carries an independent wantSuccess
boolean and the Go/.NET fixture tests assert against it instead of recomputing
the formula under test.

Shared fixtures (authenticate-user.{echoed-credential,missing-payload,
return-value-only}.reply.json) + manifest + ClientBehaviorFixtures.md +
ClientLibrariesDesign.md updated in the same change. Tracking: CLI-40/41/44 -> Done.
2026-08-07 06:42:40 -04:00
Joseph Doherty 7e7f7cad84 fix(SEC-33,SEC-34): host-meaningful path rooting; verification-cache invalidate race
SEC-33: make rooting host-meaningful and stop shipping foreign-platform literals.
- Delete IsRootedForAnyPlatform; AddIfNotRooted now uses Path.IsPathRooted (current OS).
- Promote AddIfNotRooted/AddIfInvalidPath to shared GatewayConfigPathRules so the new
  Galaxy validator reuses them and the two validators cannot drift.
- Remove Authentication:SqlitePath and Galaxy:SnapshotCachePath Windows literals from
  appsettings.json; the CommonApplicationData-derived code defaults take over. The
  Galaxy default is seeded as a configuration value before AddZbGalaxyRepository
  (SnapshotCachePath is init-only, so a PostConfigure mutation cannot compile).
- New GalaxyRepositoryOptionsValidator (ValidateOnStart) enforces a valid, host-rooted
  SnapshotCachePath when PersistSnapshot is true.
- Root-cause the stray junk-named auth DB: host start eagerly builds
  AuthSqliteConnectionFactory; under the non-rooted Windows literal on macOS SQLite
  wrote it relative to the test bin CWD. The three real-host-start tests now pin
  SqlitePath to a temp path.

SEC-34: verification cache Invalidate-vs-in-flight-repopulation race closed with a
per-key generation counter (bump-before-evict, snapshot-then-recheck). The expiry
cap (window 2) takes the documented fallback: the library verification identity
carries no ExpiresUtc, so the cache cannot cap at the key's expiry (donor-library ask).

GWC-24 rider: cap MxGateway:Events:QueueCapacity at int.MaxValue/2 so the derived
checked(2 * EventChannelCapacity) in WorkerClient cannot overflow at session creation.

SEC-35 (doc-only): note IsProduction() env-name semantics in GatewayConfiguration.md.

Docs updated same commit (GatewayConfiguration.md, Authentication.md) and tracking
registers/change-log flipped (00-tracking.md, 40-security-dashboard.md).
2026-08-07 06:36:01 -04:00
Joseph Doherty 7c2eaf09e2 test(WRK-21): size the byte-heavy drain fixture for the pipe harness
ci / java (push) Successful in 2m12s
ci / portable (push) Successful in 8m23s
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 11m46s
PipePair has no continuous read pump — the test thread drains the pipe only
while it sits in ReadUntilAsync — so multi-megabyte DrainEvents frames
interleaved with the heartbeat loop wedge both ends inside FlushFileBuffers,
each waiting for the other to read. Negotiate a 128 KiB frame maximum instead:
the 10,000 byte-heavy events still overflow it many times over, so every
assertion (bounded reply, reported truncation, no event loss across repeated
drains, surviving session) is unchanged.
2026-08-07 06:25:15 -04:00
Joseph Doherty d2bb32d97b Merge branch 'fix/gwc-28-29-30-polish'
ci / java (push) Successful in 2m44s
ci / windows-x86 (push) Successful in 1m3s
ci / nightly-windev (push) Has been skipped
ci / portable (push) Successful in 17m13s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
2026-08-07 06:20:14 -04:00
Joseph Doherty 404f7cd993 docs(archreview): close GWC-28, GWC-29, GWC-30, TST-28
Flip the four findings to Done in the 2026-07-12 tracking registers
(Gateway core + Testing) and in the per-domain registers of
10-gateway-core.md and 60-testing-docs-gaps.md; append the 2026-08-07
change-log row recording what shipped, the pre-fix red for GWC-28, and
the TST-28 mutation check.
2026-08-07 06:15:45 -04:00
Joseph Doherty eeee3e48a3 fix(GWC-30): reuse the frame reader's length-prefix scratch buffer
ReadAsync allocated a fresh 4-byte array per inbound frame; the GWC-08
pass pooled the payload buffer but left the prefix. Replaced with a
per-instance scratch field — the reader is single-consumer by
construction (one read loop per WorkerClient, handshake reads complete
before the loop starts), so a per-instance buffer is safe and the
non-reentrancy that makes it safe is now stated on the class. Pooling
four bytes via ArrayPool would cost more than the allocation it saves.

Tests: WorkerFrameProtocolTests.ReadAsync_WithMultipleFramesOnOneReader_
ParsesEveryFrame reads five frames of differing payload length through
one reader, so a stale prefix carried between calls would misparse.
2026-08-07 06:15:39 -04:00
Joseph Doherty a044f92c5d fix(GWC-29): drop the wasted request clone on the Invoke hot path
Invoke deep-cloned the whole MxCommandRequest — including its command
payload, potentially a large bulk-write graph — only to overwrite the
cloned command with commandToInvoke and discard it. MapCommand then did
the one clone actually needed. Net cost: a full wasted command deep-clone
per Invoke, worst for exactly the bulk writes that are largest.

Adds a MapCommand(MxCommand) overload (MapCommand reads nothing else off
the request) and has Invoke pass commandToInvoke directly; the request
overload delegates so other callers are untouched.

The remaining clone inside MapCommand stays and is now documented as
required rather than incidental: commandToInvoke may be the gRPC-owned
request.Command, and the caller reads it again after dispatch via
TrackCommandReply, so ownership transfer (à la GWC-07) is not safe here.
That clone is what keeps WorkerClient.CreateCommandEnvelope's no-aliasing
invariant true.

Tests: MxAccessGrpcMapperTests.MapCommandFromCommandClonesPayload
(mutating the input leaves the mapped command untouched; both overloads
produce equal results under a fixed TimeProvider).
2026-08-07 06:15:31 -04:00
Joseph Doherty f27eb28063 fix(GWC-28): stamp gateway envelope sequence at write, not construction
CreateEnvelope stamped Sequence with an interlocked increment when the
envelope was built, so two concurrent InvokeAsync callers could take 1
and 2 and then enqueue in the order 2, 1 — non-monotonic on the wire,
breaking gateway.md's "monotonic per sender" contract. Benign today
(neither side validates inbound sequence, old GWC-10 still open) but it
would fault healthy sessions the moment worker-side validation lands.

WriteLoopAsync now stamps immediately before _writer.WriteAsync. It is
the outbound channel's single consumer (SingleReader = true), so wire
order and stamp order are the same thing by construction and
_nextSequence drops to a plain ulong with no interlocking. This mirrors
the worker's WRK-04 fix, which the gateway half never received.

Also adds TST-28: a [Theory] pinning that GatewayHello.MaxFrameBytes
carries the configured worker-frame maximum (default + 2 MiB override).
The adoption half is asserted only in the Windows-only worker suite, so
a regression to sending 0 — "older gateway, use default" to the worker —
would silently downgrade the negotiated IPC-02 limit with every CI test
still green. Mutation-checked (hard-coded 0 fails both cases).

Tests: WorkerClientTests.ConcurrentInvokesEmitStrictlyIncreasingSequences
OnTheWire (32 parallel invokes; failed 3/3 pre-fix) and
.StartAsync_SendsGatewayHelloWithConfiguredMaxFrameBytes.
2026-08-07 06:15:20 -04:00
Joseph Doherty 3f854d6cbf Merge branch 'fix/sec-31-32-limiter'
ci / windows-x86 (push) Successful in 1m21s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m23s
ci / portable (push) Successful in 9m4s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
2026-08-07 06:13:14 -04:00
Joseph Doherty 5b681ee59b fix(SEC-31,SEC-32): identify a probe-slot reservation by version, not by timestamp
ReleaseProbe recognised its own reservation by comparing NextProbeAtTicks to
now + _probeIntervalTicks. RecordInto's rearm-on-trip writes that identical
expression, so a concurrent RecordFailure on the same WindowState whose `now`
lands on the claimer's tick — routine at ~1 ms clock resolution under load — was
mistaken for the caller's own claim. The release then stomped the legitimate
fresh re-arm back to the stale previousProbeAtTicks, which is already due, handing
the next arrival a free probe the re-arm had just closed.

WindowState gains a monotonic ProbeVersion bumped by every writer of
NextProbeAtTicks (TryConsumeProbe's claim and RecordInto's re-arm alike).
TryConsumeProbe returns the stamp it set as part of a ProbeClaim; ReleaseProbe
restores the previous value only while the state's version still equals that
stamp, checking and restoring in one lock(state) section and bumping the version
again on restore so no other stale release can match either.

Test: ProbeSlotRestore_DoesNotStompConcurrentRearmAtSameTick, with the clock held
still so the claim and the interleaved failure necessarily share a tick. Making it
deterministic needed a seam — the claim-to-release window is a few nanoseconds and
racing threads do not hit it (an earlier thread-based attempt passed against the
defective guard three runs out of three, and its end state was ordering-dependent
rather than correctness-dependent, so it was dropped rather than shipped as
theatre). The seam is an internal ProbeReleaseInterleaveHook, null in production,
costing one null check on the already-refused path. Verified as a genuine red
against the timestamp guard: Expected ThrottledByPeer, Actual ProbeAdmitted.
2026-08-07 06:10:14 -04:00
Joseph Doherty c836899d62 chore: untrack accidentally-committed agent worktree gitlinks; gitignore them
ci / java (push) Successful in 2m25s
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m17s
ci / portable (push) Successful in 8m41s
2026-08-07 06:09:34 -04:00
Joseph Doherty 9825c69d92 Merge branch 'fix/cli-45-credential-envvar'
ci / java (push) Successful in 2m51s
ci / windows-x86 (push) Successful in 1m21s
ci / nightly-windev (push) Has been skipped
ci / portable (push) Successful in 9m41s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
2026-08-07 06:09:03 -04:00
Joseph Doherty 9357ff2dd4 Merge branch 'fix/cli-37-38-conformance'
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m14s
ci / java (push) Successful in 2m14s
ci / portable (push) Successful in 9m13s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
2026-08-07 06:08:12 -04:00
Joseph Doherty 37cb3b0df8 fix(CLI-45): standardize the CLI credential env var and fail fast on empty passwords
All five client CLIs now share one credential contract for `authenticate-user`:
flags `--password` / `--password-env` (Go: `-password` / `-password-env`) with
default env `MXGATEWAY_VERIFY_PASSWORD`, resolution flag-then-env, and a resolved
credential that is missing *or empty* is a usage error naming the flag and the
variable. The value is never echoed and never reaches the wire.

Go and Java previously sent an empty credential when the variable was unset,
turning a misconfigured environment into a real MXAccess authentication attempt.
Go now returns the guard error before dialing; Java throws a picocli
ParameterException instead of falling back to "". Python's `--password-env`
gained the canonical default and its UsageError names the resolved variable.
Rust treats an empty flag or env value as missing, with the resolution extracted
into a testable `resolve_verify_user_password`. .NET adopts the canonical flags
and keeps `--verify-user-password`, `--verify-user-password-env`, and
MXGATEWAY_VERIFY_USER_PASSWORD as deprecated aliases for one release.

Docs same commit: CrossLanguageSmokeMatrix.md gains the credential contract and
the per-CLI subcommand-coverage table (the documented-not-fixed half of the
finding); all five READMEs name the canonical variable and the fail-fast rule,
and the .NET README carries the deprecation note. Tracking flipped to Done in
both remediation registers with a change-log row.

No .proto changed; no generated code regenerated.
2026-08-07 06:05:00 -04:00
Joseph Doherty 6092172694 Merge branch 'fix/gwc-26-27-alarm-attach'
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m17s
ci / java (push) Successful in 2m4s
ci / portable (push) Successful in 7m8s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
#	archreview/2026-07-12/remediation/10-gateway-core.md
2026-08-07 06:02:17 -04:00
Joseph Doherty d6b2f24c3f fix(CLI-37,CLI-38): make status/HRESULT reply validation conformant across all five clients
One cross-client conformance pass; also closes first-cycle CLI-08.

CLI-37: an MxStatusProxy entry is a failure iff `category !=
MX_STATUS_CATEGORY_OK`. The proto contract has always said so — `success` is
the raw 16-bit COM member carried verbatim for diagnostics, not a boolean — but
four clients branched on `success` alone and .NET required both, so the same
gateway reply produced opposite verdicts per language. An absent entry stays
success; a present entry with an UNSPECIFIED category is a failure, because the
worker always maps a category and an unmapped one is not proven OK.

CLI-38: a reply fails on HRESULT iff `hresult` is present and negative, so
positive COM success codes such as S_FALSE (1) pass. .NET/Go/Java used `!= 0`,
which errored on a parity-preserving S_FALSE that Python and Rust accepted.
This makes the existing ClientLibrariesDesign.md claim true rather than
rewriting the doc to describe the divergence.

Four shared fixtures pin both rules cross-client, and each language suite also
carries a table test for the two edges a fixture cannot express (absent entry,
UNSPECIFIED category). A Java test fake that built a status with a bare
`setSuccess(1)` and no category is fixed — under the category rule that reply
was never a success.
2026-08-07 06:00:58 -04:00
Joseph Doherty 09ccd9561f docs(GWC-26): record alarm feed repairs as at-least-once; share the channel worker fake
Code-review follow-up on fix/gwc-26-27-alarm-attach.

ApplyReconcile's snapshot-derived feed repairs are at-least-once, not
exactly-once: a reconcile reads the worker's current state while the matching
live transition may still be buffered in the monitor's lease, so both broadcast
and the duplicates are indistinguishable on the alarm feed. This pre-dates the
acked-state delta — the Raise/Clear presence repair has always had it, since
nothing serializes a reconcile pass against the in-flight live stream — so
closing it (serialization or timestamp dedup) stays out of scope for a P2 fix.
Documented instead, with the consumer contract stated explicitly (apply
transitions idempotently, never as an increment or toggle):

- ApplyReconcile gains a "Delivery semantics" comment.
- gateway.md softens the "defense in depth" prose to state the semantics.
- docs/Sessions.md carries the same caveat on the alarm-feed description.
- Tracker change-log records it as a known pre-existing characteristic and a
  candidate finding for the next review cycle.

Also hoists the ChannelWorkerClient fake — duplicated across the three alarm
test files — into TestSupport/, dropping the usings it took with it.
2026-08-07 06:00:16 -04:00
Joseph Doherty acebe18773 fix(SEC-31,SEC-32): make probe admission atomic and stop Reset clearing a shared fallback partition
Two defects found in code review of the limiter rework.

Probe admission was check-then-act across two lock scopes: Check() read
"probe due" under lock(state), released it, then re-acquired to advance
NextProbeAtTicks. A burst of requests arriving together at an interval boundary
could therefore all observe the slot as due and all be admitted, handing the
verifier the very burst the interval exists to bound. The claim is now a single
critical section (TryConsumeProbe). The two layers are still claimed one at a
time — holding two per-state locks at once would need a global lock ordering to
stay deadlock-free — so a slot claimed on the composite partition is compensated
via ReleaseProbe when the aggregate then refuses, which otherwise silently spent
the partition's next slot and pushed the legitimate holder out by a full
interval.

Reset() removed whatever partition the caller resolved to, including the
address's shared fallback partition when the caller's key id had been collapsed
into it by the per-peer cap (or when the token was junk-shaped). That bucket also
carries failures contributed by other key ids from the same address, so one
successful authentication became a reset button for an in-progress spray. Reset
now clears only a partition the caller owns (effectiveKeyId == presented key id);
the shared bucket decays by window expiry instead, and the caller still recovers
through probe admission. The key's aggregate is cleared either way, as designed.

Also applied from the review: closure-free GetOrAdd overload on _partitions, and
a remarks paragraph acknowledging the best-effort O(n) eviction scan under
sustained overflow. Threading the resolved partition key from Check through to
RecordFailure/Reset was declined: Check resolves with mint:false and RecordFailure
with mint:true, and the two can legitimately differ when a concurrent caller fills
the per-peer cap in between — reusing Check's key would record into the wrong
partition and bypass the cap, which is not worth saving one string concat.

Tests (limiter suite 11 -> 14): ProbeAdmission_UnderConcurrentArrivals_
GrantsExactlyOneSlot (200 rounds x 8 barrier-released threads at the boundary),
ProbeAdmission_WhenAggregateRefuses_ReturnsTheClaimedPeerSlot, and
Reset_WithOverCapKeyId_DoesNotClearSharedFallbackPartition. The latter two were
confirmed as genuine reds against the unfixed code; the concurrency test is a
guard — it is deterministically green on the fixed structure but did not
reproduce the original nanosecond-wide window on its own.
2026-08-07 05:57:26 -04:00
Joseph Doherty 1a75f61ebe test(GWC-26): deflake ApplyReconcileBroadcastsAcknowledgeDelta
Seeding the cache with a live Raise transition raced the first reconcile: when
the reconcile snapshot populated the cache first, the still-buffered live Raise
was applied — and broadcast — after the test's feed subscriber had registered,
so the exactly-one-transition assertion saw two. Seed through a reconcile pass
instead (forced by a provider-mode probe, as the acked step already did) so no
live transition is ever in flight. Verified: 5 consecutive full alarm-monitor
runs green, and the test still fails (timeout) with the ApplyReconcile
acked-delta branch removed.
2026-08-07 05:49:34 -04:00
Joseph Doherty cf66ebbcfb Merge branch 'fix/gwc-25-replaygap-trio'
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m10s
ci / windows-x86 (push) Successful in 1m30s
ci / portable (push) Successful in 7m39s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
#	archreview/2026-07-12/remediation/10-gateway-core.md
2026-08-07 05:49:13 -04:00
Joseph Doherty 44b8e37900 fix(GWC-25,CLI-35,CLI-36): make the empty-ring ReplayGap resumable end to end
An empty replay ring reported oldest_available_sequence = 0 even when gap was
true. Clients follow the documented after_worker_sequence = oldest - 1 formula,
so an unsigned client computed ulong.MaxValue: the follow-up resume replayed
nothing, reported no gap, and the live filter dropped every subsequent event —
a silently dead stream in the headline detach-and-resume scenario, reachable on
default config once ReplayRetentionSeconds (300) age-evicts the ring.

GWC-25: SessionEventDistributor.RegisterWithReplay's empty-ring branch now
reports _highestSequenceSeen + 1 — the next sequence that can possibly be
delivered — when gap is true, so oldest - 1 lands exactly on the highest
observed sequence and the resume delivers everything newer. Still 0 when there
is no gap, where the field is meaningless and never emitted. Nothing is lost:
the evicted interval was unrecoverable either way, and the sentinel's job is to
say "re-snapshot".

CLI-35: the Python CLI fed every stream item into MessageToDict, which raised on
the ReplayGap dataclass and aborted the command after consuming the stream. A
new _event_row helper renders a gap as {"replayGap": {...}} — the same camelCase
shape the Rust CLI emits — and leaves proto events on the existing path.

CLI-36: the Go CLI formatted result.Event on every row, but the library
deliberately clears Event on a gap, so text mode printed
"0 MX_EVENT_FAMILY_UNSPECIFIED" and JSON mode an empty object, discarding the
resume cursors. The loop now branches on result.IsReplayGap() and renders the
typed row in both modes, counting it toward -limit like any other row. The JSON
row's cursors are typed by hand rather than marshalled with protojson: the
proto3 JSON mapping renders 64-bit integers as strings ("7") while the Rust and
Python CLIs emit numbers (7), so going through protojson would have made Go the
only canonical CLI with a different value type.

Docs in the same change: docs/Sessions.md documents the empty-ring sentinel
value and that oldest - 1 is the universal resume formula in both the retained
and fully-evicted cases; docs/CrossLanguageSmokeMatrix.md gains a per-CLI
gap-rendering table covering both client findings, and records exactly what is
and is not comparable across CLIs (same keys and numeric cursors for Rust/Go/
Python; quoted cursors for .NET/Java; differing key order, whitespace, and
container), so a matrix runner compares parsed values rather than raw bytes.

Tests, all written red first and each reproducing its defect verbatim:
- SessionEventDistributorTests: RegisterWithReplayReportsNextDeliverableSequence
  WhenRingEmptiedByAge, ...WithRetentionDisabled, and
  ResumeUsingSentinelFormulaAfterEmptyRingGapDeliversLiveEvents.
- GatewayEndToEndReconnectReplayTests.ReconnectAfterFullAgeEvictionResumesWith
  SentinelFormula — fake-worker e2e resume walk on a fake clock; the fixture now
  takes a retention window and a TimeProvider.
- clients/python test_stream_events_renders_replay_gap.
- clients/go TestRunStreamEventsPrintsReplayGap.

GWC-25's ReplayGap.oldest_available_sequence proto-comment amendment is
deliberately deferred to the later codegen wave (see the tracker change log): it
is comment-only but triggers the full five-client regen fan-out.
2026-08-07 05:46:00 -04:00
Joseph Doherty 59a76da70b Merge branch 'fix/gwc-24-staging-bound'
ci / java (push) Successful in 3m4s
ci / nightly-windev (push) Has been skipped
ci / portable (push) Successful in 8m49s
ci / windows-x86 (push) Failing after 6m59s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
2026-08-07 05:41:32 -04:00
Joseph Doherty 3b6a239ed6 fix(GWC-26): attach the alarm monitor's lease before SubscribeAlarms
RunMonitorAsync issued SubscribeAlarms and the first reconcile before the
internal distributor subscriber was attached (via ISessionManager
.ReadAlarmEventsAsync). The pump has been running since MarkReady started the
dashboard mirror and only fans to subscribers registered at fan-out time, so
every transition raised in that two-round-trip window bypassed the alarm feed —
and a missed Acknowledge was never repaired, because ApplyReconcile broadcast
presence deltas only.

- The monitor now takes the internal lease directly from its session BEFORE
  SubscribeAlarms and drains it after the first reconcile; window transitions
  buffer in the lease's bounded channel. Processing them after ApplyReconcile is
  order-safe (ApplyTransition handles alarms the snapshot already placed).
- ISessionManager.ReadAlarmEventsAsync removed — zero remaining callers.
- ApplyReconcile broadcasts an Acknowledge feed transition when a both-present
  alarm's state advanced to ActiveAcked. This is a feed-level repair on the
  AlarmFeedMessage/StreamAlarms surface rebuilt from the worker's own snapshot,
  not MxEvent emission, so the "never synthesize events" rule is untouched;
  the reasoning is recorded on ApplyReconcile.

The alarm-monitor test fakes now hand the monitor a real Ready GatewaySession
with a dashboard mirror, which is what makes the window reproducible.

Docs: docs/Sessions.md and gateway.md alarm-monitor ordering notes.

Refs: archreview/2026-07-12/remediation/10-gateway-core.md GWC-26
2026-08-07 05:40:33 -04:00
Joseph Doherty df710e18a9 fix(SEC-31,SEC-32): re-partition the API-key failure limiter on (peer, key id) with probe admission
The gRPC auth failure limiter partitioned on the key id parsed out of the
*unauthenticated* token and rejected with ResourceExhausted before VerifyAsync
ran. Key ids are not secret — they ride in every token and are listed on the
dashboard — so any network peer could send 10 garbage-secret requests per minute
and deny that key indefinitely: the legitimate holder's correct secret was
refused before it was ever checked, and the success-path Reset that would clear
the block sat behind the verification the block prevented (SEC-31). The tracked
map was also flushable — any `a_b_c`-shaped junk minted a fresh partition (the
`mxgw` literal was never compared), so ~4096 throwaway tokens evicted a blocked
entry and reset the window (SEC-32).

ApiKeyFailureLimiter moves from IsBlocked/RecordFailure/Reset(string peer) to a
partition-pair API: Check/RecordFailure/Reset(ApiKeyThrottlePartition) with an
ApiKeyThrottleDecision result. Two layers share one sliding window — a composite
(transport peer, key id) partition at ApiKeyFailureLimit, and a per-key-id
aggregate across all peers at the new ApiKeyFailureAggregateLimit (default 30)
that bounds a source-rotating sprayer. An over-limit state is now a valve rather
than a wall: one request per the new ApiKeyFailureProbeIntervalSeconds (default
5) is admitted through to the real verifier, so the correct secret always reaches
the constant-time compare and resets both layers. Guarantees preserved: guessing
stays bounded per window, and the failure path still spends no store read per
attempt.

SEC-32 rides the same change set: the interceptor validates token shape (literal
`mxgw` prefix, >= 3 non-empty `_` segments, key id <= 64 chars) before minting a
key-id partition, each transport peer may mint at most 32 of them before the
overflow collapses onto its fallback partition, and eviction prefers fully
expired windows and never drops an over-limit partition below a 2x transient
overshoot ceiling. Throttled attempts increment mxgateway.auth.throttled, tagged
stage=peer|aggregate only — /metrics is unauthenticated (open SEC-14), so no key
material may appear there.

Docs in the same commit: GatewayConfiguration limiter rows plus the two new keys,
the Authentication hot-path paragraph, the Authorization SEC-11 section, and the
limiter / SecurityOptions XML remarks (the old NAT rationale described the
defective keying). Tracking rows flipped to Done with a change-log entry.

Tests: new ApiKeyFailureLimiterTests (11) covering window pruning, composite vs
aggregate trip points, probe cadence, absolute-block mode, reset across both
layers, junk-spray eviction resistance, the per-peer cap, and expired-window
eviction preference; GatewayGrpcAuthorizationInterceptorTests gains the four
SEC-31 contract tests plus NonMxgwToken_FallsBackToTransportPeerPartition (20
total); GatewayOptionsValidatorTests covers both new keys including 0 as a
supported disable value (66 total).
2026-08-07 05:39:10 -04:00
Joseph Doherty 33ba612ddd fix(WRK-21,WRK-28,WRK-23,IPC-30): byte-budget the DrainEvents reply, stop size errors from killing sessions
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m8s
ci / portable (push) Successful in 7m41s
ci / windows-x86 (push) Failing after 12m32s
WRK-21 — DrainEvents was bounded by event count only, so a byte-heavy queue
(large string/array MxValues) built a reply above the negotiated frame maximum:
the writer rejected the frame, the exception unwound the session, and the events
already dequeued were destroyed. The drain is now byte-budgeted inside the queue
lock, so an event is dequeued only once it is known to fit and one that does not
stays at the head. Truncation is reported through the reply's existing
DiagnosticMessage (no contract change); callers drain until an empty reply. Both
reply-write seams — the control-command path and ProcessCommandAsync — now catch
MessageTooLarge and answer the correlation with an InvalidRequest reply instead
of unwinding or faulting the session. Satisfies IPC-23 R1-R3.

WRK-28 — the 10,000 drain ceiling moves to GatewayContractInfo
.MaxDrainEventsPerCommand, referenced by both the gateway request validator and
the worker clamp, replacing a comment-only sync contract. C# const only; no
.proto change.

WRK-23 — WorkerFrameWriter now peek-stamps, validates, then commits the sequence
counter immediately before the stream write, so a per-frame rejection leaves no
phantom gap on the wire.

IPC-30 — an oversized event frame stays session-fatal (it is undeliverable end to
end and neither dropping nor synthesizing a replacement is allowed), but the
death is structured: the event's identity and sizes are logged (never its value),
a WorkerFault with category PROTOCOL_VIOLATION and command method EventDrain is
written, then the session exits as before.

Docs updated in the same change: MxAccessWorkerInstanceDesign.md (drain byte cap,
truncation contract, oversized-head behavior, oversized-event policy, no control
reply is session-fatal on size), WorkerFrameProtocol.md (reply pre-sizing,
non-fatal reply-size rule, oversized-event policy, rejected frames do not consume
sequence numbers), gateway.md (DrainEvents two-axis bound).
2026-08-07 05:38:23 -04:00
Joseph Doherty d4154e340c fix(GWC-24): bound the worker event staging channel and unify the depth gauge
The GWC-04 remediation decoupled the read loop from event backpressure by
staging events into an unbounded channel, so its TryWrite always succeeded and
the only overflow fault was a single timed WriteAsync exceeding
EventChannelFullModeTimeout. A consumer draining slower than the worker
produces — each individual write still completing inside the window — therefore
grew gateway memory without bound, without a fault, and without a metric: the
queue-depth gauge counted only the bounded consumer channel, so staged events
were invisible.

Bound _eventStaging at 2 x EventChannelCapacity (Wait, single reader/writer, no
synchronous continuations). A rejected staging TryWrite is the sustained
slow-drain signal and faults the client ProtocolViolation with
QueueOverflow("worker-event-staging"), guarded by IsTerminalState() so a
completed channel during shutdown stays a silent drop. SetFaulted is
non-blocking, so the read loop still never awaits behind events. The timed-write
fault is unchanged and still catches the full-stall case earlier.

Move the queue-depth increment from EnqueueWorkerEventAsync to StageWorkerEvent
so the single counter reports total undelivered events (staged + queued); the
decrement at consumer read was already correct. No new configuration key: the
bound is derived, and gateway-side buffering per session is now at most
3 x MxGateway:Events:QueueCapacity. Coordination with still-open GWC-21
(EventChannelFullModeTimeout configurability) remains open and was not blocked
on.

Tests: StagingChannelOverflowFaultsWorkerWithoutWaitingForFullModeTimeout (5-min
full-mode timeout so only the staging bound can fire; asserts an interleaved
command reply still completes) and WorkerEventQueueDepthGaugeCountsStagedEvents.
Docs updated in the same change: GatewayProcessDesign, MxAccessWorkerInstanceDesign,
GatewayConfiguration, Metrics. GWC-24 flipped to Done in both trackers.
2026-08-07 05:35:07 -04:00
Joseph Doherty 1a63fdd7db fix(GWC-27): gate AttachInternalEventSubscriber on session readiness
AttachInternalEventSubscriber ran EnsureDistributorCreated / Register /
StartPumpIfRequested with no state check, unlike AttachEventSubscriber. A
premature attach would start the pump against a not-yet-Ready worker; the pump
source throws SessionNotReady, PumpAsync completes every subscriber with that
error and latches the distributor, and _eventDistributorStarted is never reset —
so the session would reach Ready with permanently dead event streaming.

Mirror AttachEventSubscriber's gate: check _state/_workerClient.State under
_syncRoot and throw SessionManagerException(SessionNotReady) before the
distributor is created, keeping the distributor calls outside the lock.

Refs: archreview/2026-07-12/remediation/10-gateway-core.md GWC-27
2026-08-07 05:28:35 -04:00
Joseph Doherty ddb382c137 fix(TST-29): retire oldtasks.md; delete root docs-review artifacts
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m21s
ci / java (push) Successful in 2m5s
ci / portable (push) Successful in 7m31s
Migrate the durable session-resilience governance record (Phase 5
orphan-worker reattach deferred-not-planned, EnableOrphanReattach
does not yet exist, settled Phase-4 Viewer-default decision) from
oldtasks.md into a new "Session-Resilience Epic Scope" entry in
docs/DesignDecisions.md, repoint CLAUDE.md and stillpending.md's
oldtasks.md references to the new home / tasks.json, and git rm
oldtasks.md now that it has no unique content left. Flip TST-29 to
Done in the archreview tracking registers.

The five untracked root docs-review artifacts (MxAccessGateway-docs-*,
MxGatewayClient-docs-*) are absent from this worktree; they must be
deleted from the main working tree separately (gitignored, no repo
impact).
2026-08-07 05:25:42 -04:00
Joseph Doherty ead921cace docs: truth sweep — Galaxy adoption, Auth 0.1.5, redaction seam, resolved A2 caveats
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m16s
ci / java (push) Successful in 3m21s
ci / portable (push) Successful in 8m21s
Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1
2026-08-07 01:57:01 -04:00
Joseph Doherty 47c0b646a9 fix(logging): redact command values on the shared ILogRedactor seam
ci / windows-x86 (push) Failing after 18s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m13s
ci / portable (push) Successful in 7m6s
GatewayLogRedactor.RedactCommandValue had no production caller. Its only
one was GatewayLogRedactorAdapter on the abandoned
feat/adopt-zb-telemetry-serilog branch; when that work was re-implemented
on main as GatewayLogRedactorSeam the identity half was carried over and
the command-value half was not. Four unit tests kept the policy green, so
it read as wired while masking nothing.

No live leak today — no log statement currently emits a CommandValue
property — but the next one to do so would have written credential-bearing
MXAccess payloads (AuthenticateUser, WriteSecured, WriteSecured2) to every
sink in the clear, with passing tests suggesting otherwise.

Ports the missing half onto the seam: a non-null CommandValue is masked via
the existing policy, gated on CommandMethod. Value logging stays off — the
seam exposes no opt-in — so ordinary values are masked too, matching
RedactCommandValue's default. A null value stays null rather than becoming
the placeholder, and the property is never invented when absent.

Five tests added, three of which were red first on the leak itself
(operator01:hunter2 reaching the assertion unmasked). The other two pin
the null and absent guards. Identity redaction is untouched.

Full NonWindows suite: 785 pass, 45 pre-existing macOS NamedPipe-harness
failures unchanged from baseline (verified by stashing this change).
Build 0 warnings.
2026-07-27 17:01:04 -04:00
Joseph Doherty aecc50a14b docs(claude): add Sister Projects section + cross-repo index propagation rule
ci / java (push) Successful in 2m2s
ci / windows-x86 (push) Failing after 29s
ci / nightly-windev (push) Has been skipped
ci / portable (push) Successful in 7m20s
2026-07-27 15:25:14 -04:00
Joseph Doherty 2d54ace5d9 chore(health): bump ZB.MOM.WW.Health to 0.2.0
Family version-matrix alignment. No behaviour change — mxgw registers no Akka
checks, and 0.2.0's per-entry `data` object is emitted only when a check
publishes some, so its health payloads are byte-identical.

Note: this repo uses inline package pins, NOT central package management (there is
no Directory.Packages.props), so the version lives in the Server csproj.

Verified: Server builds 0 warnings.
Part of scadaproj docs/plans/2026-07-22-overview-dashboard-impl-plan.md Task 2.3.
2026-07-24 05:54:40 -04:00
Joseph Doherty 8f7ee492ba chore(secrets): bump to Secrets 0.2.3 - visible delete modal (scadaproj#2)
ci / windows-x86 (push) Successful in 1m17s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m29s
ci / portable (push) Successful in 7m59s
0.2.3's Secrets.Ui ships ConfirmDeleteModal's own styles under
collision-proof zb-secrets-* class names. This host links no Bootstrap so
it never exhibited the invisible-modal defect, but it takes the fixed
line for parity; also rides over 0.2.1/0.2.2 (Akka-replicator fixes -
inert here, no replicator in use). Tests: 780 pass, 45 fail on macOS both
before and after the bump (NamedPipeServerStream multi-instance is
Windows-only - the fake-worker pipe harness cannot run on this platform);
zero delta from the bump.

Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
2026-07-19 01:22:23 -04:00
197 changed files with 11526 additions and 1096 deletions
+22 -11
View File
@@ -60,7 +60,19 @@ jobs:
dotnet tool install --global PowerShell
echo "$HOME/.dotnet/tools" >> "$GITHUB_PATH"
# IPC-01 / IPC-19 / IPC-20: descriptor set + Contracts/Generated must match the current protos.
# IPC-25 Check 4 regenerates the Go and Python client bindings and diffs them, so the pinned
# generators must be present. protoc 34.1 is already installed above; Go and Python are set up
# above. Pin protoc-gen-go / protoc-gen-go-grpc to match the committed header stamps and grpcio
# -tools to match the committed _pb2 stamp, or Check 4 false-fails (or masks drift) under churn.
- name: Install pinned client codegen generators (Check 4)
run: |
go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.6.2
echo "$(go env GOPATH)/bin" >> "$GITHUB_PATH"
python -m pip install 'grpcio-tools==1.80.0'
# IPC-01 / IPC-19 / IPC-20 / IPC-25: descriptor set + Contracts/Generated + Go/Python bindings
# must match the current protos.
- name: Codegen / descriptor freshness
shell: pwsh
run: ./scripts/check-codegen.ps1
@@ -93,10 +105,12 @@ jobs:
python -m pytest
java:
# Java client runs on a JDK-17 Linux runner (the macOS dev box has no JRE). The protobuf gradle
# plugin rewrites MxaccessGateway.java with spurious protobuf-runtime-version churn on every
# build; when no .proto changed, revert that one file so checkGeneratedClean / a dirty tree does
# not fail the build (repo memory project_java_generated_churn).
# Java client runs on a JDK-17 Linux runner (the macOS dev box has no JRE). The grpc/protobuf
# toolchain is fully pinned (clients/java/build.gradle: grpcVersion 1.76.0 / protobufVersion
# 4.33.1), so a regeneration is byte-identical to the committed aggregates modulo real .proto
# changes — `Verify generated tree is clean` (git diff) is the true drift gate (IPC-24). The
# single-file Java aggregates are where message-level proto drift lands, so this job now catches
# a .proto edited without regenerating and committing the Java client.
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
@@ -115,13 +129,10 @@ jobs:
- name: Gradle test
working-directory: clients/java
run: gradle test
- name: Revert spurious protobuf-version churn (no .proto changed)
# Both generated aggregates can pick up protobuf-runtime-version churn on regen; revert
# both so verify-clean still catches a real, uncommitted proto/codegen change elsewhere.
run: |
git checkout -- clients/java/src/main/generated/main/java/mxaccess_gateway/v1/MxaccessGateway.java || true
git checkout -- clients/java/src/main/generated/main/java/mxaccess_worker/v1/MxaccessWorker.java || true
- name: Verify generated tree is clean
# IPC-24: the pinned grpc/protobuf toolchain regenerates byte-identical output, so this
# git-diff gate now catches message-level proto drift in the single-file Java aggregates
# (the old unconditional churn-revert step masked exactly that class and was deleted).
run: git diff --exit-code -- clients/java/src/main/generated
windows-x86:
+3
View File
@@ -152,3 +152,6 @@ generated-scratch/
*-docs-issues.md
*-docs-fixed.md
*-docs-final.md
# Agent worktrees (subagent isolation) — never commit
.claude/worktrees/
+25 -16
View File
@@ -156,8 +156,11 @@ accept a `browseSubtreeGlobs` param, so either fix is small plumbing:
Delete mxaccessgw's own `Galaxy/GalaxyRepositoryServiceCollectionExtensions.cs` registrations.
4. **Option validation** — the shared lib **binds only, ships no validator** (deliberate). mxaccessgw
already validates Galaxy options via `Configuration/GatewayOptionsValidator.cs` — **keep that**; it
stays the owner of fail-fast validation, exactly as HistorianGateway's `ConfigPreflight` does.
stays the owner of fail-fast Galaxy validation. **Updated 2026-08-07 (SEC-33):** this is now a dedicated
`Configuration/GalaxyRepositoryOptionsValidator.cs` (registered as `IValidateOptions<GalaxyRepositoryOptions>`
with `ValidateOnStart`), which enforces a valid, host-rooted `SnapshotCachePath` when `PersistSnapshot`
is true — exactly as HistorianGateway's `ConfigPreflight` does. (The original handoff pointed at
`GatewayOptionsValidator.cs`, but that validator does not see the lib-bound `GalaxyRepositoryOptions`.)
5. **Health check** — keep mxaccessgw's existing Galaxy-SQL readiness check; read the connection
string from the same `MxGateway:Galaxy` section the lib binds (HistorianGateway does this with a raw
@@ -189,20 +192,26 @@ be **deleted**. **Keep** the mxaccessgw-specific ones that exercise behavior the
## Post-adoption notes / caveats
- **Deployment config (NSSM):** the deployed services (`MxAccessGw` on 10.100.0.48; the wonder host) read
config from **NSSM environment variables, not `appsettings.json`**. The lib's `SnapshotCachePath` default
is empty (persistence no-ops). `appsettings.json` sets `MxGateway:Galaxy:SnapshotCachePath` +
`PersistSnapshot`, but the deployments must carry `MxGateway__Galaxy__SnapshotCachePath` and
`MxGateway__Galaxy__PersistSnapshot` in their NSSM env on redeploy, or snapshot persistence silently
no-ops in production.
- **Pre-existing NU1903 (unrelated):** adding the package surfaced a transitive `SQLitePCLRaw.lib.e_sqlite3`
2.1.11 advisory (GHSA-2m69-gcr7-jv3q, no upstream patch) that breaks the build under `TreatWarningsAsErrors`
— already red on `main`. Resolved with a targeted `NuGetAuditSuppress` in `src/Directory.Build.props`
(its own commit). Remove the suppression once a patched e_sqlite3 ships.
- **Pre-existing IntegrationTests break (unrelated, NOT fixed here):** `IntegrationTests/WorkerLiveMxAccessSmokeTests.cs`
constructs `EventStreamService` with 6 ctor args, but a prior event-stream refactor reduced that ctor — so
the IntegrationTests project does not compile (already broken on `main`, independent of Galaxy). The Galaxy
live tests there were rebound to the lib and compile in isolation, but the project won't build until that
unrelated call site is fixed. Track separately.
config from **NSSM environment variables, not `appsettings.json`**. **Updated 2026-08-07 (SEC-33):** the
lib's own `SnapshotCachePath` default is empty (would no-op persistence), but mxaccessgw no longer relies
on it. `appsettings.json` no longer sets `SnapshotCachePath` at all; instead the gateway seeds a
`CommonApplicationData`-derived default (`C:\ProgramData\MxGateway\galaxy-snapshot.json` on the Windows
hosts) when the bound value is blank, and `GalaxyRepositoryOptionsValidator` fails startup if
`PersistSnapshot` is true with a non-rooted/invalid path. So `MxGateway__Galaxy__SnapshotCachePath` in the
NSSM env is now **optional** (an override), not required — a deployment that omits it gets the rooted host
default and persistence works; it no longer silently no-ops. `MxGateway__Galaxy__PersistSnapshot` still
governs whether persistence runs at all.
- **Pre-existing NU1903 (unrelated) — ✅ RESOLVED (2026-07-18, commit `2f0cfe3`):** adding the package surfaced a transitive `SQLitePCLRaw.lib.e_sqlite3`
2.1.11 advisory (GHSA-2m69-gcr7-jv3q, at the time no upstream patch) that breaks the build under `TreatWarningsAsErrors`
already red on `main`. Initially resolved with a targeted `NuGetAuditSuppress` in `src/Directory.Build.props`
(its own commit). The patched e_sqlite3 (2.1.12) has since shipped: the suppression was **removed** and the
patched native lib pinned intentionally (`src/Directory.Build.props` now documents this in place of the suppression).
- **Pre-existing IntegrationTests break (unrelated, NOT fixed here) — ✅ RESOLVED since:** `IntegrationTests/WorkerLiveMxAccessSmokeTests.cs`
constructed `EventStreamService` with 6 ctor args, but a prior event-stream refactor reduced that ctor — so
the IntegrationTests project did not compile (already broken on `main`, independent of Galaxy). The call site
has since been fixed to match the 3-arg ctor (`sessionManager, options, metrics` —
`Grpc/EventStreamService.cs:11-14`; call site at `WorkerLiveMxAccessSmokeTests.cs:~1580`) and the project compiles
(verified 2026-08-07).
- **No republish needed for the lib test additions:** the browse-projector / deploy-notifier / refresh-service
tests were added to the lib AFTER 0.2.0 was published; tests aren't shipped, so 0.2.0 is unchanged.
+37 -1
View File
@@ -15,6 +15,42 @@ The architecture is a two-process design — read `gateway.md` before making str
The worker must do all MXAccess COM calls on its dedicated STA thread, and the STA loop must pump Windows messages (`MsgWaitForMultipleObjectsEx` + `PeekMessage`/`DispatchMessage`) so MXAccess events deliver. A plain blocking queue on an STA is not enough.
## Sister Projects (scadaproj umbrella)
`mxaccessgw` is one of a family of related SCADA / OT / Wonderware / OPC UA **sister
projects** cloned as sibling directories under `~/Desktop/`. They are **separate repos
and separate processes**, coupled at runtime over wire protocols (gRPC + OPC UA) — not
by project/compile references — and share the `ZB.MOM.WW.*` product namespace. **mxaccessgw
is the linchpin**: the only component that loads 32-bit MXAccess COM, so the others depend
on it over gRPC rather than touching COM directly.
- **`~/Desktop/scadaproj`** ([`../scadaproj/CLAUDE.md`](../scadaproj/CLAUDE.md)) — the
umbrella/index workspace that aggregates the whole family (purpose, location, stack,
primary commands per project) and hosts the shared `ZB.MOM.WW.*` libraries, the dev/test
GLAuth (`10.100.0.35:3893`, source of truth `scadaproj/infra/glauth/`), and the shared
**`ZB.MOM.WW.GalaxyRepository`** package this gateway consumes. **This repo is indexed
there** — see the MxAccessGateway entry in `../scadaproj/CLAUDE.md`.
- **`~/Desktop/OtOpcUa`** (`lmxopcua`, [`../OtOpcUa/CLAUDE.md`](../OtOpcUa/CLAUDE.md)) — OPC
UA server whose in-process `GalaxyDriver` **depends on this gateway** for live Galaxy
read/write/subscribe and Galaxy Repository browse.
- **`~/Desktop/ScadaBridge`** ([`../ScadaBridge/CLAUDE.md`](../ScadaBridge/CLAUDE.md)) —
distributed SCADA platform whose Data Connection Layer has a dedicated **MxGateway
adapter** that talks to this gateway directly (native MxAccess data + A&C alarms),
bypassing OtOpcUa.
- **`~/Desktop/HistorianGateway`** (`historiangw`,
[`../HistorianGateway/CLAUDE.md`](../HistorianGateway/CLAUDE.md)) — single-process gRPC
sidecar **patterned on this gateway** (session model, dashboard shell, auth interceptor,
Galaxy SQL browse) but with no COM / no x86 worker. It also consumes the shared
`ZB.MOM.WW.GalaxyRepository` package — the same library mxaccessgw adopted (2026-06-25),
so the `galaxy_repository.v1` wire contract is served from one implementation.
**Propagate cross-repo changes to the umbrella index.** When a fact the index records about
mxaccessgw changes here — remote/push status, the `.proto` contracts (`mxaccess_gateway.proto`,
`mxaccess_worker.proto`, `galaxy_repository.proto`), the two-process architecture, shared-lib
consumption, or per-project commands — update the **MxAccessGateway entry in
[`../scadaproj/CLAUDE.md`](../scadaproj/CLAUDE.md)** in the same change so the umbrella index
never drifts from this repo. (Mirrors the same rule in the peer repos.)
## Build, Test, Run
```powershell
@@ -76,7 +112,7 @@ powershell -ExecutionPolicy Bypass -File scripts/run-client-e2e-tests.ps1
- **Style guides** in `docs/style-guides/` are authoritative. Follow `CSharpStyleGuide.md` for gateway/worker/.NET-client code: file-scoped namespaces, `sealed` by default, `Async` suffix on Task-returning methods, MXAccess-aligned names (`MxStatusProxy`, `ServerHandle`, `ItemHandle`, `HResult`).
- **MXAccess parity is the contract.** Don't "fix" surprising MXAccess behavior (e.g., `WriteSecured` failing before a value-bearing NMX body, distinct `OperationComplete` semantics, invalid-handle exceptions) unless the client explicitly opts into a non-parity mode. The installed MXAccess COM component is the baseline.
- **Don't synthesize events.** The gateway forwards only events the worker emits; it never invents `OperationComplete` from write completion or command replies.
- **One worker per session** (invariant). Multi-subscriber event fan-out and reconnect-with-replay have shipped and are config-gated: `AllowMultipleEventSubscribers` (default `false`) enables fan-out up to `MaxEventSubscribersPerSession` (default `8`); `DetachGraceSeconds` (default `30`) retains a session after its last subscriber drops so clients can reconnect; `ReplayBufferCapacity` / `ReplayRetentionSeconds` control how much event history the replay ring keeps. Default config is single-subscriber (`AllowMultipleEventSubscribers` off), but detach-grace and replay retention are **on** by default (`DetachGraceSeconds=30`, `ReplayBufferCapacity=1024`, `ReplayRetentionSeconds=300`): a detached session is retained for 30 s and recent events are buffered for reconnect. The reconnect protocol is consumable end-to-end: a resuming `StreamEvents` (via `after_worker_sequence`) that predates the retained ring gets a `ReplayGap` sentinel, and all five official clients surface it as a typed signal. Orphan-worker reattach after a gateway restart is **deferred, not planned** — see `oldtasks.md` (session-resilience epic Phase 5); the invariant on the next line stands. See `docs/DesignDecisions.md` and `docs/Sessions.md`.
- **One worker per session** (invariant). Multi-subscriber event fan-out and reconnect-with-replay have shipped and are config-gated: `AllowMultipleEventSubscribers` (default `false`) enables fan-out up to `MaxEventSubscribersPerSession` (default `8`); `DetachGraceSeconds` (default `30`) retains a session after its last subscriber drops so clients can reconnect; `ReplayBufferCapacity` / `ReplayRetentionSeconds` control how much event history the replay ring keeps. Default config is single-subscriber (`AllowMultipleEventSubscribers` off), but detach-grace and replay retention are **on** by default (`DetachGraceSeconds=30`, `ReplayBufferCapacity=1024`, `ReplayRetentionSeconds=300`): a detached session is retained for 30 s and recent events are buffered for reconnect. The reconnect protocol is consumable end-to-end: a resuming `StreamEvents` (via `after_worker_sequence`) that predates the retained ring gets a `ReplayGap` sentinel, and all five official clients surface it as a typed signal. Orphan-worker reattach after a gateway restart is **deferred, not planned** — see `docs/DesignDecisions.md` (Session-Resilience Epic Scope, session-resilience epic Phase 5); the invariant on the next line stands. See `docs/DesignDecisions.md` and `docs/Sessions.md`.
- **Gateway restart does not reattach orphan workers.** The first version terminates orphaned workers on startup; do not design code paths that assume reattachment.
- **No Blazor UI component libraries.** Dashboard uses local Bootstrap CSS/JS only — do not introduce MudBlazor, Radzen, FluentUI, etc.
- **Don't log secrets or full tag values by default.** API keys, passwords, `WriteSecured` payloads, and `AuthenticateUser` credentials must never reach logs. Value logging is opt-in and redacted.
@@ -14,7 +14,7 @@
| SEC-06 | Done | **Partial** | Production hard-stop verified: `GatewayOptionsValidator.cs:161-165` (`Transport==None` in Production → startup error). Docs: `docs/GatewayConfiguration.md:244-248` (env-var override `MxGateway__Ldap__ServiceAccountPassword` documented). | **The committed dev service-account password is still in the repo at `appsettings.json:29`** (`Ldap.ServiceAccountPassword`) and has not been rotated — the doc itself says it "should be rotated". The transport guard shipped; the credential-removal/rotation half of the remediation did not. → **SEC-36** |
| SEC-07 | Done | **Yes** | `Security/Authorization/GatewayGrpcScopeResolver.cs:23` (`QueryActiveAlarmsRequest => GatewayScopes.EventsRead`); both tests now construct the real type (`Tests/Security/Authorization/GatewayGrpcAuthorizationInterceptorTests.cs:330,350`). | |
| SEC-08 | Done | **Yes** | `Security/Authentication/CachingApiKeyVerifier.cs` (15 s success-only TTL cache keyed on SHA-256 of the presented token, `:96-120`; only successes cached `:110-117`); `Security/Authentication/CoalescingMarkApiKeyStore.cs:76-113` (≤1 `last_used` write/key/60 s); wired as decorators in `Security/Authentication/AuthStoreServiceCollectionExtensions.cs:89-98`; invalidation on dashboard revoke/rotate/delete at `Dashboard/DashboardApiKeyManagementService.cs:104,144,190`; per-call constraints JSON deserialize removed via blob cache (`Security/Authentication/GatewayApiKeyIdentityMapper.cs:22-45`). Tests exist (`Tests/Security/Authentication/CachingApiKeyVerifierTests.cs`). | New surface reviewed in depth — see SEC-34 (staleness/race, bounded) below. |
| SEC-10 | Done | **Yes** | CLI: `--expires` parsed as relative `<N>d`/`<N>h` or absolute ISO-8601 with `AssumeUniversal|AdjustToUniversal` (`Security/Authentication/ApiKeyAdminCommandLineParser.cs:242-274`), threaded into `CreateKeyAsync` (`:85-117`). Dashboard: `DashboardApiKeySummary.cs:13` (`ExpiresUtc`), snapshot projection `Dashboard/DashboardSnapshotService.cs:277`, badge logic compares against `DateTimeOffset.UtcNow` with a 7-day "Expiring" warn window (`Dashboard/Components/Pages/ApiKeysPage.razor:474-497`; `StatusBadge.razor:12-13`). Verifier-side rejection is in the shared `ZB.MOM.WW.Auth.ApiKeys` 0.1.4 (documented `docs/Authentication.md:64-65`; not readable in this repo). | UTC semantics are correct end-to-end on the gateway side. Boundary note: `expiresAt <= now` shows Expired, and relative parse rejects signed values (`NumberStyles.None`). A just-expired key can still authenticate for ≤15 s via the verification cache — see SEC-34. |
| SEC-10 | Done | **Yes** | CLI: `--expires` parsed as relative `<N>d`/`<N>h` or absolute ISO-8601 with `AssumeUniversal|AdjustToUniversal` (`Security/Authentication/ApiKeyAdminCommandLineParser.cs:242-274`), threaded into `CreateKeyAsync` (`:85-117`). Dashboard: `DashboardApiKeySummary.cs:13` (`ExpiresUtc`), snapshot projection `Dashboard/DashboardSnapshotService.cs:277`, badge logic compares against `DateTimeOffset.UtcNow` with a 7-day "Expiring" warn window (`Dashboard/Components/Pages/ApiKeysPage.razor:474-497`; `StatusBadge.razor:12-13`). Verifier-side rejection is in the shared `ZB.MOM.WW.Auth.ApiKeys` 0.1.5 (pins bumped 0.1.4→0.1.5 in `e107019`, a transitive-dependency security fix only — the expiry enforcement is unchanged; documented `docs/Authentication.md:64-65`; not readable in this repo). | UTC semantics are correct end-to-end on the gateway side. Boundary note: `expiresAt <= now` shows Expired, and relative parse rejects signed values (`NumberStyles.None`). A just-expired key can still authenticate for ≤15 s via the verification cache — see SEC-34. |
| SEC-11 | Done | **Yes, with new defects** | Login: fixed-window per-remote-IP limiter policy (`Dashboard/DashboardEndpointRouteBuilderExtensions.cs:27-41`), applied to POST `/auth/login` (`:83`), registered + 429 (`GatewayApplication.cs:111-126`), middleware in pipeline (`GatewayApplication.cs:46`), test `Tests/Gateway/Dashboard/DashboardLoginRateLimitTests.cs`. gRPC: `Security/Authorization/ApiKeyFailureLimiter.cs` checked **before** the store read (`GatewayGrpcAuthorizationInterceptor.cs:72-77`), failure recorded `:89`, reset on success `:97`; interceptor test asserts the short-circuit (`GatewayGrpcAuthorizationInterceptorTests.cs:395-402`). | The limiter exists and is enforced, but its key-id partitioning creates an unauthenticated lockout DoS (**SEC-31**) and its LRU eviction is flushable (**SEC-32**). |
| SEC-12 | Done | **Yes** | `Dashboard/DashboardSessionAdminService.cs`: canonical `AuditEvent`s `dashboard-close-session`/`dashboard-kill-worker` (`:37-40`), written on Denied (`:65,147`), Success (`:89-96,171-178`), and every Failure arm (`:105,116,132,187,198,214`), category `SessionAdmin`, actor/remote/correlation captured (`:238-261`), via `IAuditWriter` (same store as API-key events). | Audit-event completeness is good: denied, not-found, faulted, and unexpected paths all emit. |
| SEC-20 | Done | **Yes** | `Metrics/GatewayMetrics.cs:374``_heartbeatFailuresCounter.Add(1)` with no `session_id` tag (rationale comment `:370-373`). | The in-memory per-session map remains dashboard-only. |
File diff suppressed because one or more lines are too long
@@ -8,19 +8,19 @@ This document turns the 2026-07-12 re-review's **new** Gateway Server Core findi
| ID | Sev | Tier | Eff | Dep | Status | Title |
|----|-----|------|-----|-----|--------|-------|
| GWC-24 | Medium | P1 | M | GWC-21 (coord) | Not started | Unbounded event staging channel: sustained slow drain grows memory silently and invisibly |
| GWC-25 | Medium | P0 | S | CLI-35/36 (coord) | Not started | Empty-ring ReplayGap sentinel carries `oldest_available_sequence = 0`, dead-streaming a compliant client |
| GWC-26 | Low | P2 | M | GWC-27 | Not started | Alarm monitor attaches its subscriber after SubscribeAlarms; window transitions bypass the feed, missed Acknowledge never repaired |
| GWC-27 | Low | P2 | S | GWC-26 | Not started | `AttachInternalEventSubscriber` bypasses the readiness gate; premature attach poisons the distributor permanently |
| GWC-28 | Low | P2 | S | GWC-10 (coord) | Not started | Gateway→worker envelope `sequence` stamped at creation, not at write — non-monotonic on the wire under concurrent invokes |
| GWC-29 | Low | — | S | — | Not started | `Invoke` deep-clones the entire request only to discard the cloned command |
| GWC-30 | Info | — | S | — | Not started | Frame reader allocates a fresh 4-byte length-prefix array per frame |
| GWC-24 | Medium | P1 | M | GWC-21 (coord) | Done | Unbounded event staging channel: sustained slow drain grows memory silently and invisibly |
| GWC-25 | Medium | P0 | S | CLI-35/36 (coord) | Done | Empty-ring ReplayGap sentinel carries `oldest_available_sequence = 0`, dead-streaming a compliant client |
| GWC-26 | Low | P2 | M | GWC-27 | Done | Alarm monitor attaches its subscriber after SubscribeAlarms; window transitions bypass the feed, missed Acknowledge never repaired |
| GWC-27 | Low | P2 | S | GWC-26 | Done | `AttachInternalEventSubscriber` bypasses the readiness gate; premature attach poisons the distributor permanently |
| GWC-28 | Low | P2 | S | GWC-10 (coord) | Done | Gateway→worker envelope `sequence` stamped at creation, not at write — non-monotonic on the wire under concurrent invokes |
| GWC-29 | Low | — | S | — | Done | `Invoke` deep-clones the entire request only to discard the cloned command |
| GWC-30 | Info | — | S | — | Done | Frame reader allocates a fresh 4-byte length-prefix array per frame |
Dependency notes: GWC-26 and GWC-27 both change the internal-subscriber attach path (`GatewaySession.AttachInternalEventSubscriber` and its `SessionManager`/alarm-monitor callers) — land GWC-27's readiness gate first (or in the same commit), then GWC-26's reorder, so the reordered monitor attach is proven against the gate. GWC-24 is the direct successor of the prior cycle's GWC-04 backpressure fix and raises the value of the still-open GWC-21 (making `EventChannelFullModeTimeout` configurable); GWC-28 is the gateway half of the worker's WRK-04 fix and must be coordinated with the still-open GWC-10 if inbound sequence enforcement is ever added. GWC-25 is server-complete on its own, but the end-to-end reconnect story also needs the client-domain CLI-35/36 fixes (Python CLI crashes on the sentinel, Go CLI destroys it).
---
## GWC-24 — Unbounded event staging channel: sustained slow drain grows memory silently and invisibly `Medium` · `P1`
## GWC-24 — Unbounded event staging channel: sustained slow drain grows memory silently and invisibly `Medium` · `P1` · **Done (2026-08-07)**
**Finding.** The GWC-04 remediation decoupled the read loop from event backpressure by staging events into `_eventStaging`, an **unbounded** channel (`Workers/WorkerClient.cs:93-100`). `StageWorkerEvent`'s `TryWrite` therefore always succeeds (`:565-573`), and the sustained-overflow `ProtocolViolation` fault fires only when a *single* timed `WriteAsync` against the bounded `_events` exceeds `EventChannelFullModeTimeout` (default 5 s, `:610-647`). The queue-depth gauge counts only `_events``_eventQueueDepth` is incremented in `EnqueueWorkerEventAsync` (`:616`, `:627`) and decremented in `ReadEventsCoreAsync` (`:289`), so staged-but-unqueued events are invisible to `SetWorkerEventQueueDepth`. The field comment (`:28-33`) claims staging "only fills during the bounded EventChannelFullModeTimeout window", which is true only for a full consumer stall, not for a consumer that drains slower than the worker produces while each individual write still completes inside the window.
@@ -61,7 +61,7 @@ Coordinate with (do not block on) open GWC-21: if `EventChannelFullModeTimeout`
The proto comment currently states "`oldest_available_sequence` itself IS still retained", which becomes false in the empty-ring case — per the docs-with-source rule, amend the field comment in the same commit to define the empty-ring value ("when nothing is retained, this is the next sequence that can be delivered — `highest observed + 1` — and the `oldest 1` resume formula remains valid; the interval evicted is unchanged"). This is a comment-only proto change (no descriptor delta), but the repo's codegen rules still apply — see the steps.
**Implementation.**
**Implementation.** (Code + `docs/Sessions.md` landed 2026-08-07 on `fix/gwc-25-replaygap-trio`; the deferred proto-comment amendment below **landed 2026-08-07** with the IPC-23 codegen wave on `fix/ipc-24-25-codegen` — GWC-25 is fully resolved.)
- `Sessions/SessionEventDistributor.cs:463-467`: replace `oldestAvailableSequence = 0;` with `oldestAvailableSequence = gap ? _highestSequenceSeen + 1 : 0;` plus a comment explaining the `oldest 1` client formula this must keep valid (cite this finding).
- `src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_gateway.proto` (`ReplayGap.oldest_available_sequence`, ~line 759): append the empty-ring sentence above. Then regenerate per repo rules: delete `src/ZB.MOM.WW.MxGateway.Contracts/Generated/*.cs`, `dotnet build src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csproj`, and **commit `Generated/`** (net48 worker builds break otherwise). Sync the vendored client copies of the proto byte-identical (`clients/*/`); a comment-only edit changes no descriptor, so: Python `*_pb2*` output is unchanged (comments are not embedded — regenerate with the pinned grpcio-tools only if the files actually differ), Go/C#/Rust generated doc comments will churn — regenerate those per each client README, and revert spurious Java aggregate-file churn if no message-level delta appears (per the established Java convention).
- `docs/Sessions.md` (~lines 228-234, ReplayGap section): document the empty-ring sentinel value and that `after_worker_sequence = oldest_available_sequence 1` is the universal resume formula in both the retained and fully-evicted cases.
@@ -16,14 +16,14 @@ members, no positional records). The worker builds and tests only on the Windows
| ID | Sev | Tier | Eff | Dep | Status | Title |
|----|-----|------|-----|-----|--------|-------|
| WRK-21 | Medium | P0 | M | IPC-23 (same defect, fix owned here); WRK-28 (same lines) | Not started | DrainEvents bound is count-based only; an oversized reply still kills the session and loses the drained events |
| WRK-22 | Low | — | S | IPC-26 (same defect, fix owned here) | Not started | Cancelled `WriteAsync` leaves its frame queued; it is still written later |
| WRK-23 | Low | — | S | WRK-21 (rejection path becomes backstop-only) | Not started | Rejected frames consume sequence numbers, producing wire gaps |
| WRK-24 | Low | — | S | — | Not started | `AdoptNegotiatedMaxMessageBytes` has no lower-bound sanity check |
| WRK-25 | Low | P2 | S | WRK-22 (both touch enqueue/dequeue) | Not started | WRK-12 flush coalescing never engages on the event hot path |
| WRK-26 | Low | P1 | S | WRK-23 (soft — sequence prose); discharges IPC-29 | Not started | Write-priority and overflow doc drift from the WRK-07 change |
| WRK-27 | Low | — | S | — | Not started | Alarm poll bypasses the watchdog's in-flight suppression (15 s vs 75 s) |
| WRK-28 | Low | — | S | WRK-21 (land in the same commit cluster) | Not started | 10,000 drain cap is a duplicated magic constant with a comment-only sync contract |
| WRK-21 | Medium | P0 | M | IPC-23 (same defect, fix owned here); WRK-28 (same lines) | Done | DrainEvents bound is count-based only; an oversized reply still kills the session and loses the drained events |
| WRK-22 | Low | — | S | IPC-26 (same defect, fix owned here) | Done | Cancelled `WriteAsync` leaves its frame queued; it is still written later |
| WRK-23 | Low | — | S | WRK-21 (rejection path becomes backstop-only) | Done | Rejected frames consume sequence numbers, producing wire gaps |
| WRK-24 | Low | — | S | — | Done | `AdoptNegotiatedMaxMessageBytes` has no lower-bound sanity check |
| WRK-25 | Low | P2 | S | WRK-22 (both touch enqueue/dequeue) | Done | WRK-12 flush coalescing never engages on the event hot path |
| WRK-26 | Low | P1 | S | WRK-23 (soft — sequence prose); discharges IPC-29 | Done | Write-priority and overflow doc drift from the WRK-07 change |
| WRK-27 | Low | — | S | — | Done | Alarm poll bypasses the watchdog's in-flight suppression (15 s vs 75 s) |
| WRK-28 | Low | — | S | WRK-21 (land in the same commit cluster) | Done | 10,000 drain cap is a duplicated magic constant with a comment-only sync contract |
---
@@ -12,16 +12,16 @@ All `path:line` citations were re-verified against the working tree at `4f5371f`
| ID | Sev | Tier | Eff | Dep | Status | Title |
|----|-----|------|-----|-----|--------|-------|
| IPC-23 | Medium | P0 | S¹ | WRK-21 | Not started | DrainEvents bound is count-based only; byte-heavy queue still builds a session-killing reply frame (contract requirements here; fix mechanics in WRK-21) |
| IPC-24 | Medium | P0 | S | — | Not started | CI's unconditional Java churn-revert masks real generated-code drift for message-level proto changes |
| IPC-25 | Medium | P0 | M | — | Not started | Committed Go/Python worker bindings are stale at HEAD; no guard covers them |
| IPC-26 | Low | P2 | S¹ | WRK-22 | Not started | Cancelled write leaves a ghost frame that is still written (contract requirement here; fix mechanics in WRK-22) |
| IPC-27 | Low | P2 | S | — | Not started | Descriptor freshness test blind to enums, enum values, services/methods, and the Galaxy contract |
| IPC-28 | Low | — | S | — | Not started | `docs/Grpc.md` omits the `CommandTooLarge``ResourceExhausted` mapping |
| IPC-29 | Low | — | S | — | Not started | Worker writer priority scheduling and write-time sequence stamping undocumented in the frame-protocol doc |
| IPC-30 | Low | P0 | M | WRK-21 (same file/batch) | Not started | Oversized worker→gateway event frame is session-fatal — make the death deliberate, structured, and diagnosable |
| IPC-23 | Medium | P0 | S¹ | WRK-21 | Done | DrainEvents bound is count-based only; byte-heavy queue still builds a session-killing reply frame (contract requirements here; fix mechanics in WRK-21) |
| IPC-24 | Medium | P0 | S | — | Done | CI's unconditional Java churn-revert masks real generated-code drift for message-level proto changes |
| IPC-25 | Medium | P0 | M | — | Done | Committed Go/Python worker bindings are stale at HEAD; no guard covers them |
| IPC-26 | Low | P2 | S¹ | WRK-22 | Done (mechanics landed in WRK-22) | Cancelled write leaves a ghost frame that is still written (contract requirement here; fix mechanics in WRK-22) |
| IPC-27 | Low | P2 | S | — | Done | Descriptor freshness test blind to enums, enum values, services/methods, and the Galaxy contract |
| IPC-28 | Low | — | S | — | Done | `docs/Grpc.md` omits the `CommandTooLarge``ResourceExhausted` mapping |
| IPC-29 | Low | — | S | — | Done (discharged by WRK-26) | Worker writer priority scheduling and write-time sequence stamping undocumented in the frame-protocol doc |
| IPC-30 | Low | P0 | M | WRK-21 (same file/batch) | Done | Oversized worker→gateway event frame is session-fatal — make the death deliberate, structured, and diagnosable |
| IPC-31 | Info | — | — | — | N/A | Gateway stamps sequence at creation, worker at write — accepted divergence; sequence is documented diagnostic-only (`gateway.md:328-330`); revisit only if sequence ever becomes load-bearing |
| IPC-32 | Info | — | S | IPC-25 | Not started | `check-codegen.ps1` check labels miscounted (folded into the IPC-25 script edit) |
| IPC-32 | Info | — | S | IPC-25 | Done | `check-codegen.ps1` check labels miscounted (folded into the IPC-25 script edit) |
¹ Effort for the work owned by *this* plan (proto comments + docs + acceptance criteria). The code mechanics are M and are tracked under WRK-21 / WRK-22 in the worker plan.
@@ -10,12 +10,12 @@ Repo rules that bind every entry: docs change in the same commit as the source (
| ID | Sev | Tier | Eff | Dep | Status | Title |
|----|-----|------|-----|-----|--------|-------|
| SEC-31 | Medium | P0 | M | — | Not started | Failure limiter partitions on attacker-controlled key id and blocks before verification (lockout DoS) |
| SEC-32 | Low | P0 | S | SEC-31 | Not started | Failure-limiter LRU is flushable by junk-token spray; token prefix never validated |
| SEC-33 | Low | P1 | M | — (co-locate SEC-23) | Not started | Any-platform path-rooting acceptance re-opens SEC-01 on Unix; Galaxy `SnapshotCachePath` unvalidated |
| SEC-34 | Low | P2 | S | — | Not started | Verification cache: expiry outlives TTL; `Invalidate` races in-flight repopulation |
| SEC-35 | Info | — | S | — | N/A (doc-only note) | Production hard-stops key on the exact `Production` environment name |
| SEC-36 | Low | P1 | M | cross-repo (`scadaproj/infra/glauth`) | Not started | Committed dev LDAP service-account password: remove from repo and rotate |
| SEC-31 | Medium | P0 | M | — | Done | Failure limiter partitions on attacker-controlled key id and blocks before verification (lockout DoS) |
| SEC-32 | Low | P0 | S | SEC-31 | Done | Failure-limiter LRU is flushable by junk-token spray; token prefix never validated |
| SEC-33 | Low | P1 | M | — (co-locate SEC-23) | Done | Any-platform path-rooting acceptance re-opens SEC-01 on Unix; Galaxy `SnapshotCachePath` unvalidated |
| SEC-34 | Low | P2 | S | — | Done | Verification cache: expiry outlives TTL; `Invalidate` races in-flight repopulation |
| SEC-35 | Info | — | S | — | N/A (doc-only note discharged 2026-08-07) | Production hard-stops key on the exact `Production` environment name |
| SEC-36 | Low | P1 | M | cross-repo (`scadaproj/infra/glauth`) | Done (repo-side; live rotation operator-pending per runbook) | Committed dev LDAP service-account password: remove from repo and rotate |
---
@@ -135,6 +135,8 @@ dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --fil
```
Post-run, verify no new `C:\*` file exists under any `bin/` (manual `find src -name 'C:*'`).
**Outcome (2026-08-07 — Done).** Implemented as designed. `IsRootedForAnyPlatform` deleted; `AddIfNotRooted`/`AddIfInvalidPath` promoted to a shared `GatewayConfigPathRules` internal helper (used by both validators) and now use `Path.IsPathRooted` (current OS). Both Windows literals removed from `appsettings.json`; the Galaxy `SnapshotCachePath` default is applied gateway-side via a configuration value seeded before `AddZbGalaxyRepository` (the package's `SnapshotCachePath` is **init-only**, so a `PostConfigure` mutation does not compile — deviation from the design's "PostConfigure default"; same effect). New `GalaxyRepositoryOptionsValidator` registered with `ValidateOnStart`. **Stray-file root cause:** starting the full host eagerly constructs `AuthSqliteConnectionFactory`, which creates the auth DB path; with the shipped Windows literal that path is non-rooted on macOS, so SQLite materialized `C:\ProgramData\MxGateway\gateway-auth.db` as a junk-named relative file under the test's `bin/` CWD (invisible to the hygiene test's bin/obj filter). After the literal removal the code default resolves under an unwritable `/usr/share` on macOS, so the three tests that start the real host (`GatewayApplicationTests.Build_MapsMetricsEndpoint`, `.StartAsync_InvalidGatewayConfiguration_FailsStartup`, `GatewayTlsBootstrapTests`) now pin `SqlitePath` to a temp path. No stray file remains (`find src -name 'C:*'` empty).
---
## SEC-34 — Verification cache: expiry outlives TTL; `Invalidate` races in-flight repopulation `Low` · `P2`
@@ -145,7 +147,7 @@ Post-run, verify no new `C:\*` file exists under any `bin/` (manual `find src -n
**Design.**
- **Expiry (window 2): eliminate, don't document.** The shared `ApiKeyIdentity` (0.1.4) carries `ExpiresUtc`; when caching a success, cap the entry lifetime at the key's expiry: `AbsoluteExpiration = min(now + ttl, ExpiresUtc)` (skip caching entirely if already ≤ now). A cached hit can then never outlive the key. Confirm during implementation that the library verifier populates `ExpiresUtc` on the returned identity; if it does not, fall back to documenting the ≤ TTL window in the remarks and `docs/Authentication.md` and file a donor-library ask.
- **Expiry (window 2): eliminate, don't document.** The shared `ApiKeyIdentity` (0.1.5 — bumped from 0.1.4 in `e107019`, no API change) carries `ExpiresUtc`; when caching a success, cap the entry lifetime at the key's expiry: `AbsoluteExpiration = min(now + ttl, ExpiresUtc)` (skip caching entirely if already ≤ now). A cached hit can then never outlive the key. Confirm during implementation that the library verifier populates `ExpiresUtc` on the returned identity; if it does not, fall back to documenting the ≤ TTL window in the remarks and `docs/Authentication.md` and file a donor-library ask.
- **Invalidate race (window 3): per-key generation check.** `ConcurrentDictionary<string, long> _generations`; `Invalidate(keyId)` increments the generation **before** evicting cache keys. `VerifyAsync` parses the key id from the token up front (same split the interceptor does — cheap, no store access), snapshots `g0` before calling the inner verifier, and after a success only `Set`s when the generation still equals `g0` — then re-reads the generation after the `Set` and self-evicts if it moved (bump-before-evict + set-then-recheck closes the remaining interleaving). Unparseable tokens skip caching already (`TryComputeCacheKey`).
- **CLI window (1): accept and keep documented** — cross-process invalidation is out of scope by design; the TTL is the backstop and the remarks already say so.
@@ -165,6 +167,8 @@ Post-run, verify no new `C:\*` file exists under any `bin/` (manual `find src -n
dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter "FullyQualifiedName~CachingApiKeyVerifier"
```
**Outcome (2026-08-07 — Done).** Window 3 (Invalidate race) implemented exactly as designed: per-key generation counter, bump-before-evict in `Invalidate`, snapshot-before-inner + set-then-recheck in `VerifyAsync`, key id parsed from the token up front. Covered by `Invalidate_DuringInFlightVerification_DiscardsStaleRepopulation`. **Window 2 (expiry cap) took the documented fallback**, not the cap: the design's confirmation step failed — the library verification identity (`ZB.MOM.WW.Auth.Abstractions.ApiKeys.ApiKeyIdentity`, the type on `ApiKeyVerification.Identity`) carries **no** `ExpiresUtc` (that property is on `ApiKeyRecord`, the store row, not the returned identity), so the cache cannot cap an entry at the key's expiry. Per the design's contingency, the ≤ TTL expiry window is documented in the class remarks and `docs/Authentication.md`, with a donor-library ask (surface expiry on the verification identity). Consequently the two expiry-cap tests (`CacheEntry_DoesNotOutliveKeyExpiry`, `AlreadyExpiredIdentity_IsNotCached`) are **not** added — they cannot be written against a type with no expiry field; window 1 (CLI) accepted and documented as before.
---
## SEC-35 — Production hard-stops key on the exact `Production` environment name `Info` · `—` (N/A: doc-only)
@@ -179,6 +183,8 @@ dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --fil
**Verification.** Doc-only; no build. Cross-read against `GatewayOptionsValidator.cs:23-27`.
**Outcome (2026-08-07 — discharged).** The documentation contract landed as a rider on the SEC-33/34 commit: `docs/GatewayConfiguration.md` gained a "Production hard-stops key on the exact environment name (SEC-35)" subsection stating that both hard-stops fire only on `IHostEnvironment.IsProduction()` (unset `ASPNETCORE_ENVIRONMENT` or the exact `Production` name) and that any other name keeps the permissive dev posture. No code change, as designed.
---
## SEC-36 — Committed dev LDAP service-account password: remove from repo and rotate `Low` · `P1` · cross-repo dependency
@@ -212,3 +218,5 @@ dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --fil
dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter "FullyQualifiedName~GatewayOptionsValidator"
```
(asserts the blank-password validation still fires with the updated message). Manual: with user-secrets set on the dev box, `dotnet run --project src/ZB.MOM.WW.MxGateway.Server/...` and a dashboard `/login` as `multi-role` succeeds against the rotated GLAuth; the deployed-host login re-check from step 1 counts as the production verification. Live-LDAP integration tests (`MXGATEWAY_RUN_LIVE_LDAP_TESTS=1`) only where the GLAuth instance is reachable; otherwise document skipped per the testing matrix.
**Outcome (2026-08-07 — Done, repo-side; live rotation operator-pending).** Landed on `fix/sec-36-ldap-secret`. **The design's baseline had already shifted:** at HEAD `appsettings.json` no longer commits the literal — it ships `"ServiceAccountPassword": "${secret:ldap/mxgateway/bind}"`, a fail-closed encrypted-store reference (documented `GatewayConfiguration.md:252`, tested by `PreHostSecretExpansionTests`) introduced by the Secrets-store adoption after this remediation was written. **Deviation from Implementation step 2:** the `${secret:}` reference was **kept, not deleted** — deleting it regresses the shipped/documented/tested store channel and the committed-plaintext finding is already resolved for `appsettings.json`. The load-bearing residual — the literal value still present in `glauth.md`'s samples (`:33,65,103,136,245`), `docs/GatewayTesting.md`, and the historical `archreview/*` SEC-06 evidence — was scrubbed to `<service-account-password>` placeholders, each with a pointer to the source of truth `scadaproj/infra/glauth/` and a rotation-required note. Steps 36 implemented as designed: `<UserSecretsId>mxaccessgw-server</UserSecretsId>` added (step 3); the `ValidateLdap` blank-password message now names both channels — dev `dotnet user-secrets set "MxGateway:Ldap:ServiceAccountPassword" <value>` and deployed `MxGateway__Ldap__ServiceAccountPassword` — plus a note on the `${secret:}` store default (step 4), asserted by the extended `Validate_Fails_WhenLdapEnabledAndServiceAccountPasswordBlank`; docs updated same commit (step 5); `git grep -i` for the old value is empty across tracked files (step 6). The cross-repo **step 1 (rotate GLAuth on `10.100.0.35`, pre-stage the NSSM env var on `10.100.0.48` and on `wonder-app-vd03` only if `Ldap.Enabled`, verify dashboard login)** is the operator's to execute, captured in the new runbook `docs/runbooks/SEC-36-ldap-credential-rotation.md`. Verification (macOS): `dotnet build …Server` 0 warnings/0 errors; `dotnet test --filter ~GatewayOptionsValidator` green.
+11 -11
View File
@@ -16,17 +16,17 @@ Operating constraints carried from prior work:
| ID | Sev | Tier | Eff | Dep | Status | Title |
|----|-----|------|-----|-----|--------|-------|
| CLI-35 | Medium | P0 | S | — | Not started | Python CLI `stream-events` crashes on a ReplayGap |
| CLI-36 | Medium | P0 | S | — | Not started | Go CLI `stream-events` silently destroys the ReplayGap signal |
| CLI-37 | Medium | P1 | M | CLI-38 | Not started | Status-array validation must branch on `category` per the proto contract (4-vs-1 divergence) |
| CLI-38 | Medium | P1 | S | — | Not started | Align .NET/Go/Java on `hresult < 0` — lands prior CLI-08 and cures the design-doc drift |
| CLI-39 | Medium | P1 | S | CLI-35..38, CLI-45 | Not started | Bump client versions off the already-published 0.1.2 before the next publish; add registry-collision guard |
| CLI-40 | Low | — | M | — | Not started | Port the exact-secret credential scrub to Rust/Java/.NET |
| CLI-41 | Low | — | M | — | Not started | Uniform malformed-reply contract for AuthenticateUser/ArchestrAUserToId/AddBufferedItem |
| CLI-42 | Low | P1 | S | — | Not started | Document the vendored Rust proto layout (CLI-02's missing doc half) |
| CLI-43 | Low | — | S | — | Not started | Java style guide still prescribes "Java 21 preferred" |
| CLI-44 | Low | — | S | — | Not started | Go event goroutine can mislabel a genuine terminal error as `ErrSlowConsumer` |
| CLI-45 | Low | P1 | M | — | Not started | Standardize CLI credential env-var name and fail fast on missing/empty passwords |
| CLI-35 | Medium | P0 | S | — | Done | Python CLI `stream-events` crashes on a ReplayGap |
| CLI-36 | Medium | P0 | S | — | Done | Go CLI `stream-events` silently destroys the ReplayGap signal |
| CLI-37 | Medium | P1 | M | CLI-38 | Done | Status-array validation must branch on `category` per the proto contract (4-vs-1 divergence) |
| CLI-38 | Medium | P1 | S | — | Done | Align .NET/Go/Java on `hresult < 0` — lands prior CLI-08 and cures the design-doc drift |
| CLI-39 | Medium | P1 | S | CLI-35..38, CLI-45 | Done | Bump client versions off the already-published 0.1.2 before the next publish; add registry-collision guard |
| CLI-40 | Low | — | M | — | Done | Port the exact-secret credential scrub to Rust/Java/.NET |
| CLI-41 | Low | — | M | — | Done | Uniform malformed-reply contract for AuthenticateUser/ArchestrAUserToId/AddBufferedItem |
| CLI-42 | Low | P1 | S | — | Done | Document the vendored Rust proto layout (CLI-02's missing doc half) |
| CLI-43 | Low | — | S | — | Done | Java style guide still prescribes "Java 21 preferred" |
| CLI-44 | Low | — | S | — | Done | Go event goroutine can mislabel a genuine terminal error as `ErrSlowConsumer` |
| CLI-45 | Low | P1 | M | — | Done | Standardize CLI credential env-var name and fail fast on missing/empty passwords |
Cross-domain dependencies: **CLI-35/CLI-36 pair with GWC-25** (gateway emits `oldest_available_sequence = 0` on an empty replay ring — the server-side half of the same reconnect story; the CLI fixes here are independently landable but the end-to-end resume walk in the smoke matrix needs both). **CLI-39 pairs with the publishing process** (`scripts/pack-clients.ps1`, `scripts/tag-go-module.ps1`, Gitea package registry).
@@ -12,10 +12,10 @@ Prior-cycle open findings (TST-05..24 where still open) are tracked in the prior
|----|-----|------|-----|-----|--------|-------|
| TST-25 | High | P1 | M | — (unlocks TST-05, TST-24) | Done | Windows/x86 test tier has zero automation — restore via SSH-driven windev CI job |
| TST-26 | Medium | P1 (folded into TST-25) | S | TST-25 | Done | docs/GatewayTesting.md, check-codegen.ps1, and ci.yml comments describe removed CI jobs |
| TST-27 | Medium | P1 (doc batch) | S | — | Not started | `ShowTagValues` config row still says "Reserved" after SEC-25 made the flag live |
| TST-28 | Low | P2 | S | relates IPC-02 | Not started | Gateway-side `max_frame_bytes` handshake field untested in the CI-run suite |
| TST-29 | Low | P2 | S | — | Not started | Retire `oldtasks.md` after folding the Phase-5 governance record into DesignDecisions.md; delete root docs-review artifacts |
| TST-30 | Low | P2 | M | — | Not started | Single shared Gitea runner is a CI throughput/availability bottleneck (cross-repo contention, no run cancel/delete) |
| TST-27 | Medium | P1 (doc batch) | S | — | Done | `ShowTagValues` config row still says "Reserved" after SEC-25 made the flag live |
| TST-28 | Low | P2 | S | relates IPC-02 | Done | Gateway-side `max_frame_bytes` handshake field untested in the CI-run suite |
| TST-29 | Low | P2 | S | — | Done | Retire `oldtasks.md` after folding the Phase-5 governance record into DesignDecisions.md; delete root docs-review artifacts |
| TST-30 | Low | P2 | M | — | Done (doc half; runner registration operator-pending per runbook) | Single shared Gitea runner is a CI throughput/availability bottleneck (cross-repo contention, no run cancel/delete) |
---
@@ -167,6 +167,8 @@ Independent of the runner count, document the **no-cancel** reality (Gitea 1.26
**Verification.** Push two branches back-to-back and confirm their runs execute concurrently (not serially) once a second runner exists; `GET /repos/dohertj2/mxaccessgw/actions/runners` (or the instance runner list) shows ≥2 runners online; `docs/GatewayTesting.md` describes the shared-runner/no-cancel reality and the bypass. Re-run the TST-25 acceptance push and confirm queue depth is materially lower under a concurrent `lmxopcua` run.
**Outcome (2026-08-07 — Done, doc half; runner registration operator-pending).** Landed on `fix/tst-30-runner-docs`. Implementation step 2 shipped: `docs/GatewayTesting.md`'s Continuous Integration section gained a "Runner capacity is shared and finite" subsection stating the `maxParallel=1` co-located runner is shared with `dohertj2/lmxopcua` at the instance level (not repo-scoped), the ~2030 minute queue latency observed under cross-repo contention, and the Gitea 1.26 no-cancel/no-delete API reality; the existing "windev tier down" degraded-mode paragraph now also covers "runner contended" as a reason to use the bypass, generalized per this finding's design note. New operator runbook `docs/runbooks/TST-30-second-ci-runner.md` carries **step 1** (register a second `act_runner` on `10.100.0.35`, option (a) recommended, same `container.network: traefik` config; option (b) dedicated labelled runner as an escalation; option (c) windev-hosted runner rejected) with the verification checklist (concurrent back-to-back pushes, `GET /repos/dohertj2/mxaccessgw/actions/runners` ≥ 2) and a note that the no-cancel reality persists regardless of runner count. **Step 3 (optional workflow-level `concurrency` group)** is documented in the runbook as unverified — explicitly framed as "verify this Gitea deployment honors it before relying on it" — and left unimplemented in `ci.yml`, since it is a `ci.yml` change out of scope for this doc-only pass. **The actual runner registration (step 1) is infrastructure work outside this repo's tree and remains the operator's to execute**, tracked in the runbook. Verification performed: `grep -n 'maxParallel\|shared\|cancel' docs/GatewayTesting.md` shows the new prose; runbook file exists at the path above; no build required (doc-only change).
---
## Cross-domain dependencies
@@ -0,0 +1,19 @@
# Candidate Findings for the Next Review Cycle (surfaced during 2026-07-12 remediation)
These were discovered while remediating the 2026-07-12 backlog but were **out of scope** for it — each is either pre-existing, by-design residual, or a new observation. They are recorded here (not fixed) so the next review cycle can triage them. None blocks the 2026-07-12 cycle, which is complete.
| ID (proposed) | Area | Severity (est.) | Summary |
|---|---|---|---|
| NEXT-01 | Testing / macOS | Low | Fake-worker/e2e gateway tests fail on macOS under the default `TMPDIR` because the `CoreFxPipe_mxaccess-gateway-{pid}-{sessionId}` path exceeds the 104-char Unix-domain-socket `sun_path` limit under `/var/folders/…/T/`. Workaround today is `TMPDIR=/tmp`. Fix options: shorten the pipe name, or document the `TMPDIR=/tmp` requirement in `docs/GatewayTesting.md`. Surfaced independently by multiple remediation agents. |
| NEXT-02 | Clients (.NET, Java) | Low | The .NET and Java CLIs render the raw `ReplayGap` sentinel `MxEvent` on `stream-events` instead of a typed gap row — Java text mode prints `0 MX_EVENT_FAMILY_UNSPECIFIED`. Same defect class as CLI-36 (Go) / CLI-35 (Python), which were fixed this cycle; the .NET/Java halves were out of scope. The cross-language smoke matrix now records this divergence honestly. |
| NEXT-03 | Gateway alarms | Low | `GatewayAlarmMonitor.ApplyReconcile` feed-repair broadcasts (the new acked-delta from GWC-26 **and** the pre-existing Raise/Clear repair) are **at-least-once, not exactly-once**: a periodic reconcile can synthesize a transition whose matching live transition is still buffered in the alarm lease, so both broadcast as indistinguishable duplicates on the alarm feed (StreamAlarms + dashboard hub). Pre-existing (the Raise/Clear repair always had it); GWC-26 documented the at-least-once contract rather than closing the race. Closing it needs reconcile/live serialization or a monotonic dedup marker. |
| NEXT-04 | Worker frame writer | Low | WRK-22/WRK-25 cancellation path: a frame `Claimed` by a concurrent lock-holder just before its caller's cancellation races in is never awaited by that caller; if the write then faults, `TrySetException` lands on a `Task` nobody observes (unobserved-task-exception). By-design residual, non-crash (no `UnobservedTaskException` handler registered), pre-existing to single-frame WRK-22 and amplified per-batch by WRK-25. Hygiene fix: attach a fault-observing continuation to abandoned/tombstoned frame completions. |
| NEXT-05 | Worker frame writer | Info | A batch whose remaining frames are tombstoned by cancellation leaves dead `PendingFrame` entries in `_eventFrames`/`_controlFrames` until a future `DequeueNext` pops and skips them. Same pre-existing behavior as single-frame WRK-22, amplified per-batch; in practice heartbeats purge them promptly, so not a real leak. |
## Operator actions still pending (from this cycle's runbooks)
These are **live-infrastructure actions the operator must execute** — the repo-side work is complete and merged:
- **SEC-36** — rotate the dev LDAP service-account credential per `docs/runbooks/SEC-36-ldap-credential-rotation.md` (generate new secret in `scadaproj/infra/glauth`, pre-stage the NSSM env var on deployed hosts, rotate GLAuth on `10.100.0.35`, verify dashboard login). The committed literal is gone from the working tree but remains recoverable from git history until rotation completes — **rotation is the load-bearing half.**
- **TST-30** — register a second Gitea `act_runner` on `10.100.0.35` per `docs/runbooks/TST-30-second-ci-runner.md` to relieve the single-shared-runner bottleneck.
- **TST-25 follow-ups** — old **TST-05** (scheduled live-MXAccess smoke) is now covered by the `nightly-windev` job; old **TST-24** (client wire tests in CI) is unblocked by the working Windows tier.
+1 -1
View File
@@ -49,7 +49,7 @@ Impact: logout (`Dashboard/DashboardEndpointRouteBuilderExtensions.cs:136-155`)
Recommendation: keep the lifetime short (or shorten to ~5 minutes given the factory refreshes per reconnect, `docs/GatewayDashboardDesign.md:497-499`), and confirm no request-path logging captures query strings (Serilog request logging is not currently enabled; keep it that way or scrub `access_token`).
**SEC-6 · Medium — LDAP is plaintext-by-default with a committed service-account password.**
Evidence: `src/ZB.MOM.WW.MxGateway.Server/Configuration/LdapOptions.cs:49-61` (defaults `Transport=None`, `AllowInsecure=true`, `ServiceAccountPassword = "serviceaccount123"`), `appsettings.json:21-33` (same values checked into the repo), `glauth.md:30,327` (dev LDAPS disabled; "binding sends passwords cleartext on the wire").
Evidence: `src/ZB.MOM.WW.MxGateway.Server/Configuration/LdapOptions.cs:49-61` (defaults `Transport=None`, `AllowInsecure=true`, `ServiceAccountPassword = "<service-account-password>"` — value redacted per SEC-36), `appsettings.json:21-33` (same values checked into the repo), `glauth.md:30,327` (dev LDAPS disabled; "binding sends passwords cleartext on the wire").
Impact: every dashboard login sends the operator's password in cleartext to `10.100.0.35:3893`, and the LDAP service-account credential is in source control. This is a documented dev posture (the shadow-options rationale at `LdapOptions.cs:20-28` is explicit that the shared library is secure-by-default), and the validator does enforce the `Transport=None ⇒ AllowInsecure` consistency rule (`GatewayOptionsValidator.cs:82-85`) — but nothing distinguishes dev from prod at runtime.
Recommendation: for production deployment docs, require `Transport=Ldaps`/`StartTls` + `AllowInsecure=false` and move `ServiceAccountPassword` to env-var/secret configuration; consider an `IsProduction` startup check mirroring SEC-4. LDAP injection risk is delegated to the shared `ZB.MOM.WW.Auth.Ldap` provider (bind-then-search per `Dashboard/DashboardAuthenticator.cs:41-47`); its escaping cannot be verified from this repo — flag for review in the donor repo.
+5 -5
View File
@@ -181,7 +181,7 @@ Full design + implementation for each row lives in the linked domain doc under i
| CLI-05 | Medium | — | S | — | Not started | .NET session cannot be re-attached to an existing session id |
| CLI-06 | Medium | — | S | — | Not started | .NET `DisposeAsync` blocks/throws on unreachable gateway |
| CLI-07 | Medium | — | S | — | Not started | .NET retry budget self-defeats on `DeadlineExceeded` |
| CLI-08 | Medium | — | S | CLI-03 | Not started | .NET/Go/Java treat any nonzero HRESULT as failure (should be `< 0`) |
| CLI-08 | Medium | — | S | CLI-03 | Done | .NET/Go/Java treat any nonzero HRESULT as failure (should be `< 0`) — landed via 2026-07-12 [CLI-38](../2026-07-12/remediation/50-clients.md#cli-38--align-netgojava-on-hresult--0-lands-prior-cli-08-cures-the-doc-drift---medium--p1) |
| CLI-09 | Medium | — | M | — | Not started | Go has no typed auth-error mapping (Unauthenticated vs PermissionDenied) |
| CLI-10 | Medium | — | M | — | Not started | Go uses deprecated `grpc.DialContext` + `grpc.WithBlock()` |
| CLI-11 | Medium | — | S | — | Not started | Go CLI cannot opt into strict TLS validation |
@@ -197,7 +197,7 @@ Full design + implementation for each row lives in the linked domain doc under i
| CLI-21 | Low | P2 | S | — | Done | Go `ClientVersion = "0.1.0-dev"` stale vs tagged releases |
| CLI-22 | Low | — | S | — | Not started | Go `newCorrelationID` swallows `crypto/rand` error → empty id |
| CLI-23 | Low | — | S | — | Not started | Go nil-vs-empty bulk short-circuit asymmetry |
| CLI-24 | Low | — | S | — | Not started | Java `MxEventStream` single-consumer constraint undocumented |
| CLI-24 | Low | — | S | — | Done | Java `MxEventStream` single-consumer constraint undocumented (closed 2026-08-07 per 2026-07-12 review old-tracker action; documented at MxEventStream.java:25 "Single consumer") |
| CLI-25 | Low | — | S | — | Not started | Java `close()` does not await channel termination |
| CLI-26 | Low | P2 | S | — | Done | Python `version.py` (0.1.0) ≠ `pyproject.toml` (0.1.2) |
| CLI-27 | Low | — | S | — | Not started | Python `Session.close()` not concurrency-safe; synthesizes reply |
@@ -207,7 +207,7 @@ Full design + implementation for each row lives in the linked domain doc under i
| CLI-31 | Low | — | M | — | Not started | Rust CLI is a single 2,699-line `main.rs` |
| CLI-32 | Low | — | S | — | Not started | Client-side bulk caps differ (.NET/Java unbounded) |
| CLI-33 | Low | — | S | CLI-01,13 | Not started | Per-language event backpressure semantics undocumented |
| CLI-34 | Low | — | S | — | Not started | Python `build/`/`.pytest_cache/` present on disk (untracked) |
| CLI-34 | Low | — | S | — | Done | Python `build/`/`.pytest_cache/` present on disk (untracked) (closed 2026-08-07 per 2026-07-12 review old-tracker action; both gitignored in clients/python/.gitignore) |
### Testing, docs & gaps — [60-testing-docs-gaps.md](60-testing-docs-gaps.md)
@@ -236,7 +236,7 @@ Full design + implementation for each row lives in the linked domain doc under i
| TST-21 | Low | — | S | — | Not started | Log rotation configured but minimal |
| TST-22 | Low | — | S | — | Not started | Config-shape JSON block omits documented keys |
| TST-23 | Low | P2 | S | — | Done | Bidirectional `Session` RPC never built |
| TST-24 | Low | P2 | M | TST-03 | Not started | Client wire behaviour has no automated verification |
| TST-24 | Low | P2 | M | TST-03 | Not started | Client wire behaviour has no automated verification. **Gate cleared:** TST-03 CI is Done (live and green 2026-07-10; Windows/x86 tier green 2026-07-13 via the TST-25/TST-26 SSH-driven windev job), so TST-24 is unblocked — deferred by choice now, not CI-gated |
## Cross-cutting clusters
@@ -276,7 +276,7 @@ Findings the review flagged as one coordinated design pass — sequence them tog
| 2026-07-09 | P1 Wave 3 (size/backpressure topology + write ordering). IPC-02/03/04 + WRK-04/07 → `Done`. **Size negotiation (IPC-02):** added `GatewayHello.max_frame_bytes` (regen `Generated/` + `clients/proto` descriptor refresh with pinned protoc 34.1); the gateway sends its negotiated worker-frame max and the worker adopts it (`WorkerFrameProtocolOptions.AdoptNegotiatedMaxMessageBytes`, 0 = keep default, >256 MiB rejected) instead of a hard-coded default. **Headroom (IPC-03):** the pipe frame max now sits `EnvelopeOverheadReserveBytes` (64 KiB) above the public gRPC cap (default `Worker.MaxMessageBytes` 16 MiB→16 MiB+64 KiB), cross-validated at startup; `WorkerClient` pre-checks command envelope size and fails only the offending correlation (`ResourceExhausted`) instead of `SetFaulted`ing the session. **Drain bound (IPC-04):** gateway request validator rejects `DrainEvents max_events` above 10 000; the worker caps each reply at `MaxDrainEventsPerReply` (10 000) and treats `max_events = 0` as that cap, not "drain all". **Sequence (WRK-04):** `WorkerFrameWriter` stamps the envelope `Sequence` at the point of writing under the write lock, so wire order and stamped sequence always agree under concurrent producers. **Priority (WRK-07):** the worker writer is now a cooperative priority scheduler — control frames (reply/fault/heartbeat/shutdown-ack) drain ahead of event frames; per-frame validation/size rejections fail only that frame, a stream failure fails all queued. Docs same-change (GatewayConfiguration, WorkerFrameProtocol, gateway.md). **Verified:** macOS NonWindows build clean + validator/grpc tests green; **windev** x86 worker builds clean, `Worker.Tests` 352 passed / 0 failed / 11 skipped (incl. new monotonic-sequence, control-before-event priority, negotiated-max, drain-bound tests), gateway `Tests` 799 passed / 3 failed — all 3 pre-existing windev-environmental (SelfSigned SAN + 2 `EventStreamServiceTests` timing, both pass in isolation). Commits `c8b3a22` (gateway half), `ebe6aea` (worker half), `309296f` (descriptor + default-expectation refresh). GWC-04 (event-channel decoupling) is the remaining Wave 3 item. |
| 2026-07-09 | P1 S-misc (dashboard/observability hardening). SEC-02/12/20 → `Done`. SEC-02: `DashboardAuthorizationHandler` restricts the loopback + `Authentication:Mode=Disabled` bypasses to read-only (they satisfy a Viewer-bearing requirement but never `AdminOnly`), closing the policy-layer gap where anonymous localhost was authorized for Admin surfaces. SEC-12: `DashboardSessionAdminService` now emits canonical `AuditEvent`s (`dashboard-close-session`/`dashboard-kill-worker`, category `SessionAdmin`) through `IAuditWriter` on Success/Failure/Denied, so Close/Kill land durable audit rows. SEC-20: dropped the unbounded `session_id` tag from the exported `mxgateway.heartbeats.failed` counter. Docs updated same-change (CLAUDE.md, GatewayDashboardDesign.md, Metrics.md). Server build clean (0 warnings); targeted classes 30/30 pass; broader Dashboard+Security+GatewayApplication+Metrics sweep 295/295 pass. |
| 2026-07-09 | P1 Wave 2b (security authz+hub). SEC-05/07/08/11 → `Done` (hub-token lifetime; QueryActiveAlarms scope arm; gateway-side verification cache + last-used coalescing; login rate limit + per-peer gRPC failure limiter). Full-suite checkpoint caught + fixed regressions the earlier narrow SEC-01/04/06 filter missed: cross-platform path-rooting, an `IHostEnvironment` fallback for minimal DI containers, a test-assembly `ASPNETCORE_ENVIRONMENT=Development` default, and a platform-correct default-path assertion. Suite: 747 passed / 42 failed, all 42 pre-existing macOS named-pipe-harness env failures. |
| 2026-07-09 | P1 Wave 2a (security). SEC-01/04/06 → `Done` (config path-rooting + production validator guards; Server build clean, validator+hygiene tests 53/53). SEC-10 → `Done`: the shared `ZB.MOM.WW.Auth.ApiKeys` gained optional `ExpiresUtc` (expired keys rejected, auth DB auto-migrates to schema v3) via a concurrent HistorianGateway-remediation session's "G-2"; this repo consumes it by bumping the four `Auth.*` refs 0.1.2→0.1.4 (commit 197731a). Remaining SEC-10 polish (`apikey create --expires` + dashboard staleness badge) tracked as a small follow-up. |
| 2026-07-09 | P1 Wave 2a (security). SEC-01/04/06 → `Done` (config path-rooting + production validator guards; Server build clean, validator+hygiene tests 53/53). SEC-10 → `Done`: the shared `ZB.MOM.WW.Auth.ApiKeys` gained optional `ExpiresUtc` (expired keys rejected, auth DB auto-migrates to schema v3) via a concurrent HistorianGateway-remediation session's "G-2"; this repo consumes it by bumping the four `Auth.*` refs 0.1.2→0.1.4 (commit 197731a; since bumped to 0.1.5 in `e107019` — 0.1.4 plus a transitive SQLitePCLRaw security pin, no API change, expiry enforcement retained). Remaining SEC-10 polish (`apikey create --expires` + dashboard staleness badge) tracked as a small follow-up. |
| 2026-07-09 | P1 Wave 1 (CI + codegen freshness + Rust buildability) via parallel agents. IPC-01/09/19/20, CLI-02 → `Done`; TST-03 → `In review` (CI pipeline authored + YAML/layout-validated, but not yet executed on a Gitea runner — proven on first push). Verified on macOS: NonWindows build clean, `ClientProtoInputTests` 5/5, `publish-client-proto-inputs.ps1 -Check` exit 0, `cargo package` (no `--no-verify`) compiles standalone. Added a vendored-Rust-proto drift guard (Check 3) to `check-codegen.ps1` closing the CLI-02 static-copy risk. IPC-09 script guards not executed end-to-end (need pinned python/JRE toolchains); logic is PATH-resolution + version assertion. |
| 2026-07-09 | WRK-01 → `Done`. Verified on Windows host (windev) via an isolated `origin/main` worktree: worker builds x86 clean, `StaRuntimeTests`+`WorkerPipeSessionTests` 33/33 pass. Fixed an `xUnit1030` build error (the new worker test used `.ConfigureAwait(false)` in `[Fact]` bodies) that the macOS tree could not surface. Also ran GWC-01's Windows-only `WorkerClientTests` on windev: 18/18 pass (incl. `ReadEventsAsync_SecondEnumerator_Throws`). All 8 P0 findings now `Done`. Not yet committed. |
| 2026-07-10 | **TST-03 → `Done`: CI is live and green** on branch `fix/ci-selfhosted-tooling`. The pipeline was authored (P1) but had never executed. Root cause it never ran: the co-located `gitea-runner` on the docker host (`10.100.0.35`) spawned job containers on an isolated network (`container.network: ""`) that could not resolve Gitea's internal clone URL `http://gitea:3000`; one-line host fix `container.network: "traefik"` + `docker restart gitea-runner`. The self-hosted `catthehacker` act image also lacks tooling GitHub-hosted runners preinstall — ci.yml now installs pwsh (dotnet global tool, for `check-codegen.ps1`) and Gradle 9.5.1 directly (act can't resolve the `gradle/actions` monorepo action; no gradle wrapper in repo). Driving to green surfaced and fixed **five real latent defects** (TST-03 doing its job): `check-codegen.ps1` `.Trim()`-on-`$null` on a clean tree; **stale vendored rust proto** (`clients/rust/protos/mxaccess_worker.proto` missing canonical `max_frame_bytes`); `OrphanWorkerTerminatorTests` hard-coded `C:\` path failing Linux `Path.GetFullPath` (0 kills); **stale java generated** `MxaccessWorker.java` (missing `max_frame_bytes`, regenerated); a sync python test building a `grpc.aio.Channel` with no current event loop on py3.12 (autouse conftest fixture). Result: `portable` **success** (NonWindows build + codegen freshness + 808/808 gateway tests + .NET/Go/Rust/Python clients) and `java` **success**. `windows` + `live-mxaccess` jobs remain `queued` pending a self-hosted **windev** runner (`10.100.0.48`) with those labels — separate follow-up, does not gate portable/java. |
@@ -132,7 +132,7 @@ This document turns every finding in the Security/Dashboard/Observability review
## SEC-06 — LDAP plaintext-by-default with a committed service password `Medium` · `P1`
**Finding.** *(review SEC-6)* `Configuration/LdapOptions.cs:49-61` defaults `Transport=None`, `AllowInsecure=true`, `ServiceAccountPassword="serviceaccount123"`; `appsettings.json:21-33` ships the same. `glauth.md:30,327` confirms dev LDAPS is disabled and binds send cleartext. The validator enforces the `Transport=None ⇒ AllowInsecure` consistency rule (`GatewayOptionsValidator.cs:82-85`) but nothing distinguishes dev from prod.
**Finding.** *(review SEC-6)* `Configuration/LdapOptions.cs:49-61` defaults `Transport=None`, `AllowInsecure=true`, `ServiceAccountPassword="<service-account-password>"` (value redacted per SEC-36); `appsettings.json:21-33` ships the same. `glauth.md:30,327` confirms dev LDAPS is disabled and binds send cleartext. The validator enforces the `Transport=None ⇒ AllowInsecure` consistency rule (`GatewayOptionsValidator.cs:82-85`) but nothing distinguishes dev from prod.
**Impact.** Every dashboard login sends the operator's password cleartext to `10.100.0.35:3893`, and a service-account credential is in source control.
@@ -140,7 +140,7 @@ This document turns every finding in the Security/Dashboard/Observability review
**Implementation.**
- `Configuration/GatewayOptionsValidator.cs`: in `ValidateLdap`, when Production and `Transport == None`, emit an error (co-locate with SEC-04's env plumbing).
- Deployment: keep `serviceaccount123` only for local GLAuth dev; document env-var override (`MxGateway__Ldap__ServiceAccountPassword`) for the NSSM-wrapped hosts; rotate the dev credential's reuse.
- Deployment: keep the dev service-account password only for local GLAuth dev; document env-var override (`MxGateway__Ldap__ServiceAccountPassword`) for the NSSM-wrapped hosts; rotate the dev credential's reuse.
- Docs: `docs/GatewayConfiguration.md` Ldap section and a production hardening note referencing `glauth.md`.
- Tests: `GatewayOptionsValidatorTests``Transport=None` + Production → invalid.
+33 -1
View File
@@ -163,6 +163,12 @@ can keep the full `MxCommandReply`, HRESULT, and status array when MXAccess
itself rejects a command. `MxAccessException.Reply` contains the raw generated
reply.
`EnsureMxAccessSuccess()` follows COM semantics: only a **negative** HRESULT is
a failure, so positive success codes such as `S_FALSE` (1) pass. A status entry
fails only when `Category` is not `MxStatusCategory.Ok``MxStatusProxy.Success`
mirrors the raw COM member for diagnostics and never decides the verdict, which
is why `IsSuccess()` branches on the category alone.
## Write Semantics And Common Pitfalls
These are MXAccess parity behaviors that surprise new callers. The gateway
@@ -258,6 +264,32 @@ optionally writes a value when `--type` and `--value` are supplied, reads a
bounded event stream, and closes the session in a `finally` block. CLI error
output redacts API keys supplied through `--api-key`.
### `authenticate-user` credentials
```powershell
$env:MXGATEWAY_VERIFY_PASSWORD = "<verify-user password>"
dotnet run --project clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli -- authenticate-user --session-id <id> --server-handle 1 --verify-user operator --json
```
The credential comes from `--password` or, preferably, the environment variable
named by `--password-env` (default `MXGATEWAY_VERIFY_PASSWORD`) so it stays out
of shell history and the process table. It is never echoed to stdout or stderr,
and error output routes it through the same redaction seam as the API key. A
missing or empty resolved credential is a usage error naming the option and the
variable: the CLI fails before the invoke rather than authenticating with an
empty password.
`MXGATEWAY_VERIFY_PASSWORD` is the canonical variable across all five client CLIs
— see [Cross-Language Smoke Matrix](../../docs/CrossLanguageSmokeMatrix.md).
**Deprecated names.** This CLI previously used `--verify-user-password`,
`--verify-user-password-env`, and `MXGATEWAY_VERIFY_USER_PASSWORD`. All three
still resolve, for one release only, so existing scripts keep working; migrate to
the canonical names above. The full resolution order is `--password`,
`--verify-user-password`, the variable named by `--password-env` (or the
deprecated `--verify-user-password-env`, default `MXGATEWAY_VERIFY_PASSWORD`),
then `MXGATEWAY_VERIFY_USER_PASSWORD`.
## Galaxy Repository Browse
`GalaxyRepositoryClient` is a separate read-only wrapper around the
@@ -455,7 +487,7 @@ dotnet nuget add source https://gitea.dohertylan.com/api/packages/dohertj2/nuget
Then add the package to your project:
````bash
dotnet add package ZB.MOM.WW.MxGateway.Client --version 0.1.1
dotnet add package ZB.MOM.WW.MxGateway.Client --version 0.2.0
````
The `ZB.MOM.WW.MxGateway.Contracts` package is pulled in transitively.
@@ -346,31 +346,78 @@ public static class MxGatewayClientCli
}
/// <summary>
/// Resolves the effective MXAccess verify-user credential from
/// <c>--verify-user-password</c> or, failing that, the
/// <c>--verify-user-password-env</c>-named environment variable (default
/// <c>MXGATEWAY_VERIFY_USER_PASSWORD</c>). The credential is never echoed;
/// this resolver exists so the error-redaction catch block can strip it
/// from any surfaced error (CLI-04), mirroring <see cref="TryResolveApiKey"/>.
/// Canonical CLI credential environment variable, shared by every official
/// client CLI (CLI-45) so one exported variable drives the same operator
/// workflow in all five languages.
/// </summary>
private const string DefaultVerifyPasswordEnvironmentName = "MXGATEWAY_VERIFY_PASSWORD";
/// <summary>
/// Pre-CLI-45 environment variable, still honoured as a deprecated fallback
/// for one release so existing scripts keep working.
/// </summary>
private const string LegacyVerifyPasswordEnvironmentName = "MXGATEWAY_VERIFY_USER_PASSWORD";
/// <summary>
/// Resolves the name of the environment variable holding the verify-user
/// credential: <c>--password-env</c>, then the deprecated
/// <c>--verify-user-password-env</c> alias, then
/// <c>MXGATEWAY_VERIFY_PASSWORD</c>.
/// </summary>
private static string ResolveVerifyPasswordEnvironmentName(CliArguments arguments)
{
string? environmentName = arguments.GetOptional("password-env");
if (!string.IsNullOrEmpty(environmentName))
{
return environmentName;
}
environmentName = arguments.GetOptional("verify-user-password-env");
return string.IsNullOrEmpty(environmentName)
? DefaultVerifyPasswordEnvironmentName
: environmentName;
}
/// <summary>
/// Resolves the effective MXAccess verify-user credential in the CLI-45
/// order: <c>--password</c>, the deprecated <c>--verify-user-password</c>
/// alias, the environment variable named by <c>--password-env</c> (default
/// <c>MXGATEWAY_VERIFY_PASSWORD</c>), then the deprecated
/// <c>MXGATEWAY_VERIFY_USER_PASSWORD</c>. An empty value from any source is
/// treated as absent. The credential is never echoed; this resolver exists so
/// the error-redaction catch block can strip it from any surfaced error
/// (CLI-04), mirroring <see cref="TryResolveApiKey" />.
/// </summary>
private static string? TryResolveVerifyUserPassword(CliArguments arguments)
{
string? password = arguments.GetOptional("verify-user-password");
string? password = arguments.GetOptional("password");
if (!string.IsNullOrEmpty(password))
{
return password;
}
string passwordEnvironmentName = arguments.GetOptional("verify-user-password-env")
?? "MXGATEWAY_VERIFY_USER_PASSWORD";
password = arguments.GetOptional("verify-user-password");
if (!string.IsNullOrEmpty(password))
{
return password;
}
return Environment.GetEnvironmentVariable(passwordEnvironmentName);
password = Environment.GetEnvironmentVariable(ResolveVerifyPasswordEnvironmentName(arguments));
if (!string.IsNullOrEmpty(password))
{
return password;
}
password = Environment.GetEnvironmentVariable(LegacyVerifyPasswordEnvironmentName);
return string.IsNullOrEmpty(password) ? null : password;
}
/// <summary>
/// Resolves the verify-user credential for <c>authenticate-user</c>, throwing
/// a redaction-safe error when neither the flag nor the env var is set. The
/// thrown message names only the option/env var, never the value.
/// a redaction-safe error when no source yields a non-empty value. Failing
/// fast keeps a misconfigured environment from becoming a real MXAccess
/// authentication attempt with an empty credential (CLI-45); the thrown
/// message names only the option/env var, never the value.
/// </summary>
private static string ResolveVerifyUserPassword(CliArguments arguments)
{
@@ -380,11 +427,10 @@ public static class MxGatewayClientCli
return password;
}
string passwordEnvironmentName = arguments.GetOptional("verify-user-password-env")
?? "MXGATEWAY_VERIFY_USER_PASSWORD";
throw new ArgumentException(
$"Verify-user password is required. Pass --verify-user-password or set {passwordEnvironmentName}.");
"Verify-user password is required. Pass --password or set "
+ $"{ResolveVerifyPasswordEnvironmentName(arguments)} (deprecated aliases: "
+ $"--verify-user-password, --verify-user-password-env, {LegacyVerifyPasswordEnvironmentName}).");
}
private static CancellationTokenSource CreateCancellation(CliArguments arguments, string command)
@@ -710,8 +756,10 @@ public static class MxGatewayClientCli
TextWriter output,
CancellationToken cancellationToken)
{
// The credential is resolved from --verify-user-password or its env var and
// is never echoed. On any surfaced error the RunCoreAsync catch block routes
// The credential is resolved from --password or its env var (default
// MXGATEWAY_VERIFY_PASSWORD) and is never echoed; a missing or empty value
// fails fast before the invoke rather than reaching the wire (CLI-45).
// On any surfaced error the RunCoreAsync catch block routes
// it through MxGatewayCliSecretRedactor so it cannot reach stderr (CLI-04).
return InvokeAndWriteAsync(
arguments,
@@ -2372,7 +2420,9 @@ public static class MxGatewayClientCli
writer.WriteLine("mxgw-dotnet activate --session-id <id> --server-handle <n> --item-handle <n> [--json]");
writer.WriteLine("mxgw-dotnet write-secured --session-id <id> --server-handle <n> --item-handle <n> --type <type> --value <value> --current-user-id <n> [--verifier-user-id <n>] [--json]");
writer.WriteLine("mxgw-dotnet write-secured2 --session-id <id> --server-handle <n> --item-handle <n> --type <type> --value <value> --current-user-id <n> [--verifier-user-id <n>] [--timestamp <iso>] [--json]");
writer.WriteLine("mxgw-dotnet authenticate-user --session-id <id> --server-handle <n> --verify-user <user> (--verify-user-password <pw> | --verify-user-password-env <ENVVAR>) [--json]");
writer.WriteLine("mxgw-dotnet authenticate-user --session-id <id> --server-handle <n> --verify-user <user> [--password <pw>] [--password-env <ENVVAR>] [--json]");
writer.WriteLine(" credential: --password, else the --password-env variable (default MXGATEWAY_VERIFY_PASSWORD); required and never empty.");
writer.WriteLine(" deprecated aliases: --verify-user-password, --verify-user-password-env, MXGATEWAY_VERIFY_USER_PASSWORD.");
writer.WriteLine("mxgw-dotnet archestra-user-to-id --session-id <id> --server-handle <n> --user-guid <guid> [--json]");
writer.WriteLine("mxgw-dotnet subscribe-bulk --session-id <id> --server-handle <n> --items <ref,ref> [--json]");
writer.WriteLine("mxgw-dotnet unsubscribe-bulk --session-id <id> --server-handle <n> --item-handles <n,n> [--json]");
@@ -32,6 +32,57 @@ public sealed class MxCommandReplyExtensionsTests
Assert.Contains("0x80040200", exception.Message);
}
/// <summary>Verifies that a non-OK status category fails even when the raw success member is set.</summary>
[Fact]
public void EnsureMxAccessSuccess_WithNonOkCategoryAndSuccessSet_Throws()
{
MxCommandReply reply = ReadReplyFixture(
"write.status-category-error-success-set.reply.json");
reply.EnsureProtocolSuccess();
MxAccessException exception = Assert.Throws<MxAccessException>(
reply.EnsureMxAccessSuccess);
Assert.Equal(1, Assert.Single(exception.Statuses).Success);
Assert.Contains("CommunicationError", exception.Message, StringComparison.Ordinal);
}
/// <summary>Verifies that an Ok status category succeeds even when the raw success member is zero.</summary>
[Fact]
public void EnsureMxAccessSuccess_WithOkCategoryAndZeroSuccess_ReturnsReply()
{
MxCommandReply reply = ReadReplyFixture(
"write.status-category-ok-success-zero.reply.json");
Assert.Equal(0, Assert.Single(reply.Statuses).Success);
Assert.Same(reply, reply.EnsureProtocolSuccess());
Assert.Same(reply, reply.EnsureMxAccessSuccess());
}
/// <summary>Verifies that a positive HResult (S_FALSE) is a COM success code, not a failure.</summary>
[Fact]
public void EnsureMxAccessSuccess_WithPositiveHResult_ReturnsReply()
{
MxCommandReply reply = ReadReplyFixture("write.hresult-s-false.reply.json");
Assert.Equal(1, reply.Hresult);
Assert.Same(reply, reply.EnsureProtocolSuccess());
Assert.Same(reply, reply.EnsureMxAccessSuccess());
}
/// <summary>Verifies that a negative HResult fails even when every status entry is Ok.</summary>
[Fact]
public void EnsureMxAccessSuccess_WithNegativeHResult_Throws()
{
MxCommandReply reply = ReadReplyFixture("write.hresult-e-fail.reply.json");
reply.EnsureProtocolSuccess();
MxAccessException exception = Assert.Throws<MxAccessException>(
reply.EnsureMxAccessSuccess);
Assert.Equal(-2147467259, exception.HResultCode);
}
/// <summary>Verifies that session-not-found protocol failures throw the correct gateway exception.</summary>
[Fact]
public void EnsureProtocolSuccess_WithSessionFailure_ThrowsSessionException()
@@ -235,7 +235,7 @@ public sealed class MxGatewayClientCliTests
"--session-id", "session-fixture",
"--server-handle", "12",
"--verify-user", "operator",
"--verify-user-password", password,
"--password", password,
],
output,
error,
@@ -246,6 +246,235 @@ public sealed class MxGatewayClientCliTests
Assert.Contains("[redacted]", error.ToString());
}
/// <summary>
/// CLI-45: <c>--password</c> is the primary credential flag, matching the other
/// four CLIs. The credential reaches the wire but never stdout/stderr.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task RunAsync_AuthenticateUser_AcceptsCanonicalPasswordFlag()
{
const string password = "canonical-flag-credential";
using var output = new StringWriter();
using var error = new StringWriter();
FakeCliClient fakeClient = new();
fakeClient.InvokeReplies.Enqueue(new MxCommandReply
{
SessionId = "session-fixture",
Kind = MxCommandKind.AuthenticateUser,
ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok },
AuthenticateUser = new AuthenticateUserReply { UserId = 11 },
});
int exitCode = await MxGatewayClientCli.RunAsync(
[
"authenticate-user",
"--endpoint", "http://localhost:5000",
"--api-key", "test-api-key",
"--session-id", "session-fixture",
"--server-handle", "12",
"--verify-user", "operator",
"--password", password,
"--json",
],
output,
error,
_ => fakeClient);
Assert.Equal(0, exitCode);
MxCommandRequest request = Assert.Single(fakeClient.InvokeRequests);
Assert.Equal(password, request.Command.AuthenticateUser.VerifyUserPassword);
Assert.DoesNotContain(password, output.ToString(), StringComparison.Ordinal);
Assert.DoesNotContain(password, error.ToString(), StringComparison.Ordinal);
}
/// <summary>
/// CLI-45: the credential is read from the environment variable named by
/// <c>--password-env</c>, whose default is the canonical
/// <c>MXGATEWAY_VERIFY_PASSWORD</c> shared by all five CLIs.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Theory]
[InlineData(null, "MXGATEWAY_VERIFY_PASSWORD")]
[InlineData("MXGW_TEST_CLI45_ENV", "MXGW_TEST_CLI45_ENV")]
public async Task RunAsync_AuthenticateUser_ReadsCredentialFromNamedEnvironmentVariable(
string? passwordEnvArgument,
string environmentName)
{
const string password = "env-sourced-credential";
using EnvironmentVariableScope scope = new(environmentName, password);
using var output = new StringWriter();
using var error = new StringWriter();
FakeCliClient fakeClient = new();
fakeClient.InvokeReplies.Enqueue(new MxCommandReply
{
SessionId = "session-fixture",
Kind = MxCommandKind.AuthenticateUser,
ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok },
AuthenticateUser = new AuthenticateUserReply { UserId = 12 },
});
List<string> args =
[
"authenticate-user",
"--endpoint", "http://localhost:5000",
"--api-key", "test-api-key",
"--session-id", "session-fixture",
"--server-handle", "12",
"--verify-user", "operator",
"--json",
];
if (passwordEnvArgument is not null)
{
args.Add("--password-env");
args.Add(passwordEnvArgument);
}
int exitCode = await MxGatewayClientCli.RunAsync([.. args], output, error, _ => fakeClient);
Assert.Equal(0, exitCode);
MxCommandRequest request = Assert.Single(fakeClient.InvokeRequests);
Assert.Equal(password, request.Command.AuthenticateUser.VerifyUserPassword);
Assert.DoesNotContain(password, output.ToString(), StringComparison.Ordinal);
}
/// <summary>
/// CLI-45: the pre-rename names stay usable for one release — the deprecated
/// <c>--verify-user-password</c> flag and the deprecated
/// <c>MXGATEWAY_VERIFY_USER_PASSWORD</c> environment variable both still resolve.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task RunAsync_AuthenticateUser_HonoursDeprecatedAliases()
{
using var flagOutput = new StringWriter();
using var flagError = new StringWriter();
FakeCliClient flagClient = new();
flagClient.InvokeReplies.Enqueue(new MxCommandReply
{
SessionId = "session-fixture",
Kind = MxCommandKind.AuthenticateUser,
ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok },
AuthenticateUser = new AuthenticateUserReply { UserId = 13 },
});
int flagExitCode = await MxGatewayClientCli.RunAsync(
[
"authenticate-user",
"--endpoint", "http://localhost:5000",
"--api-key", "test-api-key",
"--session-id", "session-fixture",
"--server-handle", "12",
"--verify-user", "operator",
"--verify-user-password", "legacy-flag-credential",
"--json",
],
flagOutput,
flagError,
_ => flagClient);
Assert.Equal(0, flagExitCode);
Assert.Equal(
"legacy-flag-credential",
Assert.Single(flagClient.InvokeRequests).Command.AuthenticateUser.VerifyUserPassword);
using EnvironmentVariableScope canonical = new("MXGATEWAY_VERIFY_PASSWORD", null);
using EnvironmentVariableScope legacy = new("MXGATEWAY_VERIFY_USER_PASSWORD", "legacy-env-credential");
using var envOutput = new StringWriter();
using var envError = new StringWriter();
FakeCliClient envClient = new();
envClient.InvokeReplies.Enqueue(new MxCommandReply
{
SessionId = "session-fixture",
Kind = MxCommandKind.AuthenticateUser,
ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok },
AuthenticateUser = new AuthenticateUserReply { UserId = 14 },
});
int envExitCode = await MxGatewayClientCli.RunAsync(
[
"authenticate-user",
"--endpoint", "http://localhost:5000",
"--api-key", "test-api-key",
"--session-id", "session-fixture",
"--server-handle", "12",
"--verify-user", "operator",
"--json",
],
envOutput,
envError,
_ => envClient);
Assert.Equal(0, envExitCode);
Assert.Equal(
"legacy-env-credential",
Assert.Single(envClient.InvokeRequests).Command.AuthenticateUser.VerifyUserPassword);
}
/// <summary>
/// CLI-45: a missing or empty credential fails fast before the invoke — the CLI
/// never sends a fabricated empty password to the wire. The error names the flag
/// and the environment variable, never a value.
/// </summary>
/// <param name="explicitEmptyFlag">Whether to pass an explicit empty --password.</param>
/// <returns>A task that represents the asynchronous operation.</returns>
[Theory]
[InlineData(false)]
[InlineData(true)]
public async Task RunAsync_AuthenticateUser_FailsFastOnMissingOrEmptyCredential(bool explicitEmptyFlag)
{
using EnvironmentVariableScope canonical = new("MXGATEWAY_VERIFY_PASSWORD", explicitEmptyFlag ? string.Empty : null);
using EnvironmentVariableScope legacy = new("MXGATEWAY_VERIFY_USER_PASSWORD", null);
using var output = new StringWriter();
using var error = new StringWriter();
FakeCliClient fakeClient = new();
List<string> args =
[
"authenticate-user",
"--endpoint", "http://localhost:5000",
"--api-key", "test-api-key",
"--session-id", "session-fixture",
"--server-handle", "12",
"--verify-user", "operator",
];
if (explicitEmptyFlag)
{
args.Add("--password");
args.Add(string.Empty);
}
int exitCode = await MxGatewayClientCli.RunAsync([.. args], output, error, _ => fakeClient);
Assert.Equal(1, exitCode);
Assert.Empty(fakeClient.InvokeRequests);
Assert.Contains("--password", error.ToString(), StringComparison.Ordinal);
Assert.Contains("MXGATEWAY_VERIFY_PASSWORD", error.ToString(), StringComparison.Ordinal);
}
/// <summary>
/// Sets an environment variable for the duration of a test and restores the
/// previous value on dispose, so credential-resolution tests do not depend on
/// (or leak into) the ambient environment.
/// </summary>
private sealed class EnvironmentVariableScope : IDisposable
{
private readonly string _name;
private readonly string? _original;
public EnvironmentVariableScope(string name, string? value)
{
_name = name;
_original = Environment.GetEnvironmentVariable(name);
Environment.SetEnvironmentVariable(name, value);
}
public void Dispose()
{
Environment.SetEnvironmentVariable(_name, _original);
}
}
/// <summary>Verifies that error output redacts sensitive API key values.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
@@ -0,0 +1,96 @@
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Client.Tests;
/// <summary>
/// Unit tests for <see cref="MxGatewaySecretRedaction"/> — the exact-substring scrub applied to
/// diagnostic text and rebuilt exceptions before they leave the client on a failure path.
/// </summary>
public sealed class MxGatewaySecretRedactionTests
{
[Fact]
public void Redact_ReplacesEveryOccurrenceOfSecret()
{
string result = MxGatewaySecretRedaction.Redact(
"pw=hunter2 retry pw=hunter2 again hunter2",
"hunter2");
Assert.DoesNotContain("hunter2", result, StringComparison.Ordinal);
Assert.Equal("pw=<redacted> retry pw=<redacted> again <redacted>", result);
}
[Fact]
public void Redact_ScrubsBothSecretsWhenOneIsSubstringOfTheOther()
{
// "secret" is a substring of "secretPassword"; both must be fully scrubbed regardless of
// supplied order — no residual leak of either verbatim value.
string result = MxGatewaySecretRedaction.Redact(
"a=secretPassword b=secret",
"secret",
"secretPassword");
Assert.DoesNotContain("secretPassword", result, StringComparison.Ordinal);
Assert.DoesNotContain("secret", result, StringComparison.Ordinal);
}
[Fact]
public void Redact_WithNullSecretsArray_ReturnsMessageUnchanged()
{
const string message = "nothing to scrub here";
string result = MxGatewaySecretRedaction.Redact(message, null!);
Assert.Equal(message, result);
}
[Fact]
public void Redact_WithEmptySecretsArray_ReturnsMessageUnchanged()
{
const string message = "nothing to scrub here";
string result = MxGatewaySecretRedaction.Redact(message);
Assert.Equal(message, result);
}
[Fact]
public void Redact_IgnoresWhitespaceOnlySecret()
{
// A whitespace-only secret must not over-redact the internal spaces of the message.
const string message = "user operator logged in";
string result = MxGatewaySecretRedaction.Redact(message, " ");
Assert.Equal(message, result);
}
[Fact]
public void Redacted_PreservesConcreteSubtypeAndDoesNotChainSecretBearingOriginal()
{
const string secret = "hunter2";
Exception transportCause = new InvalidOperationException("transport reset");
MxGatewaySessionException original = new(
$"session rejected credential '{secret}'",
"session-1",
"correlation-1",
new ProtocolStatus { Code = ProtocolStatusCode.SessionNotReady, Message = $"echoed '{secret}'" },
hResult: -1,
statuses: [new MxStatusProxy { DiagnosticText = $"denied '{secret}'" }],
innerException: transportCause);
MxGatewayException redacted = MxGatewaySecretRedaction.Redacted(original, secret);
// Concrete runtime type is preserved.
Assert.IsType<MxGatewaySessionException>(redacted);
// The secret is gone from the message and every structured accessor.
Assert.DoesNotContain(secret, redacted.Message, StringComparison.Ordinal);
Assert.DoesNotContain(secret, redacted.ToString(), StringComparison.Ordinal);
Assert.DoesNotContain(secret, redacted.ProtocolStatus!.Message, StringComparison.Ordinal);
Assert.All(redacted.Statuses, status =>
Assert.DoesNotContain(secret, status.DiagnosticText, StringComparison.Ordinal));
Assert.Contains("<redacted>", redacted.Message, StringComparison.Ordinal);
// The secret-bearing original is NOT chained; the original's transport cause is carried.
Assert.NotSame(original, redacted.InnerException);
Assert.Same(transportCause, redacted.InnerException);
}
}
@@ -0,0 +1,165 @@
using Google.Protobuf;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Client.Tests;
/// <summary>
/// Tests for the credential-scrub (CLI-40) and malformed-reply (CLI-41) contracts on the
/// credential and id-returning session helpers, driven from shared behavior fixtures.
/// </summary>
public sealed class MxGatewaySessionReplyContractTests
{
/// <summary>
/// CLI-40: when MXAccess echoes the submitted credential back in its failure diagnostic,
/// the surfaced exception message must scrub it to the library redaction marker.
/// </summary>
[Fact]
public async Task AuthenticateUserAsync_RedactsEchoedCredentialInFailureMessage()
{
const string password = "sup3rSecretVerify9f3a2b";
FakeGatewayTransport transport = CreateTransport();
transport.AddInvokeReply(ReadReplyFixture("authenticate-user.echoed-credential.reply.json"));
await using MxGatewayClient client = CreateClient(transport);
MxGatewaySession session = await client.OpenSessionAsync();
MxAccessException exception = await Assert.ThrowsAsync<MxAccessException>(
async () => await session.AuthenticateUserAsync(12, "operator", password));
Assert.DoesNotContain(password, exception.Message, StringComparison.Ordinal);
Assert.Contains("<redacted>", exception.Message, StringComparison.Ordinal);
// ToString() is what logging frameworks emit; the secret-bearing original must not be
// chained as an inner exception where it would re-surface the credential verbatim.
Assert.DoesNotContain(password, exception.ToString(), StringComparison.Ordinal);
}
/// <summary>
/// CLI-40: the redacted exception must not leak the echoed credential through any structured
/// accessor either — <see cref="MxAccessException.Reply"/> (protocol message, diagnostic
/// message, and each MXSTATUS_PROXY diagnostic text) and <see cref="MxGatewayException.Statuses"/>
/// all carry the server-echoed credential verbatim before the fix. Both the OK+negative-HRESULT
/// and the MXACCESS_FAILURE reply route to <see cref="MxAccessException"/>, so both must scrub.
/// </summary>
/// <param name="fixture">The echoed-credential reply fixture to drive.</param>
[Theory]
[InlineData("authenticate-user.echoed-credential.reply.json")]
[InlineData("authenticate-user.echoed-credential-mxaccess-failure.reply.json")]
public async Task AuthenticateUserAsync_RedactsEchoedCredentialInStructuredAccessors(string fixture)
{
const string password = "sup3rSecretVerify9f3a2b";
FakeGatewayTransport transport = CreateTransport();
transport.AddInvokeReply(ReadReplyFixture(fixture));
await using MxGatewayClient client = CreateClient(transport);
MxGatewaySession session = await client.OpenSessionAsync();
MxAccessException exception = await Assert.ThrowsAsync<MxAccessException>(
async () => await session.AuthenticateUserAsync(12, "operator", password));
Assert.DoesNotContain(password, exception.Message, StringComparison.Ordinal);
Assert.Contains("<redacted>", exception.Message, StringComparison.Ordinal);
Assert.DoesNotContain(password, exception.ToString(), StringComparison.Ordinal);
Assert.DoesNotContain(password, exception.Reply.ProtocolStatus.Message, StringComparison.Ordinal);
Assert.DoesNotContain(password, exception.Reply.DiagnosticMessage, StringComparison.Ordinal);
foreach (MxStatusProxy status in exception.Reply.Statuses)
{
Assert.DoesNotContain(password, status.DiagnosticText, StringComparison.Ordinal);
}
foreach (MxStatusProxy status in exception.Statuses)
{
Assert.DoesNotContain(password, status.DiagnosticText, StringComparison.Ordinal);
}
}
/// <summary>
/// CLI-41: an OK reply that carries neither the typed AuthenticateUser payload nor an
/// int32 return_value is a malformed reply, surfaced as a typed exception rather than an NRE.
/// </summary>
[Fact]
public async Task AuthenticateUserAsync_MissingPayloadAndReturnValue_ThrowsMalformedReply()
{
FakeGatewayTransport transport = CreateTransport();
transport.AddInvokeReply(ReadReplyFixture("authenticate-user.missing-payload.reply.json"));
await using MxGatewayClient client = CreateClient(transport);
MxGatewaySession session = await client.OpenSessionAsync();
await Assert.ThrowsAsync<MxGatewayMalformedReplyException>(
async () => await session.AuthenticateUserAsync(12, "operator", "pw"));
}
/// <summary>
/// CLI-41: an OK reply that omits the typed payload but carries an int32 return_value
/// resolves to that return value.
/// </summary>
[Fact]
public async Task AuthenticateUserAsync_ReturnValueOnly_ResolvesReturnValue()
{
FakeGatewayTransport transport = CreateTransport();
transport.AddInvokeReply(ReadReplyFixture("authenticate-user.return-value-only.reply.json"));
await using MxGatewayClient client = CreateClient(transport);
MxGatewaySession session = await client.OpenSessionAsync();
int userId = await session.AuthenticateUserAsync(12, "operator", "pw");
Assert.Equal(7, userId);
}
/// <summary>
/// CLI-41: the AddBufferedItem fallback shares the malformed-reply contract — an OK reply
/// with neither a typed item handle nor an int32 return_value throws the typed exception.
/// </summary>
[Fact]
public async Task AddBufferedItemAsync_MissingPayloadAndReturnValue_ThrowsMalformedReply()
{
FakeGatewayTransport transport = CreateTransport();
transport.AddInvokeReply(new MxCommandReply
{
SessionId = "session-fixture",
Kind = MxCommandKind.AddBufferedItem,
ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok },
});
await using MxGatewayClient client = CreateClient(transport);
MxGatewaySession session = await client.OpenSessionAsync();
await Assert.ThrowsAsync<MxGatewayMalformedReplyException>(
async () => await session.AddBufferedItemAsync(12, "Area001.Pump001.Speed", "runtime"));
}
private static MxGatewayClient CreateClient(FakeGatewayTransport transport)
{
return new MxGatewayClient(transport.Options, transport);
}
private static FakeGatewayTransport CreateTransport()
{
return new FakeGatewayTransport(new MxGatewayClientOptions
{
Endpoint = new Uri("http://localhost:5000"),
ApiKey = "test-api-key",
});
}
private static MxCommandReply ReadReplyFixture(string fileName)
{
DirectoryInfo directory = new(AppContext.BaseDirectory);
while (directory is not null)
{
string path = Path.Combine(
directory.FullName,
"clients",
"proto",
"fixtures",
"behavior",
"command-replies",
fileName);
if (File.Exists(path))
{
return JsonParser.Default.Parse<MxCommandReply>(File.ReadAllText(path));
}
directory = directory.Parent!;
}
throw new FileNotFoundException(fileName);
}
}
@@ -19,9 +19,9 @@ public sealed class MxStatusProxyExtensionsTests
{
MxStatusProxy status = JsonParser.Default.Parse<MxStatusProxy>(
testCase.GetProperty("status").GetRawText());
int success = testCase.GetProperty("status").GetProperty("success").GetInt32();
Assert.Equal(success != 0 && status.Category is MxStatusCategory.Ok, status.IsSuccess());
bool wantSuccess = testCase.GetProperty("wantSuccess").GetBoolean();
Assert.Equal(wantSuccess, status.IsSuccess());
Assert.Equal(
testCase.GetProperty("status").GetProperty("rawCategory").GetInt32(),
status.RawCategory);
@@ -31,6 +31,22 @@ public sealed class MxStatusProxyExtensionsTests
}
}
/// <summary>Verifies that the raw success member never overrides the authoritative category.</summary>
[Theory]
[InlineData(MxStatusCategory.Ok, 0, true)]
[InlineData(MxStatusCategory.Ok, 1, true)]
[InlineData(MxStatusCategory.CommunicationError, 1, false)]
[InlineData(MxStatusCategory.Unspecified, 1, false)]
public void IsSuccess_BranchesOnCategoryOnly(
MxStatusCategory category,
int success,
bool expected)
{
MxStatusProxy status = new() { Category = category, Success = success };
Assert.Equal(expected, status.IsSuccess());
}
private static string ReadFixture(string category, string fileName)
{
DirectoryInfo directory = new(AppContext.BaseDirectory);
@@ -23,7 +23,11 @@ public static class MxCommandReplyExtensions
throw CreateProtocolException(reply, code);
}
/// <summary>Validates that the reply indicates MXAccess success (no HResult or status failures), throwing MxAccessException if not.</summary>
/// <summary>
/// Validates that the reply indicates MXAccess success, throwing MxAccessException if not.
/// Following COM semantics, only a negative HResult is a failure — positive success codes
/// such as <c>S_FALSE</c> pass — and a status entry fails only when its category is not Ok.
/// </summary>
/// <param name="reply">The command reply to check.</param>
/// <returns>The same reply, for chaining.</returns>
public static MxCommandReply EnsureMxAccessSuccess(this MxCommandReply reply)
@@ -31,7 +35,7 @@ public static class MxCommandReplyExtensions
ArgumentNullException.ThrowIfNull(reply);
bool mxAccessFailure = reply.ProtocolStatus?.Code is ProtocolStatusCode.MxaccessFailure;
bool hResultFailure = reply.HasHresult && reply.Hresult != 0;
bool hResultFailure = reply.HasHresult && reply.Hresult < 0;
bool statusFailure = reply.Statuses.Any(status => !status.IsSuccess());
if (!mxAccessFailure && !hResultFailure && !statusFailure)
@@ -0,0 +1,46 @@
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Client;
/// <summary>
/// Exception thrown when the gateway returns a protocol-OK reply that carries neither the
/// expected typed payload nor an int32 <c>return_value</c>, so the client cannot resolve the
/// operation result. This replaces the historical <see cref="NullReferenceException"/> that a
/// blind <c>reply.ReturnValue.Int32Value</c> fallback would throw.
/// </summary>
public sealed class MxGatewayMalformedReplyException : MxGatewayException
{
/// <summary>Initializes a new instance with the given message.</summary>
/// <param name="message">The error message describing the malformed reply.</param>
public MxGatewayMalformedReplyException(string message)
: base(message)
{
}
/// <summary>Initializes a new instance with full diagnostic context.</summary>
/// <param name="message">The error message describing the malformed reply.</param>
/// <param name="sessionId">The session ID, if available.</param>
/// <param name="correlationId">The correlation ID for tracing, if available.</param>
/// <param name="protocolStatus">The protocol status details, if available.</param>
/// <param name="hResult">The HResult code, if available.</param>
/// <param name="statuses">The MXAccess statuses, if available.</param>
/// <param name="innerException">The underlying exception, if any.</param>
public MxGatewayMalformedReplyException(
string message,
string? sessionId = null,
string? correlationId = null,
ProtocolStatus? protocolStatus = null,
int? hResult = null,
IReadOnlyList<MxStatusProxy>? statuses = null,
Exception? innerException = null)
: base(
message,
sessionId,
correlationId,
protocolStatus,
hResult,
statuses ?? [],
innerException)
{
}
}
@@ -0,0 +1,229 @@
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Client;
/// <summary>
/// Scrubs exact secret substrings out of diagnostic text before it leaves the client on an
/// exception path. MXAccess can echo a submitted credential or secured value back inside a
/// failure diagnostic (protocol message, MXSTATUS_PROXY diagnostic text, HRESULT description);
/// this helper replaces any such verbatim occurrence with <c>&lt;redacted&gt;</c> so the raw
/// request payload never reaches a caught exception's message. The marker matches the Go, Rust,
/// and Java clients.
/// </summary>
internal static class MxGatewaySecretRedaction
{
private const string Marker = "<redacted>";
/// <summary>
/// Replaces every usable secret in <paramref name="secrets"/> with the redaction marker
/// (ordinal comparison). Returns the message unchanged when it is null or empty, or when no
/// usable secret is supplied. A secret that is null, empty, or whitespace-only is ignored so
/// it cannot over-redact ordinary separator characters in the message.
/// </summary>
/// <param name="message">The diagnostic message to scrub.</param>
/// <param name="secrets">The secret values to remove from the message.</param>
/// <returns>The scrubbed message.</returns>
internal static string Redact(string message, params string?[] secrets)
{
if (string.IsNullOrEmpty(message) || secrets is null)
{
return message;
}
string result = message;
foreach (string? secret in secrets)
{
if (!string.IsNullOrWhiteSpace(secret))
{
result = result.Replace(secret, Marker, StringComparison.Ordinal);
}
}
return result;
}
/// <summary>
/// Returns a scrubbed clone of <paramref name="reply"/>: the protocol-status message, the
/// reply-level diagnostic message, and each MXSTATUS_PROXY diagnostic text have every verbatim
/// secret replaced with the redaction marker. The original is left untouched. MXAccess can echo
/// a submitted credential into any of these fields, so a redacted exception must carry the
/// scrubbed reply rather than the secret-bearing original.
/// </summary>
/// <param name="reply">The reply to clone and scrub.</param>
/// <param name="secrets">The secret values to remove.</param>
/// <returns>A scrubbed clone of the reply.</returns>
internal static MxCommandReply RedactReply(MxCommandReply reply, params string?[] secrets)
{
ArgumentNullException.ThrowIfNull(reply);
MxCommandReply clone = reply.Clone();
if (clone.ProtocolStatus is not null)
{
clone.ProtocolStatus.Message = Redact(clone.ProtocolStatus.Message, secrets);
}
clone.DiagnosticMessage = Redact(clone.DiagnosticMessage, secrets);
foreach (MxStatusProxy status in clone.Statuses)
{
status.DiagnosticText = Redact(status.DiagnosticText, secrets);
}
return clone;
}
/// <summary>
/// Returns a scrubbed clone of <paramref name="status"/> (its message with every verbatim
/// secret removed), or <see langword="null"/> when the input is null.
/// </summary>
/// <param name="status">The protocol status to clone and scrub.</param>
/// <param name="secrets">The secret values to remove.</param>
/// <returns>A scrubbed clone, or <see langword="null"/>.</returns>
internal static ProtocolStatus? RedactStatus(ProtocolStatus? status, params string?[] secrets)
{
if (status is null)
{
return null;
}
ProtocolStatus clone = status.Clone();
clone.Message = Redact(clone.Message, secrets);
return clone;
}
/// <summary>
/// Returns a list of scrubbed clones of <paramref name="statuses"/> — each MXSTATUS_PROXY's
/// diagnostic text has every verbatim secret removed. The originals are left untouched.
/// </summary>
/// <param name="statuses">The statuses to clone and scrub.</param>
/// <param name="secrets">The secret values to remove.</param>
/// <returns>A list of scrubbed clones.</returns>
internal static IReadOnlyList<MxStatusProxy> RedactStatuses(
IReadOnlyList<MxStatusProxy> statuses,
params string?[] secrets)
{
if (statuses is null || statuses.Count is 0)
{
return statuses ?? [];
}
MxStatusProxy[] result = new MxStatusProxy[statuses.Count];
for (int i = 0; i < statuses.Count; i++)
{
MxStatusProxy clone = statuses[i].Clone();
clone.DiagnosticText = Redact(clone.DiagnosticText, secrets);
result[i] = clone;
}
return result;
}
/// <summary>
/// Returns an exception equivalent to <paramref name="ex"/> but with any verbatim secret
/// scrubbed from its message. When nothing changes, the original exception is returned
/// unchanged; otherwise a new exception of the same concrete runtime type is built and the
/// original reply/status context is preserved. The secret-bearing original is deliberately
/// <b>not</b> chained as the inner exception — doing so would let its unredacted message
/// re-surface through <see cref="Exception.ToString"/> (which logging frameworks call). The
/// original's own inner cause (a transport error, never the request payload) is carried
/// forward instead.
/// </summary>
/// <param name="ex">The exception to redact.</param>
/// <param name="secrets">The secret values to remove from the message.</param>
/// <returns>The redacted exception, or the original when no change was needed.</returns>
internal static MxGatewayException Redacted(MxGatewayException ex, params string?[] secrets)
{
ArgumentNullException.ThrowIfNull(ex);
string redacted = Redact(ex.Message, secrets);
bool messageChanged = !string.Equals(redacted, ex.Message, StringComparison.Ordinal);
Exception? cause = ex.InnerException;
// MxAccessException derives its structured fields from the raw reply, so scrubbing must
// clone and redact that reply — the message alone changing is not enough, because the reply
// can carry the echoed secret even when the message does not.
if (ex is MxAccessException access)
{
if (!messageChanged && !ReplyContainsSecret(access.Reply, secrets))
{
return ex;
}
return new MxAccessException(redacted, RedactReply(access.Reply, secrets), cause);
}
// Other subtypes carry the secret through ProtocolStatus.Message and Statuses[].DiagnosticText.
if (!messageChanged
&& !ContainsSecret(ex.ProtocolStatus?.Message, secrets)
&& !StatusesContainSecret(ex.Statuses, secrets))
{
return ex;
}
ProtocolStatus? status = RedactStatus(ex.ProtocolStatus, secrets);
IReadOnlyList<MxStatusProxy> statuses = RedactStatuses(ex.Statuses, secrets);
return ex switch
{
MxGatewaySessionException => new MxGatewaySessionException(
redacted, ex.SessionId, ex.CorrelationId, status, ex.HResultCode, statuses, cause),
MxGatewayWorkerException => new MxGatewayWorkerException(
redacted, ex.SessionId, ex.CorrelationId, status, ex.HResultCode, statuses, cause),
MxGatewayAuthenticationException => new MxGatewayAuthenticationException(
redacted, ex.SessionId, ex.CorrelationId, status, ex.HResultCode, statuses, cause),
MxGatewayAuthorizationException => new MxGatewayAuthorizationException(
redacted, ex.SessionId, ex.CorrelationId, status, ex.HResultCode, statuses, cause),
MxGatewayMalformedReplyException => new MxGatewayMalformedReplyException(
redacted, ex.SessionId, ex.CorrelationId, status, ex.HResultCode, statuses, cause),
MxGatewayCommandException => new MxGatewayCommandException(
redacted, ex.SessionId, ex.CorrelationId, status, ex.HResultCode, statuses, cause),
_ => new MxGatewayException(redacted, cause),
};
}
private static bool ContainsSecret(string? text, string?[] secrets)
{
if (string.IsNullOrEmpty(text) || secrets is null)
{
return false;
}
foreach (string? secret in secrets)
{
if (!string.IsNullOrWhiteSpace(secret) && text.Contains(secret, StringComparison.Ordinal))
{
return true;
}
}
return false;
}
private static bool StatusesContainSecret(IReadOnlyList<MxStatusProxy> statuses, string?[] secrets)
{
if (statuses is null)
{
return false;
}
foreach (MxStatusProxy status in statuses)
{
if (ContainsSecret(status.DiagnosticText, secrets))
{
return true;
}
}
return false;
}
private static bool ReplyContainsSecret(MxCommandReply reply, string?[] secrets)
{
if (reply is null)
{
return false;
}
return ContainsSecret(reply.ProtocolStatus?.Message, secrets)
|| ContainsSecret(reply.DiagnosticMessage, secrets)
|| StatusesContainSecret(reply.Statuses, secrets);
}
}
@@ -945,7 +945,7 @@ public sealed class MxGatewaySession : IAsyncDisposable
cancellationToken)
.ConfigureAwait(false);
reply.EnsureProtocolSuccess().EnsureMxAccessSuccess();
return reply.AddBufferedItem?.ItemHandle ?? reply.ReturnValue.Int32Value;
return ResolveInt32Result(reply.AddBufferedItem?.ItemHandle, reply, "AddBufferedItem");
}
/// <summary>
@@ -1141,7 +1141,14 @@ public sealed class MxGatewaySession : IAsyncDisposable
verifierUserId,
cancellationToken)
.ConfigureAwait(false);
reply.EnsureProtocolSuccess().EnsureMxAccessSuccess();
try
{
reply.EnsureProtocolSuccess().EnsureMxAccessSuccess();
}
catch (MxGatewayException ex)
{
throw MxGatewaySecretRedaction.Redacted(ex, ExtractSecretString(value));
}
}
/// <summary>
@@ -1215,7 +1222,14 @@ public sealed class MxGatewaySession : IAsyncDisposable
verifierUserId,
cancellationToken)
.ConfigureAwait(false);
reply.EnsureProtocolSuccess().EnsureMxAccessSuccess();
try
{
reply.EnsureProtocolSuccess().EnsureMxAccessSuccess();
}
catch (MxGatewayException ex)
{
throw MxGatewaySecretRedaction.Redacted(ex, ExtractSecretString(value));
}
}
/// <summary>
@@ -1285,8 +1299,15 @@ public sealed class MxGatewaySession : IAsyncDisposable
verifyUserPassword,
cancellationToken)
.ConfigureAwait(false);
reply.EnsureProtocolSuccess().EnsureMxAccessSuccess();
return reply.AuthenticateUser?.UserId ?? reply.ReturnValue.Int32Value;
try
{
reply.EnsureProtocolSuccess().EnsureMxAccessSuccess();
return ResolveInt32Result(reply.AuthenticateUser?.UserId, reply, "AuthenticateUser");
}
catch (MxGatewayException ex)
{
throw MxGatewaySecretRedaction.Redacted(ex, verifyUserPassword);
}
}
/// <summary>
@@ -1337,7 +1358,7 @@ public sealed class MxGatewaySession : IAsyncDisposable
MxCommandReply reply = await ArchestraUserToIdRawAsync(serverHandle, userIdGuid, cancellationToken)
.ConfigureAwait(false);
reply.EnsureProtocolSuccess().EnsureMxAccessSuccess();
return reply.ArchestraUserToId?.UserId ?? reply.ReturnValue.Int32Value;
return ResolveInt32Result(reply.ArchestraUserToId?.UserId, reply, "ArchestrAUserToId");
}
/// <summary>
@@ -1367,6 +1388,51 @@ public sealed class MxGatewaySession : IAsyncDisposable
cancellationToken);
}
/// <summary>
/// Resolves the int32 result of an OK command reply: the typed payload value when present,
/// otherwise an int32 <c>return_value</c> when the reply carries one. A reply that provides
/// neither is malformed and surfaces as <see cref="MxGatewayMalformedReplyException"/>
/// rather than the historical <see cref="NullReferenceException"/>.
/// </summary>
/// <param name="typedValue">The typed payload value, or <see langword="null"/> when absent.</param>
/// <param name="reply">The OK command reply.</param>
/// <param name="operation">The MXAccess operation name, for the diagnostic message.</param>
/// <returns>The resolved int32 result.</returns>
private static int ResolveInt32Result(int? typedValue, MxCommandReply reply, string operation)
{
if (typedValue.HasValue)
{
return typedValue.Value;
}
if (reply.ReturnValue is not null
&& reply.ReturnValue.KindCase == MxValue.KindOneofCase.Int32Value)
{
return reply.ReturnValue.Int32Value;
}
throw new MxGatewayMalformedReplyException(
$"{operation} returned a malformed reply: OK reply carried neither the typed payload nor an int32 return_value",
reply.SessionId,
reply.CorrelationId,
reply.ProtocolStatus,
reply.HasHresult ? reply.Hresult : null,
reply.Statuses.ToArray());
}
/// <summary>
/// Extracts the raw string form of a credential-bearing <see cref="MxValue"/> for redaction,
/// or <see langword="null"/> when the value does not carry a string.
/// </summary>
/// <param name="value">The value written by a secured write.</param>
/// <returns>The string payload, or <see langword="null"/>.</returns>
private static string? ExtractSecretString(MxValue value)
{
return value.KindCase == MxValue.KindOneofCase.StringValue
? value.StringValue
: null;
}
/// <summary>
/// Invokes an MXAccess command on this session.
/// </summary>
@@ -5,15 +5,18 @@ namespace ZB.MOM.WW.MxGateway.Client;
/// <summary>Extension methods for MxStatusProxy values.</summary>
public static class MxStatusProxyExtensions
{
/// <summary>Returns whether the status indicates success (success flag set and category is Ok).</summary>
/// <summary>
/// Returns whether the status indicates success, which the wire contract defines as
/// <see cref="MxStatusCategory.Ok"/>. The raw <c>Success</c> member is a verbatim COM
/// diagnostic, not a boolean, so it never participates in the verdict.
/// </summary>
/// <param name="status">The status to check.</param>
/// <returns><see langword="true"/> if the status indicates success; otherwise <see langword="false"/>.</returns>
public static bool IsSuccess(this MxStatusProxy status)
{
ArgumentNullException.ThrowIfNull(status);
return status.Success != 0
&& status.Category is MxStatusCategory.Ok;
return status.Category is MxStatusCategory.Ok;
}
/// <summary>Returns a formatted summary of the status for diagnostic output.</summary>
@@ -27,6 +30,6 @@ public static class MxStatusProxyExtensions
? "no diagnostic text"
: status.DiagnosticText;
return $"{status.Category} by {status.DetectedBy}; detail={status.Detail}; {diagnosticText}";
return $"success={status.Success}; {status.Category} by {status.DetectedBy}; detail={status.Detail}; {diagnosticText}";
}
}
@@ -19,7 +19,7 @@
<PropertyGroup>
<IsPackable>true</IsPackable>
<PackageId>ZB.MOM.WW.MxGateway.Client</PackageId>
<Version>0.1.2</Version>
<Version>0.2.0</Version>
<Description>.NET 10 gRPC client for the MxAccessGateway service. Provides typed wrappers, retry, and a lazy-browse walker over the Galaxy Repository hierarchy.</Description>
<PackageReadmeFile>README.md</PackageReadmeFile>
<!-- Only the shipped library generates XML docs (matching src/Contracts). The Cli and
+16 -4
View File
@@ -94,6 +94,12 @@ goroutine cleanup. Raw protobuf messages remain available through the
`errors.As` for `GatewayError`, `CommandError`, and `MxAccessError`; command
errors preserve the raw reply.
`EnsureMxAccessSuccess` follows COM semantics: only a **negative** HRESULT is a
failure, so positive success codes such as `S_FALSE` (1) pass. `StatusSucceeded`
judges each `MXSTATUS_PROXY` entry by its category — an entry fails when
`Category` is not `MX_STATUS_CATEGORY_OK`, and the raw `Success` member is a
diagnostic that never decides the verdict. A nil entry is success.
### Reconnect-replay gap
Each `EventResult` carries exactly one of `Event`, `ReplayGap`, or `Err`. When
@@ -177,7 +183,11 @@ parity holds: a `WriteSecured` issued without a matching prior `AuthenticateUser
and supervisory advise fails natively, and that failure is surfaced unchanged
rather than pre-empted. The CLI exposes `authenticate-user` (credential via
`-password-env`, default `MXGATEWAY_VERIFY_PASSWORD`, or `-password`) and
`write-secured`.
`write-secured`. The credential is required: a missing or empty resolved value is
a usage error naming the flag and the variable, so the CLI fails before dialing
instead of authenticating with an empty password. `MXGATEWAY_VERIFY_PASSWORD` is
the canonical variable across all five client CLIs — see
[Cross-Language Smoke Matrix](../../docs/CrossLanguageSmokeMatrix.md).
### Array writes replace the whole array
@@ -461,7 +471,7 @@ go run ./cmd/mxgw-go smoke -endpoint $env:MXGATEWAY_ENDPOINT -plaintext -api-key
The module is resolved directly from the git repo — no package registry:
````bash
go get gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go@v0.1.1
go get gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go@v0.2.0
````
Then import:
@@ -484,11 +494,13 @@ Go modules in monorepo subdirectories use prefixed tags. To tag a release
from this repo:
````bash
pwsh scripts/tag-go-module.ps1 -Version v0.1.1 -Push
pwsh scripts/tag-go-module.ps1 -Version v0.2.0 -Push
````
The script validates semver, refuses to tag with uncommitted tracked
changes, creates an annotated tag `clients/go/v0.1.1`, and (with `-Push`)
changes, verifies `clients/go/mxgateway/version.go`'s `ClientVersion`
matches the requested tag version (failing the tag otherwise — CLI-21/CLI-39),
creates an annotated tag `clients/go/v0.2.0`, and (with `-Push`)
pushes it to origin.
## Related Documentation
+62 -4
View File
@@ -57,6 +57,25 @@ type commandReplyOutput struct {
Reply json.RawMessage `json:"reply"`
}
// replayGapRow is the JSON row stream-events emits for a reconnect-replay gap:
// {"replayGap":{"requestedAfterSequence":N,"oldestAvailableSequence":N}}.
//
// The cursors are typed by hand rather than marshalled with protojson on
// purpose. The proto3 JSON mapping renders 64-bit integers as JSON *strings*
// ("7"), but the Rust and Python CLIs emit JSON *numbers* (7) for this row —
// routing through protojson would silently make Go the odd one out and break
// the cross-language smoke matrix's row comparison. encoding/json renders
// uint64 as a number, which is the canonical rendering here.
type replayGapRow struct {
ReplayGap replayGapCursors `json:"replayGap"`
}
// replayGapCursors is the nested cursor object of replayGapRow.
type replayGapCursors struct {
RequestedAfterSequence uint64 `json:"requestedAfterSequence"`
OldestAvailableSequence uint64 `json:"oldestAvailableSequence"`
}
func main() {
if err := runWithIO(context.Background(), os.Args[1:], os.Stdout, os.Stderr); err != nil {
fmt.Fprintln(os.Stderr, err)
@@ -427,6 +446,11 @@ func runWriteSecured(ctx context.Context, args []string, stdout, stderr io.Write
return writeCommandOutput(stdout, *jsonOutput, "write-secured", options, reply, err)
}
// defaultVerifyPasswordEnv is the canonical CLI credential environment variable,
// shared by every official client CLI (CLI-45) so one exported variable drives
// the same operator workflow in all five languages.
const defaultVerifyPasswordEnv = "MXGATEWAY_VERIFY_PASSWORD"
func runAuthenticateUser(ctx context.Context, args []string, stdout, stderr io.Writer) error {
flags := flag.NewFlagSet("authenticate-user", flag.ContinueOnError)
flags.SetOutput(stderr)
@@ -439,7 +463,7 @@ func runAuthenticateUser(ctx context.Context, args []string, stdout, stderr io.W
// prefer the environment variable so it stays out of shell history and the
// process table. The -password flag remains for non-interactive scripting.
password := flags.String("password", "", "verify-user password (prefer -password-env)")
passwordEnv := flags.String("password-env", "MXGATEWAY_VERIFY_PASSWORD", "environment variable containing the verify-user password")
passwordEnv := flags.String("password-env", defaultVerifyPasswordEnv, "environment variable containing the verify-user password")
if err := flags.Parse(args); err != nil {
return err
@@ -452,8 +476,18 @@ func runAuthenticateUser(ctx context.Context, args []string, stdout, stderr io.W
}
resolvedPassword := *password
if resolvedPassword == "" && *passwordEnv != "" {
resolvedPassword = os.Getenv(*passwordEnv)
envName := *passwordEnv
if envName == "" {
envName = defaultVerifyPasswordEnv
}
if resolvedPassword == "" {
resolvedPassword = os.Getenv(envName)
}
// Fail fast rather than dialing: an unset or empty variable must not become a
// real MXAccess authentication attempt with an empty credential. The message
// names only the flag and the variable — never the resolved value.
if resolvedPassword == "" {
return fmt.Errorf("a password is required via -password or the %s environment variable", envName)
}
client, options, err := dialForCommand(ctx, common)
@@ -970,7 +1004,31 @@ func runStreamEvents(ctx context.Context, args []string, stdout, stderr io.Write
if result.Err != nil {
return result.Err
}
if *jsonOutput {
// A reconnect-replay gap is a typed signal, not an event: the library
// clears Event on it, so formatting Event here would print a meaningless
// zero row and discard the resume cursors the operator needs. Render it
// as its own row (matching the Rust CLI) and count it toward -limit like
// any other emitted row.
if result.IsReplayGap() {
if *jsonOutput {
row, err := json.Marshal(replayGapRow{
ReplayGap: replayGapCursors{
RequestedAfterSequence: result.ReplayGap.GetRequestedAfterSequence(),
OldestAvailableSequence: result.ReplayGap.GetOldestAvailableSequence(),
},
})
if err != nil {
return err
}
fmt.Fprintln(stdout, string(row))
} else {
fmt.Fprintf(
stdout,
"REPLAY_GAP requested_after=%d oldest_available=%d\n",
result.ReplayGap.GetRequestedAfterSequence(),
result.ReplayGap.GetOldestAvailableSequence())
}
} else if *jsonOutput {
fmt.Fprintln(stdout, string(mustMarshalProto(result.Event)))
} else {
fmt.Fprintf(stdout, "%d %s\n", result.Event.GetWorkerSequence(), result.Event.GetFamily())
+180
View File
@@ -598,6 +598,69 @@ func TestRunAuthenticateUserRequiresVerifyUser(t *testing.T) {
}
}
// TestRunAuthenticateUserRejectsEmptyPassword pins the CLI-45 fail-fast contract:
// an unresolved credential must abort before dialing rather than authenticating
// with an empty password, and the usage error must name both -password and the
// canonical environment variable without echoing any value.
func TestRunAuthenticateUserRejectsEmptyPassword(t *testing.T) {
t.Setenv("MXGATEWAY_VERIFY_PASSWORD", "")
var stdout, stderr bytes.Buffer
err := runWithIO(t.Context(), []string{
"authenticate-user",
"-session-id", "s1",
"-verify-user", "operator",
"-plaintext",
"-api-key", "test",
}, &stdout, &stderr)
if err == nil {
t.Fatal("authenticate-user without a credential must fail before dialing")
}
if !strings.Contains(err.Error(), "-password") {
t.Fatalf("error must name the -password flag: %v", err)
}
if !strings.Contains(err.Error(), "MXGATEWAY_VERIFY_PASSWORD") {
t.Fatalf("error must name the canonical environment variable: %v", err)
}
}
// TestRunAuthenticateUserReadsPasswordFromCanonicalEnv pins that the default
// -password-env is MXGATEWAY_VERIFY_PASSWORD: with it set the credential guard
// passes and the command proceeds past it to the dial, which fails against an
// unused port under a short context — proving the guard was cleared without
// needing a live gateway.
func TestRunAuthenticateUserReadsPasswordFromCanonicalEnv(t *testing.T) {
t.Setenv("MXGATEWAY_VERIFY_PASSWORD", "env-sourced-credential")
ctx, cancel := context.WithTimeout(t.Context(), 200*time.Millisecond)
defer cancel()
var stdout, stderr bytes.Buffer
err := runWithIO(ctx, []string{
"authenticate-user",
"-session-id", "s1",
"-verify-user", "operator",
"-endpoint", "127.0.0.1:1",
"-plaintext",
"-api-key", "test",
"-call-timeout", "1s",
}, &stdout, &stderr)
if err == nil {
t.Fatal("expected the dial/RPC to fail against an unused port")
}
if strings.Contains(err.Error(), "flag provided but not defined") {
t.Fatalf("test invoked an unknown flag, so it never reached the guard: %v", err)
}
if strings.Contains(err.Error(), "a password is required") {
t.Fatalf("credential guard must be satisfied from %s: %v", "MXGATEWAY_VERIFY_PASSWORD", err)
}
if strings.Contains(err.Error(), "env-sourced-credential") ||
strings.Contains(stdout.String(), "env-sourced-credential") ||
strings.Contains(stderr.String(), "env-sourced-credential") {
t.Fatal("the resolved credential must never be echoed")
}
}
// TestRunWriteBulkVariantRejectsMismatchedHandlesAndValues pins the len-mismatch
// guard so a write-bulk with unequal item-handles / values counts fails fast
// before any dial.
@@ -617,3 +680,120 @@ func TestRunWriteBulkVariantRejectsMismatchedHandlesAndValues(t *testing.T) {
t.Fatalf("write-bulk mismatched handles/values error = %v", err)
}
}
// replayGapFakeGateway streams the gateway's reconnect-replay sentinel (an MxEvent
// carrying replay_gap, family UNSPECIFIED, body unset) followed by one normal data
// event — exactly what a resume whose cursor predates the retained replay ring sees.
type replayGapFakeGateway struct {
pb.UnimplementedMxAccessGatewayServer
}
func (g *replayGapFakeGateway) StreamEvents(
req *pb.StreamEventsRequest,
stream grpc.ServerStreamingServer[pb.MxEvent],
) error {
sentinel := &pb.MxEvent{
SessionId: req.GetSessionId(),
Family: pb.MxEventFamily_MX_EVENT_FAMILY_UNSPECIFIED,
ReplayGap: &pb.ReplayGap{
RequestedAfterSequence: 7,
OldestAvailableSequence: 42,
},
}
if err := stream.Send(sentinel); err != nil {
return err
}
return stream.Send(&pb.MxEvent{
SessionId: req.GetSessionId(),
Family: pb.MxEventFamily_MX_EVENT_FAMILY_ON_DATA_CHANGE,
WorkerSequence: 43,
})
}
func startReplayGapGateway(t *testing.T) string {
t.Helper()
listener, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatalf("listen: %v", err)
}
server := grpc.NewServer()
pb.RegisterMxAccessGatewayServer(server, &replayGapFakeGateway{})
go func() { _ = server.Serve(listener) }()
t.Cleanup(func() {
server.Stop()
_ = listener.Close()
})
return listener.Addr().String()
}
// TestRunStreamEventsPrintsReplayGap pins CLI-36: the CLI must render the typed
// ReplayGap signal in both output modes instead of formatting the library's
// cleared Event field (which printed "0 MX_EVENT_FAMILY_UNSPECIFIED" in text mode
// and an empty object in JSON mode, destroying the resume cursors).
func TestRunStreamEventsPrintsReplayGap(t *testing.T) {
endpoint := startReplayGapGateway(t)
baseArgs := []string{
"stream-events",
"-endpoint", endpoint,
"-plaintext",
"-api-key", "test",
"-session-id", "gap-session",
"-after-worker-sequence", "7",
"-limit", "2",
}
var stdout, stderr bytes.Buffer
if err := runWithIO(t.Context(), baseArgs, &stdout, &stderr); err != nil {
t.Fatalf("runWithIO() error = %v; stderr = %s", err, stderr.String())
}
text := stdout.String()
if !strings.Contains(text, "REPLAY_GAP requested_after=7 oldest_available=42") {
t.Fatalf("stream-events text output missing typed gap row: %q", text)
}
if strings.Contains(text, "0 MX_EVENT_FAMILY_UNSPECIFIED") {
t.Fatalf("stream-events text output destroyed the gap into a zero row: %q", text)
}
if !strings.Contains(text, "43 MX_EVENT_FAMILY_ON_DATA_CHANGE") {
t.Fatalf("stream-events text output dropped the normal event: %q", text)
}
stdout.Reset()
stderr.Reset()
if err := runWithIO(t.Context(), append(baseArgs, "-json"), &stdout, &stderr); err != nil {
t.Fatalf("runWithIO(-json) error = %v; stderr = %s", err, stderr.String())
}
lines := strings.Split(strings.TrimSpace(stdout.String()), "\n")
if len(lines) != 2 {
t.Fatalf("stream-events -json emitted %d rows, want 2: %q", len(lines), stdout.String())
}
// The cursors must decode as JSON numbers, not the strings the proto3 JSON
// mapping would produce for 64-bit fields: the Rust and Python CLIs emit
// numbers, and the cross-language matrix compares these rows across clients.
var gapRow struct {
ReplayGap *struct {
RequestedAfterSequence uint64 `json:"requestedAfterSequence"`
OldestAvailableSequence uint64 `json:"oldestAvailableSequence"`
} `json:"replayGap"`
}
if err := json.Unmarshal([]byte(lines[0]), &gapRow); err != nil {
t.Fatalf("parse gap row: %v\nrow: %s", err, lines[0])
}
if gapRow.ReplayGap == nil {
t.Fatalf("stream-events -json first row is not a replayGap row: %s", lines[0])
}
if gapRow.ReplayGap.RequestedAfterSequence != 7 || gapRow.ReplayGap.OldestAvailableSequence != 42 {
t.Fatalf("stream-events -json gap cursors = %+v, want 7/42", *gapRow.ReplayGap)
}
// Belt and braces on the value type: a protojson-rendered `"7"` already
// fails the decode above (encoding/json rejects a JSON string for an
// untagged uint64 field), but assert the raw bytes so a regression names
// the real problem instead of surfacing as an opaque unmarshal error.
if !strings.Contains(lines[0], `"requestedAfterSequence":7`) ||
!strings.Contains(lines[0], `"oldestAvailableSequence":42`) {
t.Fatalf("stream-events -json gap cursors must be JSON numbers, got: %s", lines[0])
}
}
+29 -2
View File
@@ -1,6 +1,16 @@
Set-StrictMode -Version Latest
$ErrorActionPreference = 'Stop'
# Pinned generator baseline. The committed Go bindings stamp these plugin versions in their
# headers (protoc-gen-go v1.36.11 / protoc-gen-go-grpc v1.6.2). Plugin-version drift rewrites
# those header stamps, so a regeneration on an off-pin machine would churn the tree and make
# check-codegen Check 4 false-fail (or mask real drift under churn). Assert the exact versions
# so a regen is deterministic. protoc itself is warn-only (source_code_info is normalized out of
# the committed bindings), matching publish-client-proto-inputs.ps1.
$PinnedProtocGenGoVersion = 'protoc-gen-go v1.36.11'
$PinnedProtocGenGoGrpcVersion = 'protoc-gen-go-grpc 1.6.2'
$PinnedProtocVersion = 'libprotoc 34.1'
$repoRoot = Resolve-Path (Join-Path $PSScriptRoot '..\..')
$protoRoot = Join-Path $repoRoot 'src\ZB.MOM.WW.MxGateway.Contracts\Protos'
$outputRoot = Join-Path $PSScriptRoot 'internal\generated'
@@ -36,8 +46,25 @@ $wingetProtoc = if ($env:LOCALAPPDATA) {
$goBin = if ($env:USERPROFILE) { Join-Path $env:USERPROFILE 'go\bin' } elseif ($env:HOME) { Join-Path $env:HOME 'go/bin' } else { $null }
$protoc = Resolve-Tool -Names @('protoc', 'protoc.exe') -FallbackPaths @($wingetProtoc)
$protocGenGo = Resolve-Tool -Names @('protoc-gen-go', 'protoc-gen-go.exe') -FallbackPaths @((if ($goBin) { Join-Path $goBin 'protoc-gen-go.exe' }), (if ($goBin) { Join-Path $goBin 'protoc-gen-go' }))
$protocGenGoGrpc = Resolve-Tool -Names @('protoc-gen-go-grpc', 'protoc-gen-go-grpc.exe') -FallbackPaths @((if ($goBin) { Join-Path $goBin 'protoc-gen-go-grpc.exe' }), (if ($goBin) { Join-Path $goBin 'protoc-gen-go-grpc' }))
$protocGenGo = Resolve-Tool -Names @('protoc-gen-go', 'protoc-gen-go.exe') -FallbackPaths @(($(if ($goBin) { Join-Path $goBin 'protoc-gen-go.exe' })), ($(if ($goBin) { Join-Path $goBin 'protoc-gen-go' })))
$protocGenGoGrpc = Resolve-Tool -Names @('protoc-gen-go-grpc', 'protoc-gen-go-grpc.exe') -FallbackPaths @(($(if ($goBin) { Join-Path $goBin 'protoc-gen-go-grpc.exe' })), ($(if ($goBin) { Join-Path $goBin 'protoc-gen-go-grpc' })))
# Assert the pinned plugin versions before generating so Check 4 cannot false-fail (or mask drift)
# on an off-pin machine. protoc is warn-only.
$protocGenGoVersion = (& $protocGenGo --version 2>&1 | Out-String).Trim()
if ($protocGenGoVersion -ne $PinnedProtocGenGoVersion) {
throw "protoc-gen-go reports '$protocGenGoVersion', but regeneration is pinned to '$PinnedProtocGenGoVersion'. " +
"Install the pin: go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11"
}
$protocGenGoGrpcVersion = (& $protocGenGoGrpc --version 2>&1 | Out-String).Trim()
if ($protocGenGoGrpcVersion -ne $PinnedProtocGenGoGrpcVersion) {
throw "protoc-gen-go-grpc reports '$protocGenGoGrpcVersion', but regeneration is pinned to '$PinnedProtocGenGoGrpcVersion'. " +
"Install the pin: go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.6.2"
}
$protocVersion = (& $protoc --version 2>&1 | Out-String).Trim()
if ($protocVersion -ne $PinnedProtocVersion) {
Write-Warning "protoc reports '$protocVersion', pin is '$PinnedProtocVersion'. Descriptor comments are normalized out of the committed Go bindings, so patch drift is tolerated; keep CI on the pin."
}
# protoc discovers the plugins on PATH; prepend the directories the resolved plugins live in.
$env:Path = (Split-Path $protocGenGo -Parent) + [System.IO.Path]::PathSeparator + (Split-Path $protocGenGoGrpc -Parent) + [System.IO.Path]::PathSeparator + $env:Path
@@ -5974,8 +5974,12 @@ func (x *WorkerInfoReply) GetMxaccessClsid() string {
}
type DrainEventsReply struct {
state protoimpl.MessageState `protogen:"open.v1"`
Events []*MxEvent `protobuf:"bytes,1,rep,name=events,proto3" json:"events,omitempty"`
state protoimpl.MessageState `protogen:"open.v1"`
// The reply is bounded by both a server-side count cap and the negotiated
// worker-frame byte cap; a reply may therefore carry fewer events than
// `max_events` and fewer than are queued. Callers drain iteratively until an
// empty reply.
Events []*MxEvent `protobuf:"bytes,1,rep,name=events,proto3" json:"events,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
@@ -6411,6 +6415,11 @@ type ReplayGap struct {
// after_worker_sequence = oldest_available_sequence - 1 in the next
// StreamEventsRequest, which will cause the server to replay starting at
// oldest_available_sequence (the first retained event).
// When nothing is retained (the replay ring is empty), this is the next sequence
// that can be delivered — `highest observed + 1` — and the `oldest - 1` resume
// formula remains valid: it resolves to the highest sequence already seen, so the
// follow-up resume replays nothing, reports no gap, and every newer live event
// passes. The interval evicted is unchanged.
OldestAvailableSequence uint64 `protobuf:"varint,2,opt,name=oldest_available_sequence,json=oldestAvailableSequence,proto3" json:"oldest_available_sequence,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
@@ -431,8 +431,17 @@ type GatewayHello struct {
SupportedProtocolVersion uint32 `protobuf:"varint,1,opt,name=supported_protocol_version,json=supportedProtocolVersion,proto3" json:"supported_protocol_version,omitempty"`
Nonce string `protobuf:"bytes,2,opt,name=nonce,proto3" json:"nonce,omitempty"`
GatewayVersion string `protobuf:"bytes,3,opt,name=gateway_version,json=gatewayVersion,proto3" json:"gateway_version,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
// Maximum worker-frame payload size, in bytes, negotiated by the gateway from its
// configured pipe limit. The worker adopts this as its frame-protocol MaxMessageBytes
// instead of a hard-coded default; 0 (an older gateway that never set the field) means
// "use the worker's built-in default". Sits above the public gRPC cap by an
// envelope-overhead margin so an accepted gRPC payload always fits one worker frame.
// Every worker->gateway frame — events, heartbeats, faults, and control replies
// including DrainEvents — must serialize within this limit; reply builders truncate
// to fit rather than emit an oversized frame.
MaxFrameBytes uint32 `protobuf:"varint,4,opt,name=max_frame_bytes,json=maxFrameBytes,proto3" json:"max_frame_bytes,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *GatewayHello) Reset() {
@@ -486,6 +495,13 @@ func (x *GatewayHello) GetGatewayVersion() string {
return ""
}
func (x *GatewayHello) GetMaxFrameBytes() uint32 {
if x != nil {
return x.MaxFrameBytes
}
return 0
}
type WorkerHello struct {
state protoimpl.MessageState `protogen:"open.v1"`
ProtocolVersion uint32 `protobuf:"varint,1,opt,name=protocol_version,json=protocolVersion,proto3" json:"protocol_version,omitempty"`
@@ -1109,11 +1125,12 @@ const file_mxaccess_worker_proto_rawDesc = "" +
"\fworker_event\x18\x12 \x01(\v2\x1f.mxaccess_worker.v1.WorkerEventH\x00R\vworkerEvent\x12P\n" +
"\x10worker_heartbeat\x18\x13 \x01(\v2#.mxaccess_worker.v1.WorkerHeartbeatH\x00R\x0fworkerHeartbeat\x12D\n" +
"\fworker_fault\x18\x14 \x01(\v2\x1f.mxaccess_worker.v1.WorkerFaultH\x00R\vworkerFaultB\x06\n" +
"\x04body\"\x8b\x01\n" +
"\x04body\"\xb3\x01\n" +
"\fGatewayHello\x12<\n" +
"\x1asupported_protocol_version\x18\x01 \x01(\rR\x18supportedProtocolVersion\x12\x14\n" +
"\x05nonce\x18\x02 \x01(\tR\x05nonce\x12'\n" +
"\x0fgateway_version\x18\x03 \x01(\tR\x0egatewayVersion\"\xa1\x01\n" +
"\x0fgateway_version\x18\x03 \x01(\tR\x0egatewayVersion\x12&\n" +
"\x0fmax_frame_bytes\x18\x04 \x01(\rR\rmaxFrameBytes\"\xa1\x01\n" +
"\vWorkerHello\x12)\n" +
"\x10protocol_version\x18\x01 \x01(\rR\x0fprotocolVersion\x12\x14\n" +
"\x05nonce\x18\x02 \x01(\tR\x05nonce\x12*\n" +
+148 -9
View File
@@ -10,7 +10,9 @@ import (
pb "gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go/internal/generated"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/metadata"
"google.golang.org/grpc/status"
"google.golang.org/grpc/test/bufconn"
)
@@ -200,6 +202,136 @@ func TestEventsSlowConsumerYieldsErrSlowConsumerBeforeClose(t *testing.T) {
}
}
func TestEventsFullBufferTerminalErrorKeepsRootCause(t *testing.T) {
fake := &fakeGatewayServer{
streamStarted: make(chan struct{}),
streamDone: make(chan struct{}),
streamEventCount: eventBufferSize,
streamTerminalErr: status.Error(codes.Internal, "boom"),
}
client, cleanup := newBufconnClient(t, fake)
defer cleanup()
session := NewSessionForID(client, "session-1")
events, err := session.EventsAfter(context.Background(), 0)
if err != nil {
t.Fatalf("EventsAfter() error = %v", err)
}
<-fake.streamStarted
// Do not drain until the stream has fully ended: the server sends exactly
// eventBufferSize events (filling the data slots) and then returns a genuine
// terminal gRPC error. The client must report that error as itself, using the
// reserved slot, rather than mislabeling it as ErrSlowConsumer.
select {
case <-fake.streamDone:
case <-time.After(2 * time.Second):
t.Fatal("event stream did not stop after terminal error")
}
// streamDone fires when the server returns; the client's producer goroutine
// still needs a moment to drain the gRPC stream, fill all data slots, and
// enqueue the terminal result. Let it settle before draining so the buffer is
// genuinely full when the terminal error is processed (which is what makes the
// mislabel bug observable).
time.Sleep(250 * time.Millisecond)
var last EventResult
gotResult := false
for {
select {
case res, ok := <-events:
if !ok {
if !gotResult {
t.Fatal("events channel closed without yielding any result")
}
var gwErr *GatewayError
if !errors.As(last.Err, &gwErr) {
t.Fatalf("final event result err is %T, want *GatewayError", last.Err)
}
if code := status.Code(last.Err); code != codes.Internal {
t.Fatalf("final event result gRPC code = %s, want %s", code, codes.Internal)
}
if errors.Is(last.Err, ErrSlowConsumer) {
t.Fatalf("final event result err = %v, must not be mislabeled as ErrSlowConsumer", last.Err)
}
return
}
last = res
gotResult = true
case <-time.After(2 * time.Second):
t.Fatal("events channel did not close after terminal error")
}
}
}
// TestSubscribeEventsFullBufferDeliversTerminalError is the CLI-44 regression for
// the never-drop Subscribe path. SubscribeEvents/SubscribeEventsAfter use
// cancelWhenResultBufferFull=false, so ordinary sends are blocking and uncapped and
// can fill every slot in the results channel — including the reserved terminal slot.
// A genuine terminal Recv error must still be delivered as the final result, never
// silently dropped. The server sends eventBufferSize+eventBufferReservedSlots events
// (filling every slot) and then returns a genuine gRPC error; with an unconditional
// non-blocking terminal send the error is dropped, so this fails red until the send
// path blocks for the never-drop mode.
func TestSubscribeEventsFullBufferDeliversTerminalError(t *testing.T) {
fake := &fakeGatewayServer{
streamStarted: make(chan struct{}),
streamDone: make(chan struct{}),
streamEventCount: eventBufferSize + eventBufferReservedSlots,
streamTerminalErr: status.Error(codes.Internal, "boom"),
}
client, cleanup := newBufconnClient(t, fake)
defer cleanup()
session := NewSessionForID(client, "session-1")
subscription, err := session.SubscribeEvents(context.Background())
if err != nil {
t.Fatalf("SubscribeEvents() error = %v", err)
}
defer subscription.Close()
<-fake.streamStarted
// Wait for the server to finish sending every event and return the terminal
// error, so the producer goroutine has filled every buffered slot before the
// terminal result is processed. That is what makes the dropped-terminal bug
// observable: with the buffer full, an unconditional non-blocking send discards
// the terminal error.
select {
case <-fake.streamDone:
case <-time.After(2 * time.Second):
t.Fatal("event stream did not stop after terminal error")
}
time.Sleep(250 * time.Millisecond)
// Drain fully. Every data event, then the terminal gRPC error as the final
// result, must arrive; the channel must not close without yielding it.
events := subscription.Events()
var last EventResult
gotResult := false
for {
select {
case res, ok := <-events:
if !ok {
if !gotResult {
t.Fatal("events channel closed without yielding any result")
}
var gwErr *GatewayError
if !errors.As(last.Err, &gwErr) {
t.Fatalf("final event result err is %T (%v), want the terminal *GatewayError; it was dropped", last.Err, last.Err)
}
if code := status.Code(last.Err); code != codes.Internal {
t.Fatalf("final event result gRPC code = %s, want %s", code, codes.Internal)
}
return
}
last = res
gotResult = true
case <-time.After(2 * time.Second):
t.Fatal("events channel did not close after terminal error")
}
}
}
func TestEventsSurfacesReplayGapSentinelAsTypedSignal(t *testing.T) {
fake := &fakeGatewayServer{
streamStarted: make(chan struct{}),
@@ -694,15 +826,16 @@ func newBufconnClient(t *testing.T, fake *fakeGatewayServer) (*Client, func()) {
type fakeGatewayServer struct {
pb.UnimplementedMxAccessGatewayServer
openReply *pb.OpenSessionReply
openAuth string
streamAuth string
streamStarted chan struct{}
streamDone chan struct{}
streamEventCount int
streamReplayGap *pb.ReplayGap
invokeReply *pb.MxCommandReply
invokeRequest *pb.MxCommandRequest
openReply *pb.OpenSessionReply
openAuth string
streamAuth string
streamStarted chan struct{}
streamDone chan struct{}
streamEventCount int
streamReplayGap *pb.ReplayGap
streamTerminalErr error
invokeReply *pb.MxCommandReply
invokeRequest *pb.MxCommandRequest
}
func (s *fakeGatewayServer) OpenSession(ctx context.Context, req *pb.OpenSessionRequest) (*pb.OpenSessionReply, error) {
@@ -772,6 +905,12 @@ func (s *fakeGatewayServer) StreamEvents(req *pb.StreamEventsRequest, stream grp
return err
}
}
if s.streamTerminalErr != nil {
// Return a genuine terminal stream error immediately after sending the
// events, without waiting on the client to cancel. This exercises the
// Recv-error path while the client's result buffer is still full.
return s.streamTerminalErr
}
<-stream.Context().Done()
return io.EOF
}
@@ -0,0 +1,197 @@
package mxgateway
import (
"context"
"errors"
"os"
"path/filepath"
"strings"
"testing"
pb "gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go/internal/generated"
"google.golang.org/protobuf/encoding/protojson"
)
// loadCommandReplyFixture parses a shared command-reply fixture into an
// MxCommandReply so the Go client can be driven through the same wire shapes the
// other language clients exercise.
func loadCommandReplyFixture(t *testing.T, name string) *pb.MxCommandReply {
t.Helper()
path := filepath.Join("..", "..", "proto", "fixtures", "behavior", "command-replies", name)
data, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read fixture %s: %v", name, err)
}
var reply pb.MxCommandReply
if err := protojson.Unmarshal(data, &reply); err != nil {
t.Fatalf("parse fixture %s: %v", name, err)
}
return &reply
}
func TestAuthenticateUserMissingPayloadReturnsMalformedReplyError(t *testing.T) {
fake := &fakeGatewayServer{
invokeReply: loadCommandReplyFixture(t, "authenticate-user.missing-payload.reply.json"),
}
client, cleanup := newBufconnClient(t, fake)
defer cleanup()
session := NewSessionForID(client, "session-1")
_, err := session.AuthenticateUser(context.Background(), 12, "operator", "secret")
var malformed *MalformedReplyError
if !errors.As(err, &malformed) {
t.Fatalf("AuthenticateUser() error = %v (%T), want *MalformedReplyError", err, err)
}
if malformed.Op != "authenticate user" {
t.Fatalf("MalformedReplyError.Op = %q, want %q", malformed.Op, "authenticate user")
}
}
func TestAuthenticateUserReturnValueOnlyUsesInt32ReturnValue(t *testing.T) {
fake := &fakeGatewayServer{
invokeReply: loadCommandReplyFixture(t, "authenticate-user.return-value-only.reply.json"),
}
client, cleanup := newBufconnClient(t, fake)
defer cleanup()
session := NewSessionForID(client, "session-1")
userID, err := session.AuthenticateUser(context.Background(), 12, "operator", "secret")
if err != nil {
t.Fatalf("AuthenticateUser() error = %v", err)
}
if userID != 7 {
t.Fatalf("AuthenticateUser() = %d, want 7", userID)
}
}
// AddBufferedItem shares the prefer-payload / int32-return-value / malformed
// fallback code path; cover both branches for one of the siblings.
func TestAddBufferedItemFallbackHonoursReturnValueAndReportsMalformed(t *testing.T) {
t.Run("return-value-only", func(t *testing.T) {
fake := &fakeGatewayServer{
invokeReply: loadCommandReplyFixture(t, "authenticate-user.return-value-only.reply.json"),
}
client, cleanup := newBufconnClient(t, fake)
defer cleanup()
session := NewSessionForID(client, "session-1")
itemHandle, err := session.AddBufferedItem(context.Background(), 12, "Area001.Pump001.Speed", "runtime")
if err != nil {
t.Fatalf("AddBufferedItem() error = %v", err)
}
if itemHandle != 7 {
t.Fatalf("AddBufferedItem() = %d, want 7", itemHandle)
}
})
t.Run("missing-payload", func(t *testing.T) {
fake := &fakeGatewayServer{
invokeReply: loadCommandReplyFixture(t, "authenticate-user.missing-payload.reply.json"),
}
client, cleanup := newBufconnClient(t, fake)
defer cleanup()
session := NewSessionForID(client, "session-1")
_, err := session.AddBufferedItem(context.Background(), 12, "Area001.Pump001.Speed", "runtime")
var malformed *MalformedReplyError
if !errors.As(err, &malformed) {
t.Fatalf("AddBufferedItem() error = %v (%T), want *MalformedReplyError", err, err)
}
if malformed.Op != "add buffered item" {
t.Fatalf("MalformedReplyError.Op = %q, want %q", malformed.Op, "add buffered item")
}
})
}
// TestAuthenticateUserScrubsEchoedCredentialFromError is the CLI-40 regression:
// a gateway diagnostic that echoes the raw credential back must never reach the
// caller's surfaced error text.
func TestAuthenticateUserScrubsEchoedCredentialFromError(t *testing.T) {
const credential = "sup3rSecretVerify9f3a2b"
fake := &fakeGatewayServer{
invokeReply: loadCommandReplyFixture(t, "authenticate-user.echoed-credential.reply.json"),
}
client, cleanup := newBufconnClient(t, fake)
defer cleanup()
session := NewSessionForID(client, "session-1")
_, err := session.AuthenticateUser(context.Background(), 12, "operator", credential)
if err == nil {
t.Fatal("AuthenticateUser() error = nil, want an MXAccess failure")
}
message := err.Error()
if strings.Contains(message, credential) {
t.Fatalf("surfaced error leaked the credential: %q", message)
}
if !strings.Contains(message, "<redacted>") {
t.Fatalf("surfaced error missing redaction marker: %q", message)
}
}
// TestAuthenticateUserScrubsEchoedCredentialFromStructuredReply is the CLI-40
// follow-up: redacting only the rendered Error() string is not enough. The typed
// *MxAccessError still carries the raw command reply, whose ProtocolStatus.Message,
// DiagnosticMessage, and Statuses[].DiagnosticText echo the credential verbatim. A
// logger dumping structured fields would reintroduce the leak, so the reply the
// typed error carries must be a scrubbed clone. Both the OK+negative-HRESULT and the
// MXACCESS_FAILURE fixtures route to *MxAccessError (via EnsureProtocolSuccess), so
// both must be scrubbed identically.
func TestAuthenticateUserScrubsEchoedCredentialFromStructuredReply(t *testing.T) {
const credential = "sup3rSecretVerify9f3a2b"
fixtures := []string{
"authenticate-user.echoed-credential.reply.json",
"authenticate-user.echoed-credential-mxaccess-failure.reply.json",
}
for _, fixture := range fixtures {
t.Run(fixture, func(t *testing.T) {
fake := &fakeGatewayServer{
invokeReply: loadCommandReplyFixture(t, fixture),
}
client, cleanup := newBufconnClient(t, fake)
defer cleanup()
session := NewSessionForID(client, "session-1")
_, err := session.AuthenticateUser(context.Background(), 12, "operator", credential)
if err == nil {
t.Fatal("AuthenticateUser() error = nil, want an MXAccess failure")
}
var mxErr *MxAccessError
if !errors.As(err, &mxErr) {
t.Fatalf("AuthenticateUser() error = %v (%T), want *MxAccessError", err, err)
}
reply := mxErr.Reply
if reply == nil {
t.Fatal("MxAccessError.Reply is nil, want the scrubbed command reply")
}
if got := reply.GetProtocolStatus().GetMessage(); strings.Contains(got, credential) {
t.Fatalf("MxAccessError.Reply.ProtocolStatus.Message leaked the credential: %q", got)
}
if got := reply.GetDiagnosticMessage(); strings.Contains(got, credential) {
t.Fatalf("MxAccessError.Reply.DiagnosticMessage leaked the credential: %q", got)
}
for i, status := range reply.GetStatuses() {
if got := status.GetDiagnosticText(); strings.Contains(got, credential) {
t.Fatalf("MxAccessError.Reply.Statuses[%d].DiagnosticText leaked the credential: %q", i, got)
}
}
// The wrapped CommandError's status/reply must be scrubbed too.
if mxErr.Command != nil {
if got := mxErr.Command.Status.GetMessage(); strings.Contains(got, credential) {
t.Fatalf("MxAccessError.Command.Status.Message leaked the credential: %q", got)
}
if cmdReply := mxErr.Command.Reply; cmdReply != nil {
if got := cmdReply.GetDiagnosticMessage(); strings.Contains(got, credential) {
t.Fatalf("MxAccessError.Command.Reply.DiagnosticMessage leaked the credential: %q", got)
}
}
}
if got := err.Error(); strings.Contains(got, credential) {
t.Fatalf("rendered error leaked the credential: %q", got)
}
})
}
}
+5 -4
View File
@@ -51,8 +51,9 @@ func TestStatusConversionFixtures(t *testing.T) {
var fixture struct {
Cases []struct {
ID string `json:"id"`
Status json.RawMessage `json:"status"`
ID string `json:"id"`
WantSuccess bool `json:"wantSuccess"`
Status json.RawMessage `json:"status"`
} `json:"cases"`
}
if err := json.Unmarshal(data, &fixture); err != nil {
@@ -65,8 +66,8 @@ func TestStatusConversionFixtures(t *testing.T) {
if err := protojson.Unmarshal(tc.Status, &status); err != nil {
t.Fatalf("parse status: %v", err)
}
if got, want := StatusSucceeded(&status), status.GetSuccess() != 0; got != want {
t.Fatalf("StatusSucceeded() = %v, want %v", got, want)
if got := StatusSucceeded(&status); got != tc.WantSuccess {
t.Fatalf("StatusSucceeded() = %v, want %v", got, tc.WantSuccess)
}
})
}
+119 -7
View File
@@ -6,6 +6,7 @@ import (
"strings"
pb "gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go/internal/generated"
"google.golang.org/protobuf/proto"
)
// redactedSecretMarker is the placeholder substituted for credential material in
@@ -49,20 +50,108 @@ func (e *secretRedactingError) Unwrap() error {
return e.err
}
// redactSecrets wraps err so any occurrence of a non-empty secret in the surfaced
// message is redacted, while errors.As / errors.Is still reach the wrapped typed
// error. It returns nil unchanged and skips wrapping when no non-empty secret is
// supplied, so non-secret-bearing calls keep their original error verbatim.
// scrubReplyStrings returns a clone of reply with every non-empty secret replaced
// by redactedSecretMarker in the free-text fields a gateway diagnostic could echo a
// credential into: ProtocolStatus.Message, DiagnosticMessage, and each
// Statuses[].DiagnosticText. It clones with proto.Clone so the caller's original
// reply is never mutated. A nil reply, or an empty/whitespace-only secret set, is a
// no-op (nil in, nil out; a clone otherwise).
func scrubReplyStrings(reply *pb.MxCommandReply, secrets []string) *pb.MxCommandReply {
if reply == nil {
return nil
}
clone, ok := proto.Clone(reply).(*pb.MxCommandReply)
if !ok {
return reply
}
for _, secret := range secrets {
if secret == "" {
continue
}
if clone.GetProtocolStatus() != nil {
clone.ProtocolStatus.Message = strings.ReplaceAll(clone.GetProtocolStatus().GetMessage(), secret, redactedSecretMarker)
}
clone.DiagnosticMessage = strings.ReplaceAll(clone.GetDiagnosticMessage(), secret, redactedSecretMarker)
for _, status := range clone.GetStatuses() {
status.DiagnosticText = strings.ReplaceAll(status.GetDiagnosticText(), secret, redactedSecretMarker)
}
}
return clone
}
// scrubProtocolStatusMessage returns a clone of status with every non-empty secret
// redacted from its Message, leaving the original untouched.
func scrubProtocolStatusMessage(status *ProtocolStatus, secrets []string) *ProtocolStatus {
if status == nil {
return nil
}
clone, ok := proto.Clone(status).(*ProtocolStatus)
if !ok {
return status
}
for _, secret := range secrets {
if secret != "" {
clone.Message = strings.ReplaceAll(clone.GetMessage(), secret, redactedSecretMarker)
}
}
return clone
}
// redactSecrets scrubs a non-empty secret set from the error it surfaces. When the
// wrapped error is a typed *MxAccessError or *CommandError it is rebuilt carrying
// scrubbed clones of its reply and protocol status, so a caller logging the typed
// error's structured fields cannot reintroduce the credential the rendered message
// hides. The rebuilt (or original, for other error types) value is then wrapped in
// secretRedactingError as a belt-and-suspenders scrub of any remaining rendered
// text. errors.As / errors.Is still reach the typed error through the wrapper. It
// returns nil unchanged and skips all work when no non-empty secret is supplied, so
// non-secret-bearing calls keep their original error verbatim.
func redactSecrets(err error, secrets ...string) error {
if err == nil {
return nil
}
hasSecret := false
for _, secret := range secrets {
if secret != "" {
return &secretRedactingError{err: err, secrets: secrets}
hasSecret = true
break
}
}
return err
if !hasSecret {
return err
}
rebuilt := rebuildScrubbedError(err, secrets)
return &secretRedactingError{err: rebuilt, secrets: secrets}
}
// rebuildScrubbedError rebuilds the typed error carrying scrubbed clones of any
// command reply / protocol status it holds, so credential text never survives in the
// error's structured fields. Non-reply-bearing error types are returned unchanged.
func rebuildScrubbedError(err error, secrets []string) error {
switch typed := err.(type) {
case *MxAccessError:
return &MxAccessError{
Command: scrubCommandError(typed.Command, secrets),
Reply: scrubReplyStrings(typed.Reply, secrets),
}
case *CommandError:
return scrubCommandError(typed, secrets)
default:
return err
}
}
// scrubCommandError rebuilds a CommandError with a scrubbed Status and Reply.
func scrubCommandError(cmd *CommandError, secrets []string) *CommandError {
if cmd == nil {
return nil
}
return &CommandError{
Op: cmd.Op,
Status: scrubProtocolStatusMessage(cmd.Status, secrets),
Reply: scrubReplyStrings(cmd.Reply, secrets),
}
}
// ErrSlowConsumer is the terminal error sent on the Events/EventsAfter
@@ -72,6 +161,25 @@ func redactSecrets(err error, secrets ...string) error {
// dropping events. Match it with errors.Is.
var ErrSlowConsumer = errors.New("mxgateway: event consumer fell behind; stream terminated")
// MalformedReplyError reports an OK command reply that carried neither the
// typed payload the operation expected nor a usable int32 return_value, so the
// client cannot produce a result. It gives every affected helper one uniform,
// inspectable failure instead of silently returning a zero value.
type MalformedReplyError struct {
// Op names the operation whose reply was malformed.
Op string
// Detail explains what the reply was missing.
Detail string
}
// Error returns the formatted malformed-reply message.
func (e *MalformedReplyError) Error() string {
if e == nil {
return ""
}
return fmt.Sprintf("mxgateway: %s returned a malformed reply: %s", e.Op, e.Detail)
}
// GatewayError wraps transport-level gRPC failures.
type GatewayError struct {
// Op names the operation that failed (for example "dial" or "invoke").
@@ -180,11 +288,15 @@ func EnsureProtocolSuccess(op string, status *ProtocolStatus, reply *MxCommandRe
// EnsureMxAccessSuccess returns a typed MxAccessError for failing HRESULTs or
// MXSTATUS_PROXY entries.
//
// Following COM semantics, only a negative HRESULT is a failure — positive
// success codes such as S_FALSE (1) pass. Status entries are judged by
// StatusSucceeded, which branches on the authoritative category.
func EnsureMxAccessSuccess(op string, reply *MxCommandReply) error {
if reply == nil {
return nil
}
if reply.Hresult != nil && reply.GetHresult() != 0 {
if reply.Hresult != nil && reply.GetHresult() < 0 {
return &MxAccessError{Reply: reply}
}
for _, status := range reply.GetStatuses() {
@@ -0,0 +1,115 @@
package mxgateway
import (
"errors"
"strings"
"testing"
pb "gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go/internal/generated"
)
// TestScrubReplyStringsRedactsEveryOccurrence covers the multi-occurrence case:
// one secret appearing across ProtocolStatus.Message, DiagnosticMessage, and every
// Statuses[].DiagnosticText must be fully redacted with no residue.
func TestScrubReplyStringsRedactsEveryOccurrence(t *testing.T) {
const secret = "hunter2"
reply := &pb.MxCommandReply{
ProtocolStatus: &pb.ProtocolStatus{Message: "rejected hunter2 then hunter2 again"},
DiagnosticMessage: "echoed hunter2 back",
Statuses: []*pb.MxStatusProxy{
{DiagnosticText: "first hunter2"},
{DiagnosticText: "second hunter2 and hunter2"},
},
}
scrubbed := scrubReplyStrings(reply, []string{secret})
if strings.Contains(scrubbed.GetProtocolStatus().GetMessage(), secret) {
t.Fatalf("ProtocolStatus.Message still contains the secret: %q", scrubbed.GetProtocolStatus().GetMessage())
}
if strings.Contains(scrubbed.GetDiagnosticMessage(), secret) {
t.Fatalf("DiagnosticMessage still contains the secret: %q", scrubbed.GetDiagnosticMessage())
}
for i, status := range scrubbed.GetStatuses() {
if strings.Contains(status.GetDiagnosticText(), secret) {
t.Fatalf("Statuses[%d].DiagnosticText still contains the secret: %q", i, status.GetDiagnosticText())
}
}
if !strings.Contains(scrubbed.GetProtocolStatus().GetMessage(), redactedSecretMarker) {
t.Fatalf("ProtocolStatus.Message missing redaction marker: %q", scrubbed.GetProtocolStatus().GetMessage())
}
// The original reply must be untouched (scrubReplyStrings clones).
if !strings.Contains(reply.GetDiagnosticMessage(), secret) {
t.Fatal("scrubReplyStrings mutated the original reply instead of cloning it")
}
}
// TestScrubReplyStringsRedactsOverlappingSecrets covers two secrets where one is a
// substring of the other: both must be fully redacted, with no partial leak of the
// longer secret's non-shared remainder.
func TestScrubReplyStringsRedactsOverlappingSecrets(t *testing.T) {
const shortSecret = "pass"
const longSecret = "password123"
reply := &pb.MxCommandReply{
DiagnosticMessage: "value was password123 and also pass",
}
scrubbed := scrubReplyStrings(reply, []string{longSecret, shortSecret})
got := scrubbed.GetDiagnosticMessage()
if strings.Contains(got, shortSecret) {
t.Fatalf("scrubbed message still contains a secret substring %q: %q", shortSecret, got)
}
if strings.Contains(got, longSecret) {
t.Fatalf("scrubbed message still contains %q: %q", longSecret, got)
}
// "123" is the longer secret's remainder past the shared "pass" prefix; it must
// not survive as a partial leak.
if strings.Contains(got, "123") {
t.Fatalf("scrubbed message leaked the longer secret's remainder: %q", got)
}
}
// TestRedactSecretsEmptyOrNilLeavesErrorUnchanged confirms the no-secret paths keep
// the original typed error verbatim (no wrapping, no scrubbed clone).
func TestRedactSecretsEmptyOrNilLeavesErrorUnchanged(t *testing.T) {
base := &MxAccessError{Reply: &pb.MxCommandReply{DiagnosticMessage: "boom"}}
if got := redactSecrets(base); got != error(base) {
t.Fatalf("redactSecrets with no secrets = %v, want the original error unchanged", got)
}
if got := redactSecrets(base, ""); got != error(base) {
t.Fatalf("redactSecrets with only an empty secret = %v, want the original error unchanged", got)
}
if got := redactSecrets(nil, "secret"); got != nil {
t.Fatalf("redactSecrets(nil, ...) = %v, want nil", got)
}
}
// TestRedactSecretsRebuildsTypedCommandError confirms a *CommandError (non-MXAccess
// path) is rebuilt with a scrubbed Status and Reply, and errors.As still reaches it.
func TestRedactSecretsRebuildsTypedCommandError(t *testing.T) {
const secret = "topSecretValue"
base := &CommandError{
Op: "write secured",
Status: &pb.ProtocolStatus{Message: "rejected topSecretValue"},
Reply: &pb.MxCommandReply{DiagnosticMessage: "echoed topSecretValue"},
}
redacted := redactSecrets(base, secret)
var cmdErr *CommandError
if !errors.As(redacted, &cmdErr) {
t.Fatalf("redactSecrets result %T does not unwrap to *CommandError", redacted)
}
if strings.Contains(cmdErr.Status.GetMessage(), secret) {
t.Fatalf("CommandError.Status.Message leaked the secret: %q", cmdErr.Status.GetMessage())
}
if strings.Contains(cmdErr.Reply.GetDiagnosticMessage(), secret) {
t.Fatalf("CommandError.Reply.DiagnosticMessage leaked the secret: %q", cmdErr.Reply.GetDiagnosticMessage())
}
if strings.Contains(redacted.Error(), secret) {
t.Fatalf("rendered error leaked the secret: %q", redacted.Error())
}
}
@@ -48,6 +48,89 @@ func TestGeneratedGoldenFixturesParse(t *testing.T) {
}
}
// TestCommandReplyValidationFixtures locks the shared reply-validation rules to
// the behavior fixtures: a status entry fails iff its category is not OK (the
// raw success member is diagnostics only), and an HRESULT fails iff it is
// present and negative (S_FALSE and other positive COM success codes pass).
func TestCommandReplyValidationFixtures(t *testing.T) {
tests := []struct {
fixture string
wantFailure bool
}{
{fixture: "register.ok.reply.json", wantFailure: false},
{fixture: "write.mxaccess-failure.reply.json", wantFailure: true},
{fixture: "write.status-category-error-success-set.reply.json", wantFailure: true},
{fixture: "write.status-category-ok-success-zero.reply.json", wantFailure: false},
{fixture: "write.hresult-s-false.reply.json", wantFailure: false},
{fixture: "write.hresult-e-fail.reply.json", wantFailure: true},
}
for _, tt := range tests {
t.Run(tt.fixture, func(t *testing.T) {
data, err := os.ReadFile(filepath.Join(
"..", "..", "proto", "fixtures", "behavior", "command-replies", tt.fixture))
if err != nil {
t.Fatalf("read fixture: %v", err)
}
var reply pb.MxCommandReply
if err := protojson.Unmarshal(data, &reply); err != nil {
t.Fatalf("parse fixture: %v", err)
}
err = EnsureMxAccessSuccess("invoke", &reply)
if got := err != nil; got != tt.wantFailure {
t.Fatalf("EnsureMxAccessSuccess() failed = %v (err %v), want %v", got, err, tt.wantFailure)
}
})
}
}
// TestStatusSucceededBranchesOnCategory pins the per-entry rule directly,
// including the two edges the fixtures cannot express: a nil entry is success
// and a present entry with an unspecified category is a failure.
func TestStatusSucceededBranchesOnCategory(t *testing.T) {
tests := []struct {
name string
status *MxStatusProxy
want bool
}{
{name: "nil entry", status: nil, want: true},
{
name: "ok category with zero success",
status: &pb.MxStatusProxy{
Success: 0,
Category: pb.MxStatusCategory_MX_STATUS_CATEGORY_OK,
},
want: true,
},
{
name: "error category with success set",
status: &pb.MxStatusProxy{
Success: 1,
Category: pb.MxStatusCategory_MX_STATUS_CATEGORY_COMMUNICATION_ERROR,
},
want: false,
},
{
name: "unspecified category with success set",
status: &pb.MxStatusProxy{
Success: 1,
Category: pb.MxStatusCategory_MX_STATUS_CATEGORY_UNSPECIFIED,
},
want: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := StatusSucceeded(tt.status); got != tt.want {
t.Fatalf("StatusSucceeded() = %v, want %v", got, tt.want)
}
})
}
}
func TestOpenSessionFixtureProtocolVersions(t *testing.T) {
data, err := os.ReadFile(filepath.Join("..", "..", "proto", "fixtures", "golden", "open-session-reply.ok.json"))
if err != nil {
+64 -14
View File
@@ -812,7 +812,13 @@ func (s *Session) AuthenticateUser(ctx context.Context, serverHandle int32, veri
if reply.GetAuthenticateUser() != nil {
return reply.GetAuthenticateUser().GetUserId(), nil
}
return reply.GetReturnValue().GetInt32Value(), nil
if x, ok := reply.GetReturnValue().GetKind().(*pb.MxValue_Int32Value); ok {
return x.Int32Value, nil
}
return 0, &MalformedReplyError{
Op: "authenticate user",
Detail: "reply carried neither an AuthenticateUser payload nor an int32 return_value",
}
}
// AuthenticateUserRaw invokes MXAccess AuthenticateUser and returns the raw
@@ -847,7 +853,13 @@ func (s *Session) ArchestrAUserToId(ctx context.Context, serverHandle int32, use
if reply.GetArchestraUserToId() != nil {
return reply.GetArchestraUserToId().GetUserId(), nil
}
return reply.GetReturnValue().GetInt32Value(), nil
if x, ok := reply.GetReturnValue().GetKind().(*pb.MxValue_Int32Value); ok {
return x.Int32Value, nil
}
return 0, &MalformedReplyError{
Op: "archestra user to id",
Detail: "reply carried neither an ArchestrAUserToId payload nor an int32 return_value",
}
}
// ArchestrAUserToIdRaw invokes MXAccess ArchestrAUserToId and returns the raw reply.
@@ -876,7 +888,13 @@ func (s *Session) AddBufferedItem(ctx context.Context, serverHandle int32, itemD
if reply.GetAddBufferedItem() != nil {
return reply.GetAddBufferedItem().GetItemHandle(), nil
}
return reply.GetReturnValue().GetInt32Value(), nil
if x, ok := reply.GetReturnValue().GetKind().(*pb.MxValue_Int32Value); ok {
return x.Int32Value, nil
}
return 0, &MalformedReplyError{
Op: "add buffered item",
Detail: "reply carried neither an AddBufferedItem payload nor an int32 return_value",
}
}
// AddBufferedItemRaw invokes MXAccess AddBufferedItem and returns the raw reply.
@@ -981,10 +999,12 @@ func stringSecrets(values ...*MxValue) []string {
// context cancellation stops Recv, or a terminal error is sent.
//
// The returned channel is buffered. If the consumer falls behind and the buffer
// overflows, the stream is terminated and a final EventResult carrying a
// GatewayError that wraps ErrSlowConsumer is delivered before the channel
// closes. Callers must match it with errors.Is(res.Err, ErrSlowConsumer) to
// distinguish a slow-consumer drop from a graceful server end. Use
// overflows with data, the stream is terminated and a final EventResult carrying
// a GatewayError that wraps ErrSlowConsumer is delivered before the channel
// closes; match it with errors.Is(res.Err, ErrSlowConsumer) to distinguish a
// slow-consumer drop from a graceful server end. A genuine stream error is
// reported as itself even under overflow — it is never relabeled as
// ErrSlowConsumer, so the underlying gRPC status stays inspectable. Use
// SubscribeEvents for a blocking, backpressured stream that never drops.
func (s *Session) Events(ctx context.Context) (<-chan EventResult, error) {
return s.EventsAfter(ctx, 0)
@@ -994,7 +1014,9 @@ func (s *Session) Events(ctx context.Context) (<-chan EventResult, error) {
//
// Like Events, the returned channel is buffered and terminates with a final
// EventResult wrapping ErrSlowConsumer (matchable via errors.Is) if the consumer
// falls behind and the buffer overflows, rather than silently closing.
// falls behind and the buffer overflows with data, rather than silently closing.
// A genuine stream error is reported as itself even under overflow, never
// relabeled as ErrSlowConsumer.
func (s *Session) EventsAfter(ctx context.Context, afterWorkerSequence uint64) (<-chan EventResult, error) {
subscription, err := s.subscribeEventsAfter(ctx, afterWorkerSequence, true)
if err != nil {
@@ -1048,12 +1070,11 @@ func (s *Session) subscribeEventsAfter(ctx context.Context, afterWorkerSequence
if err == io.EOF || status.Code(err) == codes.Canceled || streamCtx.Err() != nil {
return
}
sendEventResult(
streamCtx,
results,
EventResult{Err: &GatewayError{Op: "stream events", Err: err}},
cancelWhenResultBufferFull,
cancel)
// A genuine terminal stream error must be reported as itself, even
// when the data slots are full. Routing it through sendEventResult
// would let the overflow branch substitute ErrSlowConsumer and lose
// the real gRPC status, so send it directly, bypassing that branch.
sendTerminalEventResult(streamCtx, results, EventResult{Err: &GatewayError{Op: "stream events", Err: err}}, cancelWhenResultBufferFull)
return
}
}()
@@ -1072,6 +1093,35 @@ func ensureBulkSize(name string, length int) error {
return nil
}
// sendTerminalEventResult enqueues a terminal EventResult, bypassing
// sendEventResult's overflow branch so a genuine stream error is reported verbatim
// rather than relabeled as ErrSlowConsumer. How it sends depends on the mode:
//
// - cancelWhenBufferFull=true (Events/EventsAfter): ordinary data sends are capped
// at eventBufferSize, leaving eventBufferReservedSlots free, so a non-blocking
// send always lands the terminal result. Because this goroutine is the sole
// producer, at most one terminal send ever races for the reserved slot, so the
// select default only fires when the reserve is already spent — never dropping a
// first terminal error.
// - cancelWhenBufferFull=false (SubscribeEvents/SubscribeEventsAfter, never-drop):
// ordinary data sends are uncapped and blocking, so every slot including the
// reserve can hold data. A non-blocking send would then hit the full buffer and
// silently drop the terminal error, breaking the never-drop contract; instead
// block until the consumer drains a slot (or the stream context is cancelled).
func sendTerminalEventResult(ctx context.Context, results chan<- EventResult, result EventResult, cancelWhenBufferFull bool) {
if cancelWhenBufferFull {
select {
case results <- result:
default:
}
return
}
select {
case results <- result:
case <-ctx.Done():
}
}
func sendEventResult(
ctx context.Context,
results chan<- EventResult,
+12 -1
View File
@@ -1,6 +1,17 @@
package mxgateway
import (
pb "gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go/internal/generated"
)
// StatusSucceeded reports whether an MXSTATUS_PROXY entry represents success.
//
// The wire contract makes Category authoritative: an entry succeeds only when
// its category is MX_STATUS_CATEGORY_OK. The Success member mirrors the raw
// 16-bit COM value verbatim for diagnostics and is not a boolean, so it takes
// no part in the verdict. A nil entry is success (nothing was reported); a
// present entry with an unspecified category is a failure, because the worker
// always maps a category and an unmapped one is not proven OK.
func StatusSucceeded(status *MxStatusProxy) bool {
return status == nil || status.GetSuccess() != 0
return status == nil || status.GetCategory() == pb.MxStatusCategory_MX_STATUS_CATEGORY_OK
}
+1 -1
View File
@@ -3,7 +3,7 @@ package mxgateway
const (
// ClientVersion is the released semantic version of this Go client module.
// Keep it in sync with the module tag applied by scripts/tag-go-module.ps1.
ClientVersion = "0.1.2"
ClientVersion = "0.2.0"
// GatewayProtocolVersion matches GatewayContractInfo.GatewayProtocolVersion
// in the shared .NET contracts.
+14 -4
View File
@@ -139,7 +139,12 @@ commands, so you do not need to build raw `MxCommand` messages:
All of them run the same MXAccess reply validation as the bulk helpers (protocol
status plus HRESULT/`MxStatusProxy` check) via the shared `invoke` path, so an
MXAccess COM-side failure surfaces as `MxAccessException`.
MXAccess COM-side failure surfaces as `MxAccessException`. That validation
follows COM semantics: only a **negative** HRESULT is a failure, so positive
success codes such as `S_FALSE` (1) pass. `MxStatuses.succeeded` judges each
entry by its category — an entry fails when its category is not
`MX_STATUS_CATEGORY_OK`, and the raw `success` member is a diagnostic that never
decides the verdict. A `null` entry is success.
**Secret redaction.** Credentials passed to `authenticateUser` (and the
credential-sensitive values passed to `writeSecured`/`writeSecured2`) travel
@@ -167,8 +172,13 @@ session.write(serverHandle, itemHandle, value, userId);
native failure is surfaced, not papered over.
The CLI exposes `advise-supervisory`, `write-secured`, and `authenticate-user`
(credential via `--password` or `--password-env`, never echoed), and `write` /
`write2` take `--user-id`.
(credential via `--password` or the variable named by `--password-env`, default
`MXGATEWAY_VERIFY_PASSWORD`, never echoed), and `write` / `write2` take
`--user-id`. The credential is required: a missing or empty resolved value is a
picocli usage error naming the option and the variable, so the CLI fails before
connecting instead of authenticating with an empty password.
`MXGATEWAY_VERIFY_PASSWORD` is the canonical variable across all five client
CLIs — see [Cross-Language Smoke Matrix](../../docs/CrossLanguageSmokeMatrix.md).
### Array writes replace the whole array
@@ -455,7 +465,7 @@ repositories {
}
dependencies {
implementation 'com.zb.mom.ww.mxgateway:zb-mom-ww-mxgateway-client:0.1.2'
implementation 'com.zb.mom.ww.mxgateway:zb-mom-ww-mxgateway-client:0.2.1'
}
````
+8 -1
View File
@@ -13,7 +13,14 @@ ext {
subprojects {
group = 'com.zb.mom.ww.mxgateway'
version = '0.2.0'
// 0.2.0 was already published to the Gitea Maven feed on 2026-06-26,
// before the CLI-37/38/40/41 conformance fixes changed the client's
// observable behavior (status.category-based validation, hresult < 0,
// exact-secret redaction, typed malformed-reply errors). Bump to 0.2.1
// so the published coordinate matches the conformant behavior the other
// four clients ship at 0.2.0 for the first time. See CLI-39 and the
// "Versioning" section of docs/ClientPackaging.md.
version = '0.2.1'
pluginManager.withPlugin('java') {
java {
@@ -69110,24 +69110,59 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
com.google.protobuf.MessageOrBuilder {
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
java.util.List<mxaccess_gateway.v1.MxaccessGateway.MxEvent>
getEventsList();
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
mxaccess_gateway.v1.MxaccessGateway.MxEvent getEvents(int index);
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
int getEventsCount();
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
java.util.List<? extends mxaccess_gateway.v1.MxaccessGateway.MxEventOrBuilder>
getEventsOrBuilderList();
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
mxaccess_gateway.v1.MxaccessGateway.MxEventOrBuilder getEventsOrBuilder(
@@ -69175,6 +69210,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
@SuppressWarnings("serial")
private java.util.List<mxaccess_gateway.v1.MxaccessGateway.MxEvent> events_;
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
@java.lang.Override
@@ -69182,6 +69224,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
return events_;
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
@java.lang.Override
@@ -69190,6 +69239,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
return events_;
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
@java.lang.Override
@@ -69197,6 +69253,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
return events_.size();
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
@java.lang.Override
@@ -69204,6 +69267,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
return events_.get(index);
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
@java.lang.Override
@@ -69567,6 +69637,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
mxaccess_gateway.v1.MxaccessGateway.MxEvent, mxaccess_gateway.v1.MxaccessGateway.MxEvent.Builder, mxaccess_gateway.v1.MxaccessGateway.MxEventOrBuilder> eventsBuilder_;
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public java.util.List<mxaccess_gateway.v1.MxaccessGateway.MxEvent> getEventsList() {
@@ -69577,6 +69654,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
}
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public int getEventsCount() {
@@ -69587,6 +69671,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
}
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public mxaccess_gateway.v1.MxaccessGateway.MxEvent getEvents(int index) {
@@ -69597,6 +69688,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
}
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public Builder setEvents(
@@ -69614,6 +69712,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
return this;
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public Builder setEvents(
@@ -69628,6 +69733,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
return this;
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public Builder addEvents(mxaccess_gateway.v1.MxaccessGateway.MxEvent value) {
@@ -69644,6 +69756,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
return this;
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public Builder addEvents(
@@ -69661,6 +69780,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
return this;
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public Builder addEvents(
@@ -69675,6 +69801,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
return this;
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public Builder addEvents(
@@ -69689,6 +69822,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
return this;
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public Builder addAllEvents(
@@ -69704,6 +69844,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
return this;
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public Builder clearEvents() {
@@ -69717,6 +69864,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
return this;
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public Builder removeEvents(int index) {
@@ -69730,6 +69884,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
return this;
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public mxaccess_gateway.v1.MxaccessGateway.MxEvent.Builder getEventsBuilder(
@@ -69737,6 +69898,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
return internalGetEventsFieldBuilder().getBuilder(index);
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public mxaccess_gateway.v1.MxaccessGateway.MxEventOrBuilder getEventsOrBuilder(
@@ -69747,6 +69915,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
}
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public java.util.List<? extends mxaccess_gateway.v1.MxaccessGateway.MxEventOrBuilder>
@@ -69758,6 +69933,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
}
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public mxaccess_gateway.v1.MxaccessGateway.MxEvent.Builder addEventsBuilder() {
@@ -69765,6 +69947,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
mxaccess_gateway.v1.MxaccessGateway.MxEvent.getDefaultInstance());
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public mxaccess_gateway.v1.MxaccessGateway.MxEvent.Builder addEventsBuilder(
@@ -69773,6 +69962,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
index, mxaccess_gateway.v1.MxaccessGateway.MxEvent.getDefaultInstance());
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public java.util.List<mxaccess_gateway.v1.MxaccessGateway.MxEvent.Builder>
@@ -75305,6 +75501,11 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
* after_worker_sequence = oldest_available_sequence - 1 in the next
* StreamEventsRequest, which will cause the server to replay starting at
* oldest_available_sequence (the first retained event).
* When nothing is retained (the replay ring is empty), this is the next sequence
* that can be delivered `highest observed + 1` and the `oldest - 1` resume
* formula remains valid: it resolves to the highest sequence already seen, so the
* follow-up resume replays nothing, reports no gap, and every newer live event
* passes. The interval evicted is unchanged.
* </pre>
*
* <code>uint64 oldest_available_sequence = 2;</code>
@@ -75386,6 +75587,11 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
* after_worker_sequence = oldest_available_sequence - 1 in the next
* StreamEventsRequest, which will cause the server to replay starting at
* oldest_available_sequence (the first retained event).
* When nothing is retained (the replay ring is empty), this is the next sequence
* that can be delivered `highest observed + 1` and the `oldest - 1` resume
* formula remains valid: it resolves to the highest sequence already seen, so the
* follow-up resume replays nothing, reports no gap, and every newer live event
* passes. The interval evicted is unchanged.
* </pre>
*
* <code>uint64 oldest_available_sequence = 2;</code>
@@ -75781,6 +75987,11 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
* after_worker_sequence = oldest_available_sequence - 1 in the next
* StreamEventsRequest, which will cause the server to replay starting at
* oldest_available_sequence (the first retained event).
* When nothing is retained (the replay ring is empty), this is the next sequence
* that can be delivered `highest observed + 1` and the `oldest - 1` resume
* formula remains valid: it resolves to the highest sequence already seen, so the
* follow-up resume replays nothing, reports no gap, and every newer live event
* passes. The interval evicted is unchanged.
* </pre>
*
* <code>uint64 oldest_available_sequence = 2;</code>
@@ -75800,6 +76011,11 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
* after_worker_sequence = oldest_available_sequence - 1 in the next
* StreamEventsRequest, which will cause the server to replay starting at
* oldest_available_sequence (the first retained event).
* When nothing is retained (the replay ring is empty), this is the next sequence
* that can be delivered `highest observed + 1` and the `oldest - 1` resume
* formula remains valid: it resolves to the highest sequence already seen, so the
* follow-up resume replays nothing, reports no gap, and every newer live event
* passes. The interval evicted is unchanged.
* </pre>
*
* <code>uint64 oldest_available_sequence = 2;</code>
@@ -75823,6 +76039,11 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
* after_worker_sequence = oldest_available_sequence - 1 in the next
* StreamEventsRequest, which will cause the server to replay starting at
* oldest_available_sequence (the first retained event).
* When nothing is retained (the replay ring is empty), this is the next sequence
* that can be delivered `highest observed + 1` and the `oldest - 1` resume
* formula remains valid: it resolves to the highest sequence already seen, so the
* follow-up resume replays nothing, reports no gap, and every newer live event
* passes. The interval evicted is unchanged.
* </pre>
*
* <code>uint64 oldest_available_sequence = 2;</code>
@@ -3797,6 +3797,9 @@ public final class MxaccessWorker extends com.google.protobuf.GeneratedFile {
* instead of a hard-coded default; 0 (an older gateway that never set the field) means
* "use the worker's built-in default". Sits above the public gRPC cap by an
* envelope-overhead margin so an accepted gRPC payload always fits one worker frame.
* Every worker-&gt;gateway frame events, heartbeats, faults, and control replies
* including DrainEvents must serialize within this limit; reply builders truncate
* to fit rather than emit an oversized frame.
* </pre>
*
* <code>uint32 max_frame_bytes = 4;</code>
@@ -3941,6 +3944,9 @@ public final class MxaccessWorker extends com.google.protobuf.GeneratedFile {
* instead of a hard-coded default; 0 (an older gateway that never set the field) means
* "use the worker's built-in default". Sits above the public gRPC cap by an
* envelope-overhead margin so an accepted gRPC payload always fits one worker frame.
* Every worker-&gt;gateway frame events, heartbeats, faults, and control replies
* including DrainEvents must serialize within this limit; reply builders truncate
* to fit rather than emit an oversized frame.
* </pre>
*
* <code>uint32 max_frame_bytes = 4;</code>
@@ -4499,6 +4505,9 @@ public final class MxaccessWorker extends com.google.protobuf.GeneratedFile {
* instead of a hard-coded default; 0 (an older gateway that never set the field) means
* "use the worker's built-in default". Sits above the public gRPC cap by an
* envelope-overhead margin so an accepted gRPC payload always fits one worker frame.
* Every worker-&gt;gateway frame events, heartbeats, faults, and control replies
* including DrainEvents must serialize within this limit; reply builders truncate
* to fit rather than emit an oversized frame.
* </pre>
*
* <code>uint32 max_frame_bytes = 4;</code>
@@ -4515,6 +4524,9 @@ public final class MxaccessWorker extends com.google.protobuf.GeneratedFile {
* instead of a hard-coded default; 0 (an older gateway that never set the field) means
* "use the worker's built-in default". Sits above the public gRPC cap by an
* envelope-overhead margin so an accepted gRPC payload always fits one worker frame.
* Every worker-&gt;gateway frame events, heartbeats, faults, and control replies
* including DrainEvents must serialize within this limit; reply builders truncate
* to fit rather than emit an oversized frame.
* </pre>
*
* <code>uint32 max_frame_bytes = 4;</code>
@@ -4535,6 +4547,9 @@ public final class MxaccessWorker extends com.google.protobuf.GeneratedFile {
* instead of a hard-coded default; 0 (an older gateway that never set the field) means
* "use the worker's built-in default". Sits above the public gRPC cap by an
* envelope-overhead margin so an accepted gRPC payload always fits one worker frame.
* Every worker-&gt;gateway frame events, heartbeats, faults, and control replies
* including DrainEvents must serialize within this limit; reply builders truncate
* to fit rather than emit an oversized frame.
* </pre>
*
* <code>uint32 max_frame_bytes = 4;</code>
@@ -70,6 +70,7 @@ import picocli.CommandLine.Command;
import picocli.CommandLine.Mixin;
import picocli.CommandLine.Model.CommandSpec;
import picocli.CommandLine.Option;
import picocli.CommandLine.ParameterException;
import picocli.CommandLine.Spec;
/**
@@ -182,6 +183,13 @@ public final class MxGatewayCli implements Callable<Integer> {
/** Sentinel written to stdout after every command result in batch mode. */
static final String BATCH_EOR = "__MXGW_BATCH_EOR__";
/**
* Canonical CLI credential environment variable, shared by every official
* client CLI (CLI-45) so one exported variable drives the same operator
* workflow in all five languages.
*/
static final String DEFAULT_VERIFY_PASSWORD_ENV = "MXGATEWAY_VERIFY_PASSWORD";
/** Sentinel queued by {@code stream-alarms} to mark a clean end of the alarm feed. */
private static final Object ALARM_FEED_END = new Object();
@@ -1139,7 +1147,7 @@ public final class MxGatewayCli implements Callable<Integer> {
@Option(
names = "--password-env",
defaultValue = "MXGATEWAY_VERIFY_PASSWORD",
defaultValue = DEFAULT_VERIFY_PASSWORD_ENV,
description = "Environment variable holding the password when --password is omitted.")
String passwordEnv;
@@ -1151,11 +1159,20 @@ public final class MxGatewayCli implements Callable<Integer> {
public Integer call() {
// Resolve the credential from the flag or environment. It flows only
// into the request; it is never written to output, logs, or errors.
String environmentName =
passwordEnv == null || passwordEnv.isBlank() ? DEFAULT_VERIFY_PASSWORD_ENV : passwordEnv;
String resolvedPassword = password == null || password.isBlank()
? System.getenv(passwordEnv)
? System.getenv(environmentName)
: password;
if (resolvedPassword == null) {
resolvedPassword = "";
if (resolvedPassword == null || resolvedPassword.isBlank()) {
// Fail fast instead of dialing: a misconfigured environment must not
// become a real MXAccess authentication attempt with an empty
// credential (CLI-45). The message names the option and the variable
// only never the value.
throw new ParameterException(
common.spec.commandLine(),
"a password is required via --password or the " + environmentName
+ " environment variable");
}
try (MxGatewayCliClient client = clientFactory.connect(common.resolved())) {
int userId = client.session(sessionId)
@@ -2,7 +2,10 @@ package com.zb.mom.ww.mxgateway.cli;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotEquals;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.Assumptions.assumeTrue;
import com.zb.mom.ww.mxgateway.client.MxGatewayAlarmFeedSubscription;
import com.zb.mom.ww.mxgateway.client.MxGatewayClientOptions;
@@ -56,7 +59,7 @@ final class MxGatewayCliTests {
assertEquals(0, run.exitCode());
assertEquals("", run.errors());
assertTrue(run.output().contains("mxgateway-java 0.2.0"));
assertTrue(run.output().contains("mxgateway-java 0.2.1"));
assertTrue(run.output().contains("gatewayProtocolVersion=3"));
assertTrue(run.output().contains("workerProtocolVersion=1"));
}
@@ -86,7 +89,7 @@ final class MxGatewayCliTests {
CliRun run = execute(new FakeClientFactory(), "version", "--json");
assertEquals(0, run.exitCode());
assertTrue(run.output().contains("\"clientVersion\":\"0.2.0\""));
assertTrue(run.output().contains("\"clientVersion\":\"0.2.1\""));
assertTrue(run.output().contains("\"gatewayProtocolVersion\":3"));
}
@@ -211,6 +214,73 @@ final class MxGatewayCliTests {
assertFalse(run.errors().contains("super-secret-pw"), "password must never be echoed to stderr");
}
/**
* CLI-45: an unresolved credential must abort with a picocli usage error
* before the CLI dials, instead of authenticating with an empty password.
* The message names the option and the variable, never a value.
*/
@Test
void authenticateUserRejectsMissingCredentialWithUsageError() {
FakeClientFactory factory = new FakeClientFactory();
CliRun run = execute(
factory,
"authenticate-user",
"--session-id", "session-cli",
"--server-handle", "3",
"--verify-user", "operator",
"--password-env", "MXGW_CLI45_ABSENT_PASSWORD_VAR",
"--json");
assertNotEquals(0, run.exitCode(), "a missing credential must fail");
assertTrue(run.errors().contains("--password"), run.errors());
assertTrue(run.errors().contains("MXGW_CLI45_ABSENT_PASSWORD_VAR"), run.errors());
assertNull(factory.client, "the CLI must not connect without a credential");
}
/**
* CLI-45: a blank {@code --password} is treated as missing the CLI never
* sends a fabricated empty credential to the wire.
*/
@Test
void authenticateUserRejectsBlankPasswordValue() {
FakeClientFactory factory = new FakeClientFactory();
CliRun run = execute(
factory,
"authenticate-user",
"--session-id", "session-cli",
"--server-handle", "3",
"--verify-user", "operator",
"--password", "",
"--password-env", "MXGW_CLI45_ABSENT_PASSWORD_VAR",
"--json");
assertNotEquals(0, run.exitCode(), "a blank credential must fail");
assertNull(factory.client, "the CLI must not connect without a credential");
}
/**
* CLI-45: {@code --password-env} defaults to the canonical
* {@code MXGATEWAY_VERIFY_PASSWORD}, so the usage error names it when no
* explicit variable is given. Skipped if the canonical variable happens to be
* exported in the running environment (which would satisfy the credential).
*/
@Test
void authenticateUserDefaultsToCanonicalPasswordEnvName() {
assumeTrue(System.getenv(MxGatewayCli.DEFAULT_VERIFY_PASSWORD_ENV) == null);
FakeClientFactory factory = new FakeClientFactory();
CliRun run = execute(
factory,
"authenticate-user",
"--session-id", "session-cli",
"--server-handle", "3",
"--verify-user", "operator",
"--json");
assertNotEquals(0, run.exitCode());
assertTrue(run.errors().contains("MXGATEWAY_VERIFY_PASSWORD"), run.errors());
}
// ---- ping subcommand (D4) ----
@Test
@@ -63,10 +63,11 @@ protobuf {
// or a plugin/protobuf version bump, silently drifts the committed output. checkGeneratedClean
// fails when the regenerated tree differs from what is committed.
//
// Caveat (repo memory project_java_generated_churn): the protobuf gradle plugin also rewrites
// MxaccessGateway.java with a spurious protobuf-runtime-version delta on every build even when no
// .proto changed. CI reverts that one file (git checkout) before invoking this task; locally, do the
// same when you did not touch a .proto. See docs/GatewayTesting.md "Continuous Integration".
// The grpc/protobuf toolchain is pinned (build.gradle: grpcVersion / protobufVersion), so a
// regeneration is byte-identical to the committed single-file aggregates modulo real .proto
// changes no spurious protobuf-runtime-version churn (IPC-24 verified this and deleted the old
// unconditional CI churn-revert step, which masked message-level drift). Regenerate and commit
// after any .proto change. See docs/GatewayTesting.md "Continuous Integration".
tasks.register('checkGeneratedClean') {
group = 'verification'
description = 'Fails if the committed generated Java tree differs from a fresh regeneration.'
@@ -83,9 +84,9 @@ tasks.register('checkGeneratedClean') {
def dirty = stdout.toString().trim()
if (!dirty.isEmpty()) {
throw new GradleException(
"Generated Java is stale or churned:\n${dirty}\n" +
"Regenerate and commit after a .proto change, or 'git checkout' the spurious " +
"MxaccessGateway.java protobuf-version churn when no .proto changed.")
"Generated Java is stale:\n${dirty}\n" +
"Regenerate and commit the Java client after a .proto change " +
"(gradle :zb-mom-ww-mxgateway-client:generateProto).")
}
}
}
@@ -29,4 +29,18 @@ public final class MxAccessException extends MxGatewayCommandException {
public MxAccessException(String operation, MxCommandReply reply) {
super(operation, reply == null ? null : reply.getProtocolStatus(), reply);
}
/**
* Creates a new MXAccess exception with an already-built, verbatim message.
* Used to re-surface an MXAccess failure with a redacted message while
* preserving the original protocol status and reply.
*
* @param message the exact message to surface (already formatted/redacted)
* @param protocolStatus protocol status reported by the gateway
* @param reply raw command reply containing the MXAccess failure detail
* @param cause underlying error, or {@code null}
*/
public MxAccessException(String message, ProtocolStatus protocolStatus, MxCommandReply reply, Throwable cause) {
super(message, protocolStatus, reply, cause);
}
}
@@ -9,7 +9,7 @@ package com.zb.mom.ww.mxgateway.client;
public final class MxGatewayClientVersion {
private static final int GATEWAY_PROTOCOL_VERSION = 3;
private static final int WORKER_PROTOCOL_VERSION = 1;
private static final String CLIENT_VERSION = "0.2.0";
private static final String CLIENT_VERSION = "0.2.1";
private MxGatewayClientVersion() {
}
@@ -25,6 +25,23 @@ public class MxGatewayCommandException extends MxGatewayException {
this.reply = reply;
}
/**
* Creates a new command exception with an already-built, verbatim message.
* Used to re-surface a failure with a redacted message while preserving the
* original protocol status and reply.
*
* @param message the exact message to surface (already formatted/redacted)
* @param protocolStatus protocol status returned by the gateway
* @param reply raw command reply, or {@code null} when none was produced
* @param cause underlying error, or {@code null}
*/
protected MxGatewayCommandException(
String message, ProtocolStatus protocolStatus, MxCommandReply reply, Throwable cause) {
super(message, cause);
this.protocolStatus = protocolStatus;
this.reply = reply;
}
/**
* Returns the gateway protocol status that triggered this exception.
*
@@ -47,7 +47,9 @@ final class MxGatewayErrors {
if (reply == null) {
return;
}
if (reply.hasHresult() && reply.getHresult() != 0) {
// COM semantics: only a negative HRESULT is a failure. Positive success
// codes such as S_FALSE (1) pass.
if (reply.hasHresult() && reply.getHresult() < 0) {
throw new MxAccessException(operation, reply);
}
for (var status : reply.getStatusesList()) {
@@ -0,0 +1,32 @@
package com.zb.mom.ww.mxgateway.client;
/**
* Thrown when the gateway returns a protocol-OK command reply that carries
* neither the expected typed payload nor a usable {@code return_value}.
*
* <p>A successful reply for a value-returning command (for example
* {@code AuthenticateUser}, {@code ArchestrAUserToId}, or {@code AddBufferedItem})
* must supply either the command's typed payload or an int32 {@code return_value}.
* A reply that satisfies neither is malformed, and the client surfaces this
* distinct failure rather than silently returning a default {@code 0}.
*/
public final class MxGatewayMalformedReplyException extends MxGatewayException {
/**
* Creates a new malformed-reply exception with the supplied message.
*
* @param message human-readable description of the malformed reply
*/
public MxGatewayMalformedReplyException(String message) {
super(message);
}
/**
* Creates a new malformed-reply exception with the supplied message and cause.
*
* @param message human-readable description of the malformed reply
* @param cause underlying error that triggered the failure
*/
public MxGatewayMalformedReplyException(String message, Throwable cause) {
super(message, cause);
}
}
@@ -54,4 +54,34 @@ public final class MxGatewaySecrets {
}
return String.join(" ", parts);
}
/**
* Replaces every occurrence of each supplied secret with the redaction
* marker {@code "<redacted>"}. Unlike {@link #redactCredentials(String)},
* which scrubs by pattern, this performs an exact-substring scrub of the
* caller-known secrets used to strip a credential the gateway echoed back
* into a free-form failure message.
*
* @param message the message to scrub, may be {@code null}
* @param secrets the exact secret substrings to remove; {@code null}, empty,
* and blank (whitespace-only) entries and a {@code null} array are ignored
* @return {@code message} unchanged when it is {@code null} or no non-blank
* secret is supplied, otherwise the message with every secret occurrence
* replaced by {@code "<redacted>"}
*/
public static String redactExact(String message, String... secrets) {
if (message == null || secrets == null) {
return message;
}
String result = message;
for (String secret : secrets) {
if (secret == null || secret.isBlank()) {
// A blank "secret" would over-redact real whitespace; skip it.
continue;
}
result = result.replace(secret, "<redacted>");
}
return result;
}
}
@@ -31,6 +31,7 @@ import mxaccess_gateway.v1.MxaccessGateway.MxSparseElement;
import mxaccess_gateway.v1.MxaccessGateway.MxStatusProxy;
import mxaccess_gateway.v1.MxaccessGateway.MxValue;
import mxaccess_gateway.v1.MxaccessGateway.OpenSessionReply;
import mxaccess_gateway.v1.MxaccessGateway.ProtocolStatus;
import mxaccess_gateway.v1.MxaccessGateway.ReadBulkCommand;
import mxaccess_gateway.v1.MxaccessGateway.RegisterCommand;
import mxaccess_gateway.v1.MxaccessGateway.RemoveItemBulkCommand;
@@ -782,15 +783,17 @@ public final class MxGatewaySession implements AutoCloseable {
*/
public MxCommandReply writeSecuredRaw(
int serverHandle, int itemHandle, int currentUserId, int verifierUserId, MxValue value) {
return invokeCommand(MxCommand.newBuilder()
.setKind(MxCommandKind.MX_COMMAND_KIND_WRITE_SECURED)
.setWriteSecured(WriteSecuredCommand.newBuilder()
.setServerHandle(serverHandle)
.setItemHandle(itemHandle)
.setCurrentUserId(currentUserId)
.setVerifierUserId(verifierUserId)
.setValue(value))
.build());
return invokeCommandRedacted(
MxCommand.newBuilder()
.setKind(MxCommandKind.MX_COMMAND_KIND_WRITE_SECURED)
.setWriteSecured(WriteSecuredCommand.newBuilder()
.setServerHandle(serverHandle)
.setItemHandle(itemHandle)
.setCurrentUserId(currentUserId)
.setVerifierUserId(verifierUserId)
.setValue(value))
.build(),
secretStringOf(value));
}
/**
@@ -837,16 +840,18 @@ public final class MxGatewaySession implements AutoCloseable {
int verifierUserId,
MxValue value,
MxValue timestampValue) {
return invokeCommand(MxCommand.newBuilder()
.setKind(MxCommandKind.MX_COMMAND_KIND_WRITE_SECURED2)
.setWriteSecured2(WriteSecured2Command.newBuilder()
.setServerHandle(serverHandle)
.setItemHandle(itemHandle)
.setCurrentUserId(currentUserId)
.setVerifierUserId(verifierUserId)
.setValue(value)
.setTimestampValue(timestampValue))
.build());
return invokeCommandRedacted(
MxCommand.newBuilder()
.setKind(MxCommandKind.MX_COMMAND_KIND_WRITE_SECURED2)
.setWriteSecured2(WriteSecured2Command.newBuilder()
.setServerHandle(serverHandle)
.setItemHandle(itemHandle)
.setCurrentUserId(currentUserId)
.setVerifierUserId(verifierUserId)
.setValue(value)
.setTimestampValue(timestampValue))
.build(),
secretStringOf(value));
}
/**
@@ -866,17 +871,24 @@ public final class MxGatewaySession implements AutoCloseable {
* @throws MxAccessException when MXAccess rejects the credential
*/
public int authenticateUser(int serverHandle, String verifyUser, String verifyUserPassword) {
MxCommandReply reply = invokeCommand(MxCommand.newBuilder()
.setKind(MxCommandKind.MX_COMMAND_KIND_AUTHENTICATE_USER)
.setAuthenticateUser(AuthenticateUserCommand.newBuilder()
.setServerHandle(serverHandle)
.setVerifyUser(verifyUser)
.setVerifyUserPassword(verifyUserPassword))
.build());
MxCommandReply reply = invokeCommandRedacted(
MxCommand.newBuilder()
.setKind(MxCommandKind.MX_COMMAND_KIND_AUTHENTICATE_USER)
.setAuthenticateUser(AuthenticateUserCommand.newBuilder()
.setServerHandle(serverHandle)
.setVerifyUser(verifyUser)
.setVerifyUserPassword(verifyUserPassword))
.build(),
verifyUserPassword);
if (reply.hasAuthenticateUser()) {
return reply.getAuthenticateUser().getUserId();
}
return reply.getReturnValue().getInt32Value();
if (reply.hasReturnValue() && reply.getReturnValue().getKindCase() == MxValue.KindCase.INT32_VALUE) {
return reply.getReturnValue().getInt32Value();
}
throw new MxGatewayMalformedReplyException(
"AuthenticateUser returned a malformed reply: OK reply carried neither "
+ "the typed payload nor an int32 return_value");
}
/**
@@ -899,7 +911,12 @@ public final class MxGatewaySession implements AutoCloseable {
if (reply.hasArchestraUserToId()) {
return reply.getArchestraUserToId().getUserId();
}
return reply.getReturnValue().getInt32Value();
if (reply.hasReturnValue() && reply.getReturnValue().getKindCase() == MxValue.KindCase.INT32_VALUE) {
return reply.getReturnValue().getInt32Value();
}
throw new MxGatewayMalformedReplyException(
"ArchestrAUserToId returned a malformed reply: OK reply carried neither "
+ "the typed payload nor an int32 return_value");
}
/**
@@ -925,7 +942,12 @@ public final class MxGatewaySession implements AutoCloseable {
if (reply.hasAddBufferedItem()) {
return reply.getAddBufferedItem().getItemHandle();
}
return reply.getReturnValue().getInt32Value();
if (reply.hasReturnValue() && reply.getReturnValue().getKindCase() == MxValue.KindCase.INT32_VALUE) {
return reply.getReturnValue().getInt32Value();
}
throw new MxGatewayMalformedReplyException(
"AddBufferedItem returned a malformed reply: OK reply carried neither "
+ "the typed payload nor an int32 return_value");
}
/**
@@ -1027,6 +1049,162 @@ public final class MxGatewaySession implements AutoCloseable {
.build());
}
/**
* Invokes a credential-bearing command, scrubbing any exact secret the
* gateway may have echoed back into a surfaced failure message. The secret
* lives only in the request, but a non-parity gateway or provider can copy
* it into a diagnostic; this guarantees it never survives in the exception
* text a caller might log.
*
* <p>On failure both the exception's message <em>and</em> its structured
* context (the {@link ProtocolStatus} and {@link MxCommandReply} a caller can
* inspect and log) are scrubbed with {@link MxGatewaySecrets#redactExact}: the
* gateway echoes the credential into {@code protocolStatus.message},
* {@code reply.diagnosticMessage}, and each {@code statuses[i].diagnosticText}.
* If nothing carried the secret (the common case) the original exception is
* rethrown untouched. Otherwise it is re-thrown as the same concrete type
* carrying the redacted message and scrubbed context; the secret-bearing
* original is not chained as a cause, so it cannot leak through a printed
* stack trace.
*/
private MxCommandReply invokeCommandRedacted(MxCommand command, String... secrets) {
try {
return invokeCommand(command);
} catch (MxGatewayException ex) {
String original = ex.getMessage();
String redactedMessage = MxGatewaySecrets.redactExact(original, secrets);
boolean messageChanged = redactedMessage != null && !redactedMessage.equals(original);
ProtocolStatus status = protocolStatusOf(ex);
ProtocolStatus scrubbedStatus = scrubProtocolStatus(status, secrets);
boolean statusChanged = status != null && !status.equals(scrubbedStatus);
MxCommandReply reply = replyOf(ex);
MxCommandReply scrubbedReply = scrubReply(reply, secrets);
boolean replyChanged = reply != null && !reply.equals(scrubbedReply);
if (!messageChanged && !statusChanged && !replyChanged) {
throw ex;
}
String message = messageChanged ? redactedMessage : original;
throw rebuildRedacted(ex, message, scrubbedStatus, scrubbedReply);
}
}
/**
* Extracts the {@link ProtocolStatus} an exception carries, if any, so it can
* be scrubbed and re-attached to the rebuilt exception.
*/
private static ProtocolStatus protocolStatusOf(MxGatewayException ex) {
if (ex instanceof MxGatewayCommandException command) {
return command.protocolStatus();
}
if (ex instanceof MxGatewaySessionException session) {
return session.protocolStatus();
}
if (ex instanceof MxGatewayWorkerException worker) {
return worker.protocolStatus();
}
return null;
}
/**
* Extracts the raw {@link MxCommandReply} an exception carries, if any.
*/
private static MxCommandReply replyOf(MxGatewayException ex) {
if (ex instanceof MxGatewayCommandException command) {
return command.reply();
}
return null;
}
/**
* Rebuilds a gateway exception of the same concrete type with a redacted
* message and already-scrubbed context, mirroring the .NET client's
* type-switch. Only a truly-unknown subtype collapses to the base
* {@link MxGatewayException}. The original (secret-bearing) exception is
* deliberately not chained as a cause.
*/
private static MxGatewayException rebuildRedacted(
MxGatewayException ex, String message, ProtocolStatus status, MxCommandReply reply) {
if (ex instanceof MxAccessException) {
return new MxAccessException(message, status, reply, null);
}
if (ex instanceof MxGatewayCommandException) {
return new MxGatewayCommandException(message, status, reply, null);
}
if (ex instanceof MxGatewaySessionException) {
return new MxGatewaySessionException(message, status, null);
}
if (ex instanceof MxGatewayWorkerException) {
return new MxGatewayWorkerException(message, status, null);
}
if (ex instanceof MxGatewayMalformedReplyException) {
return new MxGatewayMalformedReplyException(message);
}
if (ex instanceof MxGatewayAuthenticationException) {
return new MxGatewayAuthenticationException(message, null);
}
if (ex instanceof MxGatewayAuthorizationException) {
return new MxGatewayAuthorizationException(message, null);
}
return new MxGatewayException(message);
}
/**
* Produces a scrubbed clone of a command reply, removing any exact secret the
* gateway echoed into {@code protocolStatus.message},
* {@code diagnosticMessage}, or a status's {@code diagnosticText}.
*
* @param reply the reply to scrub, or {@code null}
* @param secrets the exact secrets to strip
* @return {@code null} when {@code reply} is {@code null}, otherwise a clone
* with every echoed secret replaced by the redaction marker
*/
private static MxCommandReply scrubReply(MxCommandReply reply, String... secrets) {
if (reply == null) {
return null;
}
MxCommandReply.Builder builder = reply.toBuilder();
if (builder.hasProtocolStatus()) {
builder.setProtocolStatus(scrubProtocolStatus(builder.getProtocolStatus(), secrets));
}
builder.setDiagnosticMessage(MxGatewaySecrets.redactExact(builder.getDiagnosticMessage(), secrets));
for (int index = 0; index < builder.getStatusesCount(); index++) {
MxStatusProxy.Builder status = builder.getStatuses(index).toBuilder();
status.setDiagnosticText(MxGatewaySecrets.redactExact(status.getDiagnosticText(), secrets));
builder.setStatuses(index, status);
}
return builder.build();
}
/**
* Produces a scrubbed clone of a protocol status, removing any exact secret
* the gateway echoed into its free-form {@code message}.
*/
private static ProtocolStatus scrubProtocolStatus(ProtocolStatus status, String... secrets) {
if (status == null) {
return null;
}
return status.toBuilder()
.setMessage(MxGatewaySecrets.redactExact(status.getMessage(), secrets))
.build();
}
/**
* Extracts the string payload of a secured-write value so it can be scrubbed
* from an echoed failure message. Only string-kind values carry a
* credential-shaped secret worth redacting; other kinds return {@code null}
* (ignored by {@link MxGatewaySecrets#redactExact}).
*/
private static String secretStringOf(MxValue value) {
if (value != null && value.getKindCase() == MxValue.KindCase.STRING_VALUE) {
return value.getStringValue();
}
return null;
}
private static String newCorrelationId() {
byte[] bytes = new byte[16];
RANDOM.nextBytes(bytes);
@@ -20,6 +20,20 @@ public final class MxGatewaySessionException extends MxGatewayException {
this.protocolStatus = protocolStatus;
}
/**
* Creates a new session exception with an already-built, verbatim message.
* Used to re-surface a failure with a redacted message while preserving the
* (already scrubbed) protocol status.
*
* @param message the exact message to surface (already formatted/redacted)
* @param protocolStatus protocol status returned by the gateway
* @param cause underlying error, or {@code null}
*/
protected MxGatewaySessionException(String message, ProtocolStatus protocolStatus, Throwable cause) {
super(message, cause);
this.protocolStatus = protocolStatus;
}
/**
* Returns the gateway protocol status that triggered this exception.
*
@@ -20,6 +20,20 @@ public final class MxGatewayWorkerException extends MxGatewayException {
this.protocolStatus = protocolStatus;
}
/**
* Creates a new worker exception with an already-built, verbatim message.
* Used to re-surface a failure with a redacted message while preserving the
* (already scrubbed) protocol status.
*
* @param message the exact message to surface (already formatted/redacted)
* @param protocolStatus protocol status returned by the gateway
* @param cause underlying error, or {@code null}
*/
protected MxGatewayWorkerException(String message, ProtocolStatus protocolStatus, Throwable cause) {
super(message, cause);
this.protocolStatus = protocolStatus;
}
/**
* Returns the gateway protocol status that triggered this exception.
*
@@ -8,8 +8,11 @@ import mxaccess_gateway.v1.MxaccessGateway.MxStatusSource;
* Helpers for inspecting {@link MxStatusProxy} values returned by the gateway.
*
* <p>An {@code MxStatusProxy} mirrors the MXAccess COM {@code MXSTATUS_PROXY}
* struct. The success flag uses the MXAccess convention where any non-zero
* value indicates success.
* struct. Per the wire contract, {@code category} is the authoritative verdict:
* an entry succeeds only when its category is
* {@code MX_STATUS_CATEGORY_OK}. The {@code success} member carries the raw
* 16-bit COM value verbatim for diagnostics and is not a boolean, so it never
* decides success or failure.
*/
public final class MxStatuses {
private MxStatuses() {
@@ -18,12 +21,17 @@ public final class MxStatuses {
/**
* Returns whether the supplied status proxy reports success.
*
* <p>A {@code null} status is success because nothing was reported. A
* present entry whose category is {@code MX_STATUS_CATEGORY_UNSPECIFIED}
* is a failure: the worker always maps a category, so an unmapped one is
* not proven OK.
*
* @param status the status proxy, may be {@code null}
* @return {@code true} if {@code status} is {@code null} or its success
* flag is non-zero, {@code false} otherwise
* @return {@code true} if {@code status} is {@code null} or its category is
* {@code MX_STATUS_CATEGORY_OK}, {@code false} otherwise
*/
public static boolean succeeded(MxStatusProxy status) {
return status == null || status.getSuccess() != 0;
return status == null || status.getCategory() == MxStatusCategory.MX_STATUS_CATEGORY_OK;
}
/**
@@ -44,9 +52,11 @@ public final class MxStatuses {
*/
public record MxStatusView(MxStatusProxy raw) {
/**
* Returns the raw success flag (non-zero indicates success).
* Returns the raw {@code success} member exactly as MXAccess reported
* it. This is a diagnostic value, not a verdict use
* {@link MxStatuses#succeeded(MxStatusProxy)} to decide success.
*
* @return the success flag value
* @return the raw success member
*/
public int success() {
return raw.getSuccess();
@@ -701,14 +701,17 @@ final class MxGatewayClientSessionTests {
.setSessionId(request.getSessionId())
.setKind(request.getCommand().getKind())
.setProtocolStatus(ok());
// `category` is the authoritative success indicator, so the fake
// must set it a bare non-zero `success` is not a success.
var okStatus = mxaccess_gateway.v1.MxaccessGateway.MxStatusProxy.newBuilder()
.setSuccess(1)
.setCategory(mxaccess_gateway.v1.MxaccessGateway.MxStatusCategory.MX_STATUS_CATEGORY_OK);
if (request.getCommand().getKind() == MxCommandKind.MX_COMMAND_KIND_SUSPEND) {
reply.setSuspend(mxaccess_gateway.v1.MxaccessGateway.SuspendReply.newBuilder()
.setStatus(mxaccess_gateway.v1.MxaccessGateway.MxStatusProxy.newBuilder()
.setSuccess(1)));
.setStatus(okStatus));
} else if (request.getCommand().getKind() == MxCommandKind.MX_COMMAND_KIND_ACTIVATE) {
reply.setActivate(mxaccess_gateway.v1.MxaccessGateway.ActivateReply.newBuilder()
.setStatus(mxaccess_gateway.v1.MxaccessGateway.MxStatusProxy.newBuilder()
.setSuccess(1)));
.setStatus(okStatus));
}
responseObserver.onNext(reply.build());
responseObserver.onCompleted();
@@ -0,0 +1,201 @@
package com.zb.mom.ww.mxgateway.client;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import com.google.protobuf.util.JsonFormat;
import io.grpc.ManagedChannel;
import io.grpc.Server;
import io.grpc.inprocess.InProcessChannelBuilder;
import io.grpc.inprocess.InProcessServerBuilder;
import io.grpc.stub.StreamObserver;
import java.nio.file.Files;
import java.nio.file.Path;
import java.time.Duration;
import java.util.UUID;
import mxaccess_gateway.v1.MxAccessGatewayGrpc;
import mxaccess_gateway.v1.MxaccessGateway.MxCommandReply;
import mxaccess_gateway.v1.MxaccessGateway.MxCommandRequest;
import mxaccess_gateway.v1.MxaccessGateway.MxValue;
import mxaccess_gateway.v1.MxaccessGateway.ProtocolStatus;
import mxaccess_gateway.v1.MxaccessGateway.ProtocolStatusCode;
import org.junit.jupiter.api.Test;
final class MxGatewayCredentialReplyTests {
private static final String CREDENTIAL = "sup3rSecretVerify9f3a2b";
@Test
void authenticateUserRedactsEchoedCredentialFromReplyDrivenError() throws Exception {
assertCredentialFullyRedacted(
"authenticate-user.echoed-credential.reply.json", "auth-echo-session");
}
@Test
void authenticateUserRedactsEchoedCredentialFromMxAccessFailureReply() throws Exception {
assertCredentialFullyRedacted(
"authenticate-user.echoed-credential-mxaccess-failure.reply.json",
"auth-echo-failure-session");
}
private static void assertCredentialFullyRedacted(String fixture, String sessionId) throws Exception {
MxCommandReply reply = loadReply(fixture);
try (InProcessGateway gateway = InProcessGateway.startReturning(reply);
MxGatewayClient client = gateway.client()) {
MxGatewaySession session = MxGatewaySession.forSessionId(client, sessionId);
MxAccessException error = assertThrows(
MxAccessException.class,
() -> session.authenticateUser(12, "operator", CREDENTIAL));
assertFalse(error.getMessage().contains(CREDENTIAL),
"credential echoed by the gateway must not survive in the surfaced message");
assertTrue(error.getMessage().contains("<redacted>"),
"the echoed credential must be replaced with the redaction marker");
// The rebuilt exception must not re-expose the credential through the
// structured reply/protocolStatus a caller can inspect and log.
MxCommandReply surfaced = error.reply();
assertNotNull(surfaced, "the redacted exception must preserve a reply for inspection");
assertFalse(surfaced.getProtocolStatus().getMessage().contains(CREDENTIAL),
"reply protocol status message must not leak the echoed credential");
assertFalse(surfaced.getDiagnosticMessage().contains(CREDENTIAL),
"reply diagnostic message must not leak the echoed credential");
for (int index = 0; index < surfaced.getStatusesCount(); index++) {
assertFalse(surfaced.getStatusesList().get(index).getDiagnosticText().contains(CREDENTIAL),
"reply status diagnostic text must not leak the echoed credential");
}
assertNotNull(error.protocolStatus(), "the redacted exception must preserve a protocol status");
assertFalse(error.protocolStatus().getMessage().contains(CREDENTIAL),
"exception protocol status must not leak the echoed credential");
}
}
@Test
void authenticateUserMissingPayloadThrowsMalformedReply() throws Exception {
MxCommandReply reply = loadReply("authenticate-user.missing-payload.reply.json");
try (InProcessGateway gateway = InProcessGateway.startReturning(reply);
MxGatewayClient client = gateway.client()) {
MxGatewaySession session = MxGatewaySession.forSessionId(client, "auth-missing-session");
assertThrows(
MxGatewayMalformedReplyException.class,
() -> session.authenticateUser(3, "operator", "pw"));
}
}
@Test
void authenticateUserReturnValueOnlyReplyReturnsInt32Fallback() throws Exception {
MxCommandReply reply = loadReply("authenticate-user.return-value-only.reply.json");
try (InProcessGateway gateway = InProcessGateway.startReturning(reply);
MxGatewayClient client = gateway.client()) {
MxGatewaySession session = MxGatewaySession.forSessionId(client, "auth-return-session");
assertEquals(7, session.authenticateUser(3, "operator", "pw"));
}
}
@Test
void addBufferedItemReturnValueOnlyReplyReturnsInt32Fallback() throws Exception {
MxCommandReply reply = MxCommandReply.newBuilder()
.setProtocolStatus(ok())
.setReturnValue(MxValue.newBuilder().setInt32Value(55))
.build();
try (InProcessGateway gateway = InProcessGateway.startReturning(reply);
MxGatewayClient client = gateway.client()) {
MxGatewaySession session = MxGatewaySession.forSessionId(client, "buffered-return-session");
assertEquals(55, session.addBufferedItem(3, "Tank01.Level", "galaxy"));
}
}
@Test
void addBufferedItemMissingPayloadThrowsMalformedReply() throws Exception {
MxCommandReply reply = MxCommandReply.newBuilder().setProtocolStatus(ok()).build();
try (InProcessGateway gateway = InProcessGateway.startReturning(reply);
MxGatewayClient client = gateway.client()) {
MxGatewaySession session = MxGatewaySession.forSessionId(client, "buffered-malformed-session");
assertThrows(
MxGatewayMalformedReplyException.class,
() -> session.addBufferedItem(3, "Tank01.Level", "galaxy"));
}
}
private static ProtocolStatus ok() {
return ProtocolStatus.newBuilder()
.setCode(ProtocolStatusCode.PROTOCOL_STATUS_CODE_OK)
.build();
}
private static MxCommandReply loadReply(String fixture) throws Exception {
MxCommandReply.Builder builder = MxCommandReply.newBuilder();
JsonFormat.parser().merge(
Files.readString(fixtureRoot().resolve("command-replies/" + fixture)),
builder);
return builder.build();
}
private static Path fixtureRoot() {
Path current = Path.of(System.getProperty("user.dir")).toAbsolutePath();
for (Path path = current; path != null; path = path.getParent()) {
Path candidate = path.resolve("clients/proto/fixtures/behavior");
if (Files.exists(candidate)) {
return candidate;
}
candidate = path.resolve("../proto/fixtures/behavior").normalize();
if (Files.exists(candidate)) {
return candidate;
}
}
throw new IllegalStateException("could not locate behavior fixtures from " + current);
}
private record InProcessGateway(Server server, ManagedChannel channel) implements AutoCloseable {
static InProcessGateway startReturning(MxCommandReply reply) throws Exception {
String serverName = "mxgw-java-cred-" + UUID.randomUUID();
MxAccessGatewayGrpc.MxAccessGatewayImplBase service =
new MxAccessGatewayGrpc.MxAccessGatewayImplBase() {
@Override
public void invoke(
MxCommandRequest request, StreamObserver<MxCommandReply> responseObserver) {
responseObserver.onNext(reply);
responseObserver.onCompleted();
}
};
Server server = InProcessServerBuilder.forName(serverName)
.directExecutor()
.addService(service)
.build()
.start();
ManagedChannel channel = InProcessChannelBuilder.forName(serverName)
.directExecutor()
.build();
return new InProcessGateway(server, channel);
}
MxGatewayClient client() {
return new MxGatewayClient(
channel,
MxGatewayClientOptions.builder()
.endpoint("in-process")
.apiKey("")
.plaintext(true)
.callTimeout(Duration.ofSeconds(5))
.build());
}
@Override
public void close() {
channel.shutdownNow();
server.shutdownNow();
}
}
}
@@ -4,6 +4,7 @@ import static org.junit.jupiter.api.Assertions.assertArrayEquals;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertInstanceOf;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import com.google.gson.JsonArray;
@@ -20,6 +21,8 @@ import mxaccess_gateway.v1.MxaccessGateway.MxStatusProxy;
import mxaccess_gateway.v1.MxaccessGateway.MxValue;
import mxaccess_gateway.v1.MxaccessGateway.ProtocolStatusCode;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
final class MxGatewayFixtureTests {
@Test
@@ -89,6 +92,50 @@ final class MxGatewayFixtureTests {
throw new AssertionError("expected MxAccessException");
}
@ParameterizedTest
@CsvSource({
"register.ok.reply.json,false",
"write.status-category-error-success-set.reply.json,true",
"write.status-category-ok-success-zero.reply.json,false",
"write.hresult-s-false.reply.json,false",
"write.hresult-e-fail.reply.json,true",
})
void replyValidationFixturesBranchOnCategoryAndNegativeHresult(String fixture, boolean expectFailure)
throws Exception {
MxCommandReply.Builder builder = MxCommandReply.newBuilder();
JsonFormat.parser().merge(
Files.readString(fixtureRoot().resolve("command-replies/" + fixture)),
builder);
MxCommandReply reply = builder.build();
if (expectFailure) {
assertThrows(MxAccessException.class, () -> MxGatewayErrors.ensureMxAccessSuccess("write", reply));
} else {
MxGatewayErrors.ensureMxAccessSuccess("write", reply);
}
}
@ParameterizedTest
@CsvSource({
"MX_STATUS_CATEGORY_OK,0,true",
"MX_STATUS_CATEGORY_OK,1,true",
"MX_STATUS_CATEGORY_COMMUNICATION_ERROR,1,false",
"MX_STATUS_CATEGORY_UNSPECIFIED,1,false",
})
void statusEntryVerdictIgnoresTheRawSuccessMember(String category, int success, boolean expectSucceeded) {
MxStatusProxy status = MxStatusProxy.newBuilder()
.setCategory(MxStatusCategory.valueOf(category))
.setSuccess(success)
.build();
assertEquals(expectSucceeded, MxStatuses.succeeded(status));
}
@Test
void absentStatusEntryIsSuccess() {
assertTrue(MxStatuses.succeeded(null));
}
@Test
void grpcAuthErrorsAreClassifiedAndRedacted() {
RuntimeException authError = MxGatewayErrors.fromGrpc(
@@ -0,0 +1,50 @@
package com.zb.mom.ww.mxgateway.client;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNull;
import org.junit.jupiter.api.Test;
final class MxGatewaySecretsTests {
@Test
void redactExactReplacesEveryOccurrenceOfASecret() {
String message = "verify s3cr3t, retry s3cr3t, done s3cr3t";
String result = MxGatewaySecrets.redactExact(message, "s3cr3t");
assertFalse(result.contains("s3cr3t"), "no occurrence of the secret may survive");
assertEquals("verify <redacted>, retry <redacted>, done <redacted>", result);
}
@Test
void redactExactFullyRedactsOverlappingSecretsWhenOneIsASubstringOfTheOther() {
String message = "password=hunter2 token=hunter2extra";
String result = MxGatewaySecrets.redactExact(message, "hunter2extra", "hunter2");
assertFalse(result.contains("hunter2"), "both the secret and its superstring must be fully redacted");
assertEquals("password=<redacted> token=<redacted>", result);
}
@Test
void redactExactWithNoSecretsReturnsMessageUnchanged() {
String message = "nothing to scrub here";
assertEquals(message, MxGatewaySecrets.redactExact(message));
}
@Test
void redactExactToleratesNullMessage() {
assertNull(MxGatewaySecrets.redactExact(null, "secret"));
}
@Test
void redactExactIgnoresBlankSecretSoRealSpacesAreNotOverRedacted() {
String message = "keep these spaces intact";
String result = MxGatewaySecrets.redactExact(message, " ", "");
assertEquals(message, result);
}
}
@@ -0,0 +1,22 @@
{
"sessionId": "session-fixture",
"correlationId": "gateway-correlation-authenticate-echoed-mxaccess-failure",
"kind": "MX_COMMAND_KIND_AUTHENTICATE_USER",
"protocolStatus": {
"code": "PROTOCOL_STATUS_CODE_MXACCESS_FAILURE",
"message": "MXAccess AuthenticateUser rejected credential 'sup3rSecretVerify9f3a2b'."
},
"hresult": -2147024891,
"statuses": [
{
"success": 0,
"category": "MX_STATUS_CATEGORY_SECURITY_ERROR",
"detectedBy": "MX_STATUS_SOURCE_RESPONDING_NMX",
"detail": 5,
"rawCategory": 8,
"rawDetectedBy": 5,
"diagnosticText": "Authentication failed for password 'sup3rSecretVerify9f3a2b'."
}
],
"diagnosticMessage": "MXAccess echoed the credential 'sup3rSecretVerify9f3a2b' back in its failure diagnostic."
}
@@ -0,0 +1,22 @@
{
"sessionId": "session-fixture",
"correlationId": "gateway-correlation-authenticate-echoed",
"kind": "MX_COMMAND_KIND_AUTHENTICATE_USER",
"protocolStatus": {
"code": "PROTOCOL_STATUS_CODE_OK",
"message": "MXAccess AuthenticateUser rejected credential 'sup3rSecretVerify9f3a2b'."
},
"hresult": -2147024891,
"statuses": [
{
"success": 0,
"category": "MX_STATUS_CATEGORY_SECURITY_ERROR",
"detectedBy": "MX_STATUS_SOURCE_RESPONDING_NMX",
"detail": 5,
"rawCategory": 8,
"rawDetectedBy": 5,
"diagnosticText": "Authentication failed for password 'sup3rSecretVerify9f3a2b'."
}
],
"diagnosticMessage": "MXAccess echoed the credential 'sup3rSecretVerify9f3a2b' back in its failure diagnostic."
}
@@ -0,0 +1,10 @@
{
"sessionId": "session-fixture",
"correlationId": "gateway-correlation-authenticate-missing-payload",
"kind": "MX_COMMAND_KIND_AUTHENTICATE_USER",
"protocolStatus": {
"code": "PROTOCOL_STATUS_CODE_OK",
"message": "AuthenticateUser reached MXAccess."
},
"diagnosticMessage": "Malformed: the OK reply carried neither an AuthenticateUser payload nor a return_value."
}
@@ -0,0 +1,15 @@
{
"sessionId": "session-fixture",
"correlationId": "gateway-correlation-authenticate-return-value-only",
"kind": "MX_COMMAND_KIND_AUTHENTICATE_USER",
"protocolStatus": {
"code": "PROTOCOL_STATUS_CODE_OK",
"message": "AuthenticateUser reached MXAccess."
},
"returnValue": {
"dataType": "MX_DATA_TYPE_INTEGER",
"variantType": "VT_I4",
"int32Value": 7
},
"diagnosticMessage": "Legacy worker populated only return_value; the typed AuthenticateUser payload is absent."
}
@@ -0,0 +1,29 @@
{
"sessionId": "session-fixture",
"correlationId": "gateway-correlation-write-e-fail",
"kind": "MX_COMMAND_KIND_WRITE",
"protocolStatus": {
"code": "PROTOCOL_STATUS_CODE_OK",
"message": "Write reached MXAccess."
},
"hresult": -2147467259,
"returnValue": {
"dataType": "MX_DATA_TYPE_NO_DATA",
"variantType": "VT_EMPTY",
"isNull": true,
"rawDiagnostic": "MXAccess returned no value for the failed write.",
"rawDataType": 2
},
"statuses": [
{
"success": 1,
"category": "MX_STATUS_CATEGORY_OK",
"detectedBy": "MX_STATUS_SOURCE_RESPONDING_LMX",
"detail": 0,
"rawCategory": 0,
"rawDetectedBy": 3,
"diagnosticText": "OK"
}
],
"diagnosticMessage": "COM semantics: a negative HRESULT (E_FAIL, 0x80004005) is a failure even when every status entry is OK."
}
@@ -0,0 +1,29 @@
{
"sessionId": "session-fixture",
"correlationId": "gateway-correlation-write-s-false",
"kind": "MX_COMMAND_KIND_WRITE",
"protocolStatus": {
"code": "PROTOCOL_STATUS_CODE_OK",
"message": "Write completed with S_FALSE."
},
"hresult": 1,
"returnValue": {
"dataType": "MX_DATA_TYPE_NO_DATA",
"variantType": "VT_EMPTY",
"isNull": true,
"rawDiagnostic": "MXAccess returned no value for the write.",
"rawDataType": 2
},
"statuses": [
{
"success": 1,
"category": "MX_STATUS_CATEGORY_OK",
"detectedBy": "MX_STATUS_SOURCE_RESPONDING_LMX",
"detail": 0,
"rawCategory": 0,
"rawDetectedBy": 3,
"diagnosticText": "OK"
}
],
"diagnosticMessage": "COM semantics: a positive HRESULT such as S_FALSE (1) is a success code, not a failure."
}
@@ -0,0 +1,29 @@
{
"sessionId": "session-fixture",
"correlationId": "gateway-correlation-write-category-error",
"kind": "MX_COMMAND_KIND_WRITE",
"protocolStatus": {
"code": "PROTOCOL_STATUS_CODE_OK",
"message": "Write reached MXAccess."
},
"hresult": 0,
"returnValue": {
"dataType": "MX_DATA_TYPE_NO_DATA",
"variantType": "VT_EMPTY",
"isNull": true,
"rawDiagnostic": "MXAccess returned no value for the write.",
"rawDataType": 2
},
"statuses": [
{
"success": 1,
"category": "MX_STATUS_CATEGORY_COMMUNICATION_ERROR",
"detectedBy": "MX_STATUS_SOURCE_RESPONDING_LMX",
"detail": 77,
"rawCategory": 5,
"rawDetectedBy": 3,
"diagnosticText": "Responding LMX lost communication mid-write."
}
],
"diagnosticMessage": "Category is authoritative: a non-OK category is a failure even when the raw success member is non-zero."
}
@@ -0,0 +1,29 @@
{
"sessionId": "session-fixture",
"correlationId": "gateway-correlation-write-category-ok",
"kind": "MX_COMMAND_KIND_WRITE",
"protocolStatus": {
"code": "PROTOCOL_STATUS_CODE_OK",
"message": "Write completed."
},
"hresult": 0,
"returnValue": {
"dataType": "MX_DATA_TYPE_NO_DATA",
"variantType": "VT_EMPTY",
"isNull": true,
"rawDiagnostic": "MXAccess returned no value for the write.",
"rawDataType": 2
},
"statuses": [
{
"success": 0,
"category": "MX_STATUS_CATEGORY_OK",
"detectedBy": "MX_STATUS_SOURCE_RESPONDING_LMX",
"detail": 0,
"rawCategory": 0,
"rawDetectedBy": 3,
"diagnosticText": "OK, reported with a zero raw success member."
}
],
"diagnosticMessage": "Category is authoritative: MX_STATUS_CATEGORY_OK is success even when the raw success member is zero."
}
@@ -20,6 +20,62 @@
"path": "command-replies/write.mxaccess-failure.reply.json",
"expectation": "MXAccess failures are data-bearing replies with HRESULT and status details, not transport failures."
},
{
"id": "command-reply.write.status-category-error-success-set",
"category": "command_replies",
"messageType": "mxaccess_gateway.v1.MxCommandReply",
"path": "command-replies/write.status-category-error-success-set.reply.json",
"expectation": "A status entry fails when its category is not MX_STATUS_CATEGORY_OK, even though the raw success member is non-zero."
},
{
"id": "command-reply.write.status-category-ok-success-zero",
"category": "command_replies",
"messageType": "mxaccess_gateway.v1.MxCommandReply",
"path": "command-replies/write.status-category-ok-success-zero.reply.json",
"expectation": "A status entry succeeds when its category is MX_STATUS_CATEGORY_OK, even though the raw success member is zero."
},
{
"id": "command-reply.write.hresult-s-false",
"category": "command_replies",
"messageType": "mxaccess_gateway.v1.MxCommandReply",
"path": "command-replies/write.hresult-s-false.reply.json",
"expectation": "A positive HRESULT such as S_FALSE (1) is a COM success code and does not fail the reply."
},
{
"id": "command-reply.write.hresult-e-fail",
"category": "command_replies",
"messageType": "mxaccess_gateway.v1.MxCommandReply",
"path": "command-replies/write.hresult-e-fail.reply.json",
"expectation": "A negative HRESULT fails the reply even when every status entry reports MX_STATUS_CATEGORY_OK."
},
{
"id": "command-reply.authenticate-user.echoed-credential",
"category": "command_replies",
"messageType": "mxaccess_gateway.v1.MxCommandReply",
"path": "command-replies/authenticate-user.echoed-credential.reply.json",
"expectation": "When a gateway/MXAccess diagnostic echoes the caller's credential back (OK envelope, negative HRESULT), the surfaced error redacts the exact secret from both the rendered message and the structured reply accessors."
},
{
"id": "command-reply.authenticate-user.echoed-credential-mxaccess-failure",
"category": "command_replies",
"messageType": "mxaccess_gateway.v1.MxCommandReply",
"path": "command-replies/authenticate-user.echoed-credential-mxaccess-failure.reply.json",
"expectation": "The same echoed-credential redaction holds when the reply is coded PROTOCOL_STATUS_CODE_MXACCESS_FAILURE, which every client routes to its MXAccess error type."
},
{
"id": "command-reply.authenticate-user.missing-payload",
"category": "command_replies",
"messageType": "mxaccess_gateway.v1.MxCommandReply",
"path": "command-replies/authenticate-user.missing-payload.reply.json",
"expectation": "An OK reply with neither the typed AuthenticateUser payload nor a return_value raises a typed malformed-reply error, never a proto3 default 0 and never an NRE."
},
{
"id": "command-reply.authenticate-user.return-value-only",
"category": "command_replies",
"messageType": "mxaccess_gateway.v1.MxCommandReply",
"path": "command-replies/authenticate-user.return-value-only.reply.json",
"expectation": "An OK reply missing the typed AuthenticateUser payload but carrying an int32 return_value falls back to the return_value (legacy-worker compatibility)."
},
{
"id": "event-stream.session-ordered",
"category": "event_streams",
@@ -3,6 +3,7 @@
"cases": [
{
"id": "ok.responding-lmx",
"wantSuccess": true,
"status": {
"success": 1,
"category": "MX_STATUS_CATEGORY_OK",
@@ -15,6 +16,7 @@
},
{
"id": "security-error.requesting-lmx",
"wantSuccess": false,
"status": {
"success": 0,
"category": "MX_STATUS_CATEGORY_SECURITY_ERROR",
@@ -27,6 +29,7 @@
},
{
"id": "raw-unknown-category",
"wantSuccess": false,
"status": {
"success": 0,
"category": "MX_STATUS_CATEGORY_UNKNOWN",
+7 -1
View File
@@ -187,7 +187,13 @@ await session.write_secured(
```
The CLI mirrors these as `authenticate-user` (credential via `--password` or,
preferably, `--password-env`) and `write-secured`.
preferably, the variable named by `--password-env`, default
`MXGATEWAY_VERIFY_PASSWORD`) and `write-secured`. The credential is required: a
missing or empty resolved value raises a `UsageError` naming the option and the
variable, so the CLI fails before connecting instead of authenticating with an
empty password. `MXGATEWAY_VERIFY_PASSWORD` is the canonical variable across all
five client CLIs — see
[Cross-Language Smoke Matrix](../../docs/CrossLanguageSmokeMatrix.md).
### Array writes replace the whole array
+1 -1
View File
@@ -6,7 +6,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "zb-mom-ww-mxaccess-gateway-client"
version = "0.1.2"
version = "0.2.0"
description = "Async Python client for MXAccess Gateway."
readme = "README.md"
requires-python = ">=3.12"
@@ -11,6 +11,7 @@ from .generated.galaxy_repository_pb2 import (
)
from .events import ReplayGap
from .errors import (
MalformedReplyError,
MxAccessError,
MxGatewayAuthenticationError,
MxGatewayAuthorizationError,
@@ -35,6 +36,7 @@ __all__ = [
"GalaxyRepositoryClient",
"GatewayClient",
"LazyBrowseNode",
"MalformedReplyError",
"MxAccessError",
"MxGatewayAuthenticationError",
"MxGatewayAuthorizationError",
@@ -53,6 +53,10 @@ class MxAccessError(MxGatewayCommandError):
"""MXAccess HRESULT or status failure."""
class MalformedReplyError(MxGatewayError):
"""Raised when an OK reply lacks the expected typed payload and any usable return_value fallback."""
def map_rpc_error(operation: str, error: grpc.RpcError) -> MxGatewayTransportError:
"""Map a generated gRPC exception to the client exception hierarchy."""
@@ -137,8 +141,10 @@ def ensure_mxaccess_success(operation: str, reply: pb.MxCommandReply) -> pb.MxCo
raw_reply=reply,
)
# `category` is the authoritative verdict per the wire contract; `success`
# is the raw COM member carried verbatim for diagnostics only.
for mx_status in reply.statuses:
if mx_status.success == 0:
if mx_status.category != pb.MX_STATUS_CATEGORY_OK:
raise MxAccessError(
_mxaccess_message(operation, reply),
protocol_status=status,
@@ -151,8 +157,18 @@ def ensure_mxaccess_success(operation: str, reply: pb.MxCommandReply) -> pb.MxCo
def _mxaccess_message(operation: str, reply: pb.MxCommandReply) -> str:
status_text = reply.protocol_status.message or "MXAccess command failed"
hresult = reply.hresult if reply.HasField("hresult") else None
return (
message = (
f"{operation} failed: {status_text}; "
f"session={reply.session_id}; correlation={reply.correlation_id}; "
f"hresult={hresult}; statuses={len(reply.statuses)}"
)
# Append a per-status breakdown that carries the raw `success` COM member
# verbatim for diagnostic parity with the other clients. `category` remains
# the authoritative verdict; `success` is diagnostics only.
for status in reply.statuses:
category = pb.MxStatusCategory.Name(status.category)
message += (
f" [success={status.success}, category={category}, "
f"detail={status.detail}, {status.diagnostic_text}]"
)
return message
@@ -27,7 +27,7 @@ from google.protobuf import timestamp_pb2 as google_dot_protobuf_dot_timestamp__
import mxaccess_gateway_pb2 as mxaccess__gateway__pb2
DESCRIPTOR = _descriptor_pool.Default().AddSerializedFile(b'\n\x15mxaccess_worker.proto\x12\x12mxaccess_worker.v1\x1a\x1egoogle/protobuf/duration.proto\x1a\x1fgoogle/protobuf/timestamp.proto\x1a\x16mxaccess_gateway.proto\"\x95\x06\n\x0eWorkerEnvelope\x12\x18\n\x10protocol_version\x18\x01 \x01(\r\x12\x12\n\nsession_id\x18\x02 \x01(\t\x12\x10\n\x08sequence\x18\x03 \x01(\x04\x12\x16\n\x0e\x63orrelation_id\x18\x04 \x01(\t\x12\x39\n\rgateway_hello\x18\n \x01(\x0b\x32 .mxaccess_worker.v1.GatewayHelloH\x00\x12\x37\n\x0cworker_hello\x18\x0b \x01(\x0b\x32\x1f.mxaccess_worker.v1.WorkerHelloH\x00\x12\x37\n\x0cworker_ready\x18\x0c \x01(\x0b\x32\x1f.mxaccess_worker.v1.WorkerReadyH\x00\x12;\n\x0eworker_command\x18\r \x01(\x0b\x32!.mxaccess_worker.v1.WorkerCommandH\x00\x12\x46\n\x14worker_command_reply\x18\x0e \x01(\x0b\x32&.mxaccess_worker.v1.WorkerCommandReplyH\x00\x12\x39\n\rworker_cancel\x18\x0f \x01(\x0b\x32 .mxaccess_worker.v1.WorkerCancelH\x00\x12=\n\x0fworker_shutdown\x18\x10 \x01(\x0b\x32\".mxaccess_worker.v1.WorkerShutdownH\x00\x12\x44\n\x13worker_shutdown_ack\x18\x11 \x01(\x0b\x32%.mxaccess_worker.v1.WorkerShutdownAckH\x00\x12\x37\n\x0cworker_event\x18\x12 \x01(\x0b\x32\x1f.mxaccess_worker.v1.WorkerEventH\x00\x12?\n\x10worker_heartbeat\x18\x13 \x01(\x0b\x32#.mxaccess_worker.v1.WorkerHeartbeatH\x00\x12\x37\n\x0cworker_fault\x18\x14 \x01(\x0b\x32\x1f.mxaccess_worker.v1.WorkerFaultH\x00\x42\x06\n\x04\x62ody\"Z\n\x0cGatewayHello\x12\"\n\x1asupported_protocol_version\x18\x01 \x01(\r\x12\r\n\x05nonce\x18\x02 \x01(\t\x12\x17\n\x0fgateway_version\x18\x03 \x01(\t\"i\n\x0bWorkerHello\x12\x18\n\x10protocol_version\x18\x01 \x01(\r\x12\r\n\x05nonce\x18\x02 \x01(\t\x12\x19\n\x11worker_process_id\x18\x03 \x01(\x05\x12\x16\n\x0eworker_version\x18\x04 \x01(\t\"\x8e\x01\n\x0bWorkerReady\x12\x19\n\x11worker_process_id\x18\x01 \x01(\x05\x12\x17\n\x0fmxaccess_progid\x18\x02 \x01(\t\x12\x16\n\x0emxaccess_clsid\x18\x03 \x01(\t\x12\x33\n\x0fready_timestamp\x18\x04 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\"w\n\rWorkerCommand\x12/\n\x07\x63ommand\x18\x01 \x01(\x0b\x32\x1e.mxaccess_gateway.v1.MxCommand\x12\x35\n\x11\x65nqueue_timestamp\x18\x02 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\"\x81\x01\n\x12WorkerCommandReply\x12\x32\n\x05reply\x18\x01 \x01(\x0b\x32#.mxaccess_gateway.v1.MxCommandReply\x12\x37\n\x13\x63ompleted_timestamp\x18\x02 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\"\x1e\n\x0cWorkerCancel\x12\x0e\n\x06reason\x18\x01 \x01(\t\"Q\n\x0eWorkerShutdown\x12/\n\x0cgrace_period\x18\x01 \x01(\x0b\x32\x19.google.protobuf.Duration\x12\x0e\n\x06reason\x18\x02 \x01(\t\"H\n\x11WorkerShutdownAck\x12\x33\n\x06status\x18\x01 \x01(\x0b\x32#.mxaccess_gateway.v1.ProtocolStatus\":\n\x0bWorkerEvent\x12+\n\x05\x65vent\x18\x01 \x01(\x0b\x32\x1c.mxaccess_gateway.v1.MxEvent\"\xa5\x02\n\x0fWorkerHeartbeat\x12\x19\n\x11worker_process_id\x18\x01 \x01(\x05\x12.\n\x05state\x18\x02 \x01(\x0e\x32\x1f.mxaccess_worker.v1.WorkerState\x12?\n\x1blast_sta_activity_timestamp\x18\x03 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\x12\x1d\n\x15pending_command_count\x18\x04 \x01(\r\x12\"\n\x1aoutbound_event_queue_depth\x18\x05 \x01(\r\x12\x1b\n\x13last_event_sequence\x18\x06 \x01(\x04\x12&\n\x1e\x63urrent_command_correlation_id\x18\x07 \x01(\t\"\xf4\x01\n\x0bWorkerFault\x12\x39\n\x08\x63\x61tegory\x18\x01 \x01(\x0e\x32\'.mxaccess_worker.v1.WorkerFaultCategory\x12\x16\n\x0e\x63ommand_method\x18\x02 \x01(\t\x12\x14\n\x07hresult\x18\x03 \x01(\x05H\x00\x88\x01\x01\x12\x16\n\x0e\x65xception_type\x18\x04 \x01(\t\x12\x1a\n\x12\x64iagnostic_message\x18\x05 \x01(\t\x12<\n\x0fprotocol_status\x18\x06 \x01(\x0b\x32#.mxaccess_gateway.v1.ProtocolStatusB\n\n\x08_hresult*\x97\x02\n\x0bWorkerState\x12\x1c\n\x18WORKER_STATE_UNSPECIFIED\x10\x00\x12\x19\n\x15WORKER_STATE_STARTING\x10\x01\x12\x1c\n\x18WORKER_STATE_HANDSHAKING\x10\x02\x12!\n\x1dWORKER_STATE_INITIALIZING_STA\x10\x03\x12\x16\n\x12WORKER_STATE_READY\x10\x04\x12\"\n\x1eWORKER_STATE_EXECUTING_COMMAND\x10\x05\x12\x1e\n\x1aWORKER_STATE_SHUTTING_DOWN\x10\x06\x12\x18\n\x14WORKER_STATE_STOPPED\x10\x07\x12\x18\n\x14WORKER_STATE_FAULTED\x10\x08*\xc7\x04\n\x13WorkerFaultCategory\x12%\n!WORKER_FAULT_CATEGORY_UNSPECIFIED\x10\x00\x12+\n\'WORKER_FAULT_CATEGORY_INVALID_ARGUMENTS\x10\x01\x12\x37\n3WORKER_FAULT_CATEGORY_GATEWAY_AUTHENTICATION_FAILED\x10\x02\x12+\n\'WORKER_FAULT_CATEGORY_PROTOCOL_MISMATCH\x10\x03\x12,\n(WORKER_FAULT_CATEGORY_PROTOCOL_VIOLATION\x10\x04\x12+\n\'WORKER_FAULT_CATEGORY_PIPE_DISCONNECTED\x10\x05\x12\x32\n.WORKER_FAULT_CATEGORY_MXACCESS_CREATION_FAILED\x10\x06\x12\x31\n-WORKER_FAULT_CATEGORY_MXACCESS_COMMAND_FAILED\x10\x07\x12:\n6WORKER_FAULT_CATEGORY_MXACCESS_EVENT_CONVERSION_FAILED\x10\x08\x12\"\n\x1eWORKER_FAULT_CATEGORY_STA_HUNG\x10\t\x12(\n$WORKER_FAULT_CATEGORY_QUEUE_OVERFLOW\x10\n\x12*\n&WORKER_FAULT_CATEGORY_SHUTDOWN_TIMEOUT\x10\x0b\x42&\xaa\x02#ZB.MOM.WW.MxGateway.Contracts.Protob\x06proto3')
DESCRIPTOR = _descriptor_pool.Default().AddSerializedFile(b'\n\x15mxaccess_worker.proto\x12\x12mxaccess_worker.v1\x1a\x1egoogle/protobuf/duration.proto\x1a\x1fgoogle/protobuf/timestamp.proto\x1a\x16mxaccess_gateway.proto\"\x95\x06\n\x0eWorkerEnvelope\x12\x18\n\x10protocol_version\x18\x01 \x01(\r\x12\x12\n\nsession_id\x18\x02 \x01(\t\x12\x10\n\x08sequence\x18\x03 \x01(\x04\x12\x16\n\x0e\x63orrelation_id\x18\x04 \x01(\t\x12\x39\n\rgateway_hello\x18\n \x01(\x0b\x32 .mxaccess_worker.v1.GatewayHelloH\x00\x12\x37\n\x0cworker_hello\x18\x0b \x01(\x0b\x32\x1f.mxaccess_worker.v1.WorkerHelloH\x00\x12\x37\n\x0cworker_ready\x18\x0c \x01(\x0b\x32\x1f.mxaccess_worker.v1.WorkerReadyH\x00\x12;\n\x0eworker_command\x18\r \x01(\x0b\x32!.mxaccess_worker.v1.WorkerCommandH\x00\x12\x46\n\x14worker_command_reply\x18\x0e \x01(\x0b\x32&.mxaccess_worker.v1.WorkerCommandReplyH\x00\x12\x39\n\rworker_cancel\x18\x0f \x01(\x0b\x32 .mxaccess_worker.v1.WorkerCancelH\x00\x12=\n\x0fworker_shutdown\x18\x10 \x01(\x0b\x32\".mxaccess_worker.v1.WorkerShutdownH\x00\x12\x44\n\x13worker_shutdown_ack\x18\x11 \x01(\x0b\x32%.mxaccess_worker.v1.WorkerShutdownAckH\x00\x12\x37\n\x0cworker_event\x18\x12 \x01(\x0b\x32\x1f.mxaccess_worker.v1.WorkerEventH\x00\x12?\n\x10worker_heartbeat\x18\x13 \x01(\x0b\x32#.mxaccess_worker.v1.WorkerHeartbeatH\x00\x12\x37\n\x0cworker_fault\x18\x14 \x01(\x0b\x32\x1f.mxaccess_worker.v1.WorkerFaultH\x00\x42\x06\n\x04\x62ody\"s\n\x0cGatewayHello\x12\"\n\x1asupported_protocol_version\x18\x01 \x01(\r\x12\r\n\x05nonce\x18\x02 \x01(\t\x12\x17\n\x0fgateway_version\x18\x03 \x01(\t\x12\x17\n\x0fmax_frame_bytes\x18\x04 \x01(\r\"i\n\x0bWorkerHello\x12\x18\n\x10protocol_version\x18\x01 \x01(\r\x12\r\n\x05nonce\x18\x02 \x01(\t\x12\x19\n\x11worker_process_id\x18\x03 \x01(\x05\x12\x16\n\x0eworker_version\x18\x04 \x01(\t\"\x8e\x01\n\x0bWorkerReady\x12\x19\n\x11worker_process_id\x18\x01 \x01(\x05\x12\x17\n\x0fmxaccess_progid\x18\x02 \x01(\t\x12\x16\n\x0emxaccess_clsid\x18\x03 \x01(\t\x12\x33\n\x0fready_timestamp\x18\x04 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\"w\n\rWorkerCommand\x12/\n\x07\x63ommand\x18\x01 \x01(\x0b\x32\x1e.mxaccess_gateway.v1.MxCommand\x12\x35\n\x11\x65nqueue_timestamp\x18\x02 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\"\x81\x01\n\x12WorkerCommandReply\x12\x32\n\x05reply\x18\x01 \x01(\x0b\x32#.mxaccess_gateway.v1.MxCommandReply\x12\x37\n\x13\x63ompleted_timestamp\x18\x02 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\"\x1e\n\x0cWorkerCancel\x12\x0e\n\x06reason\x18\x01 \x01(\t\"Q\n\x0eWorkerShutdown\x12/\n\x0cgrace_period\x18\x01 \x01(\x0b\x32\x19.google.protobuf.Duration\x12\x0e\n\x06reason\x18\x02 \x01(\t\"H\n\x11WorkerShutdownAck\x12\x33\n\x06status\x18\x01 \x01(\x0b\x32#.mxaccess_gateway.v1.ProtocolStatus\":\n\x0bWorkerEvent\x12+\n\x05\x65vent\x18\x01 \x01(\x0b\x32\x1c.mxaccess_gateway.v1.MxEvent\"\xa5\x02\n\x0fWorkerHeartbeat\x12\x19\n\x11worker_process_id\x18\x01 \x01(\x05\x12.\n\x05state\x18\x02 \x01(\x0e\x32\x1f.mxaccess_worker.v1.WorkerState\x12?\n\x1blast_sta_activity_timestamp\x18\x03 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\x12\x1d\n\x15pending_command_count\x18\x04 \x01(\r\x12\"\n\x1aoutbound_event_queue_depth\x18\x05 \x01(\r\x12\x1b\n\x13last_event_sequence\x18\x06 \x01(\x04\x12&\n\x1e\x63urrent_command_correlation_id\x18\x07 \x01(\t\"\xf4\x01\n\x0bWorkerFault\x12\x39\n\x08\x63\x61tegory\x18\x01 \x01(\x0e\x32\'.mxaccess_worker.v1.WorkerFaultCategory\x12\x16\n\x0e\x63ommand_method\x18\x02 \x01(\t\x12\x14\n\x07hresult\x18\x03 \x01(\x05H\x00\x88\x01\x01\x12\x16\n\x0e\x65xception_type\x18\x04 \x01(\t\x12\x1a\n\x12\x64iagnostic_message\x18\x05 \x01(\t\x12<\n\x0fprotocol_status\x18\x06 \x01(\x0b\x32#.mxaccess_gateway.v1.ProtocolStatusB\n\n\x08_hresult*\x97\x02\n\x0bWorkerState\x12\x1c\n\x18WORKER_STATE_UNSPECIFIED\x10\x00\x12\x19\n\x15WORKER_STATE_STARTING\x10\x01\x12\x1c\n\x18WORKER_STATE_HANDSHAKING\x10\x02\x12!\n\x1dWORKER_STATE_INITIALIZING_STA\x10\x03\x12\x16\n\x12WORKER_STATE_READY\x10\x04\x12\"\n\x1eWORKER_STATE_EXECUTING_COMMAND\x10\x05\x12\x1e\n\x1aWORKER_STATE_SHUTTING_DOWN\x10\x06\x12\x18\n\x14WORKER_STATE_STOPPED\x10\x07\x12\x18\n\x14WORKER_STATE_FAULTED\x10\x08*\xc7\x04\n\x13WorkerFaultCategory\x12%\n!WORKER_FAULT_CATEGORY_UNSPECIFIED\x10\x00\x12+\n\'WORKER_FAULT_CATEGORY_INVALID_ARGUMENTS\x10\x01\x12\x37\n3WORKER_FAULT_CATEGORY_GATEWAY_AUTHENTICATION_FAILED\x10\x02\x12+\n\'WORKER_FAULT_CATEGORY_PROTOCOL_MISMATCH\x10\x03\x12,\n(WORKER_FAULT_CATEGORY_PROTOCOL_VIOLATION\x10\x04\x12+\n\'WORKER_FAULT_CATEGORY_PIPE_DISCONNECTED\x10\x05\x12\x32\n.WORKER_FAULT_CATEGORY_MXACCESS_CREATION_FAILED\x10\x06\x12\x31\n-WORKER_FAULT_CATEGORY_MXACCESS_COMMAND_FAILED\x10\x07\x12:\n6WORKER_FAULT_CATEGORY_MXACCESS_EVENT_CONVERSION_FAILED\x10\x08\x12\"\n\x1eWORKER_FAULT_CATEGORY_STA_HUNG\x10\t\x12(\n$WORKER_FAULT_CATEGORY_QUEUE_OVERFLOW\x10\n\x12*\n&WORKER_FAULT_CATEGORY_SHUTDOWN_TIMEOUT\x10\x0b\x42&\xaa\x02#ZB.MOM.WW.MxGateway.Contracts.Protob\x06proto3')
_globals = globals()
_builder.BuildMessageAndEnumDescriptors(DESCRIPTOR, _globals)
@@ -35,32 +35,32 @@ _builder.BuildTopDescriptorsAndMessages(DESCRIPTOR, 'mxaccess_worker_pb2', _glob
if not _descriptor._USE_C_DESCRIPTORS:
_globals['DESCRIPTOR']._loaded_options = None
_globals['DESCRIPTOR']._serialized_options = b'\252\002#ZB.MOM.WW.MxGateway.Contracts.Proto'
_globals['_WORKERSTATE']._serialized_start=2316
_globals['_WORKERSTATE']._serialized_end=2595
_globals['_WORKERFAULTCATEGORY']._serialized_start=2598
_globals['_WORKERFAULTCATEGORY']._serialized_end=3181
_globals['_WORKERSTATE']._serialized_start=2341
_globals['_WORKERSTATE']._serialized_end=2620
_globals['_WORKERFAULTCATEGORY']._serialized_start=2623
_globals['_WORKERFAULTCATEGORY']._serialized_end=3206
_globals['_WORKERENVELOPE']._serialized_start=135
_globals['_WORKERENVELOPE']._serialized_end=924
_globals['_GATEWAYHELLO']._serialized_start=926
_globals['_GATEWAYHELLO']._serialized_end=1016
_globals['_WORKERHELLO']._serialized_start=1018
_globals['_WORKERHELLO']._serialized_end=1123
_globals['_WORKERREADY']._serialized_start=1126
_globals['_WORKERREADY']._serialized_end=1268
_globals['_WORKERCOMMAND']._serialized_start=1270
_globals['_WORKERCOMMAND']._serialized_end=1389
_globals['_WORKERCOMMANDREPLY']._serialized_start=1392
_globals['_WORKERCOMMANDREPLY']._serialized_end=1521
_globals['_WORKERCANCEL']._serialized_start=1523
_globals['_WORKERCANCEL']._serialized_end=1553
_globals['_WORKERSHUTDOWN']._serialized_start=1555
_globals['_WORKERSHUTDOWN']._serialized_end=1636
_globals['_WORKERSHUTDOWNACK']._serialized_start=1638
_globals['_WORKERSHUTDOWNACK']._serialized_end=1710
_globals['_WORKEREVENT']._serialized_start=1712
_globals['_WORKEREVENT']._serialized_end=1770
_globals['_WORKERHEARTBEAT']._serialized_start=1773
_globals['_WORKERHEARTBEAT']._serialized_end=2066
_globals['_WORKERFAULT']._serialized_start=2069
_globals['_WORKERFAULT']._serialized_end=2313
_globals['_GATEWAYHELLO']._serialized_end=1041
_globals['_WORKERHELLO']._serialized_start=1043
_globals['_WORKERHELLO']._serialized_end=1148
_globals['_WORKERREADY']._serialized_start=1151
_globals['_WORKERREADY']._serialized_end=1293
_globals['_WORKERCOMMAND']._serialized_start=1295
_globals['_WORKERCOMMAND']._serialized_end=1414
_globals['_WORKERCOMMANDREPLY']._serialized_start=1417
_globals['_WORKERCOMMANDREPLY']._serialized_end=1546
_globals['_WORKERCANCEL']._serialized_start=1548
_globals['_WORKERCANCEL']._serialized_end=1578
_globals['_WORKERSHUTDOWN']._serialized_start=1580
_globals['_WORKERSHUTDOWN']._serialized_end=1661
_globals['_WORKERSHUTDOWNACK']._serialized_start=1663
_globals['_WORKERSHUTDOWNACK']._serialized_end=1735
_globals['_WORKEREVENT']._serialized_start=1737
_globals['_WORKEREVENT']._serialized_end=1795
_globals['_WORKERHEARTBEAT']._serialized_start=1798
_globals['_WORKERHEARTBEAT']._serialized_end=2091
_globals['_WORKERFAULT']._serialized_start=2094
_globals['_WORKERFAULT']._serialized_end=2338
# @@protoc_insertion_point(module_scope)
@@ -5,7 +5,7 @@ from __future__ import annotations
from collections.abc import AsyncIterator, Sequence
from .auth import redact_secret
from .errors import MxGatewayError, ensure_mxaccess_success
from .errors import MalformedReplyError, MxGatewayError, ensure_mxaccess_success
from .events import ReplayGap
from .generated import mxaccess_gateway_pb2 as pb
from .values import MxValueInput, to_mx_value
@@ -710,7 +710,15 @@ class Session:
correlation_id=correlation_id,
secrets=[verify_user_password],
)
return reply.authenticate_user.user_id
if reply.HasField("authenticate_user"):
return reply.authenticate_user.user_id
if reply.HasField("return_value") and reply.return_value.WhichOneof("kind") == "int32_value":
return reply.return_value.int32_value
raise MalformedReplyError(
"authenticate_user returned a malformed reply: OK reply carried "
"neither the typed payload nor an int32 return_value",
raw_reply=reply,
)
async def archestra_user_to_id(
self,
@@ -730,7 +738,15 @@ class Session:
),
correlation_id=correlation_id,
)
return reply.archestra_user_to_id.user_id
if reply.HasField("archestra_user_to_id"):
return reply.archestra_user_to_id.user_id
if reply.HasField("return_value") and reply.return_value.WhichOneof("kind") == "int32_value":
return reply.return_value.int32_value
raise MalformedReplyError(
"archestra_user_to_id returned a malformed reply: OK reply carried "
"neither the typed payload nor an int32 return_value",
raw_reply=reply,
)
async def add_buffered_item(
self,
@@ -752,7 +768,15 @@ class Session:
),
correlation_id=correlation_id,
)
return reply.add_buffered_item.item_handle
if reply.HasField("add_buffered_item"):
return reply.add_buffered_item.item_handle
if reply.HasField("return_value") and reply.return_value.WhichOneof("kind") == "int32_value":
return reply.return_value.int32_value
raise MalformedReplyError(
"add_buffered_item returned a malformed reply: OK reply carried "
"neither the typed payload nor an int32 return_value",
raw_reply=reply,
)
async def set_buffered_update_interval(
self,
@@ -895,19 +919,47 @@ def _value_secrets(value: MxValueInput) -> list[str]:
def _redact_error(error: MxGatewayError, secrets: Sequence[str | None]) -> None:
"""Scrub secret substrings from a raised error's message in place.
"""Scrub secret substrings from a raised error's message and reply in place.
Rewrites ``error.args[0]`` (the message returned by ``str(error)``) through
the shared :func:`~zb_mom_ww_mxgateway.auth.redact_secret` seam so credential
text can never reach logs or be re-raised to a caller. The
``protocol_status`` / ``raw_reply`` context is left untouched those hold the
gateway's own fields, which never echo the client-supplied secret.
text can never reach logs or be re-raised to a caller.
A misbehaving MXAccess provider can echo the client-supplied credential back
verbatim in its failure diagnostics, so ``error.raw_reply`` (the protobuf
reply) can carry the secret in ``protocol_status.message``,
``diagnostic_message``, and each ``statuses[].diagnostic_text``. A logger
dumping those structured fields would reintroduce the leak the message scrub
closes. When there is a secret to scrub and a reply is attached, this rebinds
``error.raw_reply`` to a scrubbed deep copy so the raised exception carries no
credential text on any surface. The clone leaves the original reply untouched.
"""
scrubbed = [secret for secret in secrets if secret]
if not scrubbed:
return
if error.args and isinstance(error.args[0], str):
error.args = (redact_secret(error.args[0], scrubbed), *error.args[1:])
if error.raw_reply is not None:
error.raw_reply = _redact_reply(error.raw_reply, scrubbed)
def _redact_reply(reply: pb.MxCommandReply, secrets: Sequence[str]) -> pb.MxCommandReply:
"""Return a deep copy of *reply* with credential text scrubbed from diagnostics.
Operates on a clone so the caller's original reply object is never mutated.
Only the free-text diagnostic fields that can echo a client-supplied secret
are scrubbed; the structured/enum fields the gateway itself sets are left as-is.
"""
clone = type(reply)()
clone.CopyFrom(reply)
if clone.protocol_status.message:
clone.protocol_status.message = redact_secret(clone.protocol_status.message, secrets)
if clone.diagnostic_message:
clone.diagnostic_message = redact_secret(clone.diagnostic_message, secrets)
for status in clone.statuses:
if status.diagnostic_text:
status.diagnostic_text = redact_secret(status.diagnostic_text, secrets)
return clone
from .client import GatewayClient # noqa: E402
@@ -1,3 +1,3 @@
"""Package version information."""
__version__ = "0.1.2"
__version__ = "0.2.0"
@@ -21,6 +21,7 @@ from zb_mom_ww_mxgateway import __version__
from zb_mom_ww_mxgateway.auth import redact_secret
from zb_mom_ww_mxgateway.client import GatewayClient
from zb_mom_ww_mxgateway.errors import MxGatewayError
from zb_mom_ww_mxgateway.events import ReplayGap
from zb_mom_ww_mxgateway.galaxy import GalaxyRepositoryClient
from zb_mom_ww_mxgateway.generated import galaxy_repository_pb2 as galaxy_pb
from zb_mom_ww_mxgateway.generated import mxaccess_gateway_pb2 as pb
@@ -31,6 +32,11 @@ logger = logging.getLogger(__name__)
MAX_AGGREGATE_EVENTS = 10_000
#: Canonical CLI credential environment variable, shared by every official client
#: CLI (CLI-45) so one exported variable drives the same operator workflow in all
#: five languages.
DEFAULT_VERIFY_PASSWORD_ENV = "MXGATEWAY_VERIFY_PASSWORD"
_BATCH_EOR = "__MXGW_BATCH_EOR__"
@@ -327,7 +333,8 @@ def write_secured(**kwargs: Any) -> None:
)
@click.option(
"--password-env",
default=None,
default=DEFAULT_VERIFY_PASSWORD_ENV,
show_default=True,
help="Environment variable holding the user password.",
)
@click.option("--correlation-id", default="", help="Client correlation id.")
@@ -833,17 +840,23 @@ async def _authenticate_user(**kwargs: Any) -> dict[str, Any]:
def _resolve_password(kwargs: dict[str, Any]) -> str:
"""Resolve the authenticate-user password from --password or --password-env.
Prefers the explicit flag, then falls back to the named environment
variable. The resolved secret is never echoed; callers pass it into the
``secrets`` redaction list so it cannot leak through a surfaced error.
Prefers the explicit flag, then falls back to the environment variable named
by ``--password-env`` (default :data:`DEFAULT_VERIFY_PASSWORD_ENV`). A missing
*or empty* value from either source is a usage error (CLI-45): the CLI never
sends a fabricated empty credential to the wire. The error names the option
and the variable only the resolved secret is never echoed, and callers pass
it into the ``secrets`` redaction list so it cannot leak through a surfaced
error either.
"""
env_name = kwargs.get("password_env") or DEFAULT_VERIFY_PASSWORD_ENV
password = kwargs.get("password")
if not password:
env_name = kwargs.get("password_env")
password = os.environ.get(env_name) if env_name else None
password = os.environ.get(env_name)
if not password:
raise click.UsageError("a password is required via --password or --password-env")
raise click.UsageError(
f"a password is required via --password or the {env_name} environment variable"
)
return password
@@ -1103,7 +1116,7 @@ async def _stream_events(**kwargs: Any) -> dict[str, Any]:
max_events=kwargs["max_events"],
timeout=kwargs["timeout"],
)
return {"events": [_message_dict(event) for event in events]}
return {"events": [_event_row(event) for event in events]}
async def _stream_alarms(**kwargs: Any) -> dict[str, Any]:
@@ -1500,14 +1513,14 @@ async def _collect_events(
*,
max_events: int,
timeout: float,
) -> list[pb.MxEvent]:
) -> list[pb.MxEvent | ReplayGap]:
if max_events > MAX_AGGREGATE_EVENTS:
raise click.BadParameter(
f"must be less than or equal to {MAX_AGGREGATE_EVENTS}",
param_hint="--max-events",
)
collected: list[pb.MxEvent] = []
collected: list[pb.MxEvent | ReplayGap] = []
iterator = events.__aiter__()
try:
while len(collected) < max_events:
@@ -1630,3 +1643,26 @@ def _message_dict(message: Any) -> dict[str, Any]:
preserving_proto_field_name=False,
use_integers_for_enums=False,
)
def _event_row(item: Any) -> dict[str, Any]:
"""Render one item of an event stream as a JSON row.
``Session.stream_events`` yields ``MxEvent | ReplayGap``. ``ReplayGap`` is a
plain dataclass, so it has no protobuf descriptor and cannot go through
``MessageToDict`` it gets its own distinct row instead, matching the shape
the Rust and Go CLIs emit so the cross-language matrix can compare rows.
Keys are camelCase for the same reason ``_message_dict`` uses
``preserving_proto_field_name=False``. The gap is always rendered: never
dropped, and never re-synthesized into an event.
"""
if isinstance(item, ReplayGap):
return {
"replayGap": {
"requestedAfterSequence": item.requested_after_sequence,
"oldestAvailableSequence": item.oldest_available_sequence,
},
}
return _message_dict(item)
+157 -1
View File
@@ -1,6 +1,8 @@
"""Tests for the Python CLI."""
import json
import tomllib
from pathlib import Path
import pytest
from click.testing import CliRunner
@@ -12,6 +14,21 @@ from zb_mom_ww_mxgateway_cli.commands import main
_BATCH_EOR = "__MXGW_BATCH_EOR__"
def test_version_matches_pyproject_toml() -> None:
"""`__version__` must track `pyproject.toml`'s `[project].version`.
The existing `version` command tests only assert self-consistency against
`__version__` (the two hardcoded literals could still drift from each
other without either test catching it the CLI-26 residual drift mode).
This test pins `__version__` to the single source of truth instead.
"""
pyproject_path = Path(__file__).resolve().parent.parent / "pyproject.toml"
with pyproject_path.open("rb") as handle:
pyproject = tomllib.load(handle)
assert __version__ == pyproject["project"]["version"]
def test_require_certificate_validation_flag_flows_through_connect(
monkeypatch: pytest.MonkeyPatch,
) -> None:
@@ -752,7 +769,9 @@ def test_authenticate_user_reads_password_from_env(monkeypatch: pytest.MonkeyPat
assert fake.last_request.command.authenticate_user.verify_user_password == "env-secret-pw"
def test_authenticate_user_requires_a_password() -> None:
def test_authenticate_user_requires_a_password(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.delenv("MXGATEWAY_VERIFY_PASSWORD", raising=False)
result = CliRunner().invoke(
main,
[
@@ -770,6 +789,81 @@ def test_authenticate_user_requires_a_password() -> None:
assert result.exit_code != 0
assert "password is required" in result.output
# CLI-45: the usage error names the option and the canonical env var.
assert "--password" in result.output
assert "MXGATEWAY_VERIFY_PASSWORD" in result.output
def test_authenticate_user_reads_password_from_canonical_default_env(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""CLI-45: --password-env defaults to MXGATEWAY_VERIFY_PASSWORD.
Exporting the canonical variable alone must satisfy the credential, with no
explicit --password-env flag the same operator workflow as the other CLIs.
"""
from zb_mom_ww_mxgateway.generated import mxaccess_gateway_pb2 as pb
reply = pb.MxCommandReply(
session_id="s1",
kind=pb.MX_COMMAND_KIND_AUTHENTICATE_USER,
protocol_status=pb.ProtocolStatus(code=pb.PROTOCOL_STATUS_CODE_OK),
authenticate_user=pb.AuthenticateUserReply(user_id=11),
)
fake = _FakeInvokeClient(reply)
async def fake_connect(options, **_kwargs):
return fake
monkeypatch.setattr(commands_module.GatewayClient, "connect", fake_connect)
monkeypatch.setenv("MXGATEWAY_VERIFY_PASSWORD", "canonical-env-pw")
result = CliRunner().invoke(
main,
[
"authenticate-user",
"--plaintext",
"--session-id",
"s1",
"--server-handle",
"3",
"--verify-user",
"operator",
"--json",
],
)
assert result.exit_code == 0, result.output
assert json.loads(result.output)["userId"] == 11
assert "canonical-env-pw" not in result.output
assert fake.last_request.command.authenticate_user.verify_user_password == "canonical-env-pw"
def test_authenticate_user_rejects_empty_password_value(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""CLI-45: an empty resolved credential fails fast, never reaching the wire."""
monkeypatch.setenv("MXGATEWAY_VERIFY_PASSWORD", "")
result = CliRunner().invoke(
main,
[
"authenticate-user",
"--plaintext",
"--session-id",
"s1",
"--server-handle",
"3",
"--verify-user",
"operator",
"--password",
"",
"--json",
],
)
assert result.exit_code != 0
assert "password is required" in result.output
def test_write_secured_command_does_not_echo_value_on_failure(
@@ -817,3 +911,65 @@ def test_write_secured_command_does_not_echo_value_on_failure(
def test_write_secured_and_authenticate_user_commands_are_registered() -> None:
names = set(main.commands)
assert {"write-secured", "authenticate-user"} <= names
class _FakeReplayGapSession:
"""Session stand-in whose event stream starts with a ReplayGap sentinel.
Mirrors what ``Session.stream_events`` yields on a resume that predates the
gateway's retained replay ring: the typed gap first, then normal events.
"""
def __init__(self, gap, event) -> None:
self._gap = gap
self._event = event
def stream_events(self, **_kwargs):
async def _iterate():
yield self._gap
yield self._event
return _iterate()
def test_stream_events_renders_replay_gap(monkeypatch: pytest.MonkeyPatch) -> None:
"""CLI-35: a ReplayGap renders as its own JSON row instead of crashing the command."""
from zb_mom_ww_mxgateway.events import ReplayGap
from zb_mom_ww_mxgateway.generated import mxaccess_gateway_pb2 as pb
gap = ReplayGap(requested_after_sequence=7, oldest_available_sequence=42)
event = pb.MxEvent(session_id="cli-test-session", worker_sequence=43)
async def fake_connect(options, **_kwargs):
return _FakeAsyncClient()
monkeypatch.setattr(commands_module.GatewayClient, "connect", fake_connect)
monkeypatch.setattr(
commands_module,
"_session",
lambda _client, _session_id: _FakeReplayGapSession(gap, event),
)
result = CliRunner().invoke(
main,
[
"stream-events",
"--plaintext",
"--session-id",
"cli-test-session",
"--after-worker-sequence",
"7",
"--max-events",
"2",
"--json",
],
)
assert result.exit_code == 0, result.output
rows = json.loads(result.output)["events"]
assert rows[0] == {
"replayGap": {"requestedAfterSequence": 7, "oldestAvailableSequence": 42},
}
# The gap is rendered, never swallowed, and the normal event still follows it.
assert "replayGap" not in rows[1]
assert rows[1]["workerSequence"] == "43"
+49
View File
@@ -32,6 +32,55 @@ def test_write_failure_fixture_preserves_raw_reply() -> None:
assert len(captured.value.raw_reply.statuses) == 2
@pytest.mark.parametrize(
("fixture", "expect_failure"),
[
("command-replies/register.ok.reply.json", False),
("command-replies/write.status-category-error-success-set.reply.json", True),
("command-replies/write.status-category-ok-success-zero.reply.json", False),
("command-replies/write.hresult-s-false.reply.json", False),
("command-replies/write.hresult-e-fail.reply.json", True),
],
)
def test_reply_validation_fixtures_branch_on_category_and_negative_hresult(
fixture: str,
expect_failure: bool,
) -> None:
reply = _load_reply(fixture)
if expect_failure:
with pytest.raises(MxAccessError):
ensure_mxaccess_success("write", reply)
else:
assert ensure_mxaccess_success("write", reply) is reply
@pytest.mark.parametrize(
("category", "success", "expect_failure"),
[
(pb.MX_STATUS_CATEGORY_OK, 0, False),
(pb.MX_STATUS_CATEGORY_OK, 1, False),
(pb.MX_STATUS_CATEGORY_COMMUNICATION_ERROR, 1, True),
(pb.MX_STATUS_CATEGORY_UNSPECIFIED, 1, True),
],
)
def test_status_entry_verdict_ignores_the_raw_success_member(
category: int,
success: int,
expect_failure: bool,
) -> None:
reply = pb.MxCommandReply(
protocol_status=pb.ProtocolStatus(code=pb.PROTOCOL_STATUS_CODE_OK),
statuses=[pb.MxStatusProxy(success=success, category=category)],
)
if expect_failure:
with pytest.raises(MxAccessError):
ensure_mxaccess_success("write", reply)
else:
assert ensure_mxaccess_success("write", reply) is reply
def test_session_status_maps_to_session_error() -> None:
status = pb.ProtocolStatus(
code=pb.PROTOCOL_STATUS_CODE_SESSION_NOT_FOUND,
@@ -0,0 +1,112 @@
"""Tests for the uniform malformed-reply contract (CLI-41) and the CLI-40
credential-redaction regression, driven through the shared fixtures.
CLI-41: an OK reply that carries neither the expected typed payload nor a usable
``return_value`` int32 fallback raises :class:`MalformedReplyError`; a legacy
reply that populates only ``return_value`` falls back to that int32.
CLI-40: an OK reply whose diagnostics echo the caller's credential must never
surface that credential in the raised error message.
"""
from __future__ import annotations
import json
from pathlib import Path
import pytest
from google.protobuf.json_format import ParseDict
from zb_mom_ww_mxgateway import MalformedReplyError, MxAccessError
from zb_mom_ww_mxgateway.generated import mxaccess_gateway_pb2 as pb
from test_typed_command_helpers import _session_with
FIXTURE_ROOT = Path(__file__).resolve().parents[2] / "proto" / "fixtures" / "behavior"
def _load_reply(relative: str) -> pb.MxCommandReply:
path = FIXTURE_ROOT / relative
return ParseDict(json.loads(path.read_text()), pb.MxCommandReply())
@pytest.mark.asyncio
async def test_authenticate_user_missing_payload_raises_malformed_reply() -> None:
reply = _load_reply("command-replies/authenticate-user.missing-payload.reply.json")
session, _ = await _session_with([reply])
with pytest.raises(MalformedReplyError) as captured:
await session.authenticate_user(12, "operator", "any-password")
assert captured.value.raw_reply is reply
assert "malformed reply" in str(captured.value)
@pytest.mark.asyncio
async def test_authenticate_user_return_value_only_falls_back_to_int32() -> None:
reply = _load_reply("command-replies/authenticate-user.return-value-only.reply.json")
session, _ = await _session_with([reply])
user_id = await session.authenticate_user(12, "operator", "any-password")
assert user_id == 7
@pytest.mark.asyncio
async def test_add_buffered_item_falls_back_to_return_value_int32() -> None:
reply = pb.MxCommandReply(
session_id="session-1",
kind=pb.MX_COMMAND_KIND_ADD_BUFFERED_ITEM,
protocol_status=pb.ProtocolStatus(code=pb.PROTOCOL_STATUS_CODE_OK),
return_value=pb.MxValue(int32_value=99),
)
session, _ = await _session_with([reply])
item_handle = await session.add_buffered_item(12, "Object.Attribute", "ctx")
assert item_handle == 99
@pytest.mark.asyncio
async def test_add_buffered_item_missing_payload_raises_malformed_reply() -> None:
reply = pb.MxCommandReply(
session_id="session-1",
kind=pb.MX_COMMAND_KIND_ADD_BUFFERED_ITEM,
protocol_status=pb.ProtocolStatus(code=pb.PROTOCOL_STATUS_CODE_OK),
)
session, _ = await _session_with([reply])
with pytest.raises(MalformedReplyError) as captured:
await session.add_buffered_item(12, "Object.Attribute", "ctx")
assert captured.value.raw_reply is reply
@pytest.mark.parametrize(
"fixture",
[
"command-replies/authenticate-user.echoed-credential.reply.json",
"command-replies/authenticate-user.echoed-credential-mxaccess-failure.reply.json",
],
)
@pytest.mark.asyncio
async def test_authenticate_user_echoed_credential_is_scrubbed(fixture: str) -> None:
credential = "sup3rSecretVerify9f3a2b"
reply = _load_reply(fixture)
session, _ = await _session_with([reply])
with pytest.raises(MxAccessError) as captured:
await session.authenticate_user(12, "operator", credential)
exc = captured.value
message = str(exc)
assert credential not in message
assert "[redacted]" in message
# The credential must not survive in the structured protobuf context either:
# a logger dumping raw_reply's fields would otherwise reintroduce the leak.
assert exc.raw_reply is not None
assert credential not in exc.raw_reply.protocol_status.message
assert credential not in exc.raw_reply.diagnostic_message
for status in exc.raw_reply.statuses:
assert credential not in status.diagnostic_text
@@ -140,10 +140,19 @@ async def test_write_secured_surfaces_native_failure_without_prior_authenticate(
with pytest.raises(MxAccessError) as captured:
await session.write_secured(12, 34, secret_value, current_user_id=5, verifier_user_id=6)
# Native failure is surfaced (not "fixed") and the raw reply is preserved...
assert captured.value.raw_reply is failure
# ...but the credential-sensitive value is scrubbed from the surfaced message.
# Native failure is surfaced (not "fixed"): the raw reply's structure is
# preserved so callers still see the native verdict...
raw = captured.value.raw_reply
assert raw is not None
assert raw.kind == pb.MX_COMMAND_KIND_WRITE_SECURED
assert raw.hresult == -2147217407
assert raw.protocol_status.code == pb.PROTOCOL_STATUS_CODE_MXACCESS_FAILURE
# ...but the credential-sensitive value is scrubbed from the surfaced message
# AND from the reply's echoed diagnostics, so a logger dumping raw_reply's
# structured fields cannot reintroduce the leak.
assert secret_value not in str(captured.value)
assert secret_value not in raw.protocol_status.message
assert "[redacted]" in raw.protocol_status.message
command = stub.invoke.requests[0].command
assert command.kind == pb.MX_COMMAND_KIND_WRITE_SECURED
assert command.write_secured.current_user_id == 5
+2 -2
View File
@@ -590,7 +590,7 @@ checksum = "1d87ecb2933e8aeadb3e3a02b828fed80a7528047e68b4f424523a0981a3a084"
[[package]]
name = "mxgw-cli"
version = "0.1.2"
version = "0.2.0"
dependencies = [
"clap",
"futures-util",
@@ -1490,7 +1490,7 @@ dependencies = [
[[package]]
name = "zb-mom-ww-mxgateway-client"
version = "0.1.2"
version = "0.2.0"
dependencies = [
"futures-core",
"futures-util",
+2 -2
View File
@@ -1,6 +1,6 @@
[package]
name = "zb-mom-ww-mxgateway-client"
version = "0.1.2"
version = "0.2.0"
edition = "2021"
authors = ["Joseph Doherty"]
description = "Async Rust client for the MxAccessGateway gRPC service, including a lazy-browse walker over the Galaxy Repository hierarchy."
@@ -25,7 +25,7 @@ resolver = "2"
[workspace.package]
edition = "2021"
version = "0.1.2"
version = "0.2.0"
authors = ["Joseph Doherty"]
license = "Proprietary"
repository = "https://gitea.dohertylan.com/dohertj2/mxaccessgw"
+23 -5
View File
@@ -18,9 +18,22 @@ clients/rust/
crates/mxgw-cli/
```
`build.rs` reads the `.proto` files from
`../../src/ZB.MOM.WW.MxGateway.Contracts/Protos` and generates `tonic`/`prost` bindings
into Cargo build output. `src/generated.rs` declares the Rust modules that
`build.rs` resolves the `.proto` inputs repo-path-first: it prefers the
canonical protos at `../../src/ZB.MOM.WW.MxGateway.Contracts/Protos` (two
levels above `clients/rust`) so a local in-repo `.proto` edit is picked up
live without any extra step, and falls back to the vendored copies checked
into `clients/rust/protos/` only when that canonical directory is absent —
the case for a consumer building the crate unpacked from a published
tarball, where the rest of the mxaccessgw repo does not exist. The vendored
copies are shipped in the published `.crate` via `Cargo.toml`'s `include`
list, which is what makes the crate buildable standalone; they are build
inputs only, never a second source of truth. **Refresh rule:** any commit
that edits a Contracts proto (`mxaccess_gateway.proto`, `mxaccess_worker.proto`,
`galaxy_repository.proto`) must copy the changed file(s) into
`clients/rust/protos/` in that same commit — `scripts/check-codegen.ps1`
Check 3 fails the build on byte drift between the vendored copies and the
canonical Contracts protos. `tonic`/`prost` bindings are generated into
Cargo build output. `src/generated.rs` declares the Rust modules that
include those generated files. `src/generated` remains reserved for checked-in
generator output if the crate later changes to source-tree generation.
@@ -216,7 +229,12 @@ the wire — the client never logs them and never embeds them in an `Error`'s
`Display`/`Debug`; the only error text that can surface (from `tonic::Status`
messages and reply diagnostics) is scrubbed by the credential-redaction seam.
The CLI mirrors these as `authenticate-user` (password via `--password` or the
`--password-env` env var, never echoed) and `write-secured`.
variable named by `--password-env`, default `MXGATEWAY_VERIFY_PASSWORD`, never
echoed) and `write-secured`. The credential is required: a missing or empty
resolved value is a usage error naming the flag and the variable, so the CLI
fails before dialing instead of authenticating with an empty password.
`MXGATEWAY_VERIFY_PASSWORD` is the canonical variable across all five client
CLIs — see [Cross-Language Smoke Matrix](../../docs/CrossLanguageSmokeMatrix.md).
The remaining single-item command helpers round out MXAccess parity:
`unregister`, `suspend` / `activate` (each returns the operation's
@@ -418,5 +436,5 @@ Then add the dependency:
```toml
[dependencies]
zb-mom-ww-mxgateway-client = { version = "0.1.1", registry = "dohertj2-gitea" }
zb-mom-ww-mxgateway-client = { version = "0.2.0", registry = "dohertj2-gitea" }
```
+86 -11
View File
@@ -752,17 +752,7 @@ async fn dispatch(command: Command) -> Result<(), Error> {
password_env,
json,
} => {
// Resolve the credential from --password or the named env var.
// The password is passed straight to the typed helper and is never
// echoed to stdout/stderr or embedded in an error message.
let verify_user_password = password
.or_else(|| env::var(&password_env).ok())
.ok_or_else(|| Error::InvalidArgument {
name: "password".to_owned(),
detail: format!(
"supply --password or set the environment variable `{password_env}`"
),
})?;
let verify_user_password = resolve_verify_user_password(password, &password_env)?;
let session = session_for(connection, session_id).await?;
let user_id = session
.authenticate_user(server_handle, &verify_user, &verify_user_password)
@@ -1736,6 +1726,31 @@ fn print_ok(operation: &str, use_json: bool) {
}
}
/// Resolves the `authenticate-user` credential from `--password`, falling back to
/// the environment variable named by `--password-env` (default
/// `MXGATEWAY_VERIFY_PASSWORD`).
///
/// An empty value from either source counts as missing (CLI-45): the CLI fails
/// fast with a usage error rather than sending a fabricated empty credential to
/// the wire. The error names the flag and the variable only — never the value,
/// which is never echoed to stdout/stderr or embedded in an error message.
fn resolve_verify_user_password(
password: Option<String>,
password_env: &str,
) -> Result<String, Error> {
password
.filter(|value| !value.is_empty())
.or_else(|| {
env::var(password_env)
.ok()
.filter(|value| !value.is_empty())
})
.ok_or_else(|| Error::InvalidArgument {
name: "password".to_owned(),
detail: format!("supply --password or set the environment variable `{password_env}`"),
})
}
fn print_bulk_results(
operation: &str,
results: &[zb_mom_ww_mxgateway_client::generated::mxaccess_gateway::v1::SubscribeResult],
@@ -2617,6 +2632,66 @@ mod tests {
assert!(parsed.is_ok(), "parse failed: {parsed:?}");
}
/// CLI-45: `--password-env` must default to the canonical
/// `MXGATEWAY_VERIFY_PASSWORD` shared by every official client CLI.
#[test]
fn authenticate_user_password_env_defaults_to_canonical_name() {
let parsed = Cli::try_parse_from([
"mxgw",
"authenticate-user",
"--session-id",
"session-1",
"--server-handle",
"7",
"--verify-user",
"verifier",
])
.expect("parse");
match parsed.command {
Command::AuthenticateUser { password_env, .. } => {
assert_eq!(password_env, "MXGATEWAY_VERIFY_PASSWORD");
}
other => panic!("expected authenticate-user, got {other:?}"),
}
}
/// CLI-45: a credential that resolves to an empty string — whether from the
/// flag or from the named environment variable — is treated as missing, so
/// the CLI never sends a fabricated empty password to the wire. The usage
/// error names the flag and the variable, never a value.
#[test]
fn resolve_verify_user_password_rejects_missing_and_empty_values() {
const ABSENT: &str = "MXGW_CLI45_ABSENT_PASSWORD_VAR";
const EMPTY: &str = "MXGW_CLI45_EMPTY_PASSWORD_VAR";
const PRESENT: &str = "MXGW_CLI45_PRESENT_PASSWORD_VAR";
std::env::remove_var(ABSENT);
std::env::set_var(EMPTY, "");
std::env::set_var(PRESENT, "env-sourced-credential");
for (password, env_name) in [
(None, ABSENT),
(Some(String::new()), ABSENT),
(None, EMPTY),
(Some(String::new()), EMPTY),
] {
let error = super::resolve_verify_user_password(password, env_name)
.expect_err("empty or missing credential must be a usage error");
let rendered = error.to_string();
assert!(rendered.contains("--password"), "{rendered}");
assert!(rendered.contains(env_name), "{rendered}");
}
assert_eq!(
super::resolve_verify_user_password(None, PRESENT).expect("env credential"),
"env-sourced-credential"
);
assert_eq!(
super::resolve_verify_user_password(Some("flag-credential".to_owned()), EMPTY)
.expect("flag credential"),
"flag-credential"
);
}
#[test]
fn parses_write_secured_command() {
let parsed = Cli::try_parse_from([
@@ -676,6 +676,10 @@ message WorkerInfoReply {
}
message DrainEventsReply {
// The reply is bounded by both a server-side count cap and the negotiated
// worker-frame byte cap; a reply may therefore carry fewer events than
// `max_events` and fewer than are queued. Callers drain iteratively until an
// empty reply.
repeated MxEvent events = 1;
}
@@ -760,6 +764,11 @@ message ReplayGap {
// after_worker_sequence = oldest_available_sequence - 1 in the next
// StreamEventsRequest, which will cause the server to replay starting at
// oldest_available_sequence (the first retained event).
// When nothing is retained (the replay ring is empty), this is the next sequence
// that can be delivered `highest observed + 1` and the `oldest - 1` resume
// formula remains valid: it resolves to the highest sequence already seen, so the
// follow-up resume replays nothing, reports no gap, and every newer live event
// passes. The interval evicted is unchanged.
uint64 oldest_available_sequence = 2;
}
@@ -47,6 +47,9 @@ message GatewayHello {
// instead of a hard-coded default; 0 (an older gateway that never set the field) means
// "use the worker's built-in default". Sits above the public gRPC cap by an
// envelope-overhead margin so an accepted gRPC payload always fits one worker frame.
// Every worker->gateway frame events, heartbeats, faults, and control replies
// including DrainEvents must serialize within this limit; reply builders truncate
// to fit rather than emit an oversized frame.
uint32 max_frame_bytes = 4;
}
+111 -23
View File
@@ -193,17 +193,43 @@ impl std::error::Error for CommandError {}
/// The wrapper is heap-allocated inside [`Error::MxAccess`] to keep the
/// containing enum small. Callers can recover the reply with
/// [`MxAccessError::reply`] or [`MxAccessError::into_reply`]. Its `Display`
/// summarizes the `hresult` and status entries and scrubs any credential-like
/// tokens from diagnostic text before it reaches a caller.
#[derive(Clone, Debug)]
/// summarizes the `hresult` and status entries and scrubs credentials from the
/// rendered text before it reaches a caller: credential-*shaped* tokens
/// (`mxgw_...`, `bearer`) via a pattern scrub, plus any exact caller-supplied
/// secrets registered with [`MxAccessError::with_secrets`] — the latter catches
/// a password MXAccess echoed back verbatim even though it has no token shape.
///
/// `Debug` is hand-written (not derived) so the attached exact secrets never
/// reach `{:?}` output either: it scrubs them from the reply rendering and
/// prints only the count of attached secrets, never their values.
#[derive(Clone)]
pub struct MxAccessError {
reply: MxCommandReply,
/// Exact caller-supplied secrets (e.g. an `AuthenticateUser` password or a
/// `WriteSecured` string value) scrubbed from the rendered message. Empty
/// unless a helper attaches them via [`Self::with_secrets`].
secrets: Vec<String>,
}
impl MxAccessError {
/// Wrap a reply whose MXAccess-level result reported a failure.
pub fn new(reply: MxCommandReply) -> Self {
Self { reply }
Self {
reply,
secrets: Vec::new(),
}
}
/// Register exact caller-supplied secrets to scrub from the rendered
/// message, returning the updated error.
///
/// A credential MXAccess echoes back into its diagnostic text has no
/// `mxgw_`/`bearer` shape, so the pattern scrub cannot catch it. Attaching
/// the exact secret lets `Display` replace every occurrence with
/// `<redacted>`.
pub fn with_secrets(mut self, secrets: Vec<String>) -> Self {
self.secrets = secrets;
self
}
/// Borrow the underlying reply (correlation id, hresult, statuses).
@@ -217,15 +243,43 @@ impl MxAccessError {
}
}
impl std::fmt::Debug for MxAccessError {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
// Render the reply, scrub any exact caller secret from it, and never
// print the raw secrets themselves — only how many are attached.
let mut reply = format!("{:?}", self.reply);
for secret in &self.secrets {
if !secret.is_empty() {
reply = reply.replace(secret.as_str(), "<redacted>");
}
}
formatter
.debug_struct("MxAccessError")
.field("reply", &format_args!("{reply}"))
.field(
"secrets",
&format_args!("[{} redacted]", self.secrets.len()),
)
.finish()
}
}
impl std::fmt::Display for MxAccessError {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
use std::fmt::Write as _;
let hresult = match self.reply.hresult {
Some(value) => value.to_string(),
None => "none".to_owned(),
};
// Render the whole body first so the exact-secret scrub can sweep every
// field — including diagnostic text that already went through the
// credential-shape scrub — before any of it reaches the caller.
let mut body = String::new();
write!(
formatter,
body,
"hresult={hresult}, {} status entr{}",
self.reply.statuses.len(),
if self.reply.statuses.len() == 1 {
@@ -233,20 +287,28 @@ impl std::fmt::Display for MxAccessError {
} else {
"ies"
}
)?;
)
.expect("writing to a String is infallible");
for status in &self.reply.statuses {
let category = MxStatusCategory::try_from(status.category)
.unwrap_or(MxStatusCategory::Unspecified);
let diagnostic = redact_credentials(&status.diagnostic_text);
write!(
formatter,
body,
"; [success={}, category={category:?}, detail={}, {}]",
status.success, status.detail, diagnostic
)?;
)
.expect("writing to a String is infallible");
}
Ok(())
for secret in &self.secrets {
if !secret.is_empty() {
body = body.replace(secret.as_str(), "<redacted>");
}
}
formatter.write_str(&body)
}
}
@@ -284,10 +346,17 @@ impl From<tonic::Status> for Error {
/// Promote a non-OK protocol status carried inside an [`MxCommandReply`]
/// to an [`Error::Command`].
///
/// [`ProtocolStatusCode::MxaccessFailure`] is deliberately **not** a
/// command-level failure here: it signals an MXAccess-level rejection, so it
/// falls through to [`ensure_mxaccess_success`] and surfaces as
/// [`Error::MxAccess`] — matching the .NET, Java, Go, and Python clients. Every
/// other non-`Ok` code stays [`Error::Command`].
///
/// # Errors
///
/// Returns [`Error::Command`] when `reply.protocol_status` is missing or
/// reports any code other than [`ProtocolStatusCode::Ok`].
/// reports any code other than [`ProtocolStatusCode::Ok`] or
/// [`ProtocolStatusCode::MxaccessFailure`].
pub fn ensure_command_success(reply: MxCommandReply) -> Result<MxCommandReply, Error> {
let code = reply
.protocol_status
@@ -295,7 +364,7 @@ pub fn ensure_command_success(reply: MxCommandReply) -> Result<MxCommandReply, E
.and_then(|status| ProtocolStatusCode::try_from(status.code).ok())
.unwrap_or(ProtocolStatusCode::Unspecified);
if code == ProtocolStatusCode::Ok {
if code == ProtocolStatusCode::Ok || code == ProtocolStatusCode::MxaccessFailure {
Ok(reply)
} else {
Err(Box::new(CommandError::new(reply)).into())
@@ -306,12 +375,17 @@ pub fn ensure_command_success(reply: MxCommandReply) -> Result<MxCommandReply, E
/// [`MxCommandReply`] to an [`Error::MxAccess`].
///
/// This is the second reply check applied to the typed command path, after
/// [`ensure_command_success`] confirms the protocol envelope is `Ok`. It
/// enforces MXAccess parity: a reply can carry an `Ok` protocol envelope while
/// MXAccess itself rejected the operation. Following COM semantics (and the
/// Python client), only a **negative** `hresult` is a failure — positive codes
/// such as `S_FALSE = 1` are success. A `MXSTATUS_PROXY` entry is treated as a
/// failure when its `success` member is `0`.
/// [`ensure_command_success`] confirms the protocol envelope is `Ok` (or a
/// [`ProtocolStatusCode::MxaccessFailure`] the first check lets fall through).
/// It enforces MXAccess parity: a reply can carry an `Ok` protocol envelope
/// while MXAccess itself rejected the operation, and a
/// [`ProtocolStatusCode::MxaccessFailure`] envelope is itself an MXAccess-level
/// failure regardless of `hresult`. Following COM semantics, only a
/// **negative** `hresult` is a failure — positive codes such as `S_FALSE = 1`
/// are success. A `MXSTATUS_PROXY` entry is treated as a failure when its
/// `category` is not [`MxStatusCategory::Ok`]; the `success` member mirrors the
/// raw COM value verbatim for diagnostics and never enters the verdict, so an
/// entry with an unspecified category fails even when `success` is non-zero.
///
/// Per-item bulk failures are reported inside each result entry
/// (`was_successful = false`) rather than in the top-level `hresult`/`statuses`
@@ -319,13 +393,24 @@ pub fn ensure_command_success(reply: MxCommandReply) -> Result<MxCommandReply, E
///
/// # Errors
///
/// Returns [`Error::MxAccess`] when `reply.hresult` is negative or any
/// `reply.statuses` entry reports a non-success `success` member.
/// Returns [`Error::MxAccess`] when the reply's protocol code is
/// [`ProtocolStatusCode::MxaccessFailure`], `reply.hresult` is negative, or any
/// `reply.statuses` entry reports a category other than
/// [`MxStatusCategory::Ok`].
pub fn ensure_mxaccess_success(reply: MxCommandReply) -> Result<MxCommandReply, Error> {
let protocol_code = reply
.protocol_status
.as_ref()
.and_then(|status| ProtocolStatusCode::try_from(status.code).ok())
.unwrap_or(ProtocolStatusCode::Unspecified);
let mxaccess_failure = protocol_code == ProtocolStatusCode::MxaccessFailure;
let hresult_failure = reply.hresult.is_some_and(|hresult| hresult < 0);
let status_failure = reply.statuses.iter().any(|status| status.success == 0);
let status_failure = reply
.statuses
.iter()
.any(|status| status.category != MxStatusCategory::Ok as i32);
if hresult_failure || status_failure {
if mxaccess_failure || hresult_failure || status_failure {
Err(Box::new(MxAccessError::new(reply)).into())
} else {
Ok(reply)
@@ -412,8 +497,10 @@ mod tests {
let mut reply = ok_reply();
// Positive hresult (e.g. S_FALSE = 1) is a success, not a failure.
reply.hresult = Some(1);
// A zero `success` member with an OK category is still a success: the
// category is authoritative and `success` is diagnostics only.
reply.statuses = vec![MxStatusProxy {
success: 1,
success: 0,
category: MxStatusCategory::Ok as i32,
..MxStatusProxy::default()
}];
@@ -424,8 +511,9 @@ mod tests {
#[test]
fn ensure_mxaccess_success_flags_failing_status_entry() {
let mut reply = ok_reply();
// A non-OK category fails even though the raw `success` member is set.
reply.statuses = vec![MxStatusProxy {
success: 0,
success: 1,
category: MxStatusCategory::CommunicationError as i32,
detail: 42,
diagnostic_text: "write rejected for mxgw_visible_secret".to_owned(),
+79 -11
View File
@@ -11,7 +11,7 @@
use std::sync::atomic::{AtomicU64, Ordering};
use crate::client::{EventStream, GatewayClient};
use crate::error::{ensure_protocol_success, Error};
use crate::error::{ensure_protocol_success, Error, MxAccessError};
use crate::generated::mxaccess_gateway::v1::mx_command::Payload;
use crate::generated::mxaccess_gateway::v1::mx_command_reply;
use crate::generated::mxaccess_gateway::v1::{
@@ -27,7 +27,7 @@ use crate::generated::mxaccess_gateway::v1::{
WriteSecured2BulkCommand, WriteSecured2BulkEntry, WriteSecured2Command,
WriteSecuredBulkCommand, WriteSecuredBulkEntry, WriteSecuredCommand,
};
use crate::value::{MxStatus, MxValue};
use crate::value::{MxStatus, MxValue, MxValueProjection};
const MAX_BULK_ITEMS: usize = 1_000;
@@ -801,6 +801,7 @@ impl Session {
verifier_user_id: i32,
value: MxValue,
) -> Result<(), Error> {
let secrets = string_secret(&value);
self.invoke(
MxCommandKind::WriteSecured,
Payload::WriteSecured(WriteSecuredCommand {
@@ -811,7 +812,8 @@ impl Session {
value: Some(value.into_proto()),
}),
)
.await?;
.await
.map_err(|error| attach_secrets(error, secrets))?;
Ok(())
}
@@ -831,6 +833,7 @@ impl Session {
value: MxValue,
timestamp_value: MxValue,
) -> Result<(), Error> {
let secrets = string_secret(&value);
self.invoke(
MxCommandKind::WriteSecured2,
Payload::WriteSecured2(WriteSecured2Command {
@@ -842,7 +845,8 @@ impl Session {
timestamp_value: Some(timestamp_value.into_proto()),
}),
)
.await?;
.await
.map_err(|error| attach_secrets(error, secrets))?;
Ok(())
}
@@ -882,7 +886,8 @@ impl Session {
verify_user_password: verify_user_password.to_owned(),
}),
)
.await?;
.await
.map_err(|error| attach_secrets(error, vec![verify_user_password.to_owned()]))?;
authenticate_user_id(&reply)
}
@@ -1074,18 +1079,81 @@ fn add_buffered_item_handle(reply: &MxCommandReply) -> Result<i32, Error> {
fn authenticate_user_id(reply: &MxCommandReply) -> Result<i32, Error> {
match reply.payload.as_ref() {
Some(mx_command_reply::Payload::AuthenticateUser(authenticate)) => Ok(authenticate.user_id),
_ => Err(Error::MalformedReply {
detail: "authenticate_user reply lacked an AuthenticateUser payload".to_owned(),
}),
_ => reply
.return_value
.as_ref()
.and_then(int32_reply_value)
.ok_or_else(|| Error::MalformedReply {
detail:
"authenticate_user reply lacked an AuthenticateUser payload or int32 return_value"
.to_owned(),
}),
}
}
fn archestra_user_id(reply: &MxCommandReply) -> Result<i32, Error> {
match reply.payload.as_ref() {
Some(mx_command_reply::Payload::ArchestraUserToId(archestra)) => Ok(archestra.user_id),
_ => Err(Error::MalformedReply {
detail: "archestra_user_to_id reply lacked an ArchestraUserToId payload".to_owned(),
}),
_ => reply
.return_value
.as_ref()
.and_then(int32_reply_value)
.ok_or_else(|| Error::MalformedReply {
detail:
"archestra_user_to_id reply lacked an ArchestraUserToId payload or int32 return_value"
.to_owned(),
}),
}
}
/// Extract an exact string secret from a credential-sensitive [`MxValue`] so a
/// failing `WriteSecured`/`WriteSecured2` can scrub it from the surfaced error.
/// Non-string values carry no scrubbable secret and yield an empty vector.
fn string_secret(value: &MxValue) -> Vec<String> {
match value.projection() {
MxValueProjection::String(text) if !text.is_empty() => vec![text.clone()],
_ => Vec::new(),
}
}
/// Attach caller-supplied exact secrets to an [`Error::MxAccess`] before it
/// propagates. This both scrubs the stored reply's caller-readable string
/// fields (so `reply()`/`into_reply()` cannot recover a credential MXAccess
/// echoed back verbatim) and keeps the secrets on the error as a
/// belt-and-suspenders for `Display`/`Debug`. Any other error variant is
/// returned unchanged.
fn attach_secrets(error: Error, secrets: Vec<String>) -> Error {
match error {
Error::MxAccess(boxed) => {
let mut reply = boxed.into_reply();
scrub_reply_strings(&mut reply, &secrets);
Error::MxAccess(Box::new(MxAccessError::new(reply).with_secrets(secrets)))
}
other => other,
}
}
/// Replace every non-empty secret occurrence with `<redacted>` in the reply's
/// caller-readable string fields — `protocol_status.message`,
/// `diagnostic_message`, and each `statuses[i].diagnostic_text`. A caller
/// reading the structured reply back off an [`Error::MxAccess`] would otherwise
/// reintroduce the leak that `Display`/`Debug` already close.
fn scrub_reply_strings(reply: &mut MxCommandReply, secrets: &[String]) {
for secret in secrets {
if secret.is_empty() {
continue;
}
if let Some(status) = reply.protocol_status.as_mut() {
status.message = status.message.replace(secret.as_str(), "<redacted>");
}
reply.diagnostic_message = reply
.diagnostic_message
.replace(secret.as_str(), "<redacted>");
for status in &mut reply.statuses {
status.diagnostic_text = status
.diagnostic_text
.replace(secret.as_str(), "<redacted>");
}
}
}
+5 -1
View File
@@ -282,7 +282,11 @@ impl MxStatus {
&self.raw
}
/// `MXSTATUS_PROXY.Success` flag (0 = error, non-zero = good/warning).
/// Raw `MXSTATUS_PROXY.Success` member, carried verbatim from COM.
///
/// This is a diagnostic value, not a verdict: the wire contract makes
/// [`Self::category`] authoritative, and `ensure_mxaccess_success` branches
/// on the category alone.
pub fn success(&self) -> i32 {
self.raw.success
}
+314 -2
View File
@@ -14,6 +14,7 @@ use tokio::sync::{mpsc, Mutex};
use tokio_stream::wrappers::{ReceiverStream, TcpListenerStream};
use tonic::transport::Server;
use tonic::{Request, Response, Status};
use zb_mom_ww_mxgateway_client::error::ensure_mxaccess_success;
use zb_mom_ww_mxgateway_client::generated::mxaccess_gateway::v1::mx_access_gateway_server::{
MxAccessGateway, MxAccessGatewayServer,
};
@@ -83,8 +84,10 @@ async fn session_helpers_build_commands_and_preserve_command_errors() {
.write(12, 34, ClientMxValue::int32(123), 0)
.await
.unwrap_err();
let Error::Command(error) = error else {
panic!("write failure should preserve the raw command reply: {error:?}");
// A MXACCESS_FAILURE-coded reply is an MXAccess-level failure, routed to
// Error::MxAccess (matching .NET/Java/Go/Python) rather than Error::Command.
let Error::MxAccess(error) = error else {
panic!("MXACCESS_FAILURE reply should route to Error::MxAccess: {error:?}");
};
assert_eq!(
error.reply().protocol_status.as_ref().unwrap().code,
@@ -337,6 +340,57 @@ fn authentication_and_authorization_statuses_are_distinct_and_redacted() {
assert!(!auth.to_string().contains("visible_secret"));
}
#[test]
fn command_reply_validation_fixtures_branch_on_category_and_negative_hresult() {
// The shared behavior fixtures pin both reply-validation rules: a status
// entry fails iff its category is not OK (the raw `success` member is
// diagnostics only) and an HRESULT fails iff it is present and negative.
for (fixture, expect_failure) in [
("register.ok.reply.json", false),
("write.status-category-error-success-set.reply.json", true),
("write.status-category-ok-success-zero.reply.json", false),
("write.hresult-s-false.reply.json", false),
("write.hresult-e-fail.reply.json", true),
] {
let reply = command_reply_fixture(fixture);
let result = ensure_mxaccess_success(reply);
assert_eq!(
result.is_err(),
expect_failure,
"fixture {fixture} expected failure = {expect_failure}, got {result:?}"
);
}
}
#[test]
fn status_entry_verdict_ignores_the_raw_success_member() {
// Edges the fixtures cannot express: an OK category always passes and an
// unspecified category always fails, whatever `success` carries.
for (category, success, expect_failure) in [
(MxStatusCategory::Ok, 0, false),
(MxStatusCategory::Ok, 1, false),
(MxStatusCategory::CommunicationError, 1, true),
(MxStatusCategory::Unspecified, 1, true),
] {
let reply = MxCommandReply {
protocol_status: Some(ok_status("command ok")),
statuses: vec![MxStatusProxy {
success,
category: category as i32,
..MxStatusProxy::default()
}],
..MxCommandReply::default()
};
assert_eq!(
ensure_mxaccess_success(reply).is_err(),
expect_failure,
"category {category:?} with success {success} expected failure = {expect_failure}"
);
}
}
#[test]
fn command_error_display_keeps_raw_reply_accessible() {
let reply = mxaccess_failure_reply();
@@ -752,6 +806,179 @@ async fn authenticate_user_keeps_credentials_out_of_surfaced_errors() {
);
}
#[tokio::test]
async fn authenticate_user_scrubs_exact_caller_credential_echoed_in_diagnostic() {
// CLI-40: MXAccess can echo the supplied credential back inside its failure
// diagnostic (here in statuses[0].diagnostic_text). The token has no
// mxgw_/bearer shape, so the pattern scrub alone cannot catch it — the
// exact-secret scrub must replace the caller's password with <redacted>.
let credential = "sup3rSecretVerify9f3a2b";
let state = Arc::new(FakeState::default());
*state.invoke_override.lock().await = Some(InvokeOverride::CannedReply(Box::new(
command_reply_fixture("authenticate-user.echoed-credential.reply.json"),
)));
let endpoint = spawn_fake_gateway(state.clone()).await;
let client = GatewayClient::connect(ClientOptions::new(endpoint))
.await
.unwrap();
let session = client.session("session-fixture");
let error = session
.authenticate_user(7, "verifier", credential)
.await
.unwrap_err();
assert!(
matches!(error, Error::MxAccess(_)),
"OK protocol + negative hresult must route to Error::MxAccess: {error:?}"
);
let rendered = error.to_string();
assert!(
!rendered.contains(credential),
"exact caller credential leaked into the surfaced error: {rendered}"
);
assert!(
rendered.contains("<redacted>"),
"credential occurrence must be replaced with <redacted>: {rendered}"
);
}
/// Drive `authenticate_user` against a canned reply that echoes the caller's
/// credential in every string field, then assert the surfaced
/// [`Error::MxAccess`] leaks it nowhere — neither through the structured reply a
/// caller can read back (`reply().protocol_status.message`,
/// `reply().diagnostic_message`, `reply().statuses[i].diagnostic_text`) nor
/// through `Display`/`Debug`.
async fn assert_authenticate_user_scrubs_structured_reply(fixture: &str) {
let credential = "sup3rSecretVerify9f3a2b";
let state = Arc::new(FakeState::default());
*state.invoke_override.lock().await = Some(InvokeOverride::CannedReply(Box::new(
command_reply_fixture(fixture),
)));
let endpoint = spawn_fake_gateway(state.clone()).await;
let client = GatewayClient::connect(ClientOptions::new(endpoint))
.await
.unwrap();
let session = client.session("session-fixture");
let error = session
.authenticate_user(7, "verifier", credential)
.await
.unwrap_err();
let Error::MxAccess(mx_access) = &error else {
panic!("{fixture}: credential-echoed reply must route to Error::MxAccess, got {error:?}");
};
// The structured reply a caller can read back must be scrubbed too — the raw
// MxCommandReply otherwise reintroduces the leak Display/Debug already close.
let reply = mx_access.reply();
if let Some(status) = reply.protocol_status.as_ref() {
assert!(
!status.message.contains(credential),
"{fixture}: credential leaked via reply().protocol_status.message: {}",
status.message
);
}
assert!(
!reply.diagnostic_message.contains(credential),
"{fixture}: credential leaked via reply().diagnostic_message: {}",
reply.diagnostic_message
);
for (index, status) in reply.statuses.iter().enumerate() {
assert!(
!status.diagnostic_text.contains(credential),
"{fixture}: credential leaked via reply().statuses[{index}].diagnostic_text: {}",
status.diagnostic_text
);
}
let display = error.to_string();
let debug = format!("{error:?}");
assert!(
!display.contains(credential),
"{fixture}: credential leaked into Display: {display}"
);
assert!(
!debug.contains(credential),
"{fixture}: credential leaked into Debug: {debug}"
);
assert!(
display.contains("<redacted>"),
"{fixture}: Display must mark the redaction: {display}"
);
}
#[tokio::test]
async fn authenticate_user_scrubs_credential_from_structured_reply_ok_protocol_variant() {
// OK protocol envelope + negative hresult: already Error::MxAccess before
// ISSUE 2, but the stored reply's string fields still leaked the credential.
assert_authenticate_user_scrubs_structured_reply(
"authenticate-user.echoed-credential.reply.json",
)
.await;
}
#[tokio::test]
async fn authenticate_user_scrubs_credential_from_structured_reply_mxaccess_failure_variant() {
// PROTOCOL_STATUS_CODE_MXACCESS_FAILURE: before ISSUE 2 this landed in
// Error::Command (unscrubbed, raw Display/Debug) — the red-first case.
assert_authenticate_user_scrubs_structured_reply(
"authenticate-user.echoed-credential-mxaccess-failure.reply.json",
)
.await;
}
#[tokio::test]
async fn authenticate_user_maps_missing_payload_reply_to_malformed_reply() {
// CLI-41: an OK reply with neither a typed AuthenticateUser payload nor a
// return_value is malformed.
let state = Arc::new(FakeState::default());
*state.invoke_override.lock().await = Some(InvokeOverride::CannedReply(Box::new(
command_reply_fixture("authenticate-user.missing-payload.reply.json"),
)));
let endpoint = spawn_fake_gateway(state.clone()).await;
let client = GatewayClient::connect(ClientOptions::new(endpoint))
.await
.unwrap();
let session = client.session("session-fixture");
let error = session
.authenticate_user(7, "verifier", "pw")
.await
.unwrap_err();
assert!(
matches!(error, Error::MalformedReply { .. }),
"missing payload + missing return_value must be MalformedReply, got {error:?}"
);
}
#[tokio::test]
async fn authenticate_user_falls_back_to_return_value_when_typed_payload_absent() {
// CLI-41: an OK reply that carries only a return_value (legacy worker) must
// resolve the user id from it, mirroring add_buffered_item's fallback.
let state = Arc::new(FakeState::default());
*state.invoke_override.lock().await = Some(InvokeOverride::CannedReply(Box::new(
command_reply_fixture("authenticate-user.return-value-only.reply.json"),
)));
let endpoint = spawn_fake_gateway(state.clone()).await;
let client = GatewayClient::connect(ClientOptions::new(endpoint))
.await
.unwrap();
let session = client.session("session-fixture");
let user_id = session
.authenticate_user(7, "verifier", "pw")
.await
.unwrap();
assert_eq!(
user_id, 7,
"user id must resolve from the int32 return_value"
);
}
#[tokio::test]
async fn stream_alarms_emits_snapshot_then_complete_then_transition_in_order() {
let state = Arc::new(FakeState::default());
@@ -903,6 +1130,11 @@ enum InvokeOverride {
/// `AuthenticateUser` rejected by MXAccess) so the client's
/// `ensure_mxaccess_success` check is exercised on the typed helper path.
MxAccessFailure,
/// Reply with a caller-supplied canned [`MxCommandReply`]. Lets a test
/// drive a helper with a shared behavior fixture (e.g. the
/// echoed-credential / missing-payload / return-value-only
/// authenticate-user replies). Boxed to keep the enum small.
CannedReply(Box<MxCommandReply>),
}
#[derive(Clone)]
@@ -1005,6 +1237,7 @@ impl MxAccessGateway for FakeGateway {
payload: None,
..MxCommandReply::default()
})),
InvokeOverride::CannedReply(reply) => Ok(Response::new(*reply)),
InvokeOverride::WriteOk => {
// Extract and capture the WriteCommand payload so the test
// can assert on server_handle, item_handle, user_id, and value.
@@ -1358,6 +1591,85 @@ fn event(sequence: u64) -> MxEvent {
}
}
/// Load a shared command-reply fixture into an [`MxCommandReply`].
///
/// The fixtures are protobuf JSON, which prost cannot parse directly, so this
/// reads the fields the reply-validation rules actually consume (`hresult` and
/// the status `success`/`category` pair) and rebuilds the message. Enum names
/// resolve through the generated `from_str_name`, so a fixture naming a
/// category the contract does not define fails the test rather than silently
/// degrading to `Unspecified`.
fn command_reply_fixture(file_name: &str) -> MxCommandReply {
let fixture = behavior_fixture(&format!("command-replies/{file_name}"));
let statuses = fixture["statuses"]
.as_array()
.map(Vec::as_slice)
.unwrap_or_default()
.iter()
.map(|status| {
let category_name = status["category"].as_str().unwrap();
MxStatusProxy {
success: status["success"].as_i64().unwrap() as i32,
category: MxStatusCategory::from_str_name(category_name)
.unwrap_or_else(|| panic!("unknown status category {category_name}"))
as i32,
detail: status["detail"].as_i64().unwrap_or_default() as i32,
diagnostic_text: status["diagnosticText"]
.as_str()
.unwrap_or_default()
.to_owned(),
..MxStatusProxy::default()
}
})
.collect();
// The fixtures that exercise the return_value fallback path carry a typed
// `returnValue` (VT_I4). Project it so a canned reply can drive the
// helper's payload -> return_value -> MalformedReply precedence.
let return_value = fixture.get("returnValue").and_then(|value| {
value["int32Value"].as_i64().map(|int32| MxValue {
data_type: MxDataType::Integer as i32,
variant_type: value["variantType"].as_str().unwrap_or("VT_I4").to_owned(),
kind: Some(Kind::Int32Value(int32 as i32)),
..MxValue::default()
})
});
// Honor the fixture's real protocol status (code + message) so a canned
// reply can drive the MXACCESS_FAILURE routing path, not just an OK
// envelope. Falls back to an OK envelope when the fixture omits it.
let protocol_status = fixture.get("protocolStatus").map_or_else(
|| ok_status("command ok"),
|status| {
let code_name = status["code"].as_str().unwrap_or("PROTOCOL_STATUS_CODE_OK");
ProtocolStatus {
code: ProtocolStatusCode::from_str_name(code_name)
.unwrap_or_else(|| panic!("unknown protocol status code {code_name}"))
as i32,
message: status["message"].as_str().unwrap_or_default().to_owned(),
}
},
);
MxCommandReply {
session_id: fixture["sessionId"].as_str().unwrap_or_default().to_owned(),
correlation_id: fixture["correlationId"]
.as_str()
.unwrap_or_default()
.to_owned(),
protocol_status: Some(protocol_status),
hresult: fixture["hresult"].as_i64().map(|hresult| hresult as i32),
statuses,
diagnostic_message: fixture["diagnosticMessage"]
.as_str()
.unwrap_or_default()
.to_owned(),
return_value,
..MxCommandReply::default()
}
}
fn behavior_fixture(path: &str) -> Value {
let path = std::path::Path::new(env!("CARGO_MANIFEST_DIR"))
.join("../proto/fixtures/behavior")
+42 -7
View File
@@ -99,9 +99,20 @@ library:
are skipped. Only successes are cached; failures always reach the inner verifier.
On a gateway-initiated revoke/rotate/delete the dashboard admin service calls
`IApiKeyCacheInvalidator.Invalidate(keyId)`, evicting the cached entry
immediately. The short TTL is the backstop for out-of-band mutations (a direct DB
edit, or a revoke run by the separate `apikey` CLI process, whose in-memory cache
is not the running gateway's cache).
immediately. `Invalidate` bumps a per-key generation counter **before** it evicts,
and `VerifyAsync` snapshots that generation before the inner verify and re-checks
it after writing the cache entry (set-then-recheck); a revoke that lands while a
verification is still in flight in the inner library therefore discards that
verification's repopulation instead of re-caching the just-revoked identity for a
full TTL (SEC-34). The short TTL remains the backstop for two bounded-staleness
windows it cannot close directly: (1) out-of-band mutations (a direct DB edit, or a
revoke run by the separate `apikey` CLI process, whose in-memory cache is not the
running gateway's cache); and (2) a key whose `ExpiresUtc` passes while cached keeps
authenticating until the entry's TTL elapses — expiry is enforced by the inner
library verifier, which a cache hit never reaches, and the verification identity the
library returns carries no expiry timestamp, so the cache cannot cap an entry at the
key's expiry (capping it needs the donor library to surface expiry on the
verification identity). The default 15 s TTL bounds both windows.
- **`CoalescingMarkApiKeyStore`** wraps the library `IApiKeyStore` and forwards at
most one `MarkUsed` write per key per
`MxGateway:Security:ApiKeyLastUsedCoalesceSeconds` (default 60 s), so even under a
@@ -116,6 +127,26 @@ a dictionary lookup. Both windows are configurable and may be set to `0` to disa
the respective mechanism; see
[GatewayConfiguration](./GatewayConfiguration.md).
Failures are never cached — a wrong secret always reaches the store — so the
failure path is shielded by `ApiKeyFailureLimiter` instead, consulted before
`VerifyAsync` runs. It counts failures over one sliding
`MxGateway:Security:ApiKeyFailureWindowSeconds` window in two layers: a composite
`(transport peer, key id)` partition capped at `ApiKeyFailureLimit`, and a
per-key-id aggregate across all peers capped at `ApiKeyFailureAggregateLimit`. The
key id never partitions on its own — it is public, so an attacker-supplied one
would otherwise let any peer throttle a key it does not hold — and it joins the
partition only when the presented token is validly shaped
(`mxgw_<keyId>_<secret>`, key id at most 64 characters), with at most 32 key-id
partitions per address before the overflow collapses onto that address's fallback
partition. An over-limit state admits one probe per
`ApiKeyFailureProbeIntervalSeconds` through to the real verifier and refuses
everything else with `ResourceExhausted` before the store read, so a legitimate
holder presenting the correct secret always reaches the constant-time compare and
resets both layers; the counter's LRU eviction (`ApiKeyFailureTrackedPeers`)
prefers expired windows and will not drop an over-limit partition below a 2x
overshoot ceiling, so the memory bound cannot be turned into a way to clear an
active block. See [Authorization](./Authorization.md) for the enforcement path.
## Storage
API-key state lives in a dedicated SQLite database owned by the shared library.
@@ -128,10 +159,14 @@ is derived from `Environment.GetFolderPath(SpecialFolder.CommonApplicationData)`
(`C:\ProgramData\MxGateway\gateway-auth.db` on Windows,
`/usr/share/MxGateway/gateway-auth.db` or the container equivalent elsewhere) so the
credential store is never written relative to the launch working directory on a
non-Windows host. The production hosts pin the explicit Windows path in
`appsettings.json`. `GatewayOptionsValidator` rejects a non-rooted (relative)
`SqlitePath` so a bad override fails fast at startup rather than scattering the store
by launch CWD (SEC-01).
non-Windows host. `appsettings.json` no longer ships an explicit path (SEC-33): the
removed Windows literal matched the Windows code default and, being non-rooted on a
Unix host, would have resolved against the CWD there; deployed hosts override it
through the NSSM environment (`MxGateway__Authentication__SqlitePath`).
`GatewayOptionsValidator` rejects a `SqlitePath` that is not rooted **on the host
running the gateway** (`Path.IsPathRooted`, current OS) — a relative filename or a
foreign-platform literal fails fast at startup rather than scattering the store by
launch CWD (SEC-01, SEC-33).
The library owns the SQLite schema and connection factory. The `api_keys` table
carries the key id, key prefix, secret-hash blob, display name, serialized scopes,
+7 -2
View File
@@ -89,9 +89,14 @@ The flow is:
The status codes are deliberately distinct: `Unauthenticated` signals "we do not know who you are," and `PermissionDenied` signals "we know who you are, but you cannot do this." Treating the two as the same code would make troubleshooting harder for client implementations.
### Rate limiting the auth surface (SEC-11)
### Rate limiting the auth surface (SEC-11, SEC-31, SEC-32)
Before the verification store read, the helper checks a cheap in-process per-peer failure counter (`ApiKeyFailureLimiter`). A peer that has accumulated more than `MxGateway:Security:ApiKeyFailureLimit` failed attempts inside the sliding `ApiKeyFailureWindowSeconds` window is short-circuited with `StatusCode.ResourceExhausted` — so online guessing of API-key secrets cannot spend a SQLite read (and, in a naive design, a cache miss) per attempt. The peer is keyed on the presented key id where the token parses, falling back to the transport peer address; keying on key id throttles a single abusive credential without penalizing co-located clients behind a shared NAT. A successful verification resets the peer's counter. The counter is a bounded LRU (`ApiKeyFailureTrackedPeers`) so it cannot grow without limit. `ResourceExhausted` reveals only that throttling is in effect, not whether any particular secret was valid, preserving the opaque-failure property.
Before the verification store read, the helper asks a cheap in-process failure counter (`ApiKeyFailureLimiter`) whether the attempt may proceed, so online guessing of API-key secrets cannot spend a SQLite read (and, in a naive design, a cache miss) per attempt. The counter has two layers over one sliding `ApiKeyFailureWindowSeconds` window:
- **Composite `(transport peer, key id)` partitions.** Reaching `MxGateway:Security:ApiKeyFailureLimit` failures binds the throttle to the address that produced them. The key id alone is never the partition: key ids are not secret — they ride in every token and are listed on the dashboard — so keying on them let any network peer deny a key to its legitimate holder. The key id joins the partition only after a token-shape check (literal `mxgw` prefix, at least three non-empty `_` segments, key id of at most 64 characters), and one address may mint at most 32 key-id partitions before the overflow collapses onto that address's fallback partition.
- **A per-key-id aggregate** across all peers (`ApiKeyFailureAggregateLimit`, default 30), which bounds a distributed or source-rotating sprayer that never trips any single partition.
An over-limit state is a valve, not a wall: one request per `ApiKeyFailureProbeIntervalSeconds` (default 5 s) is admitted through to the real verifier, and everything else is refused with `StatusCode.ResourceExhausted` before the store read. The slot is claimed atomically, so a burst arriving together at an interval boundary still yields exactly one admission, and a slot claimed for a request that a later layer then refuses is handed back under a per-state version stamp — never by timestamp comparison, which collides whenever a concurrent failure re-arms the same state on the same clock tick. A successful verification resets both layers — which is why the reset path stays reachable while a key is under active spray. One exception: when the caller's key id was collapsed into its address's shared fallback partition by the per-peer cap, a success clears the key's aggregate but leaves that shared partition alone, since it also holds failures contributed by other key ids from the same address. The tracked partitions form a bounded LRU (`ApiKeyFailureTrackedPeers`) whose eviction prefers fully expired windows and never removes an over-limit partition below a 2x transient overshoot ceiling, so the cap bounds memory without becoming a reset button for an active block. `ResourceExhausted` reveals only that throttling is in effect, not whether any particular secret was valid, preserving the opaque-failure property. Refusals increment `mxgateway.auth.throttled`, tagged `stage=peer|aggregate` and nothing else — `/metrics` is unauthenticated, so neither key ids nor peer addresses may appear there.
The dashboard login surface is throttled independently: `POST /auth/login` carries a fixed-window ASP.NET Core rate-limiter policy keyed per remote IP (`MxGateway:Security:LoginRateLimit*`), rejecting a burst with HTTP 429 before the LDAP bind is relayed to the directory. See [GatewayConfiguration](./GatewayConfiguration.md#security-options).

Some files were not shown because too many files have changed in this diff Show More