Compare commits

..

42 Commits

Author SHA1 Message Date
Joseph Doherty a346d514dd test(contracts): scope command-reply fixture invariants past the CLI-40/41 authenticate-user malformed-reply fixtures
ci / portable (push) Successful in 14m0s
ci / java (push) Successful in 6m50s
ci / windows-x86 (push) Failing after 1m21s
ci / nightly-windev (push) Has been skipped
The blanket loop asserted HRESULT/Statuses/ReturnValue on every command_replies fixture, but the authenticate-user.* fixtures added for the malformed-reply and credential-redaction contracts deliberately omit them (NRE on ReturnValue.DataType). Keep universal Kind/ProtocolStatus invariants for all; apply the MXAccess-detail block only to fixtures that carry it. Test-only.
2026-08-07 08:48:49 -04:00
Joseph Doherty a2d3f66b8b docs(archreview): record next-cycle candidate findings + pending operator actions surfaced during remediation
ci / windows-x86 (push) Successful in 1m19s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m19s
ci / portable (push) Failing after 4m55s
2026-08-07 08:48:03 -04:00
Joseph Doherty 93d84019b9 docs(tracking): sync IPC-23 domain register to Done (doc wave landed; Grpc.md row intentionally scoped out — DrainEvents is a worker diagnostic)
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 1m19s
ci / java (push) Successful in 2m4s
ci / portable (push) Failing after 4m43s
2026-08-07 08:47:31 -04:00
Joseph Doherty 6d26ed094c docs(tracking): close old-tracker CLI-24, CLI-34 as Done (2026-07-12 review old-tracker actions; both incidentally fixed)
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m22s
ci / windows-x86 (push) Successful in 1m20s
ci / portable (push) Failing after 4m26s
2026-08-07 08:11:57 -04:00
Joseph Doherty 4201da63d2 docs(tracking): flip IPC-24/IPC-25 to Done in the Contracts&IPC domain register (missed by the codegen-wave tracker update)
ci / windows-x86 (push) Failing after 1m19s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m17s
ci / portable (push) Failing after 4m51s
2026-08-07 08:10:39 -04:00
Joseph Doherty 9c780f8164 Merge branch 'fix/cli-39-version-train'
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 1m20s
ci / java (push) Successful in 2m12s
ci / portable (push) Failing after 4m44s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
2026-08-07 08:09:39 -04:00
Joseph Doherty 440e7cf03d fix(CLI-39): bump Contracts nupkg to 0.2.0; scope pack-clients.ps1 regexes
Code review of the CLI-39 branch caught an Important gap: Contracts.csproj
was left at the already-published 0.1.2 while the .NET Client moved to
0.2.0. Invoke-PackDotnet in scripts/pack-clients.ps1 packs and publishes
both ZB.MOM.WW.MxGateway.Contracts and .Client through the same -Publish
loop, and the new collision guard runs every nupkg it finds through
Assert-GiteaPackageNotPublished. Left as-is, the next real .NET publish
would pack Contracts at 0.1.2, the guard would correctly refuse to
republish it, and the loop would abort mid-way with Client (alphabetically
first) possibly already pushed -- the two packages permanently out of
lockstep.

- src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csproj:
  <Version> 0.1.2 -> 0.2.0, matching the .NET Client (they have always
  released together).
- src/Directory.Build.props: corrected a comment that was now stale --
  it claimed the repo-wide 0.1.2 default was kept to match the Contracts
  package, which is no longer true now that Contracts.csproj overrides it.
  The <Version> value itself is unchanged; Server/Worker/Tests staying at
  0.1.2 is a separate, not-yet-made decision, out of scope for CLI-39.
- docs/ClientPackaging.md: Contracts.csproj added as a fifth manifest in
  the Versioning section, with the near-miss recorded.

Also hardened scripts/pack-clients.ps1 per the same review: the Python
(pyproject.toml) and Rust (Cargo.toml) version-extraction regexes now
scope to the [project]/[package] section header instead of matching the
first "version = ..." line anywhere in the file (Cargo.toml has an
identical second one under [workspace.package] -- matching whichever came
first was luck of ordering, not correctness). One-line comment added on
the nuget filename-parse regex.

Verified live against the real Gitea registry: Contracts and Client both
still refuse at 0.1.2 and both now pass at 0.2.0, including running the
actual Invoke-PackDotnet filename-parse-then-guard logic against two
freshly packed real .nupkg files. dotnet build of Contracts.csproj and the
client slnx both clean. No publish performed.
2026-08-07 08:07:40 -04:00
Joseph Doherty ae605d2368 Merge remote-tracking branch 'origin/fix/wrk-22-25-seam'
ci / windows-x86 (push) Failing after 1m19s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m10s
ci / portable (push) Failing after 4m27s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
#	archreview/2026-07-12/remediation/30-contracts-ipc.md
2026-08-07 08:01:10 -04:00
Joseph Doherty 9b2abef4e1 fix(CLI-39): bump client versions off published 0.1.2; guard the publish pipeline
Converges all five clients on one version after four had drifted onto the
already-published 0.1.2/0.1.1 while their APIs kept changing underneath it:

- Rust Cargo.toml [package] + [workspace.package] -> 0.2.0 (CLIENT_VERSION
  already derives from CARGO_PKG_VERSION, no separate edit).
- Python pyproject.toml + version.py -> 0.2.0; new test asserts __version__
  matches pyproject.toml (closes the CLI-26 residual drift mode).
- Go mxgateway/version.go ClientVersion -> 0.2.0.
- .NET ZB.MOM.WW.MxGateway.Client.csproj <Version> -> 0.2.0.
- Java -> 0.2.1, not 0.2.0: the live Gitea Maven feed already had 0.2.0
  published (2026-06-26), before the CLI-37/38/40/41 conformance fixes
  changed the client's observable behavior, so reusing 0.2.0 would label
  two different APIs identically. Recorded as an exception in
  docs/ClientPackaging.md's new Versioning section.

Publish-pipeline guards:

- scripts/tag-go-module.ps1 implements the CLI-21 guard: after semver
  validation it refuses to tag unless clients/go/mxgateway/version.go's
  ClientVersion already matches the requested tag version.
- scripts/pack-clients.ps1 gains a Gitea package-registry collision guard
  wired into every per-language -Publish step; it aborts if the target
  name+version already exists rather than force-overwriting. Verified live
  against the real Gitea registry (credentials already present in this
  environment) — correctly refuses on every known-published artifact and
  passes on every unpublished target.

Docs updated in the same commit: docs/ClientPackaging.md (new Versioning
section), and the five client READMEs' stale 0.1.1/0.1.2 example versions.

No .proto changes. No publish performed.
2026-08-07 07:58:49 -04:00
Joseph Doherty 815e58d28b docs(tracking): record WRK-22/24/25/27 + IPC-26 windev evidence (377 pass)
ci / java (push) Successful in 2m7s
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 1m12s
ci / portable (push) Failing after 4m12s
2026-08-07 07:52:44 -04:00
Joseph Doherty 8df35cd63a fix(WRK-22,WRK-24,WRK-25,WRK-27,IPC-26): worker write-seam hardening
ci / java (push) Successful in 2m7s
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 1m13s
ci / portable (push) Failing after 4m6s
WRK-22/IPC-26: tombstone a WriteAsync/WriteBatchAsync cancelled while
waiting for the write lock (PendingFrame.Claimed under _gate; DequeueNext
skips cancelled, claims the frame it returns) so a cancelled write never
reaches the wire unless already claimed mid-write (documented residual).

WRK-25: add WriteBatchAsync; RunEventDrainLoopAsync submits the drained
event batch through it, so a burst of N events costs one flush not N.
IPC-30 oversized-event structured fault preserved via FindOversizedEvent.

WRK-24: reject a below-1024 negotiated frame maximum at the handshake
(MinNegotiableFrameBytes, matching GatewayOptionsValidator floor).

WRK-27: alarm poll advertises StaCallInProgress on the heartbeat snapshot
so the watchdog suppresses to the ceiling, not the grace.

Docs (WorkerFrameProtocol.md, MxAccessWorkerInstanceDesign.md) and the
2026-07-12 remediation registers/change-log updated in the same commit.
2026-08-07 07:50:38 -04:00
Joseph Doherty a55956ffa5 Merge branch 'fix/tst-30-runner-docs'
ci / java (push) Successful in 2m11s
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 1m38s
ci / portable (push) Failing after 17m43s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
2026-08-07 07:50:21 -04:00
Joseph Doherty aba22358f5 Merge branch 'fix/ipc-27-descriptor-test'
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m14s
ci / java (push) Successful in 2m8s
ci / portable (push) Failing after 4m45s
2026-08-07 07:49:33 -04:00
Joseph Doherty b604fed72b docs(TST-30): document shared-runner CI bottleneck + second-runner operator runbook
Doc half of TST-30 (single shared Gitea runner is a CI throughput/availability
bottleneck): docs/GatewayTesting.md's Continuous Integration section gains a
"Runner capacity is shared and finite" subsection covering the maxParallel=1
instance-level runner shared with dohertj2/lmxopcua, the ~20-30 min queue
latency observed under cross-repo contention, and Gitea 1.26's missing run
cancel/delete API. The existing "windev tier down" degraded-mode paragraph now
also covers "runner contended" as a reason to bypass the queue via
CI_SHA=<sha> scripts/ci/run-windev-ci.sh <mode> or the manual windev worktree
flow, generalizing it per the finding's design note.

New operator runbook docs/runbooks/TST-30-second-ci-runner.md carries the
actual runner registration (option a: second act_runner instance on
10.100.0.35 with the same container.network: traefik config, recommended;
option b: dedicated labelled runner, escalation only; option c: runner on
windev, rejected) plus verification steps and the no-cancel caveat. The
optional workflow-level concurrency group is documented as unverified --
framed as "verify before relying on it" -- and left unimplemented in ci.yml,
since registering the runner and any runs-on gating is operator/infra work
outside this repo's tree.

Tracking: TST-30 -> Done (doc half; runner registration operator-pending) in
both registers + change-log row.
2026-08-07 07:47:37 -04:00
Joseph Doherty 6060d21995 fix(IPC-27): close descriptor-freshness blind spots for enums, services, and Galaxy
ClientProtoInputTests.Descriptor_ContainsEveryContractMessageAndField only
compared messages and fields, and only enumerated the gateway/worker
descriptors, so a new enum value, a new RPC, or any galaxy_repository.proto-only
change would not redden the test even though it is documented as the primary
protoc-free CI gate.

Rename to Descriptor_ContainsEveryContractSymbol and extend the reflection walk
on both sides (published protoset and in-process contract) to also collect
enums/enum values ({enumFullName}, {enumFullName}/{valueName}) and
services/methods ({serviceFullName}, {serviceFullName}/{methodName}), and add
GalaxyRepositoryReflection.Descriptor to the enumerated files. The comparison
stays a flat, order-insensitive string-set diff with no protoc dependency.

Update docs/ClientProtoGeneration.md and docs/Contracts.md prose from
"message or field" to the full symbol coverage.

Red-path proof: pointed the test at the pre-IPC-01 stale protoset and confirmed
it failed naming max_frame_bytes, several MxCommandKind/AlarmProviderMode enum
values, MxAccessGateway/StreamAlarms and GalaxyRepository/BrowseChildren, and
the galaxy_repository.v1.* surface; restored the real path and re-ran green.

Flips IPC-27 to Done in the 2026-07-12 remediation tracker and register.
2026-08-07 07:47:18 -04:00
Joseph Doherty 1c2f3a62c1 Merge branch 'fix/sec-36-ldap-secret'
ci / windows-x86 (push) Successful in 1m16s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m31s
ci / portable (push) Failing after 4m37s
2026-08-07 07:44:43 -04:00
Joseph Doherty 0646c73e48 Merge branch 'fix/ipc-24-25-codegen'
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m15s
ci / java (push) Successful in 2m5s
ci / portable (push) Failing after 4m32s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
2026-08-07 07:42:35 -04:00
Joseph Doherty eacdd2d453 fix(IPC-23,IPC-24,IPC-25,IPC-32): proto-comment regen wave + codegen-freshness guards
Proto comments (comment-only, no wire change):
- mxaccess_worker.proto GatewayHello.max_frame_bytes: every worker->gateway frame
  must serialize within the negotiated max; reply builders truncate (IPC-23).
- mxaccess_gateway.proto DrainEventsReply: count-cap + byte-cap, drain-until-empty
  caller contract (IPC-23).
- mxaccess_gateway.proto ReplayGap.oldest_available_sequence: empty-ring value is
  highest-observed+1, oldest-1 resume formula stays valid (GWC-25 deferred amendment).

Regen wave: Contracts/Generated (C# XML doc), rust vendored protos (byte-copy),
Go bindings (worker binding was genuinely stale - lacked MaxFrameBytes entirely),
Python worker _pb2 (real descriptor delta), Java aggregates (javadoc, zero
protobuf-version churn under the pinned toolchain), client descriptor set.

IPC-24: pinned Java toolchain regenerates with no gencode-version churn, so the
unconditional churn-revert step in ci.yml is a fossil - deleted it; git diff is
now a true message-level drift gate for the single-file Java aggregates.

IPC-25: pin protoc-gen-go v1.36.11 / protoc-gen-go-grpc 1.6.2 in the Go generate
script (+ fix a latent pwsh-7 parse bug); add Check 4 to check-codegen.ps1
(regenerate Go+Python bindings, fail on diff, tool-missing fails not skips); add
the pinned-generator installs to the portable CI job.

IPC-32: relabel check-codegen banners 1/4..4/4 (folded into the Check 4 edit).

Docs: ClientProtoGeneration.md, Contracts.md, GatewayTesting.md, build.gradle
checkGeneratedClean caveat. Tracking: IPC-23/24/25/32 -> Done, GWC-25 proto note
resolved, change-log 2026-08-07.
2026-08-07 07:41:18 -04:00
Joseph Doherty 8c312c717c fix(SEC-36): scrub committed dev LDAP service-account password; add user-secrets channel + rotation runbook
Repo-side half of SEC-36. The appsettings.json plaintext was already discharged
before this branch (HEAD ships the fail-closed ${secret:ldap/mxgateway/bind}
store reference), so the residual leak was the literal value in glauth.md,
docs/GatewayTesting.md, and the historical archreview SEC-06 evidence -- all
scrubbed to <service-account-password> placeholders pointing at the source of
truth scadaproj/infra/glauth/.

- csproj: add <UserSecretsId>mxaccessgw-server</UserSecretsId> (dev channel)
- GatewayOptionsValidator: blank-password message now names both channels
  (dev user-secrets, deployed MxGateway__Ldap__ServiceAccountPassword)
- test: assert the message names both channels
- docs: GatewayConfiguration.md (three channels + rotation note), glauth.md
  (placeholders + rotation-required + runbook pointer), GatewayTesting.md
- new operator runbook docs/runbooks/SEC-36-ldap-credential-rotation.md
  (live rotation + NSSM staging remain operator-pending)
- tracking: SEC-36 -> Done (repo-side) in both registers + change-log

Deviation: kept the ${secret:} reference in appsettings.json rather than
deleting it (spec step 2 assumed the stale plaintext baseline); deleting it
would regress the shipped/documented/tested secret-store channel.

git grep -i for the old value is empty across all tracked files.
2026-08-07 07:40:41 -04:00
Joseph Doherty 10534ec906 docs(TST-27,WRK-26,CLI-42,CLI-43,IPC-28): P1 doc-drift batch, discharges IPC-29
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m17s
ci / java (push) Successful in 2m10s
ci / portable (push) Failing after 3m53s
TST-27: docs/GatewayConfiguration.md's ShowTagValues row no longer says
"Reserved" — it now states what false (default) does (DashboardEventBroadcaster
blanks tag values from a deep-cloned MxEvent before the SignalR events-hub
mirror), the security relevance (no per-session hub ACL yet, so this
redaction is the only thing between a low-trust Viewer and other sessions'
tag values), and the honest scope limit (does not cover /browse).

WRK-26 (discharges IPC-29): docs/MxAccessWorkerInstanceDesign.md's "Outbound
Queues" section rewritten from the stale five-level priority list to the
two-class Control/Event scheduler actually shipped, with the collapsed-
decision rationale, and the overflow paragraph rewritten to the implemented
fail-fast. docs/WorkerFrameProtocol.md gained a "Write Scheduling And
Sequencing" section describing HEAD truthfully: WRK-23's peek-stamp-commit
sequencing is live, WRK-25's event-batch flush coalescing is not (the drain
loop still awaits each event write individually), and WRK-22's cancellation
tombstone is not yet defined (noted as pending, not documented as shipped).

CLI-42: clients/rust/README.md and docs/ClientPackaging.md document the
vendored Rust proto layout matching build.rs — repo-path-first resolution
falling back to clients/rust/protos/, the check-codegen.ps1 Check 3 refresh
rule, and why cargo package/publish run without --no-verify.

CLI-43: docs/style-guides/JavaStyleGuide.md now says Java 17 (Ignition 8.3
baseline), mirroring CLI-12's wording, matching the shipped build.gradle.

IPC-28: docs/Grpc.md's exception-mapping prose gained CommandTooLarge ->
ResourceExhausted, and the Invoke section gained the oversized-payload
sentence, cross-referencing GatewayConfiguration.md's headroom rule.

Tracking: TST-27, WRK-26, CLI-42, CLI-43, IPC-28 flipped to Done and IPC-29
marked discharged-by-WRK-26 in 00-tracking.md and the 20/30/50/60 domain
registers, with a 2026-08-07 change-log entry.

Doc-only change; no source, proto, or test edits.
2026-08-07 07:26:58 -04:00
Joseph Doherty 97f79e79ef Merge remote-tracking branch 'origin/fix/wrk-21-drain-cluster'
ci / windows-x86 (push) Successful in 1m27s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m11s
ci / portable (push) Failing after 4m7s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
#	docs/MxAccessWorkerInstanceDesign.md
2026-08-07 07:18:58 -04:00
Joseph Doherty 758277bc62 docs(tracking): record WRK-21 review follow-ups (monotonic budget, guarded fallback) with windev evidence
ci / windows-x86 (push) Successful in 1m19s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m15s
ci / portable (push) Successful in 9m30s
2026-08-07 07:13:46 -04:00
Joseph Doherty 6bc3f9b991 fix(WRK-21): make drain budget monotonic at the reserve boundary; guard the reply-too-large fallback
ci / windows-x86 (push) Successful in 1m16s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m17s
ci / portable (push) Successful in 9m31s
Review follow-ups on the WRK-21 cluster.

1. ResolveDrainReplyByteBudget was a step function, not a floor: just above the
   64 KiB reserve the budget collapsed to a few bytes (at the validator-permitted
   floor MaxMessageBytes = 1024 + 64 KiB it was exactly 1024), too small to move a
   byte-heavy event, so DrainEvents truncated on every call and the drain-until-
   empty loop never terminated. It now takes the max of (frameMax - reserve) and
   frameMax/2, so the budget is monotonic and never below half the frame max. New
   test DrainEvents_AtValidatorFloorFrameMax_MakesProgressAndTerminates drives a
   byte-heavy queue at the exact validator floor and asserts it drains to empty
   with no head ever reported oversized.

2. The reply-too-large fallback write is now itself size-guarded
   (WriteReplyTooLargeFallbackAsync, used by both the control and STA reply seams):
   at a pathologically tiny negotiated max below the gateway's floor the fallback
   could also throw MessageTooLarge and — uncaught — kill the session, defeating the
   "no diagnostics command is session-fatal" invariant. It now log-and-swallows;
   comment notes WRK-24 adds the negotiated-max lower bound that makes it unreachable.

3. Corrected the RepeatedFieldOverheadBytes doc comments: WorkerEvent.CalculateSize()
   already includes the event's tag and length prefix (the same shape the reply's
   repeated events field packs), so the 8 bytes is pure slack over an already-
   conservative estimate, not compensation for a missing wrapper.
2026-08-07 07:09:13 -04:00
Joseph Doherty 34db678635 Merge branch 'fix/cli-40-41-44'
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m19s
ci / java (push) Successful in 2m14s
ci / portable (push) Failing after 4m12s
2026-08-07 07:08:27 -04:00
Joseph Doherty 0d874f91ee fix(CLI-40): scrub the credential from the redacted error's structured reply, route MXACCESS_FAILURE to MxAccess (Rust), fix Go Subscribe terminal-error drop
Code-review follow-up on the CLI-40/41/44 branch.

ISSUE 1 (all five, critical): the message-only scrub still leaked the
server-echoed credential through the redacted error's structured reply accessor
(.NET Reply/Statuses, Java reply()/protocolStatus(), Go MxAccessError.Reply via
errors.As, Rust reply()/into_reply(), Python raw_reply). The redacted error now
carries a scrubbed clone of the reply (protocol_status.message,
diagnostic_message, statuses[].diagnostic_text), with per-language tests asserting
the reply accessor no longer contains the credential.

ISSUE 2 (Rust, critical): ensure_command_success routed MXACCESS_FAILURE to
Error::Command (unlike the other four clients), bypassing attach_secrets and
leaking via derived Debug/Display. MXACCESS_FAILURE now routes to Error::MxAccess,
fixing the cross-client inconsistency.

ISSUE 3 (Go, important): the CLI-44 terminal send was unconditionally
non-blocking, dropping a genuine terminal error under a full buffer on the
never-drop SubscribeEvents path. It is now reserved-slot-non-blocking only for the
cancel-on-overflow path and blocking for the never-drop path.

New shared fixture authenticate-user.echoed-credential-mxaccess-failure.reply.json
wired into all five suites. Minors: whitespace-secret guard on .NET/Java redact
helpers; Java preserves exception subtype on redaction; redaction-helper unit
tests (Go/Java/.NET). Docs (ClientBehaviorFixtures.md, ClientLibrariesDesign.md)
updated to make the structured-field claim true.
2026-08-07 07:04:56 -04:00
Joseph Doherty c9925688f5 docs(tracking): record the WRK-21 cluster as Done with windev evidence
ci / java (push) Successful in 2m28s
ci / windows-x86 (push) Failing after 1m28s
ci / nightly-windev (push) Has been skipped
ci / portable (push) Successful in 8m38s
Change-log row for 2026-08-07: what landed for WRK-21/WRK-28/WRK-23/IPC-30, why
IPC-23 stays In progress (proto-comment/doc wave pending), and the verification
evidence — macOS NonWindows build + validator tests, and the documented windev
path (scripts/ci/windev-worker-ci.ps1 -Mode test) at a256560: x86 Worker build
clean, Worker.Tests 367 passed / 0 failed / 11 skipped.
2026-08-07 06:54:24 -04:00
Joseph Doherty 2aac29618e Merge branch 'fix/sec-33-34'
ci / java (push) Successful in 2m16s
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m25s
ci / portable (push) Successful in 8m32s
2026-08-07 06:51:31 -04:00
Joseph Doherty a2565604df test(WRK-21): keep the drain-to-empty walk inside the pipe harness envelope
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m14s
ci / java (push) Successful in 2m12s
ci / portable (push) Successful in 8m2s
PipePair runs both ends of a duplex pipe in one process with blocking
FlushFileBuffers under every frame write, so it wedges after roughly 85 large
round trips. Drain the full 10,000 byte-heavy events to empty at the queue layer,
where the no-loss property actually lives, and keep the pipe walk at 1,000 events
(29 replies) so it still proves the split end to end. Also give the truncation
test's budget slack: item handle 0 is a proto3 default and is not serialized, so
the probe measurement is a lower bound on the fixture's per-event cost.
2026-08-07 06:50:17 -04:00
Joseph Doherty 193daa9ee8 fix(SEC-33,SEC-34): address code review — missed docs, key-id guard comment, test consolidation
Same-commit docs rule (were missed in the prior commit):
- docs/GalaxyRepository.md: SnapshotCachePath now documents the per-OS derived
  default and the GalaxyRepositoryOptionsValidator rooting/validity enforcement.
- A2-galaxyrepository-adoption-handoff.md: correct the now-inaccurate NSSM caveat
  (SnapshotCachePath override is optional, not required; blank seeds a rooted host
  default, no silent no-op) and repoint the option-validation item at the new
  GalaxyRepositoryOptionsValidator.

SEC-34 guard confirmed and documented: TryParseKeyId's '_' split cannot truncate a
key id because both — and the only — gateway key-creation paths
(ApiKeyAdminCommandLineParser.IsValidKeyId, DashboardApiKeyManagementService.ValidateKeyId)
restrict key ids to IsAsciiLetterOrDigit || '.' || '-', and key ids are never
library-generated. Added a citing comment; no behavior change.

Test consolidation: moved the three host-start SqlitePath overrides into
TestHostEnvironmentInitializer (per-process temp store, mirroring Secrets__SqlitePath)
so future host-start tests auto-cover.
2026-08-07 06:49:24 -04:00
Joseph Doherty dc7fd16dd5 fix(CLI-40,CLI-41,CLI-44): exact-secret scrub, uniform malformed-reply contract, Go terminal-error mislabel
CLI-40: port the exact-secret credential scrub to Rust/Java/.NET (Go/Python
already did it). AuthenticateUser/WriteSecured(2) helpers now redact the exact
caller-supplied secret from any surfaced error, as defense-in-depth on top of the
by-construction guarantee. Rust hand-writes a redacting Debug (derived Debug would
leak the reply); Java/.NET rebuild the same exception type with the redacted
message and do not carry the secret-bearing original forward (so ToString/stack
traces stay clean too).

CLI-41: uniform malformed-reply contract for AuthenticateUser/ArchestrAUserToId/
AddBufferedItem across all five clients — typed payload, else a present int32
return_value, else a typed malformed-reply error. Fixes Go/Java silent-0, .NET
NRE, and Rust's own internal inconsistency.

CLI-44: the Go event goroutine's Recv-error path now uses a non-blocking
sendTerminalEventResult on the reserved slot, so a genuine terminal stream error
is reported as itself instead of being mislabeled ErrSlowConsumer under overflow.

Riders from the CLI-37/38 review: (a) .NET ToDiagnosticSummary and Python
_mxaccess_message surface the raw success member (diagnostics-only parity with
Rust); (b) the status-conversion fixture carries an independent wantSuccess
boolean and the Go/.NET fixture tests assert against it instead of recomputing
the formula under test.

Shared fixtures (authenticate-user.{echoed-credential,missing-payload,
return-value-only}.reply.json) + manifest + ClientBehaviorFixtures.md +
ClientLibrariesDesign.md updated in the same change. Tracking: CLI-40/41/44 -> Done.
2026-08-07 06:42:40 -04:00
Joseph Doherty 7e7f7cad84 fix(SEC-33,SEC-34): host-meaningful path rooting; verification-cache invalidate race
SEC-33: make rooting host-meaningful and stop shipping foreign-platform literals.
- Delete IsRootedForAnyPlatform; AddIfNotRooted now uses Path.IsPathRooted (current OS).
- Promote AddIfNotRooted/AddIfInvalidPath to shared GatewayConfigPathRules so the new
  Galaxy validator reuses them and the two validators cannot drift.
- Remove Authentication:SqlitePath and Galaxy:SnapshotCachePath Windows literals from
  appsettings.json; the CommonApplicationData-derived code defaults take over. The
  Galaxy default is seeded as a configuration value before AddZbGalaxyRepository
  (SnapshotCachePath is init-only, so a PostConfigure mutation cannot compile).
- New GalaxyRepositoryOptionsValidator (ValidateOnStart) enforces a valid, host-rooted
  SnapshotCachePath when PersistSnapshot is true.
- Root-cause the stray junk-named auth DB: host start eagerly builds
  AuthSqliteConnectionFactory; under the non-rooted Windows literal on macOS SQLite
  wrote it relative to the test bin CWD. The three real-host-start tests now pin
  SqlitePath to a temp path.

SEC-34: verification cache Invalidate-vs-in-flight-repopulation race closed with a
per-key generation counter (bump-before-evict, snapshot-then-recheck). The expiry
cap (window 2) takes the documented fallback: the library verification identity
carries no ExpiresUtc, so the cache cannot cap at the key's expiry (donor-library ask).

GWC-24 rider: cap MxGateway:Events:QueueCapacity at int.MaxValue/2 so the derived
checked(2 * EventChannelCapacity) in WorkerClient cannot overflow at session creation.

SEC-35 (doc-only): note IsProduction() env-name semantics in GatewayConfiguration.md.

Docs updated same commit (GatewayConfiguration.md, Authentication.md) and tracking
registers/change-log flipped (00-tracking.md, 40-security-dashboard.md).
2026-08-07 06:36:01 -04:00
Joseph Doherty 7c2eaf09e2 test(WRK-21): size the byte-heavy drain fixture for the pipe harness
ci / java (push) Successful in 2m12s
ci / portable (push) Successful in 8m23s
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 11m46s
PipePair has no continuous read pump — the test thread drains the pipe only
while it sits in ReadUntilAsync — so multi-megabyte DrainEvents frames
interleaved with the heartbeat loop wedge both ends inside FlushFileBuffers,
each waiting for the other to read. Negotiate a 128 KiB frame maximum instead:
the 10,000 byte-heavy events still overflow it many times over, so every
assertion (bounded reply, reported truncation, no event loss across repeated
drains, surviving session) is unchanged.
2026-08-07 06:25:15 -04:00
Joseph Doherty d2bb32d97b Merge branch 'fix/gwc-28-29-30-polish'
ci / java (push) Successful in 2m44s
ci / windows-x86 (push) Successful in 1m3s
ci / nightly-windev (push) Has been skipped
ci / portable (push) Successful in 17m13s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
2026-08-07 06:20:14 -04:00
Joseph Doherty 404f7cd993 docs(archreview): close GWC-28, GWC-29, GWC-30, TST-28
Flip the four findings to Done in the 2026-07-12 tracking registers
(Gateway core + Testing) and in the per-domain registers of
10-gateway-core.md and 60-testing-docs-gaps.md; append the 2026-08-07
change-log row recording what shipped, the pre-fix red for GWC-28, and
the TST-28 mutation check.
2026-08-07 06:15:45 -04:00
Joseph Doherty eeee3e48a3 fix(GWC-30): reuse the frame reader's length-prefix scratch buffer
ReadAsync allocated a fresh 4-byte array per inbound frame; the GWC-08
pass pooled the payload buffer but left the prefix. Replaced with a
per-instance scratch field — the reader is single-consumer by
construction (one read loop per WorkerClient, handshake reads complete
before the loop starts), so a per-instance buffer is safe and the
non-reentrancy that makes it safe is now stated on the class. Pooling
four bytes via ArrayPool would cost more than the allocation it saves.

Tests: WorkerFrameProtocolTests.ReadAsync_WithMultipleFramesOnOneReader_
ParsesEveryFrame reads five frames of differing payload length through
one reader, so a stale prefix carried between calls would misparse.
2026-08-07 06:15:39 -04:00
Joseph Doherty a044f92c5d fix(GWC-29): drop the wasted request clone on the Invoke hot path
Invoke deep-cloned the whole MxCommandRequest — including its command
payload, potentially a large bulk-write graph — only to overwrite the
cloned command with commandToInvoke and discard it. MapCommand then did
the one clone actually needed. Net cost: a full wasted command deep-clone
per Invoke, worst for exactly the bulk writes that are largest.

Adds a MapCommand(MxCommand) overload (MapCommand reads nothing else off
the request) and has Invoke pass commandToInvoke directly; the request
overload delegates so other callers are untouched.

The remaining clone inside MapCommand stays and is now documented as
required rather than incidental: commandToInvoke may be the gRPC-owned
request.Command, and the caller reads it again after dispatch via
TrackCommandReply, so ownership transfer (à la GWC-07) is not safe here.
That clone is what keeps WorkerClient.CreateCommandEnvelope's no-aliasing
invariant true.

Tests: MxAccessGrpcMapperTests.MapCommandFromCommandClonesPayload
(mutating the input leaves the mapped command untouched; both overloads
produce equal results under a fixed TimeProvider).
2026-08-07 06:15:31 -04:00
Joseph Doherty f27eb28063 fix(GWC-28): stamp gateway envelope sequence at write, not construction
CreateEnvelope stamped Sequence with an interlocked increment when the
envelope was built, so two concurrent InvokeAsync callers could take 1
and 2 and then enqueue in the order 2, 1 — non-monotonic on the wire,
breaking gateway.md's "monotonic per sender" contract. Benign today
(neither side validates inbound sequence, old GWC-10 still open) but it
would fault healthy sessions the moment worker-side validation lands.

WriteLoopAsync now stamps immediately before _writer.WriteAsync. It is
the outbound channel's single consumer (SingleReader = true), so wire
order and stamp order are the same thing by construction and
_nextSequence drops to a plain ulong with no interlocking. This mirrors
the worker's WRK-04 fix, which the gateway half never received.

Also adds TST-28: a [Theory] pinning that GatewayHello.MaxFrameBytes
carries the configured worker-frame maximum (default + 2 MiB override).
The adoption half is asserted only in the Windows-only worker suite, so
a regression to sending 0 — "older gateway, use default" to the worker —
would silently downgrade the negotiated IPC-02 limit with every CI test
still green. Mutation-checked (hard-coded 0 fails both cases).

Tests: WorkerClientTests.ConcurrentInvokesEmitStrictlyIncreasingSequences
OnTheWire (32 parallel invokes; failed 3/3 pre-fix) and
.StartAsync_SendsGatewayHelloWithConfiguredMaxFrameBytes.
2026-08-07 06:15:20 -04:00
Joseph Doherty 3f854d6cbf Merge branch 'fix/sec-31-32-limiter'
ci / windows-x86 (push) Successful in 1m21s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m23s
ci / portable (push) Successful in 9m4s
# Conflicts:
#	archreview/2026-07-12/remediation/00-tracking.md
2026-08-07 06:13:14 -04:00
Joseph Doherty 5b681ee59b fix(SEC-31,SEC-32): identify a probe-slot reservation by version, not by timestamp
ReleaseProbe recognised its own reservation by comparing NextProbeAtTicks to
now + _probeIntervalTicks. RecordInto's rearm-on-trip writes that identical
expression, so a concurrent RecordFailure on the same WindowState whose `now`
lands on the claimer's tick — routine at ~1 ms clock resolution under load — was
mistaken for the caller's own claim. The release then stomped the legitimate
fresh re-arm back to the stale previousProbeAtTicks, which is already due, handing
the next arrival a free probe the re-arm had just closed.

WindowState gains a monotonic ProbeVersion bumped by every writer of
NextProbeAtTicks (TryConsumeProbe's claim and RecordInto's re-arm alike).
TryConsumeProbe returns the stamp it set as part of a ProbeClaim; ReleaseProbe
restores the previous value only while the state's version still equals that
stamp, checking and restoring in one lock(state) section and bumping the version
again on restore so no other stale release can match either.

Test: ProbeSlotRestore_DoesNotStompConcurrentRearmAtSameTick, with the clock held
still so the claim and the interleaved failure necessarily share a tick. Making it
deterministic needed a seam — the claim-to-release window is a few nanoseconds and
racing threads do not hit it (an earlier thread-based attempt passed against the
defective guard three runs out of three, and its end state was ordering-dependent
rather than correctness-dependent, so it was dropped rather than shipped as
theatre). The seam is an internal ProbeReleaseInterleaveHook, null in production,
costing one null check on the already-refused path. Verified as a genuine red
against the timestamp guard: Expected ThrottledByPeer, Actual ProbeAdmitted.
2026-08-07 06:10:14 -04:00
Joseph Doherty acebe18773 fix(SEC-31,SEC-32): make probe admission atomic and stop Reset clearing a shared fallback partition
Two defects found in code review of the limiter rework.

Probe admission was check-then-act across two lock scopes: Check() read
"probe due" under lock(state), released it, then re-acquired to advance
NextProbeAtTicks. A burst of requests arriving together at an interval boundary
could therefore all observe the slot as due and all be admitted, handing the
verifier the very burst the interval exists to bound. The claim is now a single
critical section (TryConsumeProbe). The two layers are still claimed one at a
time — holding two per-state locks at once would need a global lock ordering to
stay deadlock-free — so a slot claimed on the composite partition is compensated
via ReleaseProbe when the aggregate then refuses, which otherwise silently spent
the partition's next slot and pushed the legitimate holder out by a full
interval.

Reset() removed whatever partition the caller resolved to, including the
address's shared fallback partition when the caller's key id had been collapsed
into it by the per-peer cap (or when the token was junk-shaped). That bucket also
carries failures contributed by other key ids from the same address, so one
successful authentication became a reset button for an in-progress spray. Reset
now clears only a partition the caller owns (effectiveKeyId == presented key id);
the shared bucket decays by window expiry instead, and the caller still recovers
through probe admission. The key's aggregate is cleared either way, as designed.

Also applied from the review: closure-free GetOrAdd overload on _partitions, and
a remarks paragraph acknowledging the best-effort O(n) eviction scan under
sustained overflow. Threading the resolved partition key from Check through to
RecordFailure/Reset was declined: Check resolves with mint:false and RecordFailure
with mint:true, and the two can legitimately differ when a concurrent caller fills
the per-peer cap in between — reusing Check's key would record into the wrong
partition and bypass the cap, which is not worth saving one string concat.

Tests (limiter suite 11 -> 14): ProbeAdmission_UnderConcurrentArrivals_
GrantsExactlyOneSlot (200 rounds x 8 barrier-released threads at the boundary),
ProbeAdmission_WhenAggregateRefuses_ReturnsTheClaimedPeerSlot, and
Reset_WithOverCapKeyId_DoesNotClearSharedFallbackPartition. The latter two were
confirmed as genuine reds against the unfixed code; the concurrency test is a
guard — it is deterministically green on the fixed structure but did not
reproduce the original nanosecond-wide window on its own.
2026-08-07 05:57:26 -04:00
Joseph Doherty df710e18a9 fix(SEC-31,SEC-32): re-partition the API-key failure limiter on (peer, key id) with probe admission
The gRPC auth failure limiter partitioned on the key id parsed out of the
*unauthenticated* token and rejected with ResourceExhausted before VerifyAsync
ran. Key ids are not secret — they ride in every token and are listed on the
dashboard — so any network peer could send 10 garbage-secret requests per minute
and deny that key indefinitely: the legitimate holder's correct secret was
refused before it was ever checked, and the success-path Reset that would clear
the block sat behind the verification the block prevented (SEC-31). The tracked
map was also flushable — any `a_b_c`-shaped junk minted a fresh partition (the
`mxgw` literal was never compared), so ~4096 throwaway tokens evicted a blocked
entry and reset the window (SEC-32).

ApiKeyFailureLimiter moves from IsBlocked/RecordFailure/Reset(string peer) to a
partition-pair API: Check/RecordFailure/Reset(ApiKeyThrottlePartition) with an
ApiKeyThrottleDecision result. Two layers share one sliding window — a composite
(transport peer, key id) partition at ApiKeyFailureLimit, and a per-key-id
aggregate across all peers at the new ApiKeyFailureAggregateLimit (default 30)
that bounds a source-rotating sprayer. An over-limit state is now a valve rather
than a wall: one request per the new ApiKeyFailureProbeIntervalSeconds (default
5) is admitted through to the real verifier, so the correct secret always reaches
the constant-time compare and resets both layers. Guarantees preserved: guessing
stays bounded per window, and the failure path still spends no store read per
attempt.

SEC-32 rides the same change set: the interceptor validates token shape (literal
`mxgw` prefix, >= 3 non-empty `_` segments, key id <= 64 chars) before minting a
key-id partition, each transport peer may mint at most 32 of them before the
overflow collapses onto its fallback partition, and eviction prefers fully
expired windows and never drops an over-limit partition below a 2x transient
overshoot ceiling. Throttled attempts increment mxgateway.auth.throttled, tagged
stage=peer|aggregate only — /metrics is unauthenticated (open SEC-14), so no key
material may appear there.

Docs in the same commit: GatewayConfiguration limiter rows plus the two new keys,
the Authentication hot-path paragraph, the Authorization SEC-11 section, and the
limiter / SecurityOptions XML remarks (the old NAT rationale described the
defective keying). Tracking rows flipped to Done with a change-log entry.

Tests: new ApiKeyFailureLimiterTests (11) covering window pruning, composite vs
aggregate trip points, probe cadence, absolute-block mode, reset across both
layers, junk-spray eviction resistance, the per-peer cap, and expired-window
eviction preference; GatewayGrpcAuthorizationInterceptorTests gains the four
SEC-31 contract tests plus NonMxgwToken_FallsBackToTransportPeerPartition (20
total); GatewayOptionsValidatorTests covers both new keys including 0 as a
supported disable value (66 total).
2026-08-07 05:39:10 -04:00
Joseph Doherty 33ba612ddd fix(WRK-21,WRK-28,WRK-23,IPC-30): byte-budget the DrainEvents reply, stop size errors from killing sessions
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m8s
ci / portable (push) Successful in 7m41s
ci / windows-x86 (push) Failing after 12m32s
WRK-21 — DrainEvents was bounded by event count only, so a byte-heavy queue
(large string/array MxValues) built a reply above the negotiated frame maximum:
the writer rejected the frame, the exception unwound the session, and the events
already dequeued were destroyed. The drain is now byte-budgeted inside the queue
lock, so an event is dequeued only once it is known to fit and one that does not
stays at the head. Truncation is reported through the reply's existing
DiagnosticMessage (no contract change); callers drain until an empty reply. Both
reply-write seams — the control-command path and ProcessCommandAsync — now catch
MessageTooLarge and answer the correlation with an InvalidRequest reply instead
of unwinding or faulting the session. Satisfies IPC-23 R1-R3.

WRK-28 — the 10,000 drain ceiling moves to GatewayContractInfo
.MaxDrainEventsPerCommand, referenced by both the gateway request validator and
the worker clamp, replacing a comment-only sync contract. C# const only; no
.proto change.

WRK-23 — WorkerFrameWriter now peek-stamps, validates, then commits the sequence
counter immediately before the stream write, so a per-frame rejection leaves no
phantom gap on the wire.

IPC-30 — an oversized event frame stays session-fatal (it is undeliverable end to
end and neither dropping nor synthesizing a replacement is allowed), but the
death is structured: the event's identity and sizes are logged (never its value),
a WorkerFault with category PROTOCOL_VIOLATION and command method EventDrain is
written, then the session exits as before.

Docs updated in the same change: MxAccessWorkerInstanceDesign.md (drain byte cap,
truncation contract, oversized-head behavior, oversized-event policy, no control
reply is session-fatal on size), WorkerFrameProtocol.md (reply pre-sizing,
non-fatal reply-size rule, oversized-event policy, rejected frames do not consume
sequence numbers), gateway.md (DrainEvents two-axis bound).
2026-08-07 05:38:23 -04:00
145 changed files with 8321 additions and 680 deletions
+22 -11
View File
@@ -60,7 +60,19 @@ jobs:
dotnet tool install --global PowerShell
echo "$HOME/.dotnet/tools" >> "$GITHUB_PATH"
# IPC-01 / IPC-19 / IPC-20: descriptor set + Contracts/Generated must match the current protos.
# IPC-25 Check 4 regenerates the Go and Python client bindings and diffs them, so the pinned
# generators must be present. protoc 34.1 is already installed above; Go and Python are set up
# above. Pin protoc-gen-go / protoc-gen-go-grpc to match the committed header stamps and grpcio
# -tools to match the committed _pb2 stamp, or Check 4 false-fails (or masks drift) under churn.
- name: Install pinned client codegen generators (Check 4)
run: |
go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.6.2
echo "$(go env GOPATH)/bin" >> "$GITHUB_PATH"
python -m pip install 'grpcio-tools==1.80.0'
# IPC-01 / IPC-19 / IPC-20 / IPC-25: descriptor set + Contracts/Generated + Go/Python bindings
# must match the current protos.
- name: Codegen / descriptor freshness
shell: pwsh
run: ./scripts/check-codegen.ps1
@@ -93,10 +105,12 @@ jobs:
python -m pytest
java:
# Java client runs on a JDK-17 Linux runner (the macOS dev box has no JRE). The protobuf gradle
# plugin rewrites MxaccessGateway.java with spurious protobuf-runtime-version churn on every
# build; when no .proto changed, revert that one file so checkGeneratedClean / a dirty tree does
# not fail the build (repo memory project_java_generated_churn).
# Java client runs on a JDK-17 Linux runner (the macOS dev box has no JRE). The grpc/protobuf
# toolchain is fully pinned (clients/java/build.gradle: grpcVersion 1.76.0 / protobufVersion
# 4.33.1), so a regeneration is byte-identical to the committed aggregates modulo real .proto
# changes — `Verify generated tree is clean` (git diff) is the true drift gate (IPC-24). The
# single-file Java aggregates are where message-level proto drift lands, so this job now catches
# a .proto edited without regenerating and committing the Java client.
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
@@ -115,13 +129,10 @@ jobs:
- name: Gradle test
working-directory: clients/java
run: gradle test
- name: Revert spurious protobuf-version churn (no .proto changed)
# Both generated aggregates can pick up protobuf-runtime-version churn on regen; revert
# both so verify-clean still catches a real, uncommitted proto/codegen change elsewhere.
run: |
git checkout -- clients/java/src/main/generated/main/java/mxaccess_gateway/v1/MxaccessGateway.java || true
git checkout -- clients/java/src/main/generated/main/java/mxaccess_worker/v1/MxaccessWorker.java || true
- name: Verify generated tree is clean
# IPC-24: the pinned grpc/protobuf toolchain regenerates byte-identical output, so this
# git-diff gate now catches message-level proto drift in the single-file Java aggregates
# (the old unconditional churn-revert step masked exactly that class and was deleted).
run: git diff --exit-code -- clients/java/src/main/generated
windows-x86:
+14 -7
View File
@@ -156,8 +156,11 @@ accept a `browseSubtreeGlobs` param, so either fix is small plumbing:
Delete mxaccessgw's own `Galaxy/GalaxyRepositoryServiceCollectionExtensions.cs` registrations.
4. **Option validation** — the shared lib **binds only, ships no validator** (deliberate). mxaccessgw
already validates Galaxy options via `Configuration/GatewayOptionsValidator.cs` — **keep that**; it
stays the owner of fail-fast validation, exactly as HistorianGateway's `ConfigPreflight` does.
stays the owner of fail-fast Galaxy validation. **Updated 2026-08-07 (SEC-33):** this is now a dedicated
`Configuration/GalaxyRepositoryOptionsValidator.cs` (registered as `IValidateOptions<GalaxyRepositoryOptions>`
with `ValidateOnStart`), which enforces a valid, host-rooted `SnapshotCachePath` when `PersistSnapshot`
is true — exactly as HistorianGateway's `ConfigPreflight` does. (The original handoff pointed at
`GatewayOptionsValidator.cs`, but that validator does not see the lib-bound `GalaxyRepositoryOptions`.)
5. **Health check** — keep mxaccessgw's existing Galaxy-SQL readiness check; read the connection
string from the same `MxGateway:Galaxy` section the lib binds (HistorianGateway does this with a raw
@@ -189,11 +192,15 @@ be **deleted**. **Keep** the mxaccessgw-specific ones that exercise behavior the
## Post-adoption notes / caveats
- **Deployment config (NSSM):** the deployed services (`MxAccessGw` on 10.100.0.48; the wonder host) read
config from **NSSM environment variables, not `appsettings.json`**. The lib's `SnapshotCachePath` default
is empty (persistence no-ops). `appsettings.json` sets `MxGateway:Galaxy:SnapshotCachePath` +
`PersistSnapshot`, but the deployments must carry `MxGateway__Galaxy__SnapshotCachePath` and
`MxGateway__Galaxy__PersistSnapshot` in their NSSM env on redeploy, or snapshot persistence silently
no-ops in production.
config from **NSSM environment variables, not `appsettings.json`**. **Updated 2026-08-07 (SEC-33):** the
lib's own `SnapshotCachePath` default is empty (would no-op persistence), but mxaccessgw no longer relies
on it. `appsettings.json` no longer sets `SnapshotCachePath` at all; instead the gateway seeds a
`CommonApplicationData`-derived default (`C:\ProgramData\MxGateway\galaxy-snapshot.json` on the Windows
hosts) when the bound value is blank, and `GalaxyRepositoryOptionsValidator` fails startup if
`PersistSnapshot` is true with a non-rooted/invalid path. So `MxGateway__Galaxy__SnapshotCachePath` in the
NSSM env is now **optional** (an override), not required — a deployment that omits it gets the rooted host
default and persistence works; it no longer silently no-ops. `MxGateway__Galaxy__PersistSnapshot` still
governs whether persistence runs at all.
- **Pre-existing NU1903 (unrelated) — ✅ RESOLVED (2026-07-18, commit `2f0cfe3`):** adding the package surfaced a transitive `SQLitePCLRaw.lib.e_sqlite3`
2.1.11 advisory (GHSA-2m69-gcr7-jv3q, at the time no upstream patch) that breaks the build under `TreatWarningsAsErrors`
— already red on `main`. Initially resolved with a targeted `NuGetAuditSuppress` in `src/Directory.Build.props`
File diff suppressed because one or more lines are too long
@@ -12,9 +12,9 @@ This document turns the 2026-07-12 re-review's **new** Gateway Server Core findi
| GWC-25 | Medium | P0 | S | CLI-35/36 (coord) | Done | Empty-ring ReplayGap sentinel carries `oldest_available_sequence = 0`, dead-streaming a compliant client |
| GWC-26 | Low | P2 | M | GWC-27 | Done | Alarm monitor attaches its subscriber after SubscribeAlarms; window transitions bypass the feed, missed Acknowledge never repaired |
| GWC-27 | Low | P2 | S | GWC-26 | Done | `AttachInternalEventSubscriber` bypasses the readiness gate; premature attach poisons the distributor permanently |
| GWC-28 | Low | P2 | S | GWC-10 (coord) | Not started | Gateway→worker envelope `sequence` stamped at creation, not at write — non-monotonic on the wire under concurrent invokes |
| GWC-29 | Low | — | S | — | Not started | `Invoke` deep-clones the entire request only to discard the cloned command |
| GWC-30 | Info | — | S | — | Not started | Frame reader allocates a fresh 4-byte length-prefix array per frame |
| GWC-28 | Low | P2 | S | GWC-10 (coord) | Done | Gateway→worker envelope `sequence` stamped at creation, not at write — non-monotonic on the wire under concurrent invokes |
| GWC-29 | Low | — | S | — | Done | `Invoke` deep-clones the entire request only to discard the cloned command |
| GWC-30 | Info | — | S | — | Done | Frame reader allocates a fresh 4-byte length-prefix array per frame |
Dependency notes: GWC-26 and GWC-27 both change the internal-subscriber attach path (`GatewaySession.AttachInternalEventSubscriber` and its `SessionManager`/alarm-monitor callers) — land GWC-27's readiness gate first (or in the same commit), then GWC-26's reorder, so the reordered monitor attach is proven against the gate. GWC-24 is the direct successor of the prior cycle's GWC-04 backpressure fix and raises the value of the still-open GWC-21 (making `EventChannelFullModeTimeout` configurable); GWC-28 is the gateway half of the worker's WRK-04 fix and must be coordinated with the still-open GWC-10 if inbound sequence enforcement is ever added. GWC-25 is server-complete on its own, but the end-to-end reconnect story also needs the client-domain CLI-35/36 fixes (Python CLI crashes on the sentinel, Go CLI destroys it).
@@ -61,7 +61,7 @@ Coordinate with (do not block on) open GWC-21: if `EventChannelFullModeTimeout`
The proto comment currently states "`oldest_available_sequence` itself IS still retained", which becomes false in the empty-ring case — per the docs-with-source rule, amend the field comment in the same commit to define the empty-ring value ("when nothing is retained, this is the next sequence that can be delivered — `highest observed + 1` — and the `oldest − 1` resume formula remains valid; the interval evicted is unchanged"). This is a comment-only proto change (no descriptor delta), but the repo's codegen rules still apply — see the steps.
**Implementation.**
**Implementation.** (Code + `docs/Sessions.md` landed 2026-08-07 on `fix/gwc-25-replaygap-trio`; the deferred proto-comment amendment below **landed 2026-08-07** with the IPC-23 codegen wave on `fix/ipc-24-25-codegen` — GWC-25 is fully resolved.)
- `Sessions/SessionEventDistributor.cs:463-467`: replace `oldestAvailableSequence = 0;` with `oldestAvailableSequence = gap ? _highestSequenceSeen + 1 : 0;` plus a comment explaining the `oldest − 1` client formula this must keep valid (cite this finding).
- `src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_gateway.proto` (`ReplayGap.oldest_available_sequence`, ~line 759): append the empty-ring sentence above. Then regenerate per repo rules: delete `src/ZB.MOM.WW.MxGateway.Contracts/Generated/*.cs`, `dotnet build src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csproj`, and **commit `Generated/`** (net48 worker builds break otherwise). Sync the vendored client copies of the proto byte-identical (`clients/*/`); a comment-only edit changes no descriptor, so: Python `*_pb2*` output is unchanged (comments are not embedded — regenerate with the pinned grpcio-tools only if the files actually differ), Go/C#/Rust generated doc comments will churn — regenerate those per each client README, and revert spurious Java aggregate-file churn if no message-level delta appears (per the established Java convention).
- `docs/Sessions.md` (~lines 228-234, ReplayGap section): document the empty-ring sentinel value and that `after_worker_sequence = oldest_available_sequence − 1` is the universal resume formula in both the retained and fully-evicted cases.
@@ -16,14 +16,14 @@ members, no positional records). The worker builds and tests only on the Windows
| ID | Sev | Tier | Eff | Dep | Status | Title |
|----|-----|------|-----|-----|--------|-------|
| WRK-21 | Medium | P0 | M | IPC-23 (same defect, fix owned here); WRK-28 (same lines) | Not started | DrainEvents bound is count-based only; an oversized reply still kills the session and loses the drained events |
| WRK-22 | Low | — | S | IPC-26 (same defect, fix owned here) | Not started | Cancelled `WriteAsync` leaves its frame queued; it is still written later |
| WRK-23 | Low | — | S | WRK-21 (rejection path becomes backstop-only) | Not started | Rejected frames consume sequence numbers, producing wire gaps |
| WRK-24 | Low | — | S | — | Not started | `AdoptNegotiatedMaxMessageBytes` has no lower-bound sanity check |
| WRK-25 | Low | P2 | S | WRK-22 (both touch enqueue/dequeue) | Not started | WRK-12 flush coalescing never engages on the event hot path |
| WRK-26 | Low | P1 | S | WRK-23 (soft — sequence prose); discharges IPC-29 | Not started | Write-priority and overflow doc drift from the WRK-07 change |
| WRK-27 | Low | — | S | — | Not started | Alarm poll bypasses the watchdog's in-flight suppression (15 s vs 75 s) |
| WRK-28 | Low | — | S | WRK-21 (land in the same commit cluster) | Not started | 10,000 drain cap is a duplicated magic constant with a comment-only sync contract |
| WRK-21 | Medium | P0 | M | IPC-23 (same defect, fix owned here); WRK-28 (same lines) | Done | DrainEvents bound is count-based only; an oversized reply still kills the session and loses the drained events |
| WRK-22 | Low | — | S | IPC-26 (same defect, fix owned here) | Done | Cancelled `WriteAsync` leaves its frame queued; it is still written later |
| WRK-23 | Low | — | S | WRK-21 (rejection path becomes backstop-only) | Done | Rejected frames consume sequence numbers, producing wire gaps |
| WRK-24 | Low | — | S | — | Done | `AdoptNegotiatedMaxMessageBytes` has no lower-bound sanity check |
| WRK-25 | Low | P2 | S | WRK-22 (both touch enqueue/dequeue) | Done | WRK-12 flush coalescing never engages on the event hot path |
| WRK-26 | Low | P1 | S | WRK-23 (soft — sequence prose); discharges IPC-29 | Done | Write-priority and overflow doc drift from the WRK-07 change |
| WRK-27 | Low | — | S | — | Done | Alarm poll bypasses the watchdog's in-flight suppression (15 s vs 75 s) |
| WRK-28 | Low | — | S | WRK-21 (land in the same commit cluster) | Done | 10,000 drain cap is a duplicated magic constant with a comment-only sync contract |
---
@@ -12,16 +12,16 @@ All `path:line` citations were re-verified against the working tree at `4f5371f`
| ID | Sev | Tier | Eff | Dep | Status | Title |
|----|-----|------|-----|-----|--------|-------|
| IPC-23 | Medium | P0 | S¹ | WRK-21 | Not started | DrainEvents bound is count-based only; byte-heavy queue still builds a session-killing reply frame (contract requirements here; fix mechanics in WRK-21) |
| IPC-24 | Medium | P0 | S | — | Not started | CI's unconditional Java churn-revert masks real generated-code drift for message-level proto changes |
| IPC-25 | Medium | P0 | M | — | Not started | Committed Go/Python worker bindings are stale at HEAD; no guard covers them |
| IPC-26 | Low | P2 | S¹ | WRK-22 | Not started | Cancelled write leaves a ghost frame that is still written (contract requirement here; fix mechanics in WRK-22) |
| IPC-27 | Low | P2 | S | — | Not started | Descriptor freshness test blind to enums, enum values, services/methods, and the Galaxy contract |
| IPC-28 | Low | — | S | — | Not started | `docs/Grpc.md` omits the `CommandTooLarge` → `ResourceExhausted` mapping |
| IPC-29 | Low | — | S | — | Not started | Worker writer priority scheduling and write-time sequence stamping undocumented in the frame-protocol doc |
| IPC-30 | Low | P0 | M | WRK-21 (same file/batch) | Not started | Oversized worker→gateway event frame is session-fatal — make the death deliberate, structured, and diagnosable |
| IPC-23 | Medium | P0 | S¹ | WRK-21 | Done | DrainEvents bound is count-based only; byte-heavy queue still builds a session-killing reply frame (contract requirements here; fix mechanics in WRK-21) |
| IPC-24 | Medium | P0 | S | — | Done | CI's unconditional Java churn-revert masks real generated-code drift for message-level proto changes |
| IPC-25 | Medium | P0 | M | — | Done | Committed Go/Python worker bindings are stale at HEAD; no guard covers them |
| IPC-26 | Low | P2 | S¹ | WRK-22 | Done (mechanics landed in WRK-22) | Cancelled write leaves a ghost frame that is still written (contract requirement here; fix mechanics in WRK-22) |
| IPC-27 | Low | P2 | S | — | Done | Descriptor freshness test blind to enums, enum values, services/methods, and the Galaxy contract |
| IPC-28 | Low | — | S | — | Done | `docs/Grpc.md` omits the `CommandTooLarge` → `ResourceExhausted` mapping |
| IPC-29 | Low | — | S | — | Done (discharged by WRK-26) | Worker writer priority scheduling and write-time sequence stamping undocumented in the frame-protocol doc |
| IPC-30 | Low | P0 | M | WRK-21 (same file/batch) | Done | Oversized worker→gateway event frame is session-fatal — make the death deliberate, structured, and diagnosable |
| IPC-31 | Info | — | — | — | N/A | Gateway stamps sequence at creation, worker at write — accepted divergence; sequence is documented diagnostic-only (`gateway.md:328-330`); revisit only if sequence ever becomes load-bearing |
| IPC-32 | Info | — | S | IPC-25 | Not started | `check-codegen.ps1` check labels miscounted (folded into the IPC-25 script edit) |
| IPC-32 | Info | — | S | IPC-25 | Done | `check-codegen.ps1` check labels miscounted (folded into the IPC-25 script edit) |
¹ Effort for the work owned by *this* plan (proto comments + docs + acceptance criteria). The code mechanics are M and are tracked under WRK-21 / WRK-22 in the worker plan.
@@ -10,12 +10,12 @@ Repo rules that bind every entry: docs change in the same commit as the source (
| ID | Sev | Tier | Eff | Dep | Status | Title |
|----|-----|------|-----|-----|--------|-------|
| SEC-31 | Medium | P0 | M | — | Not started | Failure limiter partitions on attacker-controlled key id and blocks before verification (lockout DoS) |
| SEC-32 | Low | P0 | S | SEC-31 | Not started | Failure-limiter LRU is flushable by junk-token spray; token prefix never validated |
| SEC-33 | Low | P1 | M | — (co-locate SEC-23) | Not started | Any-platform path-rooting acceptance re-opens SEC-01 on Unix; Galaxy `SnapshotCachePath` unvalidated |
| SEC-34 | Low | P2 | S | — | Not started | Verification cache: expiry outlives TTL; `Invalidate` races in-flight repopulation |
| SEC-35 | Info | — | S | — | N/A (doc-only note) | Production hard-stops key on the exact `Production` environment name |
| SEC-36 | Low | P1 | M | cross-repo (`scadaproj/infra/glauth`) | Not started | Committed dev LDAP service-account password: remove from repo and rotate |
| SEC-31 | Medium | P0 | M | — | Done | Failure limiter partitions on attacker-controlled key id and blocks before verification (lockout DoS) |
| SEC-32 | Low | P0 | S | SEC-31 | Done | Failure-limiter LRU is flushable by junk-token spray; token prefix never validated |
| SEC-33 | Low | P1 | M | — (co-locate SEC-23) | Done | Any-platform path-rooting acceptance re-opens SEC-01 on Unix; Galaxy `SnapshotCachePath` unvalidated |
| SEC-34 | Low | P2 | S | — | Done | Verification cache: expiry outlives TTL; `Invalidate` races in-flight repopulation |
| SEC-35 | Info | — | S | — | N/A (doc-only note discharged 2026-08-07) | Production hard-stops key on the exact `Production` environment name |
| SEC-36 | Low | P1 | M | cross-repo (`scadaproj/infra/glauth`) | Done (repo-side; live rotation operator-pending per runbook) | Committed dev LDAP service-account password: remove from repo and rotate |
---
@@ -135,6 +135,8 @@ dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --fil
```
Post-run, verify no new `C:\*` file exists under any `bin/` (manual `find src -name 'C:*'`).
**Outcome (2026-08-07 — Done).** Implemented as designed. `IsRootedForAnyPlatform` deleted; `AddIfNotRooted`/`AddIfInvalidPath` promoted to a shared `GatewayConfigPathRules` internal helper (used by both validators) and now use `Path.IsPathRooted` (current OS). Both Windows literals removed from `appsettings.json`; the Galaxy `SnapshotCachePath` default is applied gateway-side via a configuration value seeded before `AddZbGalaxyRepository` (the package's `SnapshotCachePath` is **init-only**, so a `PostConfigure` mutation does not compile — deviation from the design's "PostConfigure default"; same effect). New `GalaxyRepositoryOptionsValidator` registered with `ValidateOnStart`. **Stray-file root cause:** starting the full host eagerly constructs `AuthSqliteConnectionFactory`, which creates the auth DB path; with the shipped Windows literal that path is non-rooted on macOS, so SQLite materialized `C:\ProgramData\MxGateway\gateway-auth.db` as a junk-named relative file under the test's `bin/` CWD (invisible to the hygiene test's bin/obj filter). After the literal removal the code default resolves under an unwritable `/usr/share` on macOS, so the three tests that start the real host (`GatewayApplicationTests.Build_MapsMetricsEndpoint`, `.StartAsync_InvalidGatewayConfiguration_FailsStartup`, `GatewayTlsBootstrapTests`) now pin `SqlitePath` to a temp path. No stray file remains (`find src -name 'C:*'` empty).
---
## SEC-34 — Verification cache: expiry outlives TTL; `Invalidate` races in-flight repopulation `Low` · `P2`
@@ -165,6 +167,8 @@ Post-run, verify no new `C:\*` file exists under any `bin/` (manual `find src -n
dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter "FullyQualifiedName~CachingApiKeyVerifier"
```
**Outcome (2026-08-07 — Done).** Window 3 (Invalidate race) implemented exactly as designed: per-key generation counter, bump-before-evict in `Invalidate`, snapshot-before-inner + set-then-recheck in `VerifyAsync`, key id parsed from the token up front. Covered by `Invalidate_DuringInFlightVerification_DiscardsStaleRepopulation`. **Window 2 (expiry cap) took the documented fallback**, not the cap: the design's confirmation step failed — the library verification identity (`ZB.MOM.WW.Auth.Abstractions.ApiKeys.ApiKeyIdentity`, the type on `ApiKeyVerification.Identity`) carries **no** `ExpiresUtc` (that property is on `ApiKeyRecord`, the store row, not the returned identity), so the cache cannot cap an entry at the key's expiry. Per the design's contingency, the ≤ TTL expiry window is documented in the class remarks and `docs/Authentication.md`, with a donor-library ask (surface expiry on the verification identity). Consequently the two expiry-cap tests (`CacheEntry_DoesNotOutliveKeyExpiry`, `AlreadyExpiredIdentity_IsNotCached`) are **not** added — they cannot be written against a type with no expiry field; window 1 (CLI) accepted and documented as before.
---
## SEC-35 — Production hard-stops key on the exact `Production` environment name `Info` · `—` (N/A: doc-only)
@@ -179,6 +183,8 @@ dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --fil
**Verification.** Doc-only; no build. Cross-read against `GatewayOptionsValidator.cs:23-27`.
**Outcome (2026-08-07 — discharged).** The documentation contract landed as a rider on the SEC-33/34 commit: `docs/GatewayConfiguration.md` gained a "Production hard-stops key on the exact environment name (SEC-35)" subsection stating that both hard-stops fire only on `IHostEnvironment.IsProduction()` (unset `ASPNETCORE_ENVIRONMENT` or the exact `Production` name) and that any other name keeps the permissive dev posture. No code change, as designed.
---
## SEC-36 — Committed dev LDAP service-account password: remove from repo and rotate `Low` · `P1` · cross-repo dependency
@@ -212,3 +218,5 @@ dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --fil
dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter "FullyQualifiedName~GatewayOptionsValidator"
```
(asserts the blank-password validation still fires with the updated message). Manual: with user-secrets set on the dev box, `dotnet run --project src/ZB.MOM.WW.MxGateway.Server/...` and a dashboard `/login` as `multi-role` succeeds against the rotated GLAuth; the deployed-host login re-check from step 1 counts as the production verification. Live-LDAP integration tests (`MXGATEWAY_RUN_LIVE_LDAP_TESTS=1`) only where the GLAuth instance is reachable; otherwise document skipped per the testing matrix.
**Outcome (2026-08-07 — Done, repo-side; live rotation operator-pending).** Landed on `fix/sec-36-ldap-secret`. **The design's baseline had already shifted:** at HEAD `appsettings.json` no longer commits the literal — it ships `"ServiceAccountPassword": "${secret:ldap/mxgateway/bind}"`, a fail-closed encrypted-store reference (documented `GatewayConfiguration.md:252`, tested by `PreHostSecretExpansionTests`) introduced by the Secrets-store adoption after this remediation was written. **Deviation from Implementation step 2:** the `${secret:}` reference was **kept, not deleted** — deleting it regresses the shipped/documented/tested store channel and the committed-plaintext finding is already resolved for `appsettings.json`. The load-bearing residual — the literal value still present in `glauth.md`'s samples (`:33,65,103,136,245`), `docs/GatewayTesting.md`, and the historical `archreview/*` SEC-06 evidence — was scrubbed to `<service-account-password>` placeholders, each with a pointer to the source of truth `scadaproj/infra/glauth/` and a rotation-required note. Steps 3–6 implemented as designed: `<UserSecretsId>mxaccessgw-server</UserSecretsId>` added (step 3); the `ValidateLdap` blank-password message now names both channels — dev `dotnet user-secrets set "MxGateway:Ldap:ServiceAccountPassword" <value>` and deployed `MxGateway__Ldap__ServiceAccountPassword` — plus a note on the `${secret:}` store default (step 4), asserted by the extended `Validate_Fails_WhenLdapEnabledAndServiceAccountPasswordBlank`; docs updated same commit (step 5); `git grep -i` for the old value is empty across tracked files (step 6). The cross-repo **step 1 (rotate GLAuth on `10.100.0.35`, pre-stage the NSSM env var on `10.100.0.48` and on `wonder-app-vd03` only if `Ldap.Enabled`, verify dashboard login)** is the operator's to execute, captured in the new runbook `docs/runbooks/SEC-36-ldap-credential-rotation.md`. Verification (macOS): `dotnet build …Server` 0 warnings/0 errors; `dotnet test --filter ~GatewayOptionsValidator` green.
@@ -20,12 +20,12 @@ Operating constraints carried from prior work:
| CLI-36 | Medium | P0 | S | — | Done | Go CLI `stream-events` silently destroys the ReplayGap signal |
| CLI-37 | Medium | P1 | M | CLI-38 | Done | Status-array validation must branch on `category` per the proto contract (4-vs-1 divergence) |
| CLI-38 | Medium | P1 | S | — | Done | Align .NET/Go/Java on `hresult < 0` — lands prior CLI-08 and cures the design-doc drift |
| CLI-39 | Medium | P1 | S | CLI-35..38, CLI-45 | Not started | Bump client versions off the already-published 0.1.2 before the next publish; add registry-collision guard |
| CLI-40 | Low | — | M | — | Not started | Port the exact-secret credential scrub to Rust/Java/.NET |
| CLI-41 | Low | — | M | — | Not started | Uniform malformed-reply contract for AuthenticateUser/ArchestrAUserToId/AddBufferedItem |
| CLI-42 | Low | P1 | S | — | Not started | Document the vendored Rust proto layout (CLI-02's missing doc half) |
| CLI-43 | Low | — | S | — | Not started | Java style guide still prescribes "Java 21 preferred" |
| CLI-44 | Low | — | S | — | Not started | Go event goroutine can mislabel a genuine terminal error as `ErrSlowConsumer` |
| CLI-39 | Medium | P1 | S | CLI-35..38, CLI-45 | Done | Bump client versions off the already-published 0.1.2 before the next publish; add registry-collision guard |
| CLI-40 | Low | — | M | — | Done | Port the exact-secret credential scrub to Rust/Java/.NET |
| CLI-41 | Low | — | M | — | Done | Uniform malformed-reply contract for AuthenticateUser/ArchestrAUserToId/AddBufferedItem |
| CLI-42 | Low | P1 | S | — | Done | Document the vendored Rust proto layout (CLI-02's missing doc half) |
| CLI-43 | Low | — | S | — | Done | Java style guide still prescribes "Java 21 preferred" |
| CLI-44 | Low | — | S | — | Done | Go event goroutine can mislabel a genuine terminal error as `ErrSlowConsumer` |
| CLI-45 | Low | P1 | M | — | Done | Standardize CLI credential env-var name and fail fast on missing/empty passwords |
Cross-domain dependencies: **CLI-35/CLI-36 pair with GWC-25** (gateway emits `oldest_available_sequence = 0` on an empty replay ring — the server-side half of the same reconnect story; the CLI fixes here are independently landable but the end-to-end resume walk in the smoke matrix needs both). **CLI-39 pairs with the publishing process** (`scripts/pack-clients.ps1`, `scripts/tag-go-module.ps1`, Gitea package registry).
@@ -12,10 +12,10 @@ Prior-cycle open findings (TST-05..24 where still open) are tracked in the prior
|----|-----|------|-----|-----|--------|-------|
| TST-25 | High | P1 | M | — (unlocks TST-05, TST-24) | Done | Windows/x86 test tier has zero automation — restore via SSH-driven windev CI job |
| TST-26 | Medium | P1 (folded into TST-25) | S | TST-25 | Done | docs/GatewayTesting.md, check-codegen.ps1, and ci.yml comments describe removed CI jobs |
| TST-27 | Medium | P1 (doc batch) | S | — | Not started | `ShowTagValues` config row still says "Reserved" after SEC-25 made the flag live |
| TST-28 | Low | P2 | S | relates IPC-02 | Not started | Gateway-side `max_frame_bytes` handshake field untested in the CI-run suite |
| TST-27 | Medium | P1 (doc batch) | S | — | Done | `ShowTagValues` config row still says "Reserved" after SEC-25 made the flag live |
| TST-28 | Low | P2 | S | relates IPC-02 | Done | Gateway-side `max_frame_bytes` handshake field untested in the CI-run suite |
| TST-29 | Low | P2 | S | — | Done | Retire `oldtasks.md` after folding the Phase-5 governance record into DesignDecisions.md; delete root docs-review artifacts |
| TST-30 | Low | P2 | M | — | Not started | Single shared Gitea runner is a CI throughput/availability bottleneck (cross-repo contention, no run cancel/delete) |
| TST-30 | Low | P2 | M | — | Done (doc half; runner registration operator-pending per runbook) | Single shared Gitea runner is a CI throughput/availability bottleneck (cross-repo contention, no run cancel/delete) |
---
@@ -167,6 +167,8 @@ Independent of the runner count, document the **no-cancel** reality (Gitea 1.26
**Verification.** Push two branches back-to-back and confirm their runs execute concurrently (not serially) once a second runner exists; `GET /repos/dohertj2/mxaccessgw/actions/runners` (or the instance runner list) shows ≥2 runners online; `docs/GatewayTesting.md` describes the shared-runner/no-cancel reality and the bypass. Re-run the TST-25 acceptance push and confirm queue depth is materially lower under a concurrent `lmxopcua` run.
**Outcome (2026-08-07 — Done, doc half; runner registration operator-pending).** Landed on `fix/tst-30-runner-docs`. Implementation step 2 shipped: `docs/GatewayTesting.md`'s Continuous Integration section gained a "Runner capacity is shared and finite" subsection stating the `maxParallel=1` co-located runner is shared with `dohertj2/lmxopcua` at the instance level (not repo-scoped), the ~20–30 minute queue latency observed under cross-repo contention, and the Gitea 1.26 no-cancel/no-delete API reality; the existing "windev tier down" degraded-mode paragraph now also covers "runner contended" as a reason to use the bypass, generalized per this finding's design note. New operator runbook `docs/runbooks/TST-30-second-ci-runner.md` carries **step 1** (register a second `act_runner` on `10.100.0.35`, option (a) recommended, same `container.network: traefik` config; option (b) dedicated labelled runner as an escalation; option (c) windev-hosted runner rejected) with the verification checklist (concurrent back-to-back pushes, `GET /repos/dohertj2/mxaccessgw/actions/runners` ≥ 2) and a note that the no-cancel reality persists regardless of runner count. **Step 3 (optional workflow-level `concurrency` group)** is documented in the runbook as unverified — explicitly framed as "verify this Gitea deployment honors it before relying on it" — and left unimplemented in `ci.yml`, since it is a `ci.yml` change out of scope for this doc-only pass. **The actual runner registration (step 1) is infrastructure work outside this repo's tree and remains the operator's to execute**, tracked in the runbook. Verification performed: `grep -n 'maxParallel\|shared\|cancel' docs/GatewayTesting.md` shows the new prose; runbook file exists at the path above; no build required (doc-only change).
---
## Cross-domain dependencies
@@ -0,0 +1,19 @@
# Candidate Findings for the Next Review Cycle (surfaced during 2026-07-12 remediation)
These were discovered while remediating the 2026-07-12 backlog but were **out of scope** for it — each is either pre-existing, by-design residual, or a new observation. They are recorded here (not fixed) so the next review cycle can triage them. None blocks the 2026-07-12 cycle, which is complete.
| ID (proposed) | Area | Severity (est.) | Summary |
|---|---|---|---|
| NEXT-01 | Testing / macOS | Low | Fake-worker/e2e gateway tests fail on macOS under the default `TMPDIR` because the `CoreFxPipe_mxaccess-gateway-{pid}-{sessionId}` path exceeds the 104-char Unix-domain-socket `sun_path` limit under `/var/folders/…/T/`. Workaround today is `TMPDIR=/tmp`. Fix options: shorten the pipe name, or document the `TMPDIR=/tmp` requirement in `docs/GatewayTesting.md`. Surfaced independently by multiple remediation agents. |
| NEXT-02 | Clients (.NET, Java) | Low | The .NET and Java CLIs render the raw `ReplayGap` sentinel `MxEvent` on `stream-events` instead of a typed gap row — Java text mode prints `0 MX_EVENT_FAMILY_UNSPECIFIED`. Same defect class as CLI-36 (Go) / CLI-35 (Python), which were fixed this cycle; the .NET/Java halves were out of scope. The cross-language smoke matrix now records this divergence honestly. |
| NEXT-03 | Gateway alarms | Low | `GatewayAlarmMonitor.ApplyReconcile` feed-repair broadcasts (the new acked-delta from GWC-26 **and** the pre-existing Raise/Clear repair) are **at-least-once, not exactly-once**: a periodic reconcile can synthesize a transition whose matching live transition is still buffered in the alarm lease, so both broadcast as indistinguishable duplicates on the alarm feed (StreamAlarms + dashboard hub). Pre-existing (the Raise/Clear repair always had it); GWC-26 documented the at-least-once contract rather than closing the race. Closing it needs reconcile/live serialization or a monotonic dedup marker. |
| NEXT-04 | Worker frame writer | Low | WRK-22/WRK-25 cancellation path: a frame `Claimed` by a concurrent lock-holder just before its caller's cancellation races in is never awaited by that caller; if the write then faults, `TrySetException` lands on a `Task` nobody observes (unobserved-task-exception). By-design residual, non-crash (no `UnobservedTaskException` handler registered), pre-existing to single-frame WRK-22 and amplified per-batch by WRK-25. Hygiene fix: attach a fault-observing continuation to abandoned/tombstoned frame completions. |
| NEXT-05 | Worker frame writer | Info | A batch whose remaining frames are tombstoned by cancellation leaves dead `PendingFrame` entries in `_eventFrames`/`_controlFrames` until a future `DequeueNext` pops and skips them. Same pre-existing behavior as single-frame WRK-22, amplified per-batch; in practice heartbeats purge them promptly, so not a real leak. |
## Operator actions still pending (from this cycle's runbooks)
These are **live-infrastructure actions the operator must execute** — the repo-side work is complete and merged:
- **SEC-36** — rotate the dev LDAP service-account credential per `docs/runbooks/SEC-36-ldap-credential-rotation.md` (generate new secret in `scadaproj/infra/glauth`, pre-stage the NSSM env var on deployed hosts, rotate GLAuth on `10.100.0.35`, verify dashboard login). The committed literal is gone from the working tree but remains recoverable from git history until rotation completes — **rotation is the load-bearing half.**
- **TST-30** — register a second Gitea `act_runner` on `10.100.0.35` per `docs/runbooks/TST-30-second-ci-runner.md` to relieve the single-shared-runner bottleneck.
- **TST-25 follow-ups** — old **TST-05** (scheduled live-MXAccess smoke) is now covered by the `nightly-windev` job; old **TST-24** (client wire tests in CI) is unblocked by the working Windows tier.
+1 -1
View File
@@ -49,7 +49,7 @@ Impact: logout (`Dashboard/DashboardEndpointRouteBuilderExtensions.cs:136-155`)
Recommendation: keep the lifetime short (or shorten to ~5 minutes given the factory refreshes per reconnect, `docs/GatewayDashboardDesign.md:497-499`), and confirm no request-path logging captures query strings (Serilog request logging is not currently enabled; keep it that way or scrub `access_token`).
**SEC-6 · Medium — LDAP is plaintext-by-default with a committed service-account password.**
Evidence: `src/ZB.MOM.WW.MxGateway.Server/Configuration/LdapOptions.cs:49-61` (defaults `Transport=None`, `AllowInsecure=true`, `ServiceAccountPassword = "serviceaccount123"`), `appsettings.json:21-33` (same values checked into the repo), `glauth.md:30,327` (dev LDAPS disabled; "binding sends passwords cleartext on the wire").
Evidence: `src/ZB.MOM.WW.MxGateway.Server/Configuration/LdapOptions.cs:49-61` (defaults `Transport=None`, `AllowInsecure=true`, `ServiceAccountPassword = "<service-account-password>"` — value redacted per SEC-36), `appsettings.json:21-33` (same values checked into the repo), `glauth.md:30,327` (dev LDAPS disabled; "binding sends passwords cleartext on the wire").
Impact: every dashboard login sends the operator's password in cleartext to `10.100.0.35:3893`, and the LDAP service-account credential is in source control. This is a documented dev posture (the shadow-options rationale at `LdapOptions.cs:20-28` is explicit that the shared library is secure-by-default), and the validator does enforce the `Transport=None ⇒ AllowInsecure` consistency rule (`GatewayOptionsValidator.cs:82-85`) — but nothing distinguishes dev from prod at runtime.
Recommendation: for production deployment docs, require `Transport=Ldaps`/`StartTls` + `AllowInsecure=false` and move `ServiceAccountPassword` to env-var/secret configuration; consider an `IsProduction` startup check mirroring SEC-4. LDAP injection risk is delegated to the shared `ZB.MOM.WW.Auth.Ldap` provider (bind-then-search per `Dashboard/DashboardAuthenticator.cs:41-47`); its escaping cannot be verified from this repo — flag for review in the donor repo.
+2 -2
View File
@@ -197,7 +197,7 @@ Full design + implementation for each row lives in the linked domain doc under i
| CLI-21 | Low | P2 | S | — | Done | Go `ClientVersion = "0.1.0-dev"` stale vs tagged releases |
| CLI-22 | Low | — | S | — | Not started | Go `newCorrelationID` swallows `crypto/rand` error → empty id |
| CLI-23 | Low | — | S | — | Not started | Go nil-vs-empty bulk short-circuit asymmetry |
| CLI-24 | Low | — | S | — | Not started | Java `MxEventStream` single-consumer constraint undocumented |
| CLI-24 | Low | — | S | — | Done | Java `MxEventStream` single-consumer constraint undocumented (closed 2026-08-07 per 2026-07-12 review old-tracker action; documented at MxEventStream.java:25 "Single consumer") |
| CLI-25 | Low | — | S | — | Not started | Java `close()` does not await channel termination |
| CLI-26 | Low | P2 | S | — | Done | Python `version.py` (0.1.0) ≠ `pyproject.toml` (0.1.2) |
| CLI-27 | Low | — | S | — | Not started | Python `Session.close()` not concurrency-safe; synthesizes reply |
@@ -207,7 +207,7 @@ Full design + implementation for each row lives in the linked domain doc under i
| CLI-31 | Low | — | M | — | Not started | Rust CLI is a single 2,699-line `main.rs` |
| CLI-32 | Low | — | S | — | Not started | Client-side bulk caps differ (.NET/Java unbounded) |
| CLI-33 | Low | — | S | CLI-01,13 | Not started | Per-language event backpressure semantics undocumented |
| CLI-34 | Low | — | S | — | Not started | Python `build/`/`.pytest_cache/` present on disk (untracked) |
| CLI-34 | Low | — | S | — | Done | Python `build/`/`.pytest_cache/` present on disk (untracked) (closed 2026-08-07 per 2026-07-12 review old-tracker action; both gitignored in clients/python/.gitignore) |
### Testing, docs & gaps — [60-testing-docs-gaps.md](60-testing-docs-gaps.md)
@@ -132,7 +132,7 @@ This document turns every finding in the Security/Dashboard/Observability review
## SEC-06 — LDAP plaintext-by-default with a committed service password `Medium` · `P1`
**Finding.** *(review SEC-6)* `Configuration/LdapOptions.cs:49-61` defaults `Transport=None`, `AllowInsecure=true`, `ServiceAccountPassword="serviceaccount123"`; `appsettings.json:21-33` ships the same. `glauth.md:30,327` confirms dev LDAPS is disabled and binds send cleartext. The validator enforces the `Transport=None ⇒ AllowInsecure` consistency rule (`GatewayOptionsValidator.cs:82-85`) but nothing distinguishes dev from prod.
**Finding.** *(review SEC-6)* `Configuration/LdapOptions.cs:49-61` defaults `Transport=None`, `AllowInsecure=true`, `ServiceAccountPassword="<service-account-password>"` (value redacted per SEC-36); `appsettings.json:21-33` ships the same. `glauth.md:30,327` confirms dev LDAPS is disabled and binds send cleartext. The validator enforces the `Transport=None ⇒ AllowInsecure` consistency rule (`GatewayOptionsValidator.cs:82-85`) but nothing distinguishes dev from prod.
**Impact.** Every dashboard login sends the operator's password cleartext to `10.100.0.35:3893`, and a service-account credential is in source control.
@@ -140,7 +140,7 @@ This document turns every finding in the Security/Dashboard/Observability review
**Implementation.**
- `Configuration/GatewayOptionsValidator.cs`: in `ValidateLdap`, when Production and `Transport == None`, emit an error (co-locate with SEC-04's env plumbing).
- Deployment: keep `serviceaccount123` only for local GLAuth dev; document env-var override (`MxGateway__Ldap__ServiceAccountPassword`) for the NSSM-wrapped hosts; rotate the dev credential's reuse.
- Deployment: keep the dev service-account password only for local GLAuth dev; document env-var override (`MxGateway__Ldap__ServiceAccountPassword`) for the NSSM-wrapped hosts; rotate the dev credential's reuse.
- Docs: `docs/GatewayConfiguration.md` Ldap section and a production hardening note referencing `glauth.md`.
- Tests: `GatewayOptionsValidatorTests` — `Transport=None` + Production → invalid.
+1 -1
View File
@@ -487,7 +487,7 @@ dotnet nuget add source https://gitea.dohertylan.com/api/packages/dohertj2/nuget
Then add the package to your project:
````bash
dotnet add package ZB.MOM.WW.MxGateway.Client --version 0.1.1
dotnet add package ZB.MOM.WW.MxGateway.Client --version 0.2.0
````
The `ZB.MOM.WW.MxGateway.Contracts` package is pulled in transitively.
@@ -0,0 +1,96 @@
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Client.Tests;
/// <summary>
/// Unit tests for <see cref="MxGatewaySecretRedaction"/> — the exact-substring scrub applied to
/// diagnostic text and rebuilt exceptions before they leave the client on a failure path.
/// </summary>
public sealed class MxGatewaySecretRedactionTests
{
[Fact]
public void Redact_ReplacesEveryOccurrenceOfSecret()
{
string result = MxGatewaySecretRedaction.Redact(
"pw=hunter2 retry pw=hunter2 again hunter2",
"hunter2");
Assert.DoesNotContain("hunter2", result, StringComparison.Ordinal);
Assert.Equal("pw=<redacted> retry pw=<redacted> again <redacted>", result);
}
[Fact]
public void Redact_ScrubsBothSecretsWhenOneIsSubstringOfTheOther()
{
// "secret" is a substring of "secretPassword"; both must be fully scrubbed regardless of
// supplied order — no residual leak of either verbatim value.
string result = MxGatewaySecretRedaction.Redact(
"a=secretPassword b=secret",
"secret",
"secretPassword");
Assert.DoesNotContain("secretPassword", result, StringComparison.Ordinal);
Assert.DoesNotContain("secret", result, StringComparison.Ordinal);
}
[Fact]
public void Redact_WithNullSecretsArray_ReturnsMessageUnchanged()
{
const string message = "nothing to scrub here";
string result = MxGatewaySecretRedaction.Redact(message, null!);
Assert.Equal(message, result);
}
[Fact]
public void Redact_WithEmptySecretsArray_ReturnsMessageUnchanged()
{
const string message = "nothing to scrub here";
string result = MxGatewaySecretRedaction.Redact(message);
Assert.Equal(message, result);
}
[Fact]
public void Redact_IgnoresWhitespaceOnlySecret()
{
// A whitespace-only secret must not over-redact the internal spaces of the message.
const string message = "user operator logged in";
string result = MxGatewaySecretRedaction.Redact(message, " ");
Assert.Equal(message, result);
}
[Fact]
public void Redacted_PreservesConcreteSubtypeAndDoesNotChainSecretBearingOriginal()
{
const string secret = "hunter2";
Exception transportCause = new InvalidOperationException("transport reset");
MxGatewaySessionException original = new(
$"session rejected credential '{secret}'",
"session-1",
"correlation-1",
new ProtocolStatus { Code = ProtocolStatusCode.SessionNotReady, Message = $"echoed '{secret}'" },
hResult: -1,
statuses: [new MxStatusProxy { DiagnosticText = $"denied '{secret}'" }],
innerException: transportCause);
MxGatewayException redacted = MxGatewaySecretRedaction.Redacted(original, secret);
// Concrete runtime type is preserved.
Assert.IsType<MxGatewaySessionException>(redacted);
// The secret is gone from the message and every structured accessor.
Assert.DoesNotContain(secret, redacted.Message, StringComparison.Ordinal);
Assert.DoesNotContain(secret, redacted.ToString(), StringComparison.Ordinal);
Assert.DoesNotContain(secret, redacted.ProtocolStatus!.Message, StringComparison.Ordinal);
Assert.All(redacted.Statuses, status =>
Assert.DoesNotContain(secret, status.DiagnosticText, StringComparison.Ordinal));
Assert.Contains("<redacted>", redacted.Message, StringComparison.Ordinal);
// The secret-bearing original is NOT chained; the original's transport cause is carried.
Assert.NotSame(original, redacted.InnerException);
Assert.Same(transportCause, redacted.InnerException);
}
}
@@ -0,0 +1,165 @@
using Google.Protobuf;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Client.Tests;
/// <summary>
/// Tests for the credential-scrub (CLI-40) and malformed-reply (CLI-41) contracts on the
/// credential and id-returning session helpers, driven from shared behavior fixtures.
/// </summary>
public sealed class MxGatewaySessionReplyContractTests
{
/// <summary>
/// CLI-40: when MXAccess echoes the submitted credential back in its failure diagnostic,
/// the surfaced exception message must scrub it to the library redaction marker.
/// </summary>
[Fact]
public async Task AuthenticateUserAsync_RedactsEchoedCredentialInFailureMessage()
{
const string password = "sup3rSecretVerify9f3a2b";
FakeGatewayTransport transport = CreateTransport();
transport.AddInvokeReply(ReadReplyFixture("authenticate-user.echoed-credential.reply.json"));
await using MxGatewayClient client = CreateClient(transport);
MxGatewaySession session = await client.OpenSessionAsync();
MxAccessException exception = await Assert.ThrowsAsync<MxAccessException>(
async () => await session.AuthenticateUserAsync(12, "operator", password));
Assert.DoesNotContain(password, exception.Message, StringComparison.Ordinal);
Assert.Contains("<redacted>", exception.Message, StringComparison.Ordinal);
// ToString() is what logging frameworks emit; the secret-bearing original must not be
// chained as an inner exception where it would re-surface the credential verbatim.
Assert.DoesNotContain(password, exception.ToString(), StringComparison.Ordinal);
}
/// <summary>
/// CLI-40: the redacted exception must not leak the echoed credential through any structured
/// accessor either — <see cref="MxAccessException.Reply"/> (protocol message, diagnostic
/// message, and each MXSTATUS_PROXY diagnostic text) and <see cref="MxGatewayException.Statuses"/>
/// all carry the server-echoed credential verbatim before the fix. Both the OK+negative-HRESULT
/// and the MXACCESS_FAILURE reply route to <see cref="MxAccessException"/>, so both must scrub.
/// </summary>
/// <param name="fixture">The echoed-credential reply fixture to drive.</param>
[Theory]
[InlineData("authenticate-user.echoed-credential.reply.json")]
[InlineData("authenticate-user.echoed-credential-mxaccess-failure.reply.json")]
public async Task AuthenticateUserAsync_RedactsEchoedCredentialInStructuredAccessors(string fixture)
{
const string password = "sup3rSecretVerify9f3a2b";
FakeGatewayTransport transport = CreateTransport();
transport.AddInvokeReply(ReadReplyFixture(fixture));
await using MxGatewayClient client = CreateClient(transport);
MxGatewaySession session = await client.OpenSessionAsync();
MxAccessException exception = await Assert.ThrowsAsync<MxAccessException>(
async () => await session.AuthenticateUserAsync(12, "operator", password));
Assert.DoesNotContain(password, exception.Message, StringComparison.Ordinal);
Assert.Contains("<redacted>", exception.Message, StringComparison.Ordinal);
Assert.DoesNotContain(password, exception.ToString(), StringComparison.Ordinal);
Assert.DoesNotContain(password, exception.Reply.ProtocolStatus.Message, StringComparison.Ordinal);
Assert.DoesNotContain(password, exception.Reply.DiagnosticMessage, StringComparison.Ordinal);
foreach (MxStatusProxy status in exception.Reply.Statuses)
{
Assert.DoesNotContain(password, status.DiagnosticText, StringComparison.Ordinal);
}
foreach (MxStatusProxy status in exception.Statuses)
{
Assert.DoesNotContain(password, status.DiagnosticText, StringComparison.Ordinal);
}
}
/// <summary>
/// CLI-41: an OK reply that carries neither the typed AuthenticateUser payload nor an
/// int32 return_value is a malformed reply, surfaced as a typed exception rather than an NRE.
/// </summary>
[Fact]
public async Task AuthenticateUserAsync_MissingPayloadAndReturnValue_ThrowsMalformedReply()
{
FakeGatewayTransport transport = CreateTransport();
transport.AddInvokeReply(ReadReplyFixture("authenticate-user.missing-payload.reply.json"));
await using MxGatewayClient client = CreateClient(transport);
MxGatewaySession session = await client.OpenSessionAsync();
await Assert.ThrowsAsync<MxGatewayMalformedReplyException>(
async () => await session.AuthenticateUserAsync(12, "operator", "pw"));
}
/// <summary>
/// CLI-41: an OK reply that omits the typed payload but carries an int32 return_value
/// resolves to that return value.
/// </summary>
[Fact]
public async Task AuthenticateUserAsync_ReturnValueOnly_ResolvesReturnValue()
{
FakeGatewayTransport transport = CreateTransport();
transport.AddInvokeReply(ReadReplyFixture("authenticate-user.return-value-only.reply.json"));
await using MxGatewayClient client = CreateClient(transport);
MxGatewaySession session = await client.OpenSessionAsync();
int userId = await session.AuthenticateUserAsync(12, "operator", "pw");
Assert.Equal(7, userId);
}
/// <summary>
/// CLI-41: the AddBufferedItem fallback shares the malformed-reply contract — an OK reply
/// with neither a typed item handle nor an int32 return_value throws the typed exception.
/// </summary>
[Fact]
public async Task AddBufferedItemAsync_MissingPayloadAndReturnValue_ThrowsMalformedReply()
{
FakeGatewayTransport transport = CreateTransport();
transport.AddInvokeReply(new MxCommandReply
{
SessionId = "session-fixture",
Kind = MxCommandKind.AddBufferedItem,
ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok },
});
await using MxGatewayClient client = CreateClient(transport);
MxGatewaySession session = await client.OpenSessionAsync();
await Assert.ThrowsAsync<MxGatewayMalformedReplyException>(
async () => await session.AddBufferedItemAsync(12, "Area001.Pump001.Speed", "runtime"));
}
private static MxGatewayClient CreateClient(FakeGatewayTransport transport)
{
return new MxGatewayClient(transport.Options, transport);
}
private static FakeGatewayTransport CreateTransport()
{
return new FakeGatewayTransport(new MxGatewayClientOptions
{
Endpoint = new Uri("http://localhost:5000"),
ApiKey = "test-api-key",
});
}
private static MxCommandReply ReadReplyFixture(string fileName)
{
DirectoryInfo directory = new(AppContext.BaseDirectory);
while (directory is not null)
{
string path = Path.Combine(
directory.FullName,
"clients",
"proto",
"fixtures",
"behavior",
"command-replies",
fileName);
if (File.Exists(path))
{
return JsonParser.Default.Parse<MxCommandReply>(File.ReadAllText(path));
}
directory = directory.Parent!;
}
throw new FileNotFoundException(fileName);
}
}
@@ -20,7 +20,8 @@ public sealed class MxStatusProxyExtensionsTests
MxStatusProxy status = JsonParser.Default.Parse<MxStatusProxy>(
testCase.GetProperty("status").GetRawText());
Assert.Equal(status.Category is MxStatusCategory.Ok, status.IsSuccess());
bool wantSuccess = testCase.GetProperty("wantSuccess").GetBoolean();
Assert.Equal(wantSuccess, status.IsSuccess());
Assert.Equal(
testCase.GetProperty("status").GetProperty("rawCategory").GetInt32(),
status.RawCategory);
@@ -0,0 +1,46 @@
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Client;
/// <summary>
/// Exception thrown when the gateway returns a protocol-OK reply that carries neither the
/// expected typed payload nor an int32 <c>return_value</c>, so the client cannot resolve the
/// operation result. This replaces the historical <see cref="NullReferenceException"/> that a
/// blind <c>reply.ReturnValue.Int32Value</c> fallback would throw.
/// </summary>
public sealed class MxGatewayMalformedReplyException : MxGatewayException
{
/// <summary>Initializes a new instance with the given message.</summary>
/// <param name="message">The error message describing the malformed reply.</param>
public MxGatewayMalformedReplyException(string message)
: base(message)
{
}
/// <summary>Initializes a new instance with full diagnostic context.</summary>
/// <param name="message">The error message describing the malformed reply.</param>
/// <param name="sessionId">The session ID, if available.</param>
/// <param name="correlationId">The correlation ID for tracing, if available.</param>
/// <param name="protocolStatus">The protocol status details, if available.</param>
/// <param name="hResult">The HResult code, if available.</param>
/// <param name="statuses">The MXAccess statuses, if available.</param>
/// <param name="innerException">The underlying exception, if any.</param>
public MxGatewayMalformedReplyException(
string message,
string? sessionId = null,
string? correlationId = null,
ProtocolStatus? protocolStatus = null,
int? hResult = null,
IReadOnlyList<MxStatusProxy>? statuses = null,
Exception? innerException = null)
: base(
message,
sessionId,
correlationId,
protocolStatus,
hResult,
statuses ?? [],
innerException)
{
}
}
@@ -0,0 +1,229 @@
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Client;
/// <summary>
/// Scrubs exact secret substrings out of diagnostic text before it leaves the client on an
/// exception path. MXAccess can echo a submitted credential or secured value back inside a
/// failure diagnostic (protocol message, MXSTATUS_PROXY diagnostic text, HRESULT description);
/// this helper replaces any such verbatim occurrence with <c>&lt;redacted&gt;</c> so the raw
/// request payload never reaches a caught exception's message. The marker matches the Go, Rust,
/// and Java clients.
/// </summary>
internal static class MxGatewaySecretRedaction
{
private const string Marker = "<redacted>";
/// <summary>
/// Replaces every usable secret in <paramref name="secrets"/> with the redaction marker
/// (ordinal comparison). Returns the message unchanged when it is null or empty, or when no
/// usable secret is supplied. A secret that is null, empty, or whitespace-only is ignored so
/// it cannot over-redact ordinary separator characters in the message.
/// </summary>
/// <param name="message">The diagnostic message to scrub.</param>
/// <param name="secrets">The secret values to remove from the message.</param>
/// <returns>The scrubbed message.</returns>
internal static string Redact(string message, params string?[] secrets)
{
if (string.IsNullOrEmpty(message) || secrets is null)
{
return message;
}
string result = message;
foreach (string? secret in secrets)
{
if (!string.IsNullOrWhiteSpace(secret))
{
result = result.Replace(secret, Marker, StringComparison.Ordinal);
}
}
return result;
}
/// <summary>
/// Returns a scrubbed clone of <paramref name="reply"/>: the protocol-status message, the
/// reply-level diagnostic message, and each MXSTATUS_PROXY diagnostic text have every verbatim
/// secret replaced with the redaction marker. The original is left untouched. MXAccess can echo
/// a submitted credential into any of these fields, so a redacted exception must carry the
/// scrubbed reply rather than the secret-bearing original.
/// </summary>
/// <param name="reply">The reply to clone and scrub.</param>
/// <param name="secrets">The secret values to remove.</param>
/// <returns>A scrubbed clone of the reply.</returns>
internal static MxCommandReply RedactReply(MxCommandReply reply, params string?[] secrets)
{
ArgumentNullException.ThrowIfNull(reply);
MxCommandReply clone = reply.Clone();
if (clone.ProtocolStatus is not null)
{
clone.ProtocolStatus.Message = Redact(clone.ProtocolStatus.Message, secrets);
}
clone.DiagnosticMessage = Redact(clone.DiagnosticMessage, secrets);
foreach (MxStatusProxy status in clone.Statuses)
{
status.DiagnosticText = Redact(status.DiagnosticText, secrets);
}
return clone;
}
/// <summary>
/// Returns a scrubbed clone of <paramref name="status"/> (its message with every verbatim
/// secret removed), or <see langword="null"/> when the input is null.
/// </summary>
/// <param name="status">The protocol status to clone and scrub.</param>
/// <param name="secrets">The secret values to remove.</param>
/// <returns>A scrubbed clone, or <see langword="null"/>.</returns>
internal static ProtocolStatus? RedactStatus(ProtocolStatus? status, params string?[] secrets)
{
if (status is null)
{
return null;
}
ProtocolStatus clone = status.Clone();
clone.Message = Redact(clone.Message, secrets);
return clone;
}
/// <summary>
/// Returns a list of scrubbed clones of <paramref name="statuses"/> — each MXSTATUS_PROXY's
/// diagnostic text has every verbatim secret removed. The originals are left untouched.
/// </summary>
/// <param name="statuses">The statuses to clone and scrub.</param>
/// <param name="secrets">The secret values to remove.</param>
/// <returns>A list of scrubbed clones.</returns>
internal static IReadOnlyList<MxStatusProxy> RedactStatuses(
IReadOnlyList<MxStatusProxy> statuses,
params string?[] secrets)
{
if (statuses is null || statuses.Count is 0)
{
return statuses ?? [];
}
MxStatusProxy[] result = new MxStatusProxy[statuses.Count];
for (int i = 0; i < statuses.Count; i++)
{
MxStatusProxy clone = statuses[i].Clone();
clone.DiagnosticText = Redact(clone.DiagnosticText, secrets);
result[i] = clone;
}
return result;
}
/// <summary>
/// Returns an exception equivalent to <paramref name="ex"/> but with any verbatim secret
/// scrubbed from its message. When nothing changes, the original exception is returned
/// unchanged; otherwise a new exception of the same concrete runtime type is built and the
/// original reply/status context is preserved. The secret-bearing original is deliberately
/// <b>not</b> chained as the inner exception — doing so would let its unredacted message
/// re-surface through <see cref="Exception.ToString"/> (which logging frameworks call). The
/// original's own inner cause (a transport error, never the request payload) is carried
/// forward instead.
/// </summary>
/// <param name="ex">The exception to redact.</param>
/// <param name="secrets">The secret values to remove from the message.</param>
/// <returns>The redacted exception, or the original when no change was needed.</returns>
internal static MxGatewayException Redacted(MxGatewayException ex, params string?[] secrets)
{
ArgumentNullException.ThrowIfNull(ex);
string redacted = Redact(ex.Message, secrets);
bool messageChanged = !string.Equals(redacted, ex.Message, StringComparison.Ordinal);
Exception? cause = ex.InnerException;
// MxAccessException derives its structured fields from the raw reply, so scrubbing must
// clone and redact that reply — the message alone changing is not enough, because the reply
// can carry the echoed secret even when the message does not.
if (ex is MxAccessException access)
{
if (!messageChanged && !ReplyContainsSecret(access.Reply, secrets))
{
return ex;
}
return new MxAccessException(redacted, RedactReply(access.Reply, secrets), cause);
}
// Other subtypes carry the secret through ProtocolStatus.Message and Statuses[].DiagnosticText.
if (!messageChanged
&& !ContainsSecret(ex.ProtocolStatus?.Message, secrets)
&& !StatusesContainSecret(ex.Statuses, secrets))
{
return ex;
}
ProtocolStatus? status = RedactStatus(ex.ProtocolStatus, secrets);
IReadOnlyList<MxStatusProxy> statuses = RedactStatuses(ex.Statuses, secrets);
return ex switch
{
MxGatewaySessionException => new MxGatewaySessionException(
redacted, ex.SessionId, ex.CorrelationId, status, ex.HResultCode, statuses, cause),
MxGatewayWorkerException => new MxGatewayWorkerException(
redacted, ex.SessionId, ex.CorrelationId, status, ex.HResultCode, statuses, cause),
MxGatewayAuthenticationException => new MxGatewayAuthenticationException(
redacted, ex.SessionId, ex.CorrelationId, status, ex.HResultCode, statuses, cause),
MxGatewayAuthorizationException => new MxGatewayAuthorizationException(
redacted, ex.SessionId, ex.CorrelationId, status, ex.HResultCode, statuses, cause),
MxGatewayMalformedReplyException => new MxGatewayMalformedReplyException(
redacted, ex.SessionId, ex.CorrelationId, status, ex.HResultCode, statuses, cause),
MxGatewayCommandException => new MxGatewayCommandException(
redacted, ex.SessionId, ex.CorrelationId, status, ex.HResultCode, statuses, cause),
_ => new MxGatewayException(redacted, cause),
};
}
private static bool ContainsSecret(string? text, string?[] secrets)
{
if (string.IsNullOrEmpty(text) || secrets is null)
{
return false;
}
foreach (string? secret in secrets)
{
if (!string.IsNullOrWhiteSpace(secret) && text.Contains(secret, StringComparison.Ordinal))
{
return true;
}
}
return false;
}
private static bool StatusesContainSecret(IReadOnlyList<MxStatusProxy> statuses, string?[] secrets)
{
if (statuses is null)
{
return false;
}
foreach (MxStatusProxy status in statuses)
{
if (ContainsSecret(status.DiagnosticText, secrets))
{
return true;
}
}
return false;
}
private static bool ReplyContainsSecret(MxCommandReply reply, string?[] secrets)
{
if (reply is null)
{
return false;
}
return ContainsSecret(reply.ProtocolStatus?.Message, secrets)
|| ContainsSecret(reply.DiagnosticMessage, secrets)
|| StatusesContainSecret(reply.Statuses, secrets);
}
}
@@ -945,7 +945,7 @@ public sealed class MxGatewaySession : IAsyncDisposable
cancellationToken)
.ConfigureAwait(false);
reply.EnsureProtocolSuccess().EnsureMxAccessSuccess();
return reply.AddBufferedItem?.ItemHandle ?? reply.ReturnValue.Int32Value;
return ResolveInt32Result(reply.AddBufferedItem?.ItemHandle, reply, "AddBufferedItem");
}
/// <summary>
@@ -1141,7 +1141,14 @@ public sealed class MxGatewaySession : IAsyncDisposable
verifierUserId,
cancellationToken)
.ConfigureAwait(false);
reply.EnsureProtocolSuccess().EnsureMxAccessSuccess();
try
{
reply.EnsureProtocolSuccess().EnsureMxAccessSuccess();
}
catch (MxGatewayException ex)
{
throw MxGatewaySecretRedaction.Redacted(ex, ExtractSecretString(value));
}
}
/// <summary>
@@ -1215,7 +1222,14 @@ public sealed class MxGatewaySession : IAsyncDisposable
verifierUserId,
cancellationToken)
.ConfigureAwait(false);
reply.EnsureProtocolSuccess().EnsureMxAccessSuccess();
try
{
reply.EnsureProtocolSuccess().EnsureMxAccessSuccess();
}
catch (MxGatewayException ex)
{
throw MxGatewaySecretRedaction.Redacted(ex, ExtractSecretString(value));
}
}
/// <summary>
@@ -1285,8 +1299,15 @@ public sealed class MxGatewaySession : IAsyncDisposable
verifyUserPassword,
cancellationToken)
.ConfigureAwait(false);
reply.EnsureProtocolSuccess().EnsureMxAccessSuccess();
return reply.AuthenticateUser?.UserId ?? reply.ReturnValue.Int32Value;
try
{
reply.EnsureProtocolSuccess().EnsureMxAccessSuccess();
return ResolveInt32Result(reply.AuthenticateUser?.UserId, reply, "AuthenticateUser");
}
catch (MxGatewayException ex)
{
throw MxGatewaySecretRedaction.Redacted(ex, verifyUserPassword);
}
}
/// <summary>
@@ -1337,7 +1358,7 @@ public sealed class MxGatewaySession : IAsyncDisposable
MxCommandReply reply = await ArchestraUserToIdRawAsync(serverHandle, userIdGuid, cancellationToken)
.ConfigureAwait(false);
reply.EnsureProtocolSuccess().EnsureMxAccessSuccess();
return reply.ArchestraUserToId?.UserId ?? reply.ReturnValue.Int32Value;
return ResolveInt32Result(reply.ArchestraUserToId?.UserId, reply, "ArchestrAUserToId");
}
/// <summary>
@@ -1367,6 +1388,51 @@ public sealed class MxGatewaySession : IAsyncDisposable
cancellationToken);
}
/// <summary>
/// Resolves the int32 result of an OK command reply: the typed payload value when present,
/// otherwise an int32 <c>return_value</c> when the reply carries one. A reply that provides
/// neither is malformed and surfaces as <see cref="MxGatewayMalformedReplyException"/>
/// rather than the historical <see cref="NullReferenceException"/>.
/// </summary>
/// <param name="typedValue">The typed payload value, or <see langword="null"/> when absent.</param>
/// <param name="reply">The OK command reply.</param>
/// <param name="operation">The MXAccess operation name, for the diagnostic message.</param>
/// <returns>The resolved int32 result.</returns>
private static int ResolveInt32Result(int? typedValue, MxCommandReply reply, string operation)
{
if (typedValue.HasValue)
{
return typedValue.Value;
}
if (reply.ReturnValue is not null
&& reply.ReturnValue.KindCase == MxValue.KindOneofCase.Int32Value)
{
return reply.ReturnValue.Int32Value;
}
throw new MxGatewayMalformedReplyException(
$"{operation} returned a malformed reply: OK reply carried neither the typed payload nor an int32 return_value",
reply.SessionId,
reply.CorrelationId,
reply.ProtocolStatus,
reply.HasHresult ? reply.Hresult : null,
reply.Statuses.ToArray());
}
/// <summary>
/// Extracts the raw string form of a credential-bearing <see cref="MxValue"/> for redaction,
/// or <see langword="null"/> when the value does not carry a string.
/// </summary>
/// <param name="value">The value written by a secured write.</param>
/// <returns>The string payload, or <see langword="null"/>.</returns>
private static string? ExtractSecretString(MxValue value)
{
return value.KindCase == MxValue.KindOneofCase.StringValue
? value.StringValue
: null;
}
/// <summary>
/// Invokes an MXAccess command on this session.
/// </summary>
@@ -30,6 +30,6 @@ public static class MxStatusProxyExtensions
? "no diagnostic text"
: status.DiagnosticText;
return $"{status.Category} by {status.DetectedBy}; detail={status.Detail}; {diagnosticText}";
return $"success={status.Success}; {status.Category} by {status.DetectedBy}; detail={status.Detail}; {diagnosticText}";
}
}
@@ -19,7 +19,7 @@
<PropertyGroup>
<IsPackable>true</IsPackable>
<PackageId>ZB.MOM.WW.MxGateway.Client</PackageId>
<Version>0.1.2</Version>
<Version>0.2.0</Version>
<Description>.NET 10 gRPC client for the MxAccessGateway service. Provides typed wrappers, retry, and a lazy-browse walker over the Galaxy Repository hierarchy.</Description>
<PackageReadmeFile>README.md</PackageReadmeFile>
<!-- Only the shipped library generates XML docs (matching src/Contracts). The Cli and
+5 -3
View File
@@ -471,7 +471,7 @@ go run ./cmd/mxgw-go smoke -endpoint $env:MXGATEWAY_ENDPOINT -plaintext -api-key
The module is resolved directly from the git repo — no package registry:
````bash
go get gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go@v0.1.1
go get gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go@v0.2.0
````
Then import:
@@ -494,11 +494,13 @@ Go modules in monorepo subdirectories use prefixed tags. To tag a release
from this repo:
````bash
pwsh scripts/tag-go-module.ps1 -Version v0.1.1 -Push
pwsh scripts/tag-go-module.ps1 -Version v0.2.0 -Push
````
The script validates semver, refuses to tag with uncommitted tracked
changes, creates an annotated tag `clients/go/v0.1.1`, and (with `-Push`)
changes, verifies `clients/go/mxgateway/version.go`'s `ClientVersion`
matches the requested tag version (failing the tag otherwise — CLI-21/CLI-39),
creates an annotated tag `clients/go/v0.2.0`, and (with `-Push`)
pushes it to origin.
## Related Documentation
+29 -2
View File
@@ -1,6 +1,16 @@
Set-StrictMode -Version Latest
$ErrorActionPreference = 'Stop'
# Pinned generator baseline. The committed Go bindings stamp these plugin versions in their
# headers (protoc-gen-go v1.36.11 / protoc-gen-go-grpc v1.6.2). Plugin-version drift rewrites
# those header stamps, so a regeneration on an off-pin machine would churn the tree and make
# check-codegen Check 4 false-fail (or mask real drift under churn). Assert the exact versions
# so a regen is deterministic. protoc itself is warn-only (source_code_info is normalized out of
# the committed bindings), matching publish-client-proto-inputs.ps1.
$PinnedProtocGenGoVersion = 'protoc-gen-go v1.36.11'
$PinnedProtocGenGoGrpcVersion = 'protoc-gen-go-grpc 1.6.2'
$PinnedProtocVersion = 'libprotoc 34.1'
$repoRoot = Resolve-Path (Join-Path $PSScriptRoot '..\..')
$protoRoot = Join-Path $repoRoot 'src\ZB.MOM.WW.MxGateway.Contracts\Protos'
$outputRoot = Join-Path $PSScriptRoot 'internal\generated'
@@ -36,8 +46,25 @@ $wingetProtoc = if ($env:LOCALAPPDATA) {
$goBin = if ($env:USERPROFILE) { Join-Path $env:USERPROFILE 'go\bin' } elseif ($env:HOME) { Join-Path $env:HOME 'go/bin' } else { $null }
$protoc = Resolve-Tool -Names @('protoc', 'protoc.exe') -FallbackPaths @($wingetProtoc)
$protocGenGo = Resolve-Tool -Names @('protoc-gen-go', 'protoc-gen-go.exe') -FallbackPaths @((if ($goBin) { Join-Path $goBin 'protoc-gen-go.exe' }), (if ($goBin) { Join-Path $goBin 'protoc-gen-go' }))
$protocGenGoGrpc = Resolve-Tool -Names @('protoc-gen-go-grpc', 'protoc-gen-go-grpc.exe') -FallbackPaths @((if ($goBin) { Join-Path $goBin 'protoc-gen-go-grpc.exe' }), (if ($goBin) { Join-Path $goBin 'protoc-gen-go-grpc' }))
$protocGenGo = Resolve-Tool -Names @('protoc-gen-go', 'protoc-gen-go.exe') -FallbackPaths @(($(if ($goBin) { Join-Path $goBin 'protoc-gen-go.exe' })), ($(if ($goBin) { Join-Path $goBin 'protoc-gen-go' })))
$protocGenGoGrpc = Resolve-Tool -Names @('protoc-gen-go-grpc', 'protoc-gen-go-grpc.exe') -FallbackPaths @(($(if ($goBin) { Join-Path $goBin 'protoc-gen-go-grpc.exe' })), ($(if ($goBin) { Join-Path $goBin 'protoc-gen-go-grpc' })))
# Assert the pinned plugin versions before generating so Check 4 cannot false-fail (or mask drift)
# on an off-pin machine. protoc is warn-only.
$protocGenGoVersion = (& $protocGenGo --version 2>&1 | Out-String).Trim()
if ($protocGenGoVersion -ne $PinnedProtocGenGoVersion) {
throw "protoc-gen-go reports '$protocGenGoVersion', but regeneration is pinned to '$PinnedProtocGenGoVersion'. " +
"Install the pin: go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11"
}
$protocGenGoGrpcVersion = (& $protocGenGoGrpc --version 2>&1 | Out-String).Trim()
if ($protocGenGoGrpcVersion -ne $PinnedProtocGenGoGrpcVersion) {
throw "protoc-gen-go-grpc reports '$protocGenGoGrpcVersion', but regeneration is pinned to '$PinnedProtocGenGoGrpcVersion'. " +
"Install the pin: go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.6.2"
}
$protocVersion = (& $protoc --version 2>&1 | Out-String).Trim()
if ($protocVersion -ne $PinnedProtocVersion) {
Write-Warning "protoc reports '$protocVersion', pin is '$PinnedProtocVersion'. Descriptor comments are normalized out of the committed Go bindings, so patch drift is tolerated; keep CI on the pin."
}
# protoc discovers the plugins on PATH; prepend the directories the resolved plugins live in.
$env:Path = (Split-Path $protocGenGo -Parent) + [System.IO.Path]::PathSeparator + (Split-Path $protocGenGoGrpc -Parent) + [System.IO.Path]::PathSeparator + $env:Path
@@ -5974,8 +5974,12 @@ func (x *WorkerInfoReply) GetMxaccessClsid() string {
}
type DrainEventsReply struct {
state protoimpl.MessageState `protogen:"open.v1"`
Events []*MxEvent `protobuf:"bytes,1,rep,name=events,proto3" json:"events,omitempty"`
state protoimpl.MessageState `protogen:"open.v1"`
// The reply is bounded by both a server-side count cap and the negotiated
// worker-frame byte cap; a reply may therefore carry fewer events than
// `max_events` and fewer than are queued. Callers drain iteratively until an
// empty reply.
Events []*MxEvent `protobuf:"bytes,1,rep,name=events,proto3" json:"events,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
@@ -6411,6 +6415,11 @@ type ReplayGap struct {
// after_worker_sequence = oldest_available_sequence - 1 in the next
// StreamEventsRequest, which will cause the server to replay starting at
// oldest_available_sequence (the first retained event).
// When nothing is retained (the replay ring is empty), this is the next sequence
// that can be delivered — `highest observed + 1` — and the `oldest - 1` resume
// formula remains valid: it resolves to the highest sequence already seen, so the
// follow-up resume replays nothing, reports no gap, and every newer live event
// passes. The interval evicted is unchanged.
OldestAvailableSequence uint64 `protobuf:"varint,2,opt,name=oldest_available_sequence,json=oldestAvailableSequence,proto3" json:"oldest_available_sequence,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
@@ -431,8 +431,17 @@ type GatewayHello struct {
SupportedProtocolVersion uint32 `protobuf:"varint,1,opt,name=supported_protocol_version,json=supportedProtocolVersion,proto3" json:"supported_protocol_version,omitempty"`
Nonce string `protobuf:"bytes,2,opt,name=nonce,proto3" json:"nonce,omitempty"`
GatewayVersion string `protobuf:"bytes,3,opt,name=gateway_version,json=gatewayVersion,proto3" json:"gateway_version,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
// Maximum worker-frame payload size, in bytes, negotiated by the gateway from its
// configured pipe limit. The worker adopts this as its frame-protocol MaxMessageBytes
// instead of a hard-coded default; 0 (an older gateway that never set the field) means
// "use the worker's built-in default". Sits above the public gRPC cap by an
// envelope-overhead margin so an accepted gRPC payload always fits one worker frame.
// Every worker->gateway frame — events, heartbeats, faults, and control replies
// including DrainEvents — must serialize within this limit; reply builders truncate
// to fit rather than emit an oversized frame.
MaxFrameBytes uint32 `protobuf:"varint,4,opt,name=max_frame_bytes,json=maxFrameBytes,proto3" json:"max_frame_bytes,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *GatewayHello) Reset() {
@@ -486,6 +495,13 @@ func (x *GatewayHello) GetGatewayVersion() string {
return ""
}
func (x *GatewayHello) GetMaxFrameBytes() uint32 {
if x != nil {
return x.MaxFrameBytes
}
return 0
}
type WorkerHello struct {
state protoimpl.MessageState `protogen:"open.v1"`
ProtocolVersion uint32 `protobuf:"varint,1,opt,name=protocol_version,json=protocolVersion,proto3" json:"protocol_version,omitempty"`
@@ -1109,11 +1125,12 @@ const file_mxaccess_worker_proto_rawDesc = "" +
"\fworker_event\x18\x12 \x01(\v2\x1f.mxaccess_worker.v1.WorkerEventH\x00R\vworkerEvent\x12P\n" +
"\x10worker_heartbeat\x18\x13 \x01(\v2#.mxaccess_worker.v1.WorkerHeartbeatH\x00R\x0fworkerHeartbeat\x12D\n" +
"\fworker_fault\x18\x14 \x01(\v2\x1f.mxaccess_worker.v1.WorkerFaultH\x00R\vworkerFaultB\x06\n" +
"\x04body\"\x8b\x01\n" +
"\x04body\"\xb3\x01\n" +
"\fGatewayHello\x12<\n" +
"\x1asupported_protocol_version\x18\x01 \x01(\rR\x18supportedProtocolVersion\x12\x14\n" +
"\x05nonce\x18\x02 \x01(\tR\x05nonce\x12'\n" +
"\x0fgateway_version\x18\x03 \x01(\tR\x0egatewayVersion\"\xa1\x01\n" +
"\x0fgateway_version\x18\x03 \x01(\tR\x0egatewayVersion\x12&\n" +
"\x0fmax_frame_bytes\x18\x04 \x01(\rR\rmaxFrameBytes\"\xa1\x01\n" +
"\vWorkerHello\x12)\n" +
"\x10protocol_version\x18\x01 \x01(\rR\x0fprotocolVersion\x12\x14\n" +
"\x05nonce\x18\x02 \x01(\tR\x05nonce\x12*\n" +
+148 -9
View File
@@ -10,7 +10,9 @@ import (
pb "gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go/internal/generated"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/metadata"
"google.golang.org/grpc/status"
"google.golang.org/grpc/test/bufconn"
)
@@ -200,6 +202,136 @@ func TestEventsSlowConsumerYieldsErrSlowConsumerBeforeClose(t *testing.T) {
}
}
func TestEventsFullBufferTerminalErrorKeepsRootCause(t *testing.T) {
fake := &fakeGatewayServer{
streamStarted: make(chan struct{}),
streamDone: make(chan struct{}),
streamEventCount: eventBufferSize,
streamTerminalErr: status.Error(codes.Internal, "boom"),
}
client, cleanup := newBufconnClient(t, fake)
defer cleanup()
session := NewSessionForID(client, "session-1")
events, err := session.EventsAfter(context.Background(), 0)
if err != nil {
t.Fatalf("EventsAfter() error = %v", err)
}
<-fake.streamStarted
// Do not drain until the stream has fully ended: the server sends exactly
// eventBufferSize events (filling the data slots) and then returns a genuine
// terminal gRPC error. The client must report that error as itself, using the
// reserved slot, rather than mislabeling it as ErrSlowConsumer.
select {
case <-fake.streamDone:
case <-time.After(2 * time.Second):
t.Fatal("event stream did not stop after terminal error")
}
// streamDone fires when the server returns; the client's producer goroutine
// still needs a moment to drain the gRPC stream, fill all data slots, and
// enqueue the terminal result. Let it settle before draining so the buffer is
// genuinely full when the terminal error is processed (which is what makes the
// mislabel bug observable).
time.Sleep(250 * time.Millisecond)
var last EventResult
gotResult := false
for {
select {
case res, ok := <-events:
if !ok {
if !gotResult {
t.Fatal("events channel closed without yielding any result")
}
var gwErr *GatewayError
if !errors.As(last.Err, &gwErr) {
t.Fatalf("final event result err is %T, want *GatewayError", last.Err)
}
if code := status.Code(last.Err); code != codes.Internal {
t.Fatalf("final event result gRPC code = %s, want %s", code, codes.Internal)
}
if errors.Is(last.Err, ErrSlowConsumer) {
t.Fatalf("final event result err = %v, must not be mislabeled as ErrSlowConsumer", last.Err)
}
return
}
last = res
gotResult = true
case <-time.After(2 * time.Second):
t.Fatal("events channel did not close after terminal error")
}
}
}
// TestSubscribeEventsFullBufferDeliversTerminalError is the CLI-44 regression for
// the never-drop Subscribe path. SubscribeEvents/SubscribeEventsAfter use
// cancelWhenResultBufferFull=false, so ordinary sends are blocking and uncapped and
// can fill every slot in the results channel — including the reserved terminal slot.
// A genuine terminal Recv error must still be delivered as the final result, never
// silently dropped. The server sends eventBufferSize+eventBufferReservedSlots events
// (filling every slot) and then returns a genuine gRPC error; with an unconditional
// non-blocking terminal send the error is dropped, so this fails red until the send
// path blocks for the never-drop mode.
func TestSubscribeEventsFullBufferDeliversTerminalError(t *testing.T) {
fake := &fakeGatewayServer{
streamStarted: make(chan struct{}),
streamDone: make(chan struct{}),
streamEventCount: eventBufferSize + eventBufferReservedSlots,
streamTerminalErr: status.Error(codes.Internal, "boom"),
}
client, cleanup := newBufconnClient(t, fake)
defer cleanup()
session := NewSessionForID(client, "session-1")
subscription, err := session.SubscribeEvents(context.Background())
if err != nil {
t.Fatalf("SubscribeEvents() error = %v", err)
}
defer subscription.Close()
<-fake.streamStarted
// Wait for the server to finish sending every event and return the terminal
// error, so the producer goroutine has filled every buffered slot before the
// terminal result is processed. That is what makes the dropped-terminal bug
// observable: with the buffer full, an unconditional non-blocking send discards
// the terminal error.
select {
case <-fake.streamDone:
case <-time.After(2 * time.Second):
t.Fatal("event stream did not stop after terminal error")
}
time.Sleep(250 * time.Millisecond)
// Drain fully. Every data event, then the terminal gRPC error as the final
// result, must arrive; the channel must not close without yielding it.
events := subscription.Events()
var last EventResult
gotResult := false
for {
select {
case res, ok := <-events:
if !ok {
if !gotResult {
t.Fatal("events channel closed without yielding any result")
}
var gwErr *GatewayError
if !errors.As(last.Err, &gwErr) {
t.Fatalf("final event result err is %T (%v), want the terminal *GatewayError; it was dropped", last.Err, last.Err)
}
if code := status.Code(last.Err); code != codes.Internal {
t.Fatalf("final event result gRPC code = %s, want %s", code, codes.Internal)
}
return
}
last = res
gotResult = true
case <-time.After(2 * time.Second):
t.Fatal("events channel did not close after terminal error")
}
}
}
func TestEventsSurfacesReplayGapSentinelAsTypedSignal(t *testing.T) {
fake := &fakeGatewayServer{
streamStarted: make(chan struct{}),
@@ -694,15 +826,16 @@ func newBufconnClient(t *testing.T, fake *fakeGatewayServer) (*Client, func()) {
type fakeGatewayServer struct {
pb.UnimplementedMxAccessGatewayServer
openReply *pb.OpenSessionReply
openAuth string
streamAuth string
streamStarted chan struct{}
streamDone chan struct{}
streamEventCount int
streamReplayGap *pb.ReplayGap
invokeReply *pb.MxCommandReply
invokeRequest *pb.MxCommandRequest
openReply *pb.OpenSessionReply
openAuth string
streamAuth string
streamStarted chan struct{}
streamDone chan struct{}
streamEventCount int
streamReplayGap *pb.ReplayGap
streamTerminalErr error
invokeReply *pb.MxCommandReply
invokeRequest *pb.MxCommandRequest
}
func (s *fakeGatewayServer) OpenSession(ctx context.Context, req *pb.OpenSessionRequest) (*pb.OpenSessionReply, error) {
@@ -772,6 +905,12 @@ func (s *fakeGatewayServer) StreamEvents(req *pb.StreamEventsRequest, stream grp
return err
}
}
if s.streamTerminalErr != nil {
// Return a genuine terminal stream error immediately after sending the
// events, without waiting on the client to cancel. This exercises the
// Recv-error path while the client's result buffer is still full.
return s.streamTerminalErr
}
<-stream.Context().Done()
return io.EOF
}
@@ -0,0 +1,197 @@
package mxgateway
import (
"context"
"errors"
"os"
"path/filepath"
"strings"
"testing"
pb "gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go/internal/generated"
"google.golang.org/protobuf/encoding/protojson"
)
// loadCommandReplyFixture parses a shared command-reply fixture into an
// MxCommandReply so the Go client can be driven through the same wire shapes the
// other language clients exercise.
func loadCommandReplyFixture(t *testing.T, name string) *pb.MxCommandReply {
t.Helper()
path := filepath.Join("..", "..", "proto", "fixtures", "behavior", "command-replies", name)
data, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read fixture %s: %v", name, err)
}
var reply pb.MxCommandReply
if err := protojson.Unmarshal(data, &reply); err != nil {
t.Fatalf("parse fixture %s: %v", name, err)
}
return &reply
}
func TestAuthenticateUserMissingPayloadReturnsMalformedReplyError(t *testing.T) {
fake := &fakeGatewayServer{
invokeReply: loadCommandReplyFixture(t, "authenticate-user.missing-payload.reply.json"),
}
client, cleanup := newBufconnClient(t, fake)
defer cleanup()
session := NewSessionForID(client, "session-1")
_, err := session.AuthenticateUser(context.Background(), 12, "operator", "secret")
var malformed *MalformedReplyError
if !errors.As(err, &malformed) {
t.Fatalf("AuthenticateUser() error = %v (%T), want *MalformedReplyError", err, err)
}
if malformed.Op != "authenticate user" {
t.Fatalf("MalformedReplyError.Op = %q, want %q", malformed.Op, "authenticate user")
}
}
func TestAuthenticateUserReturnValueOnlyUsesInt32ReturnValue(t *testing.T) {
fake := &fakeGatewayServer{
invokeReply: loadCommandReplyFixture(t, "authenticate-user.return-value-only.reply.json"),
}
client, cleanup := newBufconnClient(t, fake)
defer cleanup()
session := NewSessionForID(client, "session-1")
userID, err := session.AuthenticateUser(context.Background(), 12, "operator", "secret")
if err != nil {
t.Fatalf("AuthenticateUser() error = %v", err)
}
if userID != 7 {
t.Fatalf("AuthenticateUser() = %d, want 7", userID)
}
}
// AddBufferedItem shares the prefer-payload / int32-return-value / malformed
// fallback code path; cover both branches for one of the siblings.
func TestAddBufferedItemFallbackHonoursReturnValueAndReportsMalformed(t *testing.T) {
t.Run("return-value-only", func(t *testing.T) {
fake := &fakeGatewayServer{
invokeReply: loadCommandReplyFixture(t, "authenticate-user.return-value-only.reply.json"),
}
client, cleanup := newBufconnClient(t, fake)
defer cleanup()
session := NewSessionForID(client, "session-1")
itemHandle, err := session.AddBufferedItem(context.Background(), 12, "Area001.Pump001.Speed", "runtime")
if err != nil {
t.Fatalf("AddBufferedItem() error = %v", err)
}
if itemHandle != 7 {
t.Fatalf("AddBufferedItem() = %d, want 7", itemHandle)
}
})
t.Run("missing-payload", func(t *testing.T) {
fake := &fakeGatewayServer{
invokeReply: loadCommandReplyFixture(t, "authenticate-user.missing-payload.reply.json"),
}
client, cleanup := newBufconnClient(t, fake)
defer cleanup()
session := NewSessionForID(client, "session-1")
_, err := session.AddBufferedItem(context.Background(), 12, "Area001.Pump001.Speed", "runtime")
var malformed *MalformedReplyError
if !errors.As(err, &malformed) {
t.Fatalf("AddBufferedItem() error = %v (%T), want *MalformedReplyError", err, err)
}
if malformed.Op != "add buffered item" {
t.Fatalf("MalformedReplyError.Op = %q, want %q", malformed.Op, "add buffered item")
}
})
}
// TestAuthenticateUserScrubsEchoedCredentialFromError is the CLI-40 regression:
// a gateway diagnostic that echoes the raw credential back must never reach the
// caller's surfaced error text.
func TestAuthenticateUserScrubsEchoedCredentialFromError(t *testing.T) {
const credential = "sup3rSecretVerify9f3a2b"
fake := &fakeGatewayServer{
invokeReply: loadCommandReplyFixture(t, "authenticate-user.echoed-credential.reply.json"),
}
client, cleanup := newBufconnClient(t, fake)
defer cleanup()
session := NewSessionForID(client, "session-1")
_, err := session.AuthenticateUser(context.Background(), 12, "operator", credential)
if err == nil {
t.Fatal("AuthenticateUser() error = nil, want an MXAccess failure")
}
message := err.Error()
if strings.Contains(message, credential) {
t.Fatalf("surfaced error leaked the credential: %q", message)
}
if !strings.Contains(message, "<redacted>") {
t.Fatalf("surfaced error missing redaction marker: %q", message)
}
}
// TestAuthenticateUserScrubsEchoedCredentialFromStructuredReply is the CLI-40
// follow-up: redacting only the rendered Error() string is not enough. The typed
// *MxAccessError still carries the raw command reply, whose ProtocolStatus.Message,
// DiagnosticMessage, and Statuses[].DiagnosticText echo the credential verbatim. A
// logger dumping structured fields would reintroduce the leak, so the reply the
// typed error carries must be a scrubbed clone. Both the OK+negative-HRESULT and the
// MXACCESS_FAILURE fixtures route to *MxAccessError (via EnsureProtocolSuccess), so
// both must be scrubbed identically.
func TestAuthenticateUserScrubsEchoedCredentialFromStructuredReply(t *testing.T) {
const credential = "sup3rSecretVerify9f3a2b"
fixtures := []string{
"authenticate-user.echoed-credential.reply.json",
"authenticate-user.echoed-credential-mxaccess-failure.reply.json",
}
for _, fixture := range fixtures {
t.Run(fixture, func(t *testing.T) {
fake := &fakeGatewayServer{
invokeReply: loadCommandReplyFixture(t, fixture),
}
client, cleanup := newBufconnClient(t, fake)
defer cleanup()
session := NewSessionForID(client, "session-1")
_, err := session.AuthenticateUser(context.Background(), 12, "operator", credential)
if err == nil {
t.Fatal("AuthenticateUser() error = nil, want an MXAccess failure")
}
var mxErr *MxAccessError
if !errors.As(err, &mxErr) {
t.Fatalf("AuthenticateUser() error = %v (%T), want *MxAccessError", err, err)
}
reply := mxErr.Reply
if reply == nil {
t.Fatal("MxAccessError.Reply is nil, want the scrubbed command reply")
}
if got := reply.GetProtocolStatus().GetMessage(); strings.Contains(got, credential) {
t.Fatalf("MxAccessError.Reply.ProtocolStatus.Message leaked the credential: %q", got)
}
if got := reply.GetDiagnosticMessage(); strings.Contains(got, credential) {
t.Fatalf("MxAccessError.Reply.DiagnosticMessage leaked the credential: %q", got)
}
for i, status := range reply.GetStatuses() {
if got := status.GetDiagnosticText(); strings.Contains(got, credential) {
t.Fatalf("MxAccessError.Reply.Statuses[%d].DiagnosticText leaked the credential: %q", i, got)
}
}
// The wrapped CommandError's status/reply must be scrubbed too.
if mxErr.Command != nil {
if got := mxErr.Command.Status.GetMessage(); strings.Contains(got, credential) {
t.Fatalf("MxAccessError.Command.Status.Message leaked the credential: %q", got)
}
if cmdReply := mxErr.Command.Reply; cmdReply != nil {
if got := cmdReply.GetDiagnosticMessage(); strings.Contains(got, credential) {
t.Fatalf("MxAccessError.Command.Reply.DiagnosticMessage leaked the credential: %q", got)
}
}
}
if got := err.Error(); strings.Contains(got, credential) {
t.Fatalf("rendered error leaked the credential: %q", got)
}
})
}
}
+5 -5
View File
@@ -51,8 +51,9 @@ func TestStatusConversionFixtures(t *testing.T) {
var fixture struct {
Cases []struct {
ID string `json:"id"`
Status json.RawMessage `json:"status"`
ID string `json:"id"`
WantSuccess bool `json:"wantSuccess"`
Status json.RawMessage `json:"status"`
} `json:"cases"`
}
if err := json.Unmarshal(data, &fixture); err != nil {
@@ -65,9 +66,8 @@ func TestStatusConversionFixtures(t *testing.T) {
if err := protojson.Unmarshal(tc.Status, &status); err != nil {
t.Fatalf("parse status: %v", err)
}
want := status.GetCategory() == pb.MxStatusCategory_MX_STATUS_CATEGORY_OK
if got := StatusSucceeded(&status); got != want {
t.Fatalf("StatusSucceeded() = %v, want %v", got, want)
if got := StatusSucceeded(&status); got != tc.WantSuccess {
t.Fatalf("StatusSucceeded() = %v, want %v", got, tc.WantSuccess)
}
})
}
+114 -6
View File
@@ -6,6 +6,7 @@ import (
"strings"
pb "gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go/internal/generated"
"google.golang.org/protobuf/proto"
)
// redactedSecretMarker is the placeholder substituted for credential material in
@@ -49,20 +50,108 @@ func (e *secretRedactingError) Unwrap() error {
return e.err
}
// redactSecrets wraps err so any occurrence of a non-empty secret in the surfaced
// message is redacted, while errors.As / errors.Is still reach the wrapped typed
// error. It returns nil unchanged and skips wrapping when no non-empty secret is
// supplied, so non-secret-bearing calls keep their original error verbatim.
// scrubReplyStrings returns a clone of reply with every non-empty secret replaced
// by redactedSecretMarker in the free-text fields a gateway diagnostic could echo a
// credential into: ProtocolStatus.Message, DiagnosticMessage, and each
// Statuses[].DiagnosticText. It clones with proto.Clone so the caller's original
// reply is never mutated. A nil reply, or an empty/whitespace-only secret set, is a
// no-op (nil in, nil out; a clone otherwise).
func scrubReplyStrings(reply *pb.MxCommandReply, secrets []string) *pb.MxCommandReply {
if reply == nil {
return nil
}
clone, ok := proto.Clone(reply).(*pb.MxCommandReply)
if !ok {
return reply
}
for _, secret := range secrets {
if secret == "" {
continue
}
if clone.GetProtocolStatus() != nil {
clone.ProtocolStatus.Message = strings.ReplaceAll(clone.GetProtocolStatus().GetMessage(), secret, redactedSecretMarker)
}
clone.DiagnosticMessage = strings.ReplaceAll(clone.GetDiagnosticMessage(), secret, redactedSecretMarker)
for _, status := range clone.GetStatuses() {
status.DiagnosticText = strings.ReplaceAll(status.GetDiagnosticText(), secret, redactedSecretMarker)
}
}
return clone
}
// scrubProtocolStatusMessage returns a clone of status with every non-empty secret
// redacted from its Message, leaving the original untouched.
func scrubProtocolStatusMessage(status *ProtocolStatus, secrets []string) *ProtocolStatus {
if status == nil {
return nil
}
clone, ok := proto.Clone(status).(*ProtocolStatus)
if !ok {
return status
}
for _, secret := range secrets {
if secret != "" {
clone.Message = strings.ReplaceAll(clone.GetMessage(), secret, redactedSecretMarker)
}
}
return clone
}
// redactSecrets scrubs a non-empty secret set from the error it surfaces. When the
// wrapped error is a typed *MxAccessError or *CommandError it is rebuilt carrying
// scrubbed clones of its reply and protocol status, so a caller logging the typed
// error's structured fields cannot reintroduce the credential the rendered message
// hides. The rebuilt (or original, for other error types) value is then wrapped in
// secretRedactingError as a belt-and-suspenders scrub of any remaining rendered
// text. errors.As / errors.Is still reach the typed error through the wrapper. It
// returns nil unchanged and skips all work when no non-empty secret is supplied, so
// non-secret-bearing calls keep their original error verbatim.
func redactSecrets(err error, secrets ...string) error {
if err == nil {
return nil
}
hasSecret := false
for _, secret := range secrets {
if secret != "" {
return &secretRedactingError{err: err, secrets: secrets}
hasSecret = true
break
}
}
return err
if !hasSecret {
return err
}
rebuilt := rebuildScrubbedError(err, secrets)
return &secretRedactingError{err: rebuilt, secrets: secrets}
}
// rebuildScrubbedError rebuilds the typed error carrying scrubbed clones of any
// command reply / protocol status it holds, so credential text never survives in the
// error's structured fields. Non-reply-bearing error types are returned unchanged.
func rebuildScrubbedError(err error, secrets []string) error {
switch typed := err.(type) {
case *MxAccessError:
return &MxAccessError{
Command: scrubCommandError(typed.Command, secrets),
Reply: scrubReplyStrings(typed.Reply, secrets),
}
case *CommandError:
return scrubCommandError(typed, secrets)
default:
return err
}
}
// scrubCommandError rebuilds a CommandError with a scrubbed Status and Reply.
func scrubCommandError(cmd *CommandError, secrets []string) *CommandError {
if cmd == nil {
return nil
}
return &CommandError{
Op: cmd.Op,
Status: scrubProtocolStatusMessage(cmd.Status, secrets),
Reply: scrubReplyStrings(cmd.Reply, secrets),
}
}
// ErrSlowConsumer is the terminal error sent on the Events/EventsAfter
@@ -72,6 +161,25 @@ func redactSecrets(err error, secrets ...string) error {
// dropping events. Match it with errors.Is.
var ErrSlowConsumer = errors.New("mxgateway: event consumer fell behind; stream terminated")
// MalformedReplyError reports an OK command reply that carried neither the
// typed payload the operation expected nor a usable int32 return_value, so the
// client cannot produce a result. It gives every affected helper one uniform,
// inspectable failure instead of silently returning a zero value.
type MalformedReplyError struct {
// Op names the operation whose reply was malformed.
Op string
// Detail explains what the reply was missing.
Detail string
}
// Error returns the formatted malformed-reply message.
func (e *MalformedReplyError) Error() string {
if e == nil {
return ""
}
return fmt.Sprintf("mxgateway: %s returned a malformed reply: %s", e.Op, e.Detail)
}
// GatewayError wraps transport-level gRPC failures.
type GatewayError struct {
// Op names the operation that failed (for example "dial" or "invoke").
@@ -0,0 +1,115 @@
package mxgateway
import (
"errors"
"strings"
"testing"
pb "gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go/internal/generated"
)
// TestScrubReplyStringsRedactsEveryOccurrence covers the multi-occurrence case:
// one secret appearing across ProtocolStatus.Message, DiagnosticMessage, and every
// Statuses[].DiagnosticText must be fully redacted with no residue.
func TestScrubReplyStringsRedactsEveryOccurrence(t *testing.T) {
const secret = "hunter2"
reply := &pb.MxCommandReply{
ProtocolStatus: &pb.ProtocolStatus{Message: "rejected hunter2 then hunter2 again"},
DiagnosticMessage: "echoed hunter2 back",
Statuses: []*pb.MxStatusProxy{
{DiagnosticText: "first hunter2"},
{DiagnosticText: "second hunter2 and hunter2"},
},
}
scrubbed := scrubReplyStrings(reply, []string{secret})
if strings.Contains(scrubbed.GetProtocolStatus().GetMessage(), secret) {
t.Fatalf("ProtocolStatus.Message still contains the secret: %q", scrubbed.GetProtocolStatus().GetMessage())
}
if strings.Contains(scrubbed.GetDiagnosticMessage(), secret) {
t.Fatalf("DiagnosticMessage still contains the secret: %q", scrubbed.GetDiagnosticMessage())
}
for i, status := range scrubbed.GetStatuses() {
if strings.Contains(status.GetDiagnosticText(), secret) {
t.Fatalf("Statuses[%d].DiagnosticText still contains the secret: %q", i, status.GetDiagnosticText())
}
}
if !strings.Contains(scrubbed.GetProtocolStatus().GetMessage(), redactedSecretMarker) {
t.Fatalf("ProtocolStatus.Message missing redaction marker: %q", scrubbed.GetProtocolStatus().GetMessage())
}
// The original reply must be untouched (scrubReplyStrings clones).
if !strings.Contains(reply.GetDiagnosticMessage(), secret) {
t.Fatal("scrubReplyStrings mutated the original reply instead of cloning it")
}
}
// TestScrubReplyStringsRedactsOverlappingSecrets covers two secrets where one is a
// substring of the other: both must be fully redacted, with no partial leak of the
// longer secret's non-shared remainder.
func TestScrubReplyStringsRedactsOverlappingSecrets(t *testing.T) {
const shortSecret = "pass"
const longSecret = "password123"
reply := &pb.MxCommandReply{
DiagnosticMessage: "value was password123 and also pass",
}
scrubbed := scrubReplyStrings(reply, []string{longSecret, shortSecret})
got := scrubbed.GetDiagnosticMessage()
if strings.Contains(got, shortSecret) {
t.Fatalf("scrubbed message still contains a secret substring %q: %q", shortSecret, got)
}
if strings.Contains(got, longSecret) {
t.Fatalf("scrubbed message still contains %q: %q", longSecret, got)
}
// "123" is the longer secret's remainder past the shared "pass" prefix; it must
// not survive as a partial leak.
if strings.Contains(got, "123") {
t.Fatalf("scrubbed message leaked the longer secret's remainder: %q", got)
}
}
// TestRedactSecretsEmptyOrNilLeavesErrorUnchanged confirms the no-secret paths keep
// the original typed error verbatim (no wrapping, no scrubbed clone).
func TestRedactSecretsEmptyOrNilLeavesErrorUnchanged(t *testing.T) {
base := &MxAccessError{Reply: &pb.MxCommandReply{DiagnosticMessage: "boom"}}
if got := redactSecrets(base); got != error(base) {
t.Fatalf("redactSecrets with no secrets = %v, want the original error unchanged", got)
}
if got := redactSecrets(base, ""); got != error(base) {
t.Fatalf("redactSecrets with only an empty secret = %v, want the original error unchanged", got)
}
if got := redactSecrets(nil, "secret"); got != nil {
t.Fatalf("redactSecrets(nil, ...) = %v, want nil", got)
}
}
// TestRedactSecretsRebuildsTypedCommandError confirms a *CommandError (non-MXAccess
// path) is rebuilt with a scrubbed Status and Reply, and errors.As still reaches it.
func TestRedactSecretsRebuildsTypedCommandError(t *testing.T) {
const secret = "topSecretValue"
base := &CommandError{
Op: "write secured",
Status: &pb.ProtocolStatus{Message: "rejected topSecretValue"},
Reply: &pb.MxCommandReply{DiagnosticMessage: "echoed topSecretValue"},
}
redacted := redactSecrets(base, secret)
var cmdErr *CommandError
if !errors.As(redacted, &cmdErr) {
t.Fatalf("redactSecrets result %T does not unwrap to *CommandError", redacted)
}
if strings.Contains(cmdErr.Status.GetMessage(), secret) {
t.Fatalf("CommandError.Status.Message leaked the secret: %q", cmdErr.Status.GetMessage())
}
if strings.Contains(cmdErr.Reply.GetDiagnosticMessage(), secret) {
t.Fatalf("CommandError.Reply.DiagnosticMessage leaked the secret: %q", cmdErr.Reply.GetDiagnosticMessage())
}
if strings.Contains(redacted.Error(), secret) {
t.Fatalf("rendered error leaked the secret: %q", redacted.Error())
}
}
+64 -14
View File
@@ -812,7 +812,13 @@ func (s *Session) AuthenticateUser(ctx context.Context, serverHandle int32, veri
if reply.GetAuthenticateUser() != nil {
return reply.GetAuthenticateUser().GetUserId(), nil
}
return reply.GetReturnValue().GetInt32Value(), nil
if x, ok := reply.GetReturnValue().GetKind().(*pb.MxValue_Int32Value); ok {
return x.Int32Value, nil
}
return 0, &MalformedReplyError{
Op: "authenticate user",
Detail: "reply carried neither an AuthenticateUser payload nor an int32 return_value",
}
}
// AuthenticateUserRaw invokes MXAccess AuthenticateUser and returns the raw
@@ -847,7 +853,13 @@ func (s *Session) ArchestrAUserToId(ctx context.Context, serverHandle int32, use
if reply.GetArchestraUserToId() != nil {
return reply.GetArchestraUserToId().GetUserId(), nil
}
return reply.GetReturnValue().GetInt32Value(), nil
if x, ok := reply.GetReturnValue().GetKind().(*pb.MxValue_Int32Value); ok {
return x.Int32Value, nil
}
return 0, &MalformedReplyError{
Op: "archestra user to id",
Detail: "reply carried neither an ArchestrAUserToId payload nor an int32 return_value",
}
}
// ArchestrAUserToIdRaw invokes MXAccess ArchestrAUserToId and returns the raw reply.
@@ -876,7 +888,13 @@ func (s *Session) AddBufferedItem(ctx context.Context, serverHandle int32, itemD
if reply.GetAddBufferedItem() != nil {
return reply.GetAddBufferedItem().GetItemHandle(), nil
}
return reply.GetReturnValue().GetInt32Value(), nil
if x, ok := reply.GetReturnValue().GetKind().(*pb.MxValue_Int32Value); ok {
return x.Int32Value, nil
}
return 0, &MalformedReplyError{
Op: "add buffered item",
Detail: "reply carried neither an AddBufferedItem payload nor an int32 return_value",
}
}
// AddBufferedItemRaw invokes MXAccess AddBufferedItem and returns the raw reply.
@@ -981,10 +999,12 @@ func stringSecrets(values ...*MxValue) []string {
// context cancellation stops Recv, or a terminal error is sent.
//
// The returned channel is buffered. If the consumer falls behind and the buffer
// overflows, the stream is terminated and a final EventResult carrying a
// GatewayError that wraps ErrSlowConsumer is delivered before the channel
// closes. Callers must match it with errors.Is(res.Err, ErrSlowConsumer) to
// distinguish a slow-consumer drop from a graceful server end. Use
// overflows with data, the stream is terminated and a final EventResult carrying
// a GatewayError that wraps ErrSlowConsumer is delivered before the channel
// closes; match it with errors.Is(res.Err, ErrSlowConsumer) to distinguish a
// slow-consumer drop from a graceful server end. A genuine stream error is
// reported as itself even under overflow — it is never relabeled as
// ErrSlowConsumer, so the underlying gRPC status stays inspectable. Use
// SubscribeEvents for a blocking, backpressured stream that never drops.
func (s *Session) Events(ctx context.Context) (<-chan EventResult, error) {
return s.EventsAfter(ctx, 0)
@@ -994,7 +1014,9 @@ func (s *Session) Events(ctx context.Context) (<-chan EventResult, error) {
//
// Like Events, the returned channel is buffered and terminates with a final
// EventResult wrapping ErrSlowConsumer (matchable via errors.Is) if the consumer
// falls behind and the buffer overflows, rather than silently closing.
// falls behind and the buffer overflows with data, rather than silently closing.
// A genuine stream error is reported as itself even under overflow, never
// relabeled as ErrSlowConsumer.
func (s *Session) EventsAfter(ctx context.Context, afterWorkerSequence uint64) (<-chan EventResult, error) {
subscription, err := s.subscribeEventsAfter(ctx, afterWorkerSequence, true)
if err != nil {
@@ -1048,12 +1070,11 @@ func (s *Session) subscribeEventsAfter(ctx context.Context, afterWorkerSequence
if err == io.EOF || status.Code(err) == codes.Canceled || streamCtx.Err() != nil {
return
}
sendEventResult(
streamCtx,
results,
EventResult{Err: &GatewayError{Op: "stream events", Err: err}},
cancelWhenResultBufferFull,
cancel)
// A genuine terminal stream error must be reported as itself, even
// when the data slots are full. Routing it through sendEventResult
// would let the overflow branch substitute ErrSlowConsumer and lose
// the real gRPC status, so send it directly, bypassing that branch.
sendTerminalEventResult(streamCtx, results, EventResult{Err: &GatewayError{Op: "stream events", Err: err}}, cancelWhenResultBufferFull)
return
}
}()
@@ -1072,6 +1093,35 @@ func ensureBulkSize(name string, length int) error {
return nil
}
// sendTerminalEventResult enqueues a terminal EventResult, bypassing
// sendEventResult's overflow branch so a genuine stream error is reported verbatim
// rather than relabeled as ErrSlowConsumer. How it sends depends on the mode:
//
// - cancelWhenBufferFull=true (Events/EventsAfter): ordinary data sends are capped
// at eventBufferSize, leaving eventBufferReservedSlots free, so a non-blocking
// send always lands the terminal result. Because this goroutine is the sole
// producer, at most one terminal send ever races for the reserved slot, so the
// select default only fires when the reserve is already spent — never dropping a
// first terminal error.
// - cancelWhenBufferFull=false (SubscribeEvents/SubscribeEventsAfter, never-drop):
// ordinary data sends are uncapped and blocking, so every slot including the
// reserve can hold data. A non-blocking send would then hit the full buffer and
// silently drop the terminal error, breaking the never-drop contract; instead
// block until the consumer drains a slot (or the stream context is cancelled).
func sendTerminalEventResult(ctx context.Context, results chan<- EventResult, result EventResult, cancelWhenBufferFull bool) {
if cancelWhenBufferFull {
select {
case results <- result:
default:
}
return
}
select {
case results <- result:
case <-ctx.Done():
}
}
func sendEventResult(
ctx context.Context,
results chan<- EventResult,
+1 -1
View File
@@ -3,7 +3,7 @@ package mxgateway
const (
// ClientVersion is the released semantic version of this Go client module.
// Keep it in sync with the module tag applied by scripts/tag-go-module.ps1.
ClientVersion = "0.1.2"
ClientVersion = "0.2.0"
// GatewayProtocolVersion matches GatewayContractInfo.GatewayProtocolVersion
// in the shared .NET contracts.
+1 -1
View File
@@ -465,7 +465,7 @@ repositories {
}
dependencies {
implementation 'com.zb.mom.ww.mxgateway:zb-mom-ww-mxgateway-client:0.1.2'
implementation 'com.zb.mom.ww.mxgateway:zb-mom-ww-mxgateway-client:0.2.1'
}
````
+8 -1
View File
@@ -13,7 +13,14 @@ ext {
subprojects {
group = 'com.zb.mom.ww.mxgateway'
version = '0.2.0'
// 0.2.0 was already published to the Gitea Maven feed on 2026-06-26,
// before the CLI-37/38/40/41 conformance fixes changed the client's
// observable behavior (status.category-based validation, hresult < 0,
// exact-secret redaction, typed malformed-reply errors). Bump to 0.2.1
// so the published coordinate matches the conformant behavior the other
// four clients ship at 0.2.0 for the first time. See CLI-39 and the
// "Versioning" section of docs/ClientPackaging.md.
version = '0.2.1'
pluginManager.withPlugin('java') {
java {
@@ -69110,24 +69110,59 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
com.google.protobuf.MessageOrBuilder {
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
java.util.List<mxaccess_gateway.v1.MxaccessGateway.MxEvent>
getEventsList();
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
mxaccess_gateway.v1.MxaccessGateway.MxEvent getEvents(int index);
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
int getEventsCount();
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
java.util.List<? extends mxaccess_gateway.v1.MxaccessGateway.MxEventOrBuilder>
getEventsOrBuilderList();
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
mxaccess_gateway.v1.MxaccessGateway.MxEventOrBuilder getEventsOrBuilder(
@@ -69175,6 +69210,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
@SuppressWarnings("serial")
private java.util.List<mxaccess_gateway.v1.MxaccessGateway.MxEvent> events_;
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
@java.lang.Override
@@ -69182,6 +69224,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
return events_;
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
@java.lang.Override
@@ -69190,6 +69239,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
return events_;
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
@java.lang.Override
@@ -69197,6 +69253,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
return events_.size();
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
@java.lang.Override
@@ -69204,6 +69267,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
return events_.get(index);
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
@java.lang.Override
@@ -69567,6 +69637,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
mxaccess_gateway.v1.MxaccessGateway.MxEvent, mxaccess_gateway.v1.MxaccessGateway.MxEvent.Builder, mxaccess_gateway.v1.MxaccessGateway.MxEventOrBuilder> eventsBuilder_;
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public java.util.List<mxaccess_gateway.v1.MxaccessGateway.MxEvent> getEventsList() {
@@ -69577,6 +69654,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
}
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public int getEventsCount() {
@@ -69587,6 +69671,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
}
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public mxaccess_gateway.v1.MxaccessGateway.MxEvent getEvents(int index) {
@@ -69597,6 +69688,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
}
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public Builder setEvents(
@@ -69614,6 +69712,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
return this;
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public Builder setEvents(
@@ -69628,6 +69733,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
return this;
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public Builder addEvents(mxaccess_gateway.v1.MxaccessGateway.MxEvent value) {
@@ -69644,6 +69756,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
return this;
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public Builder addEvents(
@@ -69661,6 +69780,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
return this;
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public Builder addEvents(
@@ -69675,6 +69801,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
return this;
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public Builder addEvents(
@@ -69689,6 +69822,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
return this;
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public Builder addAllEvents(
@@ -69704,6 +69844,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
return this;
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public Builder clearEvents() {
@@ -69717,6 +69864,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
return this;
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public Builder removeEvents(int index) {
@@ -69730,6 +69884,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
return this;
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public mxaccess_gateway.v1.MxaccessGateway.MxEvent.Builder getEventsBuilder(
@@ -69737,6 +69898,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
return internalGetEventsFieldBuilder().getBuilder(index);
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public mxaccess_gateway.v1.MxaccessGateway.MxEventOrBuilder getEventsOrBuilder(
@@ -69747,6 +69915,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
}
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public java.util.List<? extends mxaccess_gateway.v1.MxaccessGateway.MxEventOrBuilder>
@@ -69758,6 +69933,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
}
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public mxaccess_gateway.v1.MxaccessGateway.MxEvent.Builder addEventsBuilder() {
@@ -69765,6 +69947,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
mxaccess_gateway.v1.MxaccessGateway.MxEvent.getDefaultInstance());
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public mxaccess_gateway.v1.MxaccessGateway.MxEvent.Builder addEventsBuilder(
@@ -69773,6 +69962,13 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
index, mxaccess_gateway.v1.MxaccessGateway.MxEvent.getDefaultInstance());
}
/**
* <pre>
* The reply is bounded by both a server-side count cap and the negotiated
* worker-frame byte cap; a reply may therefore carry fewer events than
* `max_events` and fewer than are queued. Callers drain iteratively until an
* empty reply.
* </pre>
*
* <code>repeated .mxaccess_gateway.v1.MxEvent events = 1;</code>
*/
public java.util.List<mxaccess_gateway.v1.MxaccessGateway.MxEvent.Builder>
@@ -75305,6 +75501,11 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
* after_worker_sequence = oldest_available_sequence - 1 in the next
* StreamEventsRequest, which will cause the server to replay starting at
* oldest_available_sequence (the first retained event).
* When nothing is retained (the replay ring is empty), this is the next sequence
* that can be delivered — `highest observed + 1` — and the `oldest - 1` resume
* formula remains valid: it resolves to the highest sequence already seen, so the
* follow-up resume replays nothing, reports no gap, and every newer live event
* passes. The interval evicted is unchanged.
* </pre>
*
* <code>uint64 oldest_available_sequence = 2;</code>
@@ -75386,6 +75587,11 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
* after_worker_sequence = oldest_available_sequence - 1 in the next
* StreamEventsRequest, which will cause the server to replay starting at
* oldest_available_sequence (the first retained event).
* When nothing is retained (the replay ring is empty), this is the next sequence
* that can be delivered — `highest observed + 1` — and the `oldest - 1` resume
* formula remains valid: it resolves to the highest sequence already seen, so the
* follow-up resume replays nothing, reports no gap, and every newer live event
* passes. The interval evicted is unchanged.
* </pre>
*
* <code>uint64 oldest_available_sequence = 2;</code>
@@ -75781,6 +75987,11 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
* after_worker_sequence = oldest_available_sequence - 1 in the next
* StreamEventsRequest, which will cause the server to replay starting at
* oldest_available_sequence (the first retained event).
* When nothing is retained (the replay ring is empty), this is the next sequence
* that can be delivered — `highest observed + 1` — and the `oldest - 1` resume
* formula remains valid: it resolves to the highest sequence already seen, so the
* follow-up resume replays nothing, reports no gap, and every newer live event
* passes. The interval evicted is unchanged.
* </pre>
*
* <code>uint64 oldest_available_sequence = 2;</code>
@@ -75800,6 +76011,11 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
* after_worker_sequence = oldest_available_sequence - 1 in the next
* StreamEventsRequest, which will cause the server to replay starting at
* oldest_available_sequence (the first retained event).
* When nothing is retained (the replay ring is empty), this is the next sequence
* that can be delivered — `highest observed + 1` — and the `oldest - 1` resume
* formula remains valid: it resolves to the highest sequence already seen, so the
* follow-up resume replays nothing, reports no gap, and every newer live event
* passes. The interval evicted is unchanged.
* </pre>
*
* <code>uint64 oldest_available_sequence = 2;</code>
@@ -75823,6 +76039,11 @@ public final class MxaccessGateway extends com.google.protobuf.GeneratedFile {
* after_worker_sequence = oldest_available_sequence - 1 in the next
* StreamEventsRequest, which will cause the server to replay starting at
* oldest_available_sequence (the first retained event).
* When nothing is retained (the replay ring is empty), this is the next sequence
* that can be delivered — `highest observed + 1` — and the `oldest - 1` resume
* formula remains valid: it resolves to the highest sequence already seen, so the
* follow-up resume replays nothing, reports no gap, and every newer live event
* passes. The interval evicted is unchanged.
* </pre>
*
* <code>uint64 oldest_available_sequence = 2;</code>
@@ -3797,6 +3797,9 @@ public final class MxaccessWorker extends com.google.protobuf.GeneratedFile {
* instead of a hard-coded default; 0 (an older gateway that never set the field) means
* "use the worker's built-in default". Sits above the public gRPC cap by an
* envelope-overhead margin so an accepted gRPC payload always fits one worker frame.
* Every worker-&gt;gateway frame — events, heartbeats, faults, and control replies
* including DrainEvents — must serialize within this limit; reply builders truncate
* to fit rather than emit an oversized frame.
* </pre>
*
* <code>uint32 max_frame_bytes = 4;</code>
@@ -3941,6 +3944,9 @@ public final class MxaccessWorker extends com.google.protobuf.GeneratedFile {
* instead of a hard-coded default; 0 (an older gateway that never set the field) means
* "use the worker's built-in default". Sits above the public gRPC cap by an
* envelope-overhead margin so an accepted gRPC payload always fits one worker frame.
* Every worker-&gt;gateway frame — events, heartbeats, faults, and control replies
* including DrainEvents — must serialize within this limit; reply builders truncate
* to fit rather than emit an oversized frame.
* </pre>
*
* <code>uint32 max_frame_bytes = 4;</code>
@@ -4499,6 +4505,9 @@ public final class MxaccessWorker extends com.google.protobuf.GeneratedFile {
* instead of a hard-coded default; 0 (an older gateway that never set the field) means
* "use the worker's built-in default". Sits above the public gRPC cap by an
* envelope-overhead margin so an accepted gRPC payload always fits one worker frame.
* Every worker-&gt;gateway frame — events, heartbeats, faults, and control replies
* including DrainEvents — must serialize within this limit; reply builders truncate
* to fit rather than emit an oversized frame.
* </pre>
*
* <code>uint32 max_frame_bytes = 4;</code>
@@ -4515,6 +4524,9 @@ public final class MxaccessWorker extends com.google.protobuf.GeneratedFile {
* instead of a hard-coded default; 0 (an older gateway that never set the field) means
* "use the worker's built-in default". Sits above the public gRPC cap by an
* envelope-overhead margin so an accepted gRPC payload always fits one worker frame.
* Every worker-&gt;gateway frame — events, heartbeats, faults, and control replies
* including DrainEvents — must serialize within this limit; reply builders truncate
* to fit rather than emit an oversized frame.
* </pre>
*
* <code>uint32 max_frame_bytes = 4;</code>
@@ -4535,6 +4547,9 @@ public final class MxaccessWorker extends com.google.protobuf.GeneratedFile {
* instead of a hard-coded default; 0 (an older gateway that never set the field) means
* "use the worker's built-in default". Sits above the public gRPC cap by an
* envelope-overhead margin so an accepted gRPC payload always fits one worker frame.
* Every worker-&gt;gateway frame — events, heartbeats, faults, and control replies
* including DrainEvents — must serialize within this limit; reply builders truncate
* to fit rather than emit an oversized frame.
* </pre>
*
* <code>uint32 max_frame_bytes = 4;</code>
@@ -59,7 +59,7 @@ final class MxGatewayCliTests {
assertEquals(0, run.exitCode());
assertEquals("", run.errors());
assertTrue(run.output().contains("mxgateway-java 0.2.0"));
assertTrue(run.output().contains("mxgateway-java 0.2.1"));
assertTrue(run.output().contains("gatewayProtocolVersion=3"));
assertTrue(run.output().contains("workerProtocolVersion=1"));
}
@@ -89,7 +89,7 @@ final class MxGatewayCliTests {
CliRun run = execute(new FakeClientFactory(), "version", "--json");
assertEquals(0, run.exitCode());
assertTrue(run.output().contains("\"clientVersion\":\"0.2.0\""));
assertTrue(run.output().contains("\"clientVersion\":\"0.2.1\""));
assertTrue(run.output().contains("\"gatewayProtocolVersion\":3"));
}
@@ -63,10 +63,11 @@ protobuf {
// or a plugin/protobuf version bump, silently drifts the committed output. checkGeneratedClean
// fails when the regenerated tree differs from what is committed.
//
// Caveat (repo memory project_java_generated_churn): the protobuf gradle plugin also rewrites
// MxaccessGateway.java with a spurious protobuf-runtime-version delta on every build even when no
// .proto changed. CI reverts that one file (git checkout) before invoking this task; locally, do the
// same when you did not touch a .proto. See docs/GatewayTesting.md "Continuous Integration".
// The grpc/protobuf toolchain is pinned (build.gradle: grpcVersion / protobufVersion), so a
// regeneration is byte-identical to the committed single-file aggregates modulo real .proto
// changes — no spurious protobuf-runtime-version churn (IPC-24 verified this and deleted the old
// unconditional CI churn-revert step, which masked message-level drift). Regenerate and commit
// after any .proto change. See docs/GatewayTesting.md "Continuous Integration".
tasks.register('checkGeneratedClean') {
group = 'verification'
description = 'Fails if the committed generated Java tree differs from a fresh regeneration.'
@@ -83,9 +84,9 @@ tasks.register('checkGeneratedClean') {
def dirty = stdout.toString().trim()
if (!dirty.isEmpty()) {
throw new GradleException(
"Generated Java is stale or churned:\n${dirty}\n" +
"Regenerate and commit after a .proto change, or 'git checkout' the spurious " +
"MxaccessGateway.java protobuf-version churn when no .proto changed.")
"Generated Java is stale:\n${dirty}\n" +
"Regenerate and commit the Java client after a .proto change " +
"(gradle :zb-mom-ww-mxgateway-client:generateProto).")
}
}
}
@@ -29,4 +29,18 @@ public final class MxAccessException extends MxGatewayCommandException {
public MxAccessException(String operation, MxCommandReply reply) {
super(operation, reply == null ? null : reply.getProtocolStatus(), reply);
}
/**
* Creates a new MXAccess exception with an already-built, verbatim message.
* Used to re-surface an MXAccess failure with a redacted message while
* preserving the original protocol status and reply.
*
* @param message the exact message to surface (already formatted/redacted)
* @param protocolStatus protocol status reported by the gateway
* @param reply raw command reply containing the MXAccess failure detail
* @param cause underlying error, or {@code null}
*/
public MxAccessException(String message, ProtocolStatus protocolStatus, MxCommandReply reply, Throwable cause) {
super(message, protocolStatus, reply, cause);
}
}
@@ -9,7 +9,7 @@ package com.zb.mom.ww.mxgateway.client;
public final class MxGatewayClientVersion {
private static final int GATEWAY_PROTOCOL_VERSION = 3;
private static final int WORKER_PROTOCOL_VERSION = 1;
private static final String CLIENT_VERSION = "0.2.0";
private static final String CLIENT_VERSION = "0.2.1";
private MxGatewayClientVersion() {
}
@@ -25,6 +25,23 @@ public class MxGatewayCommandException extends MxGatewayException {
this.reply = reply;
}
/**
* Creates a new command exception with an already-built, verbatim message.
* Used to re-surface a failure with a redacted message while preserving the
* original protocol status and reply.
*
* @param message the exact message to surface (already formatted/redacted)
* @param protocolStatus protocol status returned by the gateway
* @param reply raw command reply, or {@code null} when none was produced
* @param cause underlying error, or {@code null}
*/
protected MxGatewayCommandException(
String message, ProtocolStatus protocolStatus, MxCommandReply reply, Throwable cause) {
super(message, cause);
this.protocolStatus = protocolStatus;
this.reply = reply;
}
/**
* Returns the gateway protocol status that triggered this exception.
*
@@ -0,0 +1,32 @@
package com.zb.mom.ww.mxgateway.client;
/**
* Thrown when the gateway returns a protocol-OK command reply that carries
* neither the expected typed payload nor a usable {@code return_value}.
*
* <p>A successful reply for a value-returning command (for example
* {@code AuthenticateUser}, {@code ArchestrAUserToId}, or {@code AddBufferedItem})
* must supply either the command's typed payload or an int32 {@code return_value}.
* A reply that satisfies neither is malformed, and the client surfaces this
* distinct failure rather than silently returning a default {@code 0}.
*/
public final class MxGatewayMalformedReplyException extends MxGatewayException {
/**
* Creates a new malformed-reply exception with the supplied message.
*
* @param message human-readable description of the malformed reply
*/
public MxGatewayMalformedReplyException(String message) {
super(message);
}
/**
* Creates a new malformed-reply exception with the supplied message and cause.
*
* @param message human-readable description of the malformed reply
* @param cause underlying error that triggered the failure
*/
public MxGatewayMalformedReplyException(String message, Throwable cause) {
super(message, cause);
}
}
@@ -54,4 +54,34 @@ public final class MxGatewaySecrets {
}
return String.join(" ", parts);
}
/**
* Replaces every occurrence of each supplied secret with the redaction
* marker {@code "<redacted>"}. Unlike {@link #redactCredentials(String)},
* which scrubs by pattern, this performs an exact-substring scrub of the
* caller-known secrets — used to strip a credential the gateway echoed back
* into a free-form failure message.
*
* @param message the message to scrub, may be {@code null}
* @param secrets the exact secret substrings to remove; {@code null}, empty,
* and blank (whitespace-only) entries and a {@code null} array are ignored
* @return {@code message} unchanged when it is {@code null} or no non-blank
* secret is supplied, otherwise the message with every secret occurrence
* replaced by {@code "<redacted>"}
*/
public static String redactExact(String message, String... secrets) {
if (message == null || secrets == null) {
return message;
}
String result = message;
for (String secret : secrets) {
if (secret == null || secret.isBlank()) {
// A blank "secret" would over-redact real whitespace; skip it.
continue;
}
result = result.replace(secret, "<redacted>");
}
return result;
}
}
@@ -31,6 +31,7 @@ import mxaccess_gateway.v1.MxaccessGateway.MxSparseElement;
import mxaccess_gateway.v1.MxaccessGateway.MxStatusProxy;
import mxaccess_gateway.v1.MxaccessGateway.MxValue;
import mxaccess_gateway.v1.MxaccessGateway.OpenSessionReply;
import mxaccess_gateway.v1.MxaccessGateway.ProtocolStatus;
import mxaccess_gateway.v1.MxaccessGateway.ReadBulkCommand;
import mxaccess_gateway.v1.MxaccessGateway.RegisterCommand;
import mxaccess_gateway.v1.MxaccessGateway.RemoveItemBulkCommand;
@@ -782,15 +783,17 @@ public final class MxGatewaySession implements AutoCloseable {
*/
public MxCommandReply writeSecuredRaw(
int serverHandle, int itemHandle, int currentUserId, int verifierUserId, MxValue value) {
return invokeCommand(MxCommand.newBuilder()
.setKind(MxCommandKind.MX_COMMAND_KIND_WRITE_SECURED)
.setWriteSecured(WriteSecuredCommand.newBuilder()
.setServerHandle(serverHandle)
.setItemHandle(itemHandle)
.setCurrentUserId(currentUserId)
.setVerifierUserId(verifierUserId)
.setValue(value))
.build());
return invokeCommandRedacted(
MxCommand.newBuilder()
.setKind(MxCommandKind.MX_COMMAND_KIND_WRITE_SECURED)
.setWriteSecured(WriteSecuredCommand.newBuilder()
.setServerHandle(serverHandle)
.setItemHandle(itemHandle)
.setCurrentUserId(currentUserId)
.setVerifierUserId(verifierUserId)
.setValue(value))
.build(),
secretStringOf(value));
}
/**
@@ -837,16 +840,18 @@ public final class MxGatewaySession implements AutoCloseable {
int verifierUserId,
MxValue value,
MxValue timestampValue) {
return invokeCommand(MxCommand.newBuilder()
.setKind(MxCommandKind.MX_COMMAND_KIND_WRITE_SECURED2)
.setWriteSecured2(WriteSecured2Command.newBuilder()
.setServerHandle(serverHandle)
.setItemHandle(itemHandle)
.setCurrentUserId(currentUserId)
.setVerifierUserId(verifierUserId)
.setValue(value)
.setTimestampValue(timestampValue))
.build());
return invokeCommandRedacted(
MxCommand.newBuilder()
.setKind(MxCommandKind.MX_COMMAND_KIND_WRITE_SECURED2)
.setWriteSecured2(WriteSecured2Command.newBuilder()
.setServerHandle(serverHandle)
.setItemHandle(itemHandle)
.setCurrentUserId(currentUserId)
.setVerifierUserId(verifierUserId)
.setValue(value)
.setTimestampValue(timestampValue))
.build(),
secretStringOf(value));
}
/**
@@ -866,17 +871,24 @@ public final class MxGatewaySession implements AutoCloseable {
* @throws MxAccessException when MXAccess rejects the credential
*/
public int authenticateUser(int serverHandle, String verifyUser, String verifyUserPassword) {
MxCommandReply reply = invokeCommand(MxCommand.newBuilder()
.setKind(MxCommandKind.MX_COMMAND_KIND_AUTHENTICATE_USER)
.setAuthenticateUser(AuthenticateUserCommand.newBuilder()
.setServerHandle(serverHandle)
.setVerifyUser(verifyUser)
.setVerifyUserPassword(verifyUserPassword))
.build());
MxCommandReply reply = invokeCommandRedacted(
MxCommand.newBuilder()
.setKind(MxCommandKind.MX_COMMAND_KIND_AUTHENTICATE_USER)
.setAuthenticateUser(AuthenticateUserCommand.newBuilder()
.setServerHandle(serverHandle)
.setVerifyUser(verifyUser)
.setVerifyUserPassword(verifyUserPassword))
.build(),
verifyUserPassword);
if (reply.hasAuthenticateUser()) {
return reply.getAuthenticateUser().getUserId();
}
return reply.getReturnValue().getInt32Value();
if (reply.hasReturnValue() && reply.getReturnValue().getKindCase() == MxValue.KindCase.INT32_VALUE) {
return reply.getReturnValue().getInt32Value();
}
throw new MxGatewayMalformedReplyException(
"AuthenticateUser returned a malformed reply: OK reply carried neither "
+ "the typed payload nor an int32 return_value");
}
/**
@@ -899,7 +911,12 @@ public final class MxGatewaySession implements AutoCloseable {
if (reply.hasArchestraUserToId()) {
return reply.getArchestraUserToId().getUserId();
}
return reply.getReturnValue().getInt32Value();
if (reply.hasReturnValue() && reply.getReturnValue().getKindCase() == MxValue.KindCase.INT32_VALUE) {
return reply.getReturnValue().getInt32Value();
}
throw new MxGatewayMalformedReplyException(
"ArchestrAUserToId returned a malformed reply: OK reply carried neither "
+ "the typed payload nor an int32 return_value");
}
/**
@@ -925,7 +942,12 @@ public final class MxGatewaySession implements AutoCloseable {
if (reply.hasAddBufferedItem()) {
return reply.getAddBufferedItem().getItemHandle();
}
return reply.getReturnValue().getInt32Value();
if (reply.hasReturnValue() && reply.getReturnValue().getKindCase() == MxValue.KindCase.INT32_VALUE) {
return reply.getReturnValue().getInt32Value();
}
throw new MxGatewayMalformedReplyException(
"AddBufferedItem returned a malformed reply: OK reply carried neither "
+ "the typed payload nor an int32 return_value");
}
/**
@@ -1027,6 +1049,162 @@ public final class MxGatewaySession implements AutoCloseable {
.build());
}
/**
* Invokes a credential-bearing command, scrubbing any exact secret the
* gateway may have echoed back into a surfaced failure message. The secret
* lives only in the request, but a non-parity gateway or provider can copy
* it into a diagnostic; this guarantees it never survives in the exception
* text a caller might log.
*
* <p>On failure both the exception's message <em>and</em> its structured
* context (the {@link ProtocolStatus} and {@link MxCommandReply} a caller can
* inspect and log) are scrubbed with {@link MxGatewaySecrets#redactExact}: the
* gateway echoes the credential into {@code protocolStatus.message},
* {@code reply.diagnosticMessage}, and each {@code statuses[i].diagnosticText}.
* If nothing carried the secret (the common case) the original exception is
* rethrown untouched. Otherwise it is re-thrown as the same concrete type
* carrying the redacted message and scrubbed context; the secret-bearing
* original is not chained as a cause, so it cannot leak through a printed
* stack trace.
*/
private MxCommandReply invokeCommandRedacted(MxCommand command, String... secrets) {
try {
return invokeCommand(command);
} catch (MxGatewayException ex) {
String original = ex.getMessage();
String redactedMessage = MxGatewaySecrets.redactExact(original, secrets);
boolean messageChanged = redactedMessage != null && !redactedMessage.equals(original);
ProtocolStatus status = protocolStatusOf(ex);
ProtocolStatus scrubbedStatus = scrubProtocolStatus(status, secrets);
boolean statusChanged = status != null && !status.equals(scrubbedStatus);
MxCommandReply reply = replyOf(ex);
MxCommandReply scrubbedReply = scrubReply(reply, secrets);
boolean replyChanged = reply != null && !reply.equals(scrubbedReply);
if (!messageChanged && !statusChanged && !replyChanged) {
throw ex;
}
String message = messageChanged ? redactedMessage : original;
throw rebuildRedacted(ex, message, scrubbedStatus, scrubbedReply);
}
}
/**
* Extracts the {@link ProtocolStatus} an exception carries, if any, so it can
* be scrubbed and re-attached to the rebuilt exception.
*/
private static ProtocolStatus protocolStatusOf(MxGatewayException ex) {
if (ex instanceof MxGatewayCommandException command) {
return command.protocolStatus();
}
if (ex instanceof MxGatewaySessionException session) {
return session.protocolStatus();
}
if (ex instanceof MxGatewayWorkerException worker) {
return worker.protocolStatus();
}
return null;
}
/**
* Extracts the raw {@link MxCommandReply} an exception carries, if any.
*/
private static MxCommandReply replyOf(MxGatewayException ex) {
if (ex instanceof MxGatewayCommandException command) {
return command.reply();
}
return null;
}
/**
* Rebuilds a gateway exception of the same concrete type with a redacted
* message and already-scrubbed context, mirroring the .NET client's
* type-switch. Only a truly-unknown subtype collapses to the base
* {@link MxGatewayException}. The original (secret-bearing) exception is
* deliberately not chained as a cause.
*/
private static MxGatewayException rebuildRedacted(
MxGatewayException ex, String message, ProtocolStatus status, MxCommandReply reply) {
if (ex instanceof MxAccessException) {
return new MxAccessException(message, status, reply, null);
}
if (ex instanceof MxGatewayCommandException) {
return new MxGatewayCommandException(message, status, reply, null);
}
if (ex instanceof MxGatewaySessionException) {
return new MxGatewaySessionException(message, status, null);
}
if (ex instanceof MxGatewayWorkerException) {
return new MxGatewayWorkerException(message, status, null);
}
if (ex instanceof MxGatewayMalformedReplyException) {
return new MxGatewayMalformedReplyException(message);
}
if (ex instanceof MxGatewayAuthenticationException) {
return new MxGatewayAuthenticationException(message, null);
}
if (ex instanceof MxGatewayAuthorizationException) {
return new MxGatewayAuthorizationException(message, null);
}
return new MxGatewayException(message);
}
/**
* Produces a scrubbed clone of a command reply, removing any exact secret the
* gateway echoed into {@code protocolStatus.message},
* {@code diagnosticMessage}, or a status's {@code diagnosticText}.
*
* @param reply the reply to scrub, or {@code null}
* @param secrets the exact secrets to strip
* @return {@code null} when {@code reply} is {@code null}, otherwise a clone
* with every echoed secret replaced by the redaction marker
*/
private static MxCommandReply scrubReply(MxCommandReply reply, String... secrets) {
if (reply == null) {
return null;
}
MxCommandReply.Builder builder = reply.toBuilder();
if (builder.hasProtocolStatus()) {
builder.setProtocolStatus(scrubProtocolStatus(builder.getProtocolStatus(), secrets));
}
builder.setDiagnosticMessage(MxGatewaySecrets.redactExact(builder.getDiagnosticMessage(), secrets));
for (int index = 0; index < builder.getStatusesCount(); index++) {
MxStatusProxy.Builder status = builder.getStatuses(index).toBuilder();
status.setDiagnosticText(MxGatewaySecrets.redactExact(status.getDiagnosticText(), secrets));
builder.setStatuses(index, status);
}
return builder.build();
}
/**
* Produces a scrubbed clone of a protocol status, removing any exact secret
* the gateway echoed into its free-form {@code message}.
*/
private static ProtocolStatus scrubProtocolStatus(ProtocolStatus status, String... secrets) {
if (status == null) {
return null;
}
return status.toBuilder()
.setMessage(MxGatewaySecrets.redactExact(status.getMessage(), secrets))
.build();
}
/**
* Extracts the string payload of a secured-write value so it can be scrubbed
* from an echoed failure message. Only string-kind values carry a
* credential-shaped secret worth redacting; other kinds return {@code null}
* (ignored by {@link MxGatewaySecrets#redactExact}).
*/
private static String secretStringOf(MxValue value) {
if (value != null && value.getKindCase() == MxValue.KindCase.STRING_VALUE) {
return value.getStringValue();
}
return null;
}
private static String newCorrelationId() {
byte[] bytes = new byte[16];
RANDOM.nextBytes(bytes);
@@ -20,6 +20,20 @@ public final class MxGatewaySessionException extends MxGatewayException {
this.protocolStatus = protocolStatus;
}
/**
* Creates a new session exception with an already-built, verbatim message.
* Used to re-surface a failure with a redacted message while preserving the
* (already scrubbed) protocol status.
*
* @param message the exact message to surface (already formatted/redacted)
* @param protocolStatus protocol status returned by the gateway
* @param cause underlying error, or {@code null}
*/
protected MxGatewaySessionException(String message, ProtocolStatus protocolStatus, Throwable cause) {
super(message, cause);
this.protocolStatus = protocolStatus;
}
/**
* Returns the gateway protocol status that triggered this exception.
*
@@ -20,6 +20,20 @@ public final class MxGatewayWorkerException extends MxGatewayException {
this.protocolStatus = protocolStatus;
}
/**
* Creates a new worker exception with an already-built, verbatim message.
* Used to re-surface a failure with a redacted message while preserving the
* (already scrubbed) protocol status.
*
* @param message the exact message to surface (already formatted/redacted)
* @param protocolStatus protocol status returned by the gateway
* @param cause underlying error, or {@code null}
*/
protected MxGatewayWorkerException(String message, ProtocolStatus protocolStatus, Throwable cause) {
super(message, cause);
this.protocolStatus = protocolStatus;
}
/**
* Returns the gateway protocol status that triggered this exception.
*
@@ -0,0 +1,201 @@
package com.zb.mom.ww.mxgateway.client;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import com.google.protobuf.util.JsonFormat;
import io.grpc.ManagedChannel;
import io.grpc.Server;
import io.grpc.inprocess.InProcessChannelBuilder;
import io.grpc.inprocess.InProcessServerBuilder;
import io.grpc.stub.StreamObserver;
import java.nio.file.Files;
import java.nio.file.Path;
import java.time.Duration;
import java.util.UUID;
import mxaccess_gateway.v1.MxAccessGatewayGrpc;
import mxaccess_gateway.v1.MxaccessGateway.MxCommandReply;
import mxaccess_gateway.v1.MxaccessGateway.MxCommandRequest;
import mxaccess_gateway.v1.MxaccessGateway.MxValue;
import mxaccess_gateway.v1.MxaccessGateway.ProtocolStatus;
import mxaccess_gateway.v1.MxaccessGateway.ProtocolStatusCode;
import org.junit.jupiter.api.Test;
final class MxGatewayCredentialReplyTests {
private static final String CREDENTIAL = "sup3rSecretVerify9f3a2b";
@Test
void authenticateUserRedactsEchoedCredentialFromReplyDrivenError() throws Exception {
assertCredentialFullyRedacted(
"authenticate-user.echoed-credential.reply.json", "auth-echo-session");
}
@Test
void authenticateUserRedactsEchoedCredentialFromMxAccessFailureReply() throws Exception {
assertCredentialFullyRedacted(
"authenticate-user.echoed-credential-mxaccess-failure.reply.json",
"auth-echo-failure-session");
}
private static void assertCredentialFullyRedacted(String fixture, String sessionId) throws Exception {
MxCommandReply reply = loadReply(fixture);
try (InProcessGateway gateway = InProcessGateway.startReturning(reply);
MxGatewayClient client = gateway.client()) {
MxGatewaySession session = MxGatewaySession.forSessionId(client, sessionId);
MxAccessException error = assertThrows(
MxAccessException.class,
() -> session.authenticateUser(12, "operator", CREDENTIAL));
assertFalse(error.getMessage().contains(CREDENTIAL),
"credential echoed by the gateway must not survive in the surfaced message");
assertTrue(error.getMessage().contains("<redacted>"),
"the echoed credential must be replaced with the redaction marker");
// The rebuilt exception must not re-expose the credential through the
// structured reply/protocolStatus a caller can inspect and log.
MxCommandReply surfaced = error.reply();
assertNotNull(surfaced, "the redacted exception must preserve a reply for inspection");
assertFalse(surfaced.getProtocolStatus().getMessage().contains(CREDENTIAL),
"reply protocol status message must not leak the echoed credential");
assertFalse(surfaced.getDiagnosticMessage().contains(CREDENTIAL),
"reply diagnostic message must not leak the echoed credential");
for (int index = 0; index < surfaced.getStatusesCount(); index++) {
assertFalse(surfaced.getStatusesList().get(index).getDiagnosticText().contains(CREDENTIAL),
"reply status diagnostic text must not leak the echoed credential");
}
assertNotNull(error.protocolStatus(), "the redacted exception must preserve a protocol status");
assertFalse(error.protocolStatus().getMessage().contains(CREDENTIAL),
"exception protocol status must not leak the echoed credential");
}
}
@Test
void authenticateUserMissingPayloadThrowsMalformedReply() throws Exception {
MxCommandReply reply = loadReply("authenticate-user.missing-payload.reply.json");
try (InProcessGateway gateway = InProcessGateway.startReturning(reply);
MxGatewayClient client = gateway.client()) {
MxGatewaySession session = MxGatewaySession.forSessionId(client, "auth-missing-session");
assertThrows(
MxGatewayMalformedReplyException.class,
() -> session.authenticateUser(3, "operator", "pw"));
}
}
@Test
void authenticateUserReturnValueOnlyReplyReturnsInt32Fallback() throws Exception {
MxCommandReply reply = loadReply("authenticate-user.return-value-only.reply.json");
try (InProcessGateway gateway = InProcessGateway.startReturning(reply);
MxGatewayClient client = gateway.client()) {
MxGatewaySession session = MxGatewaySession.forSessionId(client, "auth-return-session");
assertEquals(7, session.authenticateUser(3, "operator", "pw"));
}
}
@Test
void addBufferedItemReturnValueOnlyReplyReturnsInt32Fallback() throws Exception {
MxCommandReply reply = MxCommandReply.newBuilder()
.setProtocolStatus(ok())
.setReturnValue(MxValue.newBuilder().setInt32Value(55))
.build();
try (InProcessGateway gateway = InProcessGateway.startReturning(reply);
MxGatewayClient client = gateway.client()) {
MxGatewaySession session = MxGatewaySession.forSessionId(client, "buffered-return-session");
assertEquals(55, session.addBufferedItem(3, "Tank01.Level", "galaxy"));
}
}
@Test
void addBufferedItemMissingPayloadThrowsMalformedReply() throws Exception {
MxCommandReply reply = MxCommandReply.newBuilder().setProtocolStatus(ok()).build();
try (InProcessGateway gateway = InProcessGateway.startReturning(reply);
MxGatewayClient client = gateway.client()) {
MxGatewaySession session = MxGatewaySession.forSessionId(client, "buffered-malformed-session");
assertThrows(
MxGatewayMalformedReplyException.class,
() -> session.addBufferedItem(3, "Tank01.Level", "galaxy"));
}
}
private static ProtocolStatus ok() {
return ProtocolStatus.newBuilder()
.setCode(ProtocolStatusCode.PROTOCOL_STATUS_CODE_OK)
.build();
}
private static MxCommandReply loadReply(String fixture) throws Exception {
MxCommandReply.Builder builder = MxCommandReply.newBuilder();
JsonFormat.parser().merge(
Files.readString(fixtureRoot().resolve("command-replies/" + fixture)),
builder);
return builder.build();
}
private static Path fixtureRoot() {
Path current = Path.of(System.getProperty("user.dir")).toAbsolutePath();
for (Path path = current; path != null; path = path.getParent()) {
Path candidate = path.resolve("clients/proto/fixtures/behavior");
if (Files.exists(candidate)) {
return candidate;
}
candidate = path.resolve("../proto/fixtures/behavior").normalize();
if (Files.exists(candidate)) {
return candidate;
}
}
throw new IllegalStateException("could not locate behavior fixtures from " + current);
}
private record InProcessGateway(Server server, ManagedChannel channel) implements AutoCloseable {
static InProcessGateway startReturning(MxCommandReply reply) throws Exception {
String serverName = "mxgw-java-cred-" + UUID.randomUUID();
MxAccessGatewayGrpc.MxAccessGatewayImplBase service =
new MxAccessGatewayGrpc.MxAccessGatewayImplBase() {
@Override
public void invoke(
MxCommandRequest request, StreamObserver<MxCommandReply> responseObserver) {
responseObserver.onNext(reply);
responseObserver.onCompleted();
}
};
Server server = InProcessServerBuilder.forName(serverName)
.directExecutor()
.addService(service)
.build()
.start();
ManagedChannel channel = InProcessChannelBuilder.forName(serverName)
.directExecutor()
.build();
return new InProcessGateway(server, channel);
}
MxGatewayClient client() {
return new MxGatewayClient(
channel,
MxGatewayClientOptions.builder()
.endpoint("in-process")
.apiKey("")
.plaintext(true)
.callTimeout(Duration.ofSeconds(5))
.build());
}
@Override
public void close() {
channel.shutdownNow();
server.shutdownNow();
}
}
}
@@ -0,0 +1,50 @@
package com.zb.mom.ww.mxgateway.client;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNull;
import org.junit.jupiter.api.Test;
final class MxGatewaySecretsTests {
@Test
void redactExactReplacesEveryOccurrenceOfASecret() {
String message = "verify s3cr3t, retry s3cr3t, done s3cr3t";
String result = MxGatewaySecrets.redactExact(message, "s3cr3t");
assertFalse(result.contains("s3cr3t"), "no occurrence of the secret may survive");
assertEquals("verify <redacted>, retry <redacted>, done <redacted>", result);
}
@Test
void redactExactFullyRedactsOverlappingSecretsWhenOneIsASubstringOfTheOther() {
String message = "password=hunter2 token=hunter2extra";
String result = MxGatewaySecrets.redactExact(message, "hunter2extra", "hunter2");
assertFalse(result.contains("hunter2"), "both the secret and its superstring must be fully redacted");
assertEquals("password=<redacted> token=<redacted>", result);
}
@Test
void redactExactWithNoSecretsReturnsMessageUnchanged() {
String message = "nothing to scrub here";
assertEquals(message, MxGatewaySecrets.redactExact(message));
}
@Test
void redactExactToleratesNullMessage() {
assertNull(MxGatewaySecrets.redactExact(null, "secret"));
}
@Test
void redactExactIgnoresBlankSecretSoRealSpacesAreNotOverRedacted() {
String message = "keep these spaces intact";
String result = MxGatewaySecrets.redactExact(message, " ", "");
assertEquals(message, result);
}
}
@@ -0,0 +1,22 @@
{
"sessionId": "session-fixture",
"correlationId": "gateway-correlation-authenticate-echoed-mxaccess-failure",
"kind": "MX_COMMAND_KIND_AUTHENTICATE_USER",
"protocolStatus": {
"code": "PROTOCOL_STATUS_CODE_MXACCESS_FAILURE",
"message": "MXAccess AuthenticateUser rejected credential 'sup3rSecretVerify9f3a2b'."
},
"hresult": -2147024891,
"statuses": [
{
"success": 0,
"category": "MX_STATUS_CATEGORY_SECURITY_ERROR",
"detectedBy": "MX_STATUS_SOURCE_RESPONDING_NMX",
"detail": 5,
"rawCategory": 8,
"rawDetectedBy": 5,
"diagnosticText": "Authentication failed for password 'sup3rSecretVerify9f3a2b'."
}
],
"diagnosticMessage": "MXAccess echoed the credential 'sup3rSecretVerify9f3a2b' back in its failure diagnostic."
}
@@ -0,0 +1,22 @@
{
"sessionId": "session-fixture",
"correlationId": "gateway-correlation-authenticate-echoed",
"kind": "MX_COMMAND_KIND_AUTHENTICATE_USER",
"protocolStatus": {
"code": "PROTOCOL_STATUS_CODE_OK",
"message": "MXAccess AuthenticateUser rejected credential 'sup3rSecretVerify9f3a2b'."
},
"hresult": -2147024891,
"statuses": [
{
"success": 0,
"category": "MX_STATUS_CATEGORY_SECURITY_ERROR",
"detectedBy": "MX_STATUS_SOURCE_RESPONDING_NMX",
"detail": 5,
"rawCategory": 8,
"rawDetectedBy": 5,
"diagnosticText": "Authentication failed for password 'sup3rSecretVerify9f3a2b'."
}
],
"diagnosticMessage": "MXAccess echoed the credential 'sup3rSecretVerify9f3a2b' back in its failure diagnostic."
}
@@ -0,0 +1,10 @@
{
"sessionId": "session-fixture",
"correlationId": "gateway-correlation-authenticate-missing-payload",
"kind": "MX_COMMAND_KIND_AUTHENTICATE_USER",
"protocolStatus": {
"code": "PROTOCOL_STATUS_CODE_OK",
"message": "AuthenticateUser reached MXAccess."
},
"diagnosticMessage": "Malformed: the OK reply carried neither an AuthenticateUser payload nor a return_value."
}
@@ -0,0 +1,15 @@
{
"sessionId": "session-fixture",
"correlationId": "gateway-correlation-authenticate-return-value-only",
"kind": "MX_COMMAND_KIND_AUTHENTICATE_USER",
"protocolStatus": {
"code": "PROTOCOL_STATUS_CODE_OK",
"message": "AuthenticateUser reached MXAccess."
},
"returnValue": {
"dataType": "MX_DATA_TYPE_INTEGER",
"variantType": "VT_I4",
"int32Value": 7
},
"diagnosticMessage": "Legacy worker populated only return_value; the typed AuthenticateUser payload is absent."
}
@@ -48,6 +48,34 @@
"path": "command-replies/write.hresult-e-fail.reply.json",
"expectation": "A negative HRESULT fails the reply even when every status entry reports MX_STATUS_CATEGORY_OK."
},
{
"id": "command-reply.authenticate-user.echoed-credential",
"category": "command_replies",
"messageType": "mxaccess_gateway.v1.MxCommandReply",
"path": "command-replies/authenticate-user.echoed-credential.reply.json",
"expectation": "When a gateway/MXAccess diagnostic echoes the caller's credential back (OK envelope, negative HRESULT), the surfaced error redacts the exact secret from both the rendered message and the structured reply accessors."
},
{
"id": "command-reply.authenticate-user.echoed-credential-mxaccess-failure",
"category": "command_replies",
"messageType": "mxaccess_gateway.v1.MxCommandReply",
"path": "command-replies/authenticate-user.echoed-credential-mxaccess-failure.reply.json",
"expectation": "The same echoed-credential redaction holds when the reply is coded PROTOCOL_STATUS_CODE_MXACCESS_FAILURE, which every client routes to its MXAccess error type."
},
{
"id": "command-reply.authenticate-user.missing-payload",
"category": "command_replies",
"messageType": "mxaccess_gateway.v1.MxCommandReply",
"path": "command-replies/authenticate-user.missing-payload.reply.json",
"expectation": "An OK reply with neither the typed AuthenticateUser payload nor a return_value raises a typed malformed-reply error, never a proto3 default 0 and never an NRE."
},
{
"id": "command-reply.authenticate-user.return-value-only",
"category": "command_replies",
"messageType": "mxaccess_gateway.v1.MxCommandReply",
"path": "command-replies/authenticate-user.return-value-only.reply.json",
"expectation": "An OK reply missing the typed AuthenticateUser payload but carrying an int32 return_value falls back to the return_value (legacy-worker compatibility)."
},
{
"id": "event-stream.session-ordered",
"category": "event_streams",
@@ -3,6 +3,7 @@
"cases": [
{
"id": "ok.responding-lmx",
"wantSuccess": true,
"status": {
"success": 1,
"category": "MX_STATUS_CATEGORY_OK",
@@ -15,6 +16,7 @@
},
{
"id": "security-error.requesting-lmx",
"wantSuccess": false,
"status": {
"success": 0,
"category": "MX_STATUS_CATEGORY_SECURITY_ERROR",
@@ -27,6 +29,7 @@
},
{
"id": "raw-unknown-category",
"wantSuccess": false,
"status": {
"success": 0,
"category": "MX_STATUS_CATEGORY_UNKNOWN",
+1 -1
View File
@@ -6,7 +6,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "zb-mom-ww-mxaccess-gateway-client"
version = "0.1.2"
version = "0.2.0"
description = "Async Python client for MXAccess Gateway."
readme = "README.md"
requires-python = ">=3.12"
@@ -11,6 +11,7 @@ from .generated.galaxy_repository_pb2 import (
)
from .events import ReplayGap
from .errors import (
MalformedReplyError,
MxAccessError,
MxGatewayAuthenticationError,
MxGatewayAuthorizationError,
@@ -35,6 +36,7 @@ __all__ = [
"GalaxyRepositoryClient",
"GatewayClient",
"LazyBrowseNode",
"MalformedReplyError",
"MxAccessError",
"MxGatewayAuthenticationError",
"MxGatewayAuthorizationError",
@@ -53,6 +53,10 @@ class MxAccessError(MxGatewayCommandError):
"""MXAccess HRESULT or status failure."""
class MalformedReplyError(MxGatewayError):
"""Raised when an OK reply lacks the expected typed payload and any usable return_value fallback."""
def map_rpc_error(operation: str, error: grpc.RpcError) -> MxGatewayTransportError:
"""Map a generated gRPC exception to the client exception hierarchy."""
@@ -153,8 +157,18 @@ def ensure_mxaccess_success(operation: str, reply: pb.MxCommandReply) -> pb.MxCo
def _mxaccess_message(operation: str, reply: pb.MxCommandReply) -> str:
status_text = reply.protocol_status.message or "MXAccess command failed"
hresult = reply.hresult if reply.HasField("hresult") else None
return (
message = (
f"{operation} failed: {status_text}; "
f"session={reply.session_id}; correlation={reply.correlation_id}; "
f"hresult={hresult}; statuses={len(reply.statuses)}"
)
# Append a per-status breakdown that carries the raw `success` COM member
# verbatim for diagnostic parity with the other clients. `category` remains
# the authoritative verdict; `success` is diagnostics only.
for status in reply.statuses:
category = pb.MxStatusCategory.Name(status.category)
message += (
f" [success={status.success}, category={category}, "
f"detail={status.detail}, {status.diagnostic_text}]"
)
return message
@@ -27,7 +27,7 @@ from google.protobuf import timestamp_pb2 as google_dot_protobuf_dot_timestamp__
import mxaccess_gateway_pb2 as mxaccess__gateway__pb2
DESCRIPTOR = _descriptor_pool.Default().AddSerializedFile(b'\n\x15mxaccess_worker.proto\x12\x12mxaccess_worker.v1\x1a\x1egoogle/protobuf/duration.proto\x1a\x1fgoogle/protobuf/timestamp.proto\x1a\x16mxaccess_gateway.proto\"\x95\x06\n\x0eWorkerEnvelope\x12\x18\n\x10protocol_version\x18\x01 \x01(\r\x12\x12\n\nsession_id\x18\x02 \x01(\t\x12\x10\n\x08sequence\x18\x03 \x01(\x04\x12\x16\n\x0e\x63orrelation_id\x18\x04 \x01(\t\x12\x39\n\rgateway_hello\x18\n \x01(\x0b\x32 .mxaccess_worker.v1.GatewayHelloH\x00\x12\x37\n\x0cworker_hello\x18\x0b \x01(\x0b\x32\x1f.mxaccess_worker.v1.WorkerHelloH\x00\x12\x37\n\x0cworker_ready\x18\x0c \x01(\x0b\x32\x1f.mxaccess_worker.v1.WorkerReadyH\x00\x12;\n\x0eworker_command\x18\r \x01(\x0b\x32!.mxaccess_worker.v1.WorkerCommandH\x00\x12\x46\n\x14worker_command_reply\x18\x0e \x01(\x0b\x32&.mxaccess_worker.v1.WorkerCommandReplyH\x00\x12\x39\n\rworker_cancel\x18\x0f \x01(\x0b\x32 .mxaccess_worker.v1.WorkerCancelH\x00\x12=\n\x0fworker_shutdown\x18\x10 \x01(\x0b\x32\".mxaccess_worker.v1.WorkerShutdownH\x00\x12\x44\n\x13worker_shutdown_ack\x18\x11 \x01(\x0b\x32%.mxaccess_worker.v1.WorkerShutdownAckH\x00\x12\x37\n\x0cworker_event\x18\x12 \x01(\x0b\x32\x1f.mxaccess_worker.v1.WorkerEventH\x00\x12?\n\x10worker_heartbeat\x18\x13 \x01(\x0b\x32#.mxaccess_worker.v1.WorkerHeartbeatH\x00\x12\x37\n\x0cworker_fault\x18\x14 \x01(\x0b\x32\x1f.mxaccess_worker.v1.WorkerFaultH\x00\x42\x06\n\x04\x62ody\"Z\n\x0cGatewayHello\x12\"\n\x1asupported_protocol_version\x18\x01 \x01(\r\x12\r\n\x05nonce\x18\x02 \x01(\t\x12\x17\n\x0fgateway_version\x18\x03 \x01(\t\"i\n\x0bWorkerHello\x12\x18\n\x10protocol_version\x18\x01 \x01(\r\x12\r\n\x05nonce\x18\x02 \x01(\t\x12\x19\n\x11worker_process_id\x18\x03 \x01(\x05\x12\x16\n\x0eworker_version\x18\x04 \x01(\t\"\x8e\x01\n\x0bWorkerReady\x12\x19\n\x11worker_process_id\x18\x01 \x01(\x05\x12\x17\n\x0fmxaccess_progid\x18\x02 \x01(\t\x12\x16\n\x0emxaccess_clsid\x18\x03 \x01(\t\x12\x33\n\x0fready_timestamp\x18\x04 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\"w\n\rWorkerCommand\x12/\n\x07\x63ommand\x18\x01 \x01(\x0b\x32\x1e.mxaccess_gateway.v1.MxCommand\x12\x35\n\x11\x65nqueue_timestamp\x18\x02 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\"\x81\x01\n\x12WorkerCommandReply\x12\x32\n\x05reply\x18\x01 \x01(\x0b\x32#.mxaccess_gateway.v1.MxCommandReply\x12\x37\n\x13\x63ompleted_timestamp\x18\x02 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\"\x1e\n\x0cWorkerCancel\x12\x0e\n\x06reason\x18\x01 \x01(\t\"Q\n\x0eWorkerShutdown\x12/\n\x0cgrace_period\x18\x01 \x01(\x0b\x32\x19.google.protobuf.Duration\x12\x0e\n\x06reason\x18\x02 \x01(\t\"H\n\x11WorkerShutdownAck\x12\x33\n\x06status\x18\x01 \x01(\x0b\x32#.mxaccess_gateway.v1.ProtocolStatus\":\n\x0bWorkerEvent\x12+\n\x05\x65vent\x18\x01 \x01(\x0b\x32\x1c.mxaccess_gateway.v1.MxEvent\"\xa5\x02\n\x0fWorkerHeartbeat\x12\x19\n\x11worker_process_id\x18\x01 \x01(\x05\x12.\n\x05state\x18\x02 \x01(\x0e\x32\x1f.mxaccess_worker.v1.WorkerState\x12?\n\x1blast_sta_activity_timestamp\x18\x03 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\x12\x1d\n\x15pending_command_count\x18\x04 \x01(\r\x12\"\n\x1aoutbound_event_queue_depth\x18\x05 \x01(\r\x12\x1b\n\x13last_event_sequence\x18\x06 \x01(\x04\x12&\n\x1e\x63urrent_command_correlation_id\x18\x07 \x01(\t\"\xf4\x01\n\x0bWorkerFault\x12\x39\n\x08\x63\x61tegory\x18\x01 \x01(\x0e\x32\'.mxaccess_worker.v1.WorkerFaultCategory\x12\x16\n\x0e\x63ommand_method\x18\x02 \x01(\t\x12\x14\n\x07hresult\x18\x03 \x01(\x05H\x00\x88\x01\x01\x12\x16\n\x0e\x65xception_type\x18\x04 \x01(\t\x12\x1a\n\x12\x64iagnostic_message\x18\x05 \x01(\t\x12<\n\x0fprotocol_status\x18\x06 \x01(\x0b\x32#.mxaccess_gateway.v1.ProtocolStatusB\n\n\x08_hresult*\x97\x02\n\x0bWorkerState\x12\x1c\n\x18WORKER_STATE_UNSPECIFIED\x10\x00\x12\x19\n\x15WORKER_STATE_STARTING\x10\x01\x12\x1c\n\x18WORKER_STATE_HANDSHAKING\x10\x02\x12!\n\x1dWORKER_STATE_INITIALIZING_STA\x10\x03\x12\x16\n\x12WORKER_STATE_READY\x10\x04\x12\"\n\x1eWORKER_STATE_EXECUTING_COMMAND\x10\x05\x12\x1e\n\x1aWORKER_STATE_SHUTTING_DOWN\x10\x06\x12\x18\n\x14WORKER_STATE_STOPPED\x10\x07\x12\x18\n\x14WORKER_STATE_FAULTED\x10\x08*\xc7\x04\n\x13WorkerFaultCategory\x12%\n!WORKER_FAULT_CATEGORY_UNSPECIFIED\x10\x00\x12+\n\'WORKER_FAULT_CATEGORY_INVALID_ARGUMENTS\x10\x01\x12\x37\n3WORKER_FAULT_CATEGORY_GATEWAY_AUTHENTICATION_FAILED\x10\x02\x12+\n\'WORKER_FAULT_CATEGORY_PROTOCOL_MISMATCH\x10\x03\x12,\n(WORKER_FAULT_CATEGORY_PROTOCOL_VIOLATION\x10\x04\x12+\n\'WORKER_FAULT_CATEGORY_PIPE_DISCONNECTED\x10\x05\x12\x32\n.WORKER_FAULT_CATEGORY_MXACCESS_CREATION_FAILED\x10\x06\x12\x31\n-WORKER_FAULT_CATEGORY_MXACCESS_COMMAND_FAILED\x10\x07\x12:\n6WORKER_FAULT_CATEGORY_MXACCESS_EVENT_CONVERSION_FAILED\x10\x08\x12\"\n\x1eWORKER_FAULT_CATEGORY_STA_HUNG\x10\t\x12(\n$WORKER_FAULT_CATEGORY_QUEUE_OVERFLOW\x10\n\x12*\n&WORKER_FAULT_CATEGORY_SHUTDOWN_TIMEOUT\x10\x0b\x42&\xaa\x02#ZB.MOM.WW.MxGateway.Contracts.Protob\x06proto3')
DESCRIPTOR = _descriptor_pool.Default().AddSerializedFile(b'\n\x15mxaccess_worker.proto\x12\x12mxaccess_worker.v1\x1a\x1egoogle/protobuf/duration.proto\x1a\x1fgoogle/protobuf/timestamp.proto\x1a\x16mxaccess_gateway.proto\"\x95\x06\n\x0eWorkerEnvelope\x12\x18\n\x10protocol_version\x18\x01 \x01(\r\x12\x12\n\nsession_id\x18\x02 \x01(\t\x12\x10\n\x08sequence\x18\x03 \x01(\x04\x12\x16\n\x0e\x63orrelation_id\x18\x04 \x01(\t\x12\x39\n\rgateway_hello\x18\n \x01(\x0b\x32 .mxaccess_worker.v1.GatewayHelloH\x00\x12\x37\n\x0cworker_hello\x18\x0b \x01(\x0b\x32\x1f.mxaccess_worker.v1.WorkerHelloH\x00\x12\x37\n\x0cworker_ready\x18\x0c \x01(\x0b\x32\x1f.mxaccess_worker.v1.WorkerReadyH\x00\x12;\n\x0eworker_command\x18\r \x01(\x0b\x32!.mxaccess_worker.v1.WorkerCommandH\x00\x12\x46\n\x14worker_command_reply\x18\x0e \x01(\x0b\x32&.mxaccess_worker.v1.WorkerCommandReplyH\x00\x12\x39\n\rworker_cancel\x18\x0f \x01(\x0b\x32 .mxaccess_worker.v1.WorkerCancelH\x00\x12=\n\x0fworker_shutdown\x18\x10 \x01(\x0b\x32\".mxaccess_worker.v1.WorkerShutdownH\x00\x12\x44\n\x13worker_shutdown_ack\x18\x11 \x01(\x0b\x32%.mxaccess_worker.v1.WorkerShutdownAckH\x00\x12\x37\n\x0cworker_event\x18\x12 \x01(\x0b\x32\x1f.mxaccess_worker.v1.WorkerEventH\x00\x12?\n\x10worker_heartbeat\x18\x13 \x01(\x0b\x32#.mxaccess_worker.v1.WorkerHeartbeatH\x00\x12\x37\n\x0cworker_fault\x18\x14 \x01(\x0b\x32\x1f.mxaccess_worker.v1.WorkerFaultH\x00\x42\x06\n\x04\x62ody\"s\n\x0cGatewayHello\x12\"\n\x1asupported_protocol_version\x18\x01 \x01(\r\x12\r\n\x05nonce\x18\x02 \x01(\t\x12\x17\n\x0fgateway_version\x18\x03 \x01(\t\x12\x17\n\x0fmax_frame_bytes\x18\x04 \x01(\r\"i\n\x0bWorkerHello\x12\x18\n\x10protocol_version\x18\x01 \x01(\r\x12\r\n\x05nonce\x18\x02 \x01(\t\x12\x19\n\x11worker_process_id\x18\x03 \x01(\x05\x12\x16\n\x0eworker_version\x18\x04 \x01(\t\"\x8e\x01\n\x0bWorkerReady\x12\x19\n\x11worker_process_id\x18\x01 \x01(\x05\x12\x17\n\x0fmxaccess_progid\x18\x02 \x01(\t\x12\x16\n\x0emxaccess_clsid\x18\x03 \x01(\t\x12\x33\n\x0fready_timestamp\x18\x04 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\"w\n\rWorkerCommand\x12/\n\x07\x63ommand\x18\x01 \x01(\x0b\x32\x1e.mxaccess_gateway.v1.MxCommand\x12\x35\n\x11\x65nqueue_timestamp\x18\x02 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\"\x81\x01\n\x12WorkerCommandReply\x12\x32\n\x05reply\x18\x01 \x01(\x0b\x32#.mxaccess_gateway.v1.MxCommandReply\x12\x37\n\x13\x63ompleted_timestamp\x18\x02 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\"\x1e\n\x0cWorkerCancel\x12\x0e\n\x06reason\x18\x01 \x01(\t\"Q\n\x0eWorkerShutdown\x12/\n\x0cgrace_period\x18\x01 \x01(\x0b\x32\x19.google.protobuf.Duration\x12\x0e\n\x06reason\x18\x02 \x01(\t\"H\n\x11WorkerShutdownAck\x12\x33\n\x06status\x18\x01 \x01(\x0b\x32#.mxaccess_gateway.v1.ProtocolStatus\":\n\x0bWorkerEvent\x12+\n\x05\x65vent\x18\x01 \x01(\x0b\x32\x1c.mxaccess_gateway.v1.MxEvent\"\xa5\x02\n\x0fWorkerHeartbeat\x12\x19\n\x11worker_process_id\x18\x01 \x01(\x05\x12.\n\x05state\x18\x02 \x01(\x0e\x32\x1f.mxaccess_worker.v1.WorkerState\x12?\n\x1blast_sta_activity_timestamp\x18\x03 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\x12\x1d\n\x15pending_command_count\x18\x04 \x01(\r\x12\"\n\x1aoutbound_event_queue_depth\x18\x05 \x01(\r\x12\x1b\n\x13last_event_sequence\x18\x06 \x01(\x04\x12&\n\x1e\x63urrent_command_correlation_id\x18\x07 \x01(\t\"\xf4\x01\n\x0bWorkerFault\x12\x39\n\x08\x63\x61tegory\x18\x01 \x01(\x0e\x32\'.mxaccess_worker.v1.WorkerFaultCategory\x12\x16\n\x0e\x63ommand_method\x18\x02 \x01(\t\x12\x14\n\x07hresult\x18\x03 \x01(\x05H\x00\x88\x01\x01\x12\x16\n\x0e\x65xception_type\x18\x04 \x01(\t\x12\x1a\n\x12\x64iagnostic_message\x18\x05 \x01(\t\x12<\n\x0fprotocol_status\x18\x06 \x01(\x0b\x32#.mxaccess_gateway.v1.ProtocolStatusB\n\n\x08_hresult*\x97\x02\n\x0bWorkerState\x12\x1c\n\x18WORKER_STATE_UNSPECIFIED\x10\x00\x12\x19\n\x15WORKER_STATE_STARTING\x10\x01\x12\x1c\n\x18WORKER_STATE_HANDSHAKING\x10\x02\x12!\n\x1dWORKER_STATE_INITIALIZING_STA\x10\x03\x12\x16\n\x12WORKER_STATE_READY\x10\x04\x12\"\n\x1eWORKER_STATE_EXECUTING_COMMAND\x10\x05\x12\x1e\n\x1aWORKER_STATE_SHUTTING_DOWN\x10\x06\x12\x18\n\x14WORKER_STATE_STOPPED\x10\x07\x12\x18\n\x14WORKER_STATE_FAULTED\x10\x08*\xc7\x04\n\x13WorkerFaultCategory\x12%\n!WORKER_FAULT_CATEGORY_UNSPECIFIED\x10\x00\x12+\n\'WORKER_FAULT_CATEGORY_INVALID_ARGUMENTS\x10\x01\x12\x37\n3WORKER_FAULT_CATEGORY_GATEWAY_AUTHENTICATION_FAILED\x10\x02\x12+\n\'WORKER_FAULT_CATEGORY_PROTOCOL_MISMATCH\x10\x03\x12,\n(WORKER_FAULT_CATEGORY_PROTOCOL_VIOLATION\x10\x04\x12+\n\'WORKER_FAULT_CATEGORY_PIPE_DISCONNECTED\x10\x05\x12\x32\n.WORKER_FAULT_CATEGORY_MXACCESS_CREATION_FAILED\x10\x06\x12\x31\n-WORKER_FAULT_CATEGORY_MXACCESS_COMMAND_FAILED\x10\x07\x12:\n6WORKER_FAULT_CATEGORY_MXACCESS_EVENT_CONVERSION_FAILED\x10\x08\x12\"\n\x1eWORKER_FAULT_CATEGORY_STA_HUNG\x10\t\x12(\n$WORKER_FAULT_CATEGORY_QUEUE_OVERFLOW\x10\n\x12*\n&WORKER_FAULT_CATEGORY_SHUTDOWN_TIMEOUT\x10\x0b\x42&\xaa\x02#ZB.MOM.WW.MxGateway.Contracts.Protob\x06proto3')
_globals = globals()
_builder.BuildMessageAndEnumDescriptors(DESCRIPTOR, _globals)
@@ -35,32 +35,32 @@ _builder.BuildTopDescriptorsAndMessages(DESCRIPTOR, 'mxaccess_worker_pb2', _glob
if not _descriptor._USE_C_DESCRIPTORS:
_globals['DESCRIPTOR']._loaded_options = None
_globals['DESCRIPTOR']._serialized_options = b'\252\002#ZB.MOM.WW.MxGateway.Contracts.Proto'
_globals['_WORKERSTATE']._serialized_start=2316
_globals['_WORKERSTATE']._serialized_end=2595
_globals['_WORKERFAULTCATEGORY']._serialized_start=2598
_globals['_WORKERFAULTCATEGORY']._serialized_end=3181
_globals['_WORKERSTATE']._serialized_start=2341
_globals['_WORKERSTATE']._serialized_end=2620
_globals['_WORKERFAULTCATEGORY']._serialized_start=2623
_globals['_WORKERFAULTCATEGORY']._serialized_end=3206
_globals['_WORKERENVELOPE']._serialized_start=135
_globals['_WORKERENVELOPE']._serialized_end=924
_globals['_GATEWAYHELLO']._serialized_start=926
_globals['_GATEWAYHELLO']._serialized_end=1016
_globals['_WORKERHELLO']._serialized_start=1018
_globals['_WORKERHELLO']._serialized_end=1123
_globals['_WORKERREADY']._serialized_start=1126
_globals['_WORKERREADY']._serialized_end=1268
_globals['_WORKERCOMMAND']._serialized_start=1270
_globals['_WORKERCOMMAND']._serialized_end=1389
_globals['_WORKERCOMMANDREPLY']._serialized_start=1392
_globals['_WORKERCOMMANDREPLY']._serialized_end=1521
_globals['_WORKERCANCEL']._serialized_start=1523
_globals['_WORKERCANCEL']._serialized_end=1553
_globals['_WORKERSHUTDOWN']._serialized_start=1555
_globals['_WORKERSHUTDOWN']._serialized_end=1636
_globals['_WORKERSHUTDOWNACK']._serialized_start=1638
_globals['_WORKERSHUTDOWNACK']._serialized_end=1710
_globals['_WORKEREVENT']._serialized_start=1712
_globals['_WORKEREVENT']._serialized_end=1770
_globals['_WORKERHEARTBEAT']._serialized_start=1773
_globals['_WORKERHEARTBEAT']._serialized_end=2066
_globals['_WORKERFAULT']._serialized_start=2069
_globals['_WORKERFAULT']._serialized_end=2313
_globals['_GATEWAYHELLO']._serialized_end=1041
_globals['_WORKERHELLO']._serialized_start=1043
_globals['_WORKERHELLO']._serialized_end=1148
_globals['_WORKERREADY']._serialized_start=1151
_globals['_WORKERREADY']._serialized_end=1293
_globals['_WORKERCOMMAND']._serialized_start=1295
_globals['_WORKERCOMMAND']._serialized_end=1414
_globals['_WORKERCOMMANDREPLY']._serialized_start=1417
_globals['_WORKERCOMMANDREPLY']._serialized_end=1546
_globals['_WORKERCANCEL']._serialized_start=1548
_globals['_WORKERCANCEL']._serialized_end=1578
_globals['_WORKERSHUTDOWN']._serialized_start=1580
_globals['_WORKERSHUTDOWN']._serialized_end=1661
_globals['_WORKERSHUTDOWNACK']._serialized_start=1663
_globals['_WORKERSHUTDOWNACK']._serialized_end=1735
_globals['_WORKEREVENT']._serialized_start=1737
_globals['_WORKEREVENT']._serialized_end=1795
_globals['_WORKERHEARTBEAT']._serialized_start=1798
_globals['_WORKERHEARTBEAT']._serialized_end=2091
_globals['_WORKERFAULT']._serialized_start=2094
_globals['_WORKERFAULT']._serialized_end=2338
# @@protoc_insertion_point(module_scope)
@@ -5,7 +5,7 @@ from __future__ import annotations
from collections.abc import AsyncIterator, Sequence
from .auth import redact_secret
from .errors import MxGatewayError, ensure_mxaccess_success
from .errors import MalformedReplyError, MxGatewayError, ensure_mxaccess_success
from .events import ReplayGap
from .generated import mxaccess_gateway_pb2 as pb
from .values import MxValueInput, to_mx_value
@@ -710,7 +710,15 @@ class Session:
correlation_id=correlation_id,
secrets=[verify_user_password],
)
return reply.authenticate_user.user_id
if reply.HasField("authenticate_user"):
return reply.authenticate_user.user_id
if reply.HasField("return_value") and reply.return_value.WhichOneof("kind") == "int32_value":
return reply.return_value.int32_value
raise MalformedReplyError(
"authenticate_user returned a malformed reply: OK reply carried "
"neither the typed payload nor an int32 return_value",
raw_reply=reply,
)
async def archestra_user_to_id(
self,
@@ -730,7 +738,15 @@ class Session:
),
correlation_id=correlation_id,
)
return reply.archestra_user_to_id.user_id
if reply.HasField("archestra_user_to_id"):
return reply.archestra_user_to_id.user_id
if reply.HasField("return_value") and reply.return_value.WhichOneof("kind") == "int32_value":
return reply.return_value.int32_value
raise MalformedReplyError(
"archestra_user_to_id returned a malformed reply: OK reply carried "
"neither the typed payload nor an int32 return_value",
raw_reply=reply,
)
async def add_buffered_item(
self,
@@ -752,7 +768,15 @@ class Session:
),
correlation_id=correlation_id,
)
return reply.add_buffered_item.item_handle
if reply.HasField("add_buffered_item"):
return reply.add_buffered_item.item_handle
if reply.HasField("return_value") and reply.return_value.WhichOneof("kind") == "int32_value":
return reply.return_value.int32_value
raise MalformedReplyError(
"add_buffered_item returned a malformed reply: OK reply carried "
"neither the typed payload nor an int32 return_value",
raw_reply=reply,
)
async def set_buffered_update_interval(
self,
@@ -895,19 +919,47 @@ def _value_secrets(value: MxValueInput) -> list[str]:
def _redact_error(error: MxGatewayError, secrets: Sequence[str | None]) -> None:
"""Scrub secret substrings from a raised error's message in place.
"""Scrub secret substrings from a raised error's message and reply in place.
Rewrites ``error.args[0]`` (the message returned by ``str(error)``) through
the shared :func:`~zb_mom_ww_mxgateway.auth.redact_secret` seam so credential
text can never reach logs or be re-raised to a caller. The
``protocol_status`` / ``raw_reply`` context is left untouched — those hold the
gateway's own fields, which never echo the client-supplied secret.
text can never reach logs or be re-raised to a caller.
A misbehaving MXAccess provider can echo the client-supplied credential back
verbatim in its failure diagnostics, so ``error.raw_reply`` (the protobuf
reply) can carry the secret in ``protocol_status.message``,
``diagnostic_message``, and each ``statuses[].diagnostic_text``. A logger
dumping those structured fields would reintroduce the leak the message scrub
closes. When there is a secret to scrub and a reply is attached, this rebinds
``error.raw_reply`` to a scrubbed deep copy so the raised exception carries no
credential text on any surface. The clone leaves the original reply untouched.
"""
scrubbed = [secret for secret in secrets if secret]
if not scrubbed:
return
if error.args and isinstance(error.args[0], str):
error.args = (redact_secret(error.args[0], scrubbed), *error.args[1:])
if error.raw_reply is not None:
error.raw_reply = _redact_reply(error.raw_reply, scrubbed)
def _redact_reply(reply: pb.MxCommandReply, secrets: Sequence[str]) -> pb.MxCommandReply:
"""Return a deep copy of *reply* with credential text scrubbed from diagnostics.
Operates on a clone so the caller's original reply object is never mutated.
Only the free-text diagnostic fields that can echo a client-supplied secret
are scrubbed; the structured/enum fields the gateway itself sets are left as-is.
"""
clone = type(reply)()
clone.CopyFrom(reply)
if clone.protocol_status.message:
clone.protocol_status.message = redact_secret(clone.protocol_status.message, secrets)
if clone.diagnostic_message:
clone.diagnostic_message = redact_secret(clone.diagnostic_message, secrets)
for status in clone.statuses:
if status.diagnostic_text:
status.diagnostic_text = redact_secret(status.diagnostic_text, secrets)
return clone
from .client import GatewayClient # noqa: E402
@@ -1,3 +1,3 @@
"""Package version information."""
__version__ = "0.1.2"
__version__ = "0.2.0"
+17
View File
@@ -1,6 +1,8 @@
"""Tests for the Python CLI."""
import json
import tomllib
from pathlib import Path
import pytest
from click.testing import CliRunner
@@ -12,6 +14,21 @@ from zb_mom_ww_mxgateway_cli.commands import main
_BATCH_EOR = "__MXGW_BATCH_EOR__"
def test_version_matches_pyproject_toml() -> None:
"""`__version__` must track `pyproject.toml`'s `[project].version`.
The existing `version` command tests only assert self-consistency against
`__version__` (the two hardcoded literals could still drift from each
other without either test catching it — the CLI-26 residual drift mode).
This test pins `__version__` to the single source of truth instead.
"""
pyproject_path = Path(__file__).resolve().parent.parent / "pyproject.toml"
with pyproject_path.open("rb") as handle:
pyproject = tomllib.load(handle)
assert __version__ == pyproject["project"]["version"]
def test_require_certificate_validation_flag_flows_through_connect(
monkeypatch: pytest.MonkeyPatch,
) -> None:
@@ -0,0 +1,112 @@
"""Tests for the uniform malformed-reply contract (CLI-41) and the CLI-40
credential-redaction regression, driven through the shared fixtures.
CLI-41: an OK reply that carries neither the expected typed payload nor a usable
``return_value`` int32 fallback raises :class:`MalformedReplyError`; a legacy
reply that populates only ``return_value`` falls back to that int32.
CLI-40: an OK reply whose diagnostics echo the caller's credential must never
surface that credential in the raised error message.
"""
from __future__ import annotations
import json
from pathlib import Path
import pytest
from google.protobuf.json_format import ParseDict
from zb_mom_ww_mxgateway import MalformedReplyError, MxAccessError
from zb_mom_ww_mxgateway.generated import mxaccess_gateway_pb2 as pb
from test_typed_command_helpers import _session_with
FIXTURE_ROOT = Path(__file__).resolve().parents[2] / "proto" / "fixtures" / "behavior"
def _load_reply(relative: str) -> pb.MxCommandReply:
path = FIXTURE_ROOT / relative
return ParseDict(json.loads(path.read_text()), pb.MxCommandReply())
@pytest.mark.asyncio
async def test_authenticate_user_missing_payload_raises_malformed_reply() -> None:
reply = _load_reply("command-replies/authenticate-user.missing-payload.reply.json")
session, _ = await _session_with([reply])
with pytest.raises(MalformedReplyError) as captured:
await session.authenticate_user(12, "operator", "any-password")
assert captured.value.raw_reply is reply
assert "malformed reply" in str(captured.value)
@pytest.mark.asyncio
async def test_authenticate_user_return_value_only_falls_back_to_int32() -> None:
reply = _load_reply("command-replies/authenticate-user.return-value-only.reply.json")
session, _ = await _session_with([reply])
user_id = await session.authenticate_user(12, "operator", "any-password")
assert user_id == 7
@pytest.mark.asyncio
async def test_add_buffered_item_falls_back_to_return_value_int32() -> None:
reply = pb.MxCommandReply(
session_id="session-1",
kind=pb.MX_COMMAND_KIND_ADD_BUFFERED_ITEM,
protocol_status=pb.ProtocolStatus(code=pb.PROTOCOL_STATUS_CODE_OK),
return_value=pb.MxValue(int32_value=99),
)
session, _ = await _session_with([reply])
item_handle = await session.add_buffered_item(12, "Object.Attribute", "ctx")
assert item_handle == 99
@pytest.mark.asyncio
async def test_add_buffered_item_missing_payload_raises_malformed_reply() -> None:
reply = pb.MxCommandReply(
session_id="session-1",
kind=pb.MX_COMMAND_KIND_ADD_BUFFERED_ITEM,
protocol_status=pb.ProtocolStatus(code=pb.PROTOCOL_STATUS_CODE_OK),
)
session, _ = await _session_with([reply])
with pytest.raises(MalformedReplyError) as captured:
await session.add_buffered_item(12, "Object.Attribute", "ctx")
assert captured.value.raw_reply is reply
@pytest.mark.parametrize(
"fixture",
[
"command-replies/authenticate-user.echoed-credential.reply.json",
"command-replies/authenticate-user.echoed-credential-mxaccess-failure.reply.json",
],
)
@pytest.mark.asyncio
async def test_authenticate_user_echoed_credential_is_scrubbed(fixture: str) -> None:
credential = "sup3rSecretVerify9f3a2b"
reply = _load_reply(fixture)
session, _ = await _session_with([reply])
with pytest.raises(MxAccessError) as captured:
await session.authenticate_user(12, "operator", credential)
exc = captured.value
message = str(exc)
assert credential not in message
assert "[redacted]" in message
# The credential must not survive in the structured protobuf context either:
# a logger dumping raw_reply's fields would otherwise reintroduce the leak.
assert exc.raw_reply is not None
assert credential not in exc.raw_reply.protocol_status.message
assert credential not in exc.raw_reply.diagnostic_message
for status in exc.raw_reply.statuses:
assert credential not in status.diagnostic_text
@@ -140,10 +140,19 @@ async def test_write_secured_surfaces_native_failure_without_prior_authenticate(
with pytest.raises(MxAccessError) as captured:
await session.write_secured(12, 34, secret_value, current_user_id=5, verifier_user_id=6)
# Native failure is surfaced (not "fixed") and the raw reply is preserved...
assert captured.value.raw_reply is failure
# ...but the credential-sensitive value is scrubbed from the surfaced message.
# Native failure is surfaced (not "fixed"): the raw reply's structure is
# preserved so callers still see the native verdict...
raw = captured.value.raw_reply
assert raw is not None
assert raw.kind == pb.MX_COMMAND_KIND_WRITE_SECURED
assert raw.hresult == -2147217407
assert raw.protocol_status.code == pb.PROTOCOL_STATUS_CODE_MXACCESS_FAILURE
# ...but the credential-sensitive value is scrubbed from the surfaced message
# AND from the reply's echoed diagnostics, so a logger dumping raw_reply's
# structured fields cannot reintroduce the leak.
assert secret_value not in str(captured.value)
assert secret_value not in raw.protocol_status.message
assert "[redacted]" in raw.protocol_status.message
command = stub.invoke.requests[0].command
assert command.kind == pb.MX_COMMAND_KIND_WRITE_SECURED
assert command.write_secured.current_user_id == 5
+2 -2
View File
@@ -590,7 +590,7 @@ checksum = "1d87ecb2933e8aeadb3e3a02b828fed80a7528047e68b4f424523a0981a3a084"
[[package]]
name = "mxgw-cli"
version = "0.1.2"
version = "0.2.0"
dependencies = [
"clap",
"futures-util",
@@ -1490,7 +1490,7 @@ dependencies = [
[[package]]
name = "zb-mom-ww-mxgateway-client"
version = "0.1.2"
version = "0.2.0"
dependencies = [
"futures-core",
"futures-util",
+2 -2
View File
@@ -1,6 +1,6 @@
[package]
name = "zb-mom-ww-mxgateway-client"
version = "0.1.2"
version = "0.2.0"
edition = "2021"
authors = ["Joseph Doherty"]
description = "Async Rust client for the MxAccessGateway gRPC service, including a lazy-browse walker over the Galaxy Repository hierarchy."
@@ -25,7 +25,7 @@ resolver = "2"
[workspace.package]
edition = "2021"
version = "0.1.2"
version = "0.2.0"
authors = ["Joseph Doherty"]
license = "Proprietary"
repository = "https://gitea.dohertylan.com/dohertj2/mxaccessgw"
+17 -4
View File
@@ -18,9 +18,22 @@ clients/rust/
crates/mxgw-cli/
```
`build.rs` reads the `.proto` files from
`../../src/ZB.MOM.WW.MxGateway.Contracts/Protos` and generates `tonic`/`prost` bindings
into Cargo build output. `src/generated.rs` declares the Rust modules that
`build.rs` resolves the `.proto` inputs repo-path-first: it prefers the
canonical protos at `../../src/ZB.MOM.WW.MxGateway.Contracts/Protos` (two
levels above `clients/rust`) so a local in-repo `.proto` edit is picked up
live without any extra step, and falls back to the vendored copies checked
into `clients/rust/protos/` only when that canonical directory is absent —
the case for a consumer building the crate unpacked from a published
tarball, where the rest of the mxaccessgw repo does not exist. The vendored
copies are shipped in the published `.crate` via `Cargo.toml`'s `include`
list, which is what makes the crate buildable standalone; they are build
inputs only, never a second source of truth. **Refresh rule:** any commit
that edits a Contracts proto (`mxaccess_gateway.proto`, `mxaccess_worker.proto`,
`galaxy_repository.proto`) must copy the changed file(s) into
`clients/rust/protos/` in that same commit — `scripts/check-codegen.ps1`
Check 3 fails the build on byte drift between the vendored copies and the
canonical Contracts protos. `tonic`/`prost` bindings are generated into
Cargo build output. `src/generated.rs` declares the Rust modules that
include those generated files. `src/generated` remains reserved for checked-in
generator output if the crate later changes to source-tree generation.
@@ -423,5 +436,5 @@ Then add the dependency:
```toml
[dependencies]
zb-mom-ww-mxgateway-client = { version = "0.1.1", registry = "dohertj2-gitea" }
zb-mom-ww-mxgateway-client = { version = "0.2.0", registry = "dohertj2-gitea" }
```
@@ -676,6 +676,10 @@ message WorkerInfoReply {
}
message DrainEventsReply {
// The reply is bounded by both a server-side count cap and the negotiated
// worker-frame byte cap; a reply may therefore carry fewer events than
// `max_events` and fewer than are queued. Callers drain iteratively until an
// empty reply.
repeated MxEvent events = 1;
}
@@ -760,6 +764,11 @@ message ReplayGap {
// after_worker_sequence = oldest_available_sequence - 1 in the next
// StreamEventsRequest, which will cause the server to replay starting at
// oldest_available_sequence (the first retained event).
// When nothing is retained (the replay ring is empty), this is the next sequence
// that can be delivered — `highest observed + 1` — and the `oldest - 1` resume
// formula remains valid: it resolves to the highest sequence already seen, so the
// follow-up resume replays nothing, reports no gap, and every newer live event
// passes. The interval evicted is unchanged.
uint64 oldest_available_sequence = 2;
}
@@ -47,6 +47,9 @@ message GatewayHello {
// instead of a hard-coded default; 0 (an older gateway that never set the field) means
// "use the worker's built-in default". Sits above the public gRPC cap by an
// envelope-overhead margin so an accepted gRPC payload always fits one worker frame.
// Every worker->gateway frame — events, heartbeats, faults, and control replies
// including DrainEvents — must serialize within this limit; reply builders truncate
// to fit rather than emit an oversized frame.
uint32 max_frame_bytes = 4;
}
+95 -16
View File
@@ -193,17 +193,43 @@ impl std::error::Error for CommandError {}
/// The wrapper is heap-allocated inside [`Error::MxAccess`] to keep the
/// containing enum small. Callers can recover the reply with
/// [`MxAccessError::reply`] or [`MxAccessError::into_reply`]. Its `Display`
/// summarizes the `hresult` and status entries and scrubs any credential-like
/// tokens from diagnostic text before it reaches a caller.
#[derive(Clone, Debug)]
/// summarizes the `hresult` and status entries and scrubs credentials from the
/// rendered text before it reaches a caller: credential-*shaped* tokens
/// (`mxgw_...`, `bearer`) via a pattern scrub, plus any exact caller-supplied
/// secrets registered with [`MxAccessError::with_secrets`] — the latter catches
/// a password MXAccess echoed back verbatim even though it has no token shape.
///
/// `Debug` is hand-written (not derived) so the attached exact secrets never
/// reach `{:?}` output either: it scrubs them from the reply rendering and
/// prints only the count of attached secrets, never their values.
#[derive(Clone)]
pub struct MxAccessError {
reply: MxCommandReply,
/// Exact caller-supplied secrets (e.g. an `AuthenticateUser` password or a
/// `WriteSecured` string value) scrubbed from the rendered message. Empty
/// unless a helper attaches them via [`Self::with_secrets`].
secrets: Vec<String>,
}
impl MxAccessError {
/// Wrap a reply whose MXAccess-level result reported a failure.
pub fn new(reply: MxCommandReply) -> Self {
Self { reply }
Self {
reply,
secrets: Vec::new(),
}
}
/// Register exact caller-supplied secrets to scrub from the rendered
/// message, returning the updated error.
///
/// A credential MXAccess echoes back into its diagnostic text has no
/// `mxgw_`/`bearer` shape, so the pattern scrub cannot catch it. Attaching
/// the exact secret lets `Display` replace every occurrence with
/// `<redacted>`.
pub fn with_secrets(mut self, secrets: Vec<String>) -> Self {
self.secrets = secrets;
self
}
/// Borrow the underlying reply (correlation id, hresult, statuses).
@@ -217,15 +243,43 @@ impl MxAccessError {
}
}
impl std::fmt::Debug for MxAccessError {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
// Render the reply, scrub any exact caller secret from it, and never
// print the raw secrets themselves — only how many are attached.
let mut reply = format!("{:?}", self.reply);
for secret in &self.secrets {
if !secret.is_empty() {
reply = reply.replace(secret.as_str(), "<redacted>");
}
}
formatter
.debug_struct("MxAccessError")
.field("reply", &format_args!("{reply}"))
.field(
"secrets",
&format_args!("[{} redacted]", self.secrets.len()),
)
.finish()
}
}
impl std::fmt::Display for MxAccessError {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
use std::fmt::Write as _;
let hresult = match self.reply.hresult {
Some(value) => value.to_string(),
None => "none".to_owned(),
};
// Render the whole body first so the exact-secret scrub can sweep every
// field — including diagnostic text that already went through the
// credential-shape scrub — before any of it reaches the caller.
let mut body = String::new();
write!(
formatter,
body,
"hresult={hresult}, {} status entr{}",
self.reply.statuses.len(),
if self.reply.statuses.len() == 1 {
@@ -233,20 +287,28 @@ impl std::fmt::Display for MxAccessError {
} else {
"ies"
}
)?;
)
.expect("writing to a String is infallible");
for status in &self.reply.statuses {
let category = MxStatusCategory::try_from(status.category)
.unwrap_or(MxStatusCategory::Unspecified);
let diagnostic = redact_credentials(&status.diagnostic_text);
write!(
formatter,
body,
"; [success={}, category={category:?}, detail={}, {}]",
status.success, status.detail, diagnostic
)?;
)
.expect("writing to a String is infallible");
}
Ok(())
for secret in &self.secrets {
if !secret.is_empty() {
body = body.replace(secret.as_str(), "<redacted>");
}
}
formatter.write_str(&body)
}
}
@@ -284,10 +346,17 @@ impl From<tonic::Status> for Error {
/// Promote a non-OK protocol status carried inside an [`MxCommandReply`]
/// to an [`Error::Command`].
///
/// [`ProtocolStatusCode::MxaccessFailure`] is deliberately **not** a
/// command-level failure here: it signals an MXAccess-level rejection, so it
/// falls through to [`ensure_mxaccess_success`] and surfaces as
/// [`Error::MxAccess`] — matching the .NET, Java, Go, and Python clients. Every
/// other non-`Ok` code stays [`Error::Command`].
///
/// # Errors
///
/// Returns [`Error::Command`] when `reply.protocol_status` is missing or
/// reports any code other than [`ProtocolStatusCode::Ok`].
/// reports any code other than [`ProtocolStatusCode::Ok`] or
/// [`ProtocolStatusCode::MxaccessFailure`].
pub fn ensure_command_success(reply: MxCommandReply) -> Result<MxCommandReply, Error> {
let code = reply
.protocol_status
@@ -295,7 +364,7 @@ pub fn ensure_command_success(reply: MxCommandReply) -> Result<MxCommandReply, E
.and_then(|status| ProtocolStatusCode::try_from(status.code).ok())
.unwrap_or(ProtocolStatusCode::Unspecified);
if code == ProtocolStatusCode::Ok {
if code == ProtocolStatusCode::Ok || code == ProtocolStatusCode::MxaccessFailure {
Ok(reply)
} else {
Err(Box::new(CommandError::new(reply)).into())
@@ -306,9 +375,12 @@ pub fn ensure_command_success(reply: MxCommandReply) -> Result<MxCommandReply, E
/// [`MxCommandReply`] to an [`Error::MxAccess`].
///
/// This is the second reply check applied to the typed command path, after
/// [`ensure_command_success`] confirms the protocol envelope is `Ok`. It
/// enforces MXAccess parity: a reply can carry an `Ok` protocol envelope while
/// MXAccess itself rejected the operation. Following COM semantics, only a
/// [`ensure_command_success`] confirms the protocol envelope is `Ok` (or a
/// [`ProtocolStatusCode::MxaccessFailure`] the first check lets fall through).
/// It enforces MXAccess parity: a reply can carry an `Ok` protocol envelope
/// while MXAccess itself rejected the operation, and a
/// [`ProtocolStatusCode::MxaccessFailure`] envelope is itself an MXAccess-level
/// failure regardless of `hresult`. Following COM semantics, only a
/// **negative** `hresult` is a failure — positive codes such as `S_FALSE = 1`
/// are success. A `MXSTATUS_PROXY` entry is treated as a failure when its
/// `category` is not [`MxStatusCategory::Ok`]; the `success` member mirrors the
@@ -321,17 +393,24 @@ pub fn ensure_command_success(reply: MxCommandReply) -> Result<MxCommandReply, E
///
/// # Errors
///
/// Returns [`Error::MxAccess`] when `reply.hresult` is negative or any
/// Returns [`Error::MxAccess`] when the reply's protocol code is
/// [`ProtocolStatusCode::MxaccessFailure`], `reply.hresult` is negative, or any
/// `reply.statuses` entry reports a category other than
/// [`MxStatusCategory::Ok`].
pub fn ensure_mxaccess_success(reply: MxCommandReply) -> Result<MxCommandReply, Error> {
let protocol_code = reply
.protocol_status
.as_ref()
.and_then(|status| ProtocolStatusCode::try_from(status.code).ok())
.unwrap_or(ProtocolStatusCode::Unspecified);
let mxaccess_failure = protocol_code == ProtocolStatusCode::MxaccessFailure;
let hresult_failure = reply.hresult.is_some_and(|hresult| hresult < 0);
let status_failure = reply
.statuses
.iter()
.any(|status| status.category != MxStatusCategory::Ok as i32);
if hresult_failure || status_failure {
if mxaccess_failure || hresult_failure || status_failure {
Err(Box::new(MxAccessError::new(reply)).into())
} else {
Ok(reply)
+79 -11
View File
@@ -11,7 +11,7 @@
use std::sync::atomic::{AtomicU64, Ordering};
use crate::client::{EventStream, GatewayClient};
use crate::error::{ensure_protocol_success, Error};
use crate::error::{ensure_protocol_success, Error, MxAccessError};
use crate::generated::mxaccess_gateway::v1::mx_command::Payload;
use crate::generated::mxaccess_gateway::v1::mx_command_reply;
use crate::generated::mxaccess_gateway::v1::{
@@ -27,7 +27,7 @@ use crate::generated::mxaccess_gateway::v1::{
WriteSecured2BulkCommand, WriteSecured2BulkEntry, WriteSecured2Command,
WriteSecuredBulkCommand, WriteSecuredBulkEntry, WriteSecuredCommand,
};
use crate::value::{MxStatus, MxValue};
use crate::value::{MxStatus, MxValue, MxValueProjection};
const MAX_BULK_ITEMS: usize = 1_000;
@@ -801,6 +801,7 @@ impl Session {
verifier_user_id: i32,
value: MxValue,
) -> Result<(), Error> {
let secrets = string_secret(&value);
self.invoke(
MxCommandKind::WriteSecured,
Payload::WriteSecured(WriteSecuredCommand {
@@ -811,7 +812,8 @@ impl Session {
value: Some(value.into_proto()),
}),
)
.await?;
.await
.map_err(|error| attach_secrets(error, secrets))?;
Ok(())
}
@@ -831,6 +833,7 @@ impl Session {
value: MxValue,
timestamp_value: MxValue,
) -> Result<(), Error> {
let secrets = string_secret(&value);
self.invoke(
MxCommandKind::WriteSecured2,
Payload::WriteSecured2(WriteSecured2Command {
@@ -842,7 +845,8 @@ impl Session {
timestamp_value: Some(timestamp_value.into_proto()),
}),
)
.await?;
.await
.map_err(|error| attach_secrets(error, secrets))?;
Ok(())
}
@@ -882,7 +886,8 @@ impl Session {
verify_user_password: verify_user_password.to_owned(),
}),
)
.await?;
.await
.map_err(|error| attach_secrets(error, vec![verify_user_password.to_owned()]))?;
authenticate_user_id(&reply)
}
@@ -1074,18 +1079,81 @@ fn add_buffered_item_handle(reply: &MxCommandReply) -> Result<i32, Error> {
fn authenticate_user_id(reply: &MxCommandReply) -> Result<i32, Error> {
match reply.payload.as_ref() {
Some(mx_command_reply::Payload::AuthenticateUser(authenticate)) => Ok(authenticate.user_id),
_ => Err(Error::MalformedReply {
detail: "authenticate_user reply lacked an AuthenticateUser payload".to_owned(),
}),
_ => reply
.return_value
.as_ref()
.and_then(int32_reply_value)
.ok_or_else(|| Error::MalformedReply {
detail:
"authenticate_user reply lacked an AuthenticateUser payload or int32 return_value"
.to_owned(),
}),
}
}
fn archestra_user_id(reply: &MxCommandReply) -> Result<i32, Error> {
match reply.payload.as_ref() {
Some(mx_command_reply::Payload::ArchestraUserToId(archestra)) => Ok(archestra.user_id),
_ => Err(Error::MalformedReply {
detail: "archestra_user_to_id reply lacked an ArchestraUserToId payload".to_owned(),
}),
_ => reply
.return_value
.as_ref()
.and_then(int32_reply_value)
.ok_or_else(|| Error::MalformedReply {
detail:
"archestra_user_to_id reply lacked an ArchestraUserToId payload or int32 return_value"
.to_owned(),
}),
}
}
/// Extract an exact string secret from a credential-sensitive [`MxValue`] so a
/// failing `WriteSecured`/`WriteSecured2` can scrub it from the surfaced error.
/// Non-string values carry no scrubbable secret and yield an empty vector.
fn string_secret(value: &MxValue) -> Vec<String> {
match value.projection() {
MxValueProjection::String(text) if !text.is_empty() => vec![text.clone()],
_ => Vec::new(),
}
}
/// Attach caller-supplied exact secrets to an [`Error::MxAccess`] before it
/// propagates. This both scrubs the stored reply's caller-readable string
/// fields (so `reply()`/`into_reply()` cannot recover a credential MXAccess
/// echoed back verbatim) and keeps the secrets on the error as a
/// belt-and-suspenders for `Display`/`Debug`. Any other error variant is
/// returned unchanged.
fn attach_secrets(error: Error, secrets: Vec<String>) -> Error {
match error {
Error::MxAccess(boxed) => {
let mut reply = boxed.into_reply();
scrub_reply_strings(&mut reply, &secrets);
Error::MxAccess(Box::new(MxAccessError::new(reply).with_secrets(secrets)))
}
other => other,
}
}
/// Replace every non-empty secret occurrence with `<redacted>` in the reply's
/// caller-readable string fields — `protocol_status.message`,
/// `diagnostic_message`, and each `statuses[i].diagnostic_text`. A caller
/// reading the structured reply back off an [`Error::MxAccess`] would otherwise
/// reintroduce the leak that `Display`/`Debug` already close.
fn scrub_reply_strings(reply: &mut MxCommandReply, secrets: &[String]) {
for secret in secrets {
if secret.is_empty() {
continue;
}
if let Some(status) = reply.protocol_status.as_mut() {
status.message = status.message.replace(secret.as_str(), "<redacted>");
}
reply.diagnostic_message = reply
.diagnostic_message
.replace(secret.as_str(), "<redacted>");
for status in &mut reply.statuses {
status.diagnostic_text = status
.diagnostic_text
.replace(secret.as_str(), "<redacted>");
}
}
}
+217 -3
View File
@@ -84,8 +84,10 @@ async fn session_helpers_build_commands_and_preserve_command_errors() {
.write(12, 34, ClientMxValue::int32(123), 0)
.await
.unwrap_err();
let Error::Command(error) = error else {
panic!("write failure should preserve the raw command reply: {error:?}");
// A MXACCESS_FAILURE-coded reply is an MXAccess-level failure, routed to
// Error::MxAccess (matching .NET/Java/Go/Python) rather than Error::Command.
let Error::MxAccess(error) = error else {
panic!("MXACCESS_FAILURE reply should route to Error::MxAccess: {error:?}");
};
assert_eq!(
error.reply().protocol_status.as_ref().unwrap().code,
@@ -804,6 +806,179 @@ async fn authenticate_user_keeps_credentials_out_of_surfaced_errors() {
);
}
#[tokio::test]
async fn authenticate_user_scrubs_exact_caller_credential_echoed_in_diagnostic() {
// CLI-40: MXAccess can echo the supplied credential back inside its failure
// diagnostic (here in statuses[0].diagnostic_text). The token has no
// mxgw_/bearer shape, so the pattern scrub alone cannot catch it — the
// exact-secret scrub must replace the caller's password with <redacted>.
let credential = "sup3rSecretVerify9f3a2b";
let state = Arc::new(FakeState::default());
*state.invoke_override.lock().await = Some(InvokeOverride::CannedReply(Box::new(
command_reply_fixture("authenticate-user.echoed-credential.reply.json"),
)));
let endpoint = spawn_fake_gateway(state.clone()).await;
let client = GatewayClient::connect(ClientOptions::new(endpoint))
.await
.unwrap();
let session = client.session("session-fixture");
let error = session
.authenticate_user(7, "verifier", credential)
.await
.unwrap_err();
assert!(
matches!(error, Error::MxAccess(_)),
"OK protocol + negative hresult must route to Error::MxAccess: {error:?}"
);
let rendered = error.to_string();
assert!(
!rendered.contains(credential),
"exact caller credential leaked into the surfaced error: {rendered}"
);
assert!(
rendered.contains("<redacted>"),
"credential occurrence must be replaced with <redacted>: {rendered}"
);
}
/// Drive `authenticate_user` against a canned reply that echoes the caller's
/// credential in every string field, then assert the surfaced
/// [`Error::MxAccess`] leaks it nowhere — neither through the structured reply a
/// caller can read back (`reply().protocol_status.message`,
/// `reply().diagnostic_message`, `reply().statuses[i].diagnostic_text`) nor
/// through `Display`/`Debug`.
async fn assert_authenticate_user_scrubs_structured_reply(fixture: &str) {
let credential = "sup3rSecretVerify9f3a2b";
let state = Arc::new(FakeState::default());
*state.invoke_override.lock().await = Some(InvokeOverride::CannedReply(Box::new(
command_reply_fixture(fixture),
)));
let endpoint = spawn_fake_gateway(state.clone()).await;
let client = GatewayClient::connect(ClientOptions::new(endpoint))
.await
.unwrap();
let session = client.session("session-fixture");
let error = session
.authenticate_user(7, "verifier", credential)
.await
.unwrap_err();
let Error::MxAccess(mx_access) = &error else {
panic!("{fixture}: credential-echoed reply must route to Error::MxAccess, got {error:?}");
};
// The structured reply a caller can read back must be scrubbed too — the raw
// MxCommandReply otherwise reintroduces the leak Display/Debug already close.
let reply = mx_access.reply();
if let Some(status) = reply.protocol_status.as_ref() {
assert!(
!status.message.contains(credential),
"{fixture}: credential leaked via reply().protocol_status.message: {}",
status.message
);
}
assert!(
!reply.diagnostic_message.contains(credential),
"{fixture}: credential leaked via reply().diagnostic_message: {}",
reply.diagnostic_message
);
for (index, status) in reply.statuses.iter().enumerate() {
assert!(
!status.diagnostic_text.contains(credential),
"{fixture}: credential leaked via reply().statuses[{index}].diagnostic_text: {}",
status.diagnostic_text
);
}
let display = error.to_string();
let debug = format!("{error:?}");
assert!(
!display.contains(credential),
"{fixture}: credential leaked into Display: {display}"
);
assert!(
!debug.contains(credential),
"{fixture}: credential leaked into Debug: {debug}"
);
assert!(
display.contains("<redacted>"),
"{fixture}: Display must mark the redaction: {display}"
);
}
#[tokio::test]
async fn authenticate_user_scrubs_credential_from_structured_reply_ok_protocol_variant() {
// OK protocol envelope + negative hresult: already Error::MxAccess before
// ISSUE 2, but the stored reply's string fields still leaked the credential.
assert_authenticate_user_scrubs_structured_reply(
"authenticate-user.echoed-credential.reply.json",
)
.await;
}
#[tokio::test]
async fn authenticate_user_scrubs_credential_from_structured_reply_mxaccess_failure_variant() {
// PROTOCOL_STATUS_CODE_MXACCESS_FAILURE: before ISSUE 2 this landed in
// Error::Command (unscrubbed, raw Display/Debug) — the red-first case.
assert_authenticate_user_scrubs_structured_reply(
"authenticate-user.echoed-credential-mxaccess-failure.reply.json",
)
.await;
}
#[tokio::test]
async fn authenticate_user_maps_missing_payload_reply_to_malformed_reply() {
// CLI-41: an OK reply with neither a typed AuthenticateUser payload nor a
// return_value is malformed.
let state = Arc::new(FakeState::default());
*state.invoke_override.lock().await = Some(InvokeOverride::CannedReply(Box::new(
command_reply_fixture("authenticate-user.missing-payload.reply.json"),
)));
let endpoint = spawn_fake_gateway(state.clone()).await;
let client = GatewayClient::connect(ClientOptions::new(endpoint))
.await
.unwrap();
let session = client.session("session-fixture");
let error = session
.authenticate_user(7, "verifier", "pw")
.await
.unwrap_err();
assert!(
matches!(error, Error::MalformedReply { .. }),
"missing payload + missing return_value must be MalformedReply, got {error:?}"
);
}
#[tokio::test]
async fn authenticate_user_falls_back_to_return_value_when_typed_payload_absent() {
// CLI-41: an OK reply that carries only a return_value (legacy worker) must
// resolve the user id from it, mirroring add_buffered_item's fallback.
let state = Arc::new(FakeState::default());
*state.invoke_override.lock().await = Some(InvokeOverride::CannedReply(Box::new(
command_reply_fixture("authenticate-user.return-value-only.reply.json"),
)));
let endpoint = spawn_fake_gateway(state.clone()).await;
let client = GatewayClient::connect(ClientOptions::new(endpoint))
.await
.unwrap();
let session = client.session("session-fixture");
let user_id = session
.authenticate_user(7, "verifier", "pw")
.await
.unwrap();
assert_eq!(
user_id, 7,
"user id must resolve from the int32 return_value"
);
}
#[tokio::test]
async fn stream_alarms_emits_snapshot_then_complete_then_transition_in_order() {
let state = Arc::new(FakeState::default());
@@ -955,6 +1130,11 @@ enum InvokeOverride {
/// `AuthenticateUser` rejected by MXAccess) so the client's
/// `ensure_mxaccess_success` check is exercised on the typed helper path.
MxAccessFailure,
/// Reply with a caller-supplied canned [`MxCommandReply`]. Lets a test
/// drive a helper with a shared behavior fixture (e.g. the
/// echoed-credential / missing-payload / return-value-only
/// authenticate-user replies). Boxed to keep the enum small.
CannedReply(Box<MxCommandReply>),
}
#[derive(Clone)]
@@ -1057,6 +1237,7 @@ impl MxAccessGateway for FakeGateway {
payload: None,
..MxCommandReply::default()
})),
InvokeOverride::CannedReply(reply) => Ok(Response::new(*reply)),
InvokeOverride::WriteOk => {
// Extract and capture the WriteCommand payload so the test
// can assert on server_handle, item_handle, user_id, and value.
@@ -1443,15 +1624,48 @@ fn command_reply_fixture(file_name: &str) -> MxCommandReply {
})
.collect();
// The fixtures that exercise the return_value fallback path carry a typed
// `returnValue` (VT_I4). Project it so a canned reply can drive the
// helper's payload -> return_value -> MalformedReply precedence.
let return_value = fixture.get("returnValue").and_then(|value| {
value["int32Value"].as_i64().map(|int32| MxValue {
data_type: MxDataType::Integer as i32,
variant_type: value["variantType"].as_str().unwrap_or("VT_I4").to_owned(),
kind: Some(Kind::Int32Value(int32 as i32)),
..MxValue::default()
})
});
// Honor the fixture's real protocol status (code + message) so a canned
// reply can drive the MXACCESS_FAILURE routing path, not just an OK
// envelope. Falls back to an OK envelope when the fixture omits it.
let protocol_status = fixture.get("protocolStatus").map_or_else(
|| ok_status("command ok"),
|status| {
let code_name = status["code"].as_str().unwrap_or("PROTOCOL_STATUS_CODE_OK");
ProtocolStatus {
code: ProtocolStatusCode::from_str_name(code_name)
.unwrap_or_else(|| panic!("unknown protocol status code {code_name}"))
as i32,
message: status["message"].as_str().unwrap_or_default().to_owned(),
}
},
);
MxCommandReply {
session_id: fixture["sessionId"].as_str().unwrap_or_default().to_owned(),
correlation_id: fixture["correlationId"]
.as_str()
.unwrap_or_default()
.to_owned(),
protocol_status: Some(ok_status("command ok")),
protocol_status: Some(protocol_status),
hresult: fixture["hresult"].as_i64().map(|hresult| hresult as i32),
statuses,
diagnostic_message: fixture["diagnosticMessage"]
.as_str()
.unwrap_or_default()
.to_owned(),
return_value,
..MxCommandReply::default()
}
}
+42 -7
View File
@@ -99,9 +99,20 @@ library:
are skipped. Only successes are cached; failures always reach the inner verifier.
On a gateway-initiated revoke/rotate/delete the dashboard admin service calls
`IApiKeyCacheInvalidator.Invalidate(keyId)`, evicting the cached entry
immediately. The short TTL is the backstop for out-of-band mutations (a direct DB
edit, or a revoke run by the separate `apikey` CLI process, whose in-memory cache
is not the running gateway's cache).
immediately. `Invalidate` bumps a per-key generation counter **before** it evicts,
and `VerifyAsync` snapshots that generation before the inner verify and re-checks
it after writing the cache entry (set-then-recheck); a revoke that lands while a
verification is still in flight in the inner library therefore discards that
verification's repopulation instead of re-caching the just-revoked identity for a
full TTL (SEC-34). The short TTL remains the backstop for two bounded-staleness
windows it cannot close directly: (1) out-of-band mutations (a direct DB edit, or a
revoke run by the separate `apikey` CLI process, whose in-memory cache is not the
running gateway's cache); and (2) a key whose `ExpiresUtc` passes while cached keeps
authenticating until the entry's TTL elapses — expiry is enforced by the inner
library verifier, which a cache hit never reaches, and the verification identity the
library returns carries no expiry timestamp, so the cache cannot cap an entry at the
key's expiry (capping it needs the donor library to surface expiry on the
verification identity). The default 15 s TTL bounds both windows.
- **`CoalescingMarkApiKeyStore`** wraps the library `IApiKeyStore` and forwards at
most one `MarkUsed` write per key per
`MxGateway:Security:ApiKeyLastUsedCoalesceSeconds` (default 60 s), so even under a
@@ -116,6 +127,26 @@ a dictionary lookup. Both windows are configurable and may be set to `0` to disa
the respective mechanism; see
[GatewayConfiguration](./GatewayConfiguration.md).
Failures are never cached — a wrong secret always reaches the store — so the
failure path is shielded by `ApiKeyFailureLimiter` instead, consulted before
`VerifyAsync` runs. It counts failures over one sliding
`MxGateway:Security:ApiKeyFailureWindowSeconds` window in two layers: a composite
`(transport peer, key id)` partition capped at `ApiKeyFailureLimit`, and a
per-key-id aggregate across all peers capped at `ApiKeyFailureAggregateLimit`. The
key id never partitions on its own — it is public, so an attacker-supplied one
would otherwise let any peer throttle a key it does not hold — and it joins the
partition only when the presented token is validly shaped
(`mxgw_<keyId>_<secret>`, key id at most 64 characters), with at most 32 key-id
partitions per address before the overflow collapses onto that address's fallback
partition. An over-limit state admits one probe per
`ApiKeyFailureProbeIntervalSeconds` through to the real verifier and refuses
everything else with `ResourceExhausted` before the store read, so a legitimate
holder presenting the correct secret always reaches the constant-time compare and
resets both layers; the counter's LRU eviction (`ApiKeyFailureTrackedPeers`)
prefers expired windows and will not drop an over-limit partition below a 2x
overshoot ceiling, so the memory bound cannot be turned into a way to clear an
active block. See [Authorization](./Authorization.md) for the enforcement path.
## Storage
API-key state lives in a dedicated SQLite database owned by the shared library.
@@ -128,10 +159,14 @@ is derived from `Environment.GetFolderPath(SpecialFolder.CommonApplicationData)`
(`C:\ProgramData\MxGateway\gateway-auth.db` on Windows,
`/usr/share/MxGateway/gateway-auth.db` or the container equivalent elsewhere) so the
credential store is never written relative to the launch working directory on a
non-Windows host. The production hosts pin the explicit Windows path in
`appsettings.json`. `GatewayOptionsValidator` rejects a non-rooted (relative)
`SqlitePath` so a bad override fails fast at startup rather than scattering the store
by launch CWD (SEC-01).
non-Windows host. `appsettings.json` no longer ships an explicit path (SEC-33): the
removed Windows literal matched the Windows code default and, being non-rooted on a
Unix host, would have resolved against the CWD there; deployed hosts override it
through the NSSM environment (`MxGateway__Authentication__SqlitePath`).
`GatewayOptionsValidator` rejects a `SqlitePath` that is not rooted **on the host
running the gateway** (`Path.IsPathRooted`, current OS) — a relative filename or a
foreign-platform literal fails fast at startup rather than scattering the store by
launch CWD (SEC-01, SEC-33).
The library owns the SQLite schema and connection factory. The `api_keys` table
carries the key id, key prefix, secret-hash blob, display name, serialized scopes,
+7 -2
View File
@@ -89,9 +89,14 @@ The flow is:
The status codes are deliberately distinct: `Unauthenticated` signals "we do not know who you are," and `PermissionDenied` signals "we know who you are, but you cannot do this." Treating the two as the same code would make troubleshooting harder for client implementations.
### Rate limiting the auth surface (SEC-11)
### Rate limiting the auth surface (SEC-11, SEC-31, SEC-32)
Before the verification store read, the helper checks a cheap in-process per-peer failure counter (`ApiKeyFailureLimiter`). A peer that has accumulated more than `MxGateway:Security:ApiKeyFailureLimit` failed attempts inside the sliding `ApiKeyFailureWindowSeconds` window is short-circuited with `StatusCode.ResourceExhausted` — so online guessing of API-key secrets cannot spend a SQLite read (and, in a naive design, a cache miss) per attempt. The peer is keyed on the presented key id where the token parses, falling back to the transport peer address; keying on key id throttles a single abusive credential without penalizing co-located clients behind a shared NAT. A successful verification resets the peer's counter. The counter is a bounded LRU (`ApiKeyFailureTrackedPeers`) so it cannot grow without limit. `ResourceExhausted` reveals only that throttling is in effect, not whether any particular secret was valid, preserving the opaque-failure property.
Before the verification store read, the helper asks a cheap in-process failure counter (`ApiKeyFailureLimiter`) whether the attempt may proceed, so online guessing of API-key secrets cannot spend a SQLite read (and, in a naive design, a cache miss) per attempt. The counter has two layers over one sliding `ApiKeyFailureWindowSeconds` window:
- **Composite `(transport peer, key id)` partitions.** Reaching `MxGateway:Security:ApiKeyFailureLimit` failures binds the throttle to the address that produced them. The key id alone is never the partition: key ids are not secret — they ride in every token and are listed on the dashboard — so keying on them let any network peer deny a key to its legitimate holder. The key id joins the partition only after a token-shape check (literal `mxgw` prefix, at least three non-empty `_` segments, key id of at most 64 characters), and one address may mint at most 32 key-id partitions before the overflow collapses onto that address's fallback partition.
- **A per-key-id aggregate** across all peers (`ApiKeyFailureAggregateLimit`, default 30), which bounds a distributed or source-rotating sprayer that never trips any single partition.
An over-limit state is a valve, not a wall: one request per `ApiKeyFailureProbeIntervalSeconds` (default 5 s) is admitted through to the real verifier, and everything else is refused with `StatusCode.ResourceExhausted` before the store read. The slot is claimed atomically, so a burst arriving together at an interval boundary still yields exactly one admission, and a slot claimed for a request that a later layer then refuses is handed back under a per-state version stamp — never by timestamp comparison, which collides whenever a concurrent failure re-arms the same state on the same clock tick. A successful verification resets both layers — which is why the reset path stays reachable while a key is under active spray. One exception: when the caller's key id was collapsed into its address's shared fallback partition by the per-peer cap, a success clears the key's aggregate but leaves that shared partition alone, since it also holds failures contributed by other key ids from the same address. The tracked partitions form a bounded LRU (`ApiKeyFailureTrackedPeers`) whose eviction prefers fully expired windows and never removes an over-limit partition below a 2x transient overshoot ceiling, so the cap bounds memory without becoming a reset button for an active block. `ResourceExhausted` reveals only that throttling is in effect, not whether any particular secret was valid, preserving the opaque-failure property. Refusals increment `mxgateway.auth.throttled`, tagged `stage=peer|aggregate` and nothing else — `/metrics` is unauthenticated, so neither key ids nor peer addresses may appear there.
The dashboard login surface is throttled independently: `POST /auth/login` carries a fixed-window ASP.NET Core rate-limiter policy keyed per remote IP (`MxGateway:Security:LoginRateLimit*`), rejecting a burst with HTTP 429 before the LDAP bind is relayed to the directory. See [GatewayConfiguration](./GatewayConfiguration.md#security-options).
+40
View File
@@ -70,6 +70,40 @@ The rules those fixtures lock in are:
negative. Positive COM success codes such as `S_FALSE` pass, matching COM
semantics.
### Malformed-Reply And Credential-Redaction Conformance
Three further command reply fixtures pin the id/handle-extraction and
credential-redaction contracts for the credential-bearing helpers:
| Fixture | Reply | Expected behavior |
|---|---|---|
| `authenticate-user.echoed-credential.reply.json` | OK envelope, negative `hresult`, and the caller's credential echoed into `protocolStatus.message`, `statuses[0].diagnosticText`, and `diagnosticMessage` | the surfaced error redacts the exact secret from **both** the rendered message and the structured reply accessors (never leaks the verbatim value) |
| `authenticate-user.echoed-credential-mxaccess-failure.reply.json` | the same echo, but coded `PROTOCOL_STATUS_CODE_MXACCESS_FAILURE` | identical redaction; confirms every client routes the MXAccess-failure protocol code to its MXAccess error type and scrubs it |
| `authenticate-user.missing-payload.reply.json` | OK envelope, no `AuthenticateUser` payload, no `return_value` | a typed malformed-reply error, never a proto3 default `0` and never an NRE |
| `authenticate-user.return-value-only.reply.json` | OK envelope, `return_value.int32_value = 7`, no typed payload | the id resolves to `7` via the legacy `return_value` compatibility path |
The rules those fixtures lock in are:
- **Malformed-reply extraction (CLI-41).** Every helper that extracts a scalar
id/handle (`AuthenticateUser`, `ArchestrAUserToId`, `AddBufferedItem`, and the
handle extractors) prefers the typed payload; when it is absent it falls back
to `return_value` **only** when `return_value` is present with the expected
int32 variant; when neither is present it raises a typed malformed-reply error.
It never surfaces a proto3 default `0` and never throws a null-reference.
- **Credential redaction (CLI-40).** The credential-bearing helpers
(`AuthenticateUser`, `WriteSecured`/`WriteSecured2`) scrub the exact secret
values they were called with from any surfaced error — both the rendered
message text **and** the structured reply the error still exposes (a
server-echoed credential lives in `protocolStatus.message` and
`statuses[].diagnosticText`, which the error's raw-reply accessor would
otherwise re-expose to a logger dumping structured fields). The redacted error
therefore carries a scrubbed clone of the reply. This is defense-in-depth on
top of the by-construction guarantee that exceptions carry reply-derived text,
not the request. The marker is `<redacted>` in the Go, Rust, and Java clients
and `[redacted]` in the Python client and the .NET CLI; each suite asserts that
neither the surfaced message nor the exposed reply still contains the
credential, and that the message contains the client's marker.
## Event Streams
Event stream fixtures live in
@@ -100,6 +134,12 @@ behavior. A language helper may expose native booleans, integers, strings,
arrays, and timestamps, but it must keep `rawDiagnostic`, raw data type fields,
and raw byte payloads accessible when conversion is incomplete.
Each status case also carries an independent `wantSuccess` boolean alongside its
`status` object. The success/failure conformance tests assert the helper's
verdict against this fixture-declared expectation rather than recomputing it from
`category` (the same formula under test), so a regression in the verdict rule
cannot hide behind a self-consistent computation.
## Auth, Timeout, And Cancel Behavior
Authentication fixtures live in `clients/proto/fixtures/behavior/auth/`. They
+19 -2
View File
@@ -126,11 +126,28 @@ rules across all five clients (see
failing before a prior `AuthenticateUser` + `AdviseSupervisory` surfaces the
native failure unchanged — the helper does not pre-validate or reorder it.
**Malformed-reply extraction:** the id/handle-returning helpers
(`AuthenticateUser`, `ArchestrAUserToId`, `AddBufferedItem`, and the handle
extractors) follow one contract across all five clients — prefer the typed
payload; fall back to `return_value` only when it is present with the expected
int32 variant; when neither is present, raise a typed malformed-reply error.
They never surface a proto3 default `0` and never throw a null-reference (CLI-41).
**Credential handling:** `AuthenticateUser` credentials and `WriteSecured`
secured payloads route through each client's secret-redaction seam so they never
reach logs, exception text, or `ToString`/`Debug`/`Display` — the value is carried
only on the wire. Each client's test suite asserts a distinctive credential is
absent from any surfaced error.
only on the wire. In addition to that by-construction guarantee (exceptions carry
reply-derived text, not the request), every client scrubs the **exact** secret
values it was called with from any surfaced error as defense-in-depth, so a
gateway or MXAccess diagnostic that echoes a credential back cannot leak it
(CLI-40). The scrub covers **both** the rendered message and the structured reply
the error still exposes (`protocolStatus.message`, `statuses[].diagnosticText`,
`diagnosticMessage`): the redacted error carries a scrubbed clone of the reply so
a logger dumping the exception's structured fields cannot reintroduce the leak.
This holds regardless of whether the reply is coded `OK` (with a failing HRESULT)
or `MXACCESS_FAILURE` — every client routes both to its MXAccess error type. Each
client's test suite asserts the distinctive credential is absent from both the
surfaced message and the exposed reply, and that the redaction marker is present.
Shipped in all five clients (.NET / Go / Rust / Python / Java).
+83
View File
@@ -32,6 +32,73 @@ $env:MXGATEWAY_TEST_ITEM = 'TestObject.TestInt'
Use plaintext only for a local gateway. Use TLS when the gateway crosses a
machine boundary or uses a production certificate.
## Versioning
Every client's version lives in its own manifest: `clients/rust/Cargo.toml`
(`[package]` and `[workspace.package]`, both must match — `crates/mxgw-cli`
inherits via `version.workspace = true`), `clients/python/pyproject.toml`
(`[project].version`) and `clients/python/src/zb_mom_ww_mxgateway/version.py`
(`__version__`, must match `pyproject.toml`), `clients/go/mxgateway/version.go`
(`ClientVersion`), `clients/dotnet/ZB.MOM.WW.MxGateway.Client/ZB.MOM.WW.MxGateway.Client.csproj`
(`<Version>`), and `clients/java/build.gradle` (`subprojects { version = ... }`,
mirrored by the hand-maintained `MxGatewayClientVersion.CLIENT_VERSION`
constant — the two have drifted before and there is no build-time link
between them, so bump both together).
`src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csproj`
(`<Version>`) is a fifth, easy-to-miss manifest: it is not itself a
language client, but `Invoke-PackDotnet` in `scripts/pack-clients.ps1`
packs and publishes it in lockstep with the .NET Client (both
`ZB.MOM.WW.MxGateway.*` nupkgs go through the same `-Publish` loop), and
Contracts and the .NET Client have always released at the same version.
Bump Contracts' `<Version>` alongside the .NET Client's — leaving it behind
means the next `-Publish` packs a stale Contracts version, the collision
guard below correctly refuses to re-publish it, and the loop aborts
mid-way with the Client possibly already pushed (nupkgs are enumerated
alphabetically, and `Client` sorts before `Contracts`). This is distinct
from `src/Directory.Build.props`'s repo-wide `<Version>` default, which
stamps the Server/Worker/test assemblies and is not part of the published
client package set — see the comment there.
**Bump the version before every publish, never after.** A Gitea package feed
rejects re-uploading an existing name+version, and `scripts/pack-clients.ps1`
enforces this before it ever attempts a push: each per-language `-Publish`
step queries the Gitea package API
(`GET /api/v1/packages/dohertj2/{type}/{name}/{version}`) for the version
about to be published and aborts with a clear error if it already exists —
the script never force-overwrites a published artifact. `scripts/tag-go-module.ps1`
carries the equivalent guard for the Go module: it refuses to create a
`clients/go/vX.Y.Z` tag unless `clients/go/mxgateway/version.go`'s
`ClientVersion` already equals `X.Y.Z` (CLI-21/CLI-39), so a forgotten
version bump fails the tag instead of shipping a mismatched module.
As of 2026-08-07 (CLI-39) all five clients — plus `ZB.MOM.WW.MxGateway.Contracts`,
which releases in lockstep with the .NET Client — moved to **0.2.0**,
converging on one number after four of the five had drifted onto the
*already-published* 0.1.2/0.1.1 while their public APIs kept changing
underneath it (see `archreview/2026-07-12/remediation/50-clients.md` CLI-39).
A code-review follow-up on the same branch caught that the initial CLI-39
pass bumped the .NET Client but left `Contracts.csproj` at 0.1.2 — since
both publish through the same `Invoke-PackDotnet` `-Publish` loop, that
would have made the very next `.NET` publish abort on the new collision
guard partway through (Client already pushed, Contracts refused as a
re-publish of the already-published 0.1.2). Fixed in the same branch.
Verified against the live Gitea package API at that time: `nuget` had
`ZB.MOM.WW.MxGateway.Client` and `.Contracts` published through 0.1.2; `pypi` (`zb-mom-ww-mxaccess-gateway-client`)
and `cargo` (`zb-mom-ww-mxgateway-client`) had only reached 0.1.1 despite their
source pinning 0.1.2; **`maven`
(`com.zb.mom.ww.mxgateway:zb-mom-ww-mxgateway-client`) had already published
0.2.0 on 2026-06-26** — before the CLI-37/38/40/41 conformance fixes changed
the client's observable behavior (`category`-based status validation,
`hresult < 0`, exact-secret redaction, typed malformed-reply errors). Reusing
0.2.0 for the conformant Java build would have labeled two different APIs
with the same coordinate, so **Java is the one exception: it shipped as
0.2.1**, not 0.2.0. Operators publishing a future release must re-check the
target version against the live registry before assuming any of these
numbers are still unclaimed — the guards above do this automatically at
publish time, but a version bump in the source is still a manual step per
client.
## .NET
The .NET client uses .NET 10 and references
@@ -117,6 +184,22 @@ The Rust workspace builds the `mxgateway-client` library crate and the `mxgw`
CLI crate. `build.rs` generates `tonic` and `prost` modules into Cargo build
output on each build that needs updated protobuf output.
`build.rs` resolves its `.proto` inputs repo-path-first, then vendored: it
prefers the canonical protos under `src/ZB.MOM.WW.MxGateway.Contracts/Protos`
so an in-repo edit is live immediately, and falls back to the copies vendored
into `clients/rust/protos/` only when the canonical directory is absent — the
case for a published crate unpacked outside this repo. The vendored copies
are declared in `Cargo.toml`'s `include` list, so `cargo package`/`cargo
publish` ship them inside the `.crate`, making the crate buildable standalone
with no access to the rest of the mxaccessgw repo. Any Contracts proto change
must refresh `clients/rust/protos/` in the same commit; `scripts/check-codegen.ps1`
Check 3 byte-compares the vendored copies against the canonical protos and
fails on drift. Because the vendored protos make a standalone build possible,
`cargo package`/`cargo publish` run **with** verification (no `--no-verify`) —
a `cargo package` that cannot build from the vendored tree alone would mean
the vendored copies are stale, and verification is what catches that before
publish.
Regenerate and compile Rust bindings:
```powershell
+21 -11
View File
@@ -74,10 +74,12 @@ protoc-version encoding drift and does not false-fail across protoc releases
(it warns, rather than fails, when protoc is off the pin).
The gateway test project carries an independent, protoc-free freshness guard:
`ClientProtoInputTests.Descriptor_ContainsEveryContractMessageAndField` reflects
over the in-process contract descriptors and fails if any contract message or
field is missing from the committed protoset. This is the primary CI gate for
descriptor staleness; a red test means "regenerate and commit the protoset."
`ClientProtoInputTests.Descriptor_ContainsEveryContractSymbol` reflects over the
in-process contract descriptors — `mxaccess_gateway.proto`, `mxaccess_worker.proto`,
and `galaxy_repository.proto` — and fails if any contract message, field, enum,
enum value, service, or method is missing from the committed protoset. This is
the primary CI gate for descriptor staleness; a red test means "regenerate and
commit the protoset."
### Pinned generator versions
@@ -88,15 +90,23 @@ scripts assert the pin and resolve tools from `PATH`:
| Generator | Pinned version | Guard |
|-----------|----------------|-------|
| protoc (descriptor set) | 34.1 | version assertion in `scripts/publish-client-proto-inputs.ps1` |
| `Grpc.Tools` (C# `Generated/`) | 2.80.0 (contracts csproj) | `scripts/check-codegen.ps1` git-diff of `Generated/` |
| `grpcio-tools` (Python) | 1.80.0 (protobuf runtime 6.31.1) | version assertion in `clients/python/generate-proto.ps1` |
| protobuf / grpc-java (Java) | `protobufVersion` / `grpcVersion` in `clients/java/build.gradle` | `checkGeneratedClean` gradle task |
| `Grpc.Tools` (C# `Generated/`) | 2.80.0 (contracts csproj) | `scripts/check-codegen.ps1` git-diff of `Generated/` (Check 2) |
| `protoc-gen-go` (Go) | v1.36.11 | version assertion in `clients/go/generate-proto.ps1`; `check-codegen.ps1` Check 4 |
| `protoc-gen-go-grpc` (Go) | 1.6.2 | version assertion in `clients/go/generate-proto.ps1`; `check-codegen.ps1` Check 4 |
| `grpcio-tools` (Python) | 1.80.0 (protobuf runtime 6.31.1) | version assertion in `clients/python/generate-proto.ps1`; `check-codegen.ps1` Check 4 |
| protobuf / grpc-java (Java) | `protobufVersion` / `grpcVersion` in `clients/java/build.gradle` | CI `git diff --exit-code` over `clients/java/src/main/generated` after `gradle test` (the `checkGeneratedClean` gradle task is the equivalent local check) |
A newer `grpcio-tools` stamps a `GRPC_GENERATED_VERSION` above the pinned grpcio
runtime and breaks Python `pytest`; the Java protobuf plugin rewrites
`MxaccessGateway.java` with spurious protobuf-runtime-version churn on every build
(revert that one file when no `.proto` changed — see
[Gateway Testing](./GatewayTesting.md) "Continuous Integration").
runtime and breaks Python `pytest`, so the Python and Go scripts assert their
generator pins before regenerating. Under the pinned grpc/protobuf toolchain the
Java protobuf plugin regenerates byte-identical output (modulo real `.proto`
changes), so CI enforces Java freshness with a direct `git diff --exit-code -- clients/java/src/main/generated`
step after `gradle test` (which transitively regenerates via `generateProto`); the
`checkGeneratedClean` gradle task is the equivalent check for local/manual use. The
old unconditional churn-revert CI step (which masked message-level drift in the
single-file Java aggregates) was deleted (IPC-24). Go and Python committed
bindings are guarded by `check-codegen.ps1` **Check 4**, which regenerates both
and fails on any diff.
## Output Directories
+10 -5
View File
@@ -114,7 +114,11 @@ dotnet build src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csp
`scripts/check-codegen.ps1` enforces this in CI (it force-regenerates and fails on any
`git diff` against the committed `Generated/`) — that regeneration diff in the `portable`
job is the primary guard. The SSH-driven `windows-x86` job's net48 worker build is the
secondary guard (a stale `Generated/` also breaks the x86 build with `CS0246`).
secondary guard (a stale `Generated/` also breaks the x86 build with `CS0246`). The same
script runs four checks in total: the committed client descriptor set (Check 1), the C#
`Generated/` (Check 2), the Rust vendored protos (Check 3), and the Go/Python client bindings
(Check 4) each regenerate and fail on any diff. See
[Client Proto Generation](./ClientProtoGeneration.md) for the pinned generator versions.
Client generation inputs are published through
`clients/proto/proto-inputs.json` and the descriptor set under
@@ -152,10 +156,11 @@ pwsh -File scripts/publish-client-proto-inputs.ps1
Freshness is guarded two ways so a skipped regeneration cannot ship silently:
- `ClientProtoInputTests.Descriptor_ContainsEveryContractMessageAndField` (gateway test project)
reflects over the in-process contract descriptors and fails if any message or field is missing
from the committed protoset. It is semantic (symbol presence), needs no protoc, and runs in the
Linux CI. A red test means "regenerate and commit the protoset."
- `ClientProtoInputTests.Descriptor_ContainsEveryContractSymbol` (gateway test project)
reflects over the in-process contract descriptors — including `galaxy_repository.proto` — and
fails if any message, field, enum, enum value, service, or method is missing from the committed
protoset. It is semantic (symbol presence), needs no protoc, and runs in the Linux CI. A red
test means "regenerate and commit the protoset."
- `pwsh -File scripts/publish-client-proto-inputs.ps1 -Check` rebuilds the descriptor and compares
it to the committed one. The comparison normalizes both sides through the same protoc with
`source_code_info` stripped, so it does not false-fail across protoc releases.
+8 -3
View File
@@ -163,8 +163,13 @@ working across that gap, the cache persists its dataset to disk:
- After every successful **heavy** refresh (a deploy change), the raw
hierarchy and attribute rowsets are written to
`MxGateway:Galaxy:SnapshotCachePath`
(default `C:\ProgramData\MxGateway\galaxy-snapshot.json`). The write is
`MxGateway:Galaxy:SnapshotCachePath`. `appsettings.json` no longer ships an
explicit value (SEC-33): the gateway supplies a `CommonApplicationData`-derived
default when the bound value is blank — `C:\ProgramData\MxGateway\galaxy-snapshot.json`
on Windows, `/usr/share/MxGateway/galaxy-snapshot.json` (or the container
equivalent) elsewhere — and `GalaxyRepositoryOptionsValidator` rejects a
non-rooted or invalid path at startup when persistence is on, so the snapshot
can never land relative to the launch working directory. The write is
atomic — a temp file plus rename — so a crash mid-write cannot corrupt the
snapshot. Cheap no-change ticks write nothing; the file is already current.
- On the **first** refresh after startup, before any SQL runs, the cache
@@ -404,7 +409,7 @@ Bound to `MxGateway:Galaxy` via `GalaxyRepositoryOptions`.
| `MxGateway:Galaxy:ConnectionString` | `Server=localhost;Database=ZB;Integrated Security=True;TrustServerCertificate=True;Encrypt=False;` | SQL Server connection string for the Galaxy Repository. Integrated Security against `localhost` is the dev default; production deployments should override this through the standard double-underscore environment variable form, e.g. `MxGateway__Galaxy__ConnectionString`. |
| `MxGateway:Galaxy:CommandTimeoutSeconds` | `60` | Per-command SQL timeout. Applies to all three RPCs. |
| `MxGateway:Galaxy:PersistSnapshot` | `true` | Persists each successful browse dataset to disk and reloads it at startup. See [On-disk snapshot](#on-disk-snapshot). |
| `MxGateway:Galaxy:SnapshotCachePath` | `C:\ProgramData\MxGateway\galaxy-snapshot.json` | File path for the persisted browse snapshot. Ignored when `PersistSnapshot` is `false`. |
| `MxGateway:Galaxy:SnapshotCachePath` | derived from `CommonApplicationData` (`C:\ProgramData\MxGateway\galaxy-snapshot.json` on Windows, `/usr/share/MxGateway/galaxy-snapshot.json` or the container equivalent elsewhere) | File path for the persisted browse snapshot. Ignored when `PersistSnapshot` is `false`. `appsettings.json` no longer ships an explicit value (SEC-33): the gateway seeds the `CommonApplicationData`-derived default when the bound value is blank, and `GalaxyRepositoryOptionsValidator` enforces — when `PersistSnapshot` is `true` — that the path is non-blank, valid, and **rooted on the host running the gateway** (`Path.IsPathRooted`, current OS). A bare filename or a foreign-platform literal fails startup. |
The connection string is not treated as a secret in dev (`Integrated
Security`), but production deployments that use SQL authentication should set
+34 -16
View File
@@ -14,7 +14,6 @@ paths, timeouts, queue sizes, enum values, or protocol values are invalid.
"MxGateway": {
"Authentication": {
"Mode": "ApiKey",
"SqlitePath": "C:\\ProgramData\\MxGateway\\gateway-auth.db",
"PepperSecretName": "MxGateway:ApiKeyPepper",
"RunMigrationsOnStartup": true
},
@@ -71,8 +70,7 @@ paths, timeouts, queue sizes, enum values, or protocol values are invalid.
"ConnectionString": "Server=localhost;Database=ZB;Integrated Security=True;TrustServerCertificate=True;Encrypt=False;",
"CommandTimeoutSeconds": 60,
"DashboardRefreshIntervalSeconds": 30,
"PersistSnapshot": true,
"SnapshotCachePath": "C:\\ProgramData\\MxGateway\\galaxy-snapshot.json"
"PersistSnapshot": true
},
"Alarms": {
"Enabled": false,
@@ -93,15 +91,17 @@ Environment variables use the normal .NET double-underscore form. For example,
| Option | Default | Description |
|--------|---------|-------------|
| `MxGateway:Authentication:Mode` | `ApiKey` | Selects public gRPC authentication. Supported values are `ApiKey` and `Disabled`. `Disabled` bypasses API-key verification and is for local development only. |
| `MxGateway:Authentication:SqlitePath` | derived from `CommonApplicationData` (`C:\ProgramData\MxGateway\gateway-auth.db` on Windows, `/usr/share/MxGateway/gateway-auth.db` or the container equivalent elsewhere) | SQLite database path for API-key records and audit rows when API-key authentication is enabled. The code default is built from `Environment.GetFolderPath(SpecialFolder.CommonApplicationData)` so the credential store never lands in the launch working directory on a non-Windows host; the production hosts pin the explicit Windows path in `appsettings.json`, which overrides the code default. |
| `MxGateway:Authentication:SqlitePath` | derived from `CommonApplicationData` (`C:\ProgramData\MxGateway\gateway-auth.db` on Windows, `/usr/share/MxGateway/gateway-auth.db` or the container equivalent elsewhere) | SQLite database path for API-key records and audit rows when API-key authentication is enabled. The code default is built from `Environment.GetFolderPath(SpecialFolder.CommonApplicationData)` so the credential store never lands in the launch working directory on a non-Windows host. `appsettings.json` no longer ships an explicit value (SEC-33): the removed Windows literal was byte-identical to the Windows code default, and a Windows-absolute literal is **not** rooted on a Unix host, so it would have resolved against the CWD there. Deployed hosts still override the path through the NSSM environment (`MxGateway__Authentication__SqlitePath`). |
| `MxGateway:Authentication:PepperSecretName` | `MxGateway:ApiKeyPepper` | Configuration key used to read the HMAC pepper for API-key secret hashing. The dashboard effective configuration redacts this value. |
| `MxGateway:Authentication:RunMigrationsOnStartup` | `true` | Runs SQLite auth schema migrations at gateway startup when API-key authentication is enabled. |
When `Mode` is `ApiKey`, `SqlitePath` and `PepperSecretName` must be present.
`SqlitePath` must be a valid filesystem path and must be **rooted** (absolute):
the validator rejects a non-rooted path so a relative override cannot silently
resolve against the working directory and scatter the credential store by launch
CWD (SEC-01).
`SqlitePath` must be a valid filesystem path and must be **rooted** (absolute)
**on the host running the gateway**: the validator uses `Path.IsPathRooted`
(current OS), so a bare filename — or a foreign-platform literal such as a
Windows `C:\...` path on a Unix host — fails fast at startup instead of silently
resolving against the working directory and scattering the credential store by
launch CWD (SEC-01, SEC-33).
## Worker Options
@@ -149,13 +149,15 @@ All numeric session options must be greater than zero.
| Option | Default | Description |
|--------|---------|-------------|
| `MxGateway:Events:QueueCapacity` | `10000` | Capacity for bounded per-session event queues used by the gateway worker event channel and the public gRPC event stream queue. Gateway-side buffering per session is at most `3 ×` this value: the bounded worker event channel plus the read loop's staging channel, which is bounded at `2 ×` it. Overflow of either bound faults the session with `ProtocolViolation` and kills its worker. |
| `MxGateway:Events:QueueCapacity` | `10000` | Capacity for bounded per-session event queues used by the gateway worker event channel and the public gRPC event stream queue. Gateway-side buffering per session is at most `3 ×` this value: the bounded worker event channel plus the read loop's staging channel, which is bounded at `2 ×` it. Overflow of either bound faults the session with `ProtocolViolation` and kills its worker. Must be between `1` and `int.MaxValue / 2` so the derived `2 ×` staging bound cannot overflow when a worker client is created. |
| `MxGateway:Events:BackpressurePolicy` | `FailFast` | Per-subscriber event backpressure behavior when a subscriber's bounded event channel overflows. Overflow is isolated to the offending subscriber: it is always disconnected with an `EventQueueOverflow` fault while the session pump and other subscribers keep running. `FailFast` additionally faults the whole session only in the legacy single-subscriber case (the current default mode); with multiple subscribers it degrades to a per-subscriber disconnect so one slow consumer never faults a shared session. `DisconnectSubscriber` disconnects only the slow subscriber in all cases. |
| `MxGateway:Events:ReplayBufferCapacity` | `1024` | Maximum number of events retained per session in the replay ring buffer, used to re-deliver events a returning subscriber missed (reconnect/reattach). The oldest retained event is evicted once this count is exceeded. `0` disables replay retention. |
| `MxGateway:Events:ReplayRetentionSeconds` | `300` | Maximum age, in seconds, of an event retained in the replay ring buffer. Entries older than this are evicted regardless of capacity. `0` disables age-based eviction. |
| `MxGateway:Events:MaxSparseArrayLength` | `1000000` | Maximum `total_length` a sparse-array write (`MxSparseArray`) may declare. A write above this cap is rejected with `InvalidArgument` before the full array is materialized, guarding against a single write forcing a multi-GB allocation. Must be between `1` and `Array.MaxLength`. |
`QueueCapacity` must be greater than zero; it bounds each per-subscriber event
`QueueCapacity` must be greater than zero and no greater than `int.MaxValue / 2`
(the validator rejects a larger value so the derived `2 ×` staging bound cannot
throw `OverflowException` at session creation); it bounds each per-subscriber event
channel fed by the session's single event pump, and — at `2 ×` — the worker
read loop's event staging channel, so a consumer that drains slower than its
worker produces faults the session at a fixed ceiling instead of growing gateway
@@ -184,7 +186,7 @@ events (a "gap") and must re-snapshot; whatever is still retained is replayed.
| `MxGateway:Dashboard:SnapshotIntervalMilliseconds` | `1000` | Dashboard snapshot refresh interval used by the snapshot SignalR hub and the pages that subscribe to it. |
| `MxGateway:Dashboard:RecentFaultLimit` | `100` | Maximum number of fault summaries projected into each dashboard snapshot. |
| `MxGateway:Dashboard:RecentSessionLimit` | `200` | Maximum number of session summaries projected into each dashboard snapshot. |
| `MxGateway:Dashboard:ShowTagValues` | `false` | Reserved display control for tag values. The dashboard does not show full tag values by default. |
| `MxGateway:Dashboard:ShowTagValues` | `false` | Controls whether tag values reach the dashboard's SignalR events hub mirror. `false` (default): `DashboardEventBroadcaster` blanks tag values from a deep-cloned copy of each `MxEvent` before it reaches any hub subscriber — event metadata (tag reference, quality, status, timestamps) still renders; see `docs/GatewayDashboardDesign.md`'s `EventsHub` row for the mechanism. Security-relevant because the per-session hub ACL that would scope a Viewer to specific sessions does not exist yet: with no per-session scoping, this redaction is currently the only thing standing between a low-trust Viewer and other sessions' tag values, so setting this `true` exposes every session's tag values to every authenticated dashboard viewer. The flag gates only the SignalR hub mirror — it does **not** cover the `/browse` live-value display, which remains a separate, still-open residual. |
| `MxGateway:Dashboard:GroupToRole` | _(empty)_ | LDAP group → dashboard role mapping. Keys are LDAP group names (short CN or full DN — leading-RDN match). Values must be `Admin` (read/write, API-key CRUD) or `Viewer` (read-only). A user whose LDAP groups don't intersect this map cannot sign in; with no mapping at all, only the loopback bypass admits anyone. |
| `MxGateway:Dashboard:DisableLogin` | `false` | Dev/test only. When `true`, replaces the cookie authentication handler with `DashboardAutoLoginAuthenticationHandler`, which auto-authenticates every dashboard request — including requests from remote browsers, not just loopback — as `AutoLoginUser` holding both `Administrator` and `Viewer` roles. No login form, LDAP bind, or cookie is involved. A loud one-time startup warning is logged. Differs from `AllowAnonymousLocalhost`: `DisableLogin` mints a real authenticated principal (so role-gated write affordances appear), whereas `AllowAnonymousLocalhost` satisfies the authorization requirement on loopback only without minting a principal (write affordances stay hidden). Never enable in production. **Production hard-stop (SEC-04):** when the host runs in the `Production` environment and `DisableLogin` is `true`, startup validation fails and the process aborts — the flag is only accepted outside Production, where the one-time startup warning still fires. |
| `MxGateway:Dashboard:AutoLoginUser` | `(null)` | Username stamped on the synthetic principal when `DisableLogin` is `true`. Default `(null)` — a null or blank value falls back to `multi-role`. Has no effect when `DisableLogin` is `false`. |
@@ -247,7 +249,7 @@ dev/test GLAuth posture (`glauth.md`), not a production posture.
| `MxGateway:Ldap:AllowInsecure` | `true` | Permits a plaintext bind. Must be `true` when `Transport` is `None`; set `false` (with `Ldaps`/`StartTls`) in production. |
| `MxGateway:Ldap:SearchBase` | `dc=zb,dc=local` | Search base DN. |
| `MxGateway:Ldap:ServiceAccountDn` | `cn=serviceaccount,dc=zb,dc=local` | Bind DN for the search account. |
| `MxGateway:Ldap:ServiceAccountPassword` | `${secret:ldap/mxgateway/bind}` | Search-account password. **No longer a committed plaintext value:** `appsettings.json` ships the reference `${secret:ldap/mxgateway/bind}`, which the pre-host `${secret:}` expander resolves at startup from the encrypted secrets store (the code-side design default is now blank, so a missing/unresolved value fails closed rather than falling back to a leaked credential). Seed the value once with `secret set ldap/mxgateway/bind <value>` (the store's master key must be present via `ZB_SECRETS_MASTER_KEY`); startup aborts with `SecretNotFoundException` if the secret is absent. An operator may instead override it directly with the env var `MxGateway__Ldap__ServiceAccountPassword` (double-underscore form) — a plain literal there is used as-is and the secret lookup is skipped. |
| `MxGateway:Ldap:ServiceAccountPassword` | `${secret:ldap/mxgateway/bind}` | Search-account password. **Never a committed plaintext value (SEC-36):** the shared GLAuth bind credential is supplied out-of-band through one of three channels, all binding to this key. **(1) Encrypted secrets store (shipped default):** `appsettings.json` ships the reference `${secret:ldap/mxgateway/bind}`, which the pre-host `${secret:}` expander resolves at startup from the encrypted secrets store (the code-side design default is blank, so a missing/unresolved value fails closed rather than falling back to a leaked credential). Seed it once with `secret set ldap/mxgateway/bind <value>` (the store's master key must be present via `ZB_SECRETS_MASTER_KEY`); startup aborts with `SecretNotFoundException` if the secret is absent. **(2) Deployed hosts — env var:** override directly with `MxGateway__Ldap__ServiceAccountPassword` (double-underscore form) in the NSSM service environment — a plain literal there is used as-is and the store lookup is skipped. **(3) Dev boxes — user-secrets:** `dotnet user-secrets set "MxGateway:Ldap:ServiceAccountPassword" <value>` (the server carries `<UserSecretsId>mxaccessgw-server</UserSecretsId>`; user-secrets load automatically in the Development environment and live under the user profile, outside the tree). The value comes from the GLAuth source of truth `scadaproj/infra/glauth/`, never from a repo file. **Rotation:** because the credential was historically committed, rotating it in `scadaproj/infra/glauth/` (and redeploying the shared GLAuth on `10.100.0.35`) is required — see `docs/runbooks/SEC-36-ldap-credential-rotation.md` for the cutover order. A blank/unresolved value fails startup validation with a message naming the two supported channels. |
| `MxGateway:Ldap:UserNameAttribute` | `cn` | LDAP attribute holding the login user name. |
| `MxGateway:Ldap:DisplayNameAttribute` | `cn` | LDAP attribute holding the display name. |
| `MxGateway:Ldap:GroupAttribute` | `memberOf` | LDAP attribute enumerating group membership (mapped to dashboard roles via `MxGateway:Dashboard:GroupToRole`). |
@@ -257,6 +259,20 @@ When LDAP is enabled, `Server`, `SearchBase`, `ServiceAccountDn`,
must be in range. See `glauth.md` for the shared dev instance and the
dev→production hardening posture.
### Production hard-stops key on the exact environment name (SEC-35)
Both production hard-stops above — `Dashboard:DisableLogin` and the plaintext
`Ldap:Transport=None` guard — fire only when `IHostEnvironment.IsProduction()` is
true, i.e. `ASPNETCORE_ENVIRONMENT` is unset (it defaults to `Production`, which
covers the NSSM-deployed hosts) or is set to the exact string `Production`. This
is ASP.NET Core's environment-name convention. A host launched under any other
name — `Staging`, `Prod`, or a custom label — keeps the permissive dev posture
and these guards do **not** fire, by design (inverting to "anything but
Development is production-like" would refuse to boot a legitimate permissive
staging rig, e.g. one pointed at the plaintext shared GLAuth). A production-like
deployment must therefore run with the literal `Production` environment name for
the hard-stops to apply.
## Secrets Master Key
`${secret:...}` tokens in configuration — currently just
@@ -363,9 +379,11 @@ model requires otherwise.
| `MxGateway:Security:ApiKeyLastUsedCoalesceSeconds` | `60` | Coalescing window, in seconds, for the `last_used_utc` write. The library verifier writes `last_used` on every successful verification; this bounds the write to at most one per key per window, so a hammered key does not churn the WAL. `0` forwards every write. Must be zero or greater. |
| `MxGateway:Security:LoginRateLimitPermitLimit` | `10` | Maximum `POST /auth/login` attempts permitted per remote IP within `LoginRateLimitWindowSeconds` before requests are rejected with HTTP 429. Throttles LDAP credential stuffing before the bind is relayed to the directory. Must be greater than zero. |
| `MxGateway:Security:LoginRateLimitWindowSeconds` | `60` | Fixed-window length, in seconds, for the per-IP login rate limit. Must be greater than zero. |
| `MxGateway:Security:ApiKeyFailureLimit` | `10` | Failed API-key verifications, per peer, within `ApiKeyFailureWindowSeconds` that trip the in-process short-circuit. Once tripped, the gRPC auth path rejects further attempts with `ResourceExhausted` **before** the store read; a successful verification resets the peer's counter. The peer is keyed on the presented key id (falling back to the transport address) so a single abusive credential behind a shared NAT is throttled without locking out co-located clients. Must be greater than zero. |
| `MxGateway:Security:ApiKeyFailureWindowSeconds` | `60` | Sliding-window length, in seconds, over which API-key verification failures are counted per peer. Must be greater than zero. |
| `MxGateway:Security:ApiKeyFailureTrackedPeers` | `4096` | Maximum distinct peers tracked by the failure counter (a bounded LRU) so a spray of unique peer keys cannot grow memory without limit. Must be greater than zero. |
| `MxGateway:Security:ApiKeyFailureLimit` | `10` | Failed API-key verifications, per `(transport peer, key id)` partition, within `ApiKeyFailureWindowSeconds` that trip the in-process short-circuit. Once tripped, the gRPC auth path rejects further attempts from that partition with `ResourceExhausted` **before** the store read — except for the probe admitted every `ApiKeyFailureProbeIntervalSeconds` — and a successful verification resets the partition. The partition always includes the sender's transport address: key ids are public (they ride in every token and are listed on the dashboard), so keying on the key id alone let any peer deny a key to its legitimate holder. Must be greater than zero. |
| `MxGateway:Security:ApiKeyFailureWindowSeconds` | `60` | Sliding-window length, in seconds, over which API-key verification failures are counted, for both the per-partition and the per-key-id aggregate layer. Must be greater than zero. |
| `MxGateway:Security:ApiKeyFailureAggregateLimit` | `30` | Failed verifications for one key id counted across **all** transport peers within `ApiKeyFailureWindowSeconds` before that key id enters probe mode. This second layer bounds a distributed or source-rotating sprayer that never trips any single `(peer, key id)` partition. `0` disables the aggregate layer, leaving only per-partition counting. Must be zero or greater. |
| `MxGateway:Security:ApiKeyFailureProbeIntervalSeconds` | `5` | Minimum interval, in seconds, between probe admissions for an over-limit partition or key-id aggregate. An over-limit state is a valve rather than a wall: one request per interval reaches the real verifier — exactly one, even when a burst arrives together at the interval boundary — so the holder of the correct secret always gets through and clears the state, while everything else is still refused before the store read. `0` blocks absolutely instead — **not recommended**, because an unauthenticated peer can then deny the key to its holder for the whole window. Must be zero or greater. |
| `MxGateway:Security:ApiKeyFailureTrackedPeers` | `4096` | Maximum distinct partitions tracked by the failure counter (a bounded LRU) so a spray of unique tokens cannot grow memory without limit. It cannot be used to flush an active block either: only a validly shaped `mxgw_<keyId>_<secret>` token mints a key-id partition (everything else lands on the sender's transport-peer partition), each address may mint at most 32 key-id partitions before the overflow collapses onto that address's fallback partition, and eviction prefers fully expired windows, never removing an over-limit partition until the map exceeds twice this cap. Must be greater than zero. |
## Galaxy Options
@@ -375,7 +393,7 @@ model requires otherwise.
| `MxGateway:Galaxy:CommandTimeoutSeconds` | `60` | Per-command SQL timeout for all Galaxy browse RPCs. |
| `MxGateway:Galaxy:DashboardRefreshIntervalSeconds` | `30` | Interval between background refreshes of the dashboard Galaxy summary cache. SQL is hit at most once per interval regardless of dashboard render rate. |
| `MxGateway:Galaxy:PersistSnapshot` | `true` | Persists the latest successful Galaxy browse dataset to disk. When `true`, the cache reloads that snapshot at startup so clients can still browse last-known data while the Galaxy database is unreachable. The restored data is served with `Stale` status until a live query confirms it. |
| `MxGateway:Galaxy:SnapshotCachePath` | `C:\ProgramData\MxGateway\galaxy-snapshot.json` | File path for the persisted Galaxy browse snapshot. Ignored when `PersistSnapshot` is `false`. The snapshot is written atomically (temp file plus rename). Set an **absolute** path — this option is bound by the shared `ZB.MOM.WW.GalaxyRepository` package (not by `GatewayOptions`), so the gateway validator does not enforce rooting on it; a relative value would resolve against the launch working directory (SEC-01). |
| `MxGateway:Galaxy:SnapshotCachePath` | derived from `CommonApplicationData` (`C:\ProgramData\MxGateway\galaxy-snapshot.json` on Windows, `/usr/share/MxGateway/galaxy-snapshot.json` or the container equivalent elsewhere) | File path for the persisted Galaxy browse snapshot. Ignored when `PersistSnapshot` is `false`. The snapshot is written atomically (temp file plus rename). `appsettings.json` no longer ships an explicit value (SEC-33): the option is bound by the shared `ZB.MOM.WW.GalaxyRepository` package, so the gateway supplies the `CommonApplicationData`-derived default when the bound value is blank and registers `GalaxyRepositoryOptionsValidator` to enforce that — when `PersistSnapshot` is `true` — the path is non-blank, valid, and **rooted on the host running the gateway** (`Path.IsPathRooted`, current OS). A bare filename or a foreign-platform literal fails startup instead of resolving against the launch working directory (SEC-01, SEC-33). |
See [Galaxy Repository Browse](./GalaxyRepository.md) for the RPC surface and
behavior.
+52 -16
View File
@@ -241,14 +241,15 @@ an unreachable LDAP server is absorbed into a failed result rather than throwing
pre-host secrets expander, which this suite's bare `ConfigurationBuilder`
does not run). Before running the live LDAP suite, set
`MxGateway__Ldap__ServiceAccountPassword` to the real GLAuth service-account
password (dev value `serviceaccount123` for the shared GLAuth) so the suite
binds with the real password instead of the literal token.
password so the suite binds with the real password instead of the literal
token. Obtain the current value from the GLAuth source of truth
`scadaproj/infra/glauth/` (per `glauth.md`); it is not committed here.
Run the LDAP live tests explicitly:
```bash
$env:MXGATEWAY_RUN_LIVE_LDAP_TESTS = "1"
$env:MxGateway__Ldap__ServiceAccountPassword = "serviceaccount123"
$env:MxGateway__Ldap__ServiceAccountPassword = "<service-account-password>"
dotnet test src/ZB.MOM.WW.MxGateway.IntegrationTests/ZB.MOM.WW.MxGateway.IntegrationTests.csproj --filter FullyQualifiedName~DashboardLdapLiveTests
```
@@ -428,10 +429,13 @@ runtime because the x86 Worker cannot build on Linux:
gateway fake-worker tests, and builds/tests the clients that run on Linux: .NET client
build, Go (`gofmt` + `go build` + `go test`), Rust (`cargo fmt --check` + `cargo test`
+ `cargo clippy -D warnings`), and Python (`pytest`).
- **`java`** (Linux, JDK 17) — `gradle test`. The protobuf gradle plugin rewrites
`MxaccessGateway.java` with spurious protobuf-runtime-version churn on every build, so
when no `.proto` changed the job reverts that one file (`git checkout`) before asserting
the generated tree is clean. The dev Mac has no JRE, so Java verification is CI-only.
- **`java`** (Linux, JDK 17) — `gradle test`, then `git diff --exit-code` over the generated
tree. The grpc/protobuf toolchain is pinned (`clients/java/build.gradle`), so a regeneration
is byte-identical to the committed single-file aggregates modulo real `.proto` changes; the
git-diff is therefore a true drift gate that now catches message-level proto drift in the Java
client (IPC-24 deleted the old unconditional churn-revert step, which masked exactly that
class). The dev Mac has a Homebrew JDK 17, so `gradle generateProto` can be run there to refresh
the Java aggregates when a `.proto` changes.
- **`windows-x86`** (Linux runner, per push/PR) — builds the **x86 / net48 Worker and
Worker.Tests**, which are Windows-only and out of scope for the Linux jobs. It runs on a
Linux runner that always schedules and SSHes to windev (`10.100.0.48`), where
@@ -449,21 +453,53 @@ runtime because the x86 Worker cannot build on Linux:
`xUnit1030`). Gated `if: github.event_name == 'schedule'`, so it never gates a push. On
failure it opens a Gitea issue via the Actions token, since nobody watches the Actions page.
The freshness guard `scripts/check-codegen.ps1` fails the build when the committed
client descriptor set or the C# `Generated/` no longer matches the current `.proto`
sources — the codegen drift class this repo has hit repeatedly (stale client
descriptors, net48 `CS0246` on unregenerated protos). The **primary** guard for the
### Runner capacity is shared and finite
All CI runs on a co-located `gitea-runner` container on docker host `10.100.0.35` with
`maxParallel=1`, and that runner instance is **shared across repos** — it interleaves
`dohertj2/mxaccessgw` and `dohertj2/lmxopcua` jobs on the single slot rather than being
scoped to this repo (`GET /repos/dohertj2/mxaccessgw/actions/runners` returns
`total_count: 0`; the runner is registered at the instance level). Every job in a run
(`portable`, `java`, `windows-x86`) executes serially, so queue latency is additive within
a run and an active `lmxopcua` run blocks `mxaccessgw` entirely — expect ~20–30 minutes of
queue depth under cross-repo contention, not a stuck pipeline. This Gitea version (1.26)
also exposes **no run cancel or delete via the API** (`POST .../actions/runs/{id}/cancel`
returns 404, `DELETE .../actions/runs/{id}` returns 400), so a superseded or hung run cannot
be cleared and holds the slot until it finishes or times out. See
`docs/runbooks/TST-30-second-ci-runner.md` for the operator runbook that registers a second
runner to relieve this; until that lands, treat single-runner contention as expected, not a
CI outage.
When queue depth (or the missing-cancel reality) makes waiting impractical, verify a
specific commit out of band instead of waiting behind the queue: run
`CI_SHA=<sha> scripts/ci/run-windev-ci.sh <build|test|live>` from a machine with SSH access
to windev (the same script the SSH-driven `windows-x86`/`nightly-windev` jobs use — see
`scripts/ci/README.md`), or fall back to the manual windev worktree procedure below. This is
the same escape hatch used when the windev tier itself is down — TST-30 generalizes it from
"tier down" to "runner contended": either way, a stuck or slow shared runner should not
block verifying a commit.
The freshness guard `scripts/check-codegen.ps1` runs four checks and fails the build when the
committed client descriptor set (Check 1), the C# `Generated/` (Check 2), the Rust vendored
protos (Check 3), or the Go/Python client bindings (Check 4, IPC-25) no longer match the current
`.proto` sources — the codegen drift class this repo has hit repeatedly (stale client
descriptors, net48 `CS0246` on unregenerated protos, silently stale Go/Python worker bindings).
Check 4 regenerates the Go and Python bindings with their pinned generators (`protoc-gen-go`
v1.36.11 / `protoc-gen-go-grpc` 1.6.2, `grpcio-tools` 1.80.0) and fails on any diff; a missing
generator fails the check rather than skipping it. The **primary** guard for the
"regenerate and commit `Generated/`" rule is that regeneration diff in the `portable` job;
the `windows-x86` net48 compile is the **secondary** guard (a stale `Generated/` also breaks
the x86 build with `CS0246`). See [Client Proto Generation](./ClientProtoGeneration.md) and
[Contracts](./Contracts.md).
If the SSH-driven Windows tier is unavailable for infrastructure reasons (windev down, CI
key/secret rotation in flight), fall back to the manual windev worktree procedure as a
degraded mode: on windev, fast-forward an isolated `origin/main` worktree under `C:\build`
(never the dirty Desktop checkout), then run the x86 Worker build and `Worker.Tests`
(`-p:Platform=x86`) there by hand. Do this per merge for worker-touching changes until the
`windows-x86` job is green again.
key/secret rotation in flight) **or** the shared Gitea runner is contended and the queue is
impractical to wait behind (see "Runner capacity is shared and finite" above), fall back to
the manual windev worktree procedure as a degraded mode: on windev, fast-forward an isolated
`origin/main` worktree under `C:\build` (never the dirty Desktop checkout), then run the x86
Worker build and `Worker.Tests` (`-p:Platform=x86`) there by hand. Do this per merge for
worker-touching changes until the `windows-x86` job is green again (tier-down case) or the
queue clears (contention case).
## Related Documentation
+3 -1
View File
@@ -82,6 +82,8 @@ return mapper.MapCommandReply(workerReply);
Carrying the enqueue timestamp into the worker layer is what lets queue-wait time be measured separately from worker-side execution time when troubleshooting timeouts.
An accepted gRPC command payload can still be too large for the worker pipe: the envelope built around it must fit `MxGateway:Worker:MaxMessageBytes`, which is validated at startup to sit at least a fixed envelope-overhead reserve above `MaxGrpcMessageBytes` (see the headroom rule in [Gateway Configuration](./GatewayConfiguration.md)) so this should not occur for a conformant payload, but if it does, `WorkerClient` raises `WorkerClientException(CommandTooLarge)` and `Invoke` reports `ResourceExhausted` for that command — the session is not faulted, so a client can retry with a smaller payload without reopening the session.
### `StreamEvents`
`StreamEvents` is a server-streaming RPC. The handler delegates the full pipeline to `IEventStreamService` and just forwards each `MxEvent` onto the response stream. Keeping the channel and producer/consumer machinery out of the handler means cancellation, exception mapping, and metric bookkeeping live in one place.
@@ -247,7 +249,7 @@ StatusCode statusCode = exception.ErrorCode switch
};
```
`WorkerClientException` follows the same pattern: `CommandTimeout` becomes `DeadlineExceeded`, `GatewayShutdown` becomes `Cancelled`, `InvalidState` becomes `FailedPrecondition`, `ProtocolViolation` becomes `Internal`, and unmapped codes fall through to `Unavailable`.
`WorkerClientException` follows the same pattern: `CommandTimeout` becomes `DeadlineExceeded`, `GatewayShutdown` becomes `Cancelled`, `InvalidState` becomes `FailedPrecondition`, `ProtocolViolation` becomes `Internal`, `CommandTooLarge` becomes `ResourceExhausted`, and unmapped codes fall through to `Unavailable`.
## Event Streaming Model
+94 -18
View File
@@ -378,6 +378,20 @@ If event conversion throws, catch it inside the event handler, record a
structured `WorkerFault`, and keep the worker alive only if the fault policy
allows it.
The event drain loop streams queued events as `WorkerEvent` frames. A single
event whose envelope exceeds the negotiated frame maximum is **undeliverable end
to end** — the pipe maximum sits only the envelope-overhead reserve above the
public gRPC cap, so a frame the pipe rejects would also be rejected on the
client-facing stream. The session therefore faults on it rather than dropping it
(a silent drop makes the event stream unfaithful, and a synthesized placeholder
is barred by the no-synthesized-events rule), but the death is structured: the
worker logs the event's identity — family, handles, worker sequence, and sizes,
never the value — writes a `WorkerFault` with category `ProtocolViolation` and
command method `EventDrain` carrying the same identity, and only then exits.
Operator remediation is configuration: raise `MxGateway:Worker:MaxMessageBytes`
for that workload. Other per-frame rejection codes keep their previous behavior
because they indicate worker bugs, not workload size.
## Command Queue
The pipe reader converts `WorkerCommand` messages into `StaCommand` entries.
@@ -440,6 +454,29 @@ Diagnostics:
- `DrainEvents`
- `ShutdownWorker`
`DrainEvents` is answered on the message-loop thread, not the STA, and its reply
is bounded on **two** axes because no diagnostics command may be session-fatal:
- **Count** — `GatewayContractInfo.MaxDrainEventsPerCommand` (10,000) is the
single home of the ceiling, shared by the gateway's request validator (which
rejects a larger `max_events` at the public boundary) and this worker clamp
(which also interprets `max_events = 0`, "as many as available").
- **Bytes** — the count cap alone is not sufficient: byte-heavy events (large
string or array `MxValue`s) overshoot the negotiated frame maximum long before
10,000 events. The drain is therefore byte-budgeted against the negotiated
maximum less a 64 KiB envelope/reply-wrapper reserve, and the size decision
happens inside the event queue's lock, so an event is dequeued only once it is
known to fit. An event that does not fit stays at the head of the queue and is
never lost.
Truncation is reported in the reply's existing `DiagnosticMessage`
("N events returned, M remain; repeat DrainEvents for the rest") rather than in a
new field, so the contract is unchanged and callers drain iteratively until a
reply comes back empty. In the degenerate case where the head event alone exceeds
the budget, the reply returns whatever fit before it (possibly nothing) and names
the blocked event's worker sequence so an operator can find the offending tag;
that event needs a larger `MxGateway:Worker:MaxMessageBytes` to move at all.
Implement method-specific dispatch instead of a generic string method invoker.
Parity tests need stable command-specific request and reply shapes.
@@ -601,27 +638,42 @@ Rules:
## Outbound Queues
The worker should use bounded outbound queues for replies, events, heartbeats,
and faults.
`WorkerFrameWriter` is a two-class cooperative priority scheduler, not the
five-level queue an earlier design draft called for. Every outbound frame is
enqueued at one of two `WorkerFrameWritePriority` values:
Priority order when writing:
- **Control** — hello, ready, command replies, faults, heartbeats, shutdown
acknowledgements.
- **Event** — MXAccess events drained from the queue.
1. faults,
2. command replies,
3. shutdown acknowledgements,
4. heartbeats,
5. events.
A caller enqueues its frame under a lock, then contends for a single write
lock; whichever caller wins drains **every** currently-queued frame before
releasing the lock, so a reply, fault, or heartbeat enqueued while a large
event backlog is draining still gets written on the very next drain pass
rather than waiting behind it. Within a class the queue is strict FIFO — a
fault does not jump ahead of an already-queued heartbeat or reply. This
collapses the originally-specified five-level order (faults > replies >
shutdown acks > heartbeats > events) into two classes; the decision is
deliberate, not a shortfall: the control queue is shallow in practice (faults
and shutdown acks are rare, replies and heartbeats are the steady traffic), so
the FIFO delay within it is bounded, and a two-class scheduler is simpler to
reason about and test than a five-level one for that same bound. See
`docs/WorkerFrameProtocol.md`'s "Write scheduling and sequencing" section for
the wire-level detail (sequence stamping, per-frame vs. stream-failure
semantics, flush coalescing).
Event overflow policy defaults to fail-fast for parity testing. If the event
queue fills:
1. Capture overflow metrics.
2. Send `WorkerFault` if possible.
3. Stop accepting new commands.
4. Let the gateway close or kill the worker.
Production coalescing may be added later, but it must be explicit and tested.
Do not drop or coalesce events in v1.
Event overflow policy is fail-fast, not "stop accepting new commands and let
the gateway close or kill the worker." When `MxAccessEventQueue.Enqueue` finds
the queue full, it throws and the queue self-records a `WorkerFault` with
category `QueueOverflow`. The event drain loop's next pass observes the fault
through `DrainFault()`, writes the fault frame — a Control-priority frame, so
it is not stuck behind a queued event backlog — and then throws to unwind
`RunAsync`: the worker process exits rather than continuing in a state where
events are being silently lost. The exit currently uses the generic
`WorkerExitCode.UnexpectedFailure` rather than a dedicated overflow code; a
distinct exit code remains open (tracked separately). Do not drop or coalesce
events to avoid this exit — that is explicitly out of scope for v1. Production
coalescing may be added later, but it must be explicit and tested.
The gateway side of the event path is bounded to match. `WorkerClient` buffers
inbound events in a bounded consumer channel plus a staging channel bounded at
@@ -634,6 +686,13 @@ consumer that merely drains slower than this worker produces (the staging bound,
metric `QueueOverflow("worker-event-staging")`). See
`docs/GatewayProcessDesign.md`. A worker that outruns its consumer therefore
dies loudly rather than growing gateway memory silently.
No control reply is session-fatal on size. Reply builders size their payloads
against the negotiated frame maximum, and the two reply-write seams (the
control-command path and the STA command path) additionally catch a
`MessageTooLarge` per-frame rejection and answer that correlation with a small
`InvalidRequest` reply instead of unwinding the session. Oversized *event*
frames keep the opposite policy — see Event Sink — because an event above the
frame maximum cannot be delivered to the client at all.
## Heartbeat And Watchdog
@@ -688,6 +747,23 @@ heartbeat fields until dedicated thresholds own those warnings. The worker
reports stale STA activity, but the gateway owns the final kill decision
through its existing heartbeat and worker lifecycle policy.
The alarm poll runs outside the command dispatcher — `RunAlarmPollLoopAsync`
invokes `PollOnce` directly on the STA rather than through
`StaCommandDispatcher`, so it does not inflate `PendingCommandCount` or perturb
dispatch ordering for real gateway commands. Because it is not a dispatched
command it has no `CurrentCommandCorrelationId`, so a healthy-but-slow poll (a
large `GetXmlCurrentAlarms2` against a busy provider) blocking the STA past
`HeartbeatGrace` would otherwise fault a healthy session at 15 s while a
dispatched command gets the 75 s ceiling. To close that asymmetry (WRK-27) the
poll advertises itself on the heartbeat snapshot's `StaCallInProgress` flag —
set on the STA thread for exactly the span of the COM call — and the watchdog
suppression honors that flag alongside `CurrentCommandCorrelationId`. The poll
therefore receives the same grace-to-ceiling treatment as a dispatched command:
suppressed up to `HeartbeatStuckCeiling`, faulted past it (a poll that blocks
the STA more than 75 s without pumping *should* fault — that is the ceiling's
contract). The flag is named generically so any future non-dispatcher STA work
reuses it.
The in-flight-command suppression itself is bounded by
`WorkerPipeSessionOptions.HeartbeatStuckCeiling` (default 75 seconds = 5 ×
`HeartbeatGrace`). The motivating case for the suppression is a legitimately
+105
View File
@@ -29,6 +29,40 @@ default. A `max_frame_bytes` of 0 (an older gateway that never set the field)
means "use the worker's built-in default". This keeps both ends framing to the
same limit rather than depending on matched compile-time constants.
The worker accepts a negotiated value in the closed range [1024, 256 MiB]
(`MinNegotiableFrameBytes` .. `MaxNegotiableFrameBytes`); 0 keeps the default.
A value outside that range is rejected at the handshake with a fault frame
rather than adopted, because a nonsensical maximum — a gateway bug or a
foreign/old peer — would otherwise leave a session that handshakes cleanly and
then fails every subsequent frame with per-frame size errors, the worst
diagnostic shape for an operator. The 1024-byte floor matches the gateway's own
`GatewayOptionsValidator.MinimumMaxMessageBytes`, so the worker never rejects a
value the gateway's validator accepts as legal configuration, and 1024 still
guarantees hellos, heartbeats, acks, and faults fit.
Every worker-to-gateway frame must serialize within this limit, control replies
included, so reply builders truncate to fit rather than emit a frame the writer
will reject. `WorkerPipeSession` pre-sizes a `DrainEvents` reply below the
negotiated maximum (less a fixed envelope/reply-wrapper reserve) and reports the
truncation in the reply's `DiagnosticMessage`; the caller contract is to repeat
`DrainEvents` until it returns an empty reply. Should a reply still overshoot,
`MessageTooLarge` at the reply-write seam is answered with a small
`InvalidRequest` reply for that correlation, not with session teardown — no
diagnostics command may kill a session.
An oversized *event* frame is the deliberate exception. Such an event is
undeliverable end to end (the pipe maximum sits only the envelope-overhead
reserve above the public gRPC cap), so the session faults: the worker logs the
event's identity and sizes — never its value — writes a `WorkerFault` with
category `ProtocolViolation` and command method `EventDrain`, then exits.
Remediation is raising `MxGateway:Worker:MaxMessageBytes` for that workload.
A per-frame rejection does not consume an envelope `sequence`. The writer stamps
a candidate sequence, runs the empty-payload and size checks against the stamped
envelope, and commits the counter only immediately before the stream write, so
the sequences observed on the wire stay contiguous across rejections and an
operator reading a pipe capture never sees a phantom gap.
## Envelope Validation
`WorkerFrameReader` and `WorkerFrameWriter` validate each envelope against the
@@ -42,6 +76,77 @@ Protocol violations throw `WorkerFrameProtocolException` with a
`WorkerFrameProtocolErrorCode` so callers can distinguish malformed frames,
oversized frames, protocol version mismatches, and session mismatches.
## Write Scheduling And Sequencing
This section covers write scheduling (priority classes, enqueue-then-contend,
flush coalescing) and sequencing (write-time stamping) together, because both
are properties of the same single write lock.
`WorkerFrameWriter` is a two-class cooperative priority scheduler
(`WorkerFrameWritePriority.Control` and `.Event`), not a strict per-kind
priority order. A caller enqueues its frame into the control or event queue
under a lock, then contends for a single write lock; whichever caller wins
drains every frame queued at that moment, control frames first and each class
in FIFO order, so a command reply, fault, heartbeat, or shutdown
acknowledgement is never delayed behind a backlog of queued events. Priority
only reorders *which frame writes next* — it does not affect the sequence
value a frame receives (see below), so a caller cannot infer priority class
from the wire sequence.
The envelope `Sequence` is stamped by the draining lock-holder at the actual
moment of writing, not when the frame is enqueued, so the on-wire order and
the stamped sequence always agree regardless of caller concurrency or
priority reordering. Stamping uses peek-stamp-commit: a candidate sequence is
assigned and the frame is validated (size, non-empty payload) against that
stamped value, but the counter is committed only immediately before the
stream write. A per-frame rejection therefore leaves the counter untouched —
the next accepted frame reuses the candidate number, so the wire sequence
stays contiguous across rejections and an operator reading a pipe capture
never sees a phantom gap from a rejected frame.
Two failure shapes are distinguished during a drain pass:
- **Per-frame rejection** (`InvalidEnvelope`, `MessageTooLarge`,
`ProtocolVersionMismatch`, `SessionMismatch`) is specific to the one frame
that failed validation or sizing. Nothing was written for it, so it fails
only that frame's completion and draining continues with the next queued
frame.
- **Stream failure** (anything else — a broken pipe, an I/O error) means the
underlying stream itself is no longer trustworthy. It fails the frame that
triggered it, every frame already written this batch but not yet flushed,
and every frame still queued, then stops draining entirely so no caller
waits forever on a stream that will not recover.
Flushes are coalesced across a drained batch: each frame in the batch is
written to the stream without an individual flush, then one `FlushAsync`
runs after the whole batch, and only then does every successfully-written
frame's completion resolve — so a caller's `WriteAsync` still does not
complete until its bytes are both written *and* flushed, but a batch that
happened to contain several queued frames pays one flush instead of one per
frame. The event drain loop (`WorkerPipeSession.RunEventDrainLoopAsync`)
submits a whole drained event batch through `WriteBatchAsync`, which enqueues
every frame under one `_gate` acquisition, takes the write lock once, and
drains them together, so a burst of N events costs one flush rather than N —
the coalescing the batch machinery was built for now engages on the event hot
path, not only when independent producers happen to queue behind a blocked
write. Intra-batch order is preserved (FIFO enqueue under one lock), and a
concurrently queued control frame is still drained ahead of the batch. A
per-frame rejection inside a batch (for example one oversized event) surfaces
from the batch's awaited completions as that frame's
`WorkerFrameProtocolException`; the remaining completions are still observed
so none faults unobserved.
Cancellation of a `WriteAsync`/`WriteBatchAsync` call that is still waiting
for the write lock when its token fires tombstones the queued frame: the
cancelled caller marks its frame under `_gate`, and the draining lock-holder's
`DequeueNext` skips any tombstoned frame, so a cancelled call is guaranteed
never to reach the wire — *unless* a lock-holder has already claimed the frame
to write it. Claiming and cancelling are interlocked under `_gate`, so exactly
one wins; a frame already claimed is mid-write and can no longer be recalled,
so the caller observes `OperationCanceledException` while that one frame still
reaches the wire. That residual window is by design: blocking the canceller
behind the very write it is abandoning would defeat the point of cancellation.
## Verification
The frame protocol lives in `ZB.MOM.WW.MxGateway.Worker.Ipc` (`WorkerFrameReader`,
@@ -0,0 +1,119 @@
# SEC-36 — LDAP Service-Account Credential Rotation (Operator Runbook)
Operator steps to rotate the shared GLAuth service-account password after the repo-side
removal landed (SEC-36). The repo change (removal of the committed value, the two supported
secret channels, and this runbook) is already merged; the live rotation below is the
load-bearing half and is yours to execute.
> **Never put the old or new password in this repo, in a commit, in a chat, or in this file.**
> The value lives only in the GLAuth source of truth and in each host's out-of-band channel.
## Why
The dev GLAuth service-account password (`cn=serviceaccount,dc=zb,dc=local`) was historically
committed to this repo. Removal alone is insufficient — the old value is permanently recoverable
from git history — so **rotation is required**. Until the shared GLAuth on `10.100.0.35:3893`
stops honoring the old value, the repo history discloses a live directory account with LDAP
search capability over `dc=zb,dc=local`.
## Where the credential lives now (three channels, all bind `MxGateway:Ldap:ServiceAccountPassword`)
- **Source of truth:** `scadaproj/infra/glauth/config.toml` on host `10.100.0.35` (the `serviceaccount`
user's `passsha256`). `scadaproj` is a shared monorepo — stage only the explicit glauth paths.
- **Encrypted secrets store (gateway default):** `appsettings.json` ships `${secret:ldap/mxgateway/bind}`,
resolved from the local encrypted store (seed with `secret set ldap/mxgateway/bind <value>`).
- **Deployed hosts:** env var `MxGateway__Ldap__ServiceAccountPassword` in the NSSM service environment.
- **Dev boxes:** `dotnet user-secrets set "MxGateway:Ldap:ServiceAccountPassword" <value>`
(the server carries `<UserSecretsId>mxaccessgw-server</UserSecretsId>`).
See `docs/GatewayConfiguration.md` (the `ServiceAccountPassword` row) and `glauth.md`.
## Preconditions
- SSH access to the GLAuth docker host `10.100.0.35` and to the deployed gateway host(s).
- Write access to `scadaproj/infra/glauth/`.
- Know which deployed hosts run LDAP-backed dashboard login:
- **`10.100.0.48`** (`windev`) — primary; verify here.
- **`wonder-app-vd03`** — its dashboard is disabled. **Check `MxGateway:Ldap:Enabled` there first.**
If LDAP is disabled (`Enabled=false`), it has nothing to bind and needs no env var — skip it.
- A generated replacement secret (see step 1). Generate the `passsha256` per `glauth.md`
("Generate `passsha256` from a plaintext password").
## Cutover order
Follow this order so no window opens where the deployed dashboard cannot bind. **Do not rotate
GLAuth before the deployed hosts already carry the new value.**
1. **Generate the new secret in `scadaproj/infra/glauth/`.** Pick a new password, compute its
`passsha256`, and stage the change to the `serviceaccount` user in `config.toml` (do not
`docker compose up` yet — the directory must keep honoring the OLD value until the deployed
hosts carry the NEW one).
2. **Pre-stage the NEW value on every LDAP-enabled deployed host** via the env-var channel, so the
host is ready the instant GLAuth flips:
```powershell
nssm get MxAccessGw AppEnvironmentExtra
nssm set MxAccessGw AppEnvironmentExtra MxGateway__Ldap__ServiceAccountPassword=<new-value>
# restart the service so the new environment is picked up
nssm restart MxAccessGw
```
Do this on `10.100.0.48`, and on `wonder-app-vd03` **only if** `MxGateway:Ldap:Enabled=true` there.
(Alternatively seed the encrypted store with `secret set ldap/mxgateway/bind <new-value>`; the
env var overrides the store and is the simplest per-host mechanism.)
At this moment the deployed host holds the NEW value but GLAuth still honors the OLD one — binds
still fail closed against the old directory, which is expected and brief; proceed immediately.
3. **Rotate GLAuth on `10.100.0.35`** to honor the new value:
```bash
ssh 10.100.0.35
cd ~/Desktop/scadaproj/infra/glauth
docker compose up -d --force-recreate
docker compose logs -f # confirm clean startup, no TOML parse error
```
4. **Verify dashboard login on the deployed host(s).** Browse to the gateway dashboard on
`10.100.0.48` and log in as `multi-role` / `password` (Administrator) — a successful login proves
the search bind used the new service-account credential end-to-end. If `wonder-app-vd03` runs
LDAP, verify it too; if its dashboard/LDAP is disabled, no check is needed.
5. **The repo change is already landed** (removal of the committed value, `<UserSecretsId>`, the
validator message naming the two channels, and doc/scrub updates). Nothing more to commit for
the cutover.
6. **Developers set user-secrets on next pull.** After pulling, a dev box with no secret configured
will fail startup with a validation message naming the exact command. One-time per machine:
```bash
dotnet user-secrets set "MxGateway:Ldap:ServiceAccountPassword" <new-value>
```
(value from `scadaproj/infra/glauth/`, never from a repo file).
## Verifying the rotation
- **Primary:** dashboard `/login` as `multi-role` on `10.100.0.48` succeeds (step 4).
- **`wonder-app-vd03`:** only if `MxGateway:Ldap:Enabled=true`; otherwise no action.
- **Live-LDAP integration tests** (opt-in, only where the GLAuth instance is reachable):
```bash
$env:MXGATEWAY_RUN_LIVE_LDAP_TESTS = "1"
$env:MxGateway__Ldap__ServiceAccountPassword = "<new-value>" # shell env only, never committed
dotnet test src/ZB.MOM.WW.MxGateway.IntegrationTests/ZB.MOM.WW.MxGateway.IntegrationTests.csproj `
--filter FullyQualifiedName~DashboardLdapLiveTests
```
A green `DashboardLdapLiveTests` run confirms the new credential binds and searches. Where GLAuth
is unreachable, document the suite as skipped per the `docs/GatewayTesting.md` opt-in matrix.
- **Old value is dead:** after step 3, a bind with the old password must fail. Do not test this from
a shared-NAT box — GLAuth's 3-fail / 10-minute per-IP lockout can lock the whole office.
## Rollback
If dashboard login breaks after step 3, restore the previous `passsha256` in
`scadaproj/infra/glauth/config.toml`, `docker compose up -d --force-recreate`, and re-point the
deployed hosts' env var / store back to the previous value. Because the deployed hosts were
pre-staged in step 2, the exposure window is only steps 2→4.
## Done criteria
- GLAuth on `10.100.0.35` honors only the new value.
- Every LDAP-enabled deployed host binds with the new value (dashboard login verified).
- The source of truth `scadaproj/infra/glauth/config.toml` carries the new `passsha256`.
- No repo file (this one included) contains the old or new value.
- The SEC-36 tracker rows are `Done` with this runbook cited for the operator action.
+113
View File
@@ -0,0 +1,113 @@
# TST-30 — Register A Second CI Runner (Operator Runbook)
Operator steps to relieve the single shared Gitea Actions runner that CI depends on. The
repo-side half of TST-30 (documenting the shared-runner/no-cancel reality and the
`run-windev-ci.sh` bypass) is already landed in `docs/GatewayTesting.md`; registering the
second runner below is infrastructure work outside this repo's tree and is yours to execute.
## Why
All CI for this repo runs on one co-located `gitea-runner` container on docker host
`10.100.0.35` with `maxParallel=1`. That runner is registered at the **instance** level, not
scoped to this repo (`GET /repos/dohertj2/mxaccessgw/actions/runners` returns
`total_count: 0`), so it is shared with `dohertj2/lmxopcua` and every job in every run across
both repos executes serially on the single slot. A `mxaccessgw` push fans out to `portable`,
`java`, `windows-x86`, and an active `lmxopcua` run blocks all of them — queue depth of
~20–30 minutes was observed during TST-25 acceptance under cross-repo contention. Gitea 1.26
also exposes **no run cancel or delete via the API**
(`POST .../actions/runs/{id}/cancel` → 404, `DELETE .../actions/runs/{id}` → 400), so a
superseded or hung run cannot be cleared and holds the slot until it finishes or times out.
This is not a correctness problem — every job still reports accurately — but it undercuts the
fast-feedback purpose of the TST-25 Windows tier and makes CI fragile to a single host: if
`10.100.0.35` wedges or goes down, CI for both repos stops with no failover.
## Options (cheapest first)
- **(a) Register a second `act_runner` instance on `10.100.0.35` — recommended.** The host
already runs `gitea-runner`; add a second `act_runner` container (or raise the existing
runner's `maxParallel` where the docker-in-docker/resource budget allows) so at least two
jobs run concurrently. Cheapest change, and it keeps the runner co-located on the
`container.network: traefik` network that resolves `gitea:3000` — the property TST-03
depended on. **Use the same `container.network: traefik` config as the existing runner.**
- **(b) Dedicate a labelled runner to `mxaccessgw`.** Cleaner isolation — `lmxopcua` load
never blocks this repo — but needs label wiring: register the new runner with a distinct
label (e.g. `mxgw`) and change `.gitea/workflows/ci.yml`'s `runs-on:` for this repo's jobs
to gate on that label (e.g. `runs-on: [ubuntu-latest, mxgw]`). Only do this if (a) proves
insufficient — it is more moving parts for the same throughput gain, and it means `ci.yml`
changes, which is out of scope for the doc-only half of TST-30.
- **(c) Put the runner on windev / a second host — rejected as the primary fix.** windev is
the Windows build target (`10.100.0.48`), not a CI host, and co-locating a Linux runner
there loses the `gitea:3000` name resolution TST-03 relies on. Only consider if
`10.100.0.35` genuinely runs out of capacity for a second instance.
Default to **(a)**. Escalate to (b) only if `lmxopcua` contention persists after a second
instance is online (i.e., (a) is not sufficient because the two repos' combined load exceeds
two slots).
## Preconditions
- SSH/docker access to `10.100.0.35`.
- The existing `gitea-runner` container's compose/run config, to copy its
`container.network: traefik` setting and registration token flow (repo memory
`project_gitea_ci` records this configuration).
- Admin access to Gitea (`gitea.dohertylan.com`) to mint a new runner registration token.
## Steps — option (a): second runner instance
1. On `10.100.0.35`, locate the existing `gitea-runner` container/compose definition and copy
its configuration for a new instance (same `container.network: traefik`, same Docker
socket mount if it uses docker-in-docker, a distinct container name/data volume).
2. In Gitea, generate a new runner registration token (instance-level, since the existing
runner is registered at the instance level too — Admin → Actions → Runners, or
`POST /admin/actions/runners/registration-token`).
3. Register and start the second `act_runner` instance with that token, pointed at the same
Gitea origin.
4. Confirm both runners show online: instance runner list in the Gitea admin UI, or the
equivalent API listing.
## Verification
- Push two branches to `mxaccessgw` back-to-back (or trigger one `mxaccessgw` push while an
`lmxopcua` run is in flight) and confirm both runs execute **concurrently**, not serially —
the second run's jobs should start before the first finishes, not queue behind it.
- `GET /repos/dohertj2/mxaccessgw/actions/runners` (or the instance runner listing) shows
**≥2** runners online.
- Re-run the TST-25 acceptance push (a plain push to a scratch branch) and confirm queue depth
is materially lower than the ~20–30 minute baseline observed under a concurrent `lmxopcua`
run.
- Confirm `windows-x86` still resolves `gitea:3000` correctly from a job scheduled on the new
runner instance (the `traefik` network property must hold for both instances).
## The no-cancel reality does not go away
A second runner relieves contention; it does not add a cancel/delete API — Gitea 1.26 still
returns 404/400 for both. A stale or hung run on either runner still holds its slot until it
finishes or times out. Two runners just means one stale run blocks at most half the capacity
instead of all of it. Do not treat the second runner as a substitute for the escape hatch: a
specific commit can still be verified out of band without waiting on either runner via
`CI_SHA=<sha> scripts/ci/run-windev-ci.sh <build|test|live>` (Linux, needs SSH access to
windev) or the manual windev worktree flow — see the "Runner capacity is shared and finite"
section in `docs/GatewayTesting.md`.
## Optional: workflow-level `concurrency` group
As belt-and-suspenders against the missing cancel API, `.gitea/workflows/ci.yml` could add a
top-level `concurrency` group (e.g. keyed on `${{ github.ref }}`) so a newer push to the same
branch automatically supersedes an in-flight run instead of both running to completion.
**Verify this Gitea deployment actually honors `concurrency` and cancels the superseded run
before relying on it** — Gitea Actions' YAML surface does not track GitHub Actions feature
parity release-for-release, and a `concurrency` block that is silently ignored would look like
a working safeguard while doing nothing. If verified working, this is a `ci.yml` change (not
covered by this runbook) and should land as its own small change with its own verification
(push twice to the same branch quickly, confirm the first run's jobs cancel).
## Done criteria
- A second `act_runner` instance (or raised `maxParallel`) is online on `10.100.0.35` with the
same `container.network: traefik` configuration as the existing runner.
- `GET /repos/dohertj2/mxaccessgw/actions/runners` (or the instance listing) shows ≥2 runners.
- Two concurrent runs (one `mxaccessgw`, one `lmxopcua`, or two `mxaccessgw` pushes) execute
in parallel rather than serially.
- `docs/GatewayTesting.md`'s shared-runner/no-cancel prose and the `run-windev-ci.sh` bypass
remain accurate (they describe the bypass as still valid, which it is regardless of runner
count).
+3 -1
View File
@@ -5,7 +5,9 @@ library, CLI, and tests.
## Baseline
- Target the Java version defined by the client build, with Java 21 preferred.
- Target Java 17 (the Ignition 8.3 baseline; the client build enforces
`options.release = 17` with a Gradle toolchain 17). Code must compile and
run on 17; newer JDKs may host the build.
- Use Gradle unless the repository standardizes on Maven.
- Apply a formatter such as Spotless or Google Java Format when configured.
- Keep generated protobuf code separate from handwritten wrappers.
+15 -7
View File
@@ -348,9 +348,13 @@ messages, tagged from 10 upward:
Rules:
- `sequence` is a monotonic per-sender counter used as a diagnostic aid; it is
not validated for gaps or ordering on receive (the named pipe already
guarantees FIFO delivery).
- `sequence` is a monotonic per-sender counter used as a diagnostic aid. Both
sides stamp it at the point of writing, inside their single write path — the
gateway on its outbound-channel write loop, the worker on its own writer — so
the numbers stay monotonic in wire order no matter how concurrent callers
interleave while building envelopes. It is not validated for gaps or ordering
on receive (the named pipe already guarantees FIFO delivery); if inbound
enforcement is ever added, this is the property it will rely on.
- `correlation_id` links a command to its reply; it is authoritative on the
envelope, and the inner `MxCommandReply.correlation_id` echoes it for
MXAccess parity.
@@ -470,10 +474,14 @@ Optional diagnostics:
- `Ping`
- `GetSessionState`
- `GetWorkerInfo`
- `DrainEvents` — diagnostic; `max_events` is bounded (the gateway rejects requests
above a public ceiling, and the worker caps each reply at its own per-reply limit,
treating `max_events = 0` as "the default cap") so one drain cannot pack an
unbounded, session-killing reply frame.
- `DrainEvents` — diagnostic; the reply is bounded on two axes so one drain cannot
pack an unbounded, session-killing reply frame. By **count**: the gateway rejects
requests above the shared ceiling `GatewayContractInfo.MaxDrainEventsPerCommand`
and the worker clamps to the same value, treating `max_events = 0` as "the default
cap". By **bytes**: the worker sizes the reply while draining, against the
negotiated worker-frame maximum, so a byte-heavy queue truncates instead of
overshooting and events that do not fit stay queued. Truncation is reported in the
reply's `DiagnosticMessage`; callers drain iteratively until an empty reply.
- `ShutdownWorker`
Do not compress MXAccess semantics into generic verbs too early. A command enum
+14 -5
View File
@@ -30,10 +30,19 @@ gw-specific role.
| LDAPS | disabled in dev (`Transport=None`, `AllowInsecure=true`) |
| Base DN | `dc=zb,dc=local` |
| Bind DN format | `cn={username},dc=zb,dc=local` |
| Service account DN | `cn=serviceaccount,dc=zb,dc=local` / `serviceaccount123` |
| Service account DN | `cn=serviceaccount,dc=zb,dc=local` (password: `<service-account-password>`) |
| Group OU | `ou=<groupname>,ou=groups,dc=zb,dc=local` |
| Failed-bind throttle | 3 fails → 10-minute IP lockout (per `[behaviors]`) |
> **Service-account password is not committed (SEC-36).** The samples below show
> `<service-account-password>` as a placeholder, not the real value. The single source of
> truth is **`scadaproj/infra/glauth/config.toml`** on host `10.100.0.35`; the gateway consumes
> it out-of-band (encrypted secrets store reference `${secret:ldap/mxgateway/bind}`, the
> `MxGateway__Ldap__ServiceAccountPassword` env var on deployed hosts, or `dotnet user-secrets`
> on dev boxes — see `docs/GatewayConfiguration.md`). The credential was historically committed
> to this repo (and remains recoverable from git history), so **rotation is required**; the
> operator runbook is `docs/runbooks/SEC-36-ldap-credential-rotation.md`.
## Pre-existing groups (LmxOpcUa role taxonomy)
These map cleanly onto MxAccess capability boundaries — mxaccessgw
@@ -62,7 +71,7 @@ group below).
| `writeconfig` | `writeconfig123` | 5006 | WriteConfigure | — | + WriteSecured (Configure) |
| `alarmack` | `alarmack123` | 5003 | AlarmAck | — | Alarm acknowledgment |
| `admin` | `admin123` | 5004 | ReadOnly | WriteOperate, AlarmAck, WriteTune, WriteConfigure | All roles |
| `serviceaccount` | `serviceaccount123` | 5999 | ReadOnly | — | LDAP search capability (for bind-then-search) |
| `serviceaccount` | `<service-account-password>` | 5999 | ReadOnly | — | LDAP search capability (for bind-then-search) |
For mxaccessgw dev, `admin` covers every gw-side capability test;
`readonly` is the right "negative" case for proving Browse-OK /
@@ -100,7 +109,7 @@ by `sAMAccountName`, not `cn`. Use this only for dev convenience.
```
1. Bind as the service account (cn=serviceaccount,dc=zb,dc=local
/ serviceaccount123).
/ <service-account-password>).
2. Search under dc=zb,dc=local with filter
(uid=<entered-username>) — or any attribute the deployment
identifies users by. GLAuth populates uid + cn.
@@ -133,7 +142,7 @@ ldap:
allowInsecureLdap: true # dev only
searchBase: "dc=zb,dc=local"
serviceAccountDn: "cn=serviceaccount,dc=zb,dc=local"
serviceAccountPassword: "serviceaccount123"
serviceAccountPassword: "<service-account-password>" # not committed; see source-of-truth note
userNameAttribute: "uid" # GLAuth populates this; AD uses sAMAccountName
displayNameAttribute: "cn"
groupAttribute: "memberOf"
@@ -242,7 +251,7 @@ Or via `ldapsearch` if you have OpenLDAP CLI tools:
```bash
ldapsearch -x -H ldap://10.100.0.35:3893 \
-D "cn=serviceaccount,dc=zb,dc=local" -w serviceaccount123 \
-D "cn=serviceaccount,dc=zb,dc=local" -w '<service-account-password>' \
-b "dc=zb,dc=local" "(uid=multi-role)"
```
+33 -5
View File
@@ -1,7 +1,7 @@
#!/usr/bin/env pwsh
# Codegen freshness guard for CI (IPC-01, IPC-19, IPC-20, CLI-02).
# Codegen freshness guard for CI (IPC-01, IPC-19, IPC-20, IPC-25, CLI-02).
#
# Three checks, all Linux/macOS-runnable (no Server build, no x86 worker):
# Four checks, all Linux/macOS-runnable (no Server build, no x86 worker):
# 1. Published client descriptor set matches the current .proto sources (delegates to
# publish-client-proto-inputs.ps1 -Check, which normalizes source_code_info so it is
# protoc-version tolerant).
@@ -13,6 +13,12 @@
# crate buildable outside the repo, CLI-02) are byte-identical to the canonical Contracts
# protos. A drift means a .proto was edited without refreshing the vendored copies, which would
# publish a stale wire contract to crate consumers while the in-repo build stays correct.
# 4. The committed Go and Python client bindings match a fresh regeneration (IPC-25). The two
# per-client generate-proto.ps1 scripts pin their generators (protoc-gen-go v1.36.11 /
# protoc-gen-go-grpc v1.6.2 for Go; grpcio-tools 1.80.0 for Python), so a clean checkout
# regenerates deterministic output; a non-empty git diff means a .proto was edited without
# regenerating and committing those bindings. A missing generator FAILS the check (a skipped
# guard is the exact silent-drift hole IPC-25 closes), never skips it.
#
# The x86 Worker + Worker.Tests are Windows-only and are guarded by the SSH-driven `windows-x86`
# CI job (see docs/GatewayTesting.md, Continuous Integration), not here.
@@ -28,7 +34,7 @@ $generatedDir = Join-Path $repoRoot 'src/ZB.MOM.WW.MxGateway.Contracts/Generated
$contractsProject = Join-Path $repoRoot 'src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csproj'
$failures = New-Object System.Collections.Generic.List[string]
Write-Host '== Check 1/2: client descriptor set freshness =='
Write-Host '== Check 1/4: client descriptor set freshness =='
try {
& (Join-Path $PSScriptRoot 'publish-client-proto-inputs.ps1') -Check
if ($LASTEXITCODE -ne 0) {
@@ -40,7 +46,7 @@ catch {
}
Write-Host ''
Write-Host '== Check 2/2: Contracts/Generated matches a fresh regeneration =='
Write-Host '== Check 2/4: Contracts/Generated matches a fresh regeneration =='
try {
# Force a full regeneration: Grpc.Tools skips regen when the committed .cs look up to date, so
# remove them first (the documented "del Generated/*.cs to force regen" trick).
@@ -66,7 +72,7 @@ catch {
}
Write-Host ''
Write-Host '== Check 3/3: Rust vendored protos match canonical Contracts protos =='
Write-Host '== Check 3/4: Rust vendored protos match canonical Contracts protos =='
try {
$canonicalProtoDir = Join-Path $repoRoot 'src/ZB.MOM.WW.MxGateway.Contracts/Protos'
$vendoredProtoDir = Join-Path $repoRoot 'clients/rust/protos'
@@ -87,6 +93,28 @@ catch {
$failures.Add("Rust vendored proto check failed: $($_.Exception.Message)")
}
Write-Host ''
Write-Host '== Check 4/4: Go and Python client bindings match a fresh regeneration =='
try {
# Regenerate both binding sets with their pinned generators, then diff. The per-client scripts
# throw on a missing or off-pin generator, so any failure here FAILS the check rather than
# skipping it (a skipped guard is exactly the silent-drift hole IPC-25 closes).
$goBindingDir = 'clients/go/internal/generated'
$pyBindingDir = 'clients/python/src/zb_mom_ww_mxgateway/generated'
& (Join-Path $repoRoot 'clients/go/generate-proto.ps1') | Out-Host
& (Join-Path $repoRoot 'clients/python/generate-proto.ps1') | Out-Host
$bindingDiff = (& git -C $repoRoot status --porcelain -- $goBindingDir $pyBindingDir | Out-String).Trim()
if (-not [string]::IsNullOrEmpty($bindingDiff)) {
Write-Host $bindingDiff
$failures.Add("Go/Python client bindings differ from a fresh regeneration. Run clients/go/generate-proto.ps1 and clients/python/generate-proto.ps1 with the pinned generators and commit $goBindingDir and $pyBindingDir.")
}
}
catch {
$failures.Add("Go/Python codegen check failed (tool missing or regeneration error): $($_.Exception.Message)")
}
Write-Host ''
if ($failures.Count -gt 0) {
Write-Host 'Codegen freshness check FAILED:' -ForegroundColor Red
+116
View File
@@ -87,6 +87,12 @@ $GiteaNugetFeed = 'https://gitea.dohertylan.com/api/packages/dohertj2/nuget/inde
$GiteaPypiFeed = 'https://gitea.dohertylan.com/api/packages/dohertj2/pypi'
$JavaHome = '/Users/dohertj2/.local/jdks/jdk-21.0.11+10/Contents/Home'
# Generic Gitea package registry API (https://gitea.dohertylan.com/api/v1/packages/{owner}/{type}/{name}/{version}):
# returns 200 when that exact name+version already exists in the given feed
# type, 404 when it does not. Used as a pre-publish collision guard (CLI-39)
# so a re-run of this script can never silently overwrite a published artifact.
$GiteaPackageApiBase = 'https://gitea.dohertylan.com/api/v1/packages/dohertj2'
function Write-Header {
param([string]$Text)
Write-Host ''
@@ -94,6 +100,64 @@ function Write-Header {
Write-Host $Text -ForegroundColor Cyan
}
function Test-GiteaPackageExists {
<#
.SYNOPSIS
Queries the Gitea package API for an existing name+version in a feed.
.OUTPUTS
$true if the package/version already exists, $false if it does not.
Throws if the registry cannot be reached or returns anything other
than 200/404 — callers must treat "cannot verify" as "do not publish".
#>
param(
[Parameter(Mandatory)][string]$Type,
[Parameter(Mandatory)][string]$Name,
[Parameter(Mandatory)][string]$Version
)
$uri = "$GiteaPackageApiBase/$Type/$Name/$Version"
$headers = @{}
if (-not [string]::IsNullOrEmpty($env:GITEA_TOKEN)) {
$user = if ([string]::IsNullOrEmpty($env:GITEA_USERNAME)) { 'dohertj2' } else { $env:GITEA_USERNAME }
$pair = "$($user):$($env:GITEA_TOKEN)"
$basic = [Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes($pair))
$headers['Authorization'] = "Basic $basic"
}
try {
$response = Invoke-WebRequest -Uri $uri -Headers $headers -Method Get -UseBasicParsing -ErrorAction Stop
return ($response.StatusCode -eq 200)
} catch {
$statusCode = $null
if ($_.Exception.PSObject.Properties['Response'] -and $_.Exception.Response) {
$statusCode = [int]$_.Exception.Response.StatusCode
}
if ($statusCode -eq 404) {
return $false
}
throw "Unable to query the Gitea package API for '$Type/$Name/$Version' ($uri): $($_.Exception.Message). Refusing to publish without a collision check — verify manually (or check GITEA_USERNAME/GITEA_TOKEN/network) and retry."
}
}
function Assert-GiteaPackageNotPublished {
<#
.SYNOPSIS
Aborts the script if $Name/$Version already exists in the $Type feed.
Never force-overwrites a published artifact (CLI-39).
#>
param(
[Parameter(Mandatory)][string]$Type,
[Parameter(Mandatory)][string]$Name,
[Parameter(Mandatory)][string]$Version
)
Write-Host "Checking Gitea '$Type' feed for existing '$Name' $Version..."
if (Test-GiteaPackageExists -Type $Type -Name $Name -Version $Version) {
throw "Gitea package '$Name' version '$Version' already exists in the '$Type' feed. Bump the client version before publishing — this script never force-overwrites a published artifact."
}
Write-Host " Not found in the '$Type' feed — safe to publish '$Name' $Version." -ForegroundColor Green
}
# -------- .NET --------
function Invoke-PackDotnet {
@@ -121,6 +185,15 @@ function Invoke-PackDotnet {
if ($Publish) {
Write-Host 'Publishing .NET packages to Gitea...' -ForegroundColor Yellow
Get-ChildItem $OutputDir -Filter 'ZB.MOM.WW.MxGateway.*.nupkg' | ForEach-Object {
# nupkg filenames are '<PackageId>.<Version>.nupkg'; the id itself contains
# dots (e.g. 'ZB.MOM.WW.MxGateway.Client'), so the id capture is lazy and the
# version capture anchors on the leading digit to split at the right dot.
$fileBaseName = [System.IO.Path]::GetFileNameWithoutExtension($_.Name)
if ($fileBaseName -notmatch '^(?<id>.+?)\.(?<version>\d+\.\d+\.\d+(?:[-+][0-9A-Za-z.-]+)?)$') {
throw "Could not parse a NuGet package id/version out of '$($_.Name)'."
}
Assert-GiteaPackageNotPublished -Type 'nuget' -Name $Matches.id -Version $Matches.version
& dotnet nuget push $_.FullName --source $GiteaNugetFeed --api-key $env:GITEA_TOKEN
if ($LASTEXITCODE -ne 0) { throw "dotnet nuget push failed for '$($_.Name)'." }
}
@@ -159,6 +232,20 @@ function Invoke-PackPython {
Write-Host "Packed Python artifacts -> $OutputDir" -ForegroundColor Green
if ($Publish) {
$pyprojectPath = Join-Path $RepoRoot 'clients/python/pyproject.toml'
$pyprojectContent = Get-Content $pyprojectPath -Raw
# Scope to the [project] section (not just the first "version = ..." line
# in the file) — [build-system]/[tool.*] sections can carry their own
# version-shaped keys, and matching the file's first hit would be luck
# of ordering, not correctness.
if ($pyprojectContent -notmatch '(?ms)^\[project\](?<section>.*?)(?=^\[|\z)') {
throw "Could not find a [project] section in '$pyprojectPath'."
}
if ($Matches.section -notmatch '(?m)^\s*version\s*=\s*"([^"]+)"') {
throw "Could not find [project].version in '$pyprojectPath'."
}
Assert-GiteaPackageNotPublished -Type 'pypi' -Name 'zb-mom-ww-mxaccess-gateway-client' -Version $Matches[1]
Write-Host 'Publishing Python distribution to Gitea...' -ForegroundColor Yellow
$wheels = @(Get-ChildItem $OutputDir -Filter 'zb_mom_ww_mxaccess_gateway_client-*.whl')
$sdists = @(Get-ChildItem $OutputDir -Filter 'zb_mom_ww_mxaccess_gateway_client-*.tar.gz')
@@ -206,6 +293,20 @@ function Invoke-PackRust {
Write-Host "Packed Rust artifacts -> $OutputDir" -ForegroundColor Green
if ($Publish) {
$cargoTomlPath = Join-Path $rustDir 'Cargo.toml'
$cargoTomlContent = Get-Content $cargoTomlPath -Raw
# Scope to the [package] section specifically — Cargo.toml also carries a
# [workspace.package] section with its own "version = ..." line (today
# identical, by convention, not by anything this regex can rely on), and
# matching whichever comes first in the file is luck of ordering.
if ($cargoTomlContent -notmatch '(?ms)^\[package\](?<section>.*?)(?=^\[|\z)') {
throw "Could not find a [package] section in '$cargoTomlPath'."
}
if ($Matches.section -notmatch '(?m)^\s*version\s*=\s*"([^"]+)"') {
throw "Could not find [package] version in '$cargoTomlPath'."
}
Assert-GiteaPackageNotPublished -Type 'cargo' -Name 'zb-mom-ww-mxgateway-client' -Version $Matches[1]
Write-Host 'Publishing Rust crate to Gitea...' -ForegroundColor Yellow
Push-Location (Join-Path $RepoRoot 'clients/rust')
try {
@@ -269,6 +370,21 @@ function Invoke-PackJava {
Write-Host "Packed Java artifacts -> $OutputDir" -ForegroundColor Green
if ($Publish) {
$buildGradlePath = Join-Path $javaDir 'build.gradle'
$buildGradleContent = Get-Content $buildGradlePath -Raw
if ($buildGradleContent -notmatch "(?m)^\s*group\s*=\s*'([^']+)'") {
throw "Could not find subprojects { group = '...' } in '$buildGradlePath'."
}
$javaGroup = $Matches[1]
if ($buildGradleContent -notmatch "(?m)^\s*version\s*=\s*'([^']+)'") {
throw "Could not find subprojects { version = '...' } in '$buildGradlePath'."
}
$javaVersion = $Matches[1]
# Gitea's Maven package API identifies the package as "groupId:artifactId",
# not the bare artifact id — passing just the artifact id here would query
# a name that never exists and silently defeat the guard.
Assert-GiteaPackageNotPublished -Type 'maven' -Name "$javaGroup`:zb-mom-ww-mxgateway-client" -Version $javaVersion
Write-Host 'Publishing Java artifacts to Gitea Maven feed...' -ForegroundColor Yellow
Push-Location $javaDir
try {
+17
View File
@@ -36,6 +36,23 @@ if ($Version -notmatch '^v\d+\.\d+\.\d+(-[A-Za-z0-9.-]+)?$') {
throw "Version '$Version' must match semver vX.Y.Z (optionally with -prerelease suffix)."
}
# CLI-21 guard: the tag must match what the module itself reports via
# ClientVersion, or `go get <module>@vX.Y.Z` resolves a tag whose module
# code disagrees with its own version constant.
$versionGoPath = Join-Path $PSScriptRoot '..' 'clients/go/mxgateway/version.go'
if (-not (Test-Path $versionGoPath)) {
throw "Could not find '$versionGoPath' to verify ClientVersion before tagging."
}
$versionGoContent = Get-Content $versionGoPath -Raw
if ($versionGoContent -notmatch 'ClientVersion\s*=\s*"([^"]+)"') {
throw "Could not find a ClientVersion = `"...`" constant in '$versionGoPath'."
}
$clientVersion = $Matches[1]
$tagVersion = $Version.TrimStart('v')
if ($clientVersion -ne $tagVersion) {
throw "clients/go/mxgateway/version.go ClientVersion is '$clientVersion' but the requested tag is '$tagVersion'. Update ClientVersion to match before tagging."
}
$tag = "clients/go/$Version"
Write-Host "Creating Go-module tag: $tag" -ForegroundColor Cyan
+8 -4
View File
@@ -11,10 +11,14 @@
<!-- TST-11: single-source the .NET-side version for Server, Worker, Contracts, and tests
(they otherwise stamp the SDK default 1.0.0, so a deployed gateway cannot be correlated
to a release). Kept at 0.1.2 to match the Contracts package and the aligned Python/Rust/
Go clients; the Java client leads at 0.2.0 after its JDK-17 retarget. The git short SHA is
appended to InformationalVersion (0.1.2+<sha>) so support can map a running binary to a
commit; the query is guarded so a build outside a git checkout still succeeds. -->
to a release). Server/Worker/Tests stay at this default. CLI-39 (2026-08-07) moved the
published `ZB.MOM.WW.MxGateway.Contracts` and `.Client` nuget packages to 0.2.0 via an
explicit <Version> override in Contracts.csproj (MSBuild property-last-write-wins over
this Directory.Build.props default) — Server/Worker assembly stamping and the published
client packages are deliberately decoupled; a broader 0.2.0 alignment for Server/Worker
is a separate, not-yet-made decision. The git short SHA is appended to
InformationalVersion (0.1.2+<sha>) so support can map a running binary to a commit; the
query is guarded so a build outside a git checkout still succeeds. -->
<PropertyGroup>
<Version>0.1.2</Version>
</PropertyGroup>
@@ -17,6 +17,21 @@ public static class GatewayContractInfo
/// <summary>Default backend name identifying the MXAccess worker process type.</summary>
public const string DefaultBackendName = "mxaccess-worker";
/// <summary>
/// Ceiling on how many events one <c>DrainEvents</c> command may move in a single reply.
/// Shared so the gateway's request-validation ceiling
/// (<c>MxAccessGrpcRequestValidator</c>, which rejects a larger <c>max_events</c> loudly at
/// the public boundary) and the worker's per-reply clamp
/// (<c>WorkerPipeSession.CreateDrainEventsReply</c>, the backstop that also interprets
/// <c>max_events = 0</c>) cannot drift apart. A count cap alone is necessary but not
/// sufficient: the worker additionally caps the reply by serialized bytes against the
/// negotiated frame maximum (WRK-21), so a reply may carry fewer events than this ceiling
/// and fewer than are queued. Callers drain iteratively until an empty reply.
/// This is a documented behavioral bound, not wire schema — it is deliberately a C#
/// constant and not a <c>.proto</c> field.
/// </summary>
public const uint MaxDrainEventsPerCommand = 10_000;
/// <summary>
/// Environment variable name that opts an xUnit suite into running live
/// MXAccess COM tests. Single source of truth shared by both
@@ -22796,6 +22796,12 @@ namespace ZB.MOM.WW.MxGateway.Contracts.Proto {
private static readonly pb::FieldCodec<global::ZB.MOM.WW.MxGateway.Contracts.Proto.MxEvent> _repeated_events_codec
= pb::FieldCodec.ForMessage(10, global::ZB.MOM.WW.MxGateway.Contracts.Proto.MxEvent.Parser);
private readonly pbc::RepeatedField<global::ZB.MOM.WW.MxGateway.Contracts.Proto.MxEvent> events_ = new pbc::RepeatedField<global::ZB.MOM.WW.MxGateway.Contracts.Proto.MxEvent>();
/// <summary>
/// The reply is bounded by both a server-side count cap and the negotiated
/// worker-frame byte cap; a reply may therefore carry fewer events than
/// `max_events` and fewer than are queued. Callers drain iteratively until an
/// empty reply.
/// </summary>
[global::System.Diagnostics.DebuggerNonUserCodeAttribute]
[global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
public pbc::RepeatedField<global::ZB.MOM.WW.MxGateway.Contracts.Proto.MxEvent> Events {
@@ -24510,6 +24516,11 @@ namespace ZB.MOM.WW.MxGateway.Contracts.Proto {
/// after_worker_sequence = oldest_available_sequence - 1 in the next
/// StreamEventsRequest, which will cause the server to replay starting at
/// oldest_available_sequence (the first retained event).
/// When nothing is retained (the replay ring is empty), this is the next sequence
/// that can be delivered — `highest observed + 1` — and the `oldest - 1` resume
/// formula remains valid: it resolves to the highest sequence already seen, so the
/// follow-up resume replays nothing, reports no gap, and every newer live event
/// passes. The interval evicted is unchanged.
/// </summary>
[global::System.Diagnostics.DebuggerNonUserCodeAttribute]
[global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
@@ -1164,6 +1164,9 @@ namespace ZB.MOM.WW.MxGateway.Contracts.Proto {
/// instead of a hard-coded default; 0 (an older gateway that never set the field) means
/// "use the worker's built-in default". Sits above the public gRPC cap by an
/// envelope-overhead margin so an accepted gRPC payload always fits one worker frame.
/// Every worker->gateway frame — events, heartbeats, faults, and control replies
/// including DrainEvents — must serialize within this limit; reply builders truncate
/// to fit rather than emit an oversized frame.
/// </summary>
[global::System.Diagnostics.DebuggerNonUserCodeAttribute]
[global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
@@ -676,6 +676,10 @@ message WorkerInfoReply {
}
message DrainEventsReply {
// The reply is bounded by both a server-side count cap and the negotiated
// worker-frame byte cap; a reply may therefore carry fewer events than
// `max_events` and fewer than are queued. Callers drain iteratively until an
// empty reply.
repeated MxEvent events = 1;
}
@@ -760,6 +764,11 @@ message ReplayGap {
// after_worker_sequence = oldest_available_sequence - 1 in the next
// StreamEventsRequest, which will cause the server to replay starting at
// oldest_available_sequence (the first retained event).
// When nothing is retained (the replay ring is empty), this is the next sequence
// that can be delivered — `highest observed + 1` — and the `oldest - 1` resume
// formula remains valid: it resolves to the highest sequence already seen, so the
// follow-up resume replays nothing, reports no gap, and every newer live event
// passes. The interval evicted is unchanged.
uint64 oldest_available_sequence = 2;
}
@@ -47,6 +47,9 @@ message GatewayHello {
// instead of a hard-coded default; 0 (an older gateway that never set the field) means
// "use the worker's built-in default". Sits above the public gRPC cap by an
// envelope-overhead margin so an accepted gRPC payload always fits one worker frame.
// Every worker->gateway frame — events, heartbeats, faults, and control replies
// including DrainEvents — must serialize within this limit; reply builders truncate
// to fit rather than emit an oversized frame.
uint32 max_frame_bytes = 4;
}

Some files were not shown because too many files have changed in this diff Show More