diff --git a/archreview/2026-07-12/remediation/00-tracking.md b/archreview/2026-07-12/remediation/00-tracking.md index aeb4c3a..9b1cba4 100644 --- a/archreview/2026-07-12/remediation/00-tracking.md +++ b/archreview/2026-07-12/remediation/00-tracking.md @@ -114,7 +114,7 @@ Full design + implementation for each row lives in the linked domain doc under i | CLI-36 | Medium | P0 | S | GWC-25 (coord) | Done | Go CLI `stream-events` silently destroys the ReplayGap signal | | CLI-37 | Medium | P1 | M | CLI-38 (co-land) | Done | Status-array validation must branch on `category` per the proto contract (4-vs-1 divergence) | | CLI-38 | Medium | P1 | S | — | Done | Align .NET/Go/Java on `hresult < 0` — lands old CLI-08, cures design-doc drift | -| CLI-39 | Medium | P1 | S | CLI-35..38, CLI-45 (land last) | Not started | Bump client versions off published 0.1.2 (converge on 0.2.0); registry-collision guard in pack-clients.ps1 | +| CLI-39 | Medium | P1 | S | CLI-35..38, CLI-45 (land last) | Done | Bump client versions off published 0.1.2 (converge on 0.2.0); registry-collision guard in pack-clients.ps1 | | CLI-40 | Low | — | M | — | Done | Port the exact-secret credential scrub to Rust/Java/.NET | | CLI-41 | Low | — | M | — | Done | Uniform malformed-reply contract for AuthenticateUser/ArchestrAUserToId/AddBufferedItem | | CLI-42 | Low | P1 | S | — | Done | Document the vendored Rust proto layout (CLI-02's missing doc half) | @@ -142,7 +142,7 @@ Sequence these together rather than piecemeal — several are one change set spa - **Auth limiter (P0):** SEC-31 + SEC-32 — same component, one change set, one test suite. - **Codegen freshness (P0):** IPC-24 + IPC-25 (+ IPC-32 folded in). Both edit `.gitea/workflows/ci.yml` — coordinate the branch with TST-25, which touches the same file. - **Windows-tier automation (P1):** TST-25 + TST-26 (same commit). Unlocks old TST-05/TST-24 and provides CI evidence for every windev-verified cluster above; until it lands, record windev runs in this tracker's change log. -- **Client conformance + release train (P1):** CLI-37 + CLI-38 co-land (one conformance commit; closes old CLI-08), then CLI-45, with CLI-40/41 fixtures as follow-ups; **CLI-39 lands last** so published 0.2.0 carries the conformant behavior. Shared fixtures under `clients/proto/fixtures/behavior/`; update CrossLanguageSmokeMatrix.md/ClientLibrariesDesign.md same-commit. Do not republish regenerated bindings (IPC-25) before CLI-39 resolves. +- **Client conformance + release train (P1):** CLI-37 + CLI-38 co-land (one conformance commit; closes old CLI-08), then CLI-45, with CLI-40/41 fixtures as follow-ups; **CLI-39 lands last** so published 0.2.0 carries the conformant behavior — `Done` 2026-08-07 (Rust/Python/Go/.NET → 0.2.0, Java → 0.2.1; see CLI-39 change-log entry). Shared fixtures under `clients/proto/fixtures/behavior/`; update CrossLanguageSmokeMatrix.md/ClientLibrariesDesign.md same-commit. IPC-25's regenerated Go/Python bindings are still not republished — CLI-39 bumped the version constants but did not run an actual publish (explicitly out of scope; see the change-log entry). - **Doc-drift batch (P1):** TST-27 + WRK-26 (discharges IPC-29) + CLI-42 + IPC-28 + SEC-35's doc note — one sweep commit is fine. - **Backpressure follow-on (P1):** GWC-24 — `Done` 2026-08-07. Landed without a new config key (the staging bound is derived as `2 × EventChannelCapacity`), so still-open old GWC-21 (`EventChannelFullModeTimeout` configurability) remains the natural companion knob but was not a blocker. @@ -176,3 +176,4 @@ Sequence these together rather than piecemeal — several are one change set spa | 2026-08-07 | Code-review follow-ups on the same branch (commit `6bc3f9b`). (1) **Important** — `ResolveDrainReplyByteBudget` was a step, not a floor: just above the 64 KiB reserve the budget collapsed to a few bytes (exactly 1024 at the validator floor `MaxMessageBytes = 1024 + 64 KiB`), so a byte-heavy `DrainEvents` truncated on every call and the drain-until-empty loop never terminated. Now `Math.Max(frameMax - reserve, frameMax / 2)` — monotonic, never below half the frame max. New test `WorkerPipeSessionTests.DrainEvents_AtValidatorFloorFrameMax_MakesProgressAndTerminates` drives a byte-heavy queue at the exact validator floor and asserts drain-to-empty with no head reported oversized. (2) **Hardening** — the reply-too-large fallback write is now itself size-guarded (`WriteReplyTooLargeFallbackAsync`, shared by the control and STA reply seams) so a pathologically tiny negotiated max below the gateway floor (the WRK-24 gap) cannot make even the backstop session-fatal; log-and-swallow, comment points at WRK-24. (3) **Comment** — corrected the `RepeatedFieldOverheadBytes` docs: `WorkerEvent.CalculateSize()` already includes the event's tag+length, so the 8 bytes is pure slack, not wrapper compensation. **Evidence** — macOS build 0/0, validator filter 4/4. windev (`windev-worker-ci.ps1 -Sha 6bc3f9b -Mode test`, 07:07): x86 Worker build 0/0, `Worker.Tests` **368 passed / 0 failed / 11 skipped**, script exit 0. (An earlier run of the same SHA flaked on the pre-existing `RunAsync_WhenStaActivityIsStale_WritesWatchdogFault` — a 5 s CTS timeout under first-run load, untouched by this change; it passed on the clean re-run and in both prior full runs.) | | 2026-08-07 | **P1 doc-drift batch → `Done`: TST-27, WRK-26 (discharges IPC-29), CLI-42, CLI-43, IPC-28** (branch `fix/doc-drift-batch`). Doc-only; no source, proto, or test changes — cross-checked against HEAD in this worktree. **TST-27**: `docs/GatewayConfiguration.md`'s `ShowTagValues` row no longer says "Reserved" — it now states what `false` (default) does (`DashboardEventBroadcaster` blanks tag values from a deep-cloned `MxEvent` before the SignalR events-hub mirror, metadata still renders), the security relevance (the per-session hub ACL, SEC-25 roadmap item 12, still does not exist, so this redaction is the only thing between a low-trust Viewer and other sessions' tag values), and the honest scope limit (the flag does **not** cover `/browse`). **WRK-26** (discharges **IPC-29**): `docs/MxAccessWorkerInstanceDesign.md`'s "Outbound Queues" section rewritten from the stale five-level priority list to the two-class `Control`/`Event` scheduler actually shipped (`WorkerFrameWriter`/`WorkerFrameWritePriority.cs`), with the collapsed-decision rationale recorded, and the overflow paragraph rewritten to the implemented fail-fast (`WorkerFault` category `QueueOverflow` → fault frame written → `RunAsync` unwinds → generic `WorkerExitCode.UnexpectedFailure`, dedicated code still open). `docs/WorkerFrameProtocol.md` gained a new "Write Scheduling And Sequencing" section: the two priority classes, enqueue-then-contend/single-lock-holder-drains-all, write-time peek-stamp-commit sequencing, per-frame-rejection vs. stream-failure semantics, and flush coalescing — stated truthfully as landed (WRK-23's peek-stamp-commit is live at HEAD) or not (the drain loop still awaits each event `WriteAsync` individually, so WRK-25's N-events-one-flush batching has **not** landed and the section says so explicitly). Cancellation is deliberately **not** documented as a firm contract — a one-paragraph placeholder notes it is pending WRK-22, which has not landed (confirmed by reading `WorkerFrameWriter.cs`: no `Claimed`/tombstone machinery exists yet). `gateway.md:328-330` was cross-checked and left unchanged — its sequence prose (both sides stamp at write, per GWC-28) already reads true. **CLI-42**: `clients/rust/README.md` and `docs/ClientPackaging.md`'s Rust section now document the vendored proto layout matching `clients/rust/build.rs` exactly — repo-path-first resolution (`../../src/ZB.MOM.WW.MxGateway.Contracts/Protos`) falling back to `clients/rust/protos/` when the canonical path is absent (published-tarball case), the same-commit refresh rule enforced by `scripts/check-codegen.ps1` Check 3, and why `cargo package`/`cargo publish` run without `--no-verify` (matches `scripts/pack-clients.ps1:190-192`). **CLI-43**: `docs/style-guides/JavaStyleGuide.md` line 8 now says "Target Java 17 (the Ignition 8.3 baseline...)" mirroring the CLI-12 wording, matching the shipped `clients/java/build.gradle` toolchain-17 build. **IPC-28**: `docs/Grpc.md`'s exception-mapping prose gained `CommandTooLarge` → `ResourceExhausted` (verified against the live `switch` in `Grpc/MxAccessGatewayService.cs:950-960`), and the `Invoke` section gained one sentence on the oversized-payload path (`WorkerClient.InvokeAsync` rejects at the enqueue boundary per-correlation, session not faulted — verified against `WorkerClient.cs:220-234`), cross-referencing the headroom rule already documented in `docs/GatewayConfiguration.md:120-129`. Did not touch the DrainEvents-truncation row or the proto/`Generated/` trees — those belong to a parallel codegen task per the handoff note. **Source files cross-read for accuracy** (no edits): `src/ZB.MOM.WW.MxGateway.Worker/Ipc/WorkerFrameWriter.cs`, `.../WorkerFrameWritePriority.cs`, `.../WorkerPipeSession.cs` (confirmed two-class scheduler, WRK-21/23/28/30 landed, WRK-25/WRK-22 not landed), `src/ZB.MOM.WW.MxGateway.Worker/WorkerApplication.cs` (exit-code mapping), `src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessEventQueue.cs` (overflow fault path), `src/ZB.MOM.WW.MxGateway.Server/Dashboard/Hubs/DashboardEventBroadcaster.cs` + `Configuration/DashboardOptions.cs` + `docs/GatewayDashboardDesign.md:170` (ShowTagValues), `src/ZB.MOM.WW.MxGateway.Server/Grpc/MxAccessGatewayService.cs:940-963` + `Workers/WorkerClient.cs:205-244` + `Workers/WorkerClientErrorCode.cs` (CommandTooLarge mapping), `clients/rust/build.rs`, `clients/rust/Cargo.toml`, `scripts/check-codegen.ps1`, `scripts/pack-clients.ps1` (Rust vendoring), `gateway.md:326-360` (sequence-prose cross-check). Verification (greps, doc-only — no build required): `grep -n 'Reserved' docs/GatewayConfiguration.md` no longer matches the `ShowTagValues` row; `grep -n 'faults' docs/MxAccessWorkerInstanceDesign.md` shows no remaining five-level list; `grep -n 'scheduling' docs/WorkerFrameProtocol.md` finds the new section; `grep -rn 'Java 21' docs/style-guides/` empty; `grep -i vendored docs/ClientPackaging.md clients/rust/README.md` non-empty in both; `grep -n 'CommandTooLarge' docs/Grpc.md` shows the mapping. | | 2026-08-07 | **IPC-23 + IPC-24 + IPC-25 + IPC-32 → `Done`; GWC-25 deferred proto-comment resolved** (branch `fix/ipc-24-25-codegen`). The proto-comment/doc + regen wave for the P0 codegen-freshness cluster. **Proto comments (comment-only):** `mxaccess_worker.proto` `GatewayHello.max_frame_bytes` gained the every-frame-must-fit / reply-builders-truncate sentence (IPC-23); `mxaccess_gateway.proto` `DrainEventsReply` gained the count-cap+byte-cap / drain-until-empty comment (IPC-23); `mxaccess_gateway.proto` `ReplayGap.oldest_available_sequence` gained the empty-ring value definition (`highest observed + 1`, `oldest − 1` formula stays valid) resolving GWC-25's deferred amendment. **Regen wave:** `Contracts/Generated/{MxaccessGateway,MxaccessWorker}.cs` (XML-doc only, no descriptor delta), `clients/rust/protos/{mxaccess_gateway,mxaccess_worker}.proto` byte-copied, Go `internal/generated/{mxaccess_gateway,mxaccess_worker}.pb.go` (worker binding was genuinely stale — it lacked the whole `MaxFrameBytes` field/accessor/rawDesc, now refreshed; gateway.pb.go got the new doc comments), Python `generated/mxaccess_worker_pb2.py` (real descriptor delta — `max_frame_bytes` field + cascaded offsets), Java aggregates `MxaccessGateway.java`/`MxaccessWorker.java` (javadoc from the new proto comments — **zero** protobuf-version churn under the pinned toolchain), and the client descriptor set `mxaccessgw-client-v1.protoset`. **IPC-24:** regenerating Java on the pinned toolchain (grpc 1.76.0 / protobuf 4.33.1, Homebrew JDK 17) produced only the legitimate javadoc delta with no `validateProtobufGencodeVersion`/`major=/minor=/patch=` churn — so the preferred path was taken: the unconditional churn-revert step and its comment were **deleted** from `.gitea/workflows/ci.yml` (the `git diff --exit-code` gate is now a true message-level drift gate for the single-file Java aggregates). **IPC-25:** pinned the Go generators in `clients/go/generate-proto.ps1` (`protoc-gen-go` v1.36.11 assert, `protoc-gen-go-grpc` 1.6.2 assert, protoc warn-only; also fixed a latent `(if …)`→`$(if …)` parse bug that broke the script under pwsh 7); added **Check 4** to `scripts/check-codegen.ps1` (regenerate Go+Python bindings, fail on any diff, tool-missing FAILS not skips) and relabeled the banners `1/4`…`4/4` + header comment (**IPC-32** folded in); added the pinned-generator installs to the `portable` CI job before the codegen step. **Docs same commit:** `ClientProtoGeneration.md` (pinned-versions table + Go/Python Check-4 guard + Java-revert-deleted note), `Contracts.md` (four-check enumeration), `GatewayTesting.md` (java job + four-check prose), `clients/java/zb-mom-ww-mxgateway-client/build.gradle` checkGeneratedClean caveat. **IPC-23 DrainEvents-truncation doc deviation:** `docs/Grpc.md` has no DrainEvents row (it documents only the seven public RPCs; DrainEvents is a worker diagnostic), and `docs/WorkerFrameProtocol.md` (owned by the parallel doc batch) already carries the byte-cap/drain-until-empty prose via WRK-21 — so no Grpc.md edit was made. **Evidence (macOS):** `pwsh scripts/check-codegen.ps1` all four checks green (banners `1/4`…`4/4`); `dotnet build …NonWindows.slnx` 0 warnings/0 errors; `dotnet test …Tests --filter ~ClientProtoInputTests` green; `grep -c max_frame_bytes` → Go worker 2, Python worker 1 (both non-zero, were 0); `clients/go` gofmt clean + `go build` + `go test` ok; `clients/python` `pytest` 163 passed/1 skipped; Java `gradle generateProto` clean under JDK 17. **Negative-path proof:** added a scratch field to `mxaccess_worker.proto`, regenerated only `Contracts/Generated/`, reran `check-codegen.ps1` and confirmed **Check 4 fails naming both `clients/go/internal/generated` and `clients/python/…/generated`**, then reverted. **Not published** (CLI-39 gates Go/Python republish). | +| 2026-08-07 | **CLI-39 → `Done`** (branch `fix/cli-39-version-train`). The last client task in the P1 release train — lands after CLI-35/36/37/38/40/41/44/45 so the next publish carries the conformant behavior, not the pre-remediation API. Bumped all five client version constants off the already-published 0.1.2: Rust `Cargo.toml` `[package]` + `[workspace.package]` → 0.2.0 (`crates/mxgw-cli` inherits via `version.workspace = true`; `CLIENT_VERSION` already derived via `env!("CARGO_PKG_VERSION")` from CLI-29, no separate edit needed); Python `pyproject.toml` + `version.py` → 0.2.0, plus a new `test_version_matches_pyproject_toml` (`clients/python/tests/test_cli.py`) that parses `pyproject.toml` with `tomllib` and asserts `__version__` matches it — closes the CLI-26 residual (the old test only checked self-consistency against `__version__`, so the two literals could still drift from each other undetected); Go `mxgateway/version.go` `ClientVersion` → 0.2.0; .NET `ZB.MOM.WW.MxGateway.Client.csproj` `` → 0.2.0. **Java is the one exception, landed at 0.2.1, not 0.2.0**: queried the live Gitea package API (`GET /api/v1/packages/dohertj2/maven/com.zb.mom.ww.mxgateway:zb-mom-ww-mxgateway-client/0.2.0` — reachable and credentialed in this environment) and confirmed Java 0.2.0 was already published on 2026-06-26, before the CLI-37/38/40/41 conformance fixes changed the client's observable behavior; reusing 0.2.0 for the now-conformant build would have labeled two different APIs identically, so `build.gradle`'s `subprojects { version = '0.2.1' }`, the hand-maintained `MxGatewayClientVersion.CLIENT_VERSION` constant, and the two `MxGatewayCliTests` version assertions all moved to 0.2.1 together (`clients/java/README.md`'s Maven dependency example updated to match — it had drifted to a stale `0.1.2` example). Confirmed via the same live API that Python (`pypi`) and Rust (`cargo`) had only ever reached 0.1.1 in the registry (0.1.2 was set in source but never actually published), while `nuget` (`ZB.MOM.WW.MxGateway.Client`/`.Contracts`) reached 0.1.2 — recorded as historical context in the new "Versioning" section of `docs/ClientPackaging.md` rather than left implicit. **Publish-pipeline guards:** `scripts/tag-go-module.ps1` gained the CLI-21 guard that was designed but never implemented — after semver validation it now reads `clients/go/mxgateway/version.go`'s `ClientVersion` and throws if it does not equal the requested tag version (stripped of `v`), before touching git. `scripts/pack-clients.ps1` gained a Gitea package-registry collision guard (`Test-GiteaPackageExists`/`Assert-GiteaPackageNotPublished`, hitting the documented `GET /api/v1/packages/{owner}/{type}/{name}/{version}` package API) wired into all four artifact-producing `-Publish` blocks (.NET nupkg identity parsed from the packed filename so both `Client` and `Contracts` are covered; Python from `pyproject.toml`; Rust from `Cargo.toml`; Java from `build.gradle`'s `group`+`version`, **required for a real check** since Gitea's Maven package identity is `groupId:artifactId`, not the bare artifact id — an early version of this guard used the bare id and would have silently never matched anything, caught by testing it live against the known-published 0.2.0 before landing); Go is unaffected (no artifact producer in this script; the version-match guard above is its collision protection). Every non-404 registry response (including a plain 401 from an unauthenticated call) is treated as "cannot verify" and aborts — the guard fails closed, never proceeds on ambiguity. **No `.proto` changes; no publish performed.** Docs same commit: `docs/ClientPackaging.md` gained a "Versioning" section (single-source-of-truth list per client, the bump-before-publish rule, the Java exception with evidence); `clients/dotnet/README.md`, `clients/rust/README.md`, `clients/go/README.md` (install example + `tag-go-module.ps1` example + guard behavior sentence), `clients/java/README.md` had their stale `0.1.1`/`0.1.2` example version strings updated to the new released numbers. **Verification** (macOS, all four non-Java languages against the live tree): `cargo check --workspace` clean (`zb-mom-ww-mxgateway-client v0.2.0`, `mxgw-cli v0.2.0`), `cargo test --workspace` version tests pass (`version_json_output_has_protocol_versions`); `python -m pytest` 164 passed / 1 skipped (was 163; the one new test); Go `gofmt -l .` clean, `go build ./...` clean, `go test ./...` ok; `dotnet build clients/dotnet/ZB.MOM.WW.MxGateway.Client.slnx` 0 warnings/0 errors, client tests 128 passed / 1 skipped (unchanged). `JAVA_HOME=/opt/homebrew/opt/openjdk@17 gradle test` BUILD SUCCESSFUL, CLI suite 51/51 passed including both version-string assertions now at `0.2.1`, **no generated-tree churn** to revert (no `.proto` changed). Script guards: `pwsh -NoProfile` dry-run of `tag-go-module.ps1` — mismatched version (`v1.2.3`) failed with the new guard message before touching git; matched version (`v0.2.0`) passed the guard and reached the pre-existing dirty-working-tree check (expected, uncommitted changes present) — proves the new check runs first and does not block a correct version. `pack-clients.ps1`'s guard functions were exercised directly (dot-sourced in isolation, no pack/publish side effects) against the **live** Gitea registry with real credentials already present in this environment (`GITEA_USERNAME`/`GITEA_TOKEN` from `~/.zshenv`): correctly refused on every known-published artifact (nuget Client 0.1.2, maven 0.2.0, pypi 0.1.1, cargo 0.1.1) and correctly passed on every unpublished target version (0.2.0/0.2.1 across all four). **No actual publish was attempted or performed** — the task explicitly forbids it, and the sandbox additionally refuses any Bash command that inlines `$GITEA_TOKEN`/`$GITEA_USERNAME` into the command text (a credential-use guard, not a git-safety one), so the live checks above ran only by having the script's own functions read the already-exported env vars internally rather than the invoking command line referencing them. | diff --git a/archreview/2026-07-12/remediation/50-clients.md b/archreview/2026-07-12/remediation/50-clients.md index a82835a..3ba76d1 100644 --- a/archreview/2026-07-12/remediation/50-clients.md +++ b/archreview/2026-07-12/remediation/50-clients.md @@ -20,7 +20,7 @@ Operating constraints carried from prior work: | CLI-36 | Medium | P0 | S | — | Done | Go CLI `stream-events` silently destroys the ReplayGap signal | | CLI-37 | Medium | P1 | M | CLI-38 | Done | Status-array validation must branch on `category` per the proto contract (4-vs-1 divergence) | | CLI-38 | Medium | P1 | S | — | Done | Align .NET/Go/Java on `hresult < 0` — lands prior CLI-08 and cures the design-doc drift | -| CLI-39 | Medium | P1 | S | CLI-35..38, CLI-45 | Not started | Bump client versions off the already-published 0.1.2 before the next publish; add registry-collision guard | +| CLI-39 | Medium | P1 | S | CLI-35..38, CLI-45 | Done | Bump client versions off the already-published 0.1.2 before the next publish; add registry-collision guard | | CLI-40 | Low | — | M | — | Done | Port the exact-secret credential scrub to Rust/Java/.NET | | CLI-41 | Low | — | M | — | Done | Uniform malformed-reply contract for AuthenticateUser/ArchestrAUserToId/AddBufferedItem | | CLI-42 | Low | P1 | S | — | Done | Document the vendored Rust proto layout (CLI-02's missing doc half) | diff --git a/clients/dotnet/README.md b/clients/dotnet/README.md index a7b4c16..ef62fb9 100644 --- a/clients/dotnet/README.md +++ b/clients/dotnet/README.md @@ -487,7 +487,7 @@ dotnet nuget add source https://gitea.dohertylan.com/api/packages/dohertj2/nuget Then add the package to your project: ````bash -dotnet add package ZB.MOM.WW.MxGateway.Client --version 0.1.1 +dotnet add package ZB.MOM.WW.MxGateway.Client --version 0.2.0 ```` The `ZB.MOM.WW.MxGateway.Contracts` package is pulled in transitively. diff --git a/clients/dotnet/ZB.MOM.WW.MxGateway.Client/ZB.MOM.WW.MxGateway.Client.csproj b/clients/dotnet/ZB.MOM.WW.MxGateway.Client/ZB.MOM.WW.MxGateway.Client.csproj index 2da0c6f..79bfdd9 100644 --- a/clients/dotnet/ZB.MOM.WW.MxGateway.Client/ZB.MOM.WW.MxGateway.Client.csproj +++ b/clients/dotnet/ZB.MOM.WW.MxGateway.Client/ZB.MOM.WW.MxGateway.Client.csproj @@ -19,7 +19,7 @@ true ZB.MOM.WW.MxGateway.Client - 0.1.2 + 0.2.0 .NET 10 gRPC client for the MxAccessGateway service. Provides typed wrappers, retry, and a lazy-browse walker over the Galaxy Repository hierarchy. README.md