dohertj2/mxaccessgw
1
0
Fork 0
Code Issues 3 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

Merge branch 'fix/archreview-p2' into main (P2 tier: completeness & polish)

Browse Source 
# Conflicts:
#	archreview/remediation/00-tracking.md
#	clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli/MxGatewayCliSecretRedactor.cs
#	clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli/MxGatewayClientCli.cs
#	src/ZB.MOM.WW.MxGateway.Worker.Tests/Ipc/WorkerFrameProtocolTests.cs
This commit is contained in:
Joseph Doherty
2026-07-12 22:12:41 -04:00
parent b86c6bb47f dca09e7e06
commit 4f5371fdec
91 changed files with 7680 additions and 681 deletions
Download Patch File Download Diff File Expand all files Collapse all files
archreview/remediation/00-tracking.md
+50 -36
View File
@@ -64,16 +64,16 @@ Full design + implementation for each row lives in the linked domain doc under i
| GWC-03 | High | P0 | S | — | Done | Documented sparse-array max-length bound is unimplemented |
| GWC-04 | Medium | P1 | M | — | Done | Full event channel stalls the worker read loop behind command replies |
| GWC-05 | Medium | — | S | — | Not started | Worker pipe created with no ACL / no CurrentUserOnly |
| GWC-06 | Medium | P2 | S | GWC-07,08 | Not started | Stopwatch allocated per streamed event |
| GWC-07 | Medium | P2 | S | GWC-06,08 | Not started | Every mapped event is deep-cloned |
| GWC-08 | Medium | P2 | M | GWC-06,07 | Not started | Pipe framing allocates per frame and writes twice |
| GWC-06 | Medium | P2 | S | GWC-07,08 | Done | Stopwatch allocated per streamed event |
| GWC-07 | Medium | P2 | S | GWC-06,08 | Done | Every mapped event is deep-cloned |
| GWC-08 | Medium | P2 | M | GWC-06,07 | Done | Pipe framing allocates per frame and writes twice |
| GWC-09 | Medium | — | M | — | Not started | Startup probe is a no-op; its retry pipeline can never retry |
| GWC-10 | Medium | — | S | — | Not started | Envelope `sequence` monotonicity specified but not enforced |
| GWC-11 | Low | — | S | — | Not started | `_workerClient` written under lock but read lock-free |
| GWC-12 | Low | — | S | — | Not started | `WorkerClient.DisposeAsync` not safe against double-dispose |
| GWC-13 | Low | — | M | — | Not started | Worker-ready wait is a 25 ms poll loop |
| GWC-14 | Low | P2 | M | — | Not started | Replay ring is a `LinkedList` with a node alloc per event |
| GWC-15 | Low | P2 | S | — | Not started | Per-event gauge reads `ChannelReader.Count` every event |
| GWC-14 | Low | P2 | M | — | Done | Replay ring is a `LinkedList` with a node alloc per event |
| GWC-15 | Low | P2 | S | — | Done | Per-event gauge reads `ChannelReader.Count` every event |
| GWC-16 | Low | — | S | — | Not started | `Invoke` resolves the session twice per command |
| GWC-17 | Low | — | M | — | Not started | Sessions layer throws `Grpc.Core.RpcException` (layering leak) |
| GWC-18 | Low | — | S | — | Not started | `GatewaySession` implements `DisposeAsync` without `IAsyncDisposable` |
@@ -92,16 +92,16 @@ Full design + implementation for each row lives in the linked domain doc under i
| WRK-03 | Medium | — | S | WRK-02 | Not started | Commands after shutdown starts are dropped with no reply |
| WRK-04 | Medium | — | M | — | Done | Envelope `sequence` can appear out of order on the wire |
| WRK-05 | Medium | — | M | — | Not started | One transient alarm-poll failure kills the whole session |
| WRK-06 | Medium | P2 | S | — | Not started | `MXSTATUS_PROXY` conversion reflects per field, per event |
| WRK-06 | Medium | P2 | S | — | Done | `MXSTATUS_PROXY` conversion reflects per field, per event |
| WRK-07 | Medium | P1 | M | WRK-04 | Done | Documented outbound write priority not implemented |
| WRK-08 | Low | — | S | — | Not started | Residual event queue discarded at graceful shutdown, undocumented |
| WRK-09 | Low | — | S | — | Not started | No `AppDomain.UnhandledException` hook |
| WRK-10 | Low | — | S | — | Not started | Top-level catch logs exception type but never message |
| WRK-11 | Low | P2 | S | — | Not started | Every accepted event is defensively cloned on enqueue |
| WRK-12 | Low | P2 | M | WRK-04 | Not started | No event batching per envelope; one flush per event; 25 ms poll |
| WRK-11 | Low | P2 | S | — | Done | Every accepted event is defensively cloned on enqueue |
| WRK-12 | Low | P2 | M | WRK-04 | Done | No event batching per envelope; one flush per event; 25 ms poll |
| WRK-13 | Low | — | S | — | Not started | Frame writer allocates a fresh buffer per frame; reader pools |
| WRK-14 | Low | — | M | — | Not started | Private-field naming split `_camelCase` vs `camelCase` |
| WRK-15 | Low | P2 | S | — | Not started | Doc drift: STA thread name and heartbeat-counter note |
| WRK-15 | Low | P2 | S | — | Done | Doc drift: STA thread name and heartbeat-counter note |
| WRK-16 | Low | — | S | — | Not started | Boilerplate duplication in IPC envelope/ctor overloads |
| WRK-17 | Low | — | S | — | Not started | Gateway death exits with wrong exit code (6 not 5) |
| WRK-18 | Low | — | S | — | Not started | Event-queue overflow exits as generic `UnexpectedFailure` |
@@ -116,23 +116,23 @@ Full design + implementation for each row lives in the linked domain doc under i
| IPC-02 | Medium | P1 | M | IPC-03 | Done | Worker max frame size hard-coded, cannot follow gateway config |
| IPC-03 | Medium | P1 | M | IPC-02 | Done | gRPC max == pipe max with zero headroom; oversized write faults whole session |
| IPC-04 | Medium | P1 | S | IPC-03 | Done | `DrainEvents max_events=0` packs entire queue into one reply frame |
| IPC-05 | Medium | P2 | S | — | Not started | Redundant deep copies on command and event hot paths |
| IPC-06 | Medium | P2 | S | — | Not started | `gateway.md` Worker Envelope sketch no longer matches the contract |
| IPC-07 | Medium | P2 | S | — | Not started | `docs/Grpc.md` says six RPCs; there are seven |
| IPC-05 | Medium | P2 | S | — | Done | Redundant deep copies on command and event hot paths |
| IPC-06 | Medium | P2 | S | — | Done | `gateway.md` Worker Envelope sketch no longer matches the contract |
| IPC-07 | Medium | P2 | S | — | Done | `docs/Grpc.md` says six RPCs; there are seven |
| IPC-08 | Medium | — | M | — | Not started | `WorkerCancel` defined and handled but never sent |
| IPC-09 | Medium | P1 | M | IPC-01 | Done | Known codegen fragilities have no in-repo guards |
| IPC-10 | Low | — | S | — | Not started | Envelope `sequence` is write-only; monotonicity never validated |
| IPC-11 | Low | — | S | — | Not started | No protocol version negotiation despite `supported_protocol_version` name |
| IPC-12 | Low | — | S | — | Not started | Gateway frame writer has no write lock; integrity rests on undocumented invariant |
| IPC-13 | Low | P2 | S | IPC-05 | Not started | Gateway writer serializes twice and issues two stream writes |
| IPC-14 | Low | P2 | S | IPC-05 | Not started | Gateway reader allocates fresh array per frame; worker rents from `ArrayPool` |
| IPC-15 | Low | P2 | M | — | Not started | Worker event delivery 25 ms poll with per-event write+flush, no batching |
| IPC-13 | Low | P2 | S | IPC-05 | Done | Gateway writer serializes twice and issues two stream writes |
| IPC-14 | Low | P2 | S | IPC-05 | Done | Gateway reader allocates fresh array per frame; worker rents from `ArrayPool` |
| IPC-15 | Low | P2 | M | — | Done | Worker event delivery 25 ms poll with per-event write+flush, no batching |
| IPC-16 | Low | — | S | — | N/A | Correlation id carried twice per reply (Info) |
| IPC-17 | Low | P2 | S | — | Not started | Two docs name the wrong Python generated-output directory |
| IPC-17 | Low | P2 | S | — | Done | Two docs name the wrong Python generated-output directory |
| IPC-18 | Low | — | S | IPC-01 | N/A | Generated-code hygiene verified good — preserve under change (positive) |
| IPC-19 | Low | P1 | S | IPC-01 | Done | Generated/-must-be-committed rule for net48 undocumented and unguarded |
| IPC-20 | Low | P1 | S | IPC-01 | Done | Descriptor `-Check` byte-compares source-info bytes; protoc-version-sensitive |
| IPC-21 | Low | P2 | S | IPC-06 | Not started | `gateway.md` presents unimplemented `Session` RPC as live |
| IPC-21 | Low | P2 | S | IPC-06 | Done | `gateway.md` presents unimplemented `Session` RPC as live |
| IPC-22 | Low | — | S | — | N/A | Public error-detail model stops at status codes plus prose (Info) |
### Security & dashboard — [40-security-dashboard.md](40-security-dashboard.md)
@@ -141,13 +141,13 @@ Full design + implementation for each row lives in the linked domain doc under i
|---|---|:-:|:-:|---|---|---|
| SEC-01 | Medium | P1 | M | — | Done | Windows-absolute default paths become relative files off-Windows |
| SEC-02 | Medium | P1 | S | — | Done | `AllowAnonymousLocalhost` satisfies AdminOnly, not just Viewer |
| SEC-03 | Medium | P2 | S | — | Not started | Dashboard cookie lost its `__Host-` prefix; four docs still promise it |
| SEC-03 | Medium | P2 | S | — | Done | Dashboard cookie lost its `__Host-` prefix; four docs still promise it |
| SEC-04 | Medium | P1 | S | SEC-03 | Done | `DisableLogin` has no production guard |
| SEC-05 | Medium | P1 | M | — | Done | Hub bearer tokens irrevocable for 30 min, carried in query string |
| SEC-06 | Medium | P1 | M | — | Done | LDAP plaintext-by-default with a committed service password |
| SEC-07 | Medium | P1 | S | — | Done | `QueryActiveAlarmsRequest` missing from scope resolver; tests mislabeled |
| SEC-08 | Medium | P1 | L | — | Done | Per-RPC SQLite read + `last_used_utc` write; no verification cache |
| SEC-09 | Medium | P2 | S | — | Not started | Dashboard design-doc `GroupToRole` sample now fails startup validation |
| SEC-09 | Medium | P2 | S | — | Done | Dashboard design-doc `GroupToRole` sample now fails startup validation |
| SEC-10 | Medium | P1 | L | — | Done | No API-key expiry |
| SEC-11 | Medium | P1 | M | SEC-08 | Done | No rate limiting or lockout on either auth surface |
| SEC-12 | Medium | P1 | M | — | Done | Dashboard Close/Kill bypass the canonical audit store |
@@ -160,15 +160,15 @@ Full design + implementation for each row lives in the linked domain doc under i
| SEC-19 | Low | — | S | — | Not started | Canonical audit store re-issues `CREATE TABLE` per write/read |
| SEC-20 | Low | P1 | S | — | Done | `session_id` metric tag is unbounded cardinality |
| SEC-21 | Low | — | M | — | Not started | Snapshot publisher works every second regardless of audience |
| SEC-22 | Low | P2 | M | — | Not started | `docs/Authentication.md` documents types no longer in this repo |
| SEC-22 | Low | P2 | M | — | Done | `docs/Authentication.md` documents types no longer in this repo |
| SEC-23 | Low | — | S | SEC-03,04 | Not started | Validator ignores `DisableLogin`/`AutoLoginUser`/`RequireHttpsCookie`/`CookieName` |
| SEC-24 | Low | — | S | SEC-04 | Not started | Effective-config view omits the riskiest dashboard flags |
| SEC-25 | Low | P2 | M | SEC-02 | Not started | Per-session EventsHub ACL is an acknowledged TODO |
| SEC-25 | Low | P2 | M | SEC-02 | Done | Per-session EventsHub ACL is an acknowledged TODO |
| SEC-26 | Low | — | M | — | Not started | Audit trail has no retention or pruning |
| SEC-27 | Low | — | S | — | Not started | Dashboard `GatewayStatus` is hardcoded Healthy |
| SEC-28 | Info | — | S | — | N/A | Positive security observations (preserve under change) |
| SEC-29 | Info | — | S | — | N/A | UI-stack rule verified compliant |
| SEC-30 | Info | P2 | S | SEC-13 | Not started | Value-logging feature is unwired end-to-end |
| SEC-30 | Info | P2 | S | SEC-13 | Done | Value-logging feature is unwired end-to-end |
### Clients — [50-clients.md](50-clients.md)
@@ -177,7 +177,7 @@ Full design + implementation for each row lives in the linked domain doc under i
| CLI-01 | High | P0 | M | — | Done | Go `Session.Events()` silently closes stream on 16-slot overflow |
| CLI-02 | High | P1 | M | — | Done | Rust crate unbuildable outside the repo (`build.rs` path + `--no-verify`) |
| CLI-03 | High | P0 | M | — | Done | Rust `invoke` never validates HRESULT / MXSTATUS_PROXY |
| CLI-04 | High | P2 | L | CLI-15 | Not started | Typed-command parity gap across all five clients |
| CLI-04 | High | P2 | L | CLI-15 | Done | Typed-command parity gap across all five clients |
| CLI-05 | Medium | — | S | — | Not started | .NET session cannot be re-attached to an existing session id |
| CLI-06 | Medium | — | S | — | Not started | .NET `DisposeAsync` blocks/throws on unreachable gateway |
| CLI-07 | Medium | — | S | — | Not started | .NET retry budget self-defeats on `DeadlineExceeded` |
@@ -185,25 +185,25 @@ Full design + implementation for each row lives in the linked domain doc under i
| CLI-09 | Medium | — | M | — | Not started | Go has no typed auth-error mapping (Unauthenticated vs PermissionDenied) |
| CLI-10 | Medium | — | M | — | Not started | Go uses deprecated `grpc.DialContext` + `grpc.WithBlock()` |
| CLI-11 | Medium | — | S | — | Not started | Go CLI cannot opt into strict TLS validation |
| CLI-12 | Medium | P2 | S | — | Not started | Java docs still say "Java 21" after the JDK 17 retarget |
| CLI-12 | Medium | P2 | S | — | Done | Java docs still say "Java 21" after the JDK 17 retarget |
| CLI-13 | Medium | — | M | — | Not started | Java event-stream buffer hardcoded 16, cancel-on-overflow |
| CLI-14 | Medium | — | S | — | Not started | Python default TLS is blocking TOFU pin with silent `localhost` SNI |
| CLI-15 | Medium | P2 | M | — | Not started | No client handles `ReplayGap` or offers a reconnect helper |
| CLI-16 | Medium | P2 | S | CLI-12 | Not started | `docs/ClientPackaging.md` drifted from Python naming and .NET `.slnx` |
| CLI-15 | Medium | P2 | M | — | Done | No client handles `ReplayGap` or offers a reconnect helper |
| CLI-16 | Medium | P2 | S | CLI-12 | Done | `docs/ClientPackaging.md` drifted from Python naming and .NET `.slnx` |
| CLI-17 | Medium | — | M | CLI-14 | Not started | TLS default posture inconsistent across the five clients |
| CLI-18 | Low | P2 | S | — | Not started | .NET csproj records no `<Version>` |
| CLI-18 | Low | P2 | S | — | Done | .NET csproj records no `<Version>` |
| CLI-19 | Low | — | S | — | Not started | .NET duplicate `InternalsVisibleTo` (AssemblyInfo + csproj) |
| CLI-20 | Low | — | S | CLI-17 | Not started | .NET `--tls` without CA installs accept-all callback |
| CLI-21 | Low | P2 | S | — | Not started | Go `ClientVersion = "0.1.0-dev"` stale vs tagged releases |
| CLI-21 | Low | P2 | S | — | Done | Go `ClientVersion = "0.1.0-dev"` stale vs tagged releases |
| CLI-22 | Low | — | S | — | Not started | Go `newCorrelationID` swallows `crypto/rand` error → empty id |
| CLI-23 | Low | — | S | — | Not started | Go nil-vs-empty bulk short-circuit asymmetry |
| CLI-24 | Low | — | S | — | Not started | Java `MxEventStream` single-consumer constraint undocumented |
| CLI-25 | Low | — | S | — | Not started | Java `close()` does not await channel termination |
| CLI-26 | Low | P2 | S | — | Not started | Python `version.py` (0.1.0) ≠ `pyproject.toml` (0.1.2) |
| CLI-26 | Low | P2 | S | — | Done | Python `version.py` (0.1.0) ≠ `pyproject.toml` (0.1.2) |
| CLI-27 | Low | — | S | — | Not started | Python `Session.close()` not concurrency-safe; synthesizes reply |
| CLI-28 | Low | — | S | — | Not started | Python circular-import workaround at end of `session.py` |
| CLI-29 | Low | P2 | S | — | Not started | Rust `CLIENT_VERSION` (0.1.0-dev) ≠ `Cargo.toml` (0.1.2) |
| CLI-30 | Low | — | S | CLI-04 | Not started | Rust has no `unregister` typed helper |
| CLI-29 | Low | P2 | S | — | Done | Rust `CLIENT_VERSION` (0.1.0-dev) ≠ `Cargo.toml` (0.1.2) |
| CLI-30 | Low | — | S | CLI-04 | Done | Rust has no `unregister` typed helper |
| CLI-31 | Low | — | M | — | Not started | Rust CLI is a single 2,699-line `main.rs` |
| CLI-32 | Low | — | S | — | Not started | Client-side bulk caps differ (.NET/Java unbounded) |
| CLI-33 | Low | — | S | CLI-01,13 | Not started | Per-language event backpressure semantics undocumented |
@@ -213,19 +213,19 @@ Full design + implementation for each row lives in the linked domain doc under i
| ID | Sev | Tier | Eff | Dep | Status | Title |
|---|---|:-:|:-:|---|---|---|
| TST-01 | High | P2 | L | TST-04 | Not started | Reconnect/replay has no e2e test and no client consumer |
| TST-01 | High | P2 | L | TST-04 | Done | Reconnect/replay has no e2e test and no client consumer |
| TST-02 | High | P0 | M | TST-04 | Done | Reconnect owner re-validation not implemented |
| TST-03 | High | P1 | M | — | Done | No CI exists |
| TST-04 | High | P2 | L | — | Not started | Session-resilience epic 16/28 tasks unfinished |
| TST-04 | High | P2 | L | — | Done | Session-resilience epic 16/28 tasks unfinished |
| TST-05 | Medium | P1 | S | TST-03 | Not started | Real-worker control/COM paths verified opt-in only |
| TST-06 | Medium | — | M | — | Not started | Dashboard live-data path untested |
| TST-07 | Medium | — | S | — | Not started | Real-clock sleeps with negative assertions are latent flakes |
| TST-08 | Medium | P1 | M | — | Done | Full-suite orphaned testhost processes (does not reproduce; doc de-stale) |
| TST-09 | Medium | — | M | — | Not started | Health checks cover only the auth store |
| TST-10 | Medium | — | M | — | Not started | Deployment/upgrade undocumented and hand-rolled |
| TST-11 | Medium | P2 | M | — | Not started | No version discipline on server; client versions drift |
| TST-11 | Medium | P2 | M | — | Done | No version discipline on server; client versions drift |
| TST-12 | Medium | P0 | S | — | Done | CLAUDE.md misstates default retention behaviour |
| TST-13 | Medium | P2 | S | — | Not started | gateway.md carries stale design-era sketches |
| TST-13 | Medium | P2 | S | — | Done | gateway.md carries stale design-era sketches |
| TST-14 | Medium | P2 | S | — | Not started | Repo-root working artifacts need triage |
| TST-15 | Medium | P2 | M | TST-04 | Not started | Dashboard EventsHub has no per-session ACL |
| TST-16 | Medium | — | S | — | Not started | `Dashboard:ShowTagValues` is a dead flag |
@@ -235,7 +235,7 @@ Full design + implementation for each row lives in the linked domain doc under i
| TST-20 | Low | — | S | — | Not started | E2E script papers over a real advise-without-consumer sharp edge |
| TST-21 | Low | — | S | — | Not started | Log rotation configured but minimal |
| TST-22 | Low | — | S | — | Not started | Config-shape JSON block omits documented keys |
| TST-23 | Low | P2 | S | — | Not started | Bidirectional `Session` RPC never built |
| TST-23 | Low | P2 | S | — | Done | Bidirectional `Session` RPC never built |
| TST-24 | Low | P2 | M | TST-03 | Not started | Client wire behaviour has no automated verification |
## Cross-cutting clusters
@@ -253,8 +253,22 @@ Findings the review flagged as one coordinated design pass — sequence them tog
| Date | Change |
|---|---|
| 2026-07-10 | **TST-15 design fleshed out** (still `Not started` — design only, not implementation): `docs/plans/2026-07-10-dashboard-session-acl-tst15.md`. Resolves the crux the deferral left open — the dashboard is LDAP-identity (Admin/Viewer) while sessions are API-key-owned (`OwnerKeyId`), two disjoint identity domains — via a **session tag** sourced from the owning API key (rides in the existing `ApiKeyConstraints` JSON blob, no SQLite migration). Admin-sees-all; Viewer may `SubscribeSession` iff `session.Tags ∩ viewer.GrantedTags ≠ ∅` (new `Dashboard:GroupToTag` map → hub-token tag claims); untagged sessions Admin-only by default (`Dashboard:UntaggedSessionVisibility`). Includes the enforcement path (`HubTokenPayload.Tags` + `IDashboardSessionAcl` gate at `SubscribeSession`), task breakdown (epic Tasks 16–19), test plan incl. live-LDAP, and rejected alternatives (client-supplied tag; group→key-id map). Tracker + `60-testing-docs-gaps.md` TST-15 section point at the doc. **TST-03 investigated:** the CI never ran because the repo had **zero registered Gitea Actions runners** (Actions is enabled; runs are created on push/PR/nightly but fail instantly with nothing to execute them). A Mac runner proved the pipeline executes but cannot clone — this Gitea hands runners the internal `http://gitea:3000` URL, reachable only by a runner co-located on the gitea Docker network. Fix = run a co-located runner on the Gitea host (recipe prepared, `scratchpad/gitea-runner/setup-gitea-host-runner.sh`); pending host access. TST-03 stays `In review`. |
| 2026-07-09 | **P2 Epic wrap — user decision: DEFER TST-15 + TST-24, close the epic.** Epic bucket result: 5 of 7 findings `Done` (CLI-15, CLI-04, CLI-30, TST-01, TST-04); **TST-15** and **TST-24** intentionally deferred to a follow-up (kept `Not started`, not `Won't fix` — they are gated, not rejected). **TST-15** (dashboard EventsHub per-session ACL) is epic Phase 4 — a real feature needing a new session-"tag" mechanism + dashboard group→tag config, not a mechanical fix; the `EventsHub` `TODO(per-session-acl)` stays, and the already-shipped **SEC-25** mitigation (tag *values* redacted from the dashboard mirror by default) means no sensitive payload leaks through the hub today regardless of the missing ACL — so deferring carries no value-leak risk. **TST-24** (per-client wire tests) depends on **TST-03** (CI), which is `In review` (YAML authored, never run on a Gitea runner) — no point wiring client tests into a pipeline that isn't live yet. Net P2: 35/38 `Done`; remaining = TST-15 (deferred feature), TST-24 (deferred, CI-gated), TST-14 (user deletes their own untracked gitignored `*-docs-*.md` files). |
| 2026-07-09 | P2 Epic — **Java client completes CLI-15 + CLI-04 locally** (commit `1cc0fa4`); **CLI-15, CLI-04, CLI-30, TST-01 all → `Done` (5/5 clients + server e2e)**. Java CLI-15: `MxEventStreamItem` record + `MxEventStream.nextItem()` (`isReplayGap()`/`replayGap()`/`event()`); existing `Iterator<MxEvent>` path unchanged, sentinel never swallowed. Java CLI-04: Phase 1 `adviseSupervisory`/`writeSecured`/`writeSecured2`/`authenticateUser`/`archestrAUserToId` + Phase 2 `addBufferedItem`/`setBufferedUpdateInterval`/`suspend`/`activate` (unregister already present) on `MxGatewaySession`, each through `invokeCommand` → `ensureProtocolSuccess`+`ensureMxAccessSuccess`; credentials scrubbed via `MxGatewaySecrets.redactCredentials` (tests assert absent from message/toString/CLI). `gradle test` 106/0 (58 client + 48 cli), no generated churn. Built locally with `JAVA_HOME=/opt/homebrew/opt/openjdk@17` — Java toolchain now works on the Mac (see prior note). Shared docs `ClientLibrariesDesign.md` + CLAUDE.md updated to "all five clients". **TST-01 → Done** (server e2e `fed0685` + all 5 client `ReplayGap` consumers). This closes session-resilience epic Phase 3 fully. |
| 2026-07-09 | P2 Epic Wave E2 — typed-command parity (commit `bde042b`). CLI-04 → `In progress` (4/5 clients; Java pending in the SAME session — see next note), CLI-30 → `Done`. Every parity-critical single-item command now has a typed session helper across .NET/Go/Rust/Python: Phase 1 `AdviseSupervisory`/`WriteSecured`/`WriteSecured2`/`AuthenticateUser`/`ArchestrAUserToId`, Phase 2 `AddBufferedItem`/`SetBufferedUpdateInterval`/`Suspend`/`Activate`, plus `Unregister` (CLI-30: Rust + .NET added; Go/Java/Python already had it — CLI-30 fully Done). Each wraps existing raw-command machinery (no new wire surface) and runs the client's MXAccess-level reply validation (`hresult < 0` + `MxStatusProxy`). MXAccess parity preserved (WriteSecured-before-authenticate surfaces the native failure). **Credentials route through each client's secret-redaction seam** (never in logs/exceptions/ToString/Debug/Display; each suite asserts a distinctive credential is absent from surfaced errors); new CLI subcommands take credentials via flag/env, never echoed. Verified per toolchain: .NET build clean 102 passed; Go gofmt/vet/build/test clean; Rust fmt/check/test/clippy clean; Python 145 passed. Shared doc: `ClientLibrariesDesign.md` "Typed Command Parity" section. |
| 2026-07-09 | **Java client toolchain now works locally on the Mac** (user-flagged + verified): homebrew `openjdk@17` (17.0.19), @21, @26 are installed (off PATH); `JAVA_HOME=/opt/homebrew/opt/openjdk@17 gradle test` → BUILD SUCCESSFUL. Retires the "Java client must be built on windev" constraint — the remaining Java halves of CLI-15 + CLI-04 are done locally this session (no windev batch). Memory `project_java_build_host` rewritten. |
| 2026-07-09 | P2 Epic — TST-04 (session-resilience epic governance) → `Done` (commit `ed3c6c6`, docs-only umbrella). Resolved the epic's shipped-vs-planned entanglement into three decisions: **Phase 3 finished** (Task 13=TST-02 owner-scoped attach P0, Task 15=TST-01 reconnect e2e, Task 14=CLI-15 4/5 clients w/ Java pending); **Phase 4 scoped** as TST-15 with the Viewer-default decision settled (admin-sees-all, Viewer strict per owned/granted session, matching TST-02's gRPC owner binding); **Phase 5 (orphan-worker reattach) DEFERRED, not planned** — the CLAUDE.md "gateway restart does not reattach orphan workers" invariant stands and the `EnableOrphanReattach` flag does not exist / must not be referenced. Updated `oldtasks.md`, the `tasks.json` mirror (per-task statuses + governance note), and the CLAUDE.md reconnect paragraph (clients consume ReplayGap; reattach deferred). The actionable slices carry their own verification: TST-01 (done), TST-02 (done, P0), TST-15 (pending, E3). |
| 2026-07-09 | P2 Epic Wave E1 — reconnect-replay: CLI-15 + TST-01 → `In progress` (both gated on the Java client, deferred to the windev batch). **CLI-15 (commit `0c6e5b3`)**: four clients now surface the gateway's `ReplayGap` reconnect sentinel as a distinct, typed, non-terminal signal (never synthesized, never swallowed) — .NET `MxEventStreamItem`/`StreamEventItemsAsync` (build clean, 87 passed), Go `EventResult.ReplayGap`+`IsReplayGap()` (build/vet/test clean), Rust `EventItem::ReplayGap` enum (`EventStream` now yields `Result<EventItem, Error>`; fmt/check/test/clippy clean), Python `ReplayGap` dataclass yielded from `stream_events` (131 passed). Shared docs: `ClientLibrariesDesign.md` non-goals reframed (reconnect-replay protocol is consumable; auto-reconnect stays a non-goal) + `CrossLanguageSmokeMatrix.md` resume-gap note. Resume contract uniform: `after_worker_sequence = oldest_available_sequence - 1`. **TST-01 server half (commit `fed0685`)**: `GatewayEndToEndReconnectReplayTests` (2 facts) proves the default-on reconnect protocol end-to-end over the real gRPC StreamEvents path via the fake worker — replay-tail-no-gap (capacity 16, mid-batch cursor) and stale-cursor-emits-ReplayGap-first (capacity 3, 6 events force eviction, cursor=1); single-subscriber detach→reconnect, ring survives detach. macOS-verified 2/2 (needs `TMPDIR=/tmp` — default macOS TMPDIR pushes the CoreFxPipe Unix-socket path past the 104-byte `sun_path` limit; Windows CI unaffected). No product bugs; server behavior matched the contract exactly. Remaining to close both: Java `ReplayGap` surface (windev) + Java client consumer for TST-01. |
| 2026-07-09 | P2 Wave B — worker event hot-path (commit `f61c816`), **windev-verified**. WRK-06 → `Done`: `MxStatusProxyConverter` caches the four resolved `FieldInfo` per status type in a static `ConcurrentDictionary` (the `GetField` metadata scan ran 4× per status per event on the STA path); `GetValue`+`Convert.ToInt32` still run per event (late-bound RCW); both exception messages byte-identical (missing-field via `ResolveField`, not cached on throw through `GetOrAdd`; null-value unchanged). WRK-11 → `Done`: `MxAccessEventQueue.Enqueue` takes ownership of the passed `MxEvent` (stamps `WorkerSequence`/`WorkerTimestamp` in place, no `Clone()`); audited all 3 callers (base/alarm event sinks + provider-mode handler) — each builds a fresh event, none reuse it; `MxAccessValueCache.Set` now deep-copies its retained `Value`/`SourceTimestamp`/`Statuses` so the cache snapshot never aliases the queue-owned (later serialized) event. WRK-12 → `Done`: `WorkerFrameWriter` coalesces the flush across a drained batch (write each frame, one `FlushAsync` after the batch, then complete all) — preserves the written+flushed completion contract; a burst of N events costs 1 flush not N; on failure the in-flight batch + queue all fail so no caller hangs. IPC-15 → `Done`: the multi-event `WorkerEnvelope` body stays unimplemented (wire still one event per `worker_event` frame); gateway.md Performance section now distinguishes the shipped flush-coalescing from that deferred additive-proto change. **Windev:** x86 Worker builds clean (fixed a Windows-only `TreatWarningsAsErrors` CS8604 in the value-cache null-guard that macOS can't surface); Worker.Tests **356 passed / 0 failed / 11 skipped** (+4 new: converter cache-reuse, queue ownership-transfer, value-cache snapshot independence, writer batch-flush-once); gateway Tests **815 passed / 1 failed** — the 1 is the known windev-environmental SelfSigned-SAN assertion (passes on macOS), and all pipe-harness tests (which also exercise the earlier G-frame gateway frame change + WRK-12 end-to-end over a real pipe) pass. |
| 2026-07-09 | P2 Wave B — quick Mac items (commit `ec6f82b`). TST-11 → `Done`: `src/Directory.Build.props` single-sources the .NET-side version (Server/Worker/Contracts/tests stamped 0.1.2, was SDK-default 1.0.0) and appends the git short SHA to InformationalVersion (`0.1.2+<sha>`, guarded so a git-less build still works); verified Server assembly stamps `0.1.2+579282f`, Server+Tests build clean. Kept 0.1.2 (matches Contracts + aligned Python/Rust/Go); converging all to a single 0.2.0 cadence left as a release decision. WRK-15 → `Done` (docs-only path, no x86 build): STA thread-name refs corrected to the actual `MxGateway.Worker.STA` in `docs/WorkerSta.md` + `docs/MxAccessWorkerInstanceDesign.md`, and the stale heartbeat-counter note fixed (CaptureHeartbeat populates queue depth + sequence from the live queue). TST-23 → `Done`: already resolved by the Wave A gateway.md edit — the bidi `Session` RPC now sits under a "Future work: not implemented" heading (no "best long-term shape" framing), consistent with the proto. **TST-14 left open by design**: its only concrete step is deleting the user's untracked, gitignored `*-docs-*.md` working files (zero repo impact) — surfaced to the user rather than deleting files not created here. |
| 2026-07-09 | P2 Wave B — dashboard/security (commit `e15c3cb`). SEC-25 → `Done` (near-term hardening only; full per-session EventsHub ACL stays deferred to roadmap item 12 / TST-15): `DashboardEventBroadcaster` redacts tag values from a deep clone of the event before mirroring to SignalR when `Dashboard:ShowTagValues` is false (default), so the hub seam cannot leak values regardless of the missing ACL. Clears `MxEvent.value` (the OnDataChange/OnWriteComplete/OperationComplete/OnBufferedDataChange bodies are empty discriminators — values ride in the top-level field, incl. buffered arrays) + the alarm body `current_value`/`limit_value`; source event never mutated (shared with gRPC path + replay ring; verified). Makes the formerly-dead `ShowTagValues` flag live for the mirror. `EventsHub` TODO(per-session-acl) kept + tied to roadmap item 12. Test `DashboardEventBroadcasterTests` (3). SEC-30 → `Done`: `docs/Diagnostics.md` trimmed to mark opt-in command-value logging NOT YET IMPLEMENTED (no `LogCommandValues` knob, `RedactCommandValue` unwired, no values logged); wiring deferred pending secured-bulk redaction (SEC-13), not wired now. Server build clean; Dashboard tests 152/152. |
| 2026-07-09 | P2 Wave B — gateway event/frame hot-path performance (8 findings). **Frame I/O (commit `baae59c`):** GWC-08/IPC-13 gateway `WorkerFrameWriter` serializes once into one ArrayPool-rented buffer (LE length prefix + payload) and issues a single write (was: second serialization via `ToByteArray`, separate prefix array, two writes); IPC-14 `WorkerFrameReader` rents the payload buffer from ArrayPool instead of `new byte[]` per frame, read/parse length-bounded, returned in finally. Wire format byte-identical, cross-checked against the worker writer/reader. **Event/command hot path (commit `5e2e40a`):** GWC-06 `StreamEvents` timing via `Stopwatch.GetTimestamp()`/`GetElapsedTime()` (no per-event Stopwatch alloc); GWC-07/IPC-05 `MapEvent` transfers ownership of the inner `MxEvent` instead of cloning (safe — single-consumer `WorkerEvent`, MapEvent runs once pre-fan-out, all downstream consumers read-only; verified no post-mapping mutation) and `CreateCommandEnvelope` drops its redundant second clone (`MapCommand` already isolated the graph); every load-bearing isolation clone kept; GWC-15 `grpc_stream_queue.depth` converted from a per-event push counter to an `ObservableGauge` summing registered channel sources only at scrape time (name/semantics unchanged). **Replay ring (commit `cf1b3d4`):** GWC-14 replaces the `LinkedList` (node alloc per retained event) with a preallocated circular `ReplayEntry[]` (head+count), preserving ascending order, capacity eviction, time trim, oldest-read/ReplayGap math, both query paths, and the lock. → all `Done`. Server build clean (0 warnings); per-cluster tests green (WorkerFrameProtocol 10/10 incl. new near-cap round-trip, EventStream/Metrics/Distributor/Mapper 62/62, Distributor/Replay 33/33 incl. 2 new wrap-around cases). Full-suite macOS checkpoint: 769 passed / 44 failed, all 44 the known named-pipe/fake-worker-harness UDS-104-char limit (0 failures in any touched area). **Pending: windev run** to exercise the frame I/O over a real pipe (the 44 pipe-harness tests are macOS-blind). |
| 2026-07-09 | P2 Wave B — SEC-03 (dashboard `__Host-` cookie prefix) → `Done` (commit `7d42c85`). Conditionally restore the prefix instead of a pure doc fix: `DashboardServiceCollectionExtensions` PostConfigure resolves the cookie name as explicit `MxGateway:Dashboard:CookieName` override → else `__Host-MxGatewayDashboard` when `SecurePolicy==Always` (`RequireHttpsCookie` true) → else the plain `MxGatewayDashboard` default; guard never applies `__Host-` without Secure (browsers drop it). `DashboardAuthenticationDefaults.SecureCookieName` added; plain name kept as the non-secure fallback. Five stale docs corrected to the conditional contract (gateway.md, GatewayProcessDesign, ImplementationPlanGateway, GatewayDashboardDesign, CLAUDE.md; GatewayConfiguration phrasing tightened). Test `DashboardCookieOptionsTests` asserts the name flips with `RequireHttpsCookie` and that an override wins. Server build clean; Dashboard tests 149/149. |
| 2026-07-09 | P2 Wave A (doc-drift sweep + client version alignment) via parallel agents, on `fix/archreview-p2` off `main`. **Version alignment (commit `4fe1917`):** CLI-18 (.NET `<Version>0.1.2</Version>`), CLI-21 (Go `ClientVersion` 0.1.0-dev→0.1.2), CLI-26 (Python `__version__`→0.1.2), CLI-29 (Rust `CLIENT_VERSION`→`env!(CARGO_PKG_VERSION)`) → `Done`; verified dotnet build + `go build`/`test` + `cargo check`/`test`/`clippy` + pytest 127 pass. **Doc-drift (commit `06e1046`):** IPC-06/07/21 + TST-13 (gateway.md envelope sketch → proto-as-truth, six→seven RPCs incl. `QueryActiveAlarms` in `docs/Grpc.md`, `Session` RPC marked not-implemented, stale sketches removed), SEC-09 (dashboard `GroupToRole` sample `Administrator` so it passes the validator), SEC-22 (`docs/Authentication.md` rewritten to the shipped `ZB.MOM.WW.Auth.ApiKeys`-package pipeline; 18 stale type names removed, grep-verified absent; named gateway types re-verified present), IPC-17 (wrong Python gen dir `mxgateway`→`zb_mom_ww_mxgateway` in CLAUDE.md + 3 docs), CLI-12 (Java 21→17), CLI-16 (`docs/ClientPackaging.md` reconciled with `.slnx`/Python package name/gradle project names) → `Done`. Docs-only claims verified against source. 13 findings closed. |
| 2026-07-09 | Initial tracking doc generated from the six domain remediation designs. All 153 findings `Not started`. |
| 2026-07-09 | P0 tier executed via parallel agents. GWC-01/02/03, TST-02, TST-12, CLI-01, CLI-03 → `Done` (NonWindows build clean; gateway 135/135 targeted tests pass, Go `go test ./...` + `-race` pass, Rust `test`/`clippy -D warnings` pass). |
| 2026-07-09 | P0 tier executed via parallel agents. GWC-01/02/03, TST-02, TST-12, CLI-01, CLI-03 → `Done` (NonWindows build clean; gateway 135/135 targeted tests pass, Go `go test ./...` + `-race` pass, Rust `test`/`clippy -D warnings` pass). |
| 2026-07-09 | Windows full-suite temp-file-lock flakiness (discovered via TST-08) → **fixed** (commit `11a716a`). Two shared-state parallel collisions that macOS never surfaces: (1) self-signed cert generation — `SelfSignedCertificateProvider` now stages the PFX in a unique `<path>.<guid>.tmp` (was a fixed name that concurrent/interrupted writers collided on); `TestHostEnvironmentInitializer` defaults the cert path to a per-process temp dir (so the suite never writes shared `ProgramData` or fights the deployed service); `GatewayTlsBootstrapTests` moved to a `DisableParallelization` collection so its process-global env-var mutation can't bleed into parallel tests. (2) `AuthStoreHealthCheckTests` — reuse the existing `TempDatabaseDirectory` helper (`SqliteConnection.ClearAllPools()` before delete) so the Microsoft.Data.Sqlite pool releases the temp `.db` handle. **windev-verified:** 3 consecutive full-suite runs all **793 passed / 2 failed, 0 new orphans** (was flaky 2–4 fails, count-varying). The 2 stable remaining failures are pre-existing windev-environmental (`SelfSignedCertificateProviderTests.GenerateCertificate_HasExpectedSansEkuAndValidity` machine-name/FQDN SAN assertion; `EventStreamServiceTests…TracksAggregateQueueDepth` timing) — both pass on macOS, unrelated to this change. |
| 2026-07-09 | P1 TST-08 (orphaned testhost) → `Done` by evidence: the claimed leak does **not** reproduce. Full `ZB.MOM.WW.MxGateway.Tests` suite exits cleanly on macOS (0 survivors) and on the Windows dev box (windev, `origin/main` worktree: 0 new `testhost` and 0 new `ZB.MOM.WW.MxGateway.Worker` processes after a full run, isolated by StartTime). Static audit confirms the prime-suspect fixtures already dispose deterministically (WorkerClient cancels its `_stopCts` + awaits read/write/heartbeat tasks with a 5 s timeout; GatewaySession disposes the client; E2E/harness fixtures use `await using`). CLAUDE.md "Source Update Workflow" updated to drop the stale "leaves orphaned testhost processes" rationale (filtered runs kept as a speed guideline). **Discovered separately (NOT TST-08, still open):** Windows-only temp-file-lock flakiness in ~4 full-host test classes — `AuthStoreHealthCheckTests` (Microsoft.Data.Sqlite connection-pool holds the temp `.db` handle after `await using`, so the finally `File.Delete` throws "used by another process"), `GatewayTlsBootstrapTests`/`DashboardCookieOptionsTests`/`DashboardHubsRegistrationTests` (a `gw.pfx.tmp` handle from `SelfSignedCertificateProvider`'s fixed-name `path+".tmp"` atomic-write racing teardown). Count varies run-to-run (2–4 fails); macOS never sees it (Unix allows deleting open files). Fix = `SqliteConnection.ClearAllPools()` before temp delete + a unique/guarded `.tmp` name; verify on windev. |
| 2026-07-09 | SEC-10 polish (gateway-side follow-up to the G-2 auth-lib core) → done. `apikey create-key` gains an optional `--expires` (absolute ISO-8601 UTC or relative `<N>d`/`<N>h`; omit = non-expiring, opt-in) threaded through `ApiKeyAdminCommand`/parser/runner into the library's `CreateKeyAsync(..., expiresUtc, ...)`; `list-keys` shows an expiry column + `active`/`expired`/`revoked` status. Dashboard API Keys page surfaces expiry: an `Expires` column (`Never` when unset) and a status badge reading `Expired` (red) / `Expiring` (≤7 days, amber) / `Revoked` / `Active` (`DashboardApiKeySummary.ExpiresUtc` projected in `DashboardSnapshotService`; `StatusBadge` maps the new states). Server build clean; targeted tests 32/32 (parser abs/relative/invalid + end-to-end past-expiry rejection via the live verifier) + dashboard 28/28. Docs: `docs/Authentication.md` (verification flow expiry step, CLI table + examples, dashboard badge). SEC-10 was already `Done` (core); this closes the tracked polish. |
archreview/remediation/60-testing-docs-gaps.md
+1 -1
View File
@@ -327,7 +327,7 @@ If TST-02's interim mitigation (flip retention off) is chosen instead of impleme
**Impact.** Acceptable for a single-tenant dashboard; wrong the moment `GroupToRole` admits low-trust viewers. It is the dashboard-side twin of the gRPC owner-revalidation gap (TST-02).
**Design.** Covered by epic Phase 4 (Tasks 16–19, TST-04). Introduce a role/scope that scopes a hub connection to permitted sessions, plus a hub-token session tag (Task 18). The open design decision (`oldtasks.md:52`) — Viewer default of admin-sees-all vs strict per-session — must be settled first. Recommendation: admin-sees-all, Viewer strictly scoped to sessions they own/are granted, matching TST-02's gRPC owner binding for consistency. Until Phase 4 lands, keep the TODO (it correctly documents the accepted single-tenant assumption); do not silently remove it.
**Design.** Fully fleshed out in `docs/plans/2026-07-10-dashboard-session-acl-tst15.md` (epic Phase 4, Tasks 16–19, TST-04). In brief: the dashboard authenticates LDAP users (Admin/Viewer) while sessions are API-key-owned (`OwnerKeyId`) — two disjoint identity domains — so the ACL needs a bridge: a **session tag** sourced from the owning API key (riding in the existing `ApiKeyConstraints` JSON blob, no SQLite migration). Admin sees all; a Viewer may `SubscribeSession` iff `session.Tags ∩ viewer.GrantedTags ≠ ∅` (granted via a new `Dashboard:GroupToTag` map, carried into the hub token as tag claims); untagged sessions are Admin-only by default. The Viewer-default decision (admin-sees-all vs strict) is settled there. Until Phase 4 lands, keep the TODO (it correctly documents the accepted single-tenant assumption); do not silently remove it.
**Implementation.** `Dashboard/Hubs/EventsHub.cs` (ACL check on group join), hub-token minting to carry the session tag, `Configuration/DashboardOptions.cs` for any group-to-tag config (Task 17). Tests: `...Tests/Gateway/Dashboard/` hub ACL cases incl. live-LDAP users (Task 19). Docs: `docs/Sessions.md`/`gateway.md` dashboard section document the ACL model; CLAUDE.md dashboard-auth paragraph.