From 4fe1917aa68b7f8af83d5088f4dcfc601e075a51 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Thu, 9 Jul 2026 14:43:03 -0400 Subject: [PATCH 01/25] fix(clients): align client version constants to package manifests (CLI-18/21/26/29) - .NET: add explicit 0.1.2 to client csproj (CLI-18) - Go: ClientVersion 0.1.0-dev -> 0.1.2 (CLI-21) - Python: __version__ 0.1.0 -> 0.1.2 to match pyproject.toml (CLI-26) - Rust: CLIENT_VERSION now sourced from env!(CARGO_PKG_VERSION) so it cannot drift from Cargo.toml (CLI-29) Verified: dotnet build clean; go build/test ok; cargo check/test/clippy clean; pytest 127 passed. P2 client version-drift cluster. Claude-Session: https://claude.ai/code/session_01DMXXvNuPekkkrTEyPNxEkW --- .../ZB.MOM.WW.MxGateway.Client.csproj | 1 + clients/go/mxgateway/version.go | 6 +++--- clients/python/src/zb_mom_ww_mxgateway/version.py | 2 +- clients/rust/src/version.rs | 5 +++-- 4 files changed, 8 insertions(+), 6 deletions(-) diff --git a/clients/dotnet/ZB.MOM.WW.MxGateway.Client/ZB.MOM.WW.MxGateway.Client.csproj b/clients/dotnet/ZB.MOM.WW.MxGateway.Client/ZB.MOM.WW.MxGateway.Client.csproj index 4df7458..2da0c6f 100644 --- a/clients/dotnet/ZB.MOM.WW.MxGateway.Client/ZB.MOM.WW.MxGateway.Client.csproj +++ b/clients/dotnet/ZB.MOM.WW.MxGateway.Client/ZB.MOM.WW.MxGateway.Client.csproj @@ -19,6 +19,7 @@ true ZB.MOM.WW.MxGateway.Client + 0.1.2 .NET 10 gRPC client for the MxAccessGateway service. Provides typed wrappers, retry, and a lazy-browse walker over the Galaxy Repository hierarchy. README.md + + 0.1.2 + + + + + + + + $(_StampedGitSha.Trim()) + + + -- 2.52.0 From 73ce824f6c6476d8d5d345bebbefbff9e34b6ebb Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Thu, 9 Jul 2026 15:33:52 -0400 Subject: [PATCH 13/25] docs(archreview): mark TST-11/WRK-15/TST-23 Done (P2 quick items) Claude-Session: https://claude.ai/code/session_01DMXXvNuPekkkrTEyPNxEkW --- archreview/remediation/00-tracking.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/archreview/remediation/00-tracking.md b/archreview/remediation/00-tracking.md index 8151006..8a6214d 100644 --- a/archreview/remediation/00-tracking.md +++ b/archreview/remediation/00-tracking.md @@ -101,7 +101,7 @@ Full design + implementation for each row lives in the linked domain doc under i | WRK-12 | Low | P2 | M | WRK-04 | Not started | No event batching per envelope; one flush per event; 25 ms poll | | WRK-13 | Low | — | S | — | Not started | Frame writer allocates a fresh buffer per frame; reader pools | | WRK-14 | Low | — | M | — | Not started | Private-field naming split `_camelCase` vs `camelCase` | -| WRK-15 | Low | P2 | S | — | Not started | Doc drift: STA thread name and heartbeat-counter note | +| WRK-15 | Low | P2 | S | — | Done | Doc drift: STA thread name and heartbeat-counter note | | WRK-16 | Low | — | S | — | Not started | Boilerplate duplication in IPC envelope/ctor overloads | | WRK-17 | Low | — | S | — | Not started | Gateway death exits with wrong exit code (6 not 5) | | WRK-18 | Low | — | S | — | Not started | Event-queue overflow exits as generic `UnexpectedFailure` | @@ -223,7 +223,7 @@ Full design + implementation for each row lives in the linked domain doc under i | TST-08 | Medium | P1 | M | — | Done | Full-suite orphaned testhost processes (does not reproduce; doc de-stale) | | TST-09 | Medium | — | M | — | Not started | Health checks cover only the auth store | | TST-10 | Medium | — | M | — | Not started | Deployment/upgrade undocumented and hand-rolled | -| TST-11 | Medium | P2 | M | — | Not started | No version discipline on server; client versions drift | +| TST-11 | Medium | P2 | M | — | Done | No version discipline on server; client versions drift | | TST-12 | Medium | P0 | S | — | Done | CLAUDE.md misstates default retention behaviour | | TST-13 | Medium | P2 | S | — | Done | gateway.md carries stale design-era sketches | | TST-14 | Medium | P2 | S | — | Not started | Repo-root working artifacts need triage | @@ -235,7 +235,7 @@ Full design + implementation for each row lives in the linked domain doc under i | TST-20 | Low | — | S | — | Not started | E2E script papers over a real advise-without-consumer sharp edge | | TST-21 | Low | — | S | — | Not started | Log rotation configured but minimal | | TST-22 | Low | — | S | — | Not started | Config-shape JSON block omits documented keys | -| TST-23 | Low | P2 | S | — | Not started | Bidirectional `Session` RPC never built | +| TST-23 | Low | P2 | S | — | Done | Bidirectional `Session` RPC never built | | TST-24 | Low | P2 | M | TST-03 | Not started | Client wire behaviour has no automated verification | ## Cross-cutting clusters @@ -253,6 +253,7 @@ Findings the review flagged as one coordinated design pass — sequence them tog | Date | Change | |---|---| +| 2026-07-09 | P2 Wave B — quick Mac items (commit `ec6f82b`). TST-11 → `Done`: `src/Directory.Build.props` single-sources the .NET-side version (Server/Worker/Contracts/tests stamped 0.1.2, was SDK-default 1.0.0) and appends the git short SHA to InformationalVersion (`0.1.2+`, guarded so a git-less build still works); verified Server assembly stamps `0.1.2+579282f`, Server+Tests build clean. Kept 0.1.2 (matches Contracts + aligned Python/Rust/Go); converging all to a single 0.2.0 cadence left as a release decision. WRK-15 → `Done` (docs-only path, no x86 build): STA thread-name refs corrected to the actual `MxGateway.Worker.STA` in `docs/WorkerSta.md` + `docs/MxAccessWorkerInstanceDesign.md`, and the stale heartbeat-counter note fixed (CaptureHeartbeat populates queue depth + sequence from the live queue). TST-23 → `Done`: already resolved by the Wave A gateway.md edit — the bidi `Session` RPC now sits under a "Future work: not implemented" heading (no "best long-term shape" framing), consistent with the proto. **TST-14 left open by design**: its only concrete step is deleting the user's untracked, gitignored `*-docs-*.md` working files (zero repo impact) — surfaced to the user rather than deleting files not created here. | | 2026-07-09 | P2 Wave B — dashboard/security (commit `e15c3cb`). SEC-25 → `Done` (near-term hardening only; full per-session EventsHub ACL stays deferred to roadmap item 12 / TST-15): `DashboardEventBroadcaster` redacts tag values from a deep clone of the event before mirroring to SignalR when `Dashboard:ShowTagValues` is false (default), so the hub seam cannot leak values regardless of the missing ACL. Clears `MxEvent.value` (the OnDataChange/OnWriteComplete/OperationComplete/OnBufferedDataChange bodies are empty discriminators — values ride in the top-level field, incl. buffered arrays) + the alarm body `current_value`/`limit_value`; source event never mutated (shared with gRPC path + replay ring; verified). Makes the formerly-dead `ShowTagValues` flag live for the mirror. `EventsHub` TODO(per-session-acl) kept + tied to roadmap item 12. Test `DashboardEventBroadcasterTests` (3). SEC-30 → `Done`: `docs/Diagnostics.md` trimmed to mark opt-in command-value logging NOT YET IMPLEMENTED (no `LogCommandValues` knob, `RedactCommandValue` unwired, no values logged); wiring deferred pending secured-bulk redaction (SEC-13), not wired now. Server build clean; Dashboard tests 152/152. | | 2026-07-09 | P2 Wave B — gateway event/frame hot-path performance (8 findings). **Frame I/O (commit `baae59c`):** GWC-08/IPC-13 gateway `WorkerFrameWriter` serializes once into one ArrayPool-rented buffer (LE length prefix + payload) and issues a single write (was: second serialization via `ToByteArray`, separate prefix array, two writes); IPC-14 `WorkerFrameReader` rents the payload buffer from ArrayPool instead of `new byte[]` per frame, read/parse length-bounded, returned in finally. Wire format byte-identical, cross-checked against the worker writer/reader. **Event/command hot path (commit `5e2e40a`):** GWC-06 `StreamEvents` timing via `Stopwatch.GetTimestamp()`/`GetElapsedTime()` (no per-event Stopwatch alloc); GWC-07/IPC-05 `MapEvent` transfers ownership of the inner `MxEvent` instead of cloning (safe — single-consumer `WorkerEvent`, MapEvent runs once pre-fan-out, all downstream consumers read-only; verified no post-mapping mutation) and `CreateCommandEnvelope` drops its redundant second clone (`MapCommand` already isolated the graph); every load-bearing isolation clone kept; GWC-15 `grpc_stream_queue.depth` converted from a per-event push counter to an `ObservableGauge` summing registered channel sources only at scrape time (name/semantics unchanged). **Replay ring (commit `cf1b3d4`):** GWC-14 replaces the `LinkedList` (node alloc per retained event) with a preallocated circular `ReplayEntry[]` (head+count), preserving ascending order, capacity eviction, time trim, oldest-read/ReplayGap math, both query paths, and the lock. → all `Done`. Server build clean (0 warnings); per-cluster tests green (WorkerFrameProtocol 10/10 incl. new near-cap round-trip, EventStream/Metrics/Distributor/Mapper 62/62, Distributor/Replay 33/33 incl. 2 new wrap-around cases). Full-suite macOS checkpoint: 769 passed / 44 failed, all 44 the known named-pipe/fake-worker-harness UDS-104-char limit (0 failures in any touched area). **Pending: windev run** to exercise the frame I/O over a real pipe (the 44 pipe-harness tests are macOS-blind). | | 2026-07-09 | P2 Wave B — SEC-03 (dashboard `__Host-` cookie prefix) → `Done` (commit `7d42c85`). Conditionally restore the prefix instead of a pure doc fix: `DashboardServiceCollectionExtensions` PostConfigure resolves the cookie name as explicit `MxGateway:Dashboard:CookieName` override → else `__Host-MxGatewayDashboard` when `SecurePolicy==Always` (`RequireHttpsCookie` true) → else the plain `MxGatewayDashboard` default; guard never applies `__Host-` without Secure (browsers drop it). `DashboardAuthenticationDefaults.SecureCookieName` added; plain name kept as the non-secure fallback. Five stale docs corrected to the conditional contract (gateway.md, GatewayProcessDesign, ImplementationPlanGateway, GatewayDashboardDesign, CLAUDE.md; GatewayConfiguration phrasing tightened). Test `DashboardCookieOptionsTests` asserts the name flips with `RequireHttpsCookie` and that an override wins. Server build clean; Dashboard tests 149/149. | -- 2.52.0 From f61c816acf74c4e2dc7ea8588f43caaa485139c5 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Thu, 9 Jul 2026 15:52:49 -0400 Subject: [PATCH 14/25] perf(worker): event hot-path allocation + flush cuts (WRK-06/11/12, IPC-15) WRK-06: MxStatusProxyConverter caches the four resolved FieldInfo per status type in a static ConcurrentDictionary (the GetField metadata scan ran 4x per status per event on the STA path). GetValue+Convert.ToInt32 still run per event (late-bound RCW). Exceptions byte-identical: missing-field message unchanged (ResolveField, not cached on throw via GetOrAdd); null-value message unchanged. WRK-11: MxAccessEventQueue.Enqueue takes ownership of the passed MxEvent - stamps WorkerSequence/WorkerTimestamp on it in place and enqueues it, no Clone(). Audited all 3 callers (base/alarm event sinks, provider-mode handler): each builds a fresh event per Enqueue, none reuse it. MxAccessValueCache.Set now deep-copies its retained Value/SourceTimestamp/Statuses so the cache snapshot never aliases the queue-owned (later serialized) event. Net: alarm/other events clone nothing (was full clone); data-change clones payload-only. WRK-12: WorkerFrameWriter coalesces the flush across a drained batch - each frame is written but not flushed individually; one FlushAsync after the batch, then all written frames complete. Preserves the written+flushed completion contract; a burst of N events costs 1 flush, not N. On write failure the whole in-flight batch + queue fail so no caller hangs. IPC-15 (doc): the multi-event WorkerEnvelope body remains unimplemented (wire still carries one event per worker_event frame); gateway.md Performance section now distinguishes the shipped flush-coalescing from that deferred proto change. net48-safe (no init/records; readonly struct cache entry). Worker builds x86 only - verification on windev. Tests added: converter cache-reuse, queue ownership-transfer, value-cache snapshot independence, writer batch-flush-once. Claude-Session: https://claude.ai/code/session_01DMXXvNuPekkkrTEyPNxEkW --- gateway.md | 7 +- .../Conversion/MxStatusProxyConverterTests.cs | 29 ++++++ .../Ipc/WorkerFrameProtocolTests.cs | 46 +++++++++ .../MxAccess/MxAccessEventQueueTests.cs | 20 ++++ .../MxAccess/MxAccessValueCacheTests.cs | 31 ++++++ .../Conversion/MxStatusProxyConverter.cs | 95 ++++++++++++++++--- .../Ipc/WorkerFrameWriter.cs | 55 +++++++++-- .../MxAccess/MxAccessEventQueue.cs | 28 +++++- .../MxAccess/MxAccessValueCache.cs | 20 +++- 9 files changed, 303 insertions(+), 28 deletions(-) diff --git a/gateway.md b/gateway.md index 0a6c8d0..9f1e452 100644 --- a/gateway.md +++ b/gateway.md @@ -884,7 +884,12 @@ Baseline choices: Optimizations after parity: - batch commands where MXAccess semantics allow, -- batch events from worker to gateway while preserving order, +- batch events from worker to gateway while preserving order. Implemented so + far: the worker frame writer coalesces the *flush* across a drained batch of + event frames, so a burst costs one flush syscall rather than one per event + (WRK-12). Still not implemented: a multi-event `WorkerEnvelope` body that + packs several events into a single frame — the wire still carries one event + per `worker_event` frame (IPC-15), so this remains an additive-proto change, - optional data-change coalescing by item handle, - memory-mapped payload slabs for very large arrays, - shared schema for typed values to avoid raw COM marshaling at the gateway, diff --git a/src/ZB.MOM.WW.MxGateway.Worker.Tests/Conversion/MxStatusProxyConverterTests.cs b/src/ZB.MOM.WW.MxGateway.Worker.Tests/Conversion/MxStatusProxyConverterTests.cs index 4f732c3..4224573 100644 --- a/src/ZB.MOM.WW.MxGateway.Worker.Tests/Conversion/MxStatusProxyConverterTests.cs +++ b/src/ZB.MOM.WW.MxGateway.Worker.Tests/Conversion/MxStatusProxyConverterTests.cs @@ -106,6 +106,35 @@ public sealed class MxStatusProxyConverterTests Assert.Contains("success", exception.Message); } + /// + /// Verifies that repeated conversions of the same status type produce + /// identical results. WRK-06 caches the resolved FieldInfo objects per + /// type after the first conversion; the cached path must yield the same + /// message the uncached first call produced. + /// + [Fact] + public void Convert_RepeatedForSameType_ProducesIdenticalResults() + { + FakeMxStatusProxy status = new() + { + success = 0, + category = 3, + detectedBy = 1, + detail = 21, + }; + + // First call populates the per-type FieldInfo cache; the second reuses it. + MxStatusProxy first = _converter.Convert(status); + MxStatusProxy second = _converter.Convert(status); + + Assert.Equal(first, second); + Assert.Equal(MxStatusCategory.CommunicationError, second.Category); + Assert.Equal(MxStatusSource.RespondingLmx, second.DetectedBy); + Assert.Equal(0, second.Success); + Assert.Equal(21, second.Detail); + Assert.Equal("Invalid reference", second.DiagnosticText); + } + public struct FakeMxStatusProxy { public short success; diff --git a/src/ZB.MOM.WW.MxGateway.Worker.Tests/Ipc/WorkerFrameProtocolTests.cs b/src/ZB.MOM.WW.MxGateway.Worker.Tests/Ipc/WorkerFrameProtocolTests.cs index 8921762..c80b447 100644 --- a/src/ZB.MOM.WW.MxGateway.Worker.Tests/Ipc/WorkerFrameProtocolTests.cs +++ b/src/ZB.MOM.WW.MxGateway.Worker.Tests/Ipc/WorkerFrameProtocolTests.cs @@ -373,6 +373,43 @@ public sealed class WorkerFrameProtocolTests Assert.Equal(WorkerEnvelope.BodyOneofCase.WorkerEvent, frame3.BodyCase); } + /// + /// Verifies the writer coalesces the flush across a batch of frames drained together: four frames + /// queued behind an in-progress write drain in a single pass and share one FlushAsync, not four + /// (WRK-12 / IPC-15). Every frame still reaches the wire intact. + /// + /// A task that represents the asynchronous operation. + [Fact] + public async Task WriteAsync_WhenBatchDrainedTogether_FlushesOnce() + { + WorkerFrameProtocolOptions options = CreateOptions(); + using GatedWriteStream stream = new(); + WorkerFrameWriter writer = new(stream, options); + + // A blocked first write occupies the writer and holds the lock while more frames queue behind it. + Task firstWrite = writer.WriteAsync(CreateGatewayHelloEnvelope(), WorkerFrameWritePriority.Control); + await AwaitWithTimeoutAsync(stream.FirstWriteStarted); + + Task eventWrite1 = writer.WriteAsync(CreateEventEnvelope(), WorkerFrameWritePriority.Event); + Task eventWrite2 = writer.WriteAsync(CreateEventEnvelope(), WorkerFrameWritePriority.Event); + Task eventWrite3 = writer.WriteAsync(CreateEventEnvelope(), WorkerFrameWritePriority.Event); + await Task.Delay(50); + + stream.ReleaseFirstWrite(); + await AwaitWithTimeoutAsync(Task.WhenAll(firstWrite, eventWrite1, eventWrite2, eventWrite3)); + + // Four frames written in one drain pass => exactly one flush. + Assert.Equal(1, stream.FlushCount); + + stream.Position = 0; + WorkerFrameReader reader = new(stream, options); + for (int index = 0; index < 4; index++) + { + WorkerEnvelope frame = await reader.ReadAsync(); + Assert.NotEqual(WorkerEnvelope.BodyOneofCase.None, frame.BodyCase); + } + } + /// Verifies a zero negotiated frame maximum keeps the constructor default (IPC-02). [Fact] public void AdoptNegotiatedMaxMessageBytes_WithZero_KeepsDefault() @@ -451,9 +488,12 @@ public sealed class WorkerFrameProtocolTests private readonly TaskCompletionSource _firstWriteStarted = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); private int _writeCount; + private int _flushCount; public Task FirstWriteStarted => _firstWriteStarted.Task; + public int FlushCount => Volatile.Read(ref _flushCount); + public void ReleaseFirstWrite() => _release.Release(); public override async Task WriteAsync(byte[] buffer, int offset, int count, CancellationToken cancellationToken) @@ -467,6 +507,12 @@ public sealed class WorkerFrameProtocolTests await base.WriteAsync(buffer, offset, count, cancellationToken); } + public override Task FlushAsync(CancellationToken cancellationToken) + { + Interlocked.Increment(ref _flushCount); + return base.FlushAsync(cancellationToken); + } + protected override void Dispose(bool disposing) { if (disposing) diff --git a/src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/MxAccessEventQueueTests.cs b/src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/MxAccessEventQueueTests.cs index 27d684b..84f8e1d 100644 --- a/src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/MxAccessEventQueueTests.cs +++ b/src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/MxAccessEventQueueTests.cs @@ -29,6 +29,26 @@ public sealed class MxAccessEventQueueTests Assert.False(queue.TryDequeue(out _)); } + /// + /// Verifies that Enqueue takes ownership of the passed event instead of + /// cloning it: the dequeued instance is the very reference passed in, and + /// the worker sequence/timestamp are stamped on that same instance + /// (WRK-11). + /// + [Fact] + public void Enqueue_TakesOwnershipOfPassedEventInstance() + { + MxAccessEventQueue queue = new(capacity: 4); + MxEvent original = CreateEvent(MxEventFamily.OnDataChange, itemHandle: 10); + + queue.Enqueue(original); + + Assert.True(queue.TryDequeue(out WorkerEvent? dequeued)); + Assert.Same(original, dequeued?.Event); + Assert.Equal(1UL, original.WorkerSequence); + Assert.NotNull(original.WorkerTimestamp); + } + /// Verifies that Drain removes at most the requested number of events. [Fact] public void Drain_RemovesAtMostRequestedEvents() diff --git a/src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/MxAccessValueCacheTests.cs b/src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/MxAccessValueCacheTests.cs index 7e5cb4f..fded4a2 100644 --- a/src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/MxAccessValueCacheTests.cs +++ b/src/ZB.MOM.WW.MxGateway.Worker.Tests/MxAccess/MxAccessValueCacheTests.cs @@ -46,6 +46,37 @@ public sealed class MxAccessValueCacheTests Assert.Equal(999, other.Value.Int32Value); } + /// + /// Verifies that Set stores an independent deep-copied snapshot: mutating + /// the source event's protobuf sub-messages after caching does not alter + /// the cached value. WRK-11 stopped the event sink cloning before enqueue, + /// so the same MxEvent instance now flows to the outbound queue; the cache + /// must own its own copy so the two never share mutable state. + /// + [Fact] + public void Set_StoresIndependentSnapshot_UnaffectedByLaterEventMutation() + { + MxAccessValueCache cache = new(); + Timestamp sourceTimestamp = Timestamp.FromDateTime(new(2026, 5, 19, 9, 0, 0, DateTimeKind.Utc)); + MxEvent mxEvent = BuildEvent(serverHandle: 7, itemHandle: 21, intValue: 100, quality: 192, sourceTimestamp); + + cache.Set(7, 21, mxEvent); + + // Mutate the event in place after it was cached — as if it kept flowing + // through the (unrelated) outbound path. None of this must reach the cache. + mxEvent.Value.Int32Value = 999; + mxEvent.Quality = 0; + mxEvent.SourceTimestamp = Timestamp.FromDateTime(new(2030, 1, 1, 0, 0, 0, DateTimeKind.Utc)); + mxEvent.Statuses[0].Category = MxStatusCategory.SecurityError; + + Assert.True(cache.TryGet(7, 21, out MxAccessValueCache.CachedValue cached)); + Assert.Equal(100, cached.Value.Int32Value); + Assert.Equal(192, cached.Quality); + Assert.Equal(sourceTimestamp, cached.SourceTimestamp); + Assert.Single(cached.Statuses); + Assert.Equal(MxStatusCategory.Ok, cached.Statuses[0].Category); + } + /// Verifies that TryGet returns false for unknown handles. [Fact] public void TryGet_WithUnknownHandle_ReturnsFalse() diff --git a/src/ZB.MOM.WW.MxGateway.Worker/Conversion/MxStatusProxyConverter.cs b/src/ZB.MOM.WW.MxGateway.Worker/Conversion/MxStatusProxyConverter.cs index 8deff4d..036cd9c 100644 --- a/src/ZB.MOM.WW.MxGateway.Worker/Conversion/MxStatusProxyConverter.cs +++ b/src/ZB.MOM.WW.MxGateway.Worker/Conversion/MxStatusProxyConverter.cs @@ -1,4 +1,5 @@ using System; +using System.Collections.Concurrent; using System.Collections.Generic; using System.Globalization; using System.Reflection; @@ -9,6 +10,17 @@ namespace ZB.MOM.WW.MxGateway.Worker.Conversion; /// Converts MXAccess MXSTATUS_PROXY COM objects to protobuf MxStatusProxy messages. public sealed class MxStatusProxyConverter { + /// + /// Per-type cache of the four resolved objects a + /// status conversion needs. The status type is stable (the interop + /// MXSTATUS_PROXY struct in production; a fixed test double in + /// tests), so the expensive + /// metadata scan is resolved once per type and reused. Keyed by + /// so a plain-CLR test double and the real interop + /// struct each get their own entry, keeping the converter interop-agnostic. + /// + private static readonly ConcurrentDictionary FieldCache = new(); + /// Converts a single status object to a protobuf message, reflecting all fields and diagnostics. /// COM status object to convert. /// The converted protobuf status message. @@ -20,10 +32,11 @@ public sealed class MxStatusProxyConverter } Type statusType = status.GetType(); - int success = ReadInt32Field(status, statusType, "success"); - int rawCategory = ReadInt32Field(status, statusType, "category"); - int rawDetectedBy = ReadInt32Field(status, statusType, "detectedBy"); - int detail = ReadInt32Field(status, statusType, "detail"); + StatusFields fields = GetFields(statusType); + int success = ReadInt32Field(status, statusType, fields.Success); + int rawCategory = ReadInt32Field(status, statusType, fields.Category); + int rawDetectedBy = ReadInt32Field(status, statusType, fields.DetectedBy); + int detail = ReadInt32Field(status, statusType, fields.Detail); return new MxStatusProxy { @@ -82,6 +95,43 @@ public sealed class MxStatusProxyConverter private static int ReadInt32Field( object value, + Type valueType, + FieldInfo field) + { + object? fieldValue = field.GetValue(value); + if (fieldValue is null) + { + throw new MxStatusConversionException( + $"Status object field '{field.Name}' on type '{valueType.FullName}' is null."); + } + + return System.Convert.ToInt32(fieldValue, CultureInfo.InvariantCulture); + } + + /// + /// Resolves (and caches) the four objects for the + /// given status type. The first resolution for a type performs the + /// reflection scan; every subsequent conversion of that type reuses the + /// cached entry. A type missing a required field throws the same + /// the per-field lookup used to + /// throw — and, because + /// does not store a value when the factory throws, a bad type keeps + /// failing identically on every call rather than being cached. + /// + /// Runtime type of the status object being converted. + /// The resolved field set for . + private static StatusFields GetFields(Type statusType) + { + return FieldCache.GetOrAdd( + statusType, + type => new StatusFields( + ResolveField(type, "success"), + ResolveField(type, "category"), + ResolveField(type, "detectedBy"), + ResolveField(type, "detail"))); + } + + private static FieldInfo ResolveField( Type valueType, string fieldName) { @@ -92,14 +142,7 @@ public sealed class MxStatusProxyConverter $"Status object type '{valueType.FullName}' does not expose required field '{fieldName}'."); } - object? fieldValue = field.GetValue(value); - if (fieldValue is null) - { - throw new MxStatusConversionException( - $"Status object field '{fieldName}' on type '{valueType.FullName}' is null."); - } - - return System.Convert.ToInt32(fieldValue, CultureInfo.InvariantCulture); + return field; } private static MxStatusCategory MapCategory(int rawCategory) @@ -134,4 +177,32 @@ public sealed class MxStatusProxyConverter _ => MxStatusSource.Unknown, }; } + + /// + /// The four resolved status fields cached per type. Plain readonly struct + /// (not a record) so it compiles under the worker's net48 target, which + /// lacks IsExternalInit. + /// + private readonly struct StatusFields + { + public StatusFields( + FieldInfo success, + FieldInfo category, + FieldInfo detectedBy, + FieldInfo detail) + { + Success = success; + Category = category; + DetectedBy = detectedBy; + Detail = detail; + } + + public FieldInfo Success { get; } + + public FieldInfo Category { get; } + + public FieldInfo DetectedBy { get; } + + public FieldInfo Detail { get; } + } } diff --git a/src/ZB.MOM.WW.MxGateway.Worker/Ipc/WorkerFrameWriter.cs b/src/ZB.MOM.WW.MxGateway.Worker/Ipc/WorkerFrameWriter.cs index 5ae2e58..28734ad 100644 --- a/src/ZB.MOM.WW.MxGateway.Worker/Ipc/WorkerFrameWriter.cs +++ b/src/ZB.MOM.WW.MxGateway.Worker/Ipc/WorkerFrameWriter.cs @@ -112,36 +112,77 @@ public sealed class WorkerFrameWriter // Runs only under _writeLock. Drains control frames before event frames, stamping and writing each. // The stream write itself is not cancellable: a frame is written atomically or fails, never left // half-written on the pipe because a caller gave up waiting. + // + // Flushes are coalesced across the whole drained batch (WRK-12 / IPC-15): each frame is written to + // the stream but not flushed individually; a single FlushAsync runs after the batch, then every + // successfully-written frame is completed. A caller's Completion therefore still signals only after + // its bytes have been written AND flushed, so the "written and flushed" contract is unchanged — but + // a burst of N events now costs one flush syscall instead of N. private async Task DrainQueuedFramesAsync() { + List written = new List(); while (true) { PendingFrame? frame = DequeueNext(); if (frame is null) { - return; + break; } try { await WriteFrameAsync(frame.Envelope).ConfigureAwait(false); - frame.Completion.TrySetResult(true); + written.Add(frame); } catch (WorkerFrameProtocolException exception) when (IsPerFrameRejection(exception)) { // Validation, empty-payload, and oversized-frame errors are specific to this frame and - // do not damage the stream; fail only this frame and keep draining the rest. + // do not damage the stream; fail only this frame and keep draining the rest. Nothing was + // written for it, so it needs no flush. frame.Completion.TrySetException(exception); } catch (Exception exception) { - // A stream write/flush failure means the pipe is broken; fail this frame and every frame - // still queued so no caller awaits forever, then stop draining. + // A stream write failure means the pipe is broken; fail this frame, every frame already + // written this batch but not yet flushed, and every frame still queued so no caller + // awaits forever, then stop draining. frame.Completion.TrySetException(exception); + FailFrames(written, exception); FailAllQueued(exception); return; } } + + if (written.Count == 0) + { + return; + } + + try + { + await _stream.FlushAsync(CancellationToken.None).ConfigureAwait(false); + } + catch (Exception exception) + { + // The batch reached the stream but the flush that guarantees delivery failed: the pipe is + // broken. Fail every frame in the batch (the queue was already drained) so no caller treats + // an unflushed write as delivered. + FailFrames(written, exception); + return; + } + + foreach (PendingFrame frame in written) + { + frame.Completion.TrySetResult(true); + } + } + + private static void FailFrames(List frames, Exception exception) + { + foreach (PendingFrame frame in frames) + { + frame.Completion.TrySetException(exception); + } } private static bool IsPerFrameRejection(WorkerFrameProtocolException exception) @@ -211,14 +252,14 @@ public sealed class WorkerFrameWriter // Serialize once into a single buffer that carries the 4-byte length prefix followed by the // payload, then issue one stream write. This avoids a second serialization pass, a separate - // prefix array, and a separate prefix write. + // prefix array, and a separate prefix write. The flush is deferred to the end of the drained + // batch (see DrainQueuedFramesAsync) so a burst of frames shares one flush. int frameLength = sizeof(uint) + payloadLength; byte[] frame = new byte[frameLength]; WriteUInt32LittleEndian(frame, (uint)payloadLength); envelope.WriteTo(new Span(frame, sizeof(uint), payloadLength)); await _stream.WriteAsync(frame, 0, frameLength, CancellationToken.None).ConfigureAwait(false); - await _stream.FlushAsync(CancellationToken.None).ConfigureAwait(false); } private static void WriteUInt32LittleEndian( diff --git a/src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessEventQueue.cs b/src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessEventQueue.cs index 51fe92f..fe09401 100644 --- a/src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessEventQueue.cs +++ b/src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessEventQueue.cs @@ -8,6 +8,17 @@ namespace ZB.MOM.WW.MxGateway.Worker.MxAccess; /// /// Thread-safe queue for MxAccess events with capacity overflow and fault tracking. /// +/// +/// Ownership invariant: takes ownership of the +/// passed to it — it stamps the worker sequence and +/// timestamp on that same instance and enqueues it directly, without +/// cloning (WRK-11). Every caller must therefore pass a freshly built +/// that it does not retain, reuse, or mutate after the +/// call returns. All production callers (MxAccessBaseEventSink, +/// MxAccessAlarmEventSink, AlarmCommandHandler) build a new event per +/// Enqueue via the mapper and satisfy this; the value cache stores its own +/// independent snapshot (see ). +/// public sealed class MxAccessEventQueue { /// @@ -110,8 +121,11 @@ public sealed class MxAccessEventQueue /// /// Enqueues an MxAccess event, assigning a sequence number and timestamp. + /// Takes ownership of : the sequence and timestamp + /// are stamped on that same instance and it is enqueued without cloning, so + /// the caller must pass a freshly built event it does not reuse afterwards. /// - /// MXAccess event to enqueue. + /// Freshly built MXAccess event to enqueue; ownership transfers to the queue. public void Enqueue(MxEvent mxEvent) { if (mxEvent is null) @@ -132,13 +146,17 @@ public sealed class MxAccessEventQueue throw new MxAccessEventQueueOverflowException(capacity); } - MxEvent queuedEvent = mxEvent.Clone(); - queuedEvent.WorkerSequence = ++lastEventSequence; - queuedEvent.WorkerTimestamp = Timestamp.FromDateTime(DateTime.UtcNow); + // WRK-11: stamp the sequence/timestamp on the caller's own event and + // enqueue that same instance under the lock instead of cloning. See + // the ownership invariant on the class summary — the caller hands the + // event over exclusively, so a defensive Clone() here is pure + // overhead on the hottest path. + mxEvent.WorkerSequence = ++lastEventSequence; + mxEvent.WorkerTimestamp = Timestamp.FromDateTime(DateTime.UtcNow); WorkerEvent workerEvent = new() { - Event = queuedEvent, + Event = mxEvent, }; events.Enqueue(workerEvent); } diff --git a/src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessValueCache.cs b/src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessValueCache.cs index a4ba24d..f82429b 100644 --- a/src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessValueCache.cs +++ b/src/ZB.MOM.WW.MxGateway.Worker/MxAccess/MxAccessValueCache.cs @@ -40,6 +40,20 @@ public sealed class MxAccessValueCache throw new ArgumentNullException(nameof(mxEvent)); } + // WRK-11: the event sink no longer clones before enqueue, so the passed + // mxEvent is the very instance handed to the outbound queue. Deep-copy + // the value/timestamp/statuses payload we retain here so the cache's + // snapshot stays independent of the enqueued (and later serialized) + // event — the two must never share mutable protobuf sub-messages. + // Value is always set for OnDataChange; SourceTimestamp may be unset when + // the source timestamp could not be parsed, so both are cloned only when + // present. The null-forgiving result matches CachedValue's non-null- + // annotated parameters, which already accepted a runtime-null value or + // timestamp before WRK-11 (the ternary keeps the compiler's null-state + // from poisoning to maybe-null, which a plain null check would do). + MxValue cachedValue = mxEvent.Value is null ? null! : mxEvent.Value.Clone(); + Timestamp cachedTimestamp = mxEvent.SourceTimestamp is null ? null! : mxEvent.SourceTimestamp.Clone(); + long key = CreateItemKey(serverHandle, itemHandle); lock (syncRoot) { @@ -49,10 +63,10 @@ public sealed class MxAccessValueCache entries[key] = new CachedValue( nextVersion, - mxEvent.Value, + cachedValue, mxEvent.Quality, - mxEvent.SourceTimestamp, - mxEvent.Statuses); + cachedTimestamp, + mxEvent.Statuses.Clone()); } } -- 2.52.0 From c823a7b60be560c863f818fd692d272871fe0ae5 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Thu, 9 Jul 2026 16:00:29 -0400 Subject: [PATCH 15/25] docs(archreview): mark WRK-06/11/12 + IPC-15 Done (P2 worker hot-path, windev-verified) Claude-Session: https://claude.ai/code/session_01DMXXvNuPekkkrTEyPNxEkW --- archreview/remediation/00-tracking.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/archreview/remediation/00-tracking.md b/archreview/remediation/00-tracking.md index 8a6214d..85172d4 100644 --- a/archreview/remediation/00-tracking.md +++ b/archreview/remediation/00-tracking.md @@ -92,13 +92,13 @@ Full design + implementation for each row lives in the linked domain doc under i | WRK-03 | Medium | — | S | WRK-02 | Not started | Commands after shutdown starts are dropped with no reply | | WRK-04 | Medium | — | M | — | Done | Envelope `sequence` can appear out of order on the wire | | WRK-05 | Medium | — | M | — | Not started | One transient alarm-poll failure kills the whole session | -| WRK-06 | Medium | P2 | S | — | Not started | `MXSTATUS_PROXY` conversion reflects per field, per event | +| WRK-06 | Medium | P2 | S | — | Done | `MXSTATUS_PROXY` conversion reflects per field, per event | | WRK-07 | Medium | P1 | M | WRK-04 | Done | Documented outbound write priority not implemented | | WRK-08 | Low | — | S | — | Not started | Residual event queue discarded at graceful shutdown, undocumented | | WRK-09 | Low | — | S | — | Not started | No `AppDomain.UnhandledException` hook | | WRK-10 | Low | — | S | — | Not started | Top-level catch logs exception type but never message | -| WRK-11 | Low | P2 | S | — | Not started | Every accepted event is defensively cloned on enqueue | -| WRK-12 | Low | P2 | M | WRK-04 | Not started | No event batching per envelope; one flush per event; 25 ms poll | +| WRK-11 | Low | P2 | S | — | Done | Every accepted event is defensively cloned on enqueue | +| WRK-12 | Low | P2 | M | WRK-04 | Done | No event batching per envelope; one flush per event; 25 ms poll | | WRK-13 | Low | — | S | — | Not started | Frame writer allocates a fresh buffer per frame; reader pools | | WRK-14 | Low | — | M | — | Not started | Private-field naming split `_camelCase` vs `camelCase` | | WRK-15 | Low | P2 | S | — | Done | Doc drift: STA thread name and heartbeat-counter note | @@ -126,7 +126,7 @@ Full design + implementation for each row lives in the linked domain doc under i | IPC-12 | Low | — | S | — | Not started | Gateway frame writer has no write lock; integrity rests on undocumented invariant | | IPC-13 | Low | P2 | S | IPC-05 | Done | Gateway writer serializes twice and issues two stream writes | | IPC-14 | Low | P2 | S | IPC-05 | Done | Gateway reader allocates fresh array per frame; worker rents from `ArrayPool` | -| IPC-15 | Low | P2 | M | — | Not started | Worker event delivery 25 ms poll with per-event write+flush, no batching | +| IPC-15 | Low | P2 | M | — | Done | Worker event delivery 25 ms poll with per-event write+flush, no batching | | IPC-16 | Low | — | S | — | N/A | Correlation id carried twice per reply (Info) | | IPC-17 | Low | P2 | S | — | Done | Two docs name the wrong Python generated-output directory | | IPC-18 | Low | — | S | IPC-01 | N/A | Generated-code hygiene verified good — preserve under change (positive) | @@ -253,6 +253,7 @@ Findings the review flagged as one coordinated design pass — sequence them tog | Date | Change | |---|---| +| 2026-07-09 | P2 Wave B — worker event hot-path (commit `f61c816`), **windev-verified**. WRK-06 → `Done`: `MxStatusProxyConverter` caches the four resolved `FieldInfo` per status type in a static `ConcurrentDictionary` (the `GetField` metadata scan ran 4× per status per event on the STA path); `GetValue`+`Convert.ToInt32` still run per event (late-bound RCW); both exception messages byte-identical (missing-field via `ResolveField`, not cached on throw through `GetOrAdd`; null-value unchanged). WRK-11 → `Done`: `MxAccessEventQueue.Enqueue` takes ownership of the passed `MxEvent` (stamps `WorkerSequence`/`WorkerTimestamp` in place, no `Clone()`); audited all 3 callers (base/alarm event sinks + provider-mode handler) — each builds a fresh event, none reuse it; `MxAccessValueCache.Set` now deep-copies its retained `Value`/`SourceTimestamp`/`Statuses` so the cache snapshot never aliases the queue-owned (later serialized) event. WRK-12 → `Done`: `WorkerFrameWriter` coalesces the flush across a drained batch (write each frame, one `FlushAsync` after the batch, then complete all) — preserves the written+flushed completion contract; a burst of N events costs 1 flush not N; on failure the in-flight batch + queue all fail so no caller hangs. IPC-15 → `Done`: the multi-event `WorkerEnvelope` body stays unimplemented (wire still one event per `worker_event` frame); gateway.md Performance section now distinguishes the shipped flush-coalescing from that deferred additive-proto change. **Windev:** x86 Worker builds clean (fixed a Windows-only `TreatWarningsAsErrors` CS8604 in the value-cache null-guard that macOS can't surface); Worker.Tests **356 passed / 0 failed / 11 skipped** (+4 new: converter cache-reuse, queue ownership-transfer, value-cache snapshot independence, writer batch-flush-once); gateway Tests **815 passed / 1 failed** — the 1 is the known windev-environmental SelfSigned-SAN assertion (passes on macOS), and all pipe-harness tests (which also exercise the earlier G-frame gateway frame change + WRK-12 end-to-end over a real pipe) pass. | | 2026-07-09 | P2 Wave B — quick Mac items (commit `ec6f82b`). TST-11 → `Done`: `src/Directory.Build.props` single-sources the .NET-side version (Server/Worker/Contracts/tests stamped 0.1.2, was SDK-default 1.0.0) and appends the git short SHA to InformationalVersion (`0.1.2+`, guarded so a git-less build still works); verified Server assembly stamps `0.1.2+579282f`, Server+Tests build clean. Kept 0.1.2 (matches Contracts + aligned Python/Rust/Go); converging all to a single 0.2.0 cadence left as a release decision. WRK-15 → `Done` (docs-only path, no x86 build): STA thread-name refs corrected to the actual `MxGateway.Worker.STA` in `docs/WorkerSta.md` + `docs/MxAccessWorkerInstanceDesign.md`, and the stale heartbeat-counter note fixed (CaptureHeartbeat populates queue depth + sequence from the live queue). TST-23 → `Done`: already resolved by the Wave A gateway.md edit — the bidi `Session` RPC now sits under a "Future work: not implemented" heading (no "best long-term shape" framing), consistent with the proto. **TST-14 left open by design**: its only concrete step is deleting the user's untracked, gitignored `*-docs-*.md` working files (zero repo impact) — surfaced to the user rather than deleting files not created here. | | 2026-07-09 | P2 Wave B — dashboard/security (commit `e15c3cb`). SEC-25 → `Done` (near-term hardening only; full per-session EventsHub ACL stays deferred to roadmap item 12 / TST-15): `DashboardEventBroadcaster` redacts tag values from a deep clone of the event before mirroring to SignalR when `Dashboard:ShowTagValues` is false (default), so the hub seam cannot leak values regardless of the missing ACL. Clears `MxEvent.value` (the OnDataChange/OnWriteComplete/OperationComplete/OnBufferedDataChange bodies are empty discriminators — values ride in the top-level field, incl. buffered arrays) + the alarm body `current_value`/`limit_value`; source event never mutated (shared with gRPC path + replay ring; verified). Makes the formerly-dead `ShowTagValues` flag live for the mirror. `EventsHub` TODO(per-session-acl) kept + tied to roadmap item 12. Test `DashboardEventBroadcasterTests` (3). SEC-30 → `Done`: `docs/Diagnostics.md` trimmed to mark opt-in command-value logging NOT YET IMPLEMENTED (no `LogCommandValues` knob, `RedactCommandValue` unwired, no values logged); wiring deferred pending secured-bulk redaction (SEC-13), not wired now. Server build clean; Dashboard tests 152/152. | | 2026-07-09 | P2 Wave B — gateway event/frame hot-path performance (8 findings). **Frame I/O (commit `baae59c`):** GWC-08/IPC-13 gateway `WorkerFrameWriter` serializes once into one ArrayPool-rented buffer (LE length prefix + payload) and issues a single write (was: second serialization via `ToByteArray`, separate prefix array, two writes); IPC-14 `WorkerFrameReader` rents the payload buffer from ArrayPool instead of `new byte[]` per frame, read/parse length-bounded, returned in finally. Wire format byte-identical, cross-checked against the worker writer/reader. **Event/command hot path (commit `5e2e40a`):** GWC-06 `StreamEvents` timing via `Stopwatch.GetTimestamp()`/`GetElapsedTime()` (no per-event Stopwatch alloc); GWC-07/IPC-05 `MapEvent` transfers ownership of the inner `MxEvent` instead of cloning (safe — single-consumer `WorkerEvent`, MapEvent runs once pre-fan-out, all downstream consumers read-only; verified no post-mapping mutation) and `CreateCommandEnvelope` drops its redundant second clone (`MapCommand` already isolated the graph); every load-bearing isolation clone kept; GWC-15 `grpc_stream_queue.depth` converted from a per-event push counter to an `ObservableGauge` summing registered channel sources only at scrape time (name/semantics unchanged). **Replay ring (commit `cf1b3d4`):** GWC-14 replaces the `LinkedList` (node alloc per retained event) with a preallocated circular `ReplayEntry[]` (head+count), preserving ascending order, capacity eviction, time trim, oldest-read/ReplayGap math, both query paths, and the lock. → all `Done`. Server build clean (0 warnings); per-cluster tests green (WorkerFrameProtocol 10/10 incl. new near-cap round-trip, EventStream/Metrics/Distributor/Mapper 62/62, Distributor/Replay 33/33 incl. 2 new wrap-around cases). Full-suite macOS checkpoint: 769 passed / 44 failed, all 44 the known named-pipe/fake-worker-harness UDS-104-char limit (0 failures in any touched area). **Pending: windev run** to exercise the frame I/O over a real pipe (the 44 pipe-harness tests are macOS-blind). | -- 2.52.0 From 0c6e5b3acedf817d28b0668c01cd03027d7ec02a Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Thu, 9 Jul 2026 16:22:28 -0400 Subject: [PATCH 16/25] feat(clients): CLI-15 surface ReplayGap reconnect sentinel as typed signal (4/5 clients) The gateway emits a ReplayGap sentinel MxEvent at the head of a StreamEvents stream resumed via after_worker_sequence when the requested cursor predates the oldest retained event. Clients previously ignored it, silently mis-treating a lossy resume as continuous. Each client now surfaces the sentinel as a distinct, typed, non-terminal signal (never synthesized, never swallowed) so a consumer can detect the gap and re-snapshot; resume contract is after_worker_sequence = oldest_available_sequence - 1. - .NET: MxEventStreamItem (IsReplayGap/ReplayGap/Event) via new StreamEventItemsAsync + AsStreamItemsAsync extension. Build clean, 87 passed. - Go: EventResult.ReplayGap field + IsReplayGap(); ReplayGap type alias. build/vet/test clean. - Rust: EventItem enum (Event/ReplayGap); EventStream now yields Result; CLI renders REPLAY_GAP line / replayGap JSON. fmt/check/test/clippy clean. - Python: ReplayGap dataclass; stream_events yields pb.MxEvent | ReplayGap. 131 passed. - Shared docs: ClientLibrariesDesign non-goals reframed (reconnect-replay protocol is consumable; auto-reconnect stays a non-goal); CrossLanguageSmokeMatrix resume-gap note. Java client is deferred to the windev batch (no local JRE); CLI-15 stays open until it lands. Claude-Session: https://claude.ai/code/session_01DMXXvNuPekkkrTEyPNxEkW --- clients/dotnet/README.md | 42 ++++++ .../MxGatewayClientSessionTests.cs | 49 +++++++ .../MxEventStreamExtensions.cs | 40 ++++++ .../MxEventStreamItem.cs | 82 +++++++++++ .../MxGatewaySession.cs | 28 ++++ clients/go/README.md | 38 ++++++ clients/go/mxgateway/client_session_test.go | 68 +++++++++ clients/go/mxgateway/session.go | 39 +++++- clients/go/mxgateway/types.go | 6 + clients/python/README.md | 34 +++++ .../src/zb_mom_ww_mxgateway/__init__.py | 2 + .../python/src/zb_mom_ww_mxgateway/events.py | 63 +++++++++ .../python/src/zb_mom_ww_mxgateway/session.py | 50 ++++++- clients/python/tests/test_replay_gap.py | 106 ++++++++++++++ clients/rust/README.md | 41 ++++++ clients/rust/crates/mxgw-cli/src/main.rs | 46 +++++-- clients/rust/src/client.rs | 129 ++++++++++++++++-- clients/rust/src/lib.rs | 4 +- clients/rust/src/session.rs | 16 +++ clients/rust/tests/client_behavior.rs | 97 +++++++++++-- docs/ClientLibrariesDesign.md | 21 ++- docs/CrossLanguageSmokeMatrix.md | 8 ++ 22 files changed, 972 insertions(+), 37 deletions(-) create mode 100644 clients/dotnet/ZB.MOM.WW.MxGateway.Client/MxEventStreamExtensions.cs create mode 100644 clients/dotnet/ZB.MOM.WW.MxGateway.Client/MxEventStreamItem.cs create mode 100644 clients/python/src/zb_mom_ww_mxgateway/events.py create mode 100644 clients/python/tests/test_replay_gap.py diff --git a/clients/dotnet/README.md b/clients/dotnet/README.md index 71ecb15..ebdd94c 100644 --- a/clients/dotnet/README.md +++ b/clients/dotnet/README.md @@ -84,6 +84,48 @@ messages. `MxGatewaySession.OpenSessionReply` keeps the raw session-open reply available, and command helpers have `*RawAsync` variants when callers need the complete `MxCommandReply`. +### Event Streaming And Replay Gaps + +`StreamEventsAsync(afterWorkerSequence)` yields raw generated `MxEvent` +messages. Passing a non-zero `afterWorkerSequence` resumes a session's event +stream after a known worker sequence — this is the reconnect cursor. If that +cursor is *stale* — older than the oldest event the gateway still retains in the +session replay ring — the events in between were evicted and cannot be replayed. +The gateway signals this by emitting a single **replay-gap sentinel** at the head +of the resumed stream: an `MxEvent` with its `ReplayGap` field set, `Family` +unspecified, and no body. It means "you missed events — discard local state and +re-snapshot." + +Rather than force callers to inspect the raw sentinel, the client exposes a +typed surface, `StreamEventItemsAsync`, which yields `MxEventStreamItem` values: + +```csharp +await foreach (MxEventStreamItem item in session.StreamEventItemsAsync( + afterWorkerSequence: lastSeenSequence)) +{ + if (item.IsReplayGap) + { + // We missed events: throw away local state and re-snapshot. + ReplayGap gap = item.ReplayGap!; + // Resume without incurring another gap: + lastSeenSequence = gap.OldestAvailableSequence - 1; + await ReSnapshotAsync(); + continue; + } + + HandleEvent(item.Event); // normal MXAccess event, IsReplayGap == false + lastSeenSequence = item.Event.WorkerSequence; +} +``` + +The typed surface never synthesizes or drops events — it only makes the +gateway's own sentinel observable. Normal events pass through with +`IsReplayGap == false` and `ReplayGap == null`. The gap is only ever produced by +`StreamEvents`; the diagnostic drain path never emits it. If you already consume +the raw `StreamEventsAsync` (or the client-level stream), the +`AsStreamItemsAsync()` extension projects any `IAsyncEnumerable` into +the same `MxEventStreamItem` surface. + For alarms, the client exposes `QueryActiveAlarmsAsync` (one-shot snapshot of the active alarms the gateway's central monitor currently holds), `StreamAlarmsAsync` (server-streaming feed of alarm-state-change messages diff --git a/clients/dotnet/ZB.MOM.WW.MxGateway.Client.Tests/MxGatewayClientSessionTests.cs b/clients/dotnet/ZB.MOM.WW.MxGateway.Client.Tests/MxGatewayClientSessionTests.cs index d827be1..1de4290 100644 --- a/clients/dotnet/ZB.MOM.WW.MxGateway.Client.Tests/MxGatewayClientSessionTests.cs +++ b/clients/dotnet/ZB.MOM.WW.MxGateway.Client.Tests/MxGatewayClientSessionTests.cs @@ -215,6 +215,55 @@ public sealed class MxGatewayClientSessionTests Assert.Equal("session-fixture", request.SessionId); } + /// + /// Verifies that a reconnect-replay gap sentinel is surfaced as a typed + /// with the gap populated, while normal + /// events pass through unchanged with IsReplayGap false. + /// + [Fact] + public async Task StreamEventItemsAsync_SurfacesReplayGapSentinel() + { + FakeGatewayTransport transport = CreateTransport(); + transport.AddEvent(new MxEvent + { + SessionId = "session-fixture", + ReplayGap = new ReplayGap + { + RequestedAfterSequence = 5, + OldestAvailableSequence = 42, + }, + }); + transport.AddEvent(new MxEvent + { + SessionId = "session-fixture", + Family = MxEventFamily.OnDataChange, + WorkerSequence = 42, + }); + await using MxGatewayClient client = CreateClient(transport); + MxGatewaySession session = await client.OpenSessionAsync(); + + List items = []; + await foreach (MxEventStreamItem item in session.StreamEventItemsAsync(afterWorkerSequence: 5)) + { + items.Add(item); + } + + Assert.Equal(2, items.Count); + + MxEventStreamItem gap = items[0]; + Assert.True(gap.IsReplayGap); + Assert.NotNull(gap.ReplayGap); + Assert.Equal(5UL, gap.ReplayGap!.RequestedAfterSequence); + Assert.Equal(42UL, gap.ReplayGap.OldestAvailableSequence); + Assert.Same(gap.ReplayGap, gap.Event.ReplayGap); + + MxEventStreamItem normal = items[1]; + Assert.False(normal.IsReplayGap); + Assert.Null(normal.ReplayGap); + Assert.Equal(42UL, normal.Event.WorkerSequence); + Assert.Equal(MxEventFamily.OnDataChange, normal.Event.Family); + } + /// Verifies that close is explicit and idempotent. [Fact] public async Task CloseAsync_IsExplicitAndIdempotent() diff --git a/clients/dotnet/ZB.MOM.WW.MxGateway.Client/MxEventStreamExtensions.cs b/clients/dotnet/ZB.MOM.WW.MxGateway.Client/MxEventStreamExtensions.cs new file mode 100644 index 0000000..67b376a --- /dev/null +++ b/clients/dotnet/ZB.MOM.WW.MxGateway.Client/MxEventStreamExtensions.cs @@ -0,0 +1,40 @@ +using System.Runtime.CompilerServices; +using ZB.MOM.WW.MxGateway.Contracts.Proto; + +namespace ZB.MOM.WW.MxGateway.Client; + +/// +/// Extension methods that project a raw stream into the +/// typed surface, making the gateway's +/// reconnect-replay gap sentinel observable. +/// +public static class MxEventStreamExtensions +{ + /// + /// Projects a raw stream (e.g. + /// or + /// ) into typed + /// values. Normal events pass through with + /// false; the gateway's + /// reconnect-replay gap sentinel is surfaced with + /// true and + /// populated. The stream is + /// forwarded faithfully — no event is synthesized or dropped. + /// + /// The raw event stream to wrap. + /// Cancellation token for the enumeration. + /// The same events, each wrapped as an . + public static async IAsyncEnumerable AsStreamItemsAsync( + this IAsyncEnumerable source, + [EnumeratorCancellation] CancellationToken cancellationToken = default) + { + ArgumentNullException.ThrowIfNull(source); + + await foreach (MxEvent gatewayEvent in source + .WithCancellation(cancellationToken) + .ConfigureAwait(false)) + { + yield return MxEventStreamItem.From(gatewayEvent); + } + } +} diff --git a/clients/dotnet/ZB.MOM.WW.MxGateway.Client/MxEventStreamItem.cs b/clients/dotnet/ZB.MOM.WW.MxGateway.Client/MxEventStreamItem.cs new file mode 100644 index 0000000..c2f79df --- /dev/null +++ b/clients/dotnet/ZB.MOM.WW.MxGateway.Client/MxEventStreamItem.cs @@ -0,0 +1,82 @@ +using ZB.MOM.WW.MxGateway.Contracts.Proto; + +namespace ZB.MOM.WW.MxGateway.Client; + +/// +/// One item yielded by the typed event stream. It is either a normal MXAccess +/// or a reconnect-replay gap sentinel. +/// +/// +/// +/// The gateway emits a single sentinel at the head of a +/// StreamEvents stream that was resumed via +/// StreamEventsRequest.after_worker_sequence when the requested sequence +/// predates the oldest event still retained in the session replay ring — i.e. +/// events were evicted and cannot be replayed. On that sentinel the +/// MxEvent.replay_gap field is set, is +/// , the body oneof is unset, and +/// no per-item fields are populated. +/// +/// +/// This wrapper makes that sentinel observable instead of forcing the consumer +/// to inspect the raw . It never synthesizes or swallows an +/// event: the gap is exactly the gateway's own sentinel, exposed with +/// set and populated. Every +/// other event flows through unchanged with false. +/// +/// +/// When is the consumer has +/// missed events and MUST discard local state and re-snapshot. To resume without +/// incurring another gap, reconnect with +/// after_worker_sequence = ReplayGap.OldestAvailableSequence - 1. +/// +/// +public sealed class MxEventStreamItem +{ + private MxEventStreamItem(MxEvent gatewayEvent, ReplayGap? replayGap) + { + Event = gatewayEvent; + ReplayGap = replayGap; + } + + /// + /// The underlying raw . For a normal event this is the + /// MXAccess event itself; for a replay-gap item this is the gateway's + /// sentinel event whose only meaningful payload is . + /// Never . + /// + public MxEvent Event { get; } + + /// + /// The reconnect-replay gap payload when this item is a gap sentinel; + /// otherwise . Read + /// and + /// to learn + /// which events were lost. + /// + public ReplayGap? ReplayGap { get; } + + /// + /// when this item is a reconnect-replay gap sentinel: + /// the consumer missed events and must discard local state and re-snapshot. + /// Resume without another gap by reconnecting with + /// after_worker_sequence = ReplayGap.OldestAvailableSequence - 1. + /// + public bool IsReplayGap => ReplayGap is not null; + + /// + /// Wraps a raw stream as a typed item, classifying it + /// as a replay-gap sentinel when MxEvent.replay_gap is present. + /// + /// The raw event from the StreamEvents stream. + /// A typed item exposing either the normal event or the replay gap. + internal static MxEventStreamItem From(MxEvent gatewayEvent) + { + ArgumentNullException.ThrowIfNull(gatewayEvent); + + // For a message-typed proto3 field, presence is a non-null reference. + return gatewayEvent.ReplayGap is { } replayGap + ? new MxEventStreamItem(gatewayEvent, replayGap) + : new MxEventStreamItem(gatewayEvent, replayGap: null); + } +} diff --git a/clients/dotnet/ZB.MOM.WW.MxGateway.Client/MxGatewaySession.cs b/clients/dotnet/ZB.MOM.WW.MxGateway.Client/MxGatewaySession.cs index e0c6f24..1740a96 100644 --- a/clients/dotnet/ZB.MOM.WW.MxGateway.Client/MxGatewaySession.cs +++ b/clients/dotnet/ZB.MOM.WW.MxGateway.Client/MxGatewaySession.cs @@ -875,6 +875,34 @@ public sealed class MxGatewaySession : IAsyncDisposable cancellationToken); } + /// + /// Streams events as typed values, surfacing + /// the gateway's reconnect-replay gap sentinel as an observable, typed signal. + /// + /// + /// When resuming with a stale (older than + /// the oldest event still retained in the session replay ring), the gateway emits + /// a single gap sentinel at the head of the stream. It arrives here as an item + /// with true and + /// populated, meaning the consumer + /// missed events and MUST discard local state and re-snapshot. To resume without + /// incurring another gap, reconnect with + /// afterWorkerSequence = item.ReplayGap.OldestAvailableSequence - 1. + /// All other events pass through with + /// false. Use when raw generated + /// messages are needed instead. + /// + /// The sequence number to stream from. Defaults to 0. + /// Cancellation token. + /// An async enumerable of typed event items. + public IAsyncEnumerable StreamEventItemsAsync( + ulong afterWorkerSequence = 0, + CancellationToken cancellationToken = default) + { + return StreamEventsAsync(afterWorkerSequence, cancellationToken) + .AsStreamItemsAsync(cancellationToken); + } + /// /// Closes the session and releases resources. /// diff --git a/clients/go/README.md b/clients/go/README.md index 1c66881..2adfec4 100644 --- a/clients/go/README.md +++ b/clients/go/README.md @@ -92,6 +92,44 @@ goroutine cleanup. Raw protobuf messages remain available through the `errors.As` for `GatewayError`, `CommandError`, and `MxAccessError`; command errors preserve the raw reply. +### Reconnect-replay gap + +Each `EventResult` carries exactly one of `Event`, `ReplayGap`, or `Err`. When +you resume a stream with `EventsAfter`/`SubscribeEventsAfter` and a non-zero +`afterWorkerSequence`, the gateway replays buffered events from that point. If +the requested sequence predates the oldest event still retained in its replay +ring, it delivers a single **replay-gap sentinel** at the head of the resumed +stream: `res.ReplayGap` is non-nil (`res.Event` is nil, `res.IsReplayGap()` is +true) and normal events follow it. + +A gap means events were lost, so any locally cached tag/alarm state is now +stale. On seeing it, discard your cached state and re-snapshot. To resume +without provoking another gap, reconnect from just before the oldest retained +sequence: + +```go +for res := range events { + switch { + case res.Err != nil: + // terminal: stream ended (see ErrSlowConsumer for the overflow case) + return res.Err + case res.IsReplayGap(): + gap := res.ReplayGap + log.Printf("replay gap: requested after %d, oldest available %d; re-snapshotting", + gap.GetRequestedAfterSequence(), gap.GetOldestAvailableSequence()) + resnapshot() + // to resume cleanly, reconnect with: + // session.EventsAfter(ctx, gap.GetOldestAvailableSequence()-1) + default: + handle(res.Event) + } +} +``` + +The gateway sets `ReplayGap` only on `StreamEvents` results (never on a fresh, +non-resumed stream and never on `DrainEvents`). The client makes the gateway's +sentinel typed and observable; it never synthesizes or swallows it. + For alarms, the package exposes `Client.QueryActiveAlarms` for one-shot snapshots, `Client.StreamAlarms` for the server-streaming feed, and `Client.AcknowledgeAlarm` to ack an alarm by full reference. The streaming diff --git a/clients/go/mxgateway/client_session_test.go b/clients/go/mxgateway/client_session_test.go index fa1f85b..172d268 100644 --- a/clients/go/mxgateway/client_session_test.go +++ b/clients/go/mxgateway/client_session_test.go @@ -200,6 +200,63 @@ func TestEventsSlowConsumerYieldsErrSlowConsumerBeforeClose(t *testing.T) { } } +func TestEventsSurfacesReplayGapSentinelAsTypedSignal(t *testing.T) { + fake := &fakeGatewayServer{ + streamStarted: make(chan struct{}), + streamReplayGap: &pb.ReplayGap{ + RequestedAfterSequence: 5, + OldestAvailableSequence: 42, + }, + } + client, cleanup := newBufconnClient(t, fake) + defer cleanup() + session := NewSessionForID(client, "session-1") + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + + events, err := session.EventsAfter(ctx, 5) + if err != nil { + t.Fatalf("EventsAfter() error = %v", err) + } + <-fake.streamStarted + + // First result must be the typed replay-gap signal, not a normal event. + first := <-events + if first.Err != nil { + t.Fatalf("first result error = %v", first.Err) + } + if !first.IsReplayGap() { + t.Fatalf("first result IsReplayGap() = false, want true") + } + if first.Event != nil { + t.Fatalf("replay-gap result carried a non-nil Event %+v; want the sentinel to clear Event", first.Event) + } + if got := first.ReplayGap.GetRequestedAfterSequence(); got != 5 { + t.Fatalf("ReplayGap.RequestedAfterSequence = %d, want 5", got) + } + if got := first.ReplayGap.GetOldestAvailableSequence(); got != 42 { + t.Fatalf("ReplayGap.OldestAvailableSequence = %d, want 42", got) + } + + // Normal events after the sentinel are unaffected: Event set, ReplayGap nil. + second := <-events + if second.Err != nil { + t.Fatalf("second result error = %v", second.Err) + } + if second.IsReplayGap() { + t.Fatalf("second result IsReplayGap() = true, want false for a normal event") + } + if second.Event == nil { + t.Fatal("second result Event = nil, want a normal event") + } + if got := second.Event.GetWorkerSequence(); got != 1 { + t.Fatalf("normal event worker sequence = %d, want 1", got) + } + if second.Event.GetFamily() != pb.MxEventFamily_MX_EVENT_FAMILY_ON_DATA_CHANGE { + t.Fatalf("normal event family = %s, want ON_DATA_CHANGE", second.Event.GetFamily()) + } +} + func TestSessionHelpersBuildCommandsAndExposeRawReply(t *testing.T) { fake := &fakeGatewayServer{ invokeReply: &pb.MxCommandReply{ @@ -643,6 +700,7 @@ type fakeGatewayServer struct { streamStarted chan struct{} streamDone chan struct{} streamEventCount int + streamReplayGap *pb.ReplayGap invokeReply *pb.MxCommandReply invokeRequest *pb.MxCommandRequest } @@ -691,6 +749,16 @@ func (s *fakeGatewayServer) StreamEvents(req *pb.StreamEventsRequest, stream grp if s.streamStarted != nil { close(s.streamStarted) } + if s.streamReplayGap != nil { + // Emit the reconnect-replay gap sentinel at the head of the resumed + // stream: family UNSPECIFIED, body unset, only replay_gap populated. + if err := stream.Send(&pb.MxEvent{ + SessionId: req.GetSessionId(), + ReplayGap: s.streamReplayGap, + }); err != nil { + return err + } + } eventCount := s.streamEventCount if eventCount == 0 { eventCount = 1 diff --git a/clients/go/mxgateway/session.go b/clients/go/mxgateway/session.go index e61cf46..c58e00f 100644 --- a/clients/go/mxgateway/session.go +++ b/clients/go/mxgateway/session.go @@ -27,14 +27,39 @@ const eventBufferSize = 16 // non-blockingly on overflow, even when all data slots are full. const eventBufferReservedSlots = 1 -// EventResult carries either the next ordered event or a terminal stream error. +// EventResult carries the next ordered event, a replay-gap signal, or a +// terminal stream error. Exactly one of Event, ReplayGap, or Err is set on any +// delivered result. type EventResult struct { - // Event is the next event from the stream when Err is nil. + // Event is the next MXAccess event from the stream when both ReplayGap and + // Err are nil. Event *MxEvent + // ReplayGap, when non-nil, is the gateway's reconnect-replay gap sentinel: it + // is delivered at the head of a resumed stream (one opened with a non-zero + // after_worker_sequence via EventsAfter/SubscribeEventsAfter) when the + // requested sequence predates the oldest event still retained in the replay + // ring, so the events in between were lost. + // + // It is a non-terminal, observable signal — the stream continues with normal + // events after it, and Event is nil on a gap result so a gap is never + // mistaken for a normal MXAccess event. On seeing a gap the consumer must + // discard any locally cached tag/alarm state and re-snapshot. To resume + // cleanly (without provoking another gap), reconnect with EventsAfter using + // afterWorkerSequence = ReplayGap.GetOldestAvailableSequence() - 1. + // + // The gateway sets ReplayGap only on StreamEvents results, never on a normal + // (non-resumed) stream and never on DrainEvents. + ReplayGap *ReplayGap // Err is the terminal stream error; when non-nil no further results follow. Err error } +// IsReplayGap reports whether this result carries the gateway's reconnect-replay +// gap sentinel rather than a normal event or a terminal error. +func (r EventResult) IsReplayGap() bool { + return r.ReplayGap != nil +} + // EventSubscription owns a running gateway event stream. type EventSubscription struct { results <-chan EventResult @@ -736,7 +761,15 @@ func (s *Session) subscribeEventsAfter(ctx context.Context, afterWorkerSequence for { event, err := stream.Recv() if err == nil { - if !sendEventResult(streamCtx, results, EventResult{Event: event}, cancelWhenResultBufferFull, cancel) { + result := EventResult{Event: event} + // The gateway marks a reconnect-replay gap with a sentinel MxEvent + // carrying replay_gap (family UNSPECIFIED, body unset). Surface it + // as a distinct typed signal rather than a normal event: clear + // Event so consumers never process the sentinel as a data change. + if gap := event.GetReplayGap(); gap != nil { + result = EventResult{ReplayGap: gap} + } + if !sendEventResult(streamCtx, results, result, cancelWhenResultBufferFull, cancel) { return } continue diff --git a/clients/go/mxgateway/types.go b/clients/go/mxgateway/types.go index 26eba2e..8b9f402 100644 --- a/clients/go/mxgateway/types.go +++ b/clients/go/mxgateway/types.go @@ -30,6 +30,12 @@ type ( MxCommand = pb.MxCommand // MxEvent is one ordered event delivered on a session event stream. MxEvent = pb.MxEvent + // ReplayGap is the gateway sentinel payload signalling that a resumed event + // stream skipped past the oldest event still retained in the replay ring. + // RequestedAfterSequence is the after_worker_sequence the client resumed + // from; OldestAvailableSequence is the oldest sequence the gateway can still + // replay. See EventResult.ReplayGap for consumption guidance. + ReplayGap = pb.ReplayGap // MxValue is the protobuf representation of an MXAccess value. MxValue = pb.MxValue // Value is an alias for MxValue retained for symmetry with other clients. diff --git a/clients/python/README.md b/clients/python/README.md index 3833809..ef8409f 100644 --- a/clients/python/README.md +++ b/clients/python/README.md @@ -105,6 +105,40 @@ terminate the stream. Canceling a Python task cancels the client-side gRPC call or stream wait. It does not abort an in-flight MXAccess COM call inside the worker process. +### Event streaming and reconnect gaps + +`Session.stream_events()` yields an async iterator whose items are either a +normal `MxEvent` or a `ReplayGap`. Track the `worker_sequence` of the last event +you processed and pass it back as `after_worker_sequence` to resume after a +disconnect: + +```python +from zb_mom_ww_mxgateway import ReplayGap + +cursor = 0 +async for item in session.stream_events(after_worker_sequence=cursor): + if isinstance(item, ReplayGap): + # The gateway dropped events between item.requested_after_sequence and + # item.oldest_available_sequence — they are gone from the replay ring. + # Discard local tag/alarm state and re-snapshot (e.g. read_bulk / + # query_active_alarms), then resume without another gap: + cursor = item.resume_after_worker_sequence # oldest_available_sequence - 1 + continue + # Normal MXAccess event. + cursor = item.worker_sequence + handle(item) +``` + +`ReplayGap` is the gateway's reconnect-replay gap sentinel made typed and +observable. It is delivered only at the head of a stream resumed with a non-zero +`after_worker_sequence` when the requested cursor predates the oldest retained +event. It is a **non-terminal** signal — the stream continues with normal events +after it — and it is never yielded as an `MxEvent`, so it can never be mistaken +for a real MXAccess event. The client does not synthesize or swallow it; the +gateway only sets it on `StreamEvents` results (never on a fresh stream or a +`DrainEvents` reply). `GatewayClient.stream_events_raw` remains the raw protobuf +stream (the sentinel arrives there as an `MxEvent` with `replay_gap` set). + ## Write Semantics And Common Pitfalls These are MXAccess parity behaviors that surprise new callers. The gateway diff --git a/clients/python/src/zb_mom_ww_mxgateway/__init__.py b/clients/python/src/zb_mom_ww_mxgateway/__init__.py index 49bace6..0d9943d 100644 --- a/clients/python/src/zb_mom_ww_mxgateway/__init__.py +++ b/clients/python/src/zb_mom_ww_mxgateway/__init__.py @@ -9,6 +9,7 @@ from .generated.galaxy_repository_pb2 import ( GalaxyObject, WatchDeployEventsRequest, ) +from .events import ReplayGap from .errors import ( MxAccessError, MxGatewayAuthenticationError, @@ -43,6 +44,7 @@ __all__ = [ "MxGatewayTransportError", "MxGatewayWorkerError", "MxValueView", + "ReplayGap", "Session", "WatchDeployEventsRequest", "__version__", diff --git a/clients/python/src/zb_mom_ww_mxgateway/events.py b/clients/python/src/zb_mom_ww_mxgateway/events.py new file mode 100644 index 0000000..fe074de --- /dev/null +++ b/clients/python/src/zb_mom_ww_mxgateway/events.py @@ -0,0 +1,63 @@ +"""Typed event-stream signals for the MXAccess Gateway Python client.""" + +from __future__ import annotations + +from dataclasses import dataclass + +from .generated import mxaccess_gateway_pb2 as pb + + +@dataclass(frozen=True, slots=True) +class ReplayGap: + """Reconnect-replay gap signal surfaced on a resumed event stream. + + The gateway emits this at the head of a stream resumed with + :meth:`Session.stream_events`'s ``after_worker_sequence`` cursor when the + requested sequence predates the oldest event still retained in the gateway's + replay ring. That means the events between ``requested_after_sequence`` and + ``oldest_available_sequence`` were dropped from the ring and can no longer be + replayed — the client has an unrecoverable hole in its event history. + + ``ReplayGap`` is a *non-terminal, observable* signal: the stream keeps + delivering normal :class:`~zb_mom_ww_mxgateway.generated.mxaccess_gateway_pb2.MxEvent` + values after it. :meth:`Session.stream_events` yields it as a distinct type + (never as an ``MxEvent``) so a consumer can branch on + ``isinstance(item, ReplayGap)`` and never mistake a gap for a real MXAccess + event. The client neither synthesizes nor swallows the gateway's sentinel — + it only makes that sentinel typed and observable. + + On seeing a gap the consumer must discard any locally cached tag/alarm state + and re-snapshot (for example via :meth:`Session.read_bulk` or + :meth:`~zb_mom_ww_mxgateway.GatewayClient.query_active_alarms`). To resume + the stream without provoking another gap, reconnect with + ``after_worker_sequence = gap.resume_after_worker_sequence`` (that is, + ``oldest_available_sequence - 1``) so the next replayed event is the oldest + the gateway still retains. + + The gateway sets this only on ``StreamEvents`` results — never on a normal + (non-resumed) stream and never on a ``DrainEvents`` reply. + """ + + requested_after_sequence: int + """The ``after_worker_sequence`` cursor the resumed stream was opened with.""" + + oldest_available_sequence: int + """Oldest worker sequence the gateway can still replay.""" + + @classmethod + def from_proto(cls, gap: pb.ReplayGap) -> "ReplayGap": + """Build a :class:`ReplayGap` from the generated ``ReplayGap`` message.""" + return cls( + requested_after_sequence=gap.requested_after_sequence, + oldest_available_sequence=gap.oldest_available_sequence, + ) + + @property + def resume_after_worker_sequence(self) -> int: + """``after_worker_sequence`` to resume the stream without another gap. + + Equal to ``oldest_available_sequence - 1`` so the next event the gateway + replays is ``oldest_available_sequence`` — the oldest it still retains. + Clamped at ``0`` so it is never negative. + """ + return max(self.oldest_available_sequence - 1, 0) diff --git a/clients/python/src/zb_mom_ww_mxgateway/session.py b/clients/python/src/zb_mom_ww_mxgateway/session.py index 1302025..973ba9a 100644 --- a/clients/python/src/zb_mom_ww_mxgateway/session.py +++ b/clients/python/src/zb_mom_ww_mxgateway/session.py @@ -5,6 +5,7 @@ from __future__ import annotations from collections.abc import AsyncIterator, Sequence from .errors import ensure_mxaccess_success +from .events import ReplayGap from .generated import mxaccess_gateway_pb2 as pb from .values import MxValueInput, to_mx_value @@ -572,14 +573,57 @@ class Session: self, *, after_worker_sequence: int = 0, - ) -> AsyncIterator[pb.MxEvent]: - """Return an async iterator of `MxEvent` messages for this session.""" - return self.client.stream_events_raw( + ) -> AsyncIterator[pb.MxEvent | ReplayGap]: + """Return an async iterator over this session's `MxEvent` stream. + + Each yielded item is either a normal :class:`~...MxEvent` or a + :class:`ReplayGap`. Branch on ``isinstance(item, ReplayGap)`` — a gap is + never delivered as an ``MxEvent`` so it cannot be mistaken for a real + MXAccess event. + + Pass a non-zero *after_worker_sequence* to resume a previously observed + stream. If that cursor predates the oldest event the gateway still + retains in its replay ring, the stream opens with a single + :class:`ReplayGap` sentinel (events in the gap were dropped and cannot be + replayed), then continues with normal events. On a gap, discard locally + cached state, re-snapshot, and — to resume without another gap — + reconnect with ``after_worker_sequence = gap.resume_after_worker_sequence``. + See :class:`ReplayGap` for the full semantics. The underlying protobuf + stream is available raw via ``GatewayClient.stream_events_raw``. + """ + raw = self.client.stream_events_raw( pb.StreamEventsRequest( session_id=self.session_id, after_worker_sequence=after_worker_sequence, ), ) + return _surface_replay_gaps(raw) + + +async def _surface_replay_gaps( + raw: AsyncIterator[pb.MxEvent], +) -> AsyncIterator[pb.MxEvent | ReplayGap]: + """Map the gateway's ``replay_gap`` sentinel event to a typed :class:`ReplayGap`. + + Normal events pass through unchanged. The sentinel (``replay_gap`` set, + ``family`` unspecified, body unset) is converted to a distinct + :class:`ReplayGap` so a consumer can branch on it without inspecting proto + presence, and is never yielded as an ``MxEvent``. The sentinel is forwarded + faithfully — it is neither dropped nor turned into a normal event. + + Closing this generator (``aclose``) propagates to *raw* so the underlying + gRPC call is cancelled, preserving the raw stream's cancel-on-stop contract. + """ + try: + async for event in raw: + if event.HasField("replay_gap"): + yield ReplayGap.from_proto(event.replay_gap) + else: + yield event + finally: + aclose = getattr(raw, "aclose", None) + if aclose is not None: + await aclose() def _ensure_bulk_size(name: str, count: int) -> None: diff --git a/clients/python/tests/test_replay_gap.py b/clients/python/tests/test_replay_gap.py new file mode 100644 index 0000000..472e9b4 --- /dev/null +++ b/clients/python/tests/test_replay_gap.py @@ -0,0 +1,106 @@ +"""Tests for the typed ReplayGap signal on Session.stream_events (CLI-15).""" + +from __future__ import annotations + +from collections.abc import AsyncIterator + +import pytest + +from zb_mom_ww_mxgateway import ReplayGap, Session +from zb_mom_ww_mxgateway.generated import mxaccess_gateway_pb2 as pb + + +class _FakeClient: + """Minimal client stub exposing only what Session.stream_events needs.""" + + def __init__(self, events: list[pb.MxEvent]) -> None: + self._events = events + self.last_request: pb.StreamEventsRequest | None = None + + def stream_events_raw( + self, + request: pb.StreamEventsRequest, + ) -> AsyncIterator[pb.MxEvent]: + self.last_request = request + + async def _gen() -> AsyncIterator[pb.MxEvent]: + for event in self._events: + yield event + + return _gen() + + +def _gap_sentinel(*, requested: int, oldest: int) -> pb.MxEvent: + return pb.MxEvent( + session_id="session-1", + family=pb.MX_EVENT_FAMILY_UNSPECIFIED, + replay_gap=pb.ReplayGap( + requested_after_sequence=requested, + oldest_available_sequence=oldest, + ), + ) + + +def _normal_event(worker_sequence: int) -> pb.MxEvent: + return pb.MxEvent( + session_id="session-1", + worker_sequence=worker_sequence, + family=pb.MX_EVENT_FAMILY_ON_DATA_CHANGE, + ) + + +def _session(events: list[pb.MxEvent]) -> tuple[Session, _FakeClient]: + client = _FakeClient(events) + session = Session(client=client, session_id="session-1") # type: ignore[arg-type] + return session, client + + +@pytest.mark.asyncio +async def test_replay_gap_sentinel_surfaces_as_typed_signal() -> None: + session, _client = _session( + [_gap_sentinel(requested=5, oldest=10), _normal_event(10)], + ) + + items = [item async for item in session.stream_events(after_worker_sequence=5)] + + assert isinstance(items[0], ReplayGap) + assert items[0].requested_after_sequence == 5 + assert items[0].oldest_available_sequence == 10 + # Resume cursor is oldest_available_sequence - 1 so the next replayed event + # is the oldest the gateway still retains. + assert items[0].resume_after_worker_sequence == 9 + + # Normal events after the sentinel pass through unchanged as MxEvent. + assert isinstance(items[1], pb.MxEvent) + assert not items[1].HasField("replay_gap") + assert items[1].worker_sequence == 10 + + +@pytest.mark.asyncio +async def test_normal_events_are_unaffected() -> None: + session, _client = _session([_normal_event(1), _normal_event(2)]) + + items = [item async for item in session.stream_events()] + + assert all(isinstance(item, pb.MxEvent) for item in items) + assert [item.worker_sequence for item in items] == [1, 2] + assert not any(isinstance(item, ReplayGap) for item in items) + + +@pytest.mark.asyncio +async def test_stream_events_forwards_resume_cursor() -> None: + session, client = _session([]) + + async for _ in session.stream_events(after_worker_sequence=42): + pass + + assert client.last_request is not None + assert client.last_request.after_worker_sequence == 42 + assert client.last_request.session_id == "session-1" + + +def test_replay_gap_resume_cursor_never_negative() -> None: + gap = ReplayGap.from_proto( + pb.ReplayGap(requested_after_sequence=0, oldest_available_sequence=0), + ) + assert gap.resume_after_worker_sequence == 0 diff --git a/clients/rust/README.md b/clients/rust/README.md index 4548fee..e27f707 100644 --- a/clients/rust/README.md +++ b/clients/rust/README.md @@ -137,6 +137,47 @@ redaction. Per-item bulk failures are reported inside each result entry `invoke_raw` / `client.invoke_raw` escape hatch performs neither check and returns the unvalidated reply. +## Event Streaming And Reconnect-Replay Gaps + +`session.events()` / `session.events_after(after_worker_sequence)` (and the +lower-level `client.stream_events`) return an `EventStream` that yields +`EventItem` values, not bare `MxEvent`s: + +```rust +use zb_mom_ww_mxgateway_client::EventItem; + +let mut stream = session.events_after(cursor).await?; +while let Some(item) = stream.next().await { + match item? { + EventItem::Event(event) => { /* apply the MXAccess change */ } + EventItem::ReplayGap(gap) => { + // Recent history was evicted — discard local state and re-snapshot, + // then resume without provoking another gap: + let resume = gap.oldest_available_sequence.saturating_sub(1); + stream = session.events_after(resume).await?; + } + } +} +``` + +Almost every item is a normal `EventItem::Event`. `EventItem::ReplayGap` is a +faithful, typed surfacing of the gateway's reconnect-replay gap sentinel — the +client does not synthesize it. The gateway emits the sentinel at most once, at +the head of a stream **resumed** via `events_after` (`after_worker_sequence`) +when the requested sequence is older than the oldest event still retained in the +session's replay ring: events in the open interval +`(requested_after_sequence, oldest_available_sequence)` were evicted and cannot +be replayed. A `ReplayGap` therefore means "you missed events — discard any +local state and re-snapshot." To resume without a second gap, reconnect with +`events_after(gap.oldest_available_sequence - 1)`, which replays starting at the +first still-retained event. A stream opened from the beginning +(`session.events()` / `events_after(0)`) never produces a `ReplayGap`. + +`EventItem` provides `as_event()`, `into_event()`, and `replay_gap()` accessors +for callers that prefer not to `match`. The `mxgw-cli stream-events` subcommand +renders the sentinel as a distinct `REPLAY_GAP …` line (or a `replayGap` JSON +object under `--json` / `--jsonl`). + ## Write Semantics And Common Pitfalls These are MXAccess parity behaviors that surprise new callers. The gateway diff --git a/clients/rust/crates/mxgw-cli/src/main.rs b/clients/rust/crates/mxgw-cli/src/main.rs index 60d7b21..8a84ffe 100644 --- a/clients/rust/crates/mxgw-cli/src/main.rs +++ b/clients/rust/crates/mxgw-cli/src/main.rs @@ -28,8 +28,8 @@ use zb_mom_ww_mxgateway_client::generated::mxaccess_gateway::v1::{ WriteSecuredBulkEntry, }; use zb_mom_ww_mxgateway_client::{ - next_correlation_id, ApiKey, ClientOptions, Error, GalaxyClient, GatewayClient, MxValue, - MxValueProjection, CLIENT_VERSION, GATEWAY_PROTOCOL_VERSION, WORKER_PROTOCOL_VERSION, + next_correlation_id, ApiKey, ClientOptions, Error, EventItem, GalaxyClient, GatewayClient, + MxValue, MxValueProjection, CLIENT_VERSION, GATEWAY_PROTOCOL_VERSION, WORKER_PROTOCOL_VERSION, }; const MAX_AGGREGATE_EVENTS: usize = 10_000; @@ -886,17 +886,43 @@ async fn dispatch(command: Command) -> Result<(), Error> { let mut events: Vec = Vec::new(); let mut event_count = 0usize; while event_count < max_events { - let Some(event) = stream.next().await else { + let Some(item) = stream.next().await else { break; }; - let event = event?; + let item = item?; event_count += 1; - if jsonl { - println!("{}", event_to_json(&event)); - } else if json { - events.push(event_to_json(&event)); - } else { - println!("{} {}", event.worker_sequence, event.family); + match item { + EventItem::Event(event) => { + if jsonl { + println!("{}", event_to_json(&event)); + } else if json { + events.push(event_to_json(&event)); + } else { + println!("{} {}", event.worker_sequence, event.family); + } + } + // Reconnect-replay gap sentinel: recent history was evicted + // before this resumed stream could replay it. Render it as a + // distinct row so the caller can re-snapshot and resume with + // `oldest_available_sequence - 1`. + EventItem::ReplayGap(gap) => { + let value = json!({ + "replayGap": { + "requestedAfterSequence": gap.requested_after_sequence, + "oldestAvailableSequence": gap.oldest_available_sequence, + } + }); + if jsonl { + println!("{value}"); + } else if json { + events.push(value); + } else { + println!( + "REPLAY_GAP requested_after={} oldest_available={}", + gap.requested_after_sequence, gap.oldest_available_sequence + ); + } + } } } if json { diff --git a/clients/rust/src/client.rs b/clients/rust/src/client.rs index 493503c..aa3e9b1 100644 --- a/clients/rust/src/client.rs +++ b/clients/rust/src/client.rs @@ -18,7 +18,7 @@ use crate::generated::mxaccess_gateway::v1::mx_access_gateway_client::MxAccessGa use crate::generated::mxaccess_gateway::v1::{ AcknowledgeAlarmReply, AcknowledgeAlarmRequest, ActiveAlarmSnapshot, AlarmFeedMessage, CloseSessionReply, CloseSessionRequest, MxCommandReply, MxCommandRequest, MxEvent, - OpenSessionReply, OpenSessionRequest, QueryActiveAlarmsRequest, StreamAlarmsRequest, + OpenSessionReply, OpenSessionRequest, QueryActiveAlarmsRequest, ReplayGap, StreamAlarmsRequest, StreamEventsRequest, }; use crate::options::{build_tls_config, ClientOptions}; @@ -28,11 +28,120 @@ use crate::session::Session; /// [`GatewayClient`] uses internally. pub type RawGatewayClient = MxAccessGatewayClient>; -/// Pinned, boxed [`MxEvent`] stream returned by -/// [`GatewayClient::stream_events`]. Errors are pre-mapped from -/// `tonic::Status` to [`Error`]; dropping the stream cancels the call. +/// One item yielded by the per-session event stream returned by +/// [`GatewayClient::stream_events`]. +/// +/// Almost every item is an ordinary MXAccess event ([`EventItem::Event`]). +/// The one exception is the reconnect-replay gap sentinel +/// ([`EventItem::ReplayGap`]): the gateway emits it at most once, at the head +/// of a stream that was *resumed* via +/// [`Session::events_after`](crate::session::Session::events_after) +/// (`StreamEventsRequest.after_worker_sequence`) when the requested sequence is +/// older than the oldest event still retained in the session replay ring — i.e. +/// events were evicted and cannot be replayed. +/// +/// The client does **not** synthesize this signal: it faithfully forwards the +/// gateway's sentinel `MxEvent` (whose `replay_gap` field is set), and only +/// makes it a distinct, typed variant so consumers can `match` on it instead of +/// inspecting a field on a value that otherwise looks like a normal event. +/// +/// # Reacting to a gap +/// +/// A [`EventItem::ReplayGap`] means "you missed events — discard any local +/// state and re-snapshot." The events in the open interval +/// `(requested_after_sequence, oldest_available_sequence)` are gone. To resume +/// the stream without provoking another gap, reconnect with +/// [`Session::events_after`](crate::session::Session::events_after) passing +/// `oldest_available_sequence - 1`, which replays starting at the first still +/// retained event (`oldest_available_sequence`): +/// +/// ```no_run +/// # use zb_mom_ww_mxgateway_client::{EventItem, Session}; +/// # use futures_util::StreamExt; +/// # async fn run(session: Session, cursor: u64) -> Result<(), zb_mom_ww_mxgateway_client::Error> { +/// let mut stream = session.events_after(cursor).await?; +/// while let Some(item) = stream.next().await { +/// match item? { +/// EventItem::Event(event) => { +/// let _ = event; // apply the change +/// } +/// EventItem::ReplayGap(gap) => { +/// // Local state is stale — re-snapshot, then resume without a gap. +/// let resume_cursor = gap.oldest_available_sequence.saturating_sub(1); +/// stream = session.events_after(resume_cursor).await?; +/// } +/// } +/// } +/// # Ok(()) +/// # } +/// ``` +// The `Event` variant is the hot path (nearly every stream item) and is the +// large one; the rare `ReplayGap` sentinel is small. Boxing `Event` to equalize +// the variants would add a heap allocation to every streamed event — a +// regression versus the prior `Result` surface, which already +// moved `MxEvent` by value. Keep the common path allocation-free. +#[allow(clippy::large_enum_variant)] +#[derive(Clone, Debug, PartialEq)] +pub enum EventItem { + /// A normal MXAccess event forwarded from the worker. + Event(MxEvent), + /// The reconnect-replay gap sentinel — recent event history was evicted + /// before this resumed stream could replay it. See [`EventItem`] for how + /// to react. + ReplayGap(ReplayGap), +} + +impl EventItem { + /// Classify an incoming `MxEvent` into the typed stream item. + /// + /// A present `replay_gap` promotes the event to [`EventItem::ReplayGap`]; + /// otherwise it is an ordinary [`EventItem::Event`]. The sentinel is never + /// dropped and never surfaced as a normal event. + fn from_event(mut event: MxEvent) -> Self { + match event.replay_gap.take() { + Some(gap) => EventItem::ReplayGap(gap), + None => EventItem::Event(event), + } + } + + /// Borrow the inner [`MxEvent`] when this item is a normal event, or + /// `None` when it is the [`EventItem::ReplayGap`] sentinel. + #[must_use] + pub fn as_event(&self) -> Option<&MxEvent> { + match self { + EventItem::Event(event) => Some(event), + EventItem::ReplayGap(_) => None, + } + } + + /// Borrow the [`ReplayGap`] when this item is the reconnect-replay gap + /// sentinel, or `None` for a normal event. + #[must_use] + pub fn replay_gap(&self) -> Option<&ReplayGap> { + match self { + EventItem::ReplayGap(gap) => Some(gap), + EventItem::Event(_) => None, + } + } + + /// Consume the item and return the inner [`MxEvent`] when it is a normal + /// event, or `None` for the [`EventItem::ReplayGap`] sentinel. + #[must_use] + pub fn into_event(self) -> Option { + match self { + EventItem::Event(event) => Some(event), + EventItem::ReplayGap(_) => None, + } + } +} + +/// Pinned, boxed [`EventItem`] stream returned by +/// [`GatewayClient::stream_events`]. Each item is either a normal +/// [`EventItem::Event`] or the [`EventItem::ReplayGap`] reconnect-replay +/// sentinel. Errors are pre-mapped from `tonic::Status` to [`Error`]; dropping +/// the stream cancels the call. pub type EventStream = - std::pin::Pin> + Send + 'static>>; + std::pin::Pin> + Send + 'static>>; /// Pinned, boxed [`ActiveAlarmSnapshot`] stream returned by /// [`GatewayClient::query_active_alarms`]. Errors are pre-mapped from @@ -190,8 +299,12 @@ impl GatewayClient { /// Open the server-streaming `StreamEvents` RPC. /// - /// The returned [`EventStream`] yields `MxEvent` messages as the worker - /// produces them. Dropping the stream cancels the gRPC call cooperatively. + /// The returned [`EventStream`] yields [`EventItem`] values as the worker + /// produces them: ordinary MXAccess events as [`EventItem::Event`], and the + /// gateway's reconnect-replay gap sentinel — set only on resumed streams + /// whose requested sequence predates the retained replay history — as + /// [`EventItem::ReplayGap`]. Dropping the stream cancels the gRPC call + /// cooperatively. /// /// # Errors /// @@ -201,7 +314,7 @@ impl GatewayClient { let mut client = self.inner.clone(); let response = client.stream_events(self.stream_request(request)).await?; let stream = futures_util::StreamExt::map(response.into_inner(), |result| { - result.map_err(Error::from) + result.map(EventItem::from_event).map_err(Error::from) }); Ok(Box::pin(stream)) diff --git a/clients/rust/src/lib.rs b/clients/rust/src/lib.rs index 44acce7..50d4306 100644 --- a/clients/rust/src/lib.rs +++ b/clients/rust/src/lib.rs @@ -24,12 +24,14 @@ pub mod version; #[doc(inline)] pub use auth::{ApiKey, AuthInterceptor}; #[doc(inline)] -pub use client::{AlarmFeedStream, EventStream, GatewayClient}; +pub use client::{AlarmFeedStream, EventItem, EventStream, GatewayClient}; #[doc(inline)] pub use error::{CommandError, Error, MxAccessError}; #[doc(inline)] pub use galaxy::{DeployEventStream, GalaxyClient}; #[doc(inline)] +pub use generated::mxaccess_gateway::v1::ReplayGap; +#[doc(inline)] pub use options::ClientOptions; #[doc(inline)] pub use session::{next_correlation_id, Session}; diff --git a/clients/rust/src/session.rs b/clients/rust/src/session.rs index bd04339..63f8189 100644 --- a/clients/rust/src/session.rs +++ b/clients/rust/src/session.rs @@ -630,6 +630,13 @@ impl Session { /// Open the per-session event stream from the beginning. /// + /// The returned [`EventStream`] yields [`EventItem`](crate::EventItem) + /// values — normal MXAccess events as + /// [`EventItem::Event`](crate::EventItem::Event). A stream opened from the + /// beginning never produces a + /// [`EventItem::ReplayGap`](crate::EventItem::ReplayGap); that sentinel + /// only appears on a resumed stream (see [`Session::events_after`]). + /// /// # Errors /// /// Returns the `tonic::Status` mapped through [`Error::from`] when the @@ -642,6 +649,15 @@ impl Session { /// `worker_sequence` is greater than `after_worker_sequence`. Pass `0` /// to receive every buffered event. /// + /// If `after_worker_sequence` predates the oldest event still retained in + /// the gateway's replay ring, the stream opens with a single + /// [`EventItem::ReplayGap`](crate::EventItem::ReplayGap) sentinel: recent + /// history was evicted and cannot be replayed, so the caller must discard + /// any local state and re-snapshot. To resume without provoking another + /// gap, call this method again with + /// `gap.oldest_available_sequence - 1`. See + /// [`EventItem`](crate::EventItem) for the full contract. + /// /// # Errors /// /// Same conditions as [`Session::events`]. diff --git a/clients/rust/tests/client_behavior.rs b/clients/rust/tests/client_behavior.rs index ae3f3d7..477fa81 100644 --- a/clients/rust/tests/client_behavior.rs +++ b/clients/rust/tests/client_behavior.rs @@ -27,13 +27,13 @@ use zb_mom_ww_mxgateway_client::generated::mxaccess_gateway::v1::{ CloseSessionReply, CloseSessionRequest, MxCommandKind, MxCommandReply, MxDataType, MxEvent, MxEventFamily, MxSparseArray, MxSparseElement, MxStatusCategory, MxStatusProxy, MxStatusSource, MxValue, OnAlarmTransitionEvent, OpenSessionReply, OpenSessionRequest, ProtocolStatus, - ProtocolStatusCode, QueryActiveAlarmsRequest, RegisterReply, SessionState, StreamAlarmsRequest, - StreamEventsRequest, SubscribeResult, Write2BulkEntry, WriteBulkEntry, WriteCommand, - WriteSecured2BulkEntry, WriteSecuredBulkEntry, + ProtocolStatusCode, QueryActiveAlarmsRequest, RegisterReply, ReplayGap, SessionState, + StreamAlarmsRequest, StreamEventsRequest, SubscribeResult, Write2BulkEntry, WriteBulkEntry, + WriteCommand, WriteSecured2BulkEntry, WriteSecuredBulkEntry, }; use zb_mom_ww_mxgateway_client::{ - next_correlation_id, ApiKey, ClientOptions, CommandError, Error, GatewayClient, MxStatus, - MxValue as ClientMxValue, MxValueProjection, + next_correlation_id, ApiKey, ClientOptions, CommandError, Error, EventItem, GatewayClient, + MxStatus, MxValue as ClientMxValue, MxValueProjection, }; #[tokio::test] @@ -128,8 +128,28 @@ async fn event_stream_preserves_order_and_drop_cancels_server_stream() { .await .unwrap(); - assert_eq!(stream.next().await.unwrap().unwrap().worker_sequence, 1); - assert_eq!(stream.next().await.unwrap().unwrap().worker_sequence, 2); + assert_eq!( + stream + .next() + .await + .unwrap() + .unwrap() + .as_event() + .unwrap() + .worker_sequence, + 1 + ); + assert_eq!( + stream + .next() + .await + .unwrap() + .unwrap() + .as_event() + .unwrap() + .worker_sequence, + 2 + ); drop(stream); for _ in 0..20 { @@ -142,6 +162,55 @@ async fn event_stream_preserves_order_and_drop_cancels_server_stream() { assert!(state.stream_dropped.load(Ordering::SeqCst)); } +#[tokio::test] +async fn replay_gap_sentinel_surfaces_as_typed_event_item() { + let state = Arc::new(FakeState::default()); + // Script a resumed stream: the reconnect-replay gap sentinel at the head + // (family UNSPECIFIED, no body, `replay_gap` set) followed by a normal + // event. The client must promote the sentinel to `EventItem::ReplayGap` + // and leave the following event as a normal `EventItem::Event`. + *state.stream_events_script.lock().await = Some(vec![ + MxEvent { + replay_gap: Some(ReplayGap { + requested_after_sequence: 5, + oldest_available_sequence: 42, + }), + ..MxEvent::default() + }, + event(42), + ]); + let endpoint = spawn_fake_gateway(state.clone()).await; + let client = GatewayClient::connect(ClientOptions::new(endpoint)) + .await + .unwrap(); + + let mut stream = client + .stream_events(StreamEventsRequest { + session_id: "session-fixture".to_owned(), + after_worker_sequence: 5, + }) + .await + .unwrap(); + + // First item is the typed gap sentinel, not a normal event. + let first = stream.next().await.unwrap().unwrap(); + match &first { + EventItem::ReplayGap(gap) => { + assert_eq!(gap.requested_after_sequence, 5); + assert_eq!(gap.oldest_available_sequence, 42); + } + EventItem::Event(_) => panic!("expected a ReplayGap sentinel, got a normal event"), + } + // Accessor helpers reflect the variant. + assert!(first.as_event().is_none()); + assert_eq!(first.replay_gap().unwrap().oldest_available_sequence, 42); + + // The normal event that follows is unaffected. + let second = stream.next().await.unwrap().unwrap(); + assert_eq!(second.as_event().unwrap().worker_sequence, 42); + assert!(second.replay_gap().is_none()); +} + #[tokio::test] async fn acknowledge_alarm_returns_reply_with_native_status() { let state = Arc::new(FakeState::default()); @@ -672,6 +741,11 @@ struct FakeState { /// handler to emit a synthetic ConditionRefresh -> snapshot_complete /// -> transition sequence. stream_alarms_script: Mutex>>, + /// Optional per-test override that pins the fake's `StreamEvents` + /// handler to emit a scripted `MxEvent` sequence (e.g. a `replay_gap` + /// sentinel followed by a normal event). When `None`, the handler falls + /// back to the default `event(1)` / `event(2)` pair. + stream_events_script: Mutex>>, } /// Per-test override for the fake's `Invoke` handler. @@ -921,9 +995,12 @@ impl MxAccessGateway for FakeGateway { &self, _request: Request, ) -> Result, Status> { - let (sender, receiver) = mpsc::channel(4); - sender.send(Ok(event(1))).await.unwrap(); - sender.send(Ok(event(2))).await.unwrap(); + let script = self.state.stream_events_script.lock().await.take(); + let events = script.unwrap_or_else(|| vec![event(1), event(2)]); + let (sender, receiver) = mpsc::channel(events.len().max(1)); + for event in events { + sender.send(Ok(event)).await.unwrap(); + } Ok(Response::new(DropAwareStream { inner: ReceiverStream::new(receiver), diff --git a/docs/ClientLibrariesDesign.md b/docs/ClientLibrariesDesign.md index c29fb1d..d6f9326 100644 --- a/docs/ClientLibrariesDesign.md +++ b/docs/ClientLibrariesDesign.md @@ -63,12 +63,29 @@ Goals: Non-goals for v1: -- client-side reconnectable sessions, -- client-side event replay, +- automatic client-side session reconnection (the library never transparently + re-opens a dropped stream on the consumer's behalf — reconnect timing and + policy stay an application concern), +- client-side event replay buffering (the gateway owns the replay ring; the + client does not retain its own event history), - client-side command batching, - synthetic MXAccess events, - hiding MXAccess handles behind opaque client-only handles. +The gateway's reconnect-replay *protocol* is, however, consumable from every +client: each exposes the `after_worker_sequence` resume cursor on +`StreamEvents` and surfaces the gateway's `ReplayGap` sentinel as a distinct, +typed, non-terminal signal (CLI-15) so a consumer can detect an evicted-history +gap and re-snapshot. Per the no-synthesized-events invariant the client never +fabricates or swallows the sentinel — it only makes the gateway's own signal +observable. The reaction contract is identical across languages: on a gap, +discard cached state, re-snapshot, and resume with +`after_worker_sequence = oldest_available_sequence - 1`. The per-language +surface follows each idiom (.NET `MxEventStreamItem.IsReplayGap` via +`StreamEventItemsAsync`; Go `EventResult.ReplayGap`/`IsReplayGap()`; Rust +`EventItem::ReplayGap`; Python yields a typed `ReplayGap` from +`stream_events`; Java — pending, tracked with the JDK-17/windows client pass). + ## Public Client Concepts All languages should expose the same core concepts, using idiomatic naming: diff --git a/docs/CrossLanguageSmokeMatrix.md b/docs/CrossLanguageSmokeMatrix.md index e6acb2d..af7c374 100644 --- a/docs/CrossLanguageSmokeMatrix.md +++ b/docs/CrossLanguageSmokeMatrix.md @@ -30,6 +30,14 @@ Each client entry defines commands for the same required operation sequence: The optional `write` command is documented separately because writing changes provider state and should only run when the operator supplies a safe test value. +When `stream-events` is resumed with an `after_worker_sequence` cursor that +predates the oldest event still in the gateway's replay ring, the gateway emits a +single `ReplayGap` sentinel at the head of the stream. Every client surfaces this +as a distinct, typed, non-terminal signal (see each client README); the resume +contract is `after_worker_sequence = oldest_available_sequence - 1`. The default +smoke sequence opens a fresh stream (no cursor) and does not exercise the gap +path; a resume-with-gap fixture case is tracked separately (TST-24). + ## Integration Gate Cross-language smoke execution is opt-in. Runners should skip the matrix unless -- 2.52.0 From fed0685633a8c9657d9b482b4f15f1cd52542f22 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Thu, 9 Jul 2026 16:26:40 -0400 Subject: [PATCH 17/25] test(gateway): TST-01 end-to-end reconnect/replay integration test Proves the default-on reconnect protocol end to end through the real gRPC StreamEvents path via the fake worker harness (no live COM). Two facts: - ReconnectInsideRetainedWindow_ReplaysTailNoGap: capacity 16 retains all events; reconnect from a mid-batch after_worker_sequence cursor replays exactly the events with WorkerSequence > cursor (retained tail + events emitted while detached), strictly ascending, distinct, no ReplayGap. - ReconnectWithStaleCursor_EmitsReplayGapSentinelFirst: capacity 3 with 6 events forces eviction; reconnect with AfterWorkerSequence=1 yields the ReplayGap sentinel first (Family=Unspecified, no body, RequestedAfterSequence=1, OldestAvailableSequence=oldest retained), then the retained tail ascending. Single-subscriber mode: the first stream is fully detached (cancel + await the stream task runs EventStreamService's finally, dropping the subscriber count to 0) before reconnect; the distributor + replay ring are created once per session and survive detach, so events emitted while detached are retained. macOS note: fake-worker E2E tests need TMPDIR=/tmp (default macOS TMPDIR pushes the CoreFxPipe Unix-socket path past the 104-byte sun_path limit). Windows CI is unaffected. Server reconnect/replay behavior matched the contract exactly. Claude-Session: https://claude.ai/code/session_01DMXXvNuPekkkrTEyPNxEkW --- .../GatewayEndToEndReconnectReplayTests.cs | 642 ++++++++++++++++++ 1 file changed, 642 insertions(+) create mode 100644 src/ZB.MOM.WW.MxGateway.Tests/Gateway/GatewayEndToEndReconnectReplayTests.cs diff --git a/src/ZB.MOM.WW.MxGateway.Tests/Gateway/GatewayEndToEndReconnectReplayTests.cs b/src/ZB.MOM.WW.MxGateway.Tests/Gateway/GatewayEndToEndReconnectReplayTests.cs new file mode 100644 index 0000000..424bd66 --- /dev/null +++ b/src/ZB.MOM.WW.MxGateway.Tests/Gateway/GatewayEndToEndReconnectReplayTests.cs @@ -0,0 +1,642 @@ +using Google.Protobuf.WellKnownTypes; +using Grpc.Core; +using Microsoft.Extensions.Logging.Abstractions; +using Microsoft.Extensions.Options; +using ZB.MOM.WW.MxGateway.Contracts; +using ZB.MOM.WW.MxGateway.Contracts.Proto; +using ZB.MOM.WW.MxGateway.Server.Configuration; +using ZB.MOM.WW.MxGateway.Server.Grpc; +using ZB.MOM.WW.MxGateway.Server.Metrics; +using ZB.MOM.WW.MxGateway.Server.Security.Authentication; +using ZB.MOM.WW.MxGateway.Server.Security.Authorization; +using ZB.MOM.WW.MxGateway.Server.Sessions; +using ZB.MOM.WW.MxGateway.Server.Workers; +using ZB.MOM.WW.MxGateway.Tests.Gateway.Workers.Fakes; +using ZB.MOM.WW.MxGateway.Tests.TestSupport; + +namespace ZB.MOM.WW.MxGateway.Tests.Gateway; + +/// +/// End-to-end reconnect/replay tests through the real gRPC StreamEvents path via the +/// fake worker harness (TST-01, server half). A single subscriber detaches (cancel + await +/// the stream task so its lease is fully disposed), the session is retained by detach-grace +/// while the distributor pump keeps appending worker events to the replay ring, and a second +/// stream reconnects with . Covers both +/// the no-gap resume (cursor inside the retained window) and the ReplayGap sentinel +/// (cursor predates the oldest retained event after capacity eviction). +/// +/// +/// These tests run in single-subscriber mode (AllowMultipleEventSubscribers=false). +/// Reconnect works because the first stream is FULLY detached before the reconnect attaches: +/// awaiting the first stream task runs EventStreamService's finally block, which +/// disposes the subscriber lease and drops the session's active-subscriber count back to +/// zero, so the reconnect's AttachEventSubscriberWithReplay sees an empty slot rather +/// than an "already active" rejection. The distributor (and its replay ring) is created once +/// per session and survives detach, so events emitted while no subscriber is attached are +/// retained for the reconnecting stream. +/// +public sealed class GatewayEndToEndReconnectReplayTests +{ + private static readonly TimeSpan TestTimeout = TimeSpan.FromSeconds(10); + private const int ServerHandle = 3001; + private const int ItemHandle = 4002; + + /// + /// Reconnecting inside the retained window replays exactly the events newer than the + /// resume cursor — the retained tail of the first batch plus the events emitted while + /// detached — in strictly ascending order, with no duplicates and no ReplayGap + /// sentinel. + /// + /// A task that represents the asynchronous operation. + [Fact] + public async Task StreamEvents_ReconnectInsideRetainedWindow_ReplaysTailNoGap() + { + const int firstBatch = 4; + const int secondBatch = 3; + + GatedEventFakeWorkerProcessLauncher launcher = new(); + // Capacity 16 retains every event emitted here (7 total), so nothing is evicted and a + // resume from a middle cursor never produces a gap. + await using ReconnectReplayGatewayServiceFixture fixture = new(launcher, replayBufferCapacity: 16); + + string sessionId = await OpenSessionAsync(fixture, "reconnect-no-gap"); + + // ---- first connection: receive the first batch, capture its real sequences ---- + using CancellationTokenSource writer1Cts = new(); + RecordingServerStreamWriter writer1 = new(); + Task stream1Task = Task.Run(async () => + await fixture.Service.StreamEvents( + new StreamEventsRequest { SessionId = sessionId }, + writer1, + new TestServerCallContext(cancellationToken: writer1Cts.Token))); + + await fixture.WaitForSubscriberCountAsync(sessionId, n: 1, TestTimeout); + await WireUpAdviseAsync(fixture, sessionId); + + for (int i = 0; i < firstBatch; i++) + { + launcher.AllowNextEvent(); + } + + IReadOnlyList batch1 = await writer1.WaitForMessageCountAsync(firstBatch, TestTimeout); + ulong[] batch1Sequences = batch1.Select(e => e.WorkerSequence).ToArray(); + + // Resume cursor: a middle event of the first batch. Everything strictly newer than this + // must be redelivered to the reconnecting stream. + ulong cursor = batch1Sequences[1]; + int retainedTailFromBatch1 = batch1Sequences.Count(s => s > cursor); + Assert.Equal(2, retainedTailFromBatch1); // sanity: events at index 2 and 3 + + // ---- fully detach writer1 (cancel + await so its lease is disposed) ---- + await DetachAsync(writer1Cts, stream1Task); + + // ---- emit more events while detached; the ring must retain them for the reconnect ---- + for (int i = 0; i < secondBatch; i++) + { + launcher.AllowNextEvent(); + } + + // ---- reconnect with the middle cursor ---- + RecordingServerStreamWriter writer2 = new(); + Task stream2Task = Task.Run(async () => + await fixture.Service.StreamEvents( + new StreamEventsRequest { SessionId = sessionId, AfterWorkerSequence = cursor }, + writer2, + new TestServerCallContext())); + + await fixture.WaitForSubscriberCountAsync(sessionId, n: 1, TestTimeout); + + int expected = retainedTailFromBatch1 + secondBatch; + IReadOnlyList resumed = await writer2.WaitForMessageCountAsync(expected, TestTimeout); + + // ---- tear down before asserting so a hang can't wedge the run ---- + launcher.StopEmitting(); + await CloseAndDrainAsync(fixture, sessionId, stream2Task, launcher); + + // ---- assertions ---- + Assert.Equal(expected, resumed.Count); + + // No sentinel: every message is a real event, none carries a ReplayGap. + Assert.All(resumed, e => Assert.Null(e.ReplayGap)); + + // Every replayed/live event is strictly newer than the cursor. + Assert.All(resumed, e => Assert.True( + e.WorkerSequence > cursor, + $"Event sequence {e.WorkerSequence} is not newer than cursor {cursor}.")); + + // Strictly ascending, no duplicates. + for (int i = 1; i < resumed.Count; i++) + { + Assert.True( + resumed[i].WorkerSequence > resumed[i - 1].WorkerSequence, + $"Sequences must be strictly ascending: {resumed[i - 1].WorkerSequence} then {resumed[i].WorkerSequence}."); + } + + Assert.Equal(resumed.Count, resumed.Select(e => e.WorkerSequence).Distinct().Count()); + + // The retained tail of the first batch (the two events newer than the cursor) is + // replayed first, before the events emitted while detached. + Assert.Equal(batch1Sequences[2], resumed[0].WorkerSequence); + Assert.Equal(batch1Sequences[3], resumed[1].WorkerSequence); + } + + /// + /// Reconnecting with a cursor that predates the oldest retained event (after capacity + /// eviction) yields the ReplayGap sentinel FIRST — family unspecified, no body, no + /// per-item fields, correct requested/oldest sequences — followed by exactly the retained + /// tail, in ascending order. + /// + /// A task that represents the asynchronous operation. + [Fact] + public async Task StreamEvents_ReconnectWithStaleCursor_EmitsReplayGapSentinelFirst() + { + const int capacity = 3; + const int totalEvents = 6; + + GatedEventFakeWorkerProcessLauncher launcher = new(); + // Small capacity forces eviction: with 6 events emitted and a 3-slot ring, the first 3 + // are evicted and only the newest 3 remain replayable. + await using ReconnectReplayGatewayServiceFixture fixture = new(launcher, replayBufferCapacity: capacity); + + string sessionId = await OpenSessionAsync(fixture, "reconnect-gap"); + + using CancellationTokenSource writer1Cts = new(); + RecordingServerStreamWriter writer1 = new(); + Task stream1Task = Task.Run(async () => + await fixture.Service.StreamEvents( + new StreamEventsRequest { SessionId = sessionId }, + writer1, + new TestServerCallContext(cancellationToken: writer1Cts.Token))); + + await fixture.WaitForSubscriberCountAsync(sessionId, n: 1, TestTimeout); + await WireUpAdviseAsync(fixture, sessionId); + + // Emit more events than the ring can hold. Waiting for writer1 to receive all of them + // proves the pump has appended (and evicted) every event, so the ring is settled to its + // final newest-`capacity` contents before we reconnect. + for (int i = 0; i < totalEvents; i++) + { + launcher.AllowNextEvent(); + } + + IReadOnlyList all = await writer1.WaitForMessageCountAsync(totalEvents, TestTimeout); + ulong[] sequences = all.Select(e => e.WorkerSequence).ToArray(); + + // With capacity C and N total events, the newest C are retained; the oldest still + // available is the (N-C+1)th event — index N-C in emission (ascending) order. + ulong oldestAvailable = sequences[totalEvents - capacity]; + + await DetachAsync(writer1Cts, stream1Task); + + // Resume after sequence 1: the real event sequences are envelope-numbered and start well + // above 1 (startup + command-reply envelopes consume the low numbers), so cursor 1 + // predates every event and, with the oldest three already evicted, forces a gap. + const ulong staleCursor = 1; + RecordingServerStreamWriter writer2 = new(); + Task stream2Task = Task.Run(async () => + await fixture.Service.StreamEvents( + new StreamEventsRequest { SessionId = sessionId, AfterWorkerSequence = staleCursor }, + writer2, + new TestServerCallContext())); + + await fixture.WaitForSubscriberCountAsync(sessionId, n: 1, TestTimeout); + + // sentinel + the retained tail (capacity events, all newer than the stale cursor). + int expected = 1 + capacity; + IReadOnlyList resumed = await writer2.WaitForMessageCountAsync(expected, TestTimeout); + + launcher.StopEmitting(); + await CloseAndDrainAsync(fixture, sessionId, stream2Task, launcher); + + // ---- the first message is the ReplayGap sentinel ---- + Assert.Equal(expected, resumed.Count); + + MxEvent sentinel = resumed[0]; + Assert.NotNull(sentinel.ReplayGap); + Assert.Equal(MxEventFamily.Unspecified, sentinel.Family); + Assert.Equal(MxEvent.BodyOneofCase.None, sentinel.BodyCase); + Assert.Equal(sessionId, sentinel.SessionId); + Assert.Equal(staleCursor, sentinel.ReplayGap.RequestedAfterSequence); + Assert.Equal(oldestAvailable, sentinel.ReplayGap.OldestAvailableSequence); + + // ---- the sentinel is followed by the retained tail, ascending, no further sentinels ---- + IReadOnlyList tail = resumed.Skip(1).ToArray(); + Assert.Equal(capacity, tail.Count); + Assert.All(tail, e => Assert.Null(e.ReplayGap)); + Assert.All(tail, e => Assert.True( + e.WorkerSequence >= oldestAvailable, + $"Retained event {e.WorkerSequence} is older than the oldest available {oldestAvailable}.")); + + ulong[] expectedTail = sequences.Skip(totalEvents - capacity).ToArray(); + Assert.Equal(expectedTail, tail.Select(e => e.WorkerSequence).ToArray()); + } + + // ---- shared flow helpers ---- + + private static async Task OpenSessionAsync( + ReconnectReplayGatewayServiceFixture fixture, + string name) + { + OpenSessionReply openReply = await fixture.Service.OpenSession( + new OpenSessionRequest + { + ClientSessionName = name, + ClientCorrelationId = $"open-{name}", + CommandTimeout = Duration.FromTimeSpan(TestTimeout), + }, + new TestServerCallContext()).ConfigureAwait(false); + + Assert.Equal(ProtocolStatusCode.Ok, openReply.ProtocolStatus.Code); + return openReply.SessionId; + } + + private static async Task WireUpAdviseAsync( + ReconnectReplayGatewayServiceFixture fixture, + string sessionId) + { + MxCommandReply registerReply = await fixture.Service.Invoke( + CreateRegisterRequest(sessionId), + new TestServerCallContext()).ConfigureAwait(false); + Assert.Equal(ProtocolStatusCode.Ok, registerReply.ProtocolStatus.Code); + + MxCommandReply addItemReply = await fixture.Service.Invoke( + CreateAddItemRequest(sessionId, registerReply.Register.ServerHandle), + new TestServerCallContext()).ConfigureAwait(false); + Assert.Equal(ProtocolStatusCode.Ok, addItemReply.ProtocolStatus.Code); + + MxCommandReply adviseReply = await fixture.Service.Invoke( + CreateAdviseRequest(sessionId, registerReply.Register.ServerHandle, addItemReply.AddItem.ItemHandle), + new TestServerCallContext()).ConfigureAwait(false); + Assert.Equal(ProtocolStatusCode.Ok, adviseReply.ProtocolStatus.Code); + } + + // Cancels the stream's token and awaits the stream task. Awaiting is load-bearing: it lets + // EventStreamService's finally block dispose the subscriber lease (dropping the session's + // active-subscriber count to zero) BEFORE the reconnect attaches, so the reconnect never + // races a still-registered subscriber. + private static async Task DetachAsync(CancellationTokenSource cts, Task streamTask) + { + await cts.CancelAsync().ConfigureAwait(false); + try + { + await streamTask.WaitAsync(TestTimeout).ConfigureAwait(false); + } + catch (OperationCanceledException) + { + // Expected: the iterator surfaces the cancellation. + } + catch (RpcException rpc) when (rpc.StatusCode == StatusCode.Cancelled) + { + // Also acceptable depending on gRPC exception wrapping. + } + } + + private static async Task CloseAndDrainAsync( + ReconnectReplayGatewayServiceFixture fixture, + string sessionId, + Task streamTask, + GatedEventFakeWorkerProcessLauncher launcher) + { + await fixture.Service.CloseSession( + new CloseSessionRequest { SessionId = sessionId, ClientCorrelationId = "close-reconnect" }, + new TestServerCallContext()).ConfigureAwait(false); + + try + { + await streamTask.WaitAsync(TestTimeout).ConfigureAwait(false); + } + catch (OperationCanceledException) + { + } + + await launcher.WorkerTask.WaitAsync(TestTimeout).ConfigureAwait(false); + } + + // ---- request builders ---- + + private static MxCommandRequest CreateRegisterRequest(string sessionId) => + new() + { + SessionId = sessionId, + ClientCorrelationId = "register-rr", + Command = new MxCommand + { + Kind = MxCommandKind.Register, + Register = new RegisterCommand { ClientName = "reconnect-replay-e2e-client" }, + }, + }; + + private static MxCommandRequest CreateAddItemRequest(string sessionId, int serverHandle) => + new() + { + SessionId = sessionId, + ClientCorrelationId = "add-item-rr", + Command = new MxCommand + { + Kind = MxCommandKind.AddItem, + AddItem = new AddItemCommand + { + ServerHandle = serverHandle, + ItemDefinition = "Galaxy.Tag.Value", + }, + }, + }; + + private static MxCommandRequest CreateAdviseRequest( + string sessionId, + int serverHandle, + int itemHandle) => + new() + { + SessionId = sessionId, + ClientCorrelationId = "advise-rr", + Command = new MxCommand + { + Kind = MxCommandKind.Advise, + Advise = new AdviseCommand { ServerHandle = serverHandle, ItemHandle = itemHandle }, + }, + }; + + private static void ConfigureCommandReply(MxCommandReply reply, MxCommandKind kind) + { + switch (kind) + { + case MxCommandKind.Register: + reply.Register = new RegisterReply { ServerHandle = ServerHandle }; + break; + case MxCommandKind.AddItem: + reply.AddItem = new AddItemReply { ItemHandle = ItemHandle }; + break; + } + } + + // ---- fixture ---- + + /// + /// Gateway service fixture in single-subscriber mode with a configurable replay ring + /// capacity, so each test can retain the whole event history (no gap) or force capacity + /// eviction (gap). + /// + private sealed class ReconnectReplayGatewayServiceFixture : IAsyncDisposable + { + private readonly GatewayMetrics _metrics = new(); + private readonly SessionRegistry _registry = new(); + + /// Initializes a new instance of the class. + /// Fake worker process launcher backing the session manager. + /// Replay ring capacity for the session's event distributor. + public ReconnectReplayGatewayServiceFixture( + IWorkerProcessLauncher launcher, + int replayBufferCapacity) + { + IOptions options = Options.Create(CreateOptions(replayBufferCapacity)); + SessionWorkerClientFactory workerClientFactory = new( + launcher, + options, + _metrics, + NullLoggerFactory.Instance); + SessionManager sessionManager = new( + _registry, + workerClientFactory, + options, + _metrics, + logger: NullLogger.Instance, + dashboardEventBroadcaster: NullDashboardEventBroadcaster.Instance); + MxAccessGrpcMapper mapper = new(); + EventStreamService eventStreamService = new( + sessionManager, + options, + _metrics); + + Service = new MxAccessGatewayService( + sessionManager, + new GatewayRequestIdentityAccessor(), + new AllowAllConstraintEnforcer(), + new MxAccessGrpcRequestValidator(), + mapper, + eventStreamService, + _metrics, + NullLogger.Instance, + new FakeGatewayAlarmService()); + } + + /// Gets the gateway service under test. + public MxAccessGatewayService Service { get; } + + /// + /// Polls for + /// until it reaches , bounded by + /// . Fails the test on timeout. This is the deterministic + /// gate that proves the production code has (re)registered a subscriber before the + /// test drives events or reconnects. + /// + /// Identifier of the session to poll. + /// Target subscriber count to wait for. + /// Maximum time to wait before failing the test. + /// A task that represents the asynchronous operation. + public async Task WaitForSubscriberCountAsync(string sessionId, int n, TimeSpan timeout) + { + using CancellationTokenSource deadlineCts = new(timeout); + + while (true) + { + if (_registry.TryGet(sessionId, out GatewaySession? session) + && session.ActiveEventSubscriberCount >= n) + { + return; + } + + if (deadlineCts.IsCancellationRequested) + { + int actual = _registry.TryGet(sessionId, out GatewaySession? s) + ? s.ActiveEventSubscriberCount + : -1; + Assert.Fail( + $"Timed out waiting for {n} event subscriber(s) on session {sessionId}. " + + $"Actual count after {timeout.TotalSeconds:0.#}s: {actual}."); + } + + await Task.Delay(millisecondsDelay: 5, deadlineCts.Token).ConfigureAwait(false); + } + } + + /// Disposes every session in the registry and releases the fixture's metrics. + /// A task that represents the asynchronous operation. + public async ValueTask DisposeAsync() + { + foreach (GatewaySession session in _registry.Snapshot()) + { + await session.DisposeAsync().ConfigureAwait(false); + } + + _metrics.Dispose(); + } + + private static GatewayOptions CreateOptions(int replayBufferCapacity) => + new() + { + Worker = new WorkerOptions + { + StartupTimeoutSeconds = 5, + ShutdownTimeoutSeconds = 5, + HeartbeatIntervalSeconds = 30, + HeartbeatGraceSeconds = 30, + MaxMessageBytes = WorkerFrameProtocolOptions.DefaultMaxMessageBytes, + }, + Sessions = new SessionOptions + { + DefaultCommandTimeoutSeconds = 5, + MaxSessions = 4, + + // Single-subscriber mode: a fully-detached subscriber (stream task awaited) + // frees the sole slot, so the reconnect attaches cleanly without needing + // multi-subscriber fan-out. Detach-grace keeps the session Ready across the + // detach and the fixture runs no lease-reaper, so the session survives to be + // reconnected. + AllowMultipleEventSubscribers = false, + MaxEventSubscribersPerSession = 8, + }, + Events = new EventOptions + { + QueueCapacity = 32, + ReplayBufferCapacity = replayBufferCapacity, + + // Keep age-eviction effectively off for the duration of a fast test so + // capacity is the only eviction axis under test. + ReplayRetentionSeconds = 300, + }, + }; + } + + // ---- fake worker launcher ---- + + /// + /// Fake worker that emits events one at a time, gated by , so + /// the test drives event timing deterministically. Modeled on the multi-subscriber E2E + /// tests' gated launcher. The worker loop is independent of subscribers, so events emitted + /// while no gRPC stream is attached still flow through the distributor pump into the + /// session's replay ring — exactly the condition the reconnect/replay tests exercise. Call + /// before closing the session so the loop exits cleanly and can + /// process the shutdown envelope. + /// + private sealed class GatedEventFakeWorkerProcessLauncher : IWorkerProcessLauncher + { + public const int ProcessId = 7730; + + private readonly FakeWorkerProcess _process = new(ProcessId); + + // Capacity 64 so AllowNextEvent can be called ahead of time without blocking. + private readonly SemaphoreSlim _emitGate = new(0, 64); + private volatile bool _stopEmitting; + + /// Gets the task representing the fake worker's running background loop. + public Task WorkerTask { get; private set; } = Task.CompletedTask; + + /// Releases the gate so the worker emits one event. + public void AllowNextEvent() => _emitGate.Release(); + + /// + /// Signals the worker to stop waiting for the emit gate and process the shutdown + /// envelope. Must be called before CloseSession. + /// + public void StopEmitting() + { + _stopEmitting = true; + _emitGate.Release(); // unblock a pending gate wait if any + } + + /// + public Task LaunchAsync( + WorkerProcessLaunchRequest request, + CancellationToken cancellationToken = default) + { + WorkerTask = RunWorkerAsync(request, cancellationToken); + + return Task.FromResult(new WorkerProcessHandle( + _process, + new WorkerProcessCommandLine("reconnect-replay-fake-worker.exe", []), + DateTimeOffset.UtcNow)); + } + + private async Task RunWorkerAsync( + WorkerProcessLaunchRequest request, + CancellationToken cancellationToken) + { + await using FakeWorkerHarness harness = await FakeWorkerHarness.ConnectToGatewayPipeAsync( + request.SessionId, + request.Nonce, + request.PipeName, + request.ProtocolVersion, + cancellationToken: cancellationToken).ConfigureAwait(false); + await harness.CompleteStartupAsync(ProcessId, cancellationToken: cancellationToken).ConfigureAwait(false); + + int advisedServerHandle = 0; + int advisedItemHandle = 0; + int emittedCount = 0; + + while (!cancellationToken.IsCancellationRequested) + { + // While subscribed and not stopped, emit gated events using a non-blocking peek + // at the gate so incoming envelopes (including shutdown) are never starved. + while (advisedServerHandle != 0 + && !_stopEmitting + && await _emitGate.WaitAsync(millisecondsTimeout: 0).ConfigureAwait(false)) + { + int index = ++emittedCount; + await harness.EmitEventAsync( + MxEventFamily.OnDataChange, + cancellationToken, + mxEvent => + { + mxEvent.ServerHandle = advisedServerHandle; + mxEvent.ItemHandle = advisedItemHandle; + mxEvent.Quality = 192; + mxEvent.Value = new MxValue + { + DataType = MxDataType.String, + StringValue = $"reconnect-value-{index}", + }; + mxEvent.OnDataChange = new OnDataChangeEvent(); + }).ConfigureAwait(false); + } + + WorkerEnvelope? envelope; + try + { + using CancellationTokenSource readCts = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken); + readCts.CancelAfter(TimeSpan.FromMilliseconds(50)); + envelope = await harness.ReadGatewayEnvelopeAsync(readCts.Token).ConfigureAwait(false); + } + catch (OperationCanceledException) when (!cancellationToken.IsCancellationRequested) + { + // Timed out waiting for an envelope — loop back to check the gate / emit. + continue; + } + + if (envelope.BodyCase == WorkerEnvelope.BodyOneofCase.WorkerShutdown) + { + await harness.SendShutdownAckAsync(cancellationToken: cancellationToken).ConfigureAwait(false); + _process.MarkExited(0); + return; + } + + if (envelope.BodyCase != WorkerEnvelope.BodyOneofCase.WorkerCommand) + { + throw new InvalidOperationException($"Unexpected envelope {envelope.BodyCase}."); + } + + MxCommand command = envelope.WorkerCommand.Command; + await harness.ReplyToCommandAsync( + envelope, + configureReply: reply => ConfigureCommandReply(reply, command.Kind), + cancellationToken: cancellationToken).ConfigureAwait(false); + + if (command.Kind == MxCommandKind.Advise) + { + advisedServerHandle = command.Advise.ServerHandle; + advisedItemHandle = command.Advise.ItemHandle; + } + } + } + } +} -- 2.52.0 From ef498c80c237c9d63d5183ec13e7274175239a22 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Thu, 9 Jul 2026 16:27:47 -0400 Subject: [PATCH 18/25] =?UTF-8?q?docs(archreview):=20E1=20reconnect-replay?= =?UTF-8?q?=20=E2=80=94=20CLI-15=20+=20TST-01=20In=20progress?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CLI-15 (4/5 clients, commit 0c6e5b3) and TST-01 server e2e test (commit fed0685) landed; both gated on the Java client (windev batch) before flipping to Done. Claude-Session: https://claude.ai/code/session_01DMXXvNuPekkkrTEyPNxEkW --- archreview/remediation/00-tracking.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/archreview/remediation/00-tracking.md b/archreview/remediation/00-tracking.md index 85172d4..34bddc8 100644 --- a/archreview/remediation/00-tracking.md +++ b/archreview/remediation/00-tracking.md @@ -188,7 +188,7 @@ Full design + implementation for each row lives in the linked domain doc under i | CLI-12 | Medium | P2 | S | — | Done | Java docs still say "Java 21" after the JDK 17 retarget | | CLI-13 | Medium | — | M | — | Not started | Java event-stream buffer hardcoded 16, cancel-on-overflow | | CLI-14 | Medium | — | S | — | Not started | Python default TLS is blocking TOFU pin with silent `localhost` SNI | -| CLI-15 | Medium | P2 | M | — | Not started | No client handles `ReplayGap` or offers a reconnect helper | +| CLI-15 | Medium | P2 | M | — | In progress | No client handles `ReplayGap` or offers a reconnect helper | | CLI-16 | Medium | P2 | S | CLI-12 | Done | `docs/ClientPackaging.md` drifted from Python naming and .NET `.slnx` | | CLI-17 | Medium | — | M | CLI-14 | Not started | TLS default posture inconsistent across the five clients | | CLI-18 | Low | P2 | S | — | Done | .NET csproj records no `` | @@ -213,7 +213,7 @@ Full design + implementation for each row lives in the linked domain doc under i | ID | Sev | Tier | Eff | Dep | Status | Title | |---|---|:-:|:-:|---|---|---| -| TST-01 | High | P2 | L | TST-04 | Not started | Reconnect/replay has no e2e test and no client consumer | +| TST-01 | High | P2 | L | TST-04 | In progress | Reconnect/replay has no e2e test and no client consumer | | TST-02 | High | P0 | M | TST-04 | Done | Reconnect owner re-validation not implemented | | TST-03 | High | P1 | M | — | In review | No CI exists | | TST-04 | High | P2 | L | — | Not started | Session-resilience epic 16/28 tasks unfinished | @@ -253,6 +253,7 @@ Findings the review flagged as one coordinated design pass — sequence them tog | Date | Change | |---|---| +| 2026-07-09 | P2 Epic Wave E1 — reconnect-replay: CLI-15 + TST-01 → `In progress` (both gated on the Java client, deferred to the windev batch). **CLI-15 (commit `0c6e5b3`)**: four clients now surface the gateway's `ReplayGap` reconnect sentinel as a distinct, typed, non-terminal signal (never synthesized, never swallowed) — .NET `MxEventStreamItem`/`StreamEventItemsAsync` (build clean, 87 passed), Go `EventResult.ReplayGap`+`IsReplayGap()` (build/vet/test clean), Rust `EventItem::ReplayGap` enum (`EventStream` now yields `Result`; fmt/check/test/clippy clean), Python `ReplayGap` dataclass yielded from `stream_events` (131 passed). Shared docs: `ClientLibrariesDesign.md` non-goals reframed (reconnect-replay protocol is consumable; auto-reconnect stays a non-goal) + `CrossLanguageSmokeMatrix.md` resume-gap note. Resume contract uniform: `after_worker_sequence = oldest_available_sequence - 1`. **TST-01 server half (commit `fed0685`)**: `GatewayEndToEndReconnectReplayTests` (2 facts) proves the default-on reconnect protocol end-to-end over the real gRPC StreamEvents path via the fake worker — replay-tail-no-gap (capacity 16, mid-batch cursor) and stale-cursor-emits-ReplayGap-first (capacity 3, 6 events force eviction, cursor=1); single-subscriber detach→reconnect, ring survives detach. macOS-verified 2/2 (needs `TMPDIR=/tmp` — default macOS TMPDIR pushes the CoreFxPipe Unix-socket path past the 104-byte `sun_path` limit; Windows CI unaffected). No product bugs; server behavior matched the contract exactly. Remaining to close both: Java `ReplayGap` surface (windev) + Java client consumer for TST-01. | | 2026-07-09 | P2 Wave B — worker event hot-path (commit `f61c816`), **windev-verified**. WRK-06 → `Done`: `MxStatusProxyConverter` caches the four resolved `FieldInfo` per status type in a static `ConcurrentDictionary` (the `GetField` metadata scan ran 4× per status per event on the STA path); `GetValue`+`Convert.ToInt32` still run per event (late-bound RCW); both exception messages byte-identical (missing-field via `ResolveField`, not cached on throw through `GetOrAdd`; null-value unchanged). WRK-11 → `Done`: `MxAccessEventQueue.Enqueue` takes ownership of the passed `MxEvent` (stamps `WorkerSequence`/`WorkerTimestamp` in place, no `Clone()`); audited all 3 callers (base/alarm event sinks + provider-mode handler) — each builds a fresh event, none reuse it; `MxAccessValueCache.Set` now deep-copies its retained `Value`/`SourceTimestamp`/`Statuses` so the cache snapshot never aliases the queue-owned (later serialized) event. WRK-12 → `Done`: `WorkerFrameWriter` coalesces the flush across a drained batch (write each frame, one `FlushAsync` after the batch, then complete all) — preserves the written+flushed completion contract; a burst of N events costs 1 flush not N; on failure the in-flight batch + queue all fail so no caller hangs. IPC-15 → `Done`: the multi-event `WorkerEnvelope` body stays unimplemented (wire still one event per `worker_event` frame); gateway.md Performance section now distinguishes the shipped flush-coalescing from that deferred additive-proto change. **Windev:** x86 Worker builds clean (fixed a Windows-only `TreatWarningsAsErrors` CS8604 in the value-cache null-guard that macOS can't surface); Worker.Tests **356 passed / 0 failed / 11 skipped** (+4 new: converter cache-reuse, queue ownership-transfer, value-cache snapshot independence, writer batch-flush-once); gateway Tests **815 passed / 1 failed** — the 1 is the known windev-environmental SelfSigned-SAN assertion (passes on macOS), and all pipe-harness tests (which also exercise the earlier G-frame gateway frame change + WRK-12 end-to-end over a real pipe) pass. | | 2026-07-09 | P2 Wave B — quick Mac items (commit `ec6f82b`). TST-11 → `Done`: `src/Directory.Build.props` single-sources the .NET-side version (Server/Worker/Contracts/tests stamped 0.1.2, was SDK-default 1.0.0) and appends the git short SHA to InformationalVersion (`0.1.2+`, guarded so a git-less build still works); verified Server assembly stamps `0.1.2+579282f`, Server+Tests build clean. Kept 0.1.2 (matches Contracts + aligned Python/Rust/Go); converging all to a single 0.2.0 cadence left as a release decision. WRK-15 → `Done` (docs-only path, no x86 build): STA thread-name refs corrected to the actual `MxGateway.Worker.STA` in `docs/WorkerSta.md` + `docs/MxAccessWorkerInstanceDesign.md`, and the stale heartbeat-counter note fixed (CaptureHeartbeat populates queue depth + sequence from the live queue). TST-23 → `Done`: already resolved by the Wave A gateway.md edit — the bidi `Session` RPC now sits under a "Future work: not implemented" heading (no "best long-term shape" framing), consistent with the proto. **TST-14 left open by design**: its only concrete step is deleting the user's untracked, gitignored `*-docs-*.md` working files (zero repo impact) — surfaced to the user rather than deleting files not created here. | | 2026-07-09 | P2 Wave B — dashboard/security (commit `e15c3cb`). SEC-25 → `Done` (near-term hardening only; full per-session EventsHub ACL stays deferred to roadmap item 12 / TST-15): `DashboardEventBroadcaster` redacts tag values from a deep clone of the event before mirroring to SignalR when `Dashboard:ShowTagValues` is false (default), so the hub seam cannot leak values regardless of the missing ACL. Clears `MxEvent.value` (the OnDataChange/OnWriteComplete/OperationComplete/OnBufferedDataChange bodies are empty discriminators — values ride in the top-level field, incl. buffered arrays) + the alarm body `current_value`/`limit_value`; source event never mutated (shared with gRPC path + replay ring; verified). Makes the formerly-dead `ShowTagValues` flag live for the mirror. `EventsHub` TODO(per-session-acl) kept + tied to roadmap item 12. Test `DashboardEventBroadcasterTests` (3). SEC-30 → `Done`: `docs/Diagnostics.md` trimmed to mark opt-in command-value logging NOT YET IMPLEMENTED (no `LogCommandValues` knob, `RedactCommandValue` unwired, no values logged); wiring deferred pending secured-bulk redaction (SEC-13), not wired now. Server build clean; Dashboard tests 152/152. | -- 2.52.0 From ed3c6c61c50ba3374ad13fb52fdd048d7b1b762d Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Thu, 9 Jul 2026 16:33:25 -0400 Subject: [PATCH 19/25] =?UTF-8?q?docs(archreview):=20TST-04=20governance?= =?UTF-8?q?=20=E2=80=94=20Phase=203=20done,=20Phase=204=20scoped,=20Phase?= =?UTF-8?q?=205=20deferred?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Resolve the session-resilience epic's shipped-vs-planned entanglement: - Phase 3 (reconnect) finished: Task 13 = TST-02 (owner-scoped attach, P0), Task 15 = TST-01 (reconnect integration test), Task 14 = CLI-15 for 4/5 clients (Java pending, windev batch). - Phase 4 (per-session dashboard ACL) scoped as TST-15; the open Viewer-default decision is settled: admin-sees-all, Viewer strict per owned/granted session (matches TST-02's gRPC owner binding). - Phase 5 (orphan-worker reattach) marked DEFERRED, not planned. The EnableOrphanReattach flag does not exist and must not be referenced as if it does. The CLAUDE.md "gateway restart does not reattach orphan workers" invariant stands. Updates oldtasks.md, the tasks.json mirror (statuses + governance note), and the CLAUDE.md reconnect paragraph (clients now consume ReplayGap; reattach deferred). Claude-Session: https://claude.ai/code/session_01DMXXvNuPekkkrTEyPNxEkW --- CLAUDE.md | 2 +- ...026-06-15-session-resilience.md.tasks.json | 35 +++++----- oldtasks.md | 69 +++++++++++++------ 3 files changed, 67 insertions(+), 39 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 73c7c0c..a1ddf64 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -76,7 +76,7 @@ powershell -ExecutionPolicy Bypass -File scripts/run-client-e2e-tests.ps1 - **Style guides** in `docs/style-guides/` are authoritative. Follow `CSharpStyleGuide.md` for gateway/worker/.NET-client code: file-scoped namespaces, `sealed` by default, `Async` suffix on Task-returning methods, MXAccess-aligned names (`MxStatusProxy`, `ServerHandle`, `ItemHandle`, `HResult`). - **MXAccess parity is the contract.** Don't "fix" surprising MXAccess behavior (e.g., `WriteSecured` failing before a value-bearing NMX body, distinct `OperationComplete` semantics, invalid-handle exceptions) unless the client explicitly opts into a non-parity mode. The installed MXAccess COM component is the baseline. - **Don't synthesize events.** The gateway forwards only events the worker emits; it never invents `OperationComplete` from write completion or command replies. -- **One worker per session** (invariant). Multi-subscriber event fan-out and reconnect-with-replay have shipped and are config-gated: `AllowMultipleEventSubscribers` (default `false`) enables fan-out up to `MaxEventSubscribersPerSession` (default `8`); `DetachGraceSeconds` (default `30`) retains a session after its last subscriber drops so clients can reconnect; `ReplayBufferCapacity` / `ReplayRetentionSeconds` control how much event history the replay ring keeps. Default config is single-subscriber (`AllowMultipleEventSubscribers` off), but detach-grace and replay retention are **on** by default (`DetachGraceSeconds=30`, `ReplayBufferCapacity=1024`, `ReplayRetentionSeconds=300`): a detached session is retained for 30 s and recent events are buffered for reconnect. See `docs/DesignDecisions.md` and `docs/Sessions.md`. +- **One worker per session** (invariant). Multi-subscriber event fan-out and reconnect-with-replay have shipped and are config-gated: `AllowMultipleEventSubscribers` (default `false`) enables fan-out up to `MaxEventSubscribersPerSession` (default `8`); `DetachGraceSeconds` (default `30`) retains a session after its last subscriber drops so clients can reconnect; `ReplayBufferCapacity` / `ReplayRetentionSeconds` control how much event history the replay ring keeps. Default config is single-subscriber (`AllowMultipleEventSubscribers` off), but detach-grace and replay retention are **on** by default (`DetachGraceSeconds=30`, `ReplayBufferCapacity=1024`, `ReplayRetentionSeconds=300`): a detached session is retained for 30 s and recent events are buffered for reconnect. The reconnect protocol is consumable end-to-end: a resuming `StreamEvents` (via `after_worker_sequence`) that predates the retained ring gets a `ReplayGap` sentinel, and the official clients surface it as a typed signal (shipped for .NET/Go/Rust/Python; the Java client is the remaining one, batched to windev). Orphan-worker reattach after a gateway restart is **deferred, not planned** — see `oldtasks.md` (session-resilience epic Phase 5); the invariant on the next line stands. See `docs/DesignDecisions.md` and `docs/Sessions.md`. - **Gateway restart does not reattach orphan workers.** The first version terminates orphaned workers on startup; do not design code paths that assume reattachment. - **No Blazor UI component libraries.** Dashboard uses local Bootstrap CSS/JS only — do not introduce MudBlazor, Radzen, FluentUI, etc. - **Don't log secrets or full tag values by default.** API keys, passwords, `WriteSecured` payloads, and `AuthenticateUser` credentials must never reach logs. Value logging is opt-in and redacted. diff --git a/docs/plans/2026-06-15-session-resilience.md.tasks.json b/docs/plans/2026-06-15-session-resilience.md.tasks.json index 9880069..4e8f87a 100644 --- a/docs/plans/2026-06-15-session-resilience.md.tasks.json +++ b/docs/plans/2026-06-15-session-resilience.md.tasks.json @@ -13,22 +13,23 @@ {"id": 117, "subject": "Task 10: Proto - ReplayGap signal", "status": "completed", "blockedBy": [116]}, {"id": 118, "subject": "Task 11: Detach-grace session retention", "status": "completed", "blockedBy": [117]}, {"id": 119, "subject": "Task 12: Replay-on-reconnect + emit ReplayGap", "status": "completed", "blockedBy": [118, 110]}, - {"id": 120, "subject": "Task 13: Owner re-validation on reconnect", "status": "pending", "blockedBy": [119, 108]}, - {"id": 121, "subject": "Task 14: Client ReplayGap handling - all 5 clients", "status": "pending", "blockedBy": [117]}, - {"id": 122, "subject": "Task 15: Reconnect integration test (fake worker)", "status": "pending", "blockedBy": [119]}, - {"id": 123, "subject": "Task 16: gRPC session-owner gate + all-sessions admin scope", "status": "pending", "blockedBy": [116, 108]}, - {"id": 124, "subject": "Task 17: Session Tag + dashboard group-to-tag config", "status": "pending", "blockedBy": [116]}, - {"id": 125, "subject": "Task 18: EventsHub per-session ACL + hub-token tag claim", "status": "pending", "blockedBy": [124]}, - {"id": 126, "subject": "Task 19: ACL tests incl. live LDAP users", "status": "pending", "blockedBy": [125]}, - {"id": 127, "subject": "Task 20: Stable gateway-instance id + stable pipe naming", "status": "pending", "blockedBy": [126]}, - {"id": 128, "subject": "Task 21: Adoption manifest store (SQLite)", "status": "pending", "blockedBy": [127]}, - {"id": 129, "subject": "Task 22: Proto - worker adopt/reconnect frame", "status": "pending", "blockedBy": [128]}, - {"id": 130, "subject": "Task 23: Worker phone-home reconnect loop + self-terminate", "status": "pending", "blockedBy": [129]}, - {"id": 131, "subject": "Task 24: Gateway adoption - re-open pipes, nonce-validate, reject impostors", "status": "pending", "blockedBy": [130]}, - {"id": 132, "subject": "Task 25: Resync adopted worker + ReplayGap to subscribers", "status": "pending", "blockedBy": [131, 119]}, - {"id": 133, "subject": "Task 26: EnableOrphanReattach flag (default off) + terminator fallback", "status": "pending", "blockedBy": [131]}, - {"id": 134, "subject": "Task 27: Gateway-restart reattach round-trip (WINDEV + live worker)", "status": "pending", "blockedBy": [132, 133]}, - {"id": 135, "subject": "Task 28: Documented-rule reversals + stillpending refresh", "status": "pending", "blockedBy": [134]} + {"id": 120, "subject": "Task 13: Owner re-validation on reconnect", "status": "completed", "blockedBy": [119, 108], "note": "Shipped as archreview TST-02 (P0): session attach is owner-scoped."}, + {"id": 121, "subject": "Task 14: Client ReplayGap handling - all 5 clients", "status": "in_progress", "blockedBy": [117], "note": "Shipped as archreview CLI-15 for .NET/Go/Rust/Python; Java pending (windev batch)."}, + {"id": 122, "subject": "Task 15: Reconnect integration test (fake worker)", "status": "completed", "blockedBy": [119], "note": "Shipped as archreview TST-01: GatewayEndToEndReconnectReplayTests."}, + {"id": 123, "subject": "Task 16: gRPC session-owner gate + all-sessions admin scope", "status": "pending", "blockedBy": [116, 108], "note": "gRPC owner gate exists (TST-02); Phase 4 adds admin all-sessions scope. Tracked as TST-15."}, + {"id": 124, "subject": "Task 17: Session Tag + dashboard group-to-tag config", "status": "pending", "blockedBy": [116], "note": "Phase 4 / TST-15."}, + {"id": 125, "subject": "Task 18: EventsHub per-session ACL + hub-token tag claim", "status": "pending", "blockedBy": [124], "note": "Phase 4 / TST-15. Decision settled: admin-sees-all, Viewer strict per owned/granted session."}, + {"id": 126, "subject": "Task 19: ACL tests incl. live LDAP users", "status": "pending", "blockedBy": [125], "note": "Phase 4 / TST-15."}, + {"id": 127, "subject": "Task 20: Stable gateway-instance id + stable pipe naming", "status": "deferred", "blockedBy": [126], "note": "Phase 5 deferred, not planned (TST-04)."}, + {"id": 128, "subject": "Task 21: Adoption manifest store (SQLite)", "status": "deferred", "blockedBy": [127], "note": "Phase 5 deferred, not planned (TST-04)."}, + {"id": 129, "subject": "Task 22: Proto - worker adopt/reconnect frame", "status": "deferred", "blockedBy": [128], "note": "Phase 5 deferred, not planned (TST-04)."}, + {"id": 130, "subject": "Task 23: Worker phone-home reconnect loop + self-terminate", "status": "deferred", "blockedBy": [129], "note": "Phase 5 deferred, not planned (TST-04)."}, + {"id": 131, "subject": "Task 24: Gateway adoption - re-open pipes, nonce-validate, reject impostors", "status": "deferred", "blockedBy": [130], "note": "Phase 5 deferred, not planned (TST-04)."}, + {"id": 132, "subject": "Task 25: Resync adopted worker + ReplayGap to subscribers", "status": "deferred", "blockedBy": [131, 119], "note": "Phase 5 deferred, not planned (TST-04)."}, + {"id": 133, "subject": "Task 26: EnableOrphanReattach flag (default off) + terminator fallback", "status": "deferred", "blockedBy": [131], "note": "Phase 5 deferred, not planned (TST-04). The EnableOrphanReattach flag does not exist yet."}, + {"id": 134, "subject": "Task 27: Gateway-restart reattach round-trip (WINDEV + live worker)", "status": "deferred", "blockedBy": [132, 133], "note": "Phase 5 deferred, not planned (TST-04)."}, + {"id": 135, "subject": "Task 28: Documented-rule reversals + stillpending refresh", "status": "deferred", "blockedBy": [134], "note": "Phase 5 deferred, not planned (TST-04)."} ], - "lastUpdated": "2026-06-15" + "governance": "2026-07-09 (archreview TST-04): Phase 3 finished (Task 13=TST-02, Task 15=TST-01, Task 14=CLI-15 4/5, Java pending). Phase 4 (Tasks 16-19) scoped as TST-15, Viewer decision settled (admin-sees-all + Viewer strict). Phase 5 (Tasks 20-28) DEFERRED, not planned; EnableOrphanReattach does not exist.", + "lastUpdated": "2026-07-09" } diff --git a/oldtasks.md b/oldtasks.md index d675f45..16fa7f7 100644 --- a/oldtasks.md +++ b/oldtasks.md @@ -15,8 +15,27 @@ The authoritative resume state lives in ## Status -**12 of 28 tasks complete** (Phases 1–2 + reconnect core of Phase 3). All completed -work is merged to `main` (commit `c446bef`, pushed to origin). +**Governance update 2026-07-09 (archreview TST-04).** The epic is resolved into three +decisions rather than one open backlog: + +- **Phase 3 (reconnect) — finishing now, essentially complete.** Task 13 (owner + re-validation) shipped as archreview **TST-02** (P0, session attach is owner-scoped, + see CLAUDE.md Authentication). Task 15 (reconnect integration test) shipped as + **TST-01** (`GatewayEndToEndReconnectReplayTests`). Task 14 (client `ReplayGap` + handling) shipped as **CLI-15** for four of five clients (.NET/Go/Rust/Python); + the Java client is the only remainder, batched to the windev build host. +- **Phase 4 (per-session dashboard ACL) — scoped, not yet built.** Tracked as archreview + **TST-15**. The previously-open Viewer-default decision is **settled**: admin-sees-all, + Viewer strictly scoped to sessions it owns/is granted — matching TST-02's gRPC owner + binding for consistency. +- **Phase 5 (orphan-worker reattach) — DEFERRED, not planned.** It reverses the CLAUDE.md + invariant "Gateway restart does not reattach orphan workers" and adds an adoption manifest + store + worker phone-home protocol. It stays deferred unless a concrete requirement + appears. **The `EnableOrphanReattach` flag (Task 26) does not exist and must not be + referenced anywhere as if it does** until that task actually lands. + +Original snapshot (historical): **12 of 28 tasks complete** (Phases 1–2 + reconnect core of +Phase 3), merged to `main` (commit `c446bef`). ### Completed — Phase 1 (Foundation) - ✅ Task 1 (#108): Add OwnerKeyId to the session @@ -36,33 +55,41 @@ work is merged to `main` (commit `c446bef`, pushed to origin). - ✅ Task 11 (#118): Detach-grace session retention - ✅ Task 12 (#119): Replay-on-reconnect + emit ReplayGap -### Pending — Phase 3 finish -- ⏳ Task 13 (#120): Owner re-validation on reconnect — blockedBy 12, 1 -- ⏳ Task 14 (#121): Client ReplayGap handling — all 5 clients — blockedBy 10 - - Carry the per-language presence-check idiom note for `optional` message fields. -- ⏳ Task 15 (#122): Reconnect integration test (fake worker) — blockedBy 12 +### Phase 3 finish — DONE (via archreview P0/P2) +- ✅ Task 13 (#120): Owner re-validation on reconnect — shipped as **TST-02** (P0). +- 🔄 Task 14 (#121): Client ReplayGap handling — shipped as **CLI-15** for 4/5 clients + (.NET/Go/Rust/Python); Java pending (windev batch). Per-language presence-check idiom + for `optional` message fields carried in each client's surface. +- ✅ Task 15 (#122): Reconnect integration test (fake worker) — shipped as **TST-01**. -### Pending — Phase 4 (Per-session dashboard ACL) +### Phase 4 (Per-session dashboard ACL) — SCOPED, tracked as archreview TST-15 - ⏳ Task 16 (#123): gRPC session-owner gate + all-sessions admin scope — blockedBy 9, 1 + - Note: the gRPC owner gate itself already exists (TST-02); Phase 4 adds the admin + all-sessions scope + the dashboard-side twin. - ⏳ Task 17 (#124): Session Tag + dashboard group-to-tag config — blockedBy 9 - ⏳ Task 18 (#125): EventsHub per-session ACL + hub-token tag claim — blockedBy 17 - - Open decision: Viewer default (admin-sees-all vs strict per-session). + - Decision SETTLED: admin-sees-all, Viewer strictly scoped to owned/granted sessions + (matches TST-02 gRPC owner binding). - ⏳ Task 19 (#126): ACL tests incl. live LDAP users — blockedBy 18 -### Pending — Phase 5 (Orphan-worker reattach) -- ⏳ Task 20 (#127): Stable gateway-instance id + stable pipe naming — blockedBy 19 -- ⏳ Task 21 (#128): Adoption manifest store (SQLite) — blockedBy 20 -- ⏳ Task 22 (#129): Proto — worker adopt/reconnect frame — blockedBy 21 -- ⏳ Task 23 (#130): Worker phone-home reconnect loop + self-terminate — blockedBy 22 (net48/x86, windev) -- ⏳ Task 24 (#131): Gateway adoption — re-open pipes, nonce-validate, reject impostors — blockedBy 23 -- ⏳ Task 25 (#132): Resync adopted worker + ReplayGap to subscribers — blockedBy 24, 12 -- ⏳ Task 26 (#133): EnableOrphanReattach flag (default off) + terminator fallback — blockedBy 24 -- ⏳ Task 27 (#134): Gateway-restart reattach round-trip (WINDEV + live worker) — blockedBy 25, 26 -- ⏳ Task 28 (#135): Documented-rule reversals + stillpending refresh — blockedBy 27 +### Phase 5 (Orphan-worker reattach) — DEFERRED, NOT PLANNED +Deferred unless a concrete requirement appears. It reverses the CLAUDE.md invariant +"Gateway restart does not reattach orphan workers" and adds an adoption manifest store + +worker phone-home protocol. **`EnableOrphanReattach` (Task 26) does not exist** — do not +reference it as if it does until the task lands. +- 🚫 Task 20 (#127): Stable gateway-instance id + stable pipe naming +- 🚫 Task 21 (#128): Adoption manifest store (SQLite) +- 🚫 Task 22 (#129): Proto — worker adopt/reconnect frame +- 🚫 Task 23 (#130): Worker phone-home reconnect loop + self-terminate (net48/x86, windev) +- 🚫 Task 24 (#131): Gateway adoption — re-open pipes, nonce-validate, reject impostors +- 🚫 Task 25 (#132): Resync adopted worker + ReplayGap to subscribers +- 🚫 Task 26 (#133): EnableOrphanReattach flag (default off) + terminator fallback +- 🚫 Task 27 (#134): Gateway-restart reattach round-trip (WINDEV + live worker) +- 🚫 Task 28 (#135): Documented-rule reversals + stillpending refresh ## Notes -- Phase 5 reverses the documented "Gateway restart does not reattach orphan workers" - rule (CLAUDE.md) — this was explicitly approved during design. +- Phase 5 was designed to reverse the "Gateway restart does not reattach orphan workers" + rule (CLAUDE.md), but is now **deferred, not planned** (TST-04) — the invariant stands. - Two deferred follow-ups noted earlier: dashboard visibility of `DetachedAtUtc` on `DashboardSessionSummary`. - Worker (net48/x86) tasks build/test on windev; everything else builds on macOS. -- 2.52.0 From 4090a478c85c8b3053325202bced6ec81bd2c3aa Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Thu, 9 Jul 2026 16:33:51 -0400 Subject: [PATCH 20/25] docs(archreview): TST-04 -> Done (epic governance decision landed) Claude-Session: https://claude.ai/code/session_01DMXXvNuPekkkrTEyPNxEkW --- archreview/remediation/00-tracking.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/archreview/remediation/00-tracking.md b/archreview/remediation/00-tracking.md index 34bddc8..c53b954 100644 --- a/archreview/remediation/00-tracking.md +++ b/archreview/remediation/00-tracking.md @@ -216,7 +216,7 @@ Full design + implementation for each row lives in the linked domain doc under i | TST-01 | High | P2 | L | TST-04 | In progress | Reconnect/replay has no e2e test and no client consumer | | TST-02 | High | P0 | M | TST-04 | Done | Reconnect owner re-validation not implemented | | TST-03 | High | P1 | M | — | In review | No CI exists | -| TST-04 | High | P2 | L | — | Not started | Session-resilience epic 16/28 tasks unfinished | +| TST-04 | High | P2 | L | — | Done | Session-resilience epic 16/28 tasks unfinished | | TST-05 | Medium | P1 | S | TST-03 | Not started | Real-worker control/COM paths verified opt-in only | | TST-06 | Medium | — | M | — | Not started | Dashboard live-data path untested | | TST-07 | Medium | — | S | — | Not started | Real-clock sleeps with negative assertions are latent flakes | @@ -253,6 +253,7 @@ Findings the review flagged as one coordinated design pass — sequence them tog | Date | Change | |---|---| +| 2026-07-09 | P2 Epic — TST-04 (session-resilience epic governance) → `Done` (commit `ed3c6c6`, docs-only umbrella). Resolved the epic's shipped-vs-planned entanglement into three decisions: **Phase 3 finished** (Task 13=TST-02 owner-scoped attach P0, Task 15=TST-01 reconnect e2e, Task 14=CLI-15 4/5 clients w/ Java pending); **Phase 4 scoped** as TST-15 with the Viewer-default decision settled (admin-sees-all, Viewer strict per owned/granted session, matching TST-02's gRPC owner binding); **Phase 5 (orphan-worker reattach) DEFERRED, not planned** — the CLAUDE.md "gateway restart does not reattach orphan workers" invariant stands and the `EnableOrphanReattach` flag does not exist / must not be referenced. Updated `oldtasks.md`, the `tasks.json` mirror (per-task statuses + governance note), and the CLAUDE.md reconnect paragraph (clients consume ReplayGap; reattach deferred). The actionable slices carry their own verification: TST-01 (done), TST-02 (done, P0), TST-15 (pending, E3). | | 2026-07-09 | P2 Epic Wave E1 — reconnect-replay: CLI-15 + TST-01 → `In progress` (both gated on the Java client, deferred to the windev batch). **CLI-15 (commit `0c6e5b3`)**: four clients now surface the gateway's `ReplayGap` reconnect sentinel as a distinct, typed, non-terminal signal (never synthesized, never swallowed) — .NET `MxEventStreamItem`/`StreamEventItemsAsync` (build clean, 87 passed), Go `EventResult.ReplayGap`+`IsReplayGap()` (build/vet/test clean), Rust `EventItem::ReplayGap` enum (`EventStream` now yields `Result`; fmt/check/test/clippy clean), Python `ReplayGap` dataclass yielded from `stream_events` (131 passed). Shared docs: `ClientLibrariesDesign.md` non-goals reframed (reconnect-replay protocol is consumable; auto-reconnect stays a non-goal) + `CrossLanguageSmokeMatrix.md` resume-gap note. Resume contract uniform: `after_worker_sequence = oldest_available_sequence - 1`. **TST-01 server half (commit `fed0685`)**: `GatewayEndToEndReconnectReplayTests` (2 facts) proves the default-on reconnect protocol end-to-end over the real gRPC StreamEvents path via the fake worker — replay-tail-no-gap (capacity 16, mid-batch cursor) and stale-cursor-emits-ReplayGap-first (capacity 3, 6 events force eviction, cursor=1); single-subscriber detach→reconnect, ring survives detach. macOS-verified 2/2 (needs `TMPDIR=/tmp` — default macOS TMPDIR pushes the CoreFxPipe Unix-socket path past the 104-byte `sun_path` limit; Windows CI unaffected). No product bugs; server behavior matched the contract exactly. Remaining to close both: Java `ReplayGap` surface (windev) + Java client consumer for TST-01. | | 2026-07-09 | P2 Wave B — worker event hot-path (commit `f61c816`), **windev-verified**. WRK-06 → `Done`: `MxStatusProxyConverter` caches the four resolved `FieldInfo` per status type in a static `ConcurrentDictionary` (the `GetField` metadata scan ran 4× per status per event on the STA path); `GetValue`+`Convert.ToInt32` still run per event (late-bound RCW); both exception messages byte-identical (missing-field via `ResolveField`, not cached on throw through `GetOrAdd`; null-value unchanged). WRK-11 → `Done`: `MxAccessEventQueue.Enqueue` takes ownership of the passed `MxEvent` (stamps `WorkerSequence`/`WorkerTimestamp` in place, no `Clone()`); audited all 3 callers (base/alarm event sinks + provider-mode handler) — each builds a fresh event, none reuse it; `MxAccessValueCache.Set` now deep-copies its retained `Value`/`SourceTimestamp`/`Statuses` so the cache snapshot never aliases the queue-owned (later serialized) event. WRK-12 → `Done`: `WorkerFrameWriter` coalesces the flush across a drained batch (write each frame, one `FlushAsync` after the batch, then complete all) — preserves the written+flushed completion contract; a burst of N events costs 1 flush not N; on failure the in-flight batch + queue all fail so no caller hangs. IPC-15 → `Done`: the multi-event `WorkerEnvelope` body stays unimplemented (wire still one event per `worker_event` frame); gateway.md Performance section now distinguishes the shipped flush-coalescing from that deferred additive-proto change. **Windev:** x86 Worker builds clean (fixed a Windows-only `TreatWarningsAsErrors` CS8604 in the value-cache null-guard that macOS can't surface); Worker.Tests **356 passed / 0 failed / 11 skipped** (+4 new: converter cache-reuse, queue ownership-transfer, value-cache snapshot independence, writer batch-flush-once); gateway Tests **815 passed / 1 failed** — the 1 is the known windev-environmental SelfSigned-SAN assertion (passes on macOS), and all pipe-harness tests (which also exercise the earlier G-frame gateway frame change + WRK-12 end-to-end over a real pipe) pass. | | 2026-07-09 | P2 Wave B — quick Mac items (commit `ec6f82b`). TST-11 → `Done`: `src/Directory.Build.props` single-sources the .NET-side version (Server/Worker/Contracts/tests stamped 0.1.2, was SDK-default 1.0.0) and appends the git short SHA to InformationalVersion (`0.1.2+`, guarded so a git-less build still works); verified Server assembly stamps `0.1.2+579282f`, Server+Tests build clean. Kept 0.1.2 (matches Contracts + aligned Python/Rust/Go); converging all to a single 0.2.0 cadence left as a release decision. WRK-15 → `Done` (docs-only path, no x86 build): STA thread-name refs corrected to the actual `MxGateway.Worker.STA` in `docs/WorkerSta.md` + `docs/MxAccessWorkerInstanceDesign.md`, and the stale heartbeat-counter note fixed (CaptureHeartbeat populates queue depth + sequence from the live queue). TST-23 → `Done`: already resolved by the Wave A gateway.md edit — the bidi `Session` RPC now sits under a "Future work: not implemented" heading (no "best long-term shape" framing), consistent with the proto. **TST-14 left open by design**: its only concrete step is deleting the user's untracked, gitignored `*-docs-*.md` working files (zero repo impact) — surfaced to the user rather than deleting files not created here. | -- 2.52.0 From bde042b4d4833e72d258d0a4d65dab8b5f1e5ec0 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Thu, 9 Jul 2026 16:41:43 -0400 Subject: [PATCH 21/25] =?UTF-8?q?feat(clients):=20CLI-04=20typed=20single-?= =?UTF-8?q?item=20command=20parity=20(+CLI-30=20unregister)=20=E2=80=94=20?= =?UTF-8?q?4/5=20clients?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every parity-critical single-item MXAccess command now has a typed session helper instead of only a raw-Invoke escape hatch. Added per client: - Phase 1: AdviseSupervisory, WriteSecured, WriteSecured2, AuthenticateUser, ArchestrAUserToId - Phase 2: AddBufferedItem, SetBufferedUpdateInterval, Suspend, Activate - CLI-30: Unregister (Rust + .NET; Go/Python already had it) Each wraps the existing raw-command machinery (no new wire surface) and runs the same MXAccess-level reply validation (hresult < 0 + MxStatusProxy). MXAccess parity preserved: WriteSecured before AuthenticateUser+AdviseSupervisory surfaces the native failure unchanged (not pre-validated/reordered). Credentials (AuthenticateUser password, WriteSecured payloads) route through each client's secret-redaction seam and never reach logs/exceptions/ToString/Debug/Display; each suite asserts a distinctive credential is absent from surfaced errors. New CLI subcommands source credentials via flag/env, never echoed. - .NET: 21 helpers (validated + Raw), CLI subcommands, multi-secret CLI redactor. Build clean (0 warn), 102 passed. - Go: 9 helpers + *Raw variants, redactSecrets seam, promoted CLI advise-supervisory to typed. gofmt/vet/build/test clean. - Rust: 10 helpers incl. unregister; verified ensure_mxaccess_success runs on secured paths; error.rs credential scrub. fmt/check/test/clippy clean. - Python: 9 async helpers, redact_secret seam + _invoke_redacted, CLI commands. 145 passed. - Shared doc: ClientLibrariesDesign "Typed Command Parity" section. Java client typed parity is batched to windev (no local JRE); CLI-04 + CLI-30 stay open until it lands. Claude-Session: https://claude.ai/code/session_01DMXXvNuPekkkrTEyPNxEkW --- .../MxGatewayCliSecretRedactor.cs | 25 +- .../MxGatewayClientCli.cs | 297 +++++++++- .../MxGatewayClientCliTests.cs | 114 ++++ .../MxGatewayClientSessionTests.cs | 291 ++++++++++ .../MxGatewaySession.cs | 519 ++++++++++++++++++ clients/go/README.md | 39 +- clients/go/cmd/mxgw-go/main.go | 100 +++- clients/go/cmd/mxgw-go/main_test.go | 30 + clients/go/mxgateway/errors.go | 58 ++ clients/go/mxgateway/session.go | 271 +++++++++ .../mxgateway/session_parity_helpers_test.go | 224 ++++++++ clients/python/README.md | 38 +- .../python/src/zb_mom_ww_mxgateway/session.py | 281 +++++++++- .../src/zb_mom_ww_mxgateway_cli/commands.py | 105 +++- clients/python/tests/test_cli.py | 172 ++++++ .../tests/test_typed_command_helpers.py | 227 ++++++++ clients/rust/README.md | 34 +- clients/rust/crates/mxgw-cli/src/main.rs | 192 ++++++- clients/rust/src/session.rs | 366 +++++++++++- clients/rust/tests/client_behavior.rs | 187 ++++++- docs/ClientLibrariesDesign.md | 23 + 21 files changed, 3496 insertions(+), 97 deletions(-) create mode 100644 clients/go/mxgateway/session_parity_helpers_test.go create mode 100644 clients/python/tests/test_typed_command_helpers.py diff --git a/clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli/MxGatewayCliSecretRedactor.cs b/clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli/MxGatewayCliSecretRedactor.cs index d4e3003..7119c85 100644 --- a/clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli/MxGatewayCliSecretRedactor.cs +++ b/clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli/MxGatewayCliSecretRedactor.cs @@ -1,18 +1,31 @@ namespace ZB.MOM.WW.MxGateway.Client.Cli; -/// Utility to redact API keys from error messages for safe output. +/// Utility to redact secrets (API keys, MXAccess credentials) from error messages for safe output. internal static class MxGatewayCliSecretRedactor { - /// Replaces occurrences of the API key in the value with a redacted placeholder. + /// + /// Replaces every occurrence of any supplied secret in the value with a + /// redacted placeholder. Null or empty secrets are ignored, so callers can + /// pass optional credentials without pre-filtering. + /// /// The message text to redact. - /// The API key to remove; no redaction if null or empty. - public static string Redact(string value, string? apiKey) + /// The secret values to remove (API key, verify-user password, secured payloads). + public static string Redact(string value, params string?[] secrets) { - if (string.IsNullOrEmpty(value) || string.IsNullOrEmpty(apiKey)) + if (string.IsNullOrEmpty(value) || secrets is null) { return value; } - return value.Replace(apiKey, "[redacted]", StringComparison.Ordinal); + string redacted = value; + foreach (string? secret in secrets) + { + if (!string.IsNullOrEmpty(secret)) + { + redacted = redacted.Replace(secret, "[redacted]", StringComparison.Ordinal); + } + } + + return redacted; } } diff --git a/clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli/MxGatewayClientCli.cs b/clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli/MxGatewayClientCli.cs index e8d56a8..bb7250d 100644 --- a/clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli/MxGatewayClientCli.cs +++ b/clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli/MxGatewayClientCli.cs @@ -112,6 +112,24 @@ public static class MxGatewayClientCli .ConfigureAwait(false), "advise-supervisory" => await AdviseSupervisoryAsync(arguments, client, standardOutput, cancellation.Token) .ConfigureAwait(false), + "unregister" => await UnregisterAsync(arguments, client, standardOutput, cancellation.Token) + .ConfigureAwait(false), + "add-buffered-item" => await AddBufferedItemAsync(arguments, client, standardOutput, cancellation.Token) + .ConfigureAwait(false), + "set-buffered-update-interval" => await SetBufferedUpdateIntervalAsync(arguments, client, standardOutput, cancellation.Token) + .ConfigureAwait(false), + "suspend" => await SuspendAsync(arguments, client, standardOutput, cancellation.Token) + .ConfigureAwait(false), + "activate" => await ActivateAsync(arguments, client, standardOutput, cancellation.Token) + .ConfigureAwait(false), + "write-secured" => await WriteSecuredAsync(arguments, client, standardOutput, cancellation.Token) + .ConfigureAwait(false), + "write-secured2" => await WriteSecured2Async(arguments, client, standardOutput, cancellation.Token) + .ConfigureAwait(false), + "authenticate-user" => await AuthenticateUserAsync(arguments, client, standardOutput, cancellation.Token) + .ConfigureAwait(false), + "archestra-user-to-id" => await ArchestraUserToIdAsync(arguments, client, standardOutput, cancellation.Token) + .ConfigureAwait(false), "subscribe-bulk" => await SubscribeBulkAsync(arguments, client, standardOutput, cancellation.Token) .ConfigureAwait(false), "unsubscribe-bulk" => await UnsubscribeBulkAsync(arguments, client, standardOutput, cancellation.Token) @@ -157,9 +175,15 @@ public static class MxGatewayClientCli { // Client.Dotnet-028: redact the *effective* key — from --api-key or the // --api-key-env environment variable — so an env-var-sourced key echoed - // in a transport error never reaches stderr unredacted. + // in a transport error never reaches stderr unredacted. CLI-04: also + // redact MXAccess credentials (AuthenticateUser password, WriteSecured + // payloads) that could otherwise be echoed back in a surfaced error. string? apiKey = TryResolveApiKey(arguments); - string message = MxGatewayCliSecretRedactor.Redact(exception.Message, apiKey); + string message = MxGatewayCliSecretRedactor.Redact( + exception.Message, + apiKey, + TryResolveVerifyUserPassword(arguments), + arguments.GetOptional("value")); if (forceJsonErrors || arguments.HasFlag("json")) { @@ -319,6 +343,48 @@ public static class MxGatewayClientCli return Environment.GetEnvironmentVariable(apiKeyEnvironmentName); } + /// + /// Resolves the effective MXAccess verify-user credential from + /// --verify-user-password or, failing that, the + /// --verify-user-password-env-named environment variable (default + /// MXGATEWAY_VERIFY_USER_PASSWORD). The credential is never echoed; + /// this resolver exists so the error-redaction catch block can strip it + /// from any surfaced error (CLI-04), mirroring . + /// + private static string? TryResolveVerifyUserPassword(CliArguments arguments) + { + string? password = arguments.GetOptional("verify-user-password"); + if (!string.IsNullOrEmpty(password)) + { + return password; + } + + string passwordEnvironmentName = arguments.GetOptional("verify-user-password-env") + ?? "MXGATEWAY_VERIFY_USER_PASSWORD"; + + return Environment.GetEnvironmentVariable(passwordEnvironmentName); + } + + /// + /// Resolves the verify-user credential for authenticate-user, throwing + /// a redaction-safe error when neither the flag nor the env var is set. The + /// thrown message names only the option/env var, never the value. + /// + private static string ResolveVerifyUserPassword(CliArguments arguments) + { + string? password = TryResolveVerifyUserPassword(arguments); + if (!string.IsNullOrEmpty(password)) + { + return password; + } + + string passwordEnvironmentName = arguments.GetOptional("verify-user-password-env") + ?? "MXGATEWAY_VERIFY_USER_PASSWORD"; + + throw new ArgumentException( + $"Verify-user password is required. Pass --verify-user-password or set {passwordEnvironmentName}."); + } + private static CancellationTokenSource CreateCancellation(CliArguments arguments, string command) { var cancellation = new CancellationTokenSource(); @@ -475,6 +541,215 @@ public static class MxGatewayClientCli cancellationToken); } + private static Task UnregisterAsync( + CliArguments arguments, + IMxGatewayCliClient client, + TextWriter output, + CancellationToken cancellationToken) + { + return InvokeAndWriteAsync( + arguments, + client, + output, + new MxCommand + { + Kind = MxCommandKind.Unregister, + Unregister = new UnregisterCommand + { + ServerHandle = arguments.GetInt32("server-handle"), + }, + }, + cancellationToken); + } + + private static Task AddBufferedItemAsync( + CliArguments arguments, + IMxGatewayCliClient client, + TextWriter output, + CancellationToken cancellationToken) + { + return InvokeAndWriteAsync( + arguments, + client, + output, + new MxCommand + { + Kind = MxCommandKind.AddBufferedItem, + AddBufferedItem = new AddBufferedItemCommand + { + ServerHandle = arguments.GetInt32("server-handle"), + ItemDefinition = arguments.GetRequired("item"), + ItemContext = arguments.GetOptional("item-context") ?? string.Empty, + }, + }, + cancellationToken); + } + + private static Task SetBufferedUpdateIntervalAsync( + CliArguments arguments, + IMxGatewayCliClient client, + TextWriter output, + CancellationToken cancellationToken) + { + return InvokeAndWriteAsync( + arguments, + client, + output, + new MxCommand + { + Kind = MxCommandKind.SetBufferedUpdateInterval, + SetBufferedUpdateInterval = new SetBufferedUpdateIntervalCommand + { + ServerHandle = arguments.GetInt32("server-handle"), + UpdateIntervalMilliseconds = arguments.GetInt32("interval-ms"), + }, + }, + cancellationToken); + } + + private static Task SuspendAsync( + CliArguments arguments, + IMxGatewayCliClient client, + TextWriter output, + CancellationToken cancellationToken) + { + return InvokeAndWriteAsync( + arguments, + client, + output, + new MxCommand + { + Kind = MxCommandKind.Suspend, + Suspend = new SuspendCommand + { + ServerHandle = arguments.GetInt32("server-handle"), + ItemHandle = arguments.GetInt32("item-handle"), + }, + }, + cancellationToken); + } + + private static Task ActivateAsync( + CliArguments arguments, + IMxGatewayCliClient client, + TextWriter output, + CancellationToken cancellationToken) + { + return InvokeAndWriteAsync( + arguments, + client, + output, + new MxCommand + { + Kind = MxCommandKind.Activate, + Activate = new ActivateCommand + { + ServerHandle = arguments.GetInt32("server-handle"), + ItemHandle = arguments.GetInt32("item-handle"), + }, + }, + cancellationToken); + } + + private static Task WriteSecuredAsync( + CliArguments arguments, + IMxGatewayCliClient client, + TextWriter output, + CancellationToken cancellationToken) + { + return InvokeAndWriteAsync( + arguments, + client, + output, + new MxCommand + { + Kind = MxCommandKind.WriteSecured, + WriteSecured = new WriteSecuredCommand + { + ServerHandle = arguments.GetInt32("server-handle"), + ItemHandle = arguments.GetInt32("item-handle"), + CurrentUserId = arguments.GetInt32("current-user-id"), + VerifierUserId = arguments.GetInt32("verifier-user-id", 0), + Value = ParseValue(arguments), + }, + }, + cancellationToken); + } + + private static Task WriteSecured2Async( + CliArguments arguments, + IMxGatewayCliClient client, + TextWriter output, + CancellationToken cancellationToken) + { + return InvokeAndWriteAsync( + arguments, + client, + output, + new MxCommand + { + Kind = MxCommandKind.WriteSecured2, + WriteSecured2 = new WriteSecured2Command + { + ServerHandle = arguments.GetInt32("server-handle"), + ItemHandle = arguments.GetInt32("item-handle"), + CurrentUserId = arguments.GetInt32("current-user-id"), + VerifierUserId = arguments.GetInt32("verifier-user-id", 0), + Value = ParseValue(arguments), + TimestampValue = ParseTimestampValue(arguments), + }, + }, + cancellationToken); + } + + private static Task AuthenticateUserAsync( + CliArguments arguments, + IMxGatewayCliClient client, + TextWriter output, + CancellationToken cancellationToken) + { + // The credential is resolved from --verify-user-password or its env var and + // is never echoed. On any surfaced error the RunCoreAsync catch block routes + // it through MxGatewayCliSecretRedactor so it cannot reach stderr (CLI-04). + return InvokeAndWriteAsync( + arguments, + client, + output, + new MxCommand + { + Kind = MxCommandKind.AuthenticateUser, + AuthenticateUser = new AuthenticateUserCommand + { + ServerHandle = arguments.GetInt32("server-handle"), + VerifyUser = arguments.GetRequired("verify-user"), + VerifyUserPassword = ResolveVerifyUserPassword(arguments), + }, + }, + cancellationToken); + } + + private static Task ArchestraUserToIdAsync( + CliArguments arguments, + IMxGatewayCliClient client, + TextWriter output, + CancellationToken cancellationToken) + { + return InvokeAndWriteAsync( + arguments, + client, + output, + new MxCommand + { + Kind = MxCommandKind.ArchestraUserToId, + ArchestraUserToId = new ArchestrAUserToIdCommand + { + ServerHandle = arguments.GetInt32("server-handle"), + UserIdGuid = arguments.GetRequired("user-guid"), + }, + }, + cancellationToken); + } + private static Task SubscribeBulkAsync( CliArguments arguments, IMxGatewayCliClient client, @@ -2029,6 +2304,15 @@ public static class MxGatewayClientCli or "add-item" or "advise" or "advise-supervisory" + or "unregister" + or "add-buffered-item" + or "set-buffered-update-interval" + or "suspend" + or "activate" + or "write-secured" + or "write-secured2" + or "authenticate-user" + or "archestra-user-to-id" or "subscribe-bulk" or "unsubscribe-bulk" or "read-bulk" @@ -2092,6 +2376,15 @@ public static class MxGatewayClientCli writer.WriteLine("mxgw-dotnet add-item --session-id --server-handle --item [--json]"); writer.WriteLine("mxgw-dotnet advise --session-id --server-handle --item-handle [--json]"); writer.WriteLine("mxgw-dotnet advise-supervisory --session-id --server-handle --item-handle [--json]"); + writer.WriteLine("mxgw-dotnet unregister --session-id --server-handle [--json]"); + writer.WriteLine("mxgw-dotnet add-buffered-item --session-id --server-handle --item [--item-context ] [--json]"); + writer.WriteLine("mxgw-dotnet set-buffered-update-interval --session-id --server-handle --interval-ms [--json]"); + writer.WriteLine("mxgw-dotnet suspend --session-id --server-handle --item-handle [--json]"); + writer.WriteLine("mxgw-dotnet activate --session-id --server-handle --item-handle [--json]"); + writer.WriteLine("mxgw-dotnet write-secured --session-id --server-handle --item-handle --type --value --current-user-id [--verifier-user-id ] [--json]"); + writer.WriteLine("mxgw-dotnet write-secured2 --session-id --server-handle --item-handle --type --value --current-user-id [--verifier-user-id ] [--timestamp ] [--json]"); + writer.WriteLine("mxgw-dotnet authenticate-user --session-id --server-handle --verify-user (--verify-user-password | --verify-user-password-env ) [--json]"); + writer.WriteLine("mxgw-dotnet archestra-user-to-id --session-id --server-handle --user-guid [--json]"); writer.WriteLine("mxgw-dotnet subscribe-bulk --session-id --server-handle --items [--json]"); writer.WriteLine("mxgw-dotnet unsubscribe-bulk --session-id --server-handle --item-handles [--json]"); writer.WriteLine("mxgw-dotnet read-bulk --session-id --server-handle --items [--timeout-ms ] [--json]"); diff --git a/clients/dotnet/ZB.MOM.WW.MxGateway.Client.Tests/MxGatewayClientCliTests.cs b/clients/dotnet/ZB.MOM.WW.MxGateway.Client.Tests/MxGatewayClientCliTests.cs index bd39120..54642a7 100644 --- a/clients/dotnet/ZB.MOM.WW.MxGateway.Client.Tests/MxGatewayClientCliTests.cs +++ b/clients/dotnet/ZB.MOM.WW.MxGateway.Client.Tests/MxGatewayClientCliTests.cs @@ -129,6 +129,120 @@ public sealed class MxGatewayClientCliTests Assert.Equal(string.Empty, error.ToString()); } + /// Verifies that write-secured builds a WriteSecured command with the value and user ids. + [Fact] + public async Task RunAsync_WriteSecured_BuildsWriteSecuredCommand() + { + using var output = new StringWriter(); + using var error = new StringWriter(); + FakeCliClient fakeClient = new(); + fakeClient.InvokeReplies.Enqueue(new MxCommandReply + { + SessionId = "session-fixture", + Kind = MxCommandKind.WriteSecured, + ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok }, + }); + + int exitCode = await MxGatewayClientCli.RunAsync( + [ + "write-secured", + "--endpoint", "http://localhost:5000", + "--api-key", "test-api-key", + "--session-id", "session-fixture", + "--server-handle", "12", + "--item-handle", "34", + "--type", "int32", + "--value", "123", + "--current-user-id", "5", + "--verifier-user-id", "6", + "--json", + ], + output, + error, + _ => fakeClient); + + Assert.Equal(0, exitCode); + MxCommandRequest request = Assert.Single(fakeClient.InvokeRequests); + Assert.Equal(MxCommandKind.WriteSecured, request.Command.Kind); + Assert.Equal(123, request.Command.WriteSecured.Value.Int32Value); + Assert.Equal(5, request.Command.WriteSecured.CurrentUserId); + Assert.Equal(6, request.Command.WriteSecured.VerifierUserId); + Assert.Equal(string.Empty, error.ToString()); + } + + /// + /// Verifies that authenticate-user builds an AuthenticateUser command sourcing the + /// credential from the flag, and that the credential never appears in stdout/stderr. + /// + [Fact] + public async Task RunAsync_AuthenticateUser_BuildsCommandAndDoesNotEchoCredential() + { + const string password = "cli-secret-credential-987"; + using var output = new StringWriter(); + using var error = new StringWriter(); + FakeCliClient fakeClient = new(); + fakeClient.InvokeReplies.Enqueue(new MxCommandReply + { + SessionId = "session-fixture", + Kind = MxCommandKind.AuthenticateUser, + ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok }, + AuthenticateUser = new AuthenticateUserReply { UserId = 4242 }, + }); + + int exitCode = await MxGatewayClientCli.RunAsync( + [ + "authenticate-user", + "--endpoint", "http://localhost:5000", + "--api-key", "test-api-key", + "--session-id", "session-fixture", + "--server-handle", "12", + "--verify-user", "operator", + "--verify-user-password", password, + "--json", + ], + output, + error, + _ => fakeClient); + + Assert.Equal(0, exitCode); + MxCommandRequest request = Assert.Single(fakeClient.InvokeRequests); + Assert.Equal(MxCommandKind.AuthenticateUser, request.Command.Kind); + Assert.Equal("operator", request.Command.AuthenticateUser.VerifyUser); + Assert.Equal(password, request.Command.AuthenticateUser.VerifyUserPassword); + Assert.DoesNotContain(password, output.ToString()); + Assert.DoesNotContain(password, error.ToString()); + } + + /// + /// CLI-04: a surfaced error for authenticate-user must have the credential + /// redacted (never echoed to stderr), mirroring the API-key redaction seam. + /// + [Fact] + public async Task RunAsync_AuthenticateUser_ErrorOutput_RedactsCredential() + { + const string password = "leaky-credential-value"; + using var output = new StringWriter(); + using var error = new StringWriter(); + + int exitCode = await MxGatewayClientCli.RunAsync( + [ + "authenticate-user", + "--endpoint", "http://localhost:5000", + "--api-key", "test-api-key", + "--session-id", "session-fixture", + "--server-handle", "12", + "--verify-user", "operator", + "--verify-user-password", password, + ], + output, + error, + _ => throw new InvalidOperationException($"boom {password}")); + + Assert.Equal(1, exitCode); + Assert.DoesNotContain(password, error.ToString()); + Assert.Contains("[redacted]", error.ToString()); + } + /// Verifies that error output redacts sensitive API key values. [Fact] public async Task RunAsync_ErrorOutput_RedactsApiKey() diff --git a/clients/dotnet/ZB.MOM.WW.MxGateway.Client.Tests/MxGatewayClientSessionTests.cs b/clients/dotnet/ZB.MOM.WW.MxGateway.Client.Tests/MxGatewayClientSessionTests.cs index 1de4290..0b83237 100644 --- a/clients/dotnet/ZB.MOM.WW.MxGateway.Client.Tests/MxGatewayClientSessionTests.cs +++ b/clients/dotnet/ZB.MOM.WW.MxGateway.Client.Tests/MxGatewayClientSessionTests.cs @@ -415,6 +415,297 @@ public sealed class MxGatewayClientSessionTests Assert.Equal(7, el.Value.Int32Value); } + /// Verifies that unregister builds an unregister command with the server handle. + [Fact] + public async Task UnregisterAsync_BuildsUnregisterCommand() + { + FakeGatewayTransport transport = CreateTransport(); + transport.AddInvokeReply(new MxCommandReply + { + SessionId = "session-fixture", + Kind = MxCommandKind.Unregister, + ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok }, + }); + await using MxGatewayClient client = CreateClient(transport); + MxGatewaySession session = await client.OpenSessionAsync(); + + await session.UnregisterAsync(12); + + MxCommandRequest request = Assert.Single(transport.InvokeCalls).Request; + Assert.Equal(MxCommandKind.Unregister, request.Command.Kind); + Assert.Equal(12, request.Command.Unregister.ServerHandle); + } + + /// Verifies that advise-supervisory builds the supervisory advise command. + [Fact] + public async Task AdviseSupervisoryAsync_BuildsAdviseSupervisoryCommand() + { + FakeGatewayTransport transport = CreateTransport(); + transport.AddInvokeReply(new MxCommandReply + { + SessionId = "session-fixture", + Kind = MxCommandKind.AdviseSupervisory, + ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok }, + }); + await using MxGatewayClient client = CreateClient(transport); + MxGatewaySession session = await client.OpenSessionAsync(); + + await session.AdviseSupervisoryAsync(12, 34); + + MxCommandRequest request = Assert.Single(transport.InvokeCalls).Request; + Assert.Equal(MxCommandKind.AdviseSupervisory, request.Command.Kind); + Assert.Equal(12, request.Command.AdviseSupervisory.ServerHandle); + Assert.Equal(34, request.Command.AdviseSupervisory.ItemHandle); + } + + /// Verifies that add-buffered-item returns the item handle from the typed reply. + [Fact] + public async Task AddBufferedItemAsync_BuildsCommandAndReturnsItemHandle() + { + FakeGatewayTransport transport = CreateTransport(); + transport.AddInvokeReply(new MxCommandReply + { + SessionId = "session-fixture", + Kind = MxCommandKind.AddBufferedItem, + ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok }, + AddBufferedItem = new AddBufferedItemReply { ItemHandle = 77 }, + }); + await using MxGatewayClient client = CreateClient(transport); + MxGatewaySession session = await client.OpenSessionAsync(); + + int itemHandle = await session.AddBufferedItemAsync(12, "Area001.Pump001.Speed", "runtime"); + + Assert.Equal(77, itemHandle); + MxCommandRequest request = Assert.Single(transport.InvokeCalls).Request; + Assert.Equal(MxCommandKind.AddBufferedItem, request.Command.Kind); + Assert.Equal(12, request.Command.AddBufferedItem.ServerHandle); + Assert.Equal("Area001.Pump001.Speed", request.Command.AddBufferedItem.ItemDefinition); + Assert.Equal("runtime", request.Command.AddBufferedItem.ItemContext); + } + + /// Verifies that set-buffered-update-interval builds the command with the interval. + [Fact] + public async Task SetBufferedUpdateIntervalAsync_BuildsCommandWithInterval() + { + FakeGatewayTransport transport = CreateTransport(); + transport.AddInvokeReply(new MxCommandReply + { + SessionId = "session-fixture", + Kind = MxCommandKind.SetBufferedUpdateInterval, + ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok }, + }); + await using MxGatewayClient client = CreateClient(transport); + MxGatewaySession session = await client.OpenSessionAsync(); + + await session.SetBufferedUpdateIntervalAsync(12, 500); + + MxCommandRequest request = Assert.Single(transport.InvokeCalls).Request; + Assert.Equal(MxCommandKind.SetBufferedUpdateInterval, request.Command.Kind); + Assert.Equal(12, request.Command.SetBufferedUpdateInterval.ServerHandle); + Assert.Equal(500, request.Command.SetBufferedUpdateInterval.UpdateIntervalMilliseconds); + } + + /// Verifies that suspend builds the command and returns the reply status. + [Fact] + public async Task SuspendAsync_BuildsCommandAndReturnsStatus() + { + FakeGatewayTransport transport = CreateTransport(); + transport.AddInvokeReply(new MxCommandReply + { + SessionId = "session-fixture", + Kind = MxCommandKind.Suspend, + ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok }, + Suspend = new SuspendReply { Status = new MxStatusProxy { Success = 1, Category = MxStatusCategory.Ok } }, + }); + await using MxGatewayClient client = CreateClient(transport); + MxGatewaySession session = await client.OpenSessionAsync(); + + MxStatusProxy? status = await session.SuspendAsync(12, 34); + + Assert.NotNull(status); + Assert.Equal(1, status!.Success); + MxCommandRequest request = Assert.Single(transport.InvokeCalls).Request; + Assert.Equal(MxCommandKind.Suspend, request.Command.Kind); + Assert.Equal(12, request.Command.Suspend.ServerHandle); + Assert.Equal(34, request.Command.Suspend.ItemHandle); + } + + /// Verifies that activate builds the command and returns the reply status. + [Fact] + public async Task ActivateAsync_BuildsCommandAndReturnsStatus() + { + FakeGatewayTransport transport = CreateTransport(); + transport.AddInvokeReply(new MxCommandReply + { + SessionId = "session-fixture", + Kind = MxCommandKind.Activate, + ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok }, + Activate = new ActivateReply { Status = new MxStatusProxy { Success = 1, Category = MxStatusCategory.Ok } }, + }); + await using MxGatewayClient client = CreateClient(transport); + MxGatewaySession session = await client.OpenSessionAsync(); + + MxStatusProxy? status = await session.ActivateAsync(12, 34); + + Assert.NotNull(status); + MxCommandRequest request = Assert.Single(transport.InvokeCalls).Request; + Assert.Equal(MxCommandKind.Activate, request.Command.Kind); + Assert.Equal(34, request.Command.Activate.ItemHandle); + } + + /// Verifies that write-secured builds a WriteSecured command with value and user ids. + [Fact] + public async Task WriteSecuredAsync_BuildsWriteSecuredCommand() + { + FakeGatewayTransport transport = CreateTransport(); + transport.AddInvokeReply(new MxCommandReply + { + SessionId = "session-fixture", + Kind = MxCommandKind.WriteSecured, + ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok }, + }); + await using MxGatewayClient client = CreateClient(transport); + MxGatewaySession session = await client.OpenSessionAsync(); + MxValue value = 123.ToMxValue(); + + await session.WriteSecuredAsync(12, 34, value, currentUserId: 5, verifierUserId: 6); + + MxCommandRequest request = Assert.Single(transport.InvokeCalls).Request; + Assert.Equal(MxCommandKind.WriteSecured, request.Command.Kind); + Assert.Equal(12, request.Command.WriteSecured.ServerHandle); + Assert.Equal(34, request.Command.WriteSecured.ItemHandle); + Assert.Same(value, request.Command.WriteSecured.Value); + Assert.Equal(5, request.Command.WriteSecured.CurrentUserId); + Assert.Equal(6, request.Command.WriteSecured.VerifierUserId); + } + + /// + /// MXAccess parity: WriteSecured issued before a prior AuthenticateUser fails + /// natively. The client must surface that scripted failure (as an + /// ) rather than pre-validating it away. + /// + [Fact] + public async Task WriteSecuredAsync_SurfacesNativeFailureWhenNotAuthenticated() + { + FakeGatewayTransport transport = CreateTransport(); + transport.AddInvokeReply(new MxCommandReply + { + SessionId = "session-fixture", + Kind = MxCommandKind.WriteSecured, + ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.MxaccessFailure }, + Hresult = unchecked((int)0x80040200), + }); + await using MxGatewayClient client = CreateClient(transport); + MxGatewaySession session = await client.OpenSessionAsync(); + + MxAccessException exception = await Assert.ThrowsAsync( + async () => await session.WriteSecuredAsync(12, 34, 123.ToMxValue(), currentUserId: 0, verifierUserId: 0)); + + // The native HRESULT is surfaced; the request payload is never in the message. + Assert.Contains("WriteSecured", exception.Message); + Assert.Single(transport.InvokeCalls); + } + + /// Verifies that write-secured2 builds a WriteSecured2 command with value and timestamp. + [Fact] + public async Task WriteSecured2Async_BuildsWriteSecured2Command() + { + FakeGatewayTransport transport = CreateTransport(); + transport.AddInvokeReply(new MxCommandReply + { + SessionId = "session-fixture", + Kind = MxCommandKind.WriteSecured2, + ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok }, + }); + await using MxGatewayClient client = CreateClient(transport); + MxGatewaySession session = await client.OpenSessionAsync(); + MxValue value = 123.ToMxValue(); + MxValue timestampValue = DateTimeOffset.Parse("2026-01-01T00:00:00Z").ToMxValue(); + + await session.WriteSecured2Async(12, 34, value, timestampValue, currentUserId: 5, verifierUserId: 6); + + MxCommandRequest request = Assert.Single(transport.InvokeCalls).Request; + Assert.Equal(MxCommandKind.WriteSecured2, request.Command.Kind); + Assert.Same(value, request.Command.WriteSecured2.Value); + Assert.Same(timestampValue, request.Command.WriteSecured2.TimestampValue); + Assert.Equal(5, request.Command.WriteSecured2.CurrentUserId); + Assert.Equal(6, request.Command.WriteSecured2.VerifierUserId); + } + + /// Verifies that authenticate-user builds the command and returns the resolved user id. + [Fact] + public async Task AuthenticateUserAsync_BuildsCommandAndReturnsUserId() + { + FakeGatewayTransport transport = CreateTransport(); + transport.AddInvokeReply(new MxCommandReply + { + SessionId = "session-fixture", + Kind = MxCommandKind.AuthenticateUser, + ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok }, + AuthenticateUser = new AuthenticateUserReply { UserId = 4242 }, + }); + await using MxGatewayClient client = CreateClient(transport); + MxGatewaySession session = await client.OpenSessionAsync(); + + int userId = await session.AuthenticateUserAsync(12, "operator", "s3cr3t-p@ss"); + + Assert.Equal(4242, userId); + MxCommandRequest request = Assert.Single(transport.InvokeCalls).Request; + Assert.Equal(MxCommandKind.AuthenticateUser, request.Command.Kind); + Assert.Equal("operator", request.Command.AuthenticateUser.VerifyUser); + Assert.Equal("s3cr3t-p@ss", request.Command.AuthenticateUser.VerifyUserPassword); + } + + /// + /// SECRET REDACTION: when AuthenticateUser fails, the surfaced exception message + /// must never contain the credential — the error path is built only from + /// reply-derived diagnostics, not the request payload. + /// + [Fact] + public async Task AuthenticateUserAsync_FailureDoesNotLeakCredentialInErrorMessage() + { + const string password = "super-secret-credential-123"; + FakeGatewayTransport transport = CreateTransport(); + transport.AddInvokeReply(new MxCommandReply + { + SessionId = "session-fixture", + Kind = MxCommandKind.AuthenticateUser, + ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.MxaccessFailure }, + Hresult = unchecked((int)0x80040210), + }); + await using MxGatewayClient client = CreateClient(transport); + MxGatewaySession session = await client.OpenSessionAsync(); + + MxAccessException exception = await Assert.ThrowsAsync( + async () => await session.AuthenticateUserAsync(12, "operator", password)); + + Assert.DoesNotContain(password, exception.Message); + Assert.DoesNotContain(password, exception.ToString()); + } + + /// Verifies that archestra-user-to-id builds the command and returns the resolved user id. + [Fact] + public async Task ArchestraUserToIdAsync_BuildsCommandAndReturnsUserId() + { + FakeGatewayTransport transport = CreateTransport(); + transport.AddInvokeReply(new MxCommandReply + { + SessionId = "session-fixture", + Kind = MxCommandKind.ArchestraUserToId, + ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok }, + ArchestraUserToId = new ArchestrAUserToIdReply { UserId = 909 }, + }); + await using MxGatewayClient client = CreateClient(transport); + MxGatewaySession session = await client.OpenSessionAsync(); + + int userId = await session.ArchestraUserToIdAsync(12, "BCC47053-9542-4D65-BDAA-BCDEA6A32A73"); + + Assert.Equal(909, userId); + MxCommandRequest request = Assert.Single(transport.InvokeCalls).Request; + Assert.Equal(MxCommandKind.ArchestraUserToId, request.Command.Kind); + Assert.Equal("BCC47053-9542-4D65-BDAA-BCDEA6A32A73", request.Command.ArchestraUserToId.UserIdGuid); + } + private static MxGatewayClient CreateClient(FakeGatewayTransport transport) { return new MxGatewayClient(transport.Options, transport); diff --git a/clients/dotnet/ZB.MOM.WW.MxGateway.Client/MxGatewaySession.cs b/clients/dotnet/ZB.MOM.WW.MxGateway.Client/MxGatewaySession.cs index 1740a96..a8ca738 100644 --- a/clients/dotnet/ZB.MOM.WW.MxGateway.Client/MxGatewaySession.cs +++ b/clients/dotnet/ZB.MOM.WW.MxGateway.Client/MxGatewaySession.cs @@ -842,6 +842,525 @@ public sealed class MxGatewaySession : IAsyncDisposable cancellationToken); } + /// + /// Unregisters a previously registered client from the MXAccess session + /// (MXAccess Unregister), releasing its ServerHandle. + /// + /// The ServerHandle from register. + /// Cancellation token. + public async Task UnregisterAsync( + int serverHandle, + CancellationToken cancellationToken = default) + { + MxCommandReply reply = await UnregisterRawAsync(serverHandle, cancellationToken) + .ConfigureAwait(false); + reply.EnsureProtocolSuccess().EnsureMxAccessSuccess(); + } + + /// + /// Unregisters a previously registered client without error checking. + /// + /// The ServerHandle from register. + /// Cancellation token. + /// The raw server reply. + public Task UnregisterRawAsync( + int serverHandle, + CancellationToken cancellationToken = default) + { + return InvokeCommandAsync( + new MxCommand + { + Kind = MxCommandKind.Unregister, + Unregister = new UnregisterCommand { ServerHandle = serverHandle }, + }, + cancellationToken); + } + + /// + /// Subscribes to supervisory events for an item (MXAccess AdviseSupervisory). + /// + /// The ServerHandle from register. + /// The ItemHandle from add-item. + /// Cancellation token. + public async Task AdviseSupervisoryAsync( + int serverHandle, + int itemHandle, + CancellationToken cancellationToken = default) + { + MxCommandReply reply = await AdviseSupervisoryRawAsync(serverHandle, itemHandle, cancellationToken) + .ConfigureAwait(false); + reply.EnsureProtocolSuccess().EnsureMxAccessSuccess(); + } + + /// + /// Subscribes to supervisory events for an item without error checking. + /// + /// The ServerHandle from register. + /// The ItemHandle from add-item. + /// Cancellation token. + /// The raw server reply. + public Task AdviseSupervisoryRawAsync( + int serverHandle, + int itemHandle, + CancellationToken cancellationToken = default) + { + return InvokeCommandAsync( + new MxCommand + { + Kind = MxCommandKind.AdviseSupervisory, + AdviseSupervisory = new AdviseSupervisoryCommand + { + ServerHandle = serverHandle, + ItemHandle = itemHandle, + }, + }, + cancellationToken); + } + + /// + /// Adds a buffered item to the MXAccess session (MXAccess AddBufferedItem), + /// returning an ItemHandle. + /// + /// The ServerHandle from register. + /// The item tag address. + /// Additional context for the item. + /// Cancellation token. + /// The item handle assigned to the new buffered item. + public async Task AddBufferedItemAsync( + int serverHandle, + string itemDefinition, + string itemContext, + CancellationToken cancellationToken = default) + { + MxCommandReply reply = await AddBufferedItemRawAsync( + serverHandle, + itemDefinition, + itemContext, + cancellationToken) + .ConfigureAwait(false); + reply.EnsureProtocolSuccess().EnsureMxAccessSuccess(); + return reply.AddBufferedItem?.ItemHandle ?? reply.ReturnValue.Int32Value; + } + + /// + /// Adds a buffered item to the MXAccess session without error checking. + /// + /// The ServerHandle from register. + /// The item tag address. + /// Additional context for the item. + /// Cancellation token. + /// The raw server reply. + public Task AddBufferedItemRawAsync( + int serverHandle, + string itemDefinition, + string itemContext, + CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(itemDefinition); + + return InvokeCommandAsync( + new MxCommand + { + Kind = MxCommandKind.AddBufferedItem, + AddBufferedItem = new AddBufferedItemCommand + { + ServerHandle = serverHandle, + ItemDefinition = itemDefinition, + ItemContext = itemContext ?? string.Empty, + }, + }, + cancellationToken); + } + + /// + /// Sets the buffered-item update interval on the MXAccess session + /// (MXAccess SetBufferedUpdateInterval). + /// + /// The ServerHandle from register. + /// The buffered update interval, in milliseconds. + /// Cancellation token. + public async Task SetBufferedUpdateIntervalAsync( + int serverHandle, + int updateIntervalMilliseconds, + CancellationToken cancellationToken = default) + { + MxCommandReply reply = await SetBufferedUpdateIntervalRawAsync( + serverHandle, + updateIntervalMilliseconds, + cancellationToken) + .ConfigureAwait(false); + reply.EnsureProtocolSuccess().EnsureMxAccessSuccess(); + } + + /// + /// Sets the buffered-item update interval without error checking. + /// + /// The ServerHandle from register. + /// The buffered update interval, in milliseconds. + /// Cancellation token. + /// The raw server reply. + public Task SetBufferedUpdateIntervalRawAsync( + int serverHandle, + int updateIntervalMilliseconds, + CancellationToken cancellationToken = default) + { + return InvokeCommandAsync( + new MxCommand + { + Kind = MxCommandKind.SetBufferedUpdateInterval, + SetBufferedUpdateInterval = new SetBufferedUpdateIntervalCommand + { + ServerHandle = serverHandle, + UpdateIntervalMilliseconds = updateIntervalMilliseconds, + }, + }, + cancellationToken); + } + + /// + /// Suspends updates for an item (MXAccess Suspend). + /// + /// The ServerHandle from register. + /// The ItemHandle from add-item. + /// Cancellation token. + /// The item's MXSTATUS_PROXY as reported by the worker, or null if the reply omitted it. + public async Task SuspendAsync( + int serverHandle, + int itemHandle, + CancellationToken cancellationToken = default) + { + MxCommandReply reply = await SuspendRawAsync(serverHandle, itemHandle, cancellationToken) + .ConfigureAwait(false); + reply.EnsureProtocolSuccess().EnsureMxAccessSuccess(); + return reply.Suspend?.Status; + } + + /// + /// Suspends updates for an item without error checking. + /// + /// The ServerHandle from register. + /// The ItemHandle from add-item. + /// Cancellation token. + /// The raw server reply. + public Task SuspendRawAsync( + int serverHandle, + int itemHandle, + CancellationToken cancellationToken = default) + { + return InvokeCommandAsync( + new MxCommand + { + Kind = MxCommandKind.Suspend, + Suspend = new SuspendCommand + { + ServerHandle = serverHandle, + ItemHandle = itemHandle, + }, + }, + cancellationToken); + } + + /// + /// Resumes updates for a suspended item (MXAccess Activate). + /// + /// The ServerHandle from register. + /// The ItemHandle from add-item. + /// Cancellation token. + /// The item's MXSTATUS_PROXY as reported by the worker, or null if the reply omitted it. + public async Task ActivateAsync( + int serverHandle, + int itemHandle, + CancellationToken cancellationToken = default) + { + MxCommandReply reply = await ActivateRawAsync(serverHandle, itemHandle, cancellationToken) + .ConfigureAwait(false); + reply.EnsureProtocolSuccess().EnsureMxAccessSuccess(); + return reply.Activate?.Status; + } + + /// + /// Resumes updates for a suspended item without error checking. + /// + /// The ServerHandle from register. + /// The ItemHandle from add-item. + /// Cancellation token. + /// The raw server reply. + public Task ActivateRawAsync( + int serverHandle, + int itemHandle, + CancellationToken cancellationToken = default) + { + return InvokeCommandAsync( + new MxCommand + { + Kind = MxCommandKind.Activate, + Activate = new ActivateCommand + { + ServerHandle = serverHandle, + ItemHandle = itemHandle, + }, + }, + cancellationToken); + } + + /// + /// Writes a secured value to an item on the MXAccess server (MXAccess WriteSecured). + /// + /// + /// MXAccess parity: WriteSecured fails when it is issued before a value-bearing + /// NMX body or before a prior AuthenticateUser + AdviseSupervisory. That + /// native failure is surfaced unchanged — the client does not pre-validate or reorder it. + /// The is credential-sensitive and must never reach logs; the + /// client mirrors the single-item WriteSecured redaction contract. + /// + /// The ServerHandle from register. + /// The ItemHandle from add-item. + /// The secured value to write. + /// The current operator user id. + /// The verifier (secondary approver) user id. + /// Cancellation token. + public async Task WriteSecuredAsync( + int serverHandle, + int itemHandle, + MxValue value, + int currentUserId, + int verifierUserId, + CancellationToken cancellationToken = default) + { + MxCommandReply reply = await WriteSecuredRawAsync( + serverHandle, + itemHandle, + value, + currentUserId, + verifierUserId, + cancellationToken) + .ConfigureAwait(false); + reply.EnsureProtocolSuccess().EnsureMxAccessSuccess(); + } + + /// + /// Writes a secured value to an item without error checking. See + /// for the parity and redaction contract. + /// + /// The ServerHandle from register. + /// The ItemHandle from add-item. + /// The secured value to write. + /// The current operator user id. + /// The verifier (secondary approver) user id. + /// Cancellation token. + /// The raw server reply. + public Task WriteSecuredRawAsync( + int serverHandle, + int itemHandle, + MxValue value, + int currentUserId, + int verifierUserId, + CancellationToken cancellationToken = default) + { + ArgumentNullException.ThrowIfNull(value); + + return InvokeCommandAsync( + new MxCommand + { + Kind = MxCommandKind.WriteSecured, + WriteSecured = new WriteSecuredCommand + { + ServerHandle = serverHandle, + ItemHandle = itemHandle, + CurrentUserId = currentUserId, + VerifierUserId = verifierUserId, + Value = value, + }, + }, + cancellationToken); + } + + /// + /// Writes a secured value and timestamp to an item (MXAccess WriteSecured2). + /// + /// + /// Same parity and redaction contract as : the native + /// failure that occurs before a value-bearing NMX body or a prior authenticate is + /// surfaced unchanged, and the credential-sensitive must never + /// reach logs. + /// + /// The ServerHandle from register. + /// The ItemHandle from add-item. + /// The secured value to write. + /// The timestamp to write with the value. + /// The current operator user id. + /// The verifier (secondary approver) user id. + /// Cancellation token. + public async Task WriteSecured2Async( + int serverHandle, + int itemHandle, + MxValue value, + MxValue timestampValue, + int currentUserId, + int verifierUserId, + CancellationToken cancellationToken = default) + { + MxCommandReply reply = await WriteSecured2RawAsync( + serverHandle, + itemHandle, + value, + timestampValue, + currentUserId, + verifierUserId, + cancellationToken) + .ConfigureAwait(false); + reply.EnsureProtocolSuccess().EnsureMxAccessSuccess(); + } + + /// + /// Writes a secured value and timestamp to an item without error checking. See + /// for the parity and redaction contract. + /// + /// The ServerHandle from register. + /// The ItemHandle from add-item. + /// The secured value to write. + /// The timestamp to write with the value. + /// The current operator user id. + /// The verifier (secondary approver) user id. + /// Cancellation token. + /// The raw server reply. + public Task WriteSecured2RawAsync( + int serverHandle, + int itemHandle, + MxValue value, + MxValue timestampValue, + int currentUserId, + int verifierUserId, + CancellationToken cancellationToken = default) + { + ArgumentNullException.ThrowIfNull(value); + ArgumentNullException.ThrowIfNull(timestampValue); + + return InvokeCommandAsync( + new MxCommand + { + Kind = MxCommandKind.WriteSecured2, + WriteSecured2 = new WriteSecured2Command + { + ServerHandle = serverHandle, + ItemHandle = itemHandle, + CurrentUserId = currentUserId, + VerifierUserId = verifierUserId, + Value = value, + TimestampValue = timestampValue, + }, + }, + cancellationToken); + } + + /// + /// Authenticates an MXAccess verify-user (MXAccess AuthenticateUser), returning + /// the resolved user id used by WriteSecured / WriteSecured2. + /// + /// + /// The is a raw MXAccess credential. It is never + /// logged and never placed on the exception path: gateway/MXAccess failures surface only + /// reply-derived diagnostics (kind, HRESULT, MXSTATUS_PROXY), never the request payload. + /// + /// The ServerHandle from register. + /// The user to verify. + /// The verify-user credential. Never logged. + /// Cancellation token. + /// The authenticated user id. + public async Task AuthenticateUserAsync( + int serverHandle, + string verifyUser, + string verifyUserPassword, + CancellationToken cancellationToken = default) + { + MxCommandReply reply = await AuthenticateUserRawAsync( + serverHandle, + verifyUser, + verifyUserPassword, + cancellationToken) + .ConfigureAwait(false); + reply.EnsureProtocolSuccess().EnsureMxAccessSuccess(); + return reply.AuthenticateUser?.UserId ?? reply.ReturnValue.Int32Value; + } + + /// + /// Authenticates an MXAccess verify-user without error checking. See + /// for the credential-handling contract. + /// + /// The ServerHandle from register. + /// The user to verify. + /// The verify-user credential. Never logged. + /// Cancellation token. + /// The raw server reply. + public Task AuthenticateUserRawAsync( + int serverHandle, + string verifyUser, + string verifyUserPassword, + CancellationToken cancellationToken = default) + { + ArgumentNullException.ThrowIfNull(verifyUser); + ArgumentNullException.ThrowIfNull(verifyUserPassword); + + return InvokeCommandAsync( + new MxCommand + { + Kind = MxCommandKind.AuthenticateUser, + AuthenticateUser = new AuthenticateUserCommand + { + ServerHandle = serverHandle, + VerifyUser = verifyUser, + VerifyUserPassword = verifyUserPassword, + }, + }, + cancellationToken); + } + + /// + /// Resolves an ArchestrA user GUID to its MXAccess user id + /// (MXAccess ArchestrAUserToId). + /// + /// The ServerHandle from register. + /// The ArchestrA user GUID to resolve. + /// Cancellation token. + /// The resolved MXAccess user id. + public async Task ArchestraUserToIdAsync( + int serverHandle, + string userIdGuid, + CancellationToken cancellationToken = default) + { + MxCommandReply reply = await ArchestraUserToIdRawAsync(serverHandle, userIdGuid, cancellationToken) + .ConfigureAwait(false); + reply.EnsureProtocolSuccess().EnsureMxAccessSuccess(); + return reply.ArchestraUserToId?.UserId ?? reply.ReturnValue.Int32Value; + } + + /// + /// Resolves an ArchestrA user GUID to its MXAccess user id without error checking. + /// + /// The ServerHandle from register. + /// The ArchestrA user GUID to resolve. + /// Cancellation token. + /// The raw server reply. + public Task ArchestraUserToIdRawAsync( + int serverHandle, + string userIdGuid, + CancellationToken cancellationToken = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(userIdGuid); + + return InvokeCommandAsync( + new MxCommand + { + Kind = MxCommandKind.ArchestraUserToId, + ArchestraUserToId = new ArchestrAUserToIdCommand + { + ServerHandle = serverHandle, + UserIdGuid = userIdGuid, + }, + }, + cancellationToken); + } + /// /// Invokes an MXAccess command on this session. /// diff --git a/clients/go/README.md b/clients/go/README.md index 2adfec4..c76cbbe 100644 --- a/clients/go/README.md +++ b/clients/go/README.md @@ -84,7 +84,9 @@ true` to verify against the OS/system trust roots without pinning. See [Gateway Configuration](../../docs/GatewayConfiguration.md#automatic-self-signed-certificate). `Client.OpenSession` returns a `Session` with helpers for `Register`, -`AddItem`, `AddItem2`, `Advise`, `Write`, `Events`, and `Close`. Prefer +`AddItem`, `AddItem2`, `Advise`, `AdviseSupervisory`, `Write`, `WriteSecured`, +`WriteSecured2`, `AuthenticateUser`, `ArchestrAUserToId`, `AddBufferedItem`, +`SetBufferedUpdateInterval`, `Suspend`, `Activate`, `Events`, and `Close`. Prefer `SubscribeEvents` or `SubscribeEventsAfter` for long-running streams because the returned subscription owns cancellation and exposes `Close` for deterministic goroutine cleanup. Raw protobuf messages remain available through the @@ -151,29 +153,32 @@ still need the write attributed to a user id, you must first advise the item supervisory and then pass that user id on the write. Without the supervisory advise the `userID` on a plain write is ignored. -The session exposes `Advise`/`UnAdvise` but not supervisory advise, so send it -through the generic command channel: +The session exposes a typed `AdviseSupervisory` helper alongside `Advise`/`UnAdvise`: ```go -_, err := client.Invoke(ctx, &pb.MxCommandRequest{ - SessionId: session.ID(), - Command: &pb.MxCommand{ - Kind: pb.MxCommandKind_MX_COMMAND_KIND_ADVISE_SUPERVISORY, - Payload: &pb.MxCommand_AdviseSupervisory{ - AdviseSupervisory: &pb.AdviseSupervisoryCommand{ - ServerHandle: serverHandle, - ItemHandle: itemHandle, - }, - }, - }, -}) - +err := session.AdviseSupervisory(ctx, serverHandle, itemHandle) +// ... err = session.Write(ctx, serverHandle, itemHandle, value, userID) ``` The CLI exposes the same command as `advise-supervisory`, and `write` takes `-user-id`. +### Secured writes and user authentication + +The verified/secured path has typed single-item helpers too: +`AuthenticateUser(ctx, serverHandle, verifyUser, verifyUserPassword)` returns the +resolved MXAccess user id, and `WriteSecured` / `WriteSecured2` issue the secured +write. Credentials passed to `AuthenticateUser`, and the string content of a +`WriteSecured`/`WriteSecured2` value, are kept out of any error the client +surfaces (they route through the same redaction seam as the API key) and are +never logged — callers must likewise keep them out of their own logs. MXAccess +parity holds: a `WriteSecured` issued without a matching prior `AuthenticateUser` +and supervisory advise fails natively, and that failure is surfaced unchanged +rather than pre-empted. The CLI exposes `authenticate-user` (credential via +`-password-env`, default `MXGATEWAY_VERIFY_PASSWORD`, or `-password`) and +`write-secured`. + ### Array writes replace the whole array A write to an array attribute **replaces the entire array**; it is not an @@ -378,6 +383,8 @@ Every subcommand wired into the CLI. All accept the common flags | `unsubscribe-bulk` | Unadvise many item handles in one call. | | `read-bulk` | Read snapshots for many item handles in one call. | | `write` | Write one value (`-type`, `-value`). | +| `write-secured` | Secured single-item write (`-current-user-id`, `-verifier-user-id`, `-type`, `-value`). | +| `authenticate-user` | Authenticate a user, printing the resolved user id (`-verify-user`, `-password-env`/`-password`). | | `write-bulk` | Write many values (`-item-handles`, `-values`, counts must match). | | `write2-bulk` | `write-bulk` with a shared `-timestamp-value` (RFC 3339). | | `write-secured-bulk` | Secured bulk write (`-current-user-id`, `-verifier-user-id`). | diff --git a/clients/go/cmd/mxgw-go/main.go b/clients/go/cmd/mxgw-go/main.go index 4d8e4c8..3a25ade 100644 --- a/clients/go/cmd/mxgw-go/main.go +++ b/clients/go/cmd/mxgw-go/main.go @@ -21,7 +21,6 @@ import ( "syscall" "time" - pb "gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go/internal/generated" "gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go/mxgateway" "google.golang.org/protobuf/encoding/protojson" "google.golang.org/protobuf/reflect/protoreflect" @@ -90,6 +89,10 @@ func runWithIO(ctx context.Context, args []string, stdout, stderr io.Writer) err return runAdvise(ctx, args[1:], stdout, stderr) case "advise-supervisory": return runAdviseSupervisory(ctx, args[1:], stdout, stderr) + case "write-secured": + return runWriteSecured(ctx, args[1:], stdout, stderr) + case "authenticate-user": + return runAuthenticateUser(ctx, args[1:], stdout, stderr) case "subscribe-bulk": return runSubscribeBulk(ctx, args[1:], stdout, stderr) case "unsubscribe-bulk": @@ -383,21 +386,90 @@ func runAdviseSupervisory(ctx context.Context, args []string, stdout, stderr io. } defer client.Close() - reply, err := client.Invoke(ctx, &pb.MxCommandRequest{ - SessionId: *sessionID, - Command: &pb.MxCommand{ - Kind: pb.MxCommandKind_MX_COMMAND_KIND_ADVISE_SUPERVISORY, - Payload: &pb.MxCommand_AdviseSupervisory{ - AdviseSupervisory: &pb.AdviseSupervisoryCommand{ - ServerHandle: int32(*serverHandle), - ItemHandle: int32(*itemHandle), - }, - }, - }, - }) + session := mxgateway.NewSessionForID(client, *sessionID) + reply, err := session.AdviseSupervisoryRaw(ctx, int32(*serverHandle), int32(*itemHandle)) return writeCommandOutput(stdout, *jsonOutput, "advise-supervisory", options, reply, err) } +func runWriteSecured(ctx context.Context, args []string, stdout, stderr io.Writer) error { + flags := flag.NewFlagSet("write-secured", flag.ContinueOnError) + flags.SetOutput(stderr) + common := bindCommonFlags(flags) + jsonOutput := flags.Bool("json", false, "write JSON output") + sessionID := flags.String("session-id", "", "gateway session id") + serverHandle := flags.Int("server-handle", 0, "MXAccess server handle") + itemHandle := flags.Int("item-handle", 0, "MXAccess item handle") + currentUserID := flags.Int("current-user-id", 0, "MXAccess current user id") + verifierUserID := flags.Int("verifier-user-id", 0, "MXAccess verifier user id") + valueType := flags.String("type", "string", "value type: bool, int32, int64, float, double, string") + valueText := flags.String("value", "", "value text") + + if err := flags.Parse(args); err != nil { + return err + } + if *sessionID == "" { + return errors.New("session-id is required") + } + + value, err := parseValue(*valueType, *valueText) + if err != nil { + return err + } + + client, options, err := dialForCommand(ctx, common) + if err != nil { + return err + } + defer client.Close() + + session := mxgateway.NewSessionForID(client, *sessionID) + reply, err := session.WriteSecuredRaw(ctx, int32(*serverHandle), int32(*itemHandle), int32(*currentUserID), int32(*verifierUserID), value) + return writeCommandOutput(stdout, *jsonOutput, "write-secured", options, reply, err) +} + +func runAuthenticateUser(ctx context.Context, args []string, stdout, stderr io.Writer) error { + flags := flag.NewFlagSet("authenticate-user", flag.ContinueOnError) + flags.SetOutput(stderr) + common := bindCommonFlags(flags) + jsonOutput := flags.Bool("json", false, "write JSON output") + sessionID := flags.String("session-id", "", "gateway session id") + serverHandle := flags.Int("server-handle", 0, "MXAccess server handle") + verifyUser := flags.String("verify-user", "", "MXAccess user to authenticate") + // The credential is never accepted echoed on the command line by default: + // prefer the environment variable so it stays out of shell history and the + // process table. The -password flag remains for non-interactive scripting. + password := flags.String("password", "", "verify-user password (prefer -password-env)") + passwordEnv := flags.String("password-env", "MXGATEWAY_VERIFY_PASSWORD", "environment variable containing the verify-user password") + + if err := flags.Parse(args); err != nil { + return err + } + if *sessionID == "" { + return errors.New("session-id is required") + } + if *verifyUser == "" { + return errors.New("verify-user is required") + } + + resolvedPassword := *password + if resolvedPassword == "" && *passwordEnv != "" { + resolvedPassword = os.Getenv(*passwordEnv) + } + + client, options, err := dialForCommand(ctx, common) + if err != nil { + return err + } + defer client.Close() + + session := mxgateway.NewSessionForID(client, *sessionID) + // The raw reply carries only the resolved user id, never the credential, so + // writeCommandOutput can render it as-is; the credential is additionally + // scrubbed from any surfaced error by AuthenticateUserRaw. + reply, err := session.AuthenticateUserRaw(ctx, int32(*serverHandle), *verifyUser, resolvedPassword) + return writeCommandOutput(stdout, *jsonOutput, "authenticate-user", options, reply, err) +} + func runSubscribeBulk(ctx context.Context, args []string, stdout, stderr io.Writer) error { flags := flag.NewFlagSet("subscribe-bulk", flag.ContinueOnError) flags.SetOutput(stderr) @@ -1295,7 +1367,7 @@ type protojsonMessage interface { } func writeUsage(writer io.Writer) { - fmt.Fprintln(writer, "usage: mxgw-go ") + fmt.Fprintln(writer, "usage: mxgw-go ") } // batchEOR is the end-of-result sentinel emitted to stdout after every command diff --git a/clients/go/cmd/mxgw-go/main_test.go b/clients/go/cmd/mxgw-go/main_test.go index f78879b..b3102d8 100644 --- a/clients/go/cmd/mxgw-go/main_test.go +++ b/clients/go/cmd/mxgw-go/main_test.go @@ -568,6 +568,36 @@ func TestRunAdviseSupervisoryRequiresSessionID(t *testing.T) { } } +// TestRunWriteSecuredRequiresSessionID pins the write-secured session-id guard so +// it fails fast before dialing. +func TestRunWriteSecuredRequiresSessionID(t *testing.T) { + var stdout, stderr bytes.Buffer + err := runWithIO(t.Context(), []string{"write-secured", "-plaintext", "-api-key", "test"}, &stdout, &stderr) + if err == nil || !strings.Contains(err.Error(), "session-id is required") { + t.Fatalf("write-secured without -session-id error = %v", err) + } +} + +// TestRunAuthenticateUserRequiresVerifyUser pins that authenticate-user fails fast +// (before dialing) when the user is absent, and never echoes any credential in +// the guard error. +func TestRunAuthenticateUserRequiresVerifyUser(t *testing.T) { + var stdout, stderr bytes.Buffer + err := runWithIO(t.Context(), []string{ + "authenticate-user", + "-session-id", "s1", + "-password", "hunter2-password", + "-plaintext", + "-api-key", "test", + }, &stdout, &stderr) + if err == nil || !strings.Contains(err.Error(), "verify-user is required") { + t.Fatalf("authenticate-user without -verify-user error = %v", err) + } + if strings.Contains(err.Error(), "hunter2-password") { + t.Fatalf("authenticate-user guard error leaked the credential: %v", err) + } +} + // TestRunWriteBulkVariantRejectsMismatchedHandlesAndValues pins the len-mismatch // guard so a write-bulk with unequal item-handles / values counts fails fast // before any dial. diff --git a/clients/go/mxgateway/errors.go b/clients/go/mxgateway/errors.go index 778e56e..572716d 100644 --- a/clients/go/mxgateway/errors.go +++ b/clients/go/mxgateway/errors.go @@ -3,10 +3,68 @@ package mxgateway import ( "errors" "fmt" + "strings" pb "gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go/internal/generated" ) +// redactedSecretMarker is the placeholder substituted for credential material in +// surfaced error text. It matches the marker used by RedactAPIKey so the client +// presents one consistent redaction shape everywhere secrets could otherwise +// leak. +const redactedSecretMarker = "" + +// secretRedactingError wraps a typed error so any occurrence of a known +// credential in the underlying message is replaced with redactedSecretMarker in +// the surfaced text. Unwrap still exposes the wrapped error, so errors.As / +// errors.Is continue to reach the underlying MxAccessError, CommandError, or +// GatewayError. This is the seam that keeps AuthenticateUser credentials and +// WriteSecured/WriteSecured2 payload strings out of any error a caller might log, +// even if a gateway diagnostic message were to echo them back. +type secretRedactingError struct { + err error + secrets []string +} + +// Error returns the wrapped error's message with every non-empty secret redacted. +func (e *secretRedactingError) Error() string { + if e == nil || e.err == nil { + return "" + } + message := e.err.Error() + for _, secret := range e.secrets { + if secret != "" { + message = strings.ReplaceAll(message, secret, redactedSecretMarker) + } + } + return message +} + +// Unwrap returns the wrapped error so typed-error inspection still works through +// the redaction wrapper. +func (e *secretRedactingError) Unwrap() error { + if e == nil { + return nil + } + return e.err +} + +// redactSecrets wraps err so any occurrence of a non-empty secret in the surfaced +// message is redacted, while errors.As / errors.Is still reach the wrapped typed +// error. It returns nil unchanged and skips wrapping when no non-empty secret is +// supplied, so non-secret-bearing calls keep their original error verbatim. +func redactSecrets(err error, secrets ...string) error { + if err == nil { + return nil + } + for _, secret := range secrets { + if secret != "" { + return &secretRedactingError{err: err, secrets: secrets} + } + } + return err +} + // ErrSlowConsumer is the terminal error sent on the Events/EventsAfter // (cancel-when-full) path when the buffered results channel overflows because // the consumer fell behind. It is delivered as the final EventResult.Err before diff --git a/clients/go/mxgateway/session.go b/clients/go/mxgateway/session.go index c58e00f..7e920aa 100644 --- a/clients/go/mxgateway/session.go +++ b/clients/go/mxgateway/session.go @@ -706,6 +706,277 @@ func (s *Session) Write2Raw(ctx context.Context, serverHandle, itemHandle int32, }) } +// AdviseSupervisory invokes MXAccess AdviseSupervisory, advising an item on the +// supervisory (as opposed to runtime) data path. +func (s *Session) AdviseSupervisory(ctx context.Context, serverHandle, itemHandle int32) error { + _, err := s.AdviseSupervisoryRaw(ctx, serverHandle, itemHandle) + return err +} + +// AdviseSupervisoryRaw invokes MXAccess AdviseSupervisory and returns the raw reply. +func (s *Session) AdviseSupervisoryRaw(ctx context.Context, serverHandle, itemHandle int32) (*MxCommandReply, error) { + return s.invokeCommand(ctx, &pb.MxCommand{ + Kind: pb.MxCommandKind_MX_COMMAND_KIND_ADVISE_SUPERVISORY, + Payload: &pb.MxCommand_AdviseSupervisory{ + AdviseSupervisory: &pb.AdviseSupervisoryCommand{ + ServerHandle: serverHandle, + ItemHandle: itemHandle, + }, + }, + }) +} + +// WriteSecured invokes MXAccess WriteSecured (secured single-item write). +// +// The value is credential-sensitive: callers must not log it, and any error this +// call surfaces has the value's string content redacted (see WriteSecuredRaw). +// MXAccess parity is preserved — WriteSecured legitimately fails when it is not +// preceded by a matching AuthenticateUser + AdviseSupervisory or when the body +// carries no value; that native failure is surfaced as-is, never pre-empted or +// reordered by this client. +func (s *Session) WriteSecured(ctx context.Context, serverHandle, itemHandle, currentUserID, verifierUserID int32, value *MxValue) error { + _, err := s.WriteSecuredRaw(ctx, serverHandle, itemHandle, currentUserID, verifierUserID, value) + return err +} + +// WriteSecuredRaw invokes MXAccess WriteSecured and returns the raw reply. Any +// surfaced error is routed through the client's secret-redaction seam so a string +// write value never appears in error text. +func (s *Session) WriteSecuredRaw(ctx context.Context, serverHandle, itemHandle, currentUserID, verifierUserID int32, value *MxValue) (*MxCommandReply, error) { + if value == nil { + return nil, errors.New("mxgateway: write-secured value is required") + } + + reply, err := s.invokeCommand(ctx, &pb.MxCommand{ + Kind: pb.MxCommandKind_MX_COMMAND_KIND_WRITE_SECURED, + Payload: &pb.MxCommand_WriteSecured{ + WriteSecured: &pb.WriteSecuredCommand{ + ServerHandle: serverHandle, + ItemHandle: itemHandle, + CurrentUserId: currentUserID, + VerifierUserId: verifierUserID, + Value: value, + }, + }, + }) + return reply, redactSecrets(err, stringSecrets(value)...) +} + +// WriteSecured2 invokes MXAccess WriteSecured2 (secured, timestamped single-item write). +// +// Like WriteSecured, the value is credential-sensitive and its string content is +// scrubbed from any surfaced error. Native parity failures (missing prior +// AuthenticateUser/AdviseSupervisory, value-less body) are surfaced unchanged. +func (s *Session) WriteSecured2(ctx context.Context, serverHandle, itemHandle, currentUserID, verifierUserID int32, value, timestampValue *MxValue) error { + _, err := s.WriteSecured2Raw(ctx, serverHandle, itemHandle, currentUserID, verifierUserID, value, timestampValue) + return err +} + +// WriteSecured2Raw invokes MXAccess WriteSecured2 and returns the raw reply. Any +// surfaced error is routed through the client's secret-redaction seam. +func (s *Session) WriteSecured2Raw(ctx context.Context, serverHandle, itemHandle, currentUserID, verifierUserID int32, value, timestampValue *MxValue) (*MxCommandReply, error) { + if value == nil { + return nil, errors.New("mxgateway: write-secured2 value is required") + } + if timestampValue == nil { + return nil, errors.New("mxgateway: write-secured2 timestamp value is required") + } + + reply, err := s.invokeCommand(ctx, &pb.MxCommand{ + Kind: pb.MxCommandKind_MX_COMMAND_KIND_WRITE_SECURED2, + Payload: &pb.MxCommand_WriteSecured2{ + WriteSecured2: &pb.WriteSecured2Command{ + ServerHandle: serverHandle, + ItemHandle: itemHandle, + CurrentUserId: currentUserID, + VerifierUserId: verifierUserID, + Value: value, + TimestampValue: timestampValue, + }, + }, + }) + return reply, redactSecrets(err, stringSecrets(value)...) +} + +// AuthenticateUser invokes MXAccess AuthenticateUser and returns the resolved +// MXAccess user id. +// +// verifyUserPassword is a raw MXAccess credential: this client never logs it and +// scrubs it from any error it surfaces (see AuthenticateUserRaw). Callers must +// likewise keep it out of their own logs, metrics, and diagnostics. +func (s *Session) AuthenticateUser(ctx context.Context, serverHandle int32, verifyUser, verifyUserPassword string) (int32, error) { + reply, err := s.AuthenticateUserRaw(ctx, serverHandle, verifyUser, verifyUserPassword) + if err != nil { + return 0, err + } + if reply.GetAuthenticateUser() != nil { + return reply.GetAuthenticateUser().GetUserId(), nil + } + return reply.GetReturnValue().GetInt32Value(), nil +} + +// AuthenticateUserRaw invokes MXAccess AuthenticateUser and returns the raw +// reply. The credential is scrubbed from any surfaced error via the client's +// secret-redaction seam, so even a gateway diagnostic echoing the password back +// cannot leak it through this call's error. +func (s *Session) AuthenticateUserRaw(ctx context.Context, serverHandle int32, verifyUser, verifyUserPassword string) (*MxCommandReply, error) { + if verifyUser == "" { + return nil, errors.New("mxgateway: verify user is required") + } + + reply, err := s.invokeCommand(ctx, &pb.MxCommand{ + Kind: pb.MxCommandKind_MX_COMMAND_KIND_AUTHENTICATE_USER, + Payload: &pb.MxCommand_AuthenticateUser{ + AuthenticateUser: &pb.AuthenticateUserCommand{ + ServerHandle: serverHandle, + VerifyUser: verifyUser, + VerifyUserPassword: verifyUserPassword, + }, + }, + }) + return reply, redactSecrets(err, verifyUserPassword) +} + +// ArchestrAUserToId invokes MXAccess ArchestrAUserToId, resolving an ArchestrA +// user GUID to its MXAccess integer user id. +func (s *Session) ArchestrAUserToId(ctx context.Context, serverHandle int32, userIDGuid string) (int32, error) { + reply, err := s.ArchestrAUserToIdRaw(ctx, serverHandle, userIDGuid) + if err != nil { + return 0, err + } + if reply.GetArchestraUserToId() != nil { + return reply.GetArchestraUserToId().GetUserId(), nil + } + return reply.GetReturnValue().GetInt32Value(), nil +} + +// ArchestrAUserToIdRaw invokes MXAccess ArchestrAUserToId and returns the raw reply. +func (s *Session) ArchestrAUserToIdRaw(ctx context.Context, serverHandle int32, userIDGuid string) (*MxCommandReply, error) { + if userIDGuid == "" { + return nil, errors.New("mxgateway: user id GUID is required") + } + + return s.invokeCommand(ctx, &pb.MxCommand{ + Kind: pb.MxCommandKind_MX_COMMAND_KIND_ARCHESTRA_USER_TO_ID, + Payload: &pb.MxCommand_ArchestraUserToId{ + ArchestraUserToId: &pb.ArchestrAUserToIdCommand{ + ServerHandle: serverHandle, + UserIdGuid: userIDGuid, + }, + }, + }) +} + +// AddBufferedItem invokes MXAccess AddBufferedItem and returns the item handle. +func (s *Session) AddBufferedItem(ctx context.Context, serverHandle int32, itemDefinition, itemContext string) (int32, error) { + reply, err := s.AddBufferedItemRaw(ctx, serverHandle, itemDefinition, itemContext) + if err != nil { + return 0, err + } + if reply.GetAddBufferedItem() != nil { + return reply.GetAddBufferedItem().GetItemHandle(), nil + } + return reply.GetReturnValue().GetInt32Value(), nil +} + +// AddBufferedItemRaw invokes MXAccess AddBufferedItem and returns the raw reply. +func (s *Session) AddBufferedItemRaw(ctx context.Context, serverHandle int32, itemDefinition, itemContext string) (*MxCommandReply, error) { + if itemDefinition == "" { + return nil, errors.New("mxgateway: item definition is required") + } + + return s.invokeCommand(ctx, &pb.MxCommand{ + Kind: pb.MxCommandKind_MX_COMMAND_KIND_ADD_BUFFERED_ITEM, + Payload: &pb.MxCommand_AddBufferedItem{ + AddBufferedItem: &pb.AddBufferedItemCommand{ + ServerHandle: serverHandle, + ItemDefinition: itemDefinition, + ItemContext: itemContext, + }, + }, + }) +} + +// SetBufferedUpdateInterval invokes MXAccess SetBufferedUpdateInterval. +func (s *Session) SetBufferedUpdateInterval(ctx context.Context, serverHandle, updateIntervalMilliseconds int32) error { + _, err := s.SetBufferedUpdateIntervalRaw(ctx, serverHandle, updateIntervalMilliseconds) + return err +} + +// SetBufferedUpdateIntervalRaw invokes MXAccess SetBufferedUpdateInterval and returns the raw reply. +func (s *Session) SetBufferedUpdateIntervalRaw(ctx context.Context, serverHandle, updateIntervalMilliseconds int32) (*MxCommandReply, error) { + return s.invokeCommand(ctx, &pb.MxCommand{ + Kind: pb.MxCommandKind_MX_COMMAND_KIND_SET_BUFFERED_UPDATE_INTERVAL, + Payload: &pb.MxCommand_SetBufferedUpdateInterval{ + SetBufferedUpdateInterval: &pb.SetBufferedUpdateIntervalCommand{ + ServerHandle: serverHandle, + UpdateIntervalMilliseconds: updateIntervalMilliseconds, + }, + }, + }) +} + +// Suspend invokes MXAccess Suspend and returns the resulting item status. +func (s *Session) Suspend(ctx context.Context, serverHandle, itemHandle int32) (*MxStatusProxy, error) { + reply, err := s.SuspendRaw(ctx, serverHandle, itemHandle) + if err != nil { + return nil, err + } + return reply.GetSuspend().GetStatus(), nil +} + +// SuspendRaw invokes MXAccess Suspend and returns the raw reply. +func (s *Session) SuspendRaw(ctx context.Context, serverHandle, itemHandle int32) (*MxCommandReply, error) { + return s.invokeCommand(ctx, &pb.MxCommand{ + Kind: pb.MxCommandKind_MX_COMMAND_KIND_SUSPEND, + Payload: &pb.MxCommand_Suspend{ + Suspend: &pb.SuspendCommand{ + ServerHandle: serverHandle, + ItemHandle: itemHandle, + }, + }, + }) +} + +// Activate invokes MXAccess Activate and returns the resulting item status. +func (s *Session) Activate(ctx context.Context, serverHandle, itemHandle int32) (*MxStatusProxy, error) { + reply, err := s.ActivateRaw(ctx, serverHandle, itemHandle) + if err != nil { + return nil, err + } + return reply.GetActivate().GetStatus(), nil +} + +// ActivateRaw invokes MXAccess Activate and returns the raw reply. +func (s *Session) ActivateRaw(ctx context.Context, serverHandle, itemHandle int32) (*MxCommandReply, error) { + return s.invokeCommand(ctx, &pb.MxCommand{ + Kind: pb.MxCommandKind_MX_COMMAND_KIND_ACTIVATE, + Payload: &pb.MxCommand_Activate{ + Activate: &pb.ActivateCommand{ + ServerHandle: serverHandle, + ItemHandle: itemHandle, + }, + }, + }) +} + +// stringSecrets collects the non-empty string content of the given values so it +// can be scrubbed from surfaced errors. Only string-typed MxValues carry +// scrubable text; non-string values (numbers, timestamps, arrays) contribute +// nothing. +func stringSecrets(values ...*MxValue) []string { + var secrets []string + for _, value := range values { + if value == nil { + continue + } + if stringValue, ok := value.GetKind().(*pb.MxValue_StringValue); ok && stringValue.StringValue != "" { + secrets = append(secrets, stringValue.StringValue) + } + } + return secrets +} + // Events streams ordered session events until the server ends the stream, // context cancellation stops Recv, or a terminal error is sent. // diff --git a/clients/go/mxgateway/session_parity_helpers_test.go b/clients/go/mxgateway/session_parity_helpers_test.go new file mode 100644 index 0000000..79bae27 --- /dev/null +++ b/clients/go/mxgateway/session_parity_helpers_test.go @@ -0,0 +1,224 @@ +package mxgateway + +import ( + "context" + "errors" + "strings" + "testing" + + pb "gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go/internal/generated" +) + +// TestAdviseSupervisoryBuildsCommandAndExposesRawReply pins that the promoted +// typed helper emits an ADVISE_SUPERVISORY command carrying the server/item +// handles and returns the raw reply. +func TestAdviseSupervisoryBuildsCommandAndExposesRawReply(t *testing.T) { + fake := &fakeGatewayServer{ + invokeReply: &pb.MxCommandReply{ + SessionId: "session-1", + Kind: pb.MxCommandKind_MX_COMMAND_KIND_ADVISE_SUPERVISORY, + ProtocolStatus: &pb.ProtocolStatus{Code: pb.ProtocolStatusCode_PROTOCOL_STATUS_CODE_OK}, + }, + } + client, cleanup := newBufconnClient(t, fake) + defer cleanup() + session := NewSessionForID(client, "session-1") + + reply, err := session.AdviseSupervisoryRaw(context.Background(), 12, 34) + if err != nil { + t.Fatalf("AdviseSupervisoryRaw() error = %v", err) + } + if reply.GetKind() != pb.MxCommandKind_MX_COMMAND_KIND_ADVISE_SUPERVISORY { + t.Fatalf("reply kind = %s", reply.GetKind()) + } + cmd := fake.invokeRequest.GetCommand() + if cmd.GetKind() != pb.MxCommandKind_MX_COMMAND_KIND_ADVISE_SUPERVISORY { + t.Fatalf("command kind = %s", cmd.GetKind()) + } + if cmd.GetAdviseSupervisory().GetServerHandle() != 12 || cmd.GetAdviseSupervisory().GetItemHandle() != 34 { + t.Fatalf("advise-supervisory handles = (%d, %d), want (12, 34)", + cmd.GetAdviseSupervisory().GetServerHandle(), cmd.GetAdviseSupervisory().GetItemHandle()) + } + + // The error-returning wrapper drops the reply but must not error on success. + if err := session.AdviseSupervisory(context.Background(), 12, 34); err != nil { + t.Fatalf("AdviseSupervisory() error = %v", err) + } +} + +// TestWriteSecuredSurfacesNativeFailureWithoutPriorAuthenticate pins MXAccess +// parity: WriteSecured issued without a preceding AuthenticateUser is rejected +// natively, and the client surfaces that failure as a typed MxAccessError rather +// than pre-validating or reordering. The write value is also kept out of the +// surfaced error text. +func TestWriteSecuredSurfacesNativeFailureWithoutPriorAuthenticate(t *testing.T) { + hresult := int32(-2147024891) // E_ACCESSDENIED + fake := &fakeGatewayServer{ + invokeReply: &pb.MxCommandReply{ + SessionId: "session-1", + Kind: pb.MxCommandKind_MX_COMMAND_KIND_WRITE_SECURED, + Hresult: &hresult, + DiagnosticMessage: "WriteSecured requires a prior AuthenticateUser", + ProtocolStatus: &pb.ProtocolStatus{ + Code: pb.ProtocolStatusCode_PROTOCOL_STATUS_CODE_MXACCESS_FAILURE, + Message: "MXAccess failed", + }, + }, + } + client, cleanup := newBufconnClient(t, fake) + defer cleanup() + session := NewSessionForID(client, "session-1") + + securedValue := "supersecret-payload" + err := session.WriteSecured(context.Background(), 12, 34, 0, 0, StringValue(securedValue)) + + var mxErr *MxAccessError + if !errors.As(err, &mxErr) { + t.Fatalf("error %T does not support errors.As(*MxAccessError); err = %v", err, err) + } + if strings.Contains(err.Error(), securedValue) { + t.Fatalf("surfaced error leaked the secured payload: %q", err.Error()) + } + + // The command must carry the secured fields verbatim (parity: unaltered). + cmd := fake.invokeRequest.GetCommand() + if cmd.GetKind() != pb.MxCommandKind_MX_COMMAND_KIND_WRITE_SECURED { + t.Fatalf("command kind = %s", cmd.GetKind()) + } + if cmd.GetWriteSecured().GetValue().GetStringValue() != securedValue { + t.Fatalf("wire value = %q, want %q", cmd.GetWriteSecured().GetValue().GetStringValue(), securedValue) + } +} + +// TestWriteSecuredRejectsNilValueWithoutRoundTrip pins the client-side required +// guard, which never echoes the (absent) value. +func TestWriteSecuredRejectsNilValue(t *testing.T) { + fake := &fakeGatewayServer{} + client, cleanup := newBufconnClient(t, fake) + defer cleanup() + session := NewSessionForID(client, "session-1") + + if err := session.WriteSecured(context.Background(), 1, 2, 0, 0, nil); err == nil || + !strings.Contains(err.Error(), "write-secured value is required") { + t.Fatalf("WriteSecured(nil value) error = %v", err) + } +} + +// TestAuthenticateUserReturnsUserIDOnHappyPath pins the typed helper unpacking of +// AuthenticateUserReply.user_id and that the credential is carried on the wire +// but never surfaced. +func TestAuthenticateUserReturnsUserIDOnHappyPath(t *testing.T) { + fake := &fakeGatewayServer{ + invokeReply: &pb.MxCommandReply{ + SessionId: "session-1", + Kind: pb.MxCommandKind_MX_COMMAND_KIND_AUTHENTICATE_USER, + ProtocolStatus: &pb.ProtocolStatus{Code: pb.ProtocolStatusCode_PROTOCOL_STATUS_CODE_OK}, + Payload: &pb.MxCommandReply_AuthenticateUser{ + AuthenticateUser: &pb.AuthenticateUserReply{UserId: 4242}, + }, + }, + } + client, cleanup := newBufconnClient(t, fake) + defer cleanup() + session := NewSessionForID(client, "session-1") + + userID, err := session.AuthenticateUser(context.Background(), 12, "operator", "hunter2-password") + if err != nil { + t.Fatalf("AuthenticateUser() error = %v", err) + } + if userID != 4242 { + t.Fatalf("user id = %d, want 4242", userID) + } + + cmd := fake.invokeRequest.GetCommand() + if cmd.GetKind() != pb.MxCommandKind_MX_COMMAND_KIND_AUTHENTICATE_USER { + t.Fatalf("command kind = %s", cmd.GetKind()) + } + if cmd.GetAuthenticateUser().GetVerifyUser() != "operator" { + t.Fatalf("verify user = %q, want operator", cmd.GetAuthenticateUser().GetVerifyUser()) + } + if cmd.GetAuthenticateUser().GetVerifyUserPassword() != "hunter2-password" { + t.Fatalf("password not carried to the wire verbatim") + } +} + +// TestAuthenticateUserScrubsCredentialFromSurfacedError proves the redaction +// seam: even when the gateway diagnostic message echoes the credential back, the +// surfaced error redacts it while the typed MxAccessError remains reachable via +// errors.As. +func TestAuthenticateUserScrubsCredentialFromSurfacedError(t *testing.T) { + password := "hunter2-password" + fake := &fakeGatewayServer{ + invokeReply: &pb.MxCommandReply{ + SessionId: "session-1", + Kind: pb.MxCommandKind_MX_COMMAND_KIND_AUTHENTICATE_USER, + DiagnosticMessage: "authentication failed for password " + password, + // Message is intentionally left empty so MxAccessError.Error() falls + // through to the diagnostic message — the free-text field that could + // otherwise echo the credential back to a caller's log. + ProtocolStatus: &pb.ProtocolStatus{ + Code: pb.ProtocolStatusCode_PROTOCOL_STATUS_CODE_MXACCESS_FAILURE, + }, + }, + } + client, cleanup := newBufconnClient(t, fake) + defer cleanup() + session := NewSessionForID(client, "session-1") + + _, err := session.AuthenticateUser(context.Background(), 12, "operator", password) + if err == nil { + t.Fatal("AuthenticateUser() returned no error on native failure") + } + if strings.Contains(err.Error(), password) { + t.Fatalf("surfaced error leaked the credential: %q", err.Error()) + } + if !strings.Contains(err.Error(), redactedSecretMarker) { + t.Fatalf("surfaced error missing redaction marker: %q", err.Error()) + } + var mxErr *MxAccessError + if !errors.As(err, &mxErr) { + t.Fatalf("redaction wrapper broke errors.As(*MxAccessError); err = %v", err) + } +} + +// TestAuthenticateUserRequiresVerifyUser pins the client-side required guard. +func TestAuthenticateUserRequiresVerifyUser(t *testing.T) { + fake := &fakeGatewayServer{} + client, cleanup := newBufconnClient(t, fake) + defer cleanup() + session := NewSessionForID(client, "session-1") + + if _, err := session.AuthenticateUser(context.Background(), 12, "", "pw"); err == nil || + !strings.Contains(err.Error(), "verify user is required") { + t.Fatalf("AuthenticateUser(empty user) error = %v", err) + } +} + +// TestSuspendActivateReturnStatus covers two Phase 2 helpers that unpack an +// MxStatusProxy from their dedicated reply arms. +func TestSuspendActivateReturnStatus(t *testing.T) { + fake := &fakeGatewayServer{ + invokeReply: &pb.MxCommandReply{ + SessionId: "session-1", + Kind: pb.MxCommandKind_MX_COMMAND_KIND_SUSPEND, + ProtocolStatus: &pb.ProtocolStatus{Code: pb.ProtocolStatusCode_PROTOCOL_STATUS_CODE_OK}, + Payload: &pb.MxCommandReply_Suspend{ + Suspend: &pb.SuspendReply{Status: &pb.MxStatusProxy{Success: 1, DiagnosticText: "suspended"}}, + }, + }, + } + client, cleanup := newBufconnClient(t, fake) + defer cleanup() + session := NewSessionForID(client, "session-1") + + status, err := session.Suspend(context.Background(), 12, 34) + if err != nil { + t.Fatalf("Suspend() error = %v", err) + } + if status.GetDiagnosticText() != "suspended" { + t.Fatalf("status diagnostic = %q, want suspended", status.GetDiagnosticText()) + } + if fake.invokeRequest.GetCommand().GetSuspend().GetItemHandle() != 34 { + t.Fatalf("suspend item handle = %d, want 34", fake.invokeRequest.GetCommand().GetSuspend().GetItemHandle()) + } +} diff --git a/clients/python/README.md b/clients/python/README.md index ef8409f..2e14f73 100644 --- a/clients/python/README.md +++ b/clients/python/README.md @@ -153,19 +153,11 @@ but still need the write attributed to a user id, you must first advise the item supervisory and then pass that user id on the write. Without the supervisory advise the `user_id` on a plain write is ignored. -The session exposes `advise`/`unadvise` but not supervisory advise, so send it -through the generic command channel: +The session exposes a typed `advise_supervisory` helper alongside +`advise`/`unadvise`: ```python -await session.invoke( - pb.MxCommand( - kind=pb.MX_COMMAND_KIND_ADVISE_SUPERVISORY, - advise_supervisory=pb.AdviseSupervisoryCommand( - server_handle=server_handle, - item_handle=item_handle, - ), - ) -) +await session.advise_supervisory(server_handle, item_handle) await session.write(server_handle, item_handle, value, user_id=user_id) ``` @@ -173,6 +165,30 @@ await session.write(server_handle, item_handle, value, user_id=user_id) The CLI exposes the same command as `advise-supervisory`, and `write` / `write2` take `--user-id`. +For the verified/secured path, `authenticate_user`, `write_secured`, +`write_secured2`, and `archestra_user_to_id` are typed session helpers too. The +credential passed to `authenticate_user` and the values written by +`write_secured`/`write_secured2` are treated as secrets: they are never logged +and are scrubbed from any surfaced error message. MXAccess parity is preserved — +a `write_secured` that fails because no prior `authenticate_user` + +`advise_supervisory` established a supervisory context surfaces the native +failure as `MxAccessError` rather than being silently "fixed": + +```python +user_id = await session.authenticate_user(server_handle, "operator", password) +await session.advise_supervisory(server_handle, item_handle) +await session.write_secured( + server_handle, + item_handle, + value, + current_user_id=user_id, + verifier_user_id=user_id, +) +``` + +The CLI mirrors these as `authenticate-user` (credential via `--password` or, +preferably, `--password-env`) and `write-secured`. + ### Array writes replace the whole array A write to an array attribute **replaces the entire array**; it is not an diff --git a/clients/python/src/zb_mom_ww_mxgateway/session.py b/clients/python/src/zb_mom_ww_mxgateway/session.py index 973ba9a..057d37f 100644 --- a/clients/python/src/zb_mom_ww_mxgateway/session.py +++ b/clients/python/src/zb_mom_ww_mxgateway/session.py @@ -4,7 +4,8 @@ from __future__ import annotations from collections.abc import AsyncIterator, Sequence -from .errors import ensure_mxaccess_success +from .auth import redact_secret +from .errors import MxGatewayError, ensure_mxaccess_success from .events import ReplayGap from .generated import mxaccess_gateway_pb2 as pb from .values import MxValueInput, to_mx_value @@ -569,6 +570,249 @@ class Session: correlation_id=correlation_id, ) + async def _invoke_redacted( + self, + command: pb.MxCommand, + *, + correlation_id: str, + secrets: Sequence[str | None], + ) -> pb.MxCommandReply: + """Invoke a command whose request carries credential-sensitive data. + + Runs the same gateway + MXAccess validation as :meth:`invoke`, but scrubs + the supplied secret substrings from any surfaced error message before it + propagates. MXAccess parity is preserved — the native failure is still + raised as :class:`~zb_mom_ww_mxgateway.errors.MxAccessError`; only the + credential text is removed from the message so it can never reach logs. + """ + + try: + return await self.invoke(command, correlation_id=correlation_id) + except MxGatewayError as error: + _redact_error(error, secrets) + raise + + async def advise_supervisory( + self, + server_handle: int, + item_handle: int, + *, + correlation_id: str = "", + ) -> None: + """Invoke MXAccess `AdviseSupervisory` for an `ItemHandle`. + + Supervisory advise is the prerequisite for user-attributed and secured + writes: it must be established (typically after :meth:`authenticate_user`) + before a ``user_id``-bearing :meth:`write` or a :meth:`write_secured` + takes effect. + """ + await self.invoke( + pb.MxCommand( + kind=pb.MX_COMMAND_KIND_ADVISE_SUPERVISORY, + advise_supervisory=pb.AdviseSupervisoryCommand( + server_handle=server_handle, + item_handle=item_handle, + ), + ), + correlation_id=correlation_id, + ) + + async def write_secured( + self, + server_handle: int, + item_handle: int, + value: MxValueInput, + *, + current_user_id: int = 0, + verifier_user_id: int = 0, + correlation_id: str = "", + ) -> None: + """Invoke MXAccess `WriteSecured` — a signed/verified write. + + The written *value* is credential-sensitive and is scrubbed from any + surfaced error message (never logged). MXAccess parity is the contract: + ``WriteSecured`` failing before a prior :meth:`authenticate_user` + + :meth:`advise_supervisory`, or before a value-bearing body, is the native + behaviour and is surfaced as-is — it is not "fixed". + """ + await self._invoke_redacted( + pb.MxCommand( + kind=pb.MX_COMMAND_KIND_WRITE_SECURED, + write_secured=pb.WriteSecuredCommand( + server_handle=server_handle, + item_handle=item_handle, + current_user_id=current_user_id, + verifier_user_id=verifier_user_id, + value=to_mx_value(value), + ), + ), + correlation_id=correlation_id, + secrets=_value_secrets(value), + ) + + async def write_secured2( + self, + server_handle: int, + item_handle: int, + value: MxValueInput, + timestamp_value: MxValueInput, + *, + current_user_id: int = 0, + verifier_user_id: int = 0, + correlation_id: str = "", + ) -> None: + """Invoke MXAccess `WriteSecured2` — a signed/verified, timestamped write. + + Like :meth:`write_secured` but also stamps a client-supplied timestamp. + The written *value* is credential-sensitive and is scrubbed from any + surfaced error message. Native pre-condition failures are surfaced as-is. + """ + await self._invoke_redacted( + pb.MxCommand( + kind=pb.MX_COMMAND_KIND_WRITE_SECURED2, + write_secured2=pb.WriteSecured2Command( + server_handle=server_handle, + item_handle=item_handle, + current_user_id=current_user_id, + verifier_user_id=verifier_user_id, + value=to_mx_value(value), + timestamp_value=to_mx_value(timestamp_value), + ), + ), + correlation_id=correlation_id, + secrets=_value_secrets(value), + ) + + async def authenticate_user( + self, + server_handle: int, + verify_user: str, + verify_user_password: str, + *, + correlation_id: str = "", + ) -> int: + """Invoke MXAccess `AuthenticateUser` and return the resolved Galaxy user id. + + *verify_user_password* is a raw MXAccess credential: it is never logged + and is scrubbed from any surfaced error message. A native authentication + failure is surfaced as :class:`~zb_mom_ww_mxgateway.errors.MxAccessError` + with the credential removed. + """ + reply = await self._invoke_redacted( + pb.MxCommand( + kind=pb.MX_COMMAND_KIND_AUTHENTICATE_USER, + authenticate_user=pb.AuthenticateUserCommand( + server_handle=server_handle, + verify_user=verify_user, + verify_user_password=verify_user_password, + ), + ), + correlation_id=correlation_id, + secrets=[verify_user_password], + ) + return reply.authenticate_user.user_id + + async def archestra_user_to_id( + self, + server_handle: int, + user_id_guid: str, + *, + correlation_id: str = "", + ) -> int: + """Invoke MXAccess `ArchestrAUserToId` and return the resolved user id.""" + reply = await self.invoke( + pb.MxCommand( + kind=pb.MX_COMMAND_KIND_ARCHESTRA_USER_TO_ID, + archestra_user_to_id=pb.ArchestrAUserToIdCommand( + server_handle=server_handle, + user_id_guid=user_id_guid, + ), + ), + correlation_id=correlation_id, + ) + return reply.archestra_user_to_id.user_id + + async def add_buffered_item( + self, + server_handle: int, + item_definition: str, + item_context: str = "", + *, + correlation_id: str = "", + ) -> int: + """Invoke MXAccess `AddBufferedItem` and return the new `ItemHandle`.""" + reply = await self.invoke( + pb.MxCommand( + kind=pb.MX_COMMAND_KIND_ADD_BUFFERED_ITEM, + add_buffered_item=pb.AddBufferedItemCommand( + server_handle=server_handle, + item_definition=item_definition, + item_context=item_context, + ), + ), + correlation_id=correlation_id, + ) + return reply.add_buffered_item.item_handle + + async def set_buffered_update_interval( + self, + server_handle: int, + update_interval_milliseconds: int, + *, + correlation_id: str = "", + ) -> None: + """Invoke MXAccess `SetBufferedUpdateInterval` for the server handle.""" + await self.invoke( + pb.MxCommand( + kind=pb.MX_COMMAND_KIND_SET_BUFFERED_UPDATE_INTERVAL, + set_buffered_update_interval=pb.SetBufferedUpdateIntervalCommand( + server_handle=server_handle, + update_interval_milliseconds=update_interval_milliseconds, + ), + ), + correlation_id=correlation_id, + ) + + async def suspend( + self, + server_handle: int, + item_handle: int, + *, + correlation_id: str = "", + ) -> pb.MxStatusProxy: + """Invoke MXAccess `Suspend` for an `ItemHandle` and return its status.""" + reply = await self.invoke( + pb.MxCommand( + kind=pb.MX_COMMAND_KIND_SUSPEND, + suspend=pb.SuspendCommand( + server_handle=server_handle, + item_handle=item_handle, + ), + ), + correlation_id=correlation_id, + ) + return reply.suspend.status + + async def activate( + self, + server_handle: int, + item_handle: int, + *, + correlation_id: str = "", + ) -> pb.MxStatusProxy: + """Invoke MXAccess `Activate` for a suspended `ItemHandle` and return its status.""" + reply = await self.invoke( + pb.MxCommand( + kind=pb.MX_COMMAND_KIND_ACTIVATE, + activate=pb.ActivateCommand( + server_handle=server_handle, + item_handle=item_handle, + ), + ), + correlation_id=correlation_id, + ) + return reply.activate.status + def stream_events( self, *, @@ -631,4 +875,39 @@ def _ensure_bulk_size(name: str, count: int) -> None: raise ValueError(f"{name} bulk commands are limited to {MAX_BULK_ITEMS} item(s)") +def _value_secrets(value: MxValueInput) -> list[str]: + """Return the redaction candidate strings for a credential-sensitive write value. + + Secured-write payloads may carry a password or other secret. Only textual + values can appear verbatim in a surfaced error, so a string value (or a + UTF-8-decodable ``bytes`` value) is returned for scrubbing; other value + kinds have no verbatim text form to leak. + """ + if isinstance(value, str): + return [value] if value else [] + if isinstance(value, bytes): + try: + decoded = value.decode("utf-8") + except UnicodeDecodeError: + return [] + return [decoded] if decoded else [] + return [] + + +def _redact_error(error: MxGatewayError, secrets: Sequence[str | None]) -> None: + """Scrub secret substrings from a raised error's message in place. + + Rewrites ``error.args[0]`` (the message returned by ``str(error)``) through + the shared :func:`~zb_mom_ww_mxgateway.auth.redact_secret` seam so credential + text can never reach logs or be re-raised to a caller. The + ``protocol_status`` / ``raw_reply`` context is left untouched — those hold the + gateway's own fields, which never echo the client-supplied secret. + """ + scrubbed = [secret for secret in secrets if secret] + if not scrubbed: + return + if error.args and isinstance(error.args[0], str): + error.args = (redact_secret(error.args[0], scrubbed), *error.args[1:]) + + from .client import GatewayClient # noqa: E402 diff --git a/clients/python/src/zb_mom_ww_mxgateway_cli/commands.py b/clients/python/src/zb_mom_ww_mxgateway_cli/commands.py index 99876ad..154c339 100644 --- a/clients/python/src/zb_mom_ww_mxgateway_cli/commands.py +++ b/clients/python/src/zb_mom_ww_mxgateway_cli/commands.py @@ -294,6 +294,56 @@ def advise_supervisory(**kwargs: Any) -> None: ) +@main.command("write-secured") +@gateway_options +@click.option("--session-id", required=True, help="Gateway session id.") +@click.option("--server-handle", required=True, type=int, help="MXAccess server handle.") +@click.option("--item-handle", required=True, type=int, help="MXAccess item handle.") +@click.option("--type", "value_type", default="string", show_default=True) +@click.option("--value", required=True, help="Value to write (credential-sensitive; never logged).") +@click.option("--current-user-id", default=0, type=int, show_default=True) +@click.option("--verifier-user-id", default=0, type=int, show_default=True) +@click.option("--correlation-id", default="", help="Client correlation id.") +@click.option("--json", "output_json", is_flag=True, help="Emit JSON output.") +def write_secured(**kwargs: Any) -> None: + """Invoke MXAccess WriteSecured — a signed/verified write (credential-sensitive).""" + + _run( + _write_secured(**kwargs), + output_json=kwargs["output_json"], + secrets=_secrets(kwargs) + [kwargs.get("value")], + ) + + +@main.command("authenticate-user") +@gateway_options +@click.option("--session-id", required=True, help="Gateway session id.") +@click.option("--server-handle", required=True, type=int, help="MXAccess server handle.") +@click.option("--verify-user", required=True, help="MXAccess user name to authenticate.") +@click.option( + "--password", + default=None, + help="User password. Prefer --password-env so the secret is not visible on the command line.", +) +@click.option( + "--password-env", + default=None, + help="Environment variable holding the user password.", +) +@click.option("--correlation-id", default="", help="Client correlation id.") +@click.option("--json", "output_json", is_flag=True, help="Emit JSON output.") +def authenticate_user(**kwargs: Any) -> None: + """Invoke MXAccess AuthenticateUser — resolve a Galaxy user id (credential-sensitive).""" + + password = _resolve_password(kwargs) + kwargs["password"] = password + _run( + _authenticate_user(**kwargs), + output_json=kwargs["output_json"], + secrets=_secrets(kwargs) + [password], + ) + + @main.command("subscribe-bulk") @gateway_options @click.option("--session-id", required=True, help="Gateway session id.") @@ -745,19 +795,58 @@ async def _advise(**kwargs: Any) -> dict[str, Any]: async def _advise_supervisory(**kwargs: Any) -> dict[str, Any]: async with await _connect(kwargs) as client: session = _session(client, kwargs["session_id"]) - await session.invoke( - pb.MxCommand( - kind=pb.MX_COMMAND_KIND_ADVISE_SUPERVISORY, - advise_supervisory=pb.AdviseSupervisoryCommand( - server_handle=kwargs["server_handle"], - item_handle=kwargs["item_handle"], - ), - ), + await session.advise_supervisory( + kwargs["server_handle"], + kwargs["item_handle"], correlation_id=kwargs["correlation_id"], ) return {"ok": True} +async def _write_secured(**kwargs: Any) -> dict[str, Any]: + value = _parse_value(kwargs["value"], kwargs["value_type"]) + async with await _connect(kwargs) as client: + session = _session(client, kwargs["session_id"]) + await session.write_secured( + kwargs["server_handle"], + kwargs["item_handle"], + value, + current_user_id=kwargs["current_user_id"], + verifier_user_id=kwargs["verifier_user_id"], + correlation_id=kwargs["correlation_id"], + ) + return {"ok": True} + + +async def _authenticate_user(**kwargs: Any) -> dict[str, Any]: + async with await _connect(kwargs) as client: + session = _session(client, kwargs["session_id"]) + user_id = await session.authenticate_user( + kwargs["server_handle"], + kwargs["verify_user"], + kwargs["password"], + correlation_id=kwargs["correlation_id"], + ) + return {"userId": user_id} + + +def _resolve_password(kwargs: dict[str, Any]) -> str: + """Resolve the authenticate-user password from --password or --password-env. + + Prefers the explicit flag, then falls back to the named environment + variable. The resolved secret is never echoed; callers pass it into the + ``secrets`` redaction list so it cannot leak through a surfaced error. + """ + + password = kwargs.get("password") + if not password: + env_name = kwargs.get("password_env") + password = os.environ.get(env_name) if env_name else None + if not password: + raise click.UsageError("a password is required via --password or --password-env") + return password + + async def _subscribe_bulk(**kwargs: Any) -> dict[str, Any]: async with await _connect(kwargs) as client: session = _session(client, kwargs["session_id"]) diff --git a/clients/python/tests/test_cli.py b/clients/python/tests/test_cli.py index c0f34e6..856b83f 100644 --- a/clients/python/tests/test_cli.py +++ b/clients/python/tests/test_cli.py @@ -645,3 +645,175 @@ def test_galaxy_browse_help_shows_parent_gobject_id() -> None: assert result.exit_code == 0 assert "--parent-gobject-id" in result.output + + +class _FakeInvokeClient: + """Async-context-manager fake whose invoke_raw returns a scripted reply. + + Satisfies the session-backed CLI command bodies (register / write-secured / + authenticate-user) which build a Session over this client and call through to + ``invoke_raw``. Records the last command so tests can assert credentials are + carried on the wire but never echoed to stdout. + """ + + def __init__(self, reply) -> None: + self._reply = reply + self.last_request = None + + async def __aenter__(self) -> "_FakeInvokeClient": + return self + + async def __aexit__(self, *_exc: object) -> None: + return None + + async def invoke_raw(self, request): + self.last_request = request + return self._reply + + +def test_authenticate_user_command_returns_user_id_without_echoing_password( + monkeypatch: pytest.MonkeyPatch, +) -> None: + from zb_mom_ww_mxgateway.generated import mxaccess_gateway_pb2 as pb + + reply = pb.MxCommandReply( + session_id="s1", + kind=pb.MX_COMMAND_KIND_AUTHENTICATE_USER, + protocol_status=pb.ProtocolStatus(code=pb.PROTOCOL_STATUS_CODE_OK), + authenticate_user=pb.AuthenticateUserReply(user_id=42), + ) + fake = _FakeInvokeClient(reply) + + async def fake_connect(options, **_kwargs): + return fake + + monkeypatch.setattr(commands_module.GatewayClient, "connect", fake_connect) + + result = CliRunner().invoke( + main, + [ + "authenticate-user", + "--plaintext", + "--session-id", + "s1", + "--server-handle", + "3", + "--verify-user", + "operator", + "--password", + "cli-secret-pw", + "--json", + ], + ) + + assert result.exit_code == 0, result.output + assert json.loads(result.output)["userId"] == 42 + assert "cli-secret-pw" not in result.output + # Credential is carried on the wire, not echoed. + assert fake.last_request.command.authenticate_user.verify_user_password == "cli-secret-pw" + + +def test_authenticate_user_reads_password_from_env(monkeypatch: pytest.MonkeyPatch) -> None: + from zb_mom_ww_mxgateway.generated import mxaccess_gateway_pb2 as pb + + reply = pb.MxCommandReply( + session_id="s1", + kind=pb.MX_COMMAND_KIND_AUTHENTICATE_USER, + protocol_status=pb.ProtocolStatus(code=pb.PROTOCOL_STATUS_CODE_OK), + authenticate_user=pb.AuthenticateUserReply(user_id=7), + ) + fake = _FakeInvokeClient(reply) + + async def fake_connect(options, **_kwargs): + return fake + + monkeypatch.setattr(commands_module.GatewayClient, "connect", fake_connect) + monkeypatch.setenv("MXGW_TEST_PW", "env-secret-pw") + + result = CliRunner().invoke( + main, + [ + "authenticate-user", + "--plaintext", + "--session-id", + "s1", + "--server-handle", + "3", + "--verify-user", + "operator", + "--password-env", + "MXGW_TEST_PW", + "--json", + ], + ) + + assert result.exit_code == 0, result.output + assert "env-secret-pw" not in result.output + assert fake.last_request.command.authenticate_user.verify_user_password == "env-secret-pw" + + +def test_authenticate_user_requires_a_password() -> None: + result = CliRunner().invoke( + main, + [ + "authenticate-user", + "--plaintext", + "--session-id", + "s1", + "--server-handle", + "3", + "--verify-user", + "operator", + "--json", + ], + ) + + assert result.exit_code != 0 + assert "password is required" in result.output + + +def test_write_secured_command_does_not_echo_value_on_failure( + monkeypatch: pytest.MonkeyPatch, +) -> None: + from zb_mom_ww_mxgateway.generated import mxaccess_gateway_pb2 as pb + + reply = pb.MxCommandReply( + session_id="s1", + kind=pb.MX_COMMAND_KIND_WRITE_SECURED, + protocol_status=pb.ProtocolStatus( + code=pb.PROTOCOL_STATUS_CODE_MXACCESS_FAILURE, + message="WriteSecured rejected value cli-secret-value", + ), + hresult=-1, + ) + fake = _FakeInvokeClient(reply) + + async def fake_connect(options, **_kwargs): + return fake + + monkeypatch.setattr(commands_module.GatewayClient, "connect", fake_connect) + + result = CliRunner().invoke( + main, + [ + "write-secured", + "--plaintext", + "--session-id", + "s1", + "--server-handle", + "3", + "--item-handle", + "4", + "--value", + "cli-secret-value", + "--json", + ], + ) + + assert result.exit_code != 0 + assert "cli-secret-value" not in result.output + + +def test_write_secured_and_authenticate_user_commands_are_registered() -> None: + names = set(main.commands) + assert {"write-secured", "authenticate-user"} <= names diff --git a/clients/python/tests/test_typed_command_helpers.py b/clients/python/tests/test_typed_command_helpers.py new file mode 100644 index 0000000..c0f7124 --- /dev/null +++ b/clients/python/tests/test_typed_command_helpers.py @@ -0,0 +1,227 @@ +"""Tests for the typed single-item command helpers (CLI-04). + +Covers the parity-critical MXAccess commands promoted from raw ``Invoke`` to +typed async session helpers: ``advise_supervisory``, ``write_secured`` / +``write_secured2``, ``authenticate_user``, ``archestra_user_to_id``, and the +buffered/suspend/activate family. The credential-redaction contract for the +secured/auth helpers is asserted explicitly. +""" + +from __future__ import annotations + +from typing import Any + +import pytest + +from zb_mom_ww_mxgateway import ClientOptions, GatewayClient, MxAccessError +from zb_mom_ww_mxgateway.generated import mxaccess_gateway_pb2 as pb + + +class FakeUnary: + """Records requests and pops scripted replies, matching the client's call shape.""" + + def __init__(self, replies: list[Any]) -> None: + self.replies = replies + self.requests: list[Any] = [] + self.metadata: tuple[tuple[str, str], ...] | None = None + + async def __call__( + self, + request: Any, + *, + metadata: tuple[tuple[str, str], ...], + ) -> Any: + self.requests.append(request) + self.metadata = metadata + return self.replies.pop(0) + + +class FakeGatewayStub: + """Minimal stub: a fixed open-session reply plus a scriptable invoke queue.""" + + def __init__(self, invoke_replies: list[Any]) -> None: + self.open_session = FakeUnary( + [ + pb.OpenSessionReply( + session_id="session-1", + protocol_status=pb.ProtocolStatus(code=pb.PROTOCOL_STATUS_CODE_OK), + ), + ], + ) + self.invoke = FakeUnary(invoke_replies) + self.OpenSession = self.open_session + self.Invoke = self.invoke + + +async def _session_with(invoke_replies: list[Any]): + stub = FakeGatewayStub(invoke_replies) + client = await GatewayClient.connect( + ClientOptions(endpoint="fake", api_key="mxgw_test_secret", plaintext=True), + stub=stub, + ) + session = await client.open_session() + return session, stub + + +def _ok(kind: "pb.MxCommandKind.ValueType", **payload: Any) -> pb.MxCommandReply: + return pb.MxCommandReply( + session_id="session-1", + kind=kind, + protocol_status=pb.ProtocolStatus(code=pb.PROTOCOL_STATUS_CODE_OK), + **payload, + ) + + +@pytest.mark.asyncio +async def test_advise_supervisory_sends_typed_command() -> None: + session, stub = await _session_with([_ok(pb.MX_COMMAND_KIND_ADVISE_SUPERVISORY)]) + + await session.advise_supervisory(12, 34) + + command = stub.invoke.requests[0].command + assert command.kind == pb.MX_COMMAND_KIND_ADVISE_SUPERVISORY + assert command.advise_supervisory.server_handle == 12 + assert command.advise_supervisory.item_handle == 34 + + +@pytest.mark.asyncio +async def test_authenticate_user_returns_user_id_and_sends_credentials() -> None: + session, stub = await _session_with( + [_ok(pb.MX_COMMAND_KIND_AUTHENTICATE_USER, authenticate_user=pb.AuthenticateUserReply(user_id=77))], + ) + + user_id = await session.authenticate_user(12, "operator", "s3cr3t-pw") + + assert user_id == 77 + command = stub.invoke.requests[0].command + assert command.kind == pb.MX_COMMAND_KIND_AUTHENTICATE_USER + assert command.authenticate_user.verify_user == "operator" + # The credential is carried on the wire (redaction is about logs/errors, not the RPC). + assert command.authenticate_user.verify_user_password == "s3cr3t-pw" + + +@pytest.mark.asyncio +async def test_authenticate_user_scrubs_credential_from_surfaced_error() -> None: + password = "super-secret-pw" + failure = pb.MxCommandReply( + session_id="session-1", + kind=pb.MX_COMMAND_KIND_AUTHENTICATE_USER, + protocol_status=pb.ProtocolStatus( + code=pb.PROTOCOL_STATUS_CODE_MXACCESS_FAILURE, + # Simulate a gateway that unwisely echoed the credential back in the message. + message=f"authentication failed for password {password}", + ), + hresult=-1, + ) + session, _ = await _session_with([failure]) + + with pytest.raises(MxAccessError) as captured: + await session.authenticate_user(12, "operator", password) + + assert password not in str(captured.value) + assert "[redacted]" in str(captured.value) + + +@pytest.mark.asyncio +async def test_write_secured_surfaces_native_failure_without_prior_authenticate() -> None: + """Parity: WriteSecured failing before authenticate/advise-supervisory is surfaced as-is.""" + secret_value = "priv-payload" + failure = pb.MxCommandReply( + session_id="session-1", + kind=pb.MX_COMMAND_KIND_WRITE_SECURED, + protocol_status=pb.ProtocolStatus( + code=pb.PROTOCOL_STATUS_CODE_MXACCESS_FAILURE, + message=f"WriteSecured rejected value {secret_value}", + ), + hresult=-2147217407, + ) + session, stub = await _session_with([failure]) + + with pytest.raises(MxAccessError) as captured: + await session.write_secured(12, 34, secret_value, current_user_id=5, verifier_user_id=6) + + # Native failure is surfaced (not "fixed") and the raw reply is preserved... + assert captured.value.raw_reply is failure + # ...but the credential-sensitive value is scrubbed from the surfaced message. + assert secret_value not in str(captured.value) + command = stub.invoke.requests[0].command + assert command.kind == pb.MX_COMMAND_KIND_WRITE_SECURED + assert command.write_secured.current_user_id == 5 + assert command.write_secured.verifier_user_id == 6 + + +@pytest.mark.asyncio +async def test_write_secured2_sends_value_and_timestamp() -> None: + from datetime import datetime, timezone + + session, stub = await _session_with([_ok(pb.MX_COMMAND_KIND_WRITE_SECURED2)]) + + stamp = datetime(2026, 1, 2, 3, 4, 5, tzinfo=timezone.utc) + await session.write_secured2(12, 34, 42, stamp, current_user_id=5, verifier_user_id=6) + + command = stub.invoke.requests[0].command + assert command.kind == pb.MX_COMMAND_KIND_WRITE_SECURED2 + assert command.write_secured2.value.int32_value == 42 + assert command.write_secured2.HasField("timestamp_value") + + +@pytest.mark.asyncio +async def test_archestra_user_to_id_returns_user_id() -> None: + session, stub = await _session_with( + [_ok(pb.MX_COMMAND_KIND_ARCHESTRA_USER_TO_ID, archestra_user_to_id=pb.ArchestrAUserToIdReply(user_id=9))], + ) + + user_id = await session.archestra_user_to_id(12, "guid-123") + + assert user_id == 9 + assert stub.invoke.requests[0].command.archestra_user_to_id.user_id_guid == "guid-123" + + +@pytest.mark.asyncio +async def test_add_buffered_item_returns_item_handle() -> None: + session, stub = await _session_with( + [_ok(pb.MX_COMMAND_KIND_ADD_BUFFERED_ITEM, add_buffered_item=pb.AddBufferedItemReply(item_handle=55))], + ) + + item_handle = await session.add_buffered_item(12, "Object.Attribute", "ctx") + + assert item_handle == 55 + command = stub.invoke.requests[0].command + assert command.add_buffered_item.item_definition == "Object.Attribute" + assert command.add_buffered_item.item_context == "ctx" + + +@pytest.mark.asyncio +async def test_set_buffered_update_interval_sends_command() -> None: + session, stub = await _session_with([_ok(pb.MX_COMMAND_KIND_SET_BUFFERED_UPDATE_INTERVAL)]) + + await session.set_buffered_update_interval(12, 250) + + command = stub.invoke.requests[0].command + assert command.kind == pb.MX_COMMAND_KIND_SET_BUFFERED_UPDATE_INTERVAL + assert command.set_buffered_update_interval.update_interval_milliseconds == 250 + + +@pytest.mark.asyncio +async def test_suspend_and_activate_return_status() -> None: + session, _ = await _session_with( + [ + _ok( + pb.MX_COMMAND_KIND_SUSPEND, + suspend=pb.SuspendReply(status=pb.MxStatusProxy(category=pb.MX_STATUS_CATEGORY_OK)), + ), + ], + ) + status = await session.suspend(12, 34) + assert status.category == pb.MX_STATUS_CATEGORY_OK + + session, _ = await _session_with( + [ + _ok( + pb.MX_COMMAND_KIND_ACTIVATE, + activate=pb.ActivateReply(status=pb.MxStatusProxy(category=pb.MX_STATUS_CATEGORY_OK)), + ), + ], + ) + status = await session.activate(12, 34) + assert status.category == pb.MX_STATUS_CATEGORY_OK diff --git a/clients/rust/README.md b/clients/rust/README.md index e27f707..2f1b951 100644 --- a/clients/rust/README.md +++ b/clients/rust/README.md @@ -192,26 +192,36 @@ but still need the write attributed to a user id, you must first advise the item supervisory and then pass that user id on the write. Without the supervisory advise the `user_id` on a plain write is ignored. -The session exposes `advise`/`un_advise` but not supervisory advise, so send it -through the generic command channel: +The session exposes a typed `advise_supervisory` helper alongside +`advise`/`un_advise`: ```rust -session - .invoke( - MxCommandKind::AdviseSupervisory, - Payload::AdviseSupervisory(AdviseSupervisoryCommand { - server_handle, - item_handle, - }), - ) - .await?; - +session.advise_supervisory(server_handle, item_handle).await?; session.write(server_handle, item_handle, value, user_id).await?; ``` The CLI exposes the same command as `advise-supervisory`, and `write` / `write2` take `--user-id`. +### Verified / secured writes and user resolution + +The verified path has typed session helpers too: `authenticate_user` (returns +the resolved MXAccess user id), `archestra_user_to_id`, and +`write_secured` / `write_secured2`. MXAccess parity is preserved — a +`write_secured` issued before the required `authenticate_user` + +`advise_supervisory` (or before a value-bearing body) fails natively and the +failure surfaces as `Error::MxAccess`; it is not smoothed over. Credentials +passed to `authenticate_user` (and secured write payloads) are placed only on +the wire — the client never logs them and never embeds them in an `Error`'s +`Display`/`Debug`; the only error text that can surface (from `tonic::Status` +messages and reply diagnostics) is scrubbed by the credential-redaction seam. +The CLI mirrors these as `authenticate-user` (password via `--password` or the +`--password-env` env var, never echoed) and `write-secured`. + +The remaining single-item command helpers round out MXAccess parity: +`unregister`, `suspend` / `activate` (each returns the operation's +`MxStatus`), `add_buffered_item`, and `set_buffered_update_interval`. + ### Array writes replace the whole array A write to an array attribute **replaces the entire array**; it is not an diff --git a/clients/rust/crates/mxgw-cli/src/main.rs b/clients/rust/crates/mxgw-cli/src/main.rs index 8a84ffe..056d510 100644 --- a/clients/rust/crates/mxgw-cli/src/main.rs +++ b/clients/rust/crates/mxgw-cli/src/main.rs @@ -21,11 +21,10 @@ use serde_json::Value; use zb_mom_ww_mxgateway_client::galaxy::{BrowseChildrenOptions, LazyBrowseNode}; use zb_mom_ww_mxgateway_client::generated::galaxy_repository::v1::DeployEvent; use zb_mom_ww_mxgateway_client::generated::mxaccess_gateway::v1::{ - alarm_feed_message, AcknowledgeAlarmRequest, AdviseSupervisoryCommand, AlarmFeedMessage, - CloseSessionRequest, MxCommand, MxCommandKind, MxCommandRequest, MxEvent, MxEventFamily, - MxValue as ProtoMxValue, OpenSessionRequest, PingCommand, StreamAlarmsRequest, - StreamEventsRequest, Write2BulkEntry, WriteBulkEntry, WriteSecured2BulkEntry, - WriteSecuredBulkEntry, + alarm_feed_message, AcknowledgeAlarmRequest, AlarmFeedMessage, CloseSessionRequest, MxCommand, + MxCommandKind, MxCommandRequest, MxEvent, MxEventFamily, MxValue as ProtoMxValue, + OpenSessionRequest, PingCommand, StreamAlarmsRequest, StreamEventsRequest, Write2BulkEntry, + WriteBulkEntry, WriteSecured2BulkEntry, WriteSecuredBulkEntry, }; use zb_mom_ww_mxgateway_client::{ next_correlation_id, ApiKey, ClientOptions, Error, EventItem, GalaxyClient, GatewayClient, @@ -118,6 +117,67 @@ enum Command { #[arg(long)] json: bool, }, + /// Release a `ServerHandle` (and the items advised under it) via + /// MXAccess `Unregister`. + Unregister { + #[command(flatten)] + connection: ConnectionArgs, + #[arg(long)] + session_id: String, + #[arg(long)] + server_handle: i32, + #[arg(long)] + json: bool, + }, + /// Resolve an MXAccess user id from a credential via `AuthenticateUser`. + /// The password is read from `--password` or, if omitted, from the + /// environment variable named by `--password-env`; it is never echoed to + /// stdout/stderr. + AuthenticateUser { + #[command(flatten)] + connection: ConnectionArgs, + #[arg(long)] + session_id: String, + #[arg(long)] + server_handle: i32, + #[arg(long)] + verify_user: String, + /// Verifier password. Prefer `--password-env` so the secret never + /// appears in the process command line. + #[arg(long)] + password: Option, + /// Name of the environment variable holding the verifier password. + /// Used only when `--password` is not supplied. + #[arg(long, default_value = "MXGATEWAY_VERIFY_PASSWORD")] + password_env: String, + #[arg(long)] + json: bool, + }, + /// Single credential-verified write via MXAccess `WriteSecured`. + /// + /// Parity note: this fails natively unless the session has first run + /// `authenticate-user` + `advise-supervisory` and the item carries a + /// value-bearing body — the native failure is surfaced, not hidden. + WriteSecured { + #[command(flatten)] + connection: ConnectionArgs, + #[arg(long)] + session_id: String, + #[arg(long)] + server_handle: i32, + #[arg(long)] + item_handle: i32, + #[arg(long, value_enum)] + value_type: CliValueType, + #[arg(long)] + value: String, + #[arg(long, default_value_t = 0)] + current_user_id: i32, + #[arg(long, default_value_t = 0)] + verifier_user_id: i32, + #[arg(long)] + json: bool, + }, SubscribeBulk { #[command(flatten)] connection: ConnectionArgs, @@ -669,18 +729,69 @@ async fn dispatch(command: Command) -> Result<(), Error> { } => { let session = session_for(connection, session_id).await?; session - .invoke( - MxCommandKind::AdviseSupervisory, - zb_mom_ww_mxgateway_client::generated::mxaccess_gateway::v1::mx_command::Payload::AdviseSupervisory( - AdviseSupervisoryCommand { - server_handle, - item_handle, - }, - ), - ) + .advise_supervisory(server_handle, item_handle) .await?; print_ok("advise-supervisory", json); } + Command::Unregister { + connection, + session_id, + server_handle, + json, + } => { + let session = session_for(connection, session_id).await?; + session.unregister(server_handle).await?; + print_ok("unregister", json); + } + Command::AuthenticateUser { + connection, + session_id, + server_handle, + verify_user, + password, + password_env, + json, + } => { + // Resolve the credential from --password or the named env var. + // The password is passed straight to the typed helper and is never + // echoed to stdout/stderr or embedded in an error message. + let verify_user_password = password + .or_else(|| env::var(&password_env).ok()) + .ok_or_else(|| Error::InvalidArgument { + name: "password".to_owned(), + detail: format!( + "supply --password or set the environment variable `{password_env}`" + ), + })?; + let session = session_for(connection, session_id).await?; + let user_id = session + .authenticate_user(server_handle, &verify_user, &verify_user_password) + .await?; + print_handle("userId", user_id, json); + } + Command::WriteSecured { + connection, + session_id, + server_handle, + item_handle, + value_type, + value, + current_user_id, + verifier_user_id, + json, + } => { + let session = session_for(connection, session_id).await?; + session + .write_secured( + server_handle, + item_handle, + current_user_id, + verifier_user_id, + parse_value(value_type, &value)?, + ) + .await?; + print_ok("write-secured", json); + } Command::SubscribeBulk { connection, session_id, @@ -2489,6 +2600,59 @@ mod tests { assert_eq!(value["workerProtocolVersion"], 1); } + #[test] + fn parses_authenticate_user_command_with_password_env() { + let parsed = Cli::try_parse_from([ + "mxgw", + "authenticate-user", + "--session-id", + "session-1", + "--server-handle", + "7", + "--verify-user", + "verifier", + "--password-env", + "MY_PW_VAR", + ]); + assert!(parsed.is_ok(), "parse failed: {parsed:?}"); + } + + #[test] + fn parses_write_secured_command() { + let parsed = Cli::try_parse_from([ + "mxgw", + "write-secured", + "--session-id", + "session-1", + "--server-handle", + "12", + "--item-handle", + "34", + "--value-type", + "int32", + "--value", + "5", + "--current-user-id", + "1", + "--verifier-user-id", + "2", + ]); + assert!(parsed.is_ok(), "parse failed: {parsed:?}"); + } + + #[test] + fn parses_unregister_command() { + let parsed = Cli::try_parse_from([ + "mxgw", + "unregister", + "--session-id", + "session-1", + "--server-handle", + "12", + ]); + assert!(parsed.is_ok(), "parse failed: {parsed:?}"); + } + #[test] fn parses_stream_alarms_command() { let parsed = Cli::try_parse_from([ diff --git a/clients/rust/src/session.rs b/clients/rust/src/session.rs index 63f8189..621474a 100644 --- a/clients/rust/src/session.rs +++ b/clients/rust/src/session.rs @@ -15,16 +15,19 @@ use crate::error::{ensure_protocol_success, Error}; use crate::generated::mxaccess_gateway::v1::mx_command::Payload; use crate::generated::mxaccess_gateway::v1::mx_command_reply; use crate::generated::mxaccess_gateway::v1::{ - AddItem2Command, AddItemBulkCommand, AddItemCommand, AdviseCommand, AdviseItemBulkCommand, - BulkReadResult, BulkWriteResult, CloseSessionRequest, MxCommand, MxCommandKind, MxCommandReply, - MxCommandRequest, MxDataType, MxSparseArray, MxSparseElement, MxValue as ProtoMxValue, - OpenSessionRequest, ReadBulkCommand, RegisterCommand, RemoveItemBulkCommand, RemoveItemCommand, - StreamEventsRequest, SubscribeBulkCommand, SubscribeResult, UnAdviseCommand, - UnAdviseItemBulkCommand, UnsubscribeBulkCommand, Write2BulkCommand, Write2BulkEntry, - Write2Command, WriteBulkCommand, WriteBulkEntry, WriteCommand, WriteSecured2BulkCommand, - WriteSecured2BulkEntry, WriteSecuredBulkCommand, WriteSecuredBulkEntry, + ActivateCommand, AddBufferedItemCommand, AddItem2Command, AddItemBulkCommand, AddItemCommand, + AdviseCommand, AdviseItemBulkCommand, AdviseSupervisoryCommand, ArchestrAUserToIdCommand, + AuthenticateUserCommand, BulkReadResult, BulkWriteResult, CloseSessionRequest, MxCommand, + MxCommandKind, MxCommandReply, MxCommandRequest, MxDataType, MxSparseArray, MxSparseElement, + MxValue as ProtoMxValue, OpenSessionRequest, ReadBulkCommand, RegisterCommand, + RemoveItemBulkCommand, RemoveItemCommand, SetBufferedUpdateIntervalCommand, + StreamEventsRequest, SubscribeBulkCommand, SubscribeResult, SuspendCommand, UnAdviseCommand, + UnAdviseItemBulkCommand, UnregisterCommand, UnsubscribeBulkCommand, Write2BulkCommand, + Write2BulkEntry, Write2Command, WriteBulkCommand, WriteBulkEntry, WriteCommand, + WriteSecured2BulkCommand, WriteSecured2BulkEntry, WriteSecured2Command, + WriteSecuredBulkCommand, WriteSecuredBulkEntry, WriteSecuredCommand, }; -use crate::value::MxValue; +use crate::value::{MxStatus, MxValue}; const MAX_BULK_ITEMS: usize = 1_000; @@ -129,6 +132,25 @@ impl Session { register_server_handle(&reply) } + /// Run MXAccess `Unregister` to release the given `ServerHandle` and the + /// items advised under it. Mirrors [`Session::register`]; the worker + /// returns no payload, so the call resolves to `()` on success. + /// + /// # Errors + /// + /// Returns [`Error::Command`] if the worker reports a non-OK protocol + /// status and [`Error::MxAccess`] if MXAccess itself rejects the + /// unregister (negative `hresult` / non-success status), plus the usual + /// transport/status errors. + pub async fn unregister(&self, server_handle: i32) -> Result<(), Error> { + self.invoke( + MxCommandKind::Unregister, + Payload::Unregister(UnregisterCommand { server_handle }), + ) + .await?; + Ok(()) + } + /// Run MXAccess `AddItem` against `server_handle` and return the /// assigned `ItemHandle`. /// @@ -230,6 +252,133 @@ impl Session { Ok(()) } + /// Run MXAccess `AdviseSupervisory` to start supervisory-mode change + /// notifications for the given item. Mirrors [`Session::advise`]; the + /// worker returns no payload, so the call resolves to `()` on success. + /// + /// # Errors + /// + /// Returns [`Error::Command`] for a non-OK protocol status and + /// [`Error::MxAccess`] when MXAccess reports a negative `hresult` / + /// non-success status, plus the usual transport/status errors. + pub async fn advise_supervisory( + &self, + server_handle: i32, + item_handle: i32, + ) -> Result<(), Error> { + self.invoke( + MxCommandKind::AdviseSupervisory, + Payload::AdviseSupervisory(AdviseSupervisoryCommand { + server_handle, + item_handle, + }), + ) + .await?; + Ok(()) + } + + /// Run MXAccess `Suspend` on the given item and return the native + /// `MXSTATUS_PROXY` the worker reports for the operation. + /// + /// A top-level MXAccess failure (negative `hresult` or a non-success + /// top-level status) still surfaces as [`Error::MxAccess`] via the shared + /// reply validation; the returned [`MxStatus`] is the per-operation status + /// carried in the `SuspendReply` payload. + /// + /// # Errors + /// + /// Returns [`Error::Command`] for a non-OK protocol status, + /// [`Error::MxAccess`] on an MXAccess-level failure, and + /// [`Error::MalformedReply`] if the OK reply lacks the `Suspend` payload, + /// plus the usual transport/status errors. + pub async fn suspend(&self, server_handle: i32, item_handle: i32) -> Result { + let reply = self + .invoke( + MxCommandKind::Suspend, + Payload::Suspend(SuspendCommand { + server_handle, + item_handle, + }), + ) + .await?; + + suspend_status(reply) + } + + /// Run MXAccess `Activate` on the given item and return the native + /// `MXSTATUS_PROXY` the worker reports for the operation. See + /// [`Session::suspend`] for the status/error contract. + /// + /// # Errors + /// + /// Same conditions as [`Session::suspend`] (with the `Activate` payload). + pub async fn activate(&self, server_handle: i32, item_handle: i32) -> Result { + let reply = self + .invoke( + MxCommandKind::Activate, + Payload::Activate(ActivateCommand { + server_handle, + item_handle, + }), + ) + .await?; + + activate_status(reply) + } + + /// Run MXAccess `AddBufferedItem` against `server_handle` and return the + /// assigned `ItemHandle`. Mirrors [`Session::add_item2`] — the buffered + /// item carries a caller-supplied context string. + /// + /// # Errors + /// + /// Returns [`Error::Command`]/[`Error::MxAccess`] when the worker or + /// MXAccess rejects the item, [`Error::MalformedReply`] if the OK reply + /// lacks the item handle, plus the usual transport/status errors. + pub async fn add_buffered_item( + &self, + server_handle: i32, + item_definition: &str, + item_context: &str, + ) -> Result { + let reply = self + .invoke( + MxCommandKind::AddBufferedItem, + Payload::AddBufferedItem(AddBufferedItemCommand { + server_handle, + item_definition: item_definition.to_owned(), + item_context: item_context.to_owned(), + }), + ) + .await?; + + add_buffered_item_handle(&reply) + } + + /// Run MXAccess `SetBufferedUpdateInterval` for `server_handle`. The + /// worker returns no payload, so the call resolves to `()` on success. + /// + /// # Errors + /// + /// Returns [`Error::Command`]/[`Error::MxAccess`] on a non-OK protocol + /// status or an MXAccess-level failure, plus the usual transport/status + /// errors. + pub async fn set_buffered_update_interval( + &self, + server_handle: i32, + update_interval_milliseconds: i32, + ) -> Result<(), Error> { + self.invoke( + MxCommandKind::SetBufferedUpdateInterval, + Payload::SetBufferedUpdateInterval(SetBufferedUpdateIntervalCommand { + server_handle, + update_interval_milliseconds, + }), + ) + .await?; + Ok(()) + } + /// Bulk variant of [`Session::add_item`]. Each tag address yields one /// `SubscribeResult` in the returned vector. /// @@ -628,6 +777,142 @@ impl Session { Ok(()) } + /// Run MXAccess `WriteSecured` (single credential-verified write, no + /// caller-supplied timestamp). + /// + /// **MXAccess parity:** `WriteSecured` failing before a prior + /// [`Session::authenticate_user`] + [`Session::advise_supervisory`], or + /// before a value-bearing body, is the native contract, not a client bug — + /// the failure surfaces as [`Error::MxAccess`] (negative `hresult`) and is + /// **not** smoothed over. The `value` is credential-sensitive: it is placed + /// only in the wire command and is never logged or embedded in an error + /// message. + /// + /// # Errors + /// + /// Returns [`Error::Command`] for a non-OK protocol status, + /// [`Error::MxAccess`] when MXAccess rejects the secured write, plus the + /// usual transport/status errors. + pub async fn write_secured( + &self, + server_handle: i32, + item_handle: i32, + current_user_id: i32, + verifier_user_id: i32, + value: MxValue, + ) -> Result<(), Error> { + self.invoke( + MxCommandKind::WriteSecured, + Payload::WriteSecured(WriteSecuredCommand { + server_handle, + item_handle, + current_user_id, + verifier_user_id, + value: Some(value.into_proto()), + }), + ) + .await?; + Ok(()) + } + + /// Run MXAccess `WriteSecured2` (credential-verified write with a + /// caller-supplied timestamp). See [`Session::write_secured`] for the + /// parity and credential-handling contract. + /// + /// # Errors + /// + /// Same conditions as [`Session::write_secured`]. + pub async fn write_secured2( + &self, + server_handle: i32, + item_handle: i32, + current_user_id: i32, + verifier_user_id: i32, + value: MxValue, + timestamp_value: MxValue, + ) -> Result<(), Error> { + self.invoke( + MxCommandKind::WriteSecured2, + Payload::WriteSecured2(WriteSecured2Command { + server_handle, + item_handle, + current_user_id, + verifier_user_id, + value: Some(value.into_proto()), + timestamp_value: Some(timestamp_value.into_proto()), + }), + ) + .await?; + Ok(()) + } + + /// Run MXAccess `AuthenticateUser` and return the resolved MXAccess user + /// id. + /// + /// **Credential handling:** `verify_user_password` is a raw MXAccess + /// credential. It is placed only in the wire command; this helper never + /// logs it and never embeds it in an [`Error`] — the only error text that + /// can surface comes from `tonic::Status` messages and the reply's + /// diagnostic fields, both of which are scrubbed by the client's + /// credential-redaction seam (see [`crate::error`]). The reply itself + /// carries no echo of the credential. + /// + /// **MXAccess parity:** a failed authentication is a native outcome, not a + /// client error to paper over — it surfaces as [`Error::MxAccess`] + /// (negative `hresult`) with the credential absent from the message. + /// + /// # Errors + /// + /// Returns [`Error::Command`] for a non-OK protocol status, + /// [`Error::MxAccess`] when MXAccess rejects the credential, + /// [`Error::MalformedReply`] if the OK reply lacks the user id, plus the + /// usual transport/status errors. + pub async fn authenticate_user( + &self, + server_handle: i32, + verify_user: &str, + verify_user_password: &str, + ) -> Result { + let reply = self + .invoke( + MxCommandKind::AuthenticateUser, + Payload::AuthenticateUser(AuthenticateUserCommand { + server_handle, + verify_user: verify_user.to_owned(), + verify_user_password: verify_user_password.to_owned(), + }), + ) + .await?; + + authenticate_user_id(&reply) + } + + /// Run MXAccess `ArchestrAUserToId` to resolve an ArchestrA user GUID to + /// its MXAccess user id. + /// + /// # Errors + /// + /// Returns [`Error::Command`]/[`Error::MxAccess`] on a non-OK protocol + /// status or MXAccess-level failure, [`Error::MalformedReply`] if the OK + /// reply lacks the user id, plus the usual transport/status errors. + pub async fn archestra_user_to_id( + &self, + server_handle: i32, + user_id_guid: &str, + ) -> Result { + let reply = self + .invoke( + MxCommandKind::ArchestraUserToId, + Payload::ArchestraUserToId(ArchestrAUserToIdCommand { + server_handle, + user_id_guid: user_id_guid.to_owned(), + }), + ) + .await?; + + archestra_user_id(&reply) + } + /// Open the per-session event stream from the beginning. /// /// The returned [`EventStream`] yields [`EventItem`](crate::EventItem) @@ -769,6 +1054,69 @@ fn add_item2_handle(reply: &MxCommandReply) -> Result { } } +fn add_buffered_item_handle(reply: &MxCommandReply) -> Result { + match reply.payload.as_ref() { + Some(mx_command_reply::Payload::AddBufferedItem(add_buffered)) => { + Ok(add_buffered.item_handle) + } + _ => reply + .return_value + .as_ref() + .and_then(int32_reply_value) + .ok_or_else(|| Error::MalformedReply { + detail: + "add_buffered_item reply lacked an item_handle payload or int32 return_value" + .to_owned(), + }), + } +} + +fn authenticate_user_id(reply: &MxCommandReply) -> Result { + match reply.payload.as_ref() { + Some(mx_command_reply::Payload::AuthenticateUser(authenticate)) => Ok(authenticate.user_id), + _ => Err(Error::MalformedReply { + detail: "authenticate_user reply lacked an AuthenticateUser payload".to_owned(), + }), + } +} + +fn archestra_user_id(reply: &MxCommandReply) -> Result { + match reply.payload.as_ref() { + Some(mx_command_reply::Payload::ArchestraUserToId(archestra)) => Ok(archestra.user_id), + _ => Err(Error::MalformedReply { + detail: "archestra_user_to_id reply lacked an ArchestraUserToId payload".to_owned(), + }), + } +} + +fn suspend_status(reply: MxCommandReply) -> Result { + match reply.payload { + Some(mx_command_reply::Payload::Suspend(suspend)) => suspend + .status + .map(MxStatus::from_proto) + .ok_or_else(|| Error::MalformedReply { + detail: "suspend reply payload lacked a status entry".to_owned(), + }), + _ => Err(Error::MalformedReply { + detail: "suspend reply did not carry a Suspend payload".to_owned(), + }), + } +} + +fn activate_status(reply: MxCommandReply) -> Result { + match reply.payload { + Some(mx_command_reply::Payload::Activate(activate)) => activate + .status + .map(MxStatus::from_proto) + .ok_or_else(|| Error::MalformedReply { + detail: "activate reply payload lacked a status entry".to_owned(), + }), + _ => Err(Error::MalformedReply { + detail: "activate reply did not carry an Activate payload".to_owned(), + }), + } +} + enum BulkReplyKind { AddItem, AdviseItem, diff --git a/clients/rust/tests/client_behavior.rs b/clients/rust/tests/client_behavior.rs index 477fa81..faf7fba 100644 --- a/clients/rust/tests/client_behavior.rs +++ b/clients/rust/tests/client_behavior.rs @@ -23,10 +23,11 @@ use zb_mom_ww_mxgateway_client::generated::mxaccess_gateway::v1::mx_value::Kind; use zb_mom_ww_mxgateway_client::generated::mxaccess_gateway::v1::{ alarm_feed_message, AcknowledgeAlarmReply, AcknowledgeAlarmRequest, ActiveAlarmSnapshot, AddItem2Reply, AddItemReply, AlarmConditionState, AlarmFeedMessage, AlarmTransitionKind, - BulkReadReply, BulkReadResult, BulkSubscribeReply, BulkWriteReply, BulkWriteResult, - CloseSessionReply, CloseSessionRequest, MxCommandKind, MxCommandReply, MxDataType, MxEvent, - MxEventFamily, MxSparseArray, MxSparseElement, MxStatusCategory, MxStatusProxy, MxStatusSource, - MxValue, OnAlarmTransitionEvent, OpenSessionReply, OpenSessionRequest, ProtocolStatus, + AuthenticateUserCommand, AuthenticateUserReply, BulkReadReply, BulkReadResult, + BulkSubscribeReply, BulkWriteReply, BulkWriteResult, CloseSessionReply, CloseSessionRequest, + MxCommandKind, MxCommandReply, MxDataType, MxEvent, MxEventFamily, MxSparseArray, + MxSparseElement, MxStatusCategory, MxStatusProxy, MxStatusSource, MxValue, + OnAlarmTransitionEvent, OpenSessionReply, OpenSessionRequest, ProtocolStatus, ProtocolStatusCode, QueryActiveAlarmsRequest, RegisterReply, ReplayGap, SessionState, StreamAlarmsRequest, StreamEventsRequest, SubscribeResult, Write2BulkEntry, WriteBulkEntry, WriteCommand, WriteSecured2BulkEntry, WriteSecuredBulkEntry, @@ -624,6 +625,133 @@ async fn write_secured2_bulk_round_trips_through_the_fake_gateway() { assert_eq!(*last_command, Some(MxCommandKind::WriteSecured2Bulk as i32)); } +#[tokio::test] +async fn advise_supervisory_round_trips_and_sends_advise_supervisory_kind() { + let state = Arc::new(FakeState::default()); + let endpoint = spawn_fake_gateway(state.clone()).await; + let client = GatewayClient::connect(ClientOptions::new(endpoint)) + .await + .unwrap(); + let session = client.session("session-fixture"); + + session.advise_supervisory(12, 34).await.unwrap(); + + let last_command = state.last_command_kind.lock().await; + assert_eq!(*last_command, Some(MxCommandKind::AdviseSupervisory as i32)); +} + +#[tokio::test] +async fn unregister_round_trips_and_sends_unregister_kind() { + let state = Arc::new(FakeState::default()); + let endpoint = spawn_fake_gateway(state.clone()).await; + let client = GatewayClient::connect(ClientOptions::new(endpoint)) + .await + .unwrap(); + let session = client.session("session-fixture"); + + session.unregister(12).await.unwrap(); + + let last_command = state.last_command_kind.lock().await; + assert_eq!(*last_command, Some(MxCommandKind::Unregister as i32)); +} + +#[tokio::test] +async fn write_secured_surfaces_native_mxaccess_failure_and_redacts_diagnostic() { + // MXAccess parity: WriteSecured failing (e.g. before authenticate + + // advise-supervisory) is a native outcome, surfaced — not smoothed over. + // The scripted reply carries an Ok protocol envelope but a negative + // hresult, so this also proves the typed helper runs ensure_mxaccess_success. + let state = Arc::new(FakeState::default()); + *state.invoke_override.lock().await = Some(InvokeOverride::MxAccessFailure); + let endpoint = spawn_fake_gateway(state.clone()).await; + let client = GatewayClient::connect(ClientOptions::new(endpoint)) + .await + .unwrap(); + let session = client.session("session-fixture"); + + let error = session + .write_secured(12, 34, 0, 0, ClientMxValue::int32(1)) + .await + .unwrap_err(); + + let Error::MxAccess(mx_access) = &error else { + panic!("write_secured must surface the native failure as Error::MxAccess: {error:?}"); + }; + assert_eq!(mx_access.reply().hresult, Some(-2_147_217_900)); + let rendered = error.to_string(); + assert!(rendered.contains(""), "diagnostic: {rendered}"); + assert!( + !rendered.contains("leaked_secret"), + "credential-shaped diagnostic must be scrubbed: {rendered}" + ); + + let last_command = state.last_command_kind.lock().await; + assert_eq!(*last_command, Some(MxCommandKind::WriteSecured as i32)); +} + +#[tokio::test] +async fn authenticate_user_returns_user_id_and_transmits_credential_on_wire() { + let state = Arc::new(FakeState::default()); + let endpoint = spawn_fake_gateway(state.clone()).await; + let client = GatewayClient::connect(ClientOptions::new(endpoint)) + .await + .unwrap(); + let session = client.session("session-fixture"); + + let user_id = session + .authenticate_user(7, "verifier", "sup3r-s3cret-pw") + .await + .unwrap(); + + assert_eq!(user_id, 4242); + let captured = state + .last_authenticate_user + .lock() + .await + .take() + .expect("fake should have captured an AuthenticateUserCommand"); + assert_eq!(captured.server_handle, 7); + assert_eq!(captured.verify_user, "verifier"); + // The credential must reach the wire so authentication can succeed... + assert_eq!(captured.verify_user_password, "sup3r-s3cret-pw"); + let last_command = state.last_command_kind.lock().await; + assert_eq!(*last_command, Some(MxCommandKind::AuthenticateUser as i32)); +} + +#[tokio::test] +async fn authenticate_user_keeps_credentials_out_of_surfaced_errors() { + // ...but a native authentication failure must never leak the credential + // into the error's Display or Debug rendering. + let state = Arc::new(FakeState::default()); + *state.invoke_override.lock().await = Some(InvokeOverride::MxAccessFailure); + let endpoint = spawn_fake_gateway(state.clone()).await; + let client = GatewayClient::connect(ClientOptions::new(endpoint)) + .await + .unwrap(); + let session = client.session("session-fixture"); + + let password = "unique-credential-9f3b2"; + let error = session + .authenticate_user(7, "verifier", password) + .await + .unwrap_err(); + + assert!( + matches!(error, Error::MxAccess(_)), + "native auth failure should surface as Error::MxAccess: {error:?}" + ); + let display = error.to_string(); + let debug = format!("{error:?}"); + assert!( + !display.contains(password), + "credential leaked into Display: {display}" + ); + assert!( + !debug.contains(password), + "credential leaked into Debug: {debug}" + ); +} + #[tokio::test] async fn stream_alarms_emits_snapshot_then_complete_then_transition_in_order() { let state = Arc::new(FakeState::default()); @@ -732,6 +860,10 @@ struct FakeState { /// Captures the last `WriteCommand` payload received, populated when the /// `WriteOk` override is active. Used by `write_array_elements` e2e test. last_write_command: Mutex>, + /// Captures the last `AuthenticateUserCommand` payload received, populated + /// by the `AuthenticateUser` happy-path handler so a test can confirm the + /// credential reaches the wire (but never a surfaced error). + last_authenticate_user: Mutex>, stream_dropped: Arc, /// Optional per-test override that pins the fake's `Invoke` handler to /// a specific reply shape (or `Err(Status)`). The default of `None` @@ -765,6 +897,12 @@ enum InvokeOverride { /// and capture the decoded `WriteCommand` in /// `FakeState::last_write_command` for inspection. WriteOk, + /// Reply with an `Ok` protocol envelope but a negative `hresult` and a + /// non-success status entry carrying a credential-shaped diagnostic. This + /// mimics the worker's COMException path (e.g. `WriteSecured` / + /// `AuthenticateUser` rejected by MXAccess) so the client's + /// `ensure_mxaccess_success` check is exercised on the typed helper path. + MxAccessFailure, } #[derive(Clone)] @@ -846,6 +984,27 @@ impl MxAccessGateway for FakeGateway { ..MxCommandReply::default() })), InvokeOverride::Unavailable(message) => Err(Status::unavailable(message)), + InvokeOverride::MxAccessFailure => Ok(Response::new(MxCommandReply { + session_id: request.session_id, + correlation_id: "fake-correlation".to_owned(), + kind, + // Protocol envelope succeeds; MXAccess itself failed. + protocol_status: Some(ok_status("command ok")), + // 0x80040E14 (a COM failure) as a signed 32-bit value. + hresult: Some(-2_147_217_900), + statuses: vec![MxStatusProxy { + success: 0, + category: MxStatusCategory::SecurityError as i32, + detected_by: MxStatusSource::RespondingLmx as i32, + detail: 123, + // A credential-shaped token that must be scrubbed from + // any surfaced diagnostic text. + diagnostic_text: "denied for mxgw_leaked_secret".to_owned(), + ..MxStatusProxy::default() + }], + payload: None, + ..MxCommandReply::default() + })), InvokeOverride::WriteOk => { // Extract and capture the WriteCommand payload so the test // can assert on server_handle, item_handle, user_id, and value. @@ -977,6 +1136,26 @@ impl MxAccessGateway for FakeGateway { ))); } + if kind == MxCommandKind::AuthenticateUser as i32 { + // Capture the transmitted command so a test can confirm the + // credential reaches the wire but never an error message. + if let Some(mx_command::Payload::AuthenticateUser(auth)) = + request.command.and_then(|command| command.payload) + { + *self.state.last_authenticate_user.lock().await = Some(auth); + } + return Ok(Response::new(MxCommandReply { + session_id: request.session_id, + correlation_id: "fake-correlation".to_owned(), + kind, + protocol_status: Some(ok_status("command ok")), + payload: Some(mx_command_reply::Payload::AuthenticateUser( + AuthenticateUserReply { user_id: 4242 }, + )), + ..MxCommandReply::default() + })); + } + Ok(Response::new(MxCommandReply { session_id: request.session_id, correlation_id: "fake-correlation".to_owned(), diff --git a/docs/ClientLibrariesDesign.md b/docs/ClientLibrariesDesign.md index d6f9326..a616291 100644 --- a/docs/ClientLibrariesDesign.md +++ b/docs/ClientLibrariesDesign.md @@ -104,6 +104,29 @@ All languages should expose the same core concepts, using idiomatic naming: The gateway session id and MXAccess handles must remain visible. The library may offer helper methods, but it must not invent alternate handle semantics. +## Typed Command Parity + +Every command kind in the wire contract has a typed single-item session helper, +not just a raw-`Invoke` escape hatch (CLI-04). Beyond the register/add/advise/ +remove/write family, the parity-critical single-item helpers are: +`AdviseSupervisory`, `WriteSecured` / `WriteSecured2`, `AuthenticateUser`, +`ArchestrAUserToId`, `AddBufferedItem`, `SetBufferedUpdateInterval`, `Suspend`, +`Activate`, and `Unregister`. Each is a thin wrapper over the same raw-command +machinery the bulk helpers use — it adds no wire surface — and runs the same +MXAccess-level reply validation (HRESULT `< 0` + per-item `MxStatusProxy`) as the +rest of the client. **MXAccess parity is preserved exactly**: e.g. `WriteSecured` +failing before a prior `AuthenticateUser` + `AdviseSupervisory` surfaces the +native failure unchanged — the helper does not pre-validate or reorder it. + +**Credential handling:** `AuthenticateUser` credentials and `WriteSecured` +secured payloads route through each client's secret-redaction seam so they never +reach logs, exception text, or `ToString`/`Debug`/`Display` — the value is carried +only on the wire. Each client's test suite asserts a distinctive credential is +absent from any surfaced error. + +Shipped for .NET / Go / Rust / Python; the Java client's typed parity helpers are +batched to the windows build host (no local JRE on the primary dev box). + ## Shared API Shape Each language should support this conceptual API: -- 2.52.0 From 908951be9dad7552808a24bcf5f3d651d15a04ce Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Thu, 9 Jul 2026 16:44:24 -0400 Subject: [PATCH 22/25] docs(archreview): E2 CLI-04 In progress (4/5) + CLI-30 Done; Java now local Java client toolchain (homebrew openjdk@17) works on the Mac, so the remaining Java halves of CLI-15/CLI-04 are done locally this session, not batched to windev. Claude-Session: https://claude.ai/code/session_01DMXXvNuPekkkrTEyPNxEkW --- archreview/remediation/00-tracking.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/archreview/remediation/00-tracking.md b/archreview/remediation/00-tracking.md index c53b954..5227d5d 100644 --- a/archreview/remediation/00-tracking.md +++ b/archreview/remediation/00-tracking.md @@ -177,7 +177,7 @@ Full design + implementation for each row lives in the linked domain doc under i | CLI-01 | High | P0 | M | — | Done | Go `Session.Events()` silently closes stream on 16-slot overflow | | CLI-02 | High | P1 | M | — | Done | Rust crate unbuildable outside the repo (`build.rs` path + `--no-verify`) | | CLI-03 | High | P0 | M | — | Done | Rust `invoke` never validates HRESULT / MXSTATUS_PROXY | -| CLI-04 | High | P2 | L | CLI-15 | Not started | Typed-command parity gap across all five clients | +| CLI-04 | High | P2 | L | CLI-15 | In progress | Typed-command parity gap across all five clients | | CLI-05 | Medium | — | S | — | Not started | .NET session cannot be re-attached to an existing session id | | CLI-06 | Medium | — | S | — | Not started | .NET `DisposeAsync` blocks/throws on unreachable gateway | | CLI-07 | Medium | — | S | — | Not started | .NET retry budget self-defeats on `DeadlineExceeded` | @@ -203,7 +203,7 @@ Full design + implementation for each row lives in the linked domain doc under i | CLI-27 | Low | — | S | — | Not started | Python `Session.close()` not concurrency-safe; synthesizes reply | | CLI-28 | Low | — | S | — | Not started | Python circular-import workaround at end of `session.py` | | CLI-29 | Low | P2 | S | — | Done | Rust `CLIENT_VERSION` (0.1.0-dev) ≠ `Cargo.toml` (0.1.2) | -| CLI-30 | Low | — | S | CLI-04 | Not started | Rust has no `unregister` typed helper | +| CLI-30 | Low | — | S | CLI-04 | Done | Rust has no `unregister` typed helper | | CLI-31 | Low | — | M | — | Not started | Rust CLI is a single 2,699-line `main.rs` | | CLI-32 | Low | — | S | — | Not started | Client-side bulk caps differ (.NET/Java unbounded) | | CLI-33 | Low | — | S | CLI-01,13 | Not started | Per-language event backpressure semantics undocumented | @@ -253,6 +253,8 @@ Findings the review flagged as one coordinated design pass — sequence them tog | Date | Change | |---|---| +| 2026-07-09 | P2 Epic Wave E2 — typed-command parity (commit `bde042b`). CLI-04 → `In progress` (4/5 clients; Java pending in the SAME session — see next note), CLI-30 → `Done`. Every parity-critical single-item command now has a typed session helper across .NET/Go/Rust/Python: Phase 1 `AdviseSupervisory`/`WriteSecured`/`WriteSecured2`/`AuthenticateUser`/`ArchestrAUserToId`, Phase 2 `AddBufferedItem`/`SetBufferedUpdateInterval`/`Suspend`/`Activate`, plus `Unregister` (CLI-30: Rust + .NET added; Go/Java/Python already had it — CLI-30 fully Done). Each wraps existing raw-command machinery (no new wire surface) and runs the client's MXAccess-level reply validation (`hresult < 0` + `MxStatusProxy`). MXAccess parity preserved (WriteSecured-before-authenticate surfaces the native failure). **Credentials route through each client's secret-redaction seam** (never in logs/exceptions/ToString/Debug/Display; each suite asserts a distinctive credential is absent from surfaced errors); new CLI subcommands take credentials via flag/env, never echoed. Verified per toolchain: .NET build clean 102 passed; Go gofmt/vet/build/test clean; Rust fmt/check/test/clippy clean; Python 145 passed. Shared doc: `ClientLibrariesDesign.md` "Typed Command Parity" section. | +| 2026-07-09 | **Java client toolchain now works locally on the Mac** (user-flagged + verified): homebrew `openjdk@17` (17.0.19), @21, @26 are installed (off PATH); `JAVA_HOME=/opt/homebrew/opt/openjdk@17 gradle test` → BUILD SUCCESSFUL. Retires the "Java client must be built on windev" constraint — the remaining Java halves of CLI-15 + CLI-04 are done locally this session (no windev batch). Memory `project_java_build_host` rewritten. | | 2026-07-09 | P2 Epic — TST-04 (session-resilience epic governance) → `Done` (commit `ed3c6c6`, docs-only umbrella). Resolved the epic's shipped-vs-planned entanglement into three decisions: **Phase 3 finished** (Task 13=TST-02 owner-scoped attach P0, Task 15=TST-01 reconnect e2e, Task 14=CLI-15 4/5 clients w/ Java pending); **Phase 4 scoped** as TST-15 with the Viewer-default decision settled (admin-sees-all, Viewer strict per owned/granted session, matching TST-02's gRPC owner binding); **Phase 5 (orphan-worker reattach) DEFERRED, not planned** — the CLAUDE.md "gateway restart does not reattach orphan workers" invariant stands and the `EnableOrphanReattach` flag does not exist / must not be referenced. Updated `oldtasks.md`, the `tasks.json` mirror (per-task statuses + governance note), and the CLAUDE.md reconnect paragraph (clients consume ReplayGap; reattach deferred). The actionable slices carry their own verification: TST-01 (done), TST-02 (done, P0), TST-15 (pending, E3). | | 2026-07-09 | P2 Epic Wave E1 — reconnect-replay: CLI-15 + TST-01 → `In progress` (both gated on the Java client, deferred to the windev batch). **CLI-15 (commit `0c6e5b3`)**: four clients now surface the gateway's `ReplayGap` reconnect sentinel as a distinct, typed, non-terminal signal (never synthesized, never swallowed) — .NET `MxEventStreamItem`/`StreamEventItemsAsync` (build clean, 87 passed), Go `EventResult.ReplayGap`+`IsReplayGap()` (build/vet/test clean), Rust `EventItem::ReplayGap` enum (`EventStream` now yields `Result`; fmt/check/test/clippy clean), Python `ReplayGap` dataclass yielded from `stream_events` (131 passed). Shared docs: `ClientLibrariesDesign.md` non-goals reframed (reconnect-replay protocol is consumable; auto-reconnect stays a non-goal) + `CrossLanguageSmokeMatrix.md` resume-gap note. Resume contract uniform: `after_worker_sequence = oldest_available_sequence - 1`. **TST-01 server half (commit `fed0685`)**: `GatewayEndToEndReconnectReplayTests` (2 facts) proves the default-on reconnect protocol end-to-end over the real gRPC StreamEvents path via the fake worker — replay-tail-no-gap (capacity 16, mid-batch cursor) and stale-cursor-emits-ReplayGap-first (capacity 3, 6 events force eviction, cursor=1); single-subscriber detach→reconnect, ring survives detach. macOS-verified 2/2 (needs `TMPDIR=/tmp` — default macOS TMPDIR pushes the CoreFxPipe Unix-socket path past the 104-byte `sun_path` limit; Windows CI unaffected). No product bugs; server behavior matched the contract exactly. Remaining to close both: Java `ReplayGap` surface (windev) + Java client consumer for TST-01. | | 2026-07-09 | P2 Wave B — worker event hot-path (commit `f61c816`), **windev-verified**. WRK-06 → `Done`: `MxStatusProxyConverter` caches the four resolved `FieldInfo` per status type in a static `ConcurrentDictionary` (the `GetField` metadata scan ran 4× per status per event on the STA path); `GetValue`+`Convert.ToInt32` still run per event (late-bound RCW); both exception messages byte-identical (missing-field via `ResolveField`, not cached on throw through `GetOrAdd`; null-value unchanged). WRK-11 → `Done`: `MxAccessEventQueue.Enqueue` takes ownership of the passed `MxEvent` (stamps `WorkerSequence`/`WorkerTimestamp` in place, no `Clone()`); audited all 3 callers (base/alarm event sinks + provider-mode handler) — each builds a fresh event, none reuse it; `MxAccessValueCache.Set` now deep-copies its retained `Value`/`SourceTimestamp`/`Statuses` so the cache snapshot never aliases the queue-owned (later serialized) event. WRK-12 → `Done`: `WorkerFrameWriter` coalesces the flush across a drained batch (write each frame, one `FlushAsync` after the batch, then complete all) — preserves the written+flushed completion contract; a burst of N events costs 1 flush not N; on failure the in-flight batch + queue all fail so no caller hangs. IPC-15 → `Done`: the multi-event `WorkerEnvelope` body stays unimplemented (wire still one event per `worker_event` frame); gateway.md Performance section now distinguishes the shipped flush-coalescing from that deferred additive-proto change. **Windev:** x86 Worker builds clean (fixed a Windows-only `TreatWarningsAsErrors` CS8604 in the value-cache null-guard that macOS can't surface); Worker.Tests **356 passed / 0 failed / 11 skipped** (+4 new: converter cache-reuse, queue ownership-transfer, value-cache snapshot independence, writer batch-flush-once); gateway Tests **815 passed / 1 failed** — the 1 is the known windev-environmental SelfSigned-SAN assertion (passes on macOS), and all pipe-harness tests (which also exercise the earlier G-frame gateway frame change + WRK-12 end-to-end over a real pipe) pass. | -- 2.52.0 From 1cc0fa4007007a426e8b1f2c3963271cfb59421b Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Thu, 9 Jul 2026 16:57:13 -0400 Subject: [PATCH 23/25] feat(java-client): CLI-15 ReplayGap surface + CLI-04 typed-command parity (5/5 complete) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Completes both findings across all five clients — done locally on the Mac now that homebrew openjdk@17 is available (JAVA_HOME=/opt/homebrew/opt/openjdk@17). CLI-15: new MxEventStreamItem record + MxEventStream.nextItem() surfaces the gateway's ReplayGap sentinel as a typed, non-terminal signal (isReplayGap()/ replayGap()/event()); existing Iterator path unchanged, sentinel never swallowed/synthesized. Javadoc covers gap semantics + resume contract. CLI-04: typed single-item helpers on MxGatewaySession — Phase 1 adviseSupervisory/writeSecured/writeSecured2/authenticateUser/archestrAUserToId, Phase 2 addBufferedItem/setBufferedUpdateInterval/suspend/activate (unregister already present). Each routes through invokeCommand -> ensureProtocolSuccess + ensureMxAccessSuccess (same validation as bulk). MXAccess parity preserved. Credentials flow only into the request proto; exceptions carry only the reply and gRPC status text is scrubbed via MxGatewaySecrets.redactCredentials — tests assert the password/secured value is absent from getMessage()/toString()/CLI output. New CLI subcommands write-secured/authenticate-user (credential via --password/--password-env, prints only the user id). gradle test: 106 tests, 0 failures (58 client + 48 cli); no generated churn. Shared docs: ClientLibrariesDesign + CLAUDE.md updated to "all five clients". Claude-Session: https://claude.ai/code/session_01DMXXvNuPekkkrTEyPNxEkW --- CLAUDE.md | 2 +- clients/java/README.md | 88 +++++- .../zb/mom/ww/mxgateway/cli/MxGatewayCli.java | 126 +++++++- .../ww/mxgateway/cli/MxGatewayCliTests.java | 67 ++++ .../ww/mxgateway/client/MxEventStream.java | 36 +++ .../mxgateway/client/MxEventStreamItem.java | 71 +++++ .../ww/mxgateway/client/MxGatewaySession.java | 289 ++++++++++++++++++ .../client/MxGatewayClientSessionTests.java | 230 ++++++++++++++ docs/ClientLibrariesDesign.md | 8 +- 9 files changed, 890 insertions(+), 27 deletions(-) create mode 100644 clients/java/zb-mom-ww-mxgateway-client/src/main/java/com/zb/mom/ww/mxgateway/client/MxEventStreamItem.java diff --git a/CLAUDE.md b/CLAUDE.md index a1ddf64..bf44902 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -76,7 +76,7 @@ powershell -ExecutionPolicy Bypass -File scripts/run-client-e2e-tests.ps1 - **Style guides** in `docs/style-guides/` are authoritative. Follow `CSharpStyleGuide.md` for gateway/worker/.NET-client code: file-scoped namespaces, `sealed` by default, `Async` suffix on Task-returning methods, MXAccess-aligned names (`MxStatusProxy`, `ServerHandle`, `ItemHandle`, `HResult`). - **MXAccess parity is the contract.** Don't "fix" surprising MXAccess behavior (e.g., `WriteSecured` failing before a value-bearing NMX body, distinct `OperationComplete` semantics, invalid-handle exceptions) unless the client explicitly opts into a non-parity mode. The installed MXAccess COM component is the baseline. - **Don't synthesize events.** The gateway forwards only events the worker emits; it never invents `OperationComplete` from write completion or command replies. -- **One worker per session** (invariant). Multi-subscriber event fan-out and reconnect-with-replay have shipped and are config-gated: `AllowMultipleEventSubscribers` (default `false`) enables fan-out up to `MaxEventSubscribersPerSession` (default `8`); `DetachGraceSeconds` (default `30`) retains a session after its last subscriber drops so clients can reconnect; `ReplayBufferCapacity` / `ReplayRetentionSeconds` control how much event history the replay ring keeps. Default config is single-subscriber (`AllowMultipleEventSubscribers` off), but detach-grace and replay retention are **on** by default (`DetachGraceSeconds=30`, `ReplayBufferCapacity=1024`, `ReplayRetentionSeconds=300`): a detached session is retained for 30 s and recent events are buffered for reconnect. The reconnect protocol is consumable end-to-end: a resuming `StreamEvents` (via `after_worker_sequence`) that predates the retained ring gets a `ReplayGap` sentinel, and the official clients surface it as a typed signal (shipped for .NET/Go/Rust/Python; the Java client is the remaining one, batched to windev). Orphan-worker reattach after a gateway restart is **deferred, not planned** — see `oldtasks.md` (session-resilience epic Phase 5); the invariant on the next line stands. See `docs/DesignDecisions.md` and `docs/Sessions.md`. +- **One worker per session** (invariant). Multi-subscriber event fan-out and reconnect-with-replay have shipped and are config-gated: `AllowMultipleEventSubscribers` (default `false`) enables fan-out up to `MaxEventSubscribersPerSession` (default `8`); `DetachGraceSeconds` (default `30`) retains a session after its last subscriber drops so clients can reconnect; `ReplayBufferCapacity` / `ReplayRetentionSeconds` control how much event history the replay ring keeps. Default config is single-subscriber (`AllowMultipleEventSubscribers` off), but detach-grace and replay retention are **on** by default (`DetachGraceSeconds=30`, `ReplayBufferCapacity=1024`, `ReplayRetentionSeconds=300`): a detached session is retained for 30 s and recent events are buffered for reconnect. The reconnect protocol is consumable end-to-end: a resuming `StreamEvents` (via `after_worker_sequence`) that predates the retained ring gets a `ReplayGap` sentinel, and all five official clients surface it as a typed signal. Orphan-worker reattach after a gateway restart is **deferred, not planned** — see `oldtasks.md` (session-resilience epic Phase 5); the invariant on the next line stands. See `docs/DesignDecisions.md` and `docs/Sessions.md`. - **Gateway restart does not reattach orphan workers.** The first version terminates orphaned workers on startup; do not design code paths that assume reattachment. - **No Blazor UI component libraries.** Dashboard uses local Bootstrap CSS/JS only — do not introduce MudBlazor, Radzen, FluentUI, etc. - **Don't log secrets or full tag values by default.** API keys, passwords, `WriteSecured` payloads, and `AuthenticateUser` credentials must never reach logs. Value logging is opt-in and redacted. diff --git a/clients/java/README.md b/clients/java/README.md index d7bead6..2e0b117 100644 --- a/clients/java/README.md +++ b/clients/java/README.md @@ -76,7 +76,40 @@ data-bearing MXAccess failure. `MxEventStream` implements `Iterator` and `AutoCloseable`. Closing it cancels the underlying gRPC stream. Canceling or timing out a Java client call only stops the client from waiting; it does not abort an in-flight MXAccess COM -call on the worker STA. +call on the worker STA. It is a **single-consumer** surface: drive +`hasNext()`/`next()` (or `nextItem()`) from one thread only. + +### Reconnect-replay gap signal + +When you resume a stream with `streamEventsAfter(afterWorkerSequence)` and the +requested cursor predates the oldest event the gateway still retains, the +gateway emits a single **replay-gap sentinel** at the head of the stream: an +`MxEvent` with its `replay_gap` field set, `family` unspecified, and the body +oneof unset. It means "you missed events — discard cached state and +re-snapshot": the events in the open interval `(requested_after_sequence, +oldest_available_sequence)` were evicted and cannot be replayed. The gateway +never synthesizes this signal from anything else, and the client never swallows +it. + +Use `nextItem()` to branch on it as a distinct typed item; `next()` still +returns the sentinel as a plain `MxEvent` (test `event.hasReplayGap()`). After a +gap, re-snapshot, then resume without another gap by requesting +`oldestAvailableSequence - 1` as the next `afterWorkerSequence`: + +```java +try (MxEventStream events = session.streamEventsAfter(lastSeenSequence)) { + while (events.hasNext()) { + MxEventStreamItem item = events.nextItem(); + if (item.isReplayGap()) { + long resumeFrom = item.replayGap().getOldestAvailableSequence() - 1; + // discard cached per-item state, re-snapshot, then resume from resumeFrom + continue; + } + MxEvent event = item.event(); + // normal event handling + } +} +``` For alarms, `MxGatewayClient` exposes `queryActiveAlarms` (one-shot snapshot), `streamAlarms` (returns an `MxGatewayAlarmFeedSubscription` whose iterator @@ -89,30 +122,52 @@ ack target). Close the subscription to cancel the underlying gRPC stream. These are MXAccess parity behaviors that surprise new callers. The gateway forwards them unchanged — it does not paper over them. +### Typed single-item command helpers + +`MxGatewaySession` exposes typed helpers for the parity-critical single-item +commands, so you do not need to build raw `MxCommand` messages: + +- `adviseSupervisory(serverHandle, itemHandle)` (and `adviseSupervisoryRaw`) +- `writeSecured(serverHandle, itemHandle, currentUserId, verifierUserId, value)` + and `writeSecured2(..., timestampValue)` (plus `*Raw` variants) +- `authenticateUser(serverHandle, verifyUser, verifyUserPassword)` → user id +- `archestrAUserToId(serverHandle, userIdGuid)` → user id +- `addBufferedItem(serverHandle, itemDefinition, itemContext)` → item handle +- `setBufferedUpdateInterval(serverHandle, updateIntervalMs)` +- `suspend(serverHandle, itemHandle)` / `activate(serverHandle, itemHandle)` → + the reply's `MxStatusProxy` + +All of them run the same MXAccess reply validation as the bulk helpers (protocol +status plus HRESULT/`MxStatusProxy` check) via the shared `invoke` path, so an +MXAccess COM-side failure surfaces as `MxAccessException`. + +**Secret redaction.** Credentials passed to `authenticateUser` (and the +credential-sensitive values passed to `writeSecured`/`writeSecured2`) travel +only in the request. They never appear in logs, exception messages, or +`toString()`: gateway status text is scrubbed through `MxGatewaySecrets`, and +MXAccess failures carry only the reply (never the request). Do not log the +credentials yourself. + ### Attributing a write to a user without `authenticateUser` MXAccess only stamps a plain `write`/`write2` with a Galaxy user id when the item carries an active *supervisory* advise. If you are **not** using the verified/secured path (`authenticateUser` → `writeSecured`/`writeSecured2`) but -still need the write attributed to a user id, you must first advise the item -supervisory and then pass that user id on the write. Without the supervisory -advise the `userId` on a plain write is ignored. - -The session exposes `advise`/`unAdvise` but not supervisory advise, so send it -through the generic command channel: +still need the write attributed to a user id, first advise the item supervisory +and then pass that user id on the write. Without the supervisory advise the +`userId` on a plain write is ignored. ```java -session.invokeCommand(MxCommand.newBuilder() - .setKind(MxCommandKind.MX_COMMAND_KIND_ADVISE_SUPERVISORY) - .setAdviseSupervisory(AdviseSupervisoryCommand.newBuilder() - .setServerHandle(serverHandle) - .setItemHandle(itemHandle)) - .build()); - +session.adviseSupervisory(serverHandle, itemHandle); session.write(serverHandle, itemHandle, value, userId); ``` -The CLI exposes the same command as `advise-supervisory`, and `write` / +**MXAccess parity:** `writeSecured` failing before a prior `authenticateUser` + +`adviseSupervisory`, or before a value-bearing body, is correct behavior — the +native failure is surfaced, not papered over. + +The CLI exposes `advise-supervisory`, `write-secured`, and `authenticate-user` +(credential via `--password` or `--password-env`, never echoed), and `write` / `write2` take `--user-id`. ### Array writes replace the whole array @@ -324,6 +379,9 @@ gradle :zb-mom-ww-mxgateway-cli:run --args="register --endpoint localhost:5000 - gradle :zb-mom-ww-mxgateway-cli:run --args="add-item --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --session-id --server-handle 1 --item TestObject.TestInt --json" gradle :zb-mom-ww-mxgateway-cli:run --args="advise --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --session-id --server-handle 1 --item-handle 1 --json" gradle :zb-mom-ww-mxgateway-cli:run --args="write --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --session-id --server-handle 1 --item-handle 1 --type int32 --value 123 --json" +gradle :zb-mom-ww-mxgateway-cli:run --args="advise-supervisory --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --session-id --server-handle 1 --item-handle 1 --json" +gradle :zb-mom-ww-mxgateway-cli:run --args="authenticate-user --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --session-id --server-handle 1 --verify-user operator --password-env MXGATEWAY_VERIFY_PASSWORD --json" +gradle :zb-mom-ww-mxgateway-cli:run --args="write-secured --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --session-id --server-handle 1 --item-handle 1 --type int32 --value 123 --current-user-id 100 --verifier-user-id 100 --json" gradle :zb-mom-ww-mxgateway-cli:run --args="stream-events --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --session-id --limit 1 --json" gradle :zb-mom-ww-mxgateway-cli:run --args="stream-alarms --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --filter-prefix Galaxy --limit 1 --json" gradle :zb-mom-ww-mxgateway-cli:run --args="acknowledge-alarm --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --reference \"\\Galaxy\Area001.Pump001.PumpFault\" --json" diff --git a/clients/java/zb-mom-ww-mxgateway-cli/src/main/java/com/zb/mom/ww/mxgateway/cli/MxGatewayCli.java b/clients/java/zb-mom-ww-mxgateway-cli/src/main/java/com/zb/mom/ww/mxgateway/cli/MxGatewayCli.java index 32ef7a5..181f14b 100644 --- a/clients/java/zb-mom-ww-mxgateway-cli/src/main/java/com/zb/mom/ww/mxgateway/cli/MxGatewayCli.java +++ b/clients/java/zb-mom-ww-mxgateway-cli/src/main/java/com/zb/mom/ww/mxgateway/cli/MxGatewayCli.java @@ -155,6 +155,8 @@ public final class MxGatewayCli implements Callable { commandLine.addSubcommand("advise", new AdviseCommand(clientFactory)); commandLine.addSubcommand( "advise-supervisory", new AdviseSupervisoryCommand(clientFactory)); + commandLine.addSubcommand("write-secured", new WriteSecuredCommand(clientFactory)); + commandLine.addSubcommand("authenticate-user", new AuthenticateUserCommand(clientFactory)); commandLine.addSubcommand("subscribe-bulk", new SubscribeBulkCommand(clientFactory)); commandLine.addSubcommand("unsubscribe-bulk", new UnsubscribeBulkCommand(clientFactory)); commandLine.addSubcommand("read-bulk", new ReadBulkCommand(clientFactory)); @@ -1074,6 +1076,106 @@ public final class MxGatewayCli implements Callable { } } + @Command( + name = "write-secured", + description = "Invokes MXAccess WriteSecured (verified single-item write).") + static final class WriteSecuredCommand extends GatewayCommand { + @Option(names = "--session-id", required = true, description = "Gateway session id.") + String sessionId; + + @Option(names = "--server-handle", required = true, description = "MXAccess server handle.") + int serverHandle; + + @Option(names = "--item-handle", required = true, description = "MXAccess item handle.") + int itemHandle; + + @Option(names = "--type", defaultValue = "string", description = "Value type.") + String type; + + @Option(names = "--value", required = true, description = "Value text (credential-sensitive; never echoed).") + String value; + + @Option(names = "--current-user-id", defaultValue = "0", description = "MXAccess current user id.") + int currentUserId; + + @Option(names = "--verifier-user-id", defaultValue = "0", description = "MXAccess verifier user id.") + int verifierUserId; + + WriteSecuredCommand(MxGatewayCliClientFactory clientFactory) { + super(clientFactory); + } + + @Override + public Integer call() { + try (MxGatewayCliClient client = clientFactory.connect(common.resolved())) { + // The secured write value is credential-sensitive: it goes only + // into the request. The reply printed below never carries it. + MxCommandReply reply = client.session(sessionId) + .writeSecuredRaw( + serverHandle, itemHandle, currentUserId, verifierUserId, parseValue(type, value)); + writeOutput("write-secured", common, json, reply, () -> reply.getKind().name()); + } + return 0; + } + } + + @Command( + name = "authenticate-user", + description = "Invokes MXAccess AuthenticateUser and prints the resolved user id.") + static final class AuthenticateUserCommand extends GatewayCommand { + @Option(names = "--session-id", required = true, description = "Gateway session id.") + String sessionId; + + @Option(names = "--server-handle", required = true, description = "MXAccess server handle.") + int serverHandle; + + @Option(names = "--verify-user", required = true, description = "Galaxy user name to authenticate.") + String verifyUser; + + @Option( + names = "--password", + description = "Galaxy user password (credential; prefer --password-env). Never echoed.") + String password = ""; + + @Option( + names = "--password-env", + defaultValue = "MXGATEWAY_VERIFY_PASSWORD", + description = "Environment variable holding the password when --password is omitted.") + String passwordEnv; + + AuthenticateUserCommand(MxGatewayCliClientFactory clientFactory) { + super(clientFactory); + } + + @Override + public Integer call() { + // Resolve the credential from the flag or environment. It flows only + // into the request; it is never written to output, logs, or errors. + String resolvedPassword = password == null || password.isBlank() + ? System.getenv(passwordEnv) + : password; + if (resolvedPassword == null) { + resolvedPassword = ""; + } + try (MxGatewayCliClient client = clientFactory.connect(common.resolved())) { + int userId = client.session(sessionId) + .authenticateUser(serverHandle, verifyUser, resolvedPassword); + PrintWriter out = common.spec.commandLine().getOut(); + if (json) { + Map output = new LinkedHashMap<>(); + output.put("command", "authenticate-user"); + output.put("options", common.redactedJsonMap()); + output.put("verifyUser", verifyUser); + output.put("userId", userId); + out.println(jsonObject(output)); + } else { + out.println(userId); + } + } + return 0; + } + } + @Command(name = "subscribe-bulk", description = "Invokes MXAccess SubscribeBulk.") static final class SubscribeBulkCommand extends GatewayCommand { @Option(names = "--session-id", required = true, description = "Gateway session id.") @@ -1864,6 +1966,11 @@ public final class MxGatewayCli implements Callable { MxCommandReply writeRaw(int serverHandle, int itemHandle, MxValue value, int userId); + MxCommandReply writeSecuredRaw( + int serverHandle, int itemHandle, int currentUserId, int verifierUserId, MxValue value); + + int authenticateUser(int serverHandle, String verifyUser, String verifyUserPassword); + List subscribeBulk(int serverHandle, List items); List unsubscribeBulk(int serverHandle, List itemHandles); @@ -1982,13 +2089,7 @@ public final class MxGatewayCli implements Callable { @Override public MxCommandReply adviseSupervisoryRaw(int serverHandle, int itemHandle) { - return session.invokeCommand(MxCommand.newBuilder() - .setKind(MxCommandKind.MX_COMMAND_KIND_ADVISE_SUPERVISORY) - .setAdviseSupervisory( - mxaccess_gateway.v1.MxaccessGateway.AdviseSupervisoryCommand.newBuilder() - .setServerHandle(serverHandle) - .setItemHandle(itemHandle)) - .build()); + return session.adviseSupervisoryRaw(serverHandle, itemHandle); } @Override @@ -1996,6 +2097,17 @@ public final class MxGatewayCli implements Callable { return session.writeRaw(serverHandle, itemHandle, value, userId); } + @Override + public MxCommandReply writeSecuredRaw( + int serverHandle, int itemHandle, int currentUserId, int verifierUserId, MxValue value) { + return session.writeSecuredRaw(serverHandle, itemHandle, currentUserId, verifierUserId, value); + } + + @Override + public int authenticateUser(int serverHandle, String verifyUser, String verifyUserPassword) { + return session.authenticateUser(serverHandle, verifyUser, verifyUserPassword); + } + @Override public List subscribeBulk(int serverHandle, List items) { return session.subscribeBulk(serverHandle, items); diff --git a/clients/java/zb-mom-ww-mxgateway-cli/src/test/java/com/zb/mom/ww/mxgateway/cli/MxGatewayCliTests.java b/clients/java/zb-mom-ww-mxgateway-cli/src/test/java/com/zb/mom/ww/mxgateway/cli/MxGatewayCliTests.java index 2f26c63..9639cc2 100644 --- a/clients/java/zb-mom-ww-mxgateway-cli/src/test/java/com/zb/mom/ww/mxgateway/cli/MxGatewayCliTests.java +++ b/clients/java/zb-mom-ww-mxgateway-cli/src/test/java/com/zb/mom/ww/mxgateway/cli/MxGatewayCliTests.java @@ -168,6 +168,49 @@ final class MxGatewayCliTests { assertTrue(run.output().contains("\"kind\":\"MX_COMMAND_KIND_ADVISE_SUPERVISORY\"")); } + @Test + void writeSecuredCommandForwardsUserIdsAndValue() { + FakeClientFactory factory = new FakeClientFactory(); + CliRun run = execute( + factory, + "write-secured", + "--session-id", "session-cli", + "--server-handle", "12", + "--item-handle", "34", + "--type", "int32", + "--value", "77", + "--current-user-id", "100", + "--verifier-user-id", "200", + "--json"); + + assertEquals(0, run.exitCode()); + assertEquals(77, factory.client.session.lastWriteSecuredValue.getInt32Value()); + assertEquals(100, factory.client.session.lastWriteSecuredCurrentUserId); + assertEquals(200, factory.client.session.lastWriteSecuredVerifierUserId); + assertTrue(run.output().contains("\"kind\":\"MX_COMMAND_KIND_WRITE_SECURED\"")); + } + + @Test + void authenticateUserCommandForwardsCredentialAndPrintsUserIdWithoutEchoingPassword() { + FakeClientFactory factory = new FakeClientFactory(); + CliRun run = execute( + factory, + "authenticate-user", + "--session-id", "session-cli", + "--server-handle", "3", + "--verify-user", "operator", + "--password", "super-secret-pw", + "--json"); + + assertEquals(0, run.exitCode()); + // The credential reaches the session (request) but never the output. + assertEquals("operator", factory.client.session.lastAuthenticateUser); + assertEquals("super-secret-pw", factory.client.session.lastAuthenticatePassword); + assertTrue(run.output().contains("\"userId\":4242"), run.output()); + assertFalse(run.output().contains("super-secret-pw"), "password must never be echoed"); + assertFalse(run.errors().contains("super-secret-pw"), "password must never be echoed to stderr"); + } + // ---- ping subcommand (D4) ---- @Test @@ -1257,6 +1300,11 @@ final class MxGatewayCliTests { private boolean adviseCalled; private boolean adviseSupervisoryCalled; private MxValue lastWriteValue; + private MxValue lastWriteSecuredValue; + private int lastWriteSecuredCurrentUserId; + private int lastWriteSecuredVerifierUserId; + private String lastAuthenticateUser; + private String lastAuthenticatePassword; private String lastPingMessage; private long lastReadBulkTimeoutMs; private List lastReadBulkItems; @@ -1341,6 +1389,25 @@ final class MxGatewayCliTests { .build(); } + @Override + public MxCommandReply writeSecuredRaw( + int serverHandle, int itemHandle, int currentUserId, int verifierUserId, MxValue value) { + lastWriteSecuredValue = value; + lastWriteSecuredCurrentUserId = currentUserId; + lastWriteSecuredVerifierUserId = verifierUserId; + return MxCommandReply.newBuilder() + .setKind(MxCommandKind.MX_COMMAND_KIND_WRITE_SECURED) + .setProtocolStatus(ok()) + .build(); + } + + @Override + public int authenticateUser(int serverHandle, String verifyUser, String verifyUserPassword) { + lastAuthenticateUser = verifyUser; + lastAuthenticatePassword = verifyUserPassword; + return 4242; + } + @Override public List subscribeBulk(int serverHandle, List items) { List results = new ArrayList<>(); diff --git a/clients/java/zb-mom-ww-mxgateway-client/src/main/java/com/zb/mom/ww/mxgateway/client/MxEventStream.java b/clients/java/zb-mom-ww-mxgateway-client/src/main/java/com/zb/mom/ww/mxgateway/client/MxEventStream.java index 0fa27e4..e884c97 100644 --- a/clients/java/zb-mom-ww-mxgateway-client/src/main/java/com/zb/mom/ww/mxgateway/client/MxEventStream.java +++ b/clients/java/zb-mom-ww-mxgateway-client/src/main/java/com/zb/mom/ww/mxgateway/client/MxEventStream.java @@ -21,6 +21,24 @@ import mxaccess_gateway.v1.MxaccessGateway.StreamEventsRequest; * stream cancels the underlying gRPC call. If the queue overflows the call is * cancelled and a follow-up call to {@link #next()} throws * {@link MxGatewayException}. + * + *

Single consumer. This stream is not safe to drain from + * more than one thread. Interleave {@link #hasNext()}/{@link #next()} (or + * {@link #nextItem()}) from a single consumer only; concurrent drains race on + * the internal cursor. + * + *

Reconnect-replay gap. When the stream was resumed via + * {@code StreamEventsRequest.after_worker_sequence} and the requested cursor + * predates the oldest event the gateway still retains, the gateway emits a + * single gap sentinel {@link MxEvent} at the head of the stream with its + * {@code replay_gap} field set (family unspecified, body oneof unset). It is a + * distinct, non-terminal signal, forwarded verbatim — never synthesized and + * never swallowed. {@link #next()} returns it as a normal {@link MxEvent} + * (callers can test {@code event.hasReplayGap()}); {@link #nextItem()} wraps it + * in an {@link MxEventStreamItem} whose {@link MxEventStreamItem#isReplayGap()} + * is {@code true}. On a gap the consumer must discard cached per-item state and + * re-snapshot, then resume without a further gap by requesting + * {@code oldest_available_sequence - 1} as the next {@code after_worker_sequence}. */ public final class MxEventStream implements Iterator, AutoCloseable { private static final Object END = new Object(); @@ -91,6 +109,24 @@ public final class MxEventStream implements Iterator, AutoCloseable { return (MxEvent) value; } + /** + * Drains the next stream element as a typed {@link MxEventStreamItem} so a + * consumer can branch on the reconnect-replay gap sentinel via + * {@link MxEventStreamItem#isReplayGap()} without inspecting the raw event. + * + *

Equivalent to wrapping {@link #next()}; the gap sentinel is surfaced as + * a distinct typed item rather than being swallowed, and normal events are + * returned unchanged on {@link MxEventStreamItem#event()}. Do not mix + * {@link #next()} and {@code nextItem()} on the same element — each call + * advances the single shared cursor. + * + * @return the next stream item + * @throws NoSuchElementException if the stream is exhausted + */ + public MxEventStreamItem nextItem() { + return new MxEventStreamItem(next()); + } + @Override public void close() { closed = true; diff --git a/clients/java/zb-mom-ww-mxgateway-client/src/main/java/com/zb/mom/ww/mxgateway/client/MxEventStreamItem.java b/clients/java/zb-mom-ww-mxgateway-client/src/main/java/com/zb/mom/ww/mxgateway/client/MxEventStreamItem.java new file mode 100644 index 0000000..1b369fb --- /dev/null +++ b/clients/java/zb-mom-ww-mxgateway-client/src/main/java/com/zb/mom/ww/mxgateway/client/MxEventStreamItem.java @@ -0,0 +1,71 @@ +package com.zb.mom.ww.mxgateway.client; + +import java.util.Objects; +import mxaccess_gateway.v1.MxaccessGateway.MxEvent; +import mxaccess_gateway.v1.MxaccessGateway.ReplayGap; + +/** + * Typed view over a single item drained from an {@link MxEventStream}. + * + *

A {@code StreamEvents} stream resumed via {@code after_worker_sequence} + * may begin with a gateway reconnect-replay gap sentinel: a single + * {@link MxEvent} whose {@code replay_gap} field is set, whose + * {@code family} is {@code MX_EVENT_FAMILY_UNSPECIFIED}, and whose {@code body} + * oneof is unset. It means the requested resume cursor predates the oldest + * event the gateway still retains, so the events in the open interval + * {@code (requested_after_sequence, oldest_available_sequence)} were evicted and + * cannot be replayed. + * + *

This wrapper lets a consumer branch on that sentinel without inspecting + * the raw event: {@link #isReplayGap()} is {@code true} only for the sentinel, + * and {@link #replayGap()} exposes the typed {@link ReplayGap} cursors. Normal + * MXAccess events return {@code false} from {@link #isReplayGap()} and carry + * their payload on {@link #event()} exactly as before. + * + *

The gateway never synthesizes an {@code OperationComplete} or any other + * event from the gap; the sentinel is the gateway's own forwarded signal and is + * surfaced here untouched. On receiving a gap the consumer must discard any + * cached per-item state and re-snapshot, then resume without incurring another + * gap by requesting {@code oldest_available_sequence - 1} as the new + * {@code after_worker_sequence} cursor. + * + * @param event the raw event; for a gap sentinel this is the sentinel event + * itself (family unspecified, body unset, {@code replay_gap} set) + */ +public record MxEventStreamItem(MxEvent event) { + /** + * Creates a stream-item view over the supplied event. + * + * @param event the raw event; must not be {@code null} + */ + public MxEventStreamItem { + Objects.requireNonNull(event, "event"); + } + + /** + * Returns whether this item is the reconnect-replay gap sentinel rather + * than a normal MXAccess event. Detected via the generated + * {@code MxEvent.hasReplayGap()} presence flag. + * + * @return {@code true} for the gap sentinel, {@code false} for normal events + */ + public boolean isReplayGap() { + return event.hasReplayGap(); + } + + /** + * Returns the typed reconnect-replay gap descriptor when this item is the + * gap sentinel. + * + * @return the {@link ReplayGap} carrying {@code requested_after_sequence} + * and {@code oldest_available_sequence} + * @throws IllegalStateException if this item is a normal event (check + * {@link #isReplayGap()} first) + */ + public ReplayGap replayGap() { + if (!event.hasReplayGap()) { + throw new IllegalStateException("stream item is not a replay-gap sentinel"); + } + return event.getReplayGap(); + } +} diff --git a/clients/java/zb-mom-ww-mxgateway-client/src/main/java/com/zb/mom/ww/mxgateway/client/MxGatewaySession.java b/clients/java/zb-mom-ww-mxgateway-client/src/main/java/com/zb/mom/ww/mxgateway/client/MxGatewaySession.java index 8dd00e1..68c3843 100644 --- a/clients/java/zb-mom-ww-mxgateway-client/src/main/java/com/zb/mom/ww/mxgateway/client/MxGatewaySession.java +++ b/clients/java/zb-mom-ww-mxgateway-client/src/main/java/com/zb/mom/ww/mxgateway/client/MxGatewaySession.java @@ -7,11 +7,16 @@ import java.util.List; import java.util.Map; import java.util.Objects; import java.util.TreeMap; +import mxaccess_gateway.v1.MxaccessGateway.ActivateCommand; +import mxaccess_gateway.v1.MxaccessGateway.AddBufferedItemCommand; import mxaccess_gateway.v1.MxaccessGateway.AddItem2Command; import mxaccess_gateway.v1.MxaccessGateway.AddItemBulkCommand; import mxaccess_gateway.v1.MxaccessGateway.AddItemCommand; import mxaccess_gateway.v1.MxaccessGateway.AdviseItemBulkCommand; import mxaccess_gateway.v1.MxaccessGateway.AdviseCommand; +import mxaccess_gateway.v1.MxaccessGateway.AdviseSupervisoryCommand; +import mxaccess_gateway.v1.MxaccessGateway.ArchestrAUserToIdCommand; +import mxaccess_gateway.v1.MxaccessGateway.AuthenticateUserCommand; import mxaccess_gateway.v1.MxaccessGateway.BulkReadResult; import mxaccess_gateway.v1.MxaccessGateway.BulkWriteResult; import mxaccess_gateway.v1.MxaccessGateway.CloseSessionReply; @@ -23,15 +28,18 @@ import mxaccess_gateway.v1.MxaccessGateway.MxCommandRequest; import mxaccess_gateway.v1.MxaccessGateway.MxDataType; import mxaccess_gateway.v1.MxaccessGateway.MxSparseArray; import mxaccess_gateway.v1.MxaccessGateway.MxSparseElement; +import mxaccess_gateway.v1.MxaccessGateway.MxStatusProxy; import mxaccess_gateway.v1.MxaccessGateway.MxValue; import mxaccess_gateway.v1.MxaccessGateway.OpenSessionReply; import mxaccess_gateway.v1.MxaccessGateway.ReadBulkCommand; import mxaccess_gateway.v1.MxaccessGateway.RegisterCommand; import mxaccess_gateway.v1.MxaccessGateway.RemoveItemBulkCommand; import mxaccess_gateway.v1.MxaccessGateway.RemoveItemCommand; +import mxaccess_gateway.v1.MxaccessGateway.SetBufferedUpdateIntervalCommand; import mxaccess_gateway.v1.MxaccessGateway.StreamEventsRequest; import mxaccess_gateway.v1.MxaccessGateway.SubscribeBulkCommand; import mxaccess_gateway.v1.MxaccessGateway.SubscribeResult; +import mxaccess_gateway.v1.MxaccessGateway.SuspendCommand; import mxaccess_gateway.v1.MxaccessGateway.UnAdviseCommand; import mxaccess_gateway.v1.MxaccessGateway.UnAdviseItemBulkCommand; import mxaccess_gateway.v1.MxaccessGateway.UnsubscribeBulkCommand; @@ -42,6 +50,8 @@ import mxaccess_gateway.v1.MxaccessGateway.Write2Command; import mxaccess_gateway.v1.MxaccessGateway.WriteBulkCommand; import mxaccess_gateway.v1.MxaccessGateway.WriteBulkEntry; import mxaccess_gateway.v1.MxaccessGateway.WriteCommand; +import mxaccess_gateway.v1.MxaccessGateway.WriteSecured2Command; +import mxaccess_gateway.v1.MxaccessGateway.WriteSecuredCommand; import mxaccess_gateway.v1.MxaccessGateway.WriteSecured2BulkCommand; import mxaccess_gateway.v1.MxaccessGateway.WriteSecured2BulkEntry; import mxaccess_gateway.v1.MxaccessGateway.WriteSecuredBulkCommand; @@ -697,6 +707,285 @@ public final class MxGatewaySession implements AutoCloseable { .build()); } + /** + * Invokes MXAccess {@code AdviseSupervisory} so the item accepts + * supervisory-attributed writes. Required before a plain {@link #write} + * can stamp a Galaxy user id without the verified/secured path. + * + * @param serverHandle the {@code ServerHandle} owning the item + * @param itemHandle the {@code ItemHandle} to advise supervisory + * @throws MxGatewayException on transport or protocol failure + * @throws MxAccessException when MXAccess reports a COM-side failure + */ + public void adviseSupervisory(int serverHandle, int itemHandle) { + adviseSupervisoryRaw(serverHandle, itemHandle); + } + + /** + * Invokes MXAccess {@code AdviseSupervisory} and returns the raw reply. + * + * @param serverHandle the {@code ServerHandle} owning the item + * @param itemHandle the {@code ItemHandle} to advise supervisory + * @return the raw command reply + * @throws MxGatewayException on transport or protocol failure + * @throws MxAccessException when MXAccess reports a COM-side failure + */ + public MxCommandReply adviseSupervisoryRaw(int serverHandle, int itemHandle) { + return invokeCommand(MxCommand.newBuilder() + .setKind(MxCommandKind.MX_COMMAND_KIND_ADVISE_SUPERVISORY) + .setAdviseSupervisory(AdviseSupervisoryCommand.newBuilder() + .setServerHandle(serverHandle) + .setItemHandle(itemHandle)) + .build()); + } + + /** + * Invokes MXAccess {@code WriteSecured} — a verified write gated by a + * previously authenticated Galaxy user. + * + *

MXAccess parity: the native call fails if the caller + * has not first {@link #authenticateUser authenticated} and + * {@link #adviseSupervisory advised supervisory}, or if the value body is + * absent; that native failure is surfaced (as {@link MxAccessException}), + * not papered over. + * + *

Secret handling: {@code value} may carry a + * credential-sensitive payload. It is placed only in the request and never + * appears in any surfaced error (exceptions carry the reply, not the + * request), so callers must likewise avoid logging it. + * + * @param serverHandle the {@code ServerHandle} owning the item + * @param itemHandle the {@code ItemHandle} to write + * @param currentUserId the authenticated (current) Galaxy user id + * @param verifierUserId the verifier Galaxy user id (second-signature); use + * the same value as {@code currentUserId} when no separate verifier applies + * @param value the credential-sensitive value to write + * @throws MxGatewayException on transport or protocol failure + * @throws MxAccessException when MXAccess reports a COM-side failure + */ + public void writeSecured( + int serverHandle, int itemHandle, int currentUserId, int verifierUserId, MxValue value) { + writeSecuredRaw(serverHandle, itemHandle, currentUserId, verifierUserId, value); + } + + /** + * Invokes MXAccess {@code WriteSecured} and returns the raw reply. + * + * @param serverHandle the {@code ServerHandle} owning the item + * @param itemHandle the {@code ItemHandle} to write + * @param currentUserId the authenticated (current) Galaxy user id + * @param verifierUserId the verifier Galaxy user id + * @param value the credential-sensitive value to write + * @return the raw command reply + * @throws MxGatewayException on transport or protocol failure + * @throws MxAccessException when MXAccess reports a COM-side failure + */ + public MxCommandReply writeSecuredRaw( + int serverHandle, int itemHandle, int currentUserId, int verifierUserId, MxValue value) { + return invokeCommand(MxCommand.newBuilder() + .setKind(MxCommandKind.MX_COMMAND_KIND_WRITE_SECURED) + .setWriteSecured(WriteSecuredCommand.newBuilder() + .setServerHandle(serverHandle) + .setItemHandle(itemHandle) + .setCurrentUserId(currentUserId) + .setVerifierUserId(verifierUserId) + .setValue(value)) + .build()); + } + + /** + * Invokes MXAccess {@code WriteSecured2} — a verified, explicitly + * timestamped write. Parity and secret-handling notes mirror + * {@link #writeSecured}. + * + * @param serverHandle the {@code ServerHandle} owning the item + * @param itemHandle the {@code ItemHandle} to write + * @param currentUserId the authenticated (current) Galaxy user id + * @param verifierUserId the verifier Galaxy user id + * @param value the credential-sensitive value to write + * @param timestampValue the timestamp value to associate with the write + * @throws MxGatewayException on transport or protocol failure + * @throws MxAccessException when MXAccess reports a COM-side failure + */ + public void writeSecured2( + int serverHandle, + int itemHandle, + int currentUserId, + int verifierUserId, + MxValue value, + MxValue timestampValue) { + writeSecured2Raw(serverHandle, itemHandle, currentUserId, verifierUserId, value, timestampValue); + } + + /** + * Invokes MXAccess {@code WriteSecured2} and returns the raw reply. + * + * @param serverHandle the {@code ServerHandle} owning the item + * @param itemHandle the {@code ItemHandle} to write + * @param currentUserId the authenticated (current) Galaxy user id + * @param verifierUserId the verifier Galaxy user id + * @param value the credential-sensitive value to write + * @param timestampValue the timestamp value to associate with the write + * @return the raw command reply + * @throws MxGatewayException on transport or protocol failure + * @throws MxAccessException when MXAccess reports a COM-side failure + */ + public MxCommandReply writeSecured2Raw( + int serverHandle, + int itemHandle, + int currentUserId, + int verifierUserId, + MxValue value, + MxValue timestampValue) { + return invokeCommand(MxCommand.newBuilder() + .setKind(MxCommandKind.MX_COMMAND_KIND_WRITE_SECURED2) + .setWriteSecured2(WriteSecured2Command.newBuilder() + .setServerHandle(serverHandle) + .setItemHandle(itemHandle) + .setCurrentUserId(currentUserId) + .setVerifierUserId(verifierUserId) + .setValue(value) + .setTimestampValue(timestampValue)) + .build()); + } + + /** + * Invokes MXAccess {@code AuthenticateUser} and returns the resolved + * Galaxy user id used to attribute subsequent secured writes. + * + *

Secret handling: {@code verifyUserPassword} is a raw + * MXAccess credential. It is placed only in the request and never appears in + * any surfaced error, log, or {@code toString()} (exceptions carry the + * reply, not the request). Callers must not log it either. + * + * @param serverHandle the {@code ServerHandle} for the session + * @param verifyUser the Galaxy user name to authenticate + * @param verifyUserPassword the user's credential; never logged or surfaced + * @return the authenticated Galaxy user id + * @throws MxGatewayException on transport or protocol failure + * @throws MxAccessException when MXAccess rejects the credential + */ + public int authenticateUser(int serverHandle, String verifyUser, String verifyUserPassword) { + MxCommandReply reply = invokeCommand(MxCommand.newBuilder() + .setKind(MxCommandKind.MX_COMMAND_KIND_AUTHENTICATE_USER) + .setAuthenticateUser(AuthenticateUserCommand.newBuilder() + .setServerHandle(serverHandle) + .setVerifyUser(verifyUser) + .setVerifyUserPassword(verifyUserPassword)) + .build()); + if (reply.hasAuthenticateUser()) { + return reply.getAuthenticateUser().getUserId(); + } + return reply.getReturnValue().getInt32Value(); + } + + /** + * Invokes MXAccess {@code ArchestrAUserToId}, resolving a Galaxy user GUID + * to its integer user id. + * + * @param serverHandle the {@code ServerHandle} for the session + * @param userIdGuid the Galaxy user GUID string + * @return the resolved Galaxy user id + * @throws MxGatewayException on transport or protocol failure + * @throws MxAccessException when MXAccess reports a COM-side failure + */ + public int archestrAUserToId(int serverHandle, String userIdGuid) { + MxCommandReply reply = invokeCommand(MxCommand.newBuilder() + .setKind(MxCommandKind.MX_COMMAND_KIND_ARCHESTRA_USER_TO_ID) + .setArchestraUserToId(ArchestrAUserToIdCommand.newBuilder() + .setServerHandle(serverHandle) + .setUserIdGuid(userIdGuid)) + .build()); + if (reply.hasArchestraUserToId()) { + return reply.getArchestraUserToId().getUserId(); + } + return reply.getReturnValue().getInt32Value(); + } + + /** + * Invokes MXAccess {@code AddBufferedItem} and returns the new item handle. + * The buffered add family delivers coalesced {@code OnBufferedDataChange} + * updates on the interval set by {@link #setBufferedUpdateInterval}. + * + * @param serverHandle the {@code ServerHandle} owning the item + * @param itemDefinition the MXAccess item definition (tag reference) + * @param itemContext the MXAccess item context (e.g. galaxy/object scope) + * @return the {@code ItemHandle} assigned by MXAccess + * @throws MxGatewayException on transport or protocol failure + * @throws MxAccessException when MXAccess reports a COM-side failure + */ + public int addBufferedItem(int serverHandle, String itemDefinition, String itemContext) { + MxCommandReply reply = invokeCommand(MxCommand.newBuilder() + .setKind(MxCommandKind.MX_COMMAND_KIND_ADD_BUFFERED_ITEM) + .setAddBufferedItem(AddBufferedItemCommand.newBuilder() + .setServerHandle(serverHandle) + .setItemDefinition(itemDefinition) + .setItemContext(itemContext)) + .build()); + if (reply.hasAddBufferedItem()) { + return reply.getAddBufferedItem().getItemHandle(); + } + return reply.getReturnValue().getInt32Value(); + } + + /** + * Invokes MXAccess {@code SetBufferedUpdateInterval}, controlling how often + * the worker coalesces buffered updates for the given server handle. + * + * @param serverHandle the {@code ServerHandle} to configure + * @param updateIntervalMilliseconds the buffered update interval in milliseconds + * @throws MxGatewayException on transport or protocol failure + * @throws MxAccessException when MXAccess reports a COM-side failure + */ + public void setBufferedUpdateInterval(int serverHandle, int updateIntervalMilliseconds) { + invokeCommand(MxCommand.newBuilder() + .setKind(MxCommandKind.MX_COMMAND_KIND_SET_BUFFERED_UPDATE_INTERVAL) + .setSetBufferedUpdateInterval(SetBufferedUpdateIntervalCommand.newBuilder() + .setServerHandle(serverHandle) + .setUpdateIntervalMilliseconds(updateIntervalMilliseconds)) + .build()); + } + + /** + * Invokes MXAccess {@code Suspend} on an item and returns the reply's + * {@link MxStatusProxy}. + * + * @param serverHandle the {@code ServerHandle} owning the item + * @param itemHandle the {@code ItemHandle} to suspend + * @return the {@code MxStatusProxy} carried by the suspend reply + * @throws MxGatewayException on transport or protocol failure + * @throws MxAccessException when MXAccess reports a COM-side failure + */ + public MxStatusProxy suspend(int serverHandle, int itemHandle) { + MxCommandReply reply = invokeCommand(MxCommand.newBuilder() + .setKind(MxCommandKind.MX_COMMAND_KIND_SUSPEND) + .setSuspend(SuspendCommand.newBuilder() + .setServerHandle(serverHandle) + .setItemHandle(itemHandle)) + .build()); + return reply.getSuspend().getStatus(); + } + + /** + * Invokes MXAccess {@code Activate} on a previously suspended item and + * returns the reply's {@link MxStatusProxy}. + * + * @param serverHandle the {@code ServerHandle} owning the item + * @param itemHandle the {@code ItemHandle} to activate + * @return the {@code MxStatusProxy} carried by the activate reply + * @throws MxGatewayException on transport or protocol failure + * @throws MxAccessException when MXAccess reports a COM-side failure + */ + public MxStatusProxy activate(int serverHandle, int itemHandle) { + MxCommandReply reply = invokeCommand(MxCommand.newBuilder() + .setKind(MxCommandKind.MX_COMMAND_KIND_ACTIVATE) + .setActivate(ActivateCommand.newBuilder() + .setServerHandle(serverHandle) + .setItemHandle(itemHandle)) + .build()); + return reply.getActivate().getStatus(); + } + /** * Subscribes to gateway events for this session starting from the * beginning of the worker event log. diff --git a/clients/java/zb-mom-ww-mxgateway-client/src/test/java/com/zb/mom/ww/mxgateway/client/MxGatewayClientSessionTests.java b/clients/java/zb-mom-ww-mxgateway-client/src/test/java/com/zb/mom/ww/mxgateway/client/MxGatewayClientSessionTests.java index 16cc720..bfc3a2f 100644 --- a/clients/java/zb-mom-ww-mxgateway-client/src/test/java/com/zb/mom/ww/mxgateway/client/MxGatewayClientSessionTests.java +++ b/clients/java/zb-mom-ww-mxgateway-client/src/test/java/com/zb/mom/ww/mxgateway/client/MxGatewayClientSessionTests.java @@ -1,6 +1,7 @@ package com.zb.mom.ww.mxgateway.client; import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; import static org.junit.jupiter.api.Assertions.assertNotNull; import static org.junit.jupiter.api.Assertions.assertNull; import static org.junit.jupiter.api.Assertions.assertThrows; @@ -30,6 +31,7 @@ import mxaccess_gateway.v1.MxaccessGateway.AddItemReply; import mxaccess_gateway.v1.MxaccessGateway.AlarmConditionState; import mxaccess_gateway.v1.MxaccessGateway.AlarmFeedMessage; import mxaccess_gateway.v1.MxaccessGateway.AlarmTransitionKind; +import mxaccess_gateway.v1.MxaccessGateway.AuthenticateUserReply; import mxaccess_gateway.v1.MxaccessGateway.BulkSubscribeReply; import mxaccess_gateway.v1.MxaccessGateway.OnAlarmTransitionEvent; import mxaccess_gateway.v1.MxaccessGateway.CloseSessionReply; @@ -39,6 +41,7 @@ import mxaccess_gateway.v1.MxaccessGateway.MxCommandReply; import mxaccess_gateway.v1.MxaccessGateway.MxCommandRequest; import mxaccess_gateway.v1.MxaccessGateway.MxDataType; import mxaccess_gateway.v1.MxaccessGateway.MxEvent; +import mxaccess_gateway.v1.MxaccessGateway.MxEventFamily; import mxaccess_gateway.v1.MxaccessGateway.MxSparseElement; import mxaccess_gateway.v1.MxaccessGateway.MxValue; import mxaccess_gateway.v1.MxaccessGateway.OpenSessionReply; @@ -47,6 +50,7 @@ import mxaccess_gateway.v1.MxaccessGateway.ProtocolStatus; import mxaccess_gateway.v1.MxaccessGateway.ProtocolStatusCode; import mxaccess_gateway.v1.MxaccessGateway.QueryActiveAlarmsRequest; import mxaccess_gateway.v1.MxaccessGateway.RegisterReply; +import mxaccess_gateway.v1.MxaccessGateway.ReplayGap; import mxaccess_gateway.v1.MxaccessGateway.SessionState; import mxaccess_gateway.v1.MxaccessGateway.StreamAlarmsRequest; import mxaccess_gateway.v1.MxaccessGateway.StreamEventsRequest; @@ -509,6 +513,232 @@ final class MxGatewayClientSessionTests { } } + @Test + void streamEventsSurfacesReplayGapSentinelAsTypedItem() throws Exception { + // CLI-15: a resumed stream whose cursor predates the retained window + // begins with the gateway's replay-gap sentinel. It must surface as a + // distinct typed item, and normal events must be unaffected. + TestGatewayService service = new TestGatewayService() { + @Override + public void streamEvents(StreamEventsRequest request, StreamObserver responseObserver) { + responseObserver.onNext(MxEvent.newBuilder() + .setSessionId(request.getSessionId()) + .setReplayGap(ReplayGap.newBuilder() + .setRequestedAfterSequence(5) + .setOldestAvailableSequence(9)) + .build()); + responseObserver.onNext(MxEvent.newBuilder() + .setSessionId(request.getSessionId()) + .setFamily(MxEventFamily.MX_EVENT_FAMILY_ON_DATA_CHANGE) + .setWorkerSequence(9) + .build()); + responseObserver.onCompleted(); + } + }; + + try (InProcessGateway gateway = InProcessGateway.start(service, new AtomicReference<>()); + MxGatewayClient client = gateway.client("", Duration.ofSeconds(5)); + MxEventStream events = + MxGatewaySession.forSessionId(client, "resume-session").streamEventsAfter(5)) { + assertTrue(events.hasNext()); + MxEventStreamItem gap = events.nextItem(); + assertTrue(gap.isReplayGap(), "head sentinel must classify as a replay gap"); + assertEquals(5, gap.replayGap().getRequestedAfterSequence()); + assertEquals(9, gap.replayGap().getOldestAvailableSequence()); + // Sentinel carries no MXAccess payload. + assertEquals(MxEventFamily.MX_EVENT_FAMILY_UNSPECIFIED, gap.event().getFamily()); + + assertTrue(events.hasNext()); + MxEventStreamItem normal = events.nextItem(); + assertFalse(normal.isReplayGap(), "normal event must not classify as a replay gap"); + assertEquals(9, normal.event().getWorkerSequence()); + assertEquals(MxEventFamily.MX_EVENT_FAMILY_ON_DATA_CHANGE, normal.event().getFamily()); + } + } + + @Test + void adviseSupervisoryBuildsSupervisoryCommand() throws Exception { + AtomicReference commandRequest = new AtomicReference<>(); + TestGatewayService service = okInvokeService(commandRequest); + + try (InProcessGateway gateway = InProcessGateway.start(service, new AtomicReference<>()); + MxGatewayClient client = gateway.client("", Duration.ofSeconds(5))) { + MxGatewaySession session = MxGatewaySession.forSessionId(client, "advise-super-session"); + + session.adviseSupervisory(12, 34); + + assertEquals( + MxCommandKind.MX_COMMAND_KIND_ADVISE_SUPERVISORY, + commandRequest.get().getCommand().getKind()); + assertEquals(12, commandRequest.get().getCommand().getAdviseSupervisory().getServerHandle()); + assertEquals(34, commandRequest.get().getCommand().getAdviseSupervisory().getItemHandle()); + } + } + + @Test + void writeSecuredSurfacesNativeMxAccessFailure() throws Exception { + // CLI-04 parity: WriteSecured before authenticate/advise fails natively; + // the client surfaces the failure rather than papering over it. + TestGatewayService service = new TestGatewayService() { + @Override + public void invoke(MxCommandRequest request, StreamObserver responseObserver) { + responseObserver.onNext(MxCommandReply.newBuilder() + .setSessionId(request.getSessionId()) + .setKind(request.getCommand().getKind()) + .setProtocolStatus(ProtocolStatus.newBuilder() + .setCode(ProtocolStatusCode.PROTOCOL_STATUS_CODE_MXACCESS_FAILURE) + .setMessage("WriteSecured rejected: user not authenticated.")) + .setHresult(-2147220992) + .build()); + responseObserver.onCompleted(); + } + }; + + try (InProcessGateway gateway = InProcessGateway.start(service, new AtomicReference<>()); + MxGatewayClient client = gateway.client("", Duration.ofSeconds(5))) { + MxGatewaySession session = MxGatewaySession.forSessionId(client, "secured-session"); + + MxAccessException error = assertThrows( + MxAccessException.class, + () -> session.writeSecured(1, 2, 100, 100, MxValues.stringValue("secret-payload"))); + + assertEquals(-2147220992, error.reply().getHresult()); + // The credential-bearing value lives only in the request, so it must + // not appear in any surfaced error text. + assertFalse(String.valueOf(error.getMessage()).contains("secret-payload")); + } + } + + @Test + void writeSecuredScriptedSuccessSendsSecuredCommand() throws Exception { + AtomicReference commandRequest = new AtomicReference<>(); + TestGatewayService service = okInvokeService(commandRequest); + + try (InProcessGateway gateway = InProcessGateway.start(service, new AtomicReference<>()); + MxGatewayClient client = gateway.client("", Duration.ofSeconds(5))) { + MxGatewaySession session = MxGatewaySession.forSessionId(client, "secured-ok-session"); + + session.writeSecured(7, 8, 100, 200, MxValues.int32Value(42)); + + var command = commandRequest.get().getCommand(); + assertEquals(MxCommandKind.MX_COMMAND_KIND_WRITE_SECURED, command.getKind()); + assertEquals(7, command.getWriteSecured().getServerHandle()); + assertEquals(8, command.getWriteSecured().getItemHandle()); + assertEquals(100, command.getWriteSecured().getCurrentUserId()); + assertEquals(200, command.getWriteSecured().getVerifierUserId()); + assertEquals(42, command.getWriteSecured().getValue().getInt32Value()); + } + } + + @Test + void authenticateUserReturnsUserIdAndForwardsCredential() throws Exception { + AtomicReference commandRequest = new AtomicReference<>(); + TestGatewayService service = new TestGatewayService() { + @Override + public void invoke(MxCommandRequest request, StreamObserver responseObserver) { + commandRequest.set(request); + responseObserver.onNext(MxCommandReply.newBuilder() + .setSessionId(request.getSessionId()) + .setKind(request.getCommand().getKind()) + .setProtocolStatus(ok()) + .setAuthenticateUser(AuthenticateUserReply.newBuilder().setUserId(4242)) + .build()); + responseObserver.onCompleted(); + } + }; + + try (InProcessGateway gateway = InProcessGateway.start(service, new AtomicReference<>()); + MxGatewayClient client = gateway.client("", Duration.ofSeconds(5))) { + MxGatewaySession session = MxGatewaySession.forSessionId(client, "auth-session"); + + int userId = session.authenticateUser(3, "operator", "super-secret-pw"); + + assertEquals(4242, userId); + // The credential is forwarded in the request only. + assertEquals( + "super-secret-pw", + commandRequest.get().getCommand().getAuthenticateUser().getVerifyUserPassword()); + } + } + + @Test + void authenticateUserFailureKeepsCredentialOutOfSurfacedError() throws Exception { + TestGatewayService service = new TestGatewayService() { + @Override + public void invoke(MxCommandRequest request, StreamObserver responseObserver) { + responseObserver.onNext(MxCommandReply.newBuilder() + .setSessionId(request.getSessionId()) + .setKind(request.getCommand().getKind()) + .setProtocolStatus(ProtocolStatus.newBuilder() + .setCode(ProtocolStatusCode.PROTOCOL_STATUS_CODE_MXACCESS_FAILURE) + .setMessage("AuthenticateUser rejected the credential.")) + .setHresult(-2147220992) + .build()); + responseObserver.onCompleted(); + } + }; + + try (InProcessGateway gateway = InProcessGateway.start(service, new AtomicReference<>()); + MxGatewayClient client = gateway.client("", Duration.ofSeconds(5))) { + MxGatewaySession session = MxGatewaySession.forSessionId(client, "auth-fail-session"); + + MxAccessException error = assertThrows( + MxAccessException.class, + () -> session.authenticateUser(3, "operator", "super-secret-pw")); + + // The password must never reach the exception message or toString(). + assertFalse(String.valueOf(error.getMessage()).contains("super-secret-pw")); + assertFalse(error.toString().contains("super-secret-pw")); + } + } + + @Test + void suspendAndActivateReturnStatusProxy() throws Exception { + TestGatewayService service = new TestGatewayService() { + @Override + public void invoke(MxCommandRequest request, StreamObserver responseObserver) { + MxCommandReply.Builder reply = MxCommandReply.newBuilder() + .setSessionId(request.getSessionId()) + .setKind(request.getCommand().getKind()) + .setProtocolStatus(ok()); + if (request.getCommand().getKind() == MxCommandKind.MX_COMMAND_KIND_SUSPEND) { + reply.setSuspend(mxaccess_gateway.v1.MxaccessGateway.SuspendReply.newBuilder() + .setStatus(mxaccess_gateway.v1.MxaccessGateway.MxStatusProxy.newBuilder() + .setSuccess(1))); + } else if (request.getCommand().getKind() == MxCommandKind.MX_COMMAND_KIND_ACTIVATE) { + reply.setActivate(mxaccess_gateway.v1.MxaccessGateway.ActivateReply.newBuilder() + .setStatus(mxaccess_gateway.v1.MxaccessGateway.MxStatusProxy.newBuilder() + .setSuccess(1))); + } + responseObserver.onNext(reply.build()); + responseObserver.onCompleted(); + } + }; + + try (InProcessGateway gateway = InProcessGateway.start(service, new AtomicReference<>()); + MxGatewayClient client = gateway.client("", Duration.ofSeconds(5))) { + MxGatewaySession session = MxGatewaySession.forSessionId(client, "suspend-session"); + + assertTrue(MxStatuses.succeeded(session.suspend(1, 2))); + assertTrue(MxStatuses.succeeded(session.activate(1, 2))); + } + } + + private static TestGatewayService okInvokeService(AtomicReference commandRequest) { + return new TestGatewayService() { + @Override + public void invoke(MxCommandRequest request, StreamObserver responseObserver) { + commandRequest.set(request); + responseObserver.onNext(MxCommandReply.newBuilder() + .setSessionId(request.getSessionId()) + .setKind(request.getCommand().getKind()) + .setProtocolStatus(ok()) + .build()); + responseObserver.onCompleted(); + } + }; + } + private static ProtocolStatus ok() { return ProtocolStatus.newBuilder() .setCode(ProtocolStatusCode.PROTOCOL_STATUS_CODE_OK) diff --git a/docs/ClientLibrariesDesign.md b/docs/ClientLibrariesDesign.md index a616291..77356a0 100644 --- a/docs/ClientLibrariesDesign.md +++ b/docs/ClientLibrariesDesign.md @@ -83,8 +83,9 @@ discard cached state, re-snapshot, and resume with `after_worker_sequence = oldest_available_sequence - 1`. The per-language surface follows each idiom (.NET `MxEventStreamItem.IsReplayGap` via `StreamEventItemsAsync`; Go `EventResult.ReplayGap`/`IsReplayGap()`; Rust -`EventItem::ReplayGap`; Python yields a typed `ReplayGap` from -`stream_events`; Java — pending, tracked with the JDK-17/windows client pass). +`EventItem::ReplayGap`; Python yields a typed `ReplayGap` from `stream_events`; +Java `MxEventStreamItem.isReplayGap()` via `MxEventStream.nextItem()`). Shipped +in all five clients. ## Public Client Concepts @@ -124,8 +125,7 @@ reach logs, exception text, or `ToString`/`Debug`/`Display` — the value is car only on the wire. Each client's test suite asserts a distinctive credential is absent from any surfaced error. -Shipped for .NET / Go / Rust / Python; the Java client's typed parity helpers are -batched to the windows build host (no local JRE on the primary dev box). +Shipped in all five clients (.NET / Go / Rust / Python / Java). ## Shared API Shape -- 2.52.0 From f1d9ad5ca2aa194c717450d58c9ea02076c8810d Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Thu, 9 Jul 2026 16:57:47 -0400 Subject: [PATCH 24/25] docs(archreview): CLI-15/CLI-04/CLI-30/TST-01 -> Done (Java completes 5/5) Session-resilience epic Phase 3 fully closed (reconnect protocol + client consumers across all five clients + server e2e test). Claude-Session: https://claude.ai/code/session_01DMXXvNuPekkkrTEyPNxEkW --- archreview/remediation/00-tracking.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/archreview/remediation/00-tracking.md b/archreview/remediation/00-tracking.md index 5227d5d..ef194df 100644 --- a/archreview/remediation/00-tracking.md +++ b/archreview/remediation/00-tracking.md @@ -177,7 +177,7 @@ Full design + implementation for each row lives in the linked domain doc under i | CLI-01 | High | P0 | M | — | Done | Go `Session.Events()` silently closes stream on 16-slot overflow | | CLI-02 | High | P1 | M | — | Done | Rust crate unbuildable outside the repo (`build.rs` path + `--no-verify`) | | CLI-03 | High | P0 | M | — | Done | Rust `invoke` never validates HRESULT / MXSTATUS_PROXY | -| CLI-04 | High | P2 | L | CLI-15 | In progress | Typed-command parity gap across all five clients | +| CLI-04 | High | P2 | L | CLI-15 | Done | Typed-command parity gap across all five clients | | CLI-05 | Medium | — | S | — | Not started | .NET session cannot be re-attached to an existing session id | | CLI-06 | Medium | — | S | — | Not started | .NET `DisposeAsync` blocks/throws on unreachable gateway | | CLI-07 | Medium | — | S | — | Not started | .NET retry budget self-defeats on `DeadlineExceeded` | @@ -188,7 +188,7 @@ Full design + implementation for each row lives in the linked domain doc under i | CLI-12 | Medium | P2 | S | — | Done | Java docs still say "Java 21" after the JDK 17 retarget | | CLI-13 | Medium | — | M | — | Not started | Java event-stream buffer hardcoded 16, cancel-on-overflow | | CLI-14 | Medium | — | S | — | Not started | Python default TLS is blocking TOFU pin with silent `localhost` SNI | -| CLI-15 | Medium | P2 | M | — | In progress | No client handles `ReplayGap` or offers a reconnect helper | +| CLI-15 | Medium | P2 | M | — | Done | No client handles `ReplayGap` or offers a reconnect helper | | CLI-16 | Medium | P2 | S | CLI-12 | Done | `docs/ClientPackaging.md` drifted from Python naming and .NET `.slnx` | | CLI-17 | Medium | — | M | CLI-14 | Not started | TLS default posture inconsistent across the five clients | | CLI-18 | Low | P2 | S | — | Done | .NET csproj records no `` | @@ -213,7 +213,7 @@ Full design + implementation for each row lives in the linked domain doc under i | ID | Sev | Tier | Eff | Dep | Status | Title | |---|---|:-:|:-:|---|---|---| -| TST-01 | High | P2 | L | TST-04 | In progress | Reconnect/replay has no e2e test and no client consumer | +| TST-01 | High | P2 | L | TST-04 | Done | Reconnect/replay has no e2e test and no client consumer | | TST-02 | High | P0 | M | TST-04 | Done | Reconnect owner re-validation not implemented | | TST-03 | High | P1 | M | — | In review | No CI exists | | TST-04 | High | P2 | L | — | Done | Session-resilience epic 16/28 tasks unfinished | @@ -253,6 +253,7 @@ Findings the review flagged as one coordinated design pass — sequence them tog | Date | Change | |---|---| +| 2026-07-09 | P2 Epic — **Java client completes CLI-15 + CLI-04 locally** (commit `1cc0fa4`); **CLI-15, CLI-04, CLI-30, TST-01 all → `Done` (5/5 clients + server e2e)**. Java CLI-15: `MxEventStreamItem` record + `MxEventStream.nextItem()` (`isReplayGap()`/`replayGap()`/`event()`); existing `Iterator` path unchanged, sentinel never swallowed. Java CLI-04: Phase 1 `adviseSupervisory`/`writeSecured`/`writeSecured2`/`authenticateUser`/`archestrAUserToId` + Phase 2 `addBufferedItem`/`setBufferedUpdateInterval`/`suspend`/`activate` (unregister already present) on `MxGatewaySession`, each through `invokeCommand` → `ensureProtocolSuccess`+`ensureMxAccessSuccess`; credentials scrubbed via `MxGatewaySecrets.redactCredentials` (tests assert absent from message/toString/CLI). `gradle test` 106/0 (58 client + 48 cli), no generated churn. Built locally with `JAVA_HOME=/opt/homebrew/opt/openjdk@17` — Java toolchain now works on the Mac (see prior note). Shared docs `ClientLibrariesDesign.md` + CLAUDE.md updated to "all five clients". **TST-01 → Done** (server e2e `fed0685` + all 5 client `ReplayGap` consumers). This closes session-resilience epic Phase 3 fully. | | 2026-07-09 | P2 Epic Wave E2 — typed-command parity (commit `bde042b`). CLI-04 → `In progress` (4/5 clients; Java pending in the SAME session — see next note), CLI-30 → `Done`. Every parity-critical single-item command now has a typed session helper across .NET/Go/Rust/Python: Phase 1 `AdviseSupervisory`/`WriteSecured`/`WriteSecured2`/`AuthenticateUser`/`ArchestrAUserToId`, Phase 2 `AddBufferedItem`/`SetBufferedUpdateInterval`/`Suspend`/`Activate`, plus `Unregister` (CLI-30: Rust + .NET added; Go/Java/Python already had it — CLI-30 fully Done). Each wraps existing raw-command machinery (no new wire surface) and runs the client's MXAccess-level reply validation (`hresult < 0` + `MxStatusProxy`). MXAccess parity preserved (WriteSecured-before-authenticate surfaces the native failure). **Credentials route through each client's secret-redaction seam** (never in logs/exceptions/ToString/Debug/Display; each suite asserts a distinctive credential is absent from surfaced errors); new CLI subcommands take credentials via flag/env, never echoed. Verified per toolchain: .NET build clean 102 passed; Go gofmt/vet/build/test clean; Rust fmt/check/test/clippy clean; Python 145 passed. Shared doc: `ClientLibrariesDesign.md` "Typed Command Parity" section. | | 2026-07-09 | **Java client toolchain now works locally on the Mac** (user-flagged + verified): homebrew `openjdk@17` (17.0.19), @21, @26 are installed (off PATH); `JAVA_HOME=/opt/homebrew/opt/openjdk@17 gradle test` → BUILD SUCCESSFUL. Retires the "Java client must be built on windev" constraint — the remaining Java halves of CLI-15 + CLI-04 are done locally this session (no windev batch). Memory `project_java_build_host` rewritten. | | 2026-07-09 | P2 Epic — TST-04 (session-resilience epic governance) → `Done` (commit `ed3c6c6`, docs-only umbrella). Resolved the epic's shipped-vs-planned entanglement into three decisions: **Phase 3 finished** (Task 13=TST-02 owner-scoped attach P0, Task 15=TST-01 reconnect e2e, Task 14=CLI-15 4/5 clients w/ Java pending); **Phase 4 scoped** as TST-15 with the Viewer-default decision settled (admin-sees-all, Viewer strict per owned/granted session, matching TST-02's gRPC owner binding); **Phase 5 (orphan-worker reattach) DEFERRED, not planned** — the CLAUDE.md "gateway restart does not reattach orphan workers" invariant stands and the `EnableOrphanReattach` flag does not exist / must not be referenced. Updated `oldtasks.md`, the `tasks.json` mirror (per-task statuses + governance note), and the CLAUDE.md reconnect paragraph (clients consume ReplayGap; reattach deferred). The actionable slices carry their own verification: TST-01 (done), TST-02 (done, P0), TST-15 (pending, E3). | -- 2.52.0 From 2201fd8828957ae5265883ccc28a77ba8308a75d Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Thu, 9 Jul 2026 17:04:22 -0400 Subject: [PATCH 25/25] =?UTF-8?q?docs(archreview):=20wrap=20P2=20epic=20?= =?UTF-8?q?=E2=80=94=20defer=20TST-15=20+=20TST-24=20(user=20decision)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Epic bucket: 5/7 done (CLI-15, CLI-04, CLI-30, TST-01, TST-04). TST-15 (Phase-4 ACL feature; SEC-25 already prevents value leakage) and TST-24 (CI-gated on TST-03) deferred to a follow-up. Net P2: 35/38 Done. Claude-Session: https://claude.ai/code/session_01DMXXvNuPekkkrTEyPNxEkW --- archreview/remediation/00-tracking.md | 1 + 1 file changed, 1 insertion(+) diff --git a/archreview/remediation/00-tracking.md b/archreview/remediation/00-tracking.md index ef194df..8bf4044 100644 --- a/archreview/remediation/00-tracking.md +++ b/archreview/remediation/00-tracking.md @@ -253,6 +253,7 @@ Findings the review flagged as one coordinated design pass — sequence them tog | Date | Change | |---|---| +| 2026-07-09 | **P2 Epic wrap — user decision: DEFER TST-15 + TST-24, close the epic.** Epic bucket result: 5 of 7 findings `Done` (CLI-15, CLI-04, CLI-30, TST-01, TST-04); **TST-15** and **TST-24** intentionally deferred to a follow-up (kept `Not started`, not `Won't fix` — they are gated, not rejected). **TST-15** (dashboard EventsHub per-session ACL) is epic Phase 4 — a real feature needing a new session-"tag" mechanism + dashboard group→tag config, not a mechanical fix; the `EventsHub` `TODO(per-session-acl)` stays, and the already-shipped **SEC-25** mitigation (tag *values* redacted from the dashboard mirror by default) means no sensitive payload leaks through the hub today regardless of the missing ACL — so deferring carries no value-leak risk. **TST-24** (per-client wire tests) depends on **TST-03** (CI), which is `In review` (YAML authored, never run on a Gitea runner) — no point wiring client tests into a pipeline that isn't live yet. Net P2: 35/38 `Done`; remaining = TST-15 (deferred feature), TST-24 (deferred, CI-gated), TST-14 (user deletes their own untracked gitignored `*-docs-*.md` files). | | 2026-07-09 | P2 Epic — **Java client completes CLI-15 + CLI-04 locally** (commit `1cc0fa4`); **CLI-15, CLI-04, CLI-30, TST-01 all → `Done` (5/5 clients + server e2e)**. Java CLI-15: `MxEventStreamItem` record + `MxEventStream.nextItem()` (`isReplayGap()`/`replayGap()`/`event()`); existing `Iterator` path unchanged, sentinel never swallowed. Java CLI-04: Phase 1 `adviseSupervisory`/`writeSecured`/`writeSecured2`/`authenticateUser`/`archestrAUserToId` + Phase 2 `addBufferedItem`/`setBufferedUpdateInterval`/`suspend`/`activate` (unregister already present) on `MxGatewaySession`, each through `invokeCommand` → `ensureProtocolSuccess`+`ensureMxAccessSuccess`; credentials scrubbed via `MxGatewaySecrets.redactCredentials` (tests assert absent from message/toString/CLI). `gradle test` 106/0 (58 client + 48 cli), no generated churn. Built locally with `JAVA_HOME=/opt/homebrew/opt/openjdk@17` — Java toolchain now works on the Mac (see prior note). Shared docs `ClientLibrariesDesign.md` + CLAUDE.md updated to "all five clients". **TST-01 → Done** (server e2e `fed0685` + all 5 client `ReplayGap` consumers). This closes session-resilience epic Phase 3 fully. | | 2026-07-09 | P2 Epic Wave E2 — typed-command parity (commit `bde042b`). CLI-04 → `In progress` (4/5 clients; Java pending in the SAME session — see next note), CLI-30 → `Done`. Every parity-critical single-item command now has a typed session helper across .NET/Go/Rust/Python: Phase 1 `AdviseSupervisory`/`WriteSecured`/`WriteSecured2`/`AuthenticateUser`/`ArchestrAUserToId`, Phase 2 `AddBufferedItem`/`SetBufferedUpdateInterval`/`Suspend`/`Activate`, plus `Unregister` (CLI-30: Rust + .NET added; Go/Java/Python already had it — CLI-30 fully Done). Each wraps existing raw-command machinery (no new wire surface) and runs the client's MXAccess-level reply validation (`hresult < 0` + `MxStatusProxy`). MXAccess parity preserved (WriteSecured-before-authenticate surfaces the native failure). **Credentials route through each client's secret-redaction seam** (never in logs/exceptions/ToString/Debug/Display; each suite asserts a distinctive credential is absent from surfaced errors); new CLI subcommands take credentials via flag/env, never echoed. Verified per toolchain: .NET build clean 102 passed; Go gofmt/vet/build/test clean; Rust fmt/check/test/clippy clean; Python 145 passed. Shared doc: `ClientLibrariesDesign.md` "Typed Command Parity" section. | | 2026-07-09 | **Java client toolchain now works locally on the Mac** (user-flagged + verified): homebrew `openjdk@17` (17.0.19), @21, @26 are installed (off PATH); `JAVA_HOME=/opt/homebrew/opt/openjdk@17 gradle test` → BUILD SUCCESSFUL. Retires the "Java client must be built on windev" constraint — the remaining Java halves of CLI-15 + CLI-04 are done locally this session (no windev batch). Memory `project_java_build_host` rewritten. | -- 2.52.0