Same-commit docs rule (were missed in the prior commit):
- docs/GalaxyRepository.md: SnapshotCachePath now documents the per-OS derived
default and the GalaxyRepositoryOptionsValidator rooting/validity enforcement.
- A2-galaxyrepository-adoption-handoff.md: correct the now-inaccurate NSSM caveat
(SnapshotCachePath override is optional, not required; blank seeds a rooted host
default, no silent no-op) and repoint the option-validation item at the new
GalaxyRepositoryOptionsValidator.
SEC-34 guard confirmed and documented: TryParseKeyId's '_' split cannot truncate a
key id because both — and the only — gateway key-creation paths
(ApiKeyAdminCommandLineParser.IsValidKeyId, DashboardApiKeyManagementService.ValidateKeyId)
restrict key ids to IsAsciiLetterOrDigit || '.' || '-', and key ids are never
library-generated. Added a citing comment; no behavior change.
Test consolidation: moved the three host-start SqlitePath overrides into
TestHostEnvironmentInitializer (per-process temp store, mirroring Secrets__SqlitePath)
so future host-start tests auto-cover.
SEC-33: make rooting host-meaningful and stop shipping foreign-platform literals.
- Delete IsRootedForAnyPlatform; AddIfNotRooted now uses Path.IsPathRooted (current OS).
- Promote AddIfNotRooted/AddIfInvalidPath to shared GatewayConfigPathRules so the new
Galaxy validator reuses them and the two validators cannot drift.
- Remove Authentication:SqlitePath and Galaxy:SnapshotCachePath Windows literals from
appsettings.json; the CommonApplicationData-derived code defaults take over. The
Galaxy default is seeded as a configuration value before AddZbGalaxyRepository
(SnapshotCachePath is init-only, so a PostConfigure mutation cannot compile).
- New GalaxyRepositoryOptionsValidator (ValidateOnStart) enforces a valid, host-rooted
SnapshotCachePath when PersistSnapshot is true.
- Root-cause the stray junk-named auth DB: host start eagerly builds
AuthSqliteConnectionFactory; under the non-rooted Windows literal on macOS SQLite
wrote it relative to the test bin CWD. The three real-host-start tests now pin
SqlitePath to a temp path.
SEC-34: verification cache Invalidate-vs-in-flight-repopulation race closed with a
per-key generation counter (bump-before-evict, snapshot-then-recheck). The expiry
cap (window 2) takes the documented fallback: the library verification identity
carries no ExpiresUtc, so the cache cannot cap at the key's expiry (donor-library ask).
GWC-24 rider: cap MxGateway:Events:QueueCapacity at int.MaxValue/2 so the derived
checked(2 * EventChannelCapacity) in WorkerClient cannot overflow at session creation.
SEC-35 (doc-only): note IsProduction() env-name semantics in GatewayConfiguration.md.
Docs updated same commit (GatewayConfiguration.md, Authentication.md) and tracking
registers/change-log flipped (00-tracking.md, 40-security-dashboard.md).
Flip the four findings to Done in the 2026-07-12 tracking registers
(Gateway core + Testing) and in the per-domain registers of
10-gateway-core.md and 60-testing-docs-gaps.md; append the 2026-08-07
change-log row recording what shipped, the pre-fix red for GWC-28, and
the TST-28 mutation check.
ReadAsync allocated a fresh 4-byte array per inbound frame; the GWC-08
pass pooled the payload buffer but left the prefix. Replaced with a
per-instance scratch field — the reader is single-consumer by
construction (one read loop per WorkerClient, handshake reads complete
before the loop starts), so a per-instance buffer is safe and the
non-reentrancy that makes it safe is now stated on the class. Pooling
four bytes via ArrayPool would cost more than the allocation it saves.
Tests: WorkerFrameProtocolTests.ReadAsync_WithMultipleFramesOnOneReader_
ParsesEveryFrame reads five frames of differing payload length through
one reader, so a stale prefix carried between calls would misparse.
Invoke deep-cloned the whole MxCommandRequest — including its command
payload, potentially a large bulk-write graph — only to overwrite the
cloned command with commandToInvoke and discard it. MapCommand then did
the one clone actually needed. Net cost: a full wasted command deep-clone
per Invoke, worst for exactly the bulk writes that are largest.
Adds a MapCommand(MxCommand) overload (MapCommand reads nothing else off
the request) and has Invoke pass commandToInvoke directly; the request
overload delegates so other callers are untouched.
The remaining clone inside MapCommand stays and is now documented as
required rather than incidental: commandToInvoke may be the gRPC-owned
request.Command, and the caller reads it again after dispatch via
TrackCommandReply, so ownership transfer (à la GWC-07) is not safe here.
That clone is what keeps WorkerClient.CreateCommandEnvelope's no-aliasing
invariant true.
Tests: MxAccessGrpcMapperTests.MapCommandFromCommandClonesPayload
(mutating the input leaves the mapped command untouched; both overloads
produce equal results under a fixed TimeProvider).
CreateEnvelope stamped Sequence with an interlocked increment when the
envelope was built, so two concurrent InvokeAsync callers could take 1
and 2 and then enqueue in the order 2, 1 — non-monotonic on the wire,
breaking gateway.md's "monotonic per sender" contract. Benign today
(neither side validates inbound sequence, old GWC-10 still open) but it
would fault healthy sessions the moment worker-side validation lands.
WriteLoopAsync now stamps immediately before _writer.WriteAsync. It is
the outbound channel's single consumer (SingleReader = true), so wire
order and stamp order are the same thing by construction and
_nextSequence drops to a plain ulong with no interlocking. This mirrors
the worker's WRK-04 fix, which the gateway half never received.
Also adds TST-28: a [Theory] pinning that GatewayHello.MaxFrameBytes
carries the configured worker-frame maximum (default + 2 MiB override).
The adoption half is asserted only in the Windows-only worker suite, so
a regression to sending 0 — "older gateway, use default" to the worker —
would silently downgrade the negotiated IPC-02 limit with every CI test
still green. Mutation-checked (hard-coded 0 fails both cases).
Tests: WorkerClientTests.ConcurrentInvokesEmitStrictlyIncreasingSequences
OnTheWire (32 parallel invokes; failed 3/3 pre-fix) and
.StartAsync_SendsGatewayHelloWithConfiguredMaxFrameBytes.
ReleaseProbe recognised its own reservation by comparing NextProbeAtTicks to
now + _probeIntervalTicks. RecordInto's rearm-on-trip writes that identical
expression, so a concurrent RecordFailure on the same WindowState whose `now`
lands on the claimer's tick — routine at ~1 ms clock resolution under load — was
mistaken for the caller's own claim. The release then stomped the legitimate
fresh re-arm back to the stale previousProbeAtTicks, which is already due, handing
the next arrival a free probe the re-arm had just closed.
WindowState gains a monotonic ProbeVersion bumped by every writer of
NextProbeAtTicks (TryConsumeProbe's claim and RecordInto's re-arm alike).
TryConsumeProbe returns the stamp it set as part of a ProbeClaim; ReleaseProbe
restores the previous value only while the state's version still equals that
stamp, checking and restoring in one lock(state) section and bumping the version
again on restore so no other stale release can match either.
Test: ProbeSlotRestore_DoesNotStompConcurrentRearmAtSameTick, with the clock held
still so the claim and the interleaved failure necessarily share a tick. Making it
deterministic needed a seam — the claim-to-release window is a few nanoseconds and
racing threads do not hit it (an earlier thread-based attempt passed against the
defective guard three runs out of three, and its end state was ordering-dependent
rather than correctness-dependent, so it was dropped rather than shipped as
theatre). The seam is an internal ProbeReleaseInterleaveHook, null in production,
costing one null check on the already-refused path. Verified as a genuine red
against the timestamp guard: Expected ThrottledByPeer, Actual ProbeAdmitted.
All five client CLIs now share one credential contract for `authenticate-user`:
flags `--password` / `--password-env` (Go: `-password` / `-password-env`) with
default env `MXGATEWAY_VERIFY_PASSWORD`, resolution flag-then-env, and a resolved
credential that is missing *or empty* is a usage error naming the flag and the
variable. The value is never echoed and never reaches the wire.
Go and Java previously sent an empty credential when the variable was unset,
turning a misconfigured environment into a real MXAccess authentication attempt.
Go now returns the guard error before dialing; Java throws a picocli
ParameterException instead of falling back to "". Python's `--password-env`
gained the canonical default and its UsageError names the resolved variable.
Rust treats an empty flag or env value as missing, with the resolution extracted
into a testable `resolve_verify_user_password`. .NET adopts the canonical flags
and keeps `--verify-user-password`, `--verify-user-password-env`, and
MXGATEWAY_VERIFY_USER_PASSWORD as deprecated aliases for one release.
Docs same commit: CrossLanguageSmokeMatrix.md gains the credential contract and
the per-CLI subcommand-coverage table (the documented-not-fixed half of the
finding); all five READMEs name the canonical variable and the fail-fast rule,
and the .NET README carries the deprecation note. Tracking flipped to Done in
both remediation registers with a change-log row.
No .proto changed; no generated code regenerated.
One cross-client conformance pass; also closes first-cycle CLI-08.
CLI-37: an MxStatusProxy entry is a failure iff `category !=
MX_STATUS_CATEGORY_OK`. The proto contract has always said so — `success` is
the raw 16-bit COM member carried verbatim for diagnostics, not a boolean — but
four clients branched on `success` alone and .NET required both, so the same
gateway reply produced opposite verdicts per language. An absent entry stays
success; a present entry with an UNSPECIFIED category is a failure, because the
worker always maps a category and an unmapped one is not proven OK.
CLI-38: a reply fails on HRESULT iff `hresult` is present and negative, so
positive COM success codes such as S_FALSE (1) pass. .NET/Go/Java used `!= 0`,
which errored on a parity-preserving S_FALSE that Python and Rust accepted.
This makes the existing ClientLibrariesDesign.md claim true rather than
rewriting the doc to describe the divergence.
Four shared fixtures pin both rules cross-client, and each language suite also
carries a table test for the two edges a fixture cannot express (absent entry,
UNSPECIFIED category). A Java test fake that built a status with a bare
`setSuccess(1)` and no category is fixed — under the category rule that reply
was never a success.
Code-review follow-up on fix/gwc-26-27-alarm-attach.
ApplyReconcile's snapshot-derived feed repairs are at-least-once, not
exactly-once: a reconcile reads the worker's current state while the matching
live transition may still be buffered in the monitor's lease, so both broadcast
and the duplicates are indistinguishable on the alarm feed. This pre-dates the
acked-state delta — the Raise/Clear presence repair has always had it, since
nothing serializes a reconcile pass against the in-flight live stream — so
closing it (serialization or timestamp dedup) stays out of scope for a P2 fix.
Documented instead, with the consumer contract stated explicitly (apply
transitions idempotently, never as an increment or toggle):
- ApplyReconcile gains a "Delivery semantics" comment.
- gateway.md softens the "defense in depth" prose to state the semantics.
- docs/Sessions.md carries the same caveat on the alarm-feed description.
- Tracker change-log records it as a known pre-existing characteristic and a
candidate finding for the next review cycle.
Also hoists the ChannelWorkerClient fake — duplicated across the three alarm
test files — into TestSupport/, dropping the usings it took with it.
Two defects found in code review of the limiter rework.
Probe admission was check-then-act across two lock scopes: Check() read
"probe due" under lock(state), released it, then re-acquired to advance
NextProbeAtTicks. A burst of requests arriving together at an interval boundary
could therefore all observe the slot as due and all be admitted, handing the
verifier the very burst the interval exists to bound. The claim is now a single
critical section (TryConsumeProbe). The two layers are still claimed one at a
time — holding two per-state locks at once would need a global lock ordering to
stay deadlock-free — so a slot claimed on the composite partition is compensated
via ReleaseProbe when the aggregate then refuses, which otherwise silently spent
the partition's next slot and pushed the legitimate holder out by a full
interval.
Reset() removed whatever partition the caller resolved to, including the
address's shared fallback partition when the caller's key id had been collapsed
into it by the per-peer cap (or when the token was junk-shaped). That bucket also
carries failures contributed by other key ids from the same address, so one
successful authentication became a reset button for an in-progress spray. Reset
now clears only a partition the caller owns (effectiveKeyId == presented key id);
the shared bucket decays by window expiry instead, and the caller still recovers
through probe admission. The key's aggregate is cleared either way, as designed.
Also applied from the review: closure-free GetOrAdd overload on _partitions, and
a remarks paragraph acknowledging the best-effort O(n) eviction scan under
sustained overflow. Threading the resolved partition key from Check through to
RecordFailure/Reset was declined: Check resolves with mint:false and RecordFailure
with mint:true, and the two can legitimately differ when a concurrent caller fills
the per-peer cap in between — reusing Check's key would record into the wrong
partition and bypass the cap, which is not worth saving one string concat.
Tests (limiter suite 11 -> 14): ProbeAdmission_UnderConcurrentArrivals_
GrantsExactlyOneSlot (200 rounds x 8 barrier-released threads at the boundary),
ProbeAdmission_WhenAggregateRefuses_ReturnsTheClaimedPeerSlot, and
Reset_WithOverCapKeyId_DoesNotClearSharedFallbackPartition. The latter two were
confirmed as genuine reds against the unfixed code; the concurrency test is a
guard — it is deterministically green on the fixed structure but did not
reproduce the original nanosecond-wide window on its own.
Seeding the cache with a live Raise transition raced the first reconcile: when
the reconcile snapshot populated the cache first, the still-buffered live Raise
was applied — and broadcast — after the test's feed subscriber had registered,
so the exactly-one-transition assertion saw two. Seed through a reconcile pass
instead (forced by a provider-mode probe, as the acked step already did) so no
live transition is ever in flight. Verified: 5 consecutive full alarm-monitor
runs green, and the test still fails (timeout) with the ApplyReconcile
acked-delta branch removed.
An empty replay ring reported oldest_available_sequence = 0 even when gap was
true. Clients follow the documented after_worker_sequence = oldest - 1 formula,
so an unsigned client computed ulong.MaxValue: the follow-up resume replayed
nothing, reported no gap, and the live filter dropped every subsequent event —
a silently dead stream in the headline detach-and-resume scenario, reachable on
default config once ReplayRetentionSeconds (300) age-evicts the ring.
GWC-25: SessionEventDistributor.RegisterWithReplay's empty-ring branch now
reports _highestSequenceSeen + 1 — the next sequence that can possibly be
delivered — when gap is true, so oldest - 1 lands exactly on the highest
observed sequence and the resume delivers everything newer. Still 0 when there
is no gap, where the field is meaningless and never emitted. Nothing is lost:
the evicted interval was unrecoverable either way, and the sentinel's job is to
say "re-snapshot".
CLI-35: the Python CLI fed every stream item into MessageToDict, which raised on
the ReplayGap dataclass and aborted the command after consuming the stream. A
new _event_row helper renders a gap as {"replayGap": {...}} — the same camelCase
shape the Rust CLI emits — and leaves proto events on the existing path.
CLI-36: the Go CLI formatted result.Event on every row, but the library
deliberately clears Event on a gap, so text mode printed
"0 MX_EVENT_FAMILY_UNSPECIFIED" and JSON mode an empty object, discarding the
resume cursors. The loop now branches on result.IsReplayGap() and renders the
typed row in both modes, counting it toward -limit like any other row. The JSON
row's cursors are typed by hand rather than marshalled with protojson: the
proto3 JSON mapping renders 64-bit integers as strings ("7") while the Rust and
Python CLIs emit numbers (7), so going through protojson would have made Go the
only canonical CLI with a different value type.
Docs in the same change: docs/Sessions.md documents the empty-ring sentinel
value and that oldest - 1 is the universal resume formula in both the retained
and fully-evicted cases; docs/CrossLanguageSmokeMatrix.md gains a per-CLI
gap-rendering table covering both client findings, and records exactly what is
and is not comparable across CLIs (same keys and numeric cursors for Rust/Go/
Python; quoted cursors for .NET/Java; differing key order, whitespace, and
container), so a matrix runner compares parsed values rather than raw bytes.
Tests, all written red first and each reproducing its defect verbatim:
- SessionEventDistributorTests: RegisterWithReplayReportsNextDeliverableSequence
WhenRingEmptiedByAge, ...WithRetentionDisabled, and
ResumeUsingSentinelFormulaAfterEmptyRingGapDeliversLiveEvents.
- GatewayEndToEndReconnectReplayTests.ReconnectAfterFullAgeEvictionResumesWith
SentinelFormula — fake-worker e2e resume walk on a fake clock; the fixture now
takes a retention window and a TimeProvider.
- clients/python test_stream_events_renders_replay_gap.
- clients/go TestRunStreamEventsPrintsReplayGap.
GWC-25's ReplayGap.oldest_available_sequence proto-comment amendment is
deliberately deferred to the later codegen wave (see the tracker change log): it
is comment-only but triggers the full five-client regen fan-out.
RunMonitorAsync issued SubscribeAlarms and the first reconcile before the
internal distributor subscriber was attached (via ISessionManager
.ReadAlarmEventsAsync). The pump has been running since MarkReady started the
dashboard mirror and only fans to subscribers registered at fan-out time, so
every transition raised in that two-round-trip window bypassed the alarm feed —
and a missed Acknowledge was never repaired, because ApplyReconcile broadcast
presence deltas only.
- The monitor now takes the internal lease directly from its session BEFORE
SubscribeAlarms and drains it after the first reconcile; window transitions
buffer in the lease's bounded channel. Processing them after ApplyReconcile is
order-safe (ApplyTransition handles alarms the snapshot already placed).
- ISessionManager.ReadAlarmEventsAsync removed — zero remaining callers.
- ApplyReconcile broadcasts an Acknowledge feed transition when a both-present
alarm's state advanced to ActiveAcked. This is a feed-level repair on the
AlarmFeedMessage/StreamAlarms surface rebuilt from the worker's own snapshot,
not MxEvent emission, so the "never synthesize events" rule is untouched;
the reasoning is recorded on ApplyReconcile.
The alarm-monitor test fakes now hand the monitor a real Ready GatewaySession
with a dashboard mirror, which is what makes the window reproducible.
Docs: docs/Sessions.md and gateway.md alarm-monitor ordering notes.
Refs: archreview/2026-07-12/remediation/10-gateway-core.md GWC-26
The gRPC auth failure limiter partitioned on the key id parsed out of the
*unauthenticated* token and rejected with ResourceExhausted before VerifyAsync
ran. Key ids are not secret — they ride in every token and are listed on the
dashboard — so any network peer could send 10 garbage-secret requests per minute
and deny that key indefinitely: the legitimate holder's correct secret was
refused before it was ever checked, and the success-path Reset that would clear
the block sat behind the verification the block prevented (SEC-31). The tracked
map was also flushable — any `a_b_c`-shaped junk minted a fresh partition (the
`mxgw` literal was never compared), so ~4096 throwaway tokens evicted a blocked
entry and reset the window (SEC-32).
ApiKeyFailureLimiter moves from IsBlocked/RecordFailure/Reset(string peer) to a
partition-pair API: Check/RecordFailure/Reset(ApiKeyThrottlePartition) with an
ApiKeyThrottleDecision result. Two layers share one sliding window — a composite
(transport peer, key id) partition at ApiKeyFailureLimit, and a per-key-id
aggregate across all peers at the new ApiKeyFailureAggregateLimit (default 30)
that bounds a source-rotating sprayer. An over-limit state is now a valve rather
than a wall: one request per the new ApiKeyFailureProbeIntervalSeconds (default
5) is admitted through to the real verifier, so the correct secret always reaches
the constant-time compare and resets both layers. Guarantees preserved: guessing
stays bounded per window, and the failure path still spends no store read per
attempt.
SEC-32 rides the same change set: the interceptor validates token shape (literal
`mxgw` prefix, >= 3 non-empty `_` segments, key id <= 64 chars) before minting a
key-id partition, each transport peer may mint at most 32 of them before the
overflow collapses onto its fallback partition, and eviction prefers fully
expired windows and never drops an over-limit partition below a 2x transient
overshoot ceiling. Throttled attempts increment mxgateway.auth.throttled, tagged
stage=peer|aggregate only — /metrics is unauthenticated (open SEC-14), so no key
material may appear there.
Docs in the same commit: GatewayConfiguration limiter rows plus the two new keys,
the Authentication hot-path paragraph, the Authorization SEC-11 section, and the
limiter / SecurityOptions XML remarks (the old NAT rationale described the
defective keying). Tracking rows flipped to Done with a change-log entry.
Tests: new ApiKeyFailureLimiterTests (11) covering window pruning, composite vs
aggregate trip points, probe cadence, absolute-block mode, reset across both
layers, junk-spray eviction resistance, the per-peer cap, and expired-window
eviction preference; GatewayGrpcAuthorizationInterceptorTests gains the four
SEC-31 contract tests plus NonMxgwToken_FallsBackToTransportPeerPartition (20
total); GatewayOptionsValidatorTests covers both new keys including 0 as a
supported disable value (66 total).
The GWC-04 remediation decoupled the read loop from event backpressure by
staging events into an unbounded channel, so its TryWrite always succeeded and
the only overflow fault was a single timed WriteAsync exceeding
EventChannelFullModeTimeout. A consumer draining slower than the worker
produces — each individual write still completing inside the window — therefore
grew gateway memory without bound, without a fault, and without a metric: the
queue-depth gauge counted only the bounded consumer channel, so staged events
were invisible.
Bound _eventStaging at 2 x EventChannelCapacity (Wait, single reader/writer, no
synchronous continuations). A rejected staging TryWrite is the sustained
slow-drain signal and faults the client ProtocolViolation with
QueueOverflow("worker-event-staging"), guarded by IsTerminalState() so a
completed channel during shutdown stays a silent drop. SetFaulted is
non-blocking, so the read loop still never awaits behind events. The timed-write
fault is unchanged and still catches the full-stall case earlier.
Move the queue-depth increment from EnqueueWorkerEventAsync to StageWorkerEvent
so the single counter reports total undelivered events (staged + queued); the
decrement at consumer read was already correct. No new configuration key: the
bound is derived, and gateway-side buffering per session is now at most
3 x MxGateway:Events:QueueCapacity. Coordination with still-open GWC-21
(EventChannelFullModeTimeout configurability) remains open and was not blocked
on.
Tests: StagingChannelOverflowFaultsWorkerWithoutWaitingForFullModeTimeout (5-min
full-mode timeout so only the staging bound can fire; asserts an interleaved
command reply still completes) and WorkerEventQueueDepthGaugeCountsStagedEvents.
Docs updated in the same change: GatewayProcessDesign, MxAccessWorkerInstanceDesign,
GatewayConfiguration, Metrics. GWC-24 flipped to Done in both trackers.
AttachInternalEventSubscriber ran EnsureDistributorCreated / Register /
StartPumpIfRequested with no state check, unlike AttachEventSubscriber. A
premature attach would start the pump against a not-yet-Ready worker; the pump
source throws SessionNotReady, PumpAsync completes every subscriber with that
error and latches the distributor, and _eventDistributorStarted is never reset —
so the session would reach Ready with permanently dead event streaming.
Mirror AttachEventSubscriber's gate: check _state/_workerClient.State under
_syncRoot and throw SessionManagerException(SessionNotReady) before the
distributor is created, keeping the distributor calls outside the lock.
Refs: archreview/2026-07-12/remediation/10-gateway-core.md GWC-27
Migrate the durable session-resilience governance record (Phase 5
orphan-worker reattach deferred-not-planned, EnableOrphanReattach
does not yet exist, settled Phase-4 Viewer-default decision) from
oldtasks.md into a new "Session-Resilience Epic Scope" entry in
docs/DesignDecisions.md, repoint CLAUDE.md and stillpending.md's
oldtasks.md references to the new home / tasks.json, and git rm
oldtasks.md now that it has no unique content left. Flip TST-29 to
Done in the archreview tracking registers.
The five untracked root docs-review artifacts (MxAccessGateway-docs-*,
MxGatewayClient-docs-*) are absent from this worktree; they must be
deleted from the main working tree separately (gitignored, no repo
impact).
GatewayLogRedactor.RedactCommandValue had no production caller. Its only
one was GatewayLogRedactorAdapter on the abandoned
feat/adopt-zb-telemetry-serilog branch; when that work was re-implemented
on main as GatewayLogRedactorSeam the identity half was carried over and
the command-value half was not. Four unit tests kept the policy green, so
it read as wired while masking nothing.
No live leak today — no log statement currently emits a CommandValue
property — but the next one to do so would have written credential-bearing
MXAccess payloads (AuthenticateUser, WriteSecured, WriteSecured2) to every
sink in the clear, with passing tests suggesting otherwise.
Ports the missing half onto the seam: a non-null CommandValue is masked via
the existing policy, gated on CommandMethod. Value logging stays off — the
seam exposes no opt-in — so ordinary values are masked too, matching
RedactCommandValue's default. A null value stays null rather than becoming
the placeholder, and the property is never invented when absent.
Five tests added, three of which were red first on the leak itself
(operator01:hunter2 reaching the assertion unmasked). The other two pin
the null and absent guards. Identity redaction is untouched.
Full NonWindows suite: 785 pass, 45 pre-existing macOS NamedPipe-harness
failures unchanged from baseline (verified by stashing this change).
Build 0 warnings.
Family version-matrix alignment. No behaviour change — mxgw registers no Akka
checks, and 0.2.0's per-entry `data` object is emitted only when a check
publishes some, so its health payloads are byte-identical.
Note: this repo uses inline package pins, NOT central package management (there is
no Directory.Packages.props), so the version lives in the Server csproj.
Verified: Server builds 0 warnings.
Part of scadaproj docs/plans/2026-07-22-overview-dashboard-impl-plan.md Task 2.3.
0.2.3's Secrets.Ui ships ConfirmDeleteModal's own styles under
collision-proof zb-secrets-* class names. This host links no Bootstrap so
it never exhibited the invisible-modal defect, but it takes the fixed
line for parity; also rides over 0.2.1/0.2.2 (Akka-replicator fixes -
inert here, no replicator in use). Tests: 780 pass, 45 fail on macOS both
before and after the bump (NamedPipeServerStream multi-instance is
Windows-only - the fake-worker pipe harness cannot run on this platform);
zero delta from the bump.
Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
The suppression's own comment said 'Remove once an upstream fix ships'. It has:
SQLitePCLRaw.lib.e_sqlite3 2.1.12 patches this advisory within the 2.1.x line, so
the premise that no patched release existed is no longer true.
The Server project already resolved 2.1.12, but only incidentally - transitively via
ZB.MOM.WW.Auth.ApiKeys. An explicit PackageReference makes that floor intentional, so
a change to the Auth dependency graph cannot silently regress to the vulnerable 2.1.11.
Removing the suppression restores auditing for exactly that case.
Verified: forced restore of the NonWindows solution reports no NU1903; build is
0 warnings / 0 errors; test results unchanged from baseline (781 passed, 44 failed -
all 44 pre-existing macOS Unix-domain-socket path-length failures in the fake-worker
harness, identical count before and after).
Version hygiene + picks up the G-8 KEK-rotation surface.
NOT a security fix for this repo. An earlier version of this message claimed it
closed GHSA-2m69-gcr7-jv3q; that was wrong. A/B against the 0.1.2 baseline shows
SQLitePCLRaw.lib.e_sqlite3 already resolved 2.1.12, supplied transitively by the
pre-existing ZB.MOM.WW.Auth.ApiKeys 0.1.5 reference.
Note: src/Directory.Build.props still suppresses GHSA-2m69-gcr7-jv3q on the
now-outdated rationale that no patched e_sqlite3 exists. 2.1.12 is patched and
already resolving, so that suppression looks removable - verify separately.
Auth.ApiKeys 0.1.4 pulled SQLitePCLRaw.lib.e_sqlite3 2.1.11, which carries
high-severity advisory GHSA-2m69-gcr7-jv3q. This gateway was genuinely exposed --
verified 2.1.11 resolving before the bump and 2.1.12 after, with the vulnerability
scan now clean.
Auth 0.1.5 is 0.1.4 plus a transitive pin, so there is no API change here.
Suite unchanged from the documented baseline: 781 pass / 44 pre-existing worker-COM
failures on macOS, none Auth/ApiKey/SQLite related.
Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
The nightly (run #47) portable job failed CreateAsync_WhenFakeWorkerNeverSendsReady
_TimesOutAndKillsWorker: it asserts the factory's TimeoutException carries 'did not
complete startup' (thrown at the session's 1s startup timeout), but the test's
.WaitAsync(TestTimeout=5s) anti-hang net tripped first under CI load and surfaced
.NET's generic 'The operation has timed out.' — same commit passed on the push run
(#46), so it's a load-sensitive timeout race, not a regression. Raising runner
capacity to 4 makes concurrent-load flakes like this more likely.
Fix: give the two CreateAsync failure/timeout tests a dedicated 30s HangGuardTimeout,
far above the 1s semantic timeout under test, so the factory's own exception is
always the one observed. The net still catches a genuine hang.
Note: this class cannot run on macOS (named pipes -> Unix domain sockets fail on
the pipe path); verification is via CI Linux.
Surfaced during TST-25 acceptance verification: all CI runs on one shared
gitea-runner (maxParallel=1, co-located 10.100.0.35) interleaved with
dohertj2/lmxopcua, and Gitea 1.26 has no run cancel/delete API, so queue
latency is unbounded under cross-repo contention and the runner is a single
point of failure. Design: add a second/labelled runner; document the no-cancel
reality and the run-windev-ci.sh queue-bypass. Low/P2. Roll-ups updated.
Fixes found by running the TST-25 CI acceptance checks:
- CI SSH private key no longer leaks in windows-x86 logs (base64 secret + decode;
key rotated on windev). Masking confirmed (WINDEV_SSH_KEY: *** on run #38).
- Bootstrap git fetch/checkout now runs under the worktree lock, so concurrent
runs serialize instead of colliding on .git/index.lock.
- Deflaked SessionManagerTests fail-fast timing assertions (portable flake).
Merge target df7e20d verified GREEN via the local windev path (x86 Worker build
+ Worker.Tests 356 passed); main's own CI run provides the canonical evidence.
CI (portable, run #38) intermittently failed SessionManagerTests
.InvokeAsync_WhenTimeoutZero_FailsFastUnchanged with 'expected immediate
fail-fast but took 155ms'. The fail-fast paths throw synchronously under the
lock (GatewaySession.GetReadyWorkerClientAsync) and never enter the poll loop,
so the absolute <100ms wall-clock bound measured only host load, not behavior,
and flaked under CI contention.
- WhenWorkerFaulted_FailsFastWithBothStates: anchor the bound to a large (5000ms)
ready-wait timeout and assert fail-fast returns in < timeout/3 — a regression
that burned the timeout is still caught, but scheduling jitter can't trip it.
- WhenTimeoutZero_FailsFastUnchanged: drop the wall-clock assertion entirely
(a zero timeout has no wait window to burn); the error code, both-states
message, and InvokeCount == 0 already pin the immediate fail-fast.
Verified: the 3 SessionManager timing tests pass (net10.0).
Acceptance-check finding (concurrency): the worktree lock in windev-worker-ci.ps1
guarded the build stage, but run-windev-ci.sh's bootstrap did git fetch + git
checkout on the shared C:\build\mxaccessgw-ci clone BEFORE that lock was taken.
Two concurrent runs therefore collided on .git/index.lock at the bootstrap stage
(the second failed 'Another git process seems to be running', exit 1) instead of
the second waiting — defeating the lock's purpose for manual/degraded-mode overlap
or a future second runner. (The single Gitea runner serializes jobs, so CI pushes
never actually overlapped; this is defense-in-depth being restored.)
Fix: the bootstrap now acquires the same mkdir worktree lock around its fetch/
checkout and exports MXGW_CI_LOCK_HELD; windev-worker-ci.ps1 re-uses that lock
(skips re-acquire/release) when the env is set, and still self-locks for a
standalone/manual run. Now the second concurrent run waits at the bootstrap.
The windows-x86 acceptance check 'no key material in logs' failed: run #37's
job log printed the full WINDEV_SSH_KEY PEM in the step env echo. Gitea's
secret masker is line-oriented, so a multiline PEM rendered as one line with
literal \n escapes never matches the real-newline secret value and is not
redacted.
Fix: store WINDEV_SSH_KEY base64-encoded (single line) so the masker redacts
it to ***; run-windev-ci.sh auto-decodes a base64 PEM (still accepts a raw PEM
for local hand-testing). Drop the redundant WINDEV_SSH_KNOWN_HOSTS from the job
env (host keys are public and come from the committed windev.known_hosts pin),
removing another cleartext env line. Document the base64 requirement in the
bring-up README.
Operationally: the previously-exposed CI key has been rotated on windev
(old pubkey revoked from administrators_authorized_keys, new key installed) and
the Gitea WINDEV_SSH_KEY secret replaced with the new key's base64.
Restores the Windows/x86 test tier that had zero automation since the native
windows CI jobs were removed. A Linux CI job SSHes to windev (10.100.0.48),
checks out the pushed SHA in an isolated locked clone, and runs the x86 Worker
build + Worker.Tests (per push) / live-MXAccess smoke (nightly). Verified GREEN
under Gitea run #37 on d769244.
Joseph Doherty