Update the gateway docs for the central alarm monitor reversal:
Grpc.md replaces QueryActiveAlarms with the session-less StreamAlarms
RPC and notes AcknowledgeAlarm no longer needs a session;
Authorization.md maps StreamAlarmsRequest to events:read;
GatewayConfiguration.md adds the MxGateway:Alarms options block; and
GatewayDashboardDesign.md points the Alarms page at the central
monitor cache instead of a per-session subscription.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
gateway.md describes the always-on GatewayAlarmMonitor, the session-less
StreamAlarms feed, and session-less AcknowledgeAlarm. DesignDecisions.md
records that the v1 "one subscriber per session / no fan-out" rule is
superseded for the alarm subsystem: alarm state is gateway-wide, so the
gateway monitors it centrally and fans it out to all clients.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Browse renders the Galaxy hierarchy tree from IGalaxyHierarchyCache:
expandable areas/objects with attribute name, data type and the
alarm/historized flags, plus a name/reference filter. Right-click or
double-click an attribute to add it to a subscription panel that polls
live value, quality and source timestamp every two seconds.
Alarms lists the worker's currently-active alarm set via
IAlarmRpcDispatcher, defaulting to unacknowledged Active alarms with
filters for acknowledged alarms, area, severity range and text. It is
read-only and warns when alarm auto-subscribe is disabled.
Both tabs read live MXAccess data through a new singleton
DashboardLiveDataService that owns one shared, lazily-opened gateway
session (one worker) for the whole dashboard, re-opened transparently
if it faults or its lease expires.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The cross-language e2e matrix spawned one CLI process per operation —
~250 per client — paying a process (and, for the Java CLI, a full JVM)
cold-start every time. The Java leg alone ran ~16 minutes.
Each client CLI (dotnet, go, rust, python, java) gains a `batch`
subcommand: a single process that reads one command line from stdin,
runs it through the normal subcommand dispatch, writes the JSON result,
then a line containing exactly `__MXGW_BATCH_EOR__`. A failing command
writes its `{"error":...}` envelope and the loop continues.
run-client-e2e-tests.ps1 now launches one batch process per client and
pings every operation through its stdin/stdout, so startup is paid once
per client. The orchestration and assertions are unchanged; the parity
and auth phases now read the `{"error":...}` envelope instead of a
process exit code.
Full 5-client matrix with -VerifyWrite: ~15 min, down from ~35; the Java
leg dropped from ~16 min to ~2-3.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The cross-language client e2e matrix failed for dotnet and Java. Both
failures were in the harness, not the client code.
1. Per-call toolchain cold-start. The matrix issues ~250 CLI calls per
client; it invoked `dotnet run` / `gradle :mxgateway-cli:run` every
time, rebuilding and cold-starting the toolchain per call. Build each
CLI once up front (`dotnet build`, `gradle :mxgateway-cli:installDist`)
and invoke the compiled artifact directly. This alone fixes dotnet.
2. Worker event-channel overflow. The per-tag advise loop advises every
discovered tag with no StreamEvents consumer attached, so change
events accumulate in the worker event channel
(MxGateway:Events:QueueCapacity) until FailFast faults the worker.
dotnet's faster loop slipped under the window; the Java CLI's
process-per-call JVM cold-start did not. Every -DrainEveryTags advised
tags (default 15) the loop connects a short StreamEvents drain; the
gateway's per-stream producer empties the channel the instant a
subscriber attaches, so a small bounded read suffices.
Full 5-client matrix (dotnet, go, rust, python, java) now passes with
-VerifyWrite against a live gateway.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Second re-review pass at commit a020350 caught 48 new findings — including
one High-severity regression I introduced in the prior sweep — and fixed
them all in one parallel wave.
High (1)
- Client.Python-018: prior sweep set `license = "Proprietary"` in
pyproject.toml. setuptools >= 77 enforces PEP 639 and rejects the
string (it must be a valid SPDX expression), so `pip wheel .` and
`pip install -e .` both fail before any source compiles. Tests
still pass because pytest bypasses the build backend via
`pythonpath`. Dropped the invalid license string, kept the
`License :: Other/Proprietary License` classifier, and added
`tests/test_packaging.py` so a future regression of the same shape
is caught in CI.
Mediums (6)
- Worker-023: `HeartbeatStuckCeiling` (default 75s = 5x HeartbeatGrace)
on WorkerPipeSessionOptions bounds the in-flight-command watchdog
suppression so a truly stuck COM call still triggers StaHung
instead of permanently defeating the watchdog.
- Client.Rust-018: reverted Rust's `latencyMs` split so the
cross-language bench comparison is apples-to-apples again;
`failureLatencyMs` kept as Rust-only enrichment.
- Client.Java-021: applied Client.Java-002's terminal-state
serialisation pattern to DeployEventStream so close() arriving
after queue-overflow can't erase the overflow exception.
- IntegrationTests-017: teardown-parity test now uses a two-window
stability check after UnAdvise instead of strict equality against
the pre-UnAdvise count (which raced against in-flight events).
- IntegrationTests-019: new RecordingTestOutputHelper wraps every
log sink the WriteSecured live test owns (worker stdout/stderr,
gateway logs, direct WriteLine) so the credential is proven
absent from the full output buffer, not just the diagnostic
message.
- Tests-020: added MxAccessGatewayServiceConstraintTests coverage
for the previously-uncovered Write2Bulk and WriteSecured2Bulk
arms of WriteBulkConstraintPlan.SetPayload.
Lows (41 — highlights)
- Server: Galaxy glob cache eviction is race-free (Server-024);
GalaxyRepositoryGrpcService takes IGalaxyRepository (Server-025);
AlarmsOptions validated at startup (Server-026); Authorization.md
Constraint Enforcement snippet/prose enumerate the bulk write/read
family (Server-027); bulk-read-commands and bulk-write-commands
capability tokens added to OpenSession (Server-029);
NotWiredAlarmRpcDispatcher XML doc and missing scope-resolver and
state-machine tests cleaned up (023, 028).
- Worker: AlarmCommandHandler now invokes the same STA-affinity
guard the poll path uses, at every command entry (Worker-024);
RunAsync null-checks the runtime-session factory result
(Worker-025).
- Worker.Tests: shared LiveMxAccessOptInVariableName lives on
GatewayContractInfo (Worker.Tests-025); MxAccessSession.CreateForTesting
rejects production sinks (Worker.Tests-026); FakeRuntimeSession's
CancelCommandReturnValue serialised under lock (Worker.Tests-027);
Probes namespace lifted to MxGateway.Worker.Tests.Probes
(Worker.Tests-029); cancel-envelope sequence numbers monotonised
(Worker.Tests-030); docs/GatewayTesting.md gains a "Dev-rig Probes"
section (Worker.Tests-028).
- Tests: ManualTimeProvider consolidated into one TestSupport/ copy
(Tests-021); SessionManagerBulkTests adds a mid-flight cancellation
test backed by a TaskCompletionSource fake (Tests-022); companion
FakeWorkerProcess.WaitForExitAsync no longer fakes its exit signal
(Tests-023); constraint plan reply-count divergence pinned
(Tests-024).
- IntegrationTests: TryGetSession chain carries [MaybeNullWhen(false)]
end-to-end (IntegrationTests-018); abnormal-exit keyword set
tightened to pipe-disconnected/end-of-stream and the test now
asserts streamTask.IsFaulted (020, 021).
- Client.Dotnet: bench commands added to isLongRunning so the
default 30s wall-clock budget doesn't kill them (015);
BenchStreamEventsAsync observes the inner stream task on every
exit path (016).
- Client.Go: parseValue wraps strconv errors with flag context and
%w (017); bench loops honour ctx.Done() (018); galaxy-watch parses
RFC3339Nano with fractional seconds (019); runStreamEvents installs
signal.NotifyContext like runGalaxyWatch (020); five new CLI-level
table-driven tests cover the bulk/bench subcommands (021).
- Client.Java: toCompletable Javadoc rewritten to match the actual
cancellation contract Client.Java-015 established (022); stream-events
text path uses Long.toUnsignedString for worker_sequence (023);
bench-read-bulk no longer pollutes success-latency histogram with
failure durations (024); --shutdown-timeout CLI option propagates
through to ClientOptions (025); seven new MxGatewayCliTests cover
the bulk and bench commands (026).
- Client.Python: mxgateway_cli ships its own py.typed marker (019);
wheel-build smoke test added under tests/test_packaging.py (020);
README documents the Galaxy CLI parity gap explicitly (021).
- Client.Rust: RustClientDesign.md signatures match session.rs and
document the AsRef<str> read_bulk genericism (019);
next_correlation_id re-exported at the crate root, with a
property-style doc contract and an explicit disclaimer that the
literal textual format is not part of the contract (020).
- Contracts: BulkWriteResult comment names the actual
IConstraintEnforcer mechanism instead of "tag-allowlist filter"
(014); BulkReadResult gains explicit per-arm payload-population
documentation for the success vs failure cases (015).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Re-reviewed every module/client against the 10-category checklist
(REVIEW-PROCESS.md) at commit 1cd51bb, filed 72 new findings, and
fixed them in three priority waves (3 High, 17 Medium, 52 Low).
Highs
- Server-017: enumerate AcknowledgeAlarm / QueryActiveAlarms in
GatewayGrpcScopeResolver so non-admin keys can use them; document
the mapping in docs/Authorization.md; add interceptor tests.
- Client.Java-013: add the five missing bulk-method stubs to the
CLI FakeSession so the test module compiles on a clean tree.
- Client.Rust-013: fix the clippy::doc_lazy_continuation regression
in generated tonic code by reformatting the ReadBulkCommand proto
comment and scoping a #![allow(...)] to the generated submodules.
Mediums (highlights)
- Server: unify GatewaySession state-lock discipline (-015) and
make DisposeAsync race-safe against in-flight CloseAsync (-016);
add constraint-enforcement test coverage for the bulk-plan path
(-021).
- Worker: introduce StaRuntimeShutdownException so RunAlarmPollLoop
can distinguish graceful shutdown from a real STA-affinity
violation (-016); have the watchdog skip StaHung while
CurrentCommandCorrelationId is non-empty so a legitimate slow
ReadBulk no longer self-faults (-017).
- Tests: add per-method round-trip + cancellation coverage for the
11 GatewaySession bulk methods (-013); replace the real TCP probe
in GalaxyHierarchyCacheTests with an IGalaxyRepository fake
(-016).
- IntegrationTests: drive the StreamEvents writer in the live Write
test and assert OnWriteComplete (-012); add live tests for
Unadvise/RemoveItem/Unregister ordering, WriteSecured, and
abnormal worker exit (-014).
- Worker.Tests: replace MxAccessSession reflection with an internal
CreateForTesting factory (-016); cover WorkerCancel and
unexpected-body envelope branches (-017).
- Client.Java: cancel MxEventStream when close() races
beforeStart() (-014); return a CancellingCompletableFuture that
actually forwards cancellation through .thenApply chains (-015).
- Client.Python: drop the silent localhost-plaintext downgrade in
the CLI; require explicit --plaintext (-013).
- Client.Rust: stop bench-read-bulk from polluting success-latency
histograms with failed-call durations (-015); add coverage for
the five MalformedReply paths, the bulk-write helpers, the
Error::Unavailable mapping, and the unary-fault path (-016).
- Contracts: extend docs/Contracts.md with the bulk read/write
command family (-009).
Lows (highlights)
- Server: cap GalaxyGlobMatcher.RegexCache; align
WorkerAlarmRpcDispatcher missing-session handling; drop the
duplicate dashboard @page routes; refresh IAlarmRpcDispatcher
XML doc.
- Worker: surface SetXmlAlarmQuery COM failures; remove dead
subscriptionExpression / ExecutingCommand arms; preserve
factory-supplied runtime sessions; split MxAlarmSnapshot.cs into
three files.
- Tests: dispose the WebApplication in seven test classes; rebuild
FakeWorkerProcess.WaitForExitAsync against a real TaskCompletion
source; switch the heartbeat-expires test to ManualTimeProvider;
add InvariantCulture to the remaining DateTimeOffset.Parse sites;
document GalaxyFilterInputSafetyTests in GatewayTesting.md.
- IntegrationTests: comment fixes, RecordingServerStreamWriter
IDisposable, class-level [Trait], single-source ZB default
connection string.
- Worker.Tests: replace silent-return gating with LiveMxAccessFact
so absent env vars SKIP not pass; PascalCase rename of probe
[Fact]s; deterministic deadline test; new frame-protocol error
tests; ComputeTransitions diff-coverage; relocate dev-rig probes
to Probes/.
- Contracts: add round-trip coverage and per-field redaction /
Galaxy-identifier comments to the protos.
- Client.Dotnet: introduce clients/dotnet/Directory.Build.props so
TreatWarningsAsErrors / analysers apply; document
DiscoverHierarchyOptions and IMxGatewayCliClient; require typed
bulk-read handles in CLI; surface AcknowledgeAlarm transport
faults through Translate().
- Client.Go: kill dead code in alarms_test / fakeGalaxyServer /
runWriteBulkVariant; document the six new subcommands in
writeUsage; drain galaxy-watch events on limit; switch io.EOF
comparisons to errors.Is.
- Client.Java: shared shutdown helpers + new shutdownTimeout
option; regex-based credential redaction; Long.toUnsignedString
for uint64 sequence; doc fixes.
- Client.Python: combine duplicate imports; add coverage for
_percentile / bench-read-bulk / MAX_AGGREGATE_EVENTS /
_api_key_from_env; populate pyproject metadata and ship py.typed.
- Client.Rust: expose next_correlation_id() so CLI ping/close
stop hard-coding correlation IDs; resync RustClientDesign.md
with the current Session / Error surface and CLI subcommand set.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds five new MXAccess command kinds (WriteBulk, Write2Bulk,
WriteSecuredBulk, WriteSecured2Bulk, ReadBulk) that ride the existing
"one round-trip, per-entry results" bulk shape used by AddItemBulk and
SubscribeBulk today. MXAccess COM has no native bulk API; the worker
runs each bulk operation as a sequential loop on its STA, returning
one BulkWriteResult / BulkReadResult per requested entry so per-item
MXAccess failures surface as was_successful=false rather than throwing.
ReadBulk has no MXAccess analogue. The worker satisfies it by:
- Returning the last cached OnDataChange payload (was_cached=true)
when the requested tag is already in the session''s item registry
AND advised — the existing subscription is NOT touched, since the
caller did not create it.
- Otherwise taking the AddItem + Advise + wait-for-OnDataChange +
UnAdvise + RemoveItem snapshot lifecycle itself (was_cached=false)
and leaving the session exactly as it was. The wait pumps Windows
messages on the STA so the inbound MXAccess event can dispatch
while the executor still holds the thread.
The new MxAccessValueCache lives on each MxAccessSession, shared with
MxAccessBaseEventSink which populates it on every OnDataChange after
the event clears the outbound queue. Eviction on RemoveItem keeps
reused MXAccess handles from serving stale values from a previous
lifetime.
Gateway-side authorization wires WriteBulk/Write2Bulk to invoke:write,
WriteSecuredBulk/WriteSecured2Bulk to invoke:secure, ReadBulk to
invoke:read. The constraint-filter pipeline is refactored from a single
BulkConstraintPlan record into an abstract base plus three concretes
(SubscribeBulk, WriteBulk, ReadBulk), each owning its own denied-entry
merge so the dispatch site never branches on reply shape. A new
FilterWriteBulkAsync<TEntry> generic over the four write-entry shapes
runs CheckWriteHandleAsync per entry; denied entries surface as the
BulkWriteResult shape, preserving original-index order.
All five language clients (.NET, Go, Rust, Python, Java) gained the
five new methods following their existing bulk pattern, with regenerated
protobufs.
Tests added:
- MxAccessValueCacheTests (6 cases) — Set/TryGet, Remove resets the
version, TryWaitForUpdate signals on Set, pump step fires each poll.
- MxAccessBaseEventSinkTests — OnDataChange populates the cache,
ValueCache property exposes the bound instance.
- MxAccessCommandExecutorTests — four bulk-write variants (per-entry
success/failure, value+timestamp forwarding, secured user ids),
ReadBulk snapshot lifecycle on uncached tag (timeout surfaces as
was_successful=false), invalid-payload reply.
- GatewayGrpcScopeResolverTests — five new MxCommandKind cases.
- SessionManagerTests — WriteBulk and ReadBulk forwarding through
FakeWorkerHarness; ReadBulk forwards timeout_ms.
- Per-client (.NET, Go, Rust, Python, Java) — WriteBulk builds the
right command and returns per-entry results, ReadBulk forwards the
timeout and unpacks the was_cached flag.
Cross-language e2e CLI subcommands for the new bulks are deliberately
scoped out of this change (each of the five client CLIs would need
five new subcommands plus matching phases in
scripts/run-client-e2e-tests.ps1); coverage equivalent to the existing
bulk-subscribe coverage is provided by worker + gateway + per-client
unit tests.
Docs updated in the same commit: gateway.md (Public MXAccess Command
Surface), docs/DesignDecisions.md (new "Bulk Command Family" section
with the ReadBulk cache-then-snapshot rationale), and every client
README.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Running the matrix against a live gateway surfaced several issues:
- The write phase is now opt-in (-VerifyWrite, was -SkipWrite). It runs
right after register so only a small event backlog precedes the write,
and asserts the reliable OnWriteComplete signal (the written value is
not echoed back by a provider-driven attribute like TestChangingInt, so
the value compare is best-effort).
- Java was launched as bare "gradle", which .NET's Process.Start cannot
exec (it is gradle.bat) — resolve the launcher and run it via cmd.exe.
- The Java client's MxEventStream queue capacity was 16, which overflows
on any active session's backlog-replay burst; raised to 1024.
- The Rust stream-events CLI now renders the event family as the proto
enum name, matching the protobuf-JSON the other four clients emit.
Update docs/GatewayTesting.md for the reworked write phase.
Verified live: the full five-client matrix passes with -VerifyWrite.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Close the notable gaps in scripts/run-client-e2e-tests.ps1:
- Write round-trip: write a per-client sentinel value to a configurable
writable attribute, then assert it is echoed back through the event
stream. Extends the Rust mxgw-cli stream-events output with full
per-event JSON (itemHandle + protojson-shaped value) so all five
language clients run an identical value compare.
- Parity: assert an invalid item handle and an unknown session id are
rejected rather than silently succeeding.
- Auth rejection: assert open-session is rejected with a missing API key
and, when -RejectScopeApiKeyEnv is supplied, with an insufficient-scope
key.
- Parallel: -Parallel runs each language client as an isolated child
process and merges their JSON reports.
Update docs/GatewayTesting.md for the new phases and flags.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contracts-001: docs/Grpc.md still described "four MxAccessGateway RPCs" —
updated to the actual six (adding AcknowledgeAlarm and QueryActiveAlarms to
the handler and validation-rule sections).
Contracts-003 (Won't Fix): the finding is factually wrong — the <Protobuf>
item for mxaccess_worker.proto already sets ProtoRoot="Protos"; all three
items are consistent (confirmed back to the reviewed commit).
Contracts-004: corrected the stale GatewayContractInfo XML summary
("before generated protobuf contracts are introduced").
Contracts-005: no proto field/enum value was ever removed, so no reserved
ranges were invented. Added a wire-compatibility policy comment to all three
.proto files instructing future editors to reserve removed numbers.
Contracts-006: documented MxStatusProxy.success — it mirrors the COM
MXSTATUS_PROXY numeric success member, is not a boolean, and clients should
branch on category.
Contracts-007: added 13 round-trip tests covering galaxy_repository.proto
messages, bulk-subscribe payloads, and raw-value/IPC worker bodies.
Contracts-008: WorkerAlarmRpcDispatcher never assigns AcknowledgeAlarmReply.
status, so the old "native status" proto comment was misleading. Corrected
the hresult/status proto comments and documented the worker native_status →
public reply mapping in AlarmClientDiscovery.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Server-007: GalaxyHierarchyProjector re-filtered the whole hierarchy per
page (O(total) paging). It now memoizes the filtered list per cache-entry +
filter signature so subsequent pages are an O(pageSize) slice.
Server-008: WatchDeployEvents re-resolved browse subtrees and rebuilt globs
per streamed event. ResolveBrowseSubtrees is hoisted out of the loop and
GalaxyGlobMatcher caches compiled Regex instances per pattern.
Server-009: auth-store connections used no busy timeout or WAL. A new
OpenConnectionAsync applies journal_mode=WAL and a busy_timeout; all auth
call sites use it. docs/Authentication.md updated.
Server-010: the dashboard rendered Rotate/Revoke for revoked keys, where
Rotate silently reactivates them. ApiKeysPage now shows actions only for
Active keys. docs/Authentication.md updated.
Server-011: WorkerAlarmRpcDispatcher converted to a primary constructor and
brought in line with module conventions.
Server-012: CLAUDE.md corrected to the canonical *:* scope strings.
Server-013 (partly re-triaged): three named coverage gaps were already
closed; the genuine gap (WorkerExecutableValidator) is now covered.
Server-014: rewrote stale "alarm path not yet wired" comments in
MxAccessGatewayService to describe the production WorkerAlarmRpcDispatcher.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
MxCommandReply.payload has no by-name ack case: MX_COMMAND_KIND_ACKNOWLEDGE_
ALARM_BY_NAME reuses the acknowledge_alarm reply payload. Verified the worker
(MxAccessCommandExecutor.ExecuteAcknowledgeAlarmByName) and gateway
(WorkerAlarmRpcDispatcher) already implement this correctly — the gap was
purely undocumented contract asymmetry. Documented the reuse on the proto
oneof case and the AcknowledgeAlarmReplyPayload message comment (regenerating
the .NET contract), and in docs/AlarmClientDiscovery.md. Added
ProtobufContractRoundTripTests.MxCommandReply_AcknowledgeAlarmByName_Reuses
AcknowledgeAlarmPayloadCase to pin the contract.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
IntegrationTests-003: the live MXAccess smoke test asserted on the first
streamed event, which a registration/quality bootstrap event could occupy.
The recording writer now waits for the first event matching a predicate
(Family == OnDataChange).
IntegrationTests-004: the cleanup `finally` could throw and mask an original
assertion failure. Shutdown now routes through a helper that logs cleanup
exceptions instead of propagating them.
IntegrationTests-005: added live MXAccess parity tests — a Write round-trip
to an advised item, and an invalid-handle command surfacing the MXAccess
failure without a transport fault.
IntegrationTests-006: added live LDAP failure-path tests — wrong password
(no password leak), unknown username, and server-unreachable.
docs/GatewayTesting.md updated to describe the new cases.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Worker.Tests-003: removed the wall-clock `Elapsed < 2s` assertion from
InvokeAsync_WakesIdlePumpForQueuedCommand; the awaited completion against a
30s idle period already proves the wake event drove dispatch.
Worker.Tests-004: MxAccessStaSession.Dispose now joins the alarm poll task
after cancelling the CTS (consistent with ShutdownGracefullyAsync), and
Dispose_StopsAlarmPollLoop asserts deterministically instead of via Task.Delay.
Worker.Tests-005: undisposed MemoryStream instances across the frame-protocol
and pipe-session tests are now `using` declarations.
Worker.Tests-006: Dispose_StopsAlarmPollLoop now constructs MxAccessStaSession
with `using` so a failed assertion cannot leak the STA poll loop.
Worker.Tests-007: docs/WorkerFrameProtocol.md verification section corrected
to target MxGateway.Worker.Tests / MxGateway.Worker with -p:Platform=x86.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
IntegrationTests-001: documented the live Galaxy Repository test suite and
its MXGATEWAY_RUN_LIVE_GALAXY_TESTS / MXGATEWAY_LIVE_GALAXY_CONN gating in
docs/GatewayTesting.md.
IntegrationTests-002: documented the live LDAP test suite in
docs/GatewayTesting.md and added a concrete "Provisioning the GwAdmin group"
step to glauth.md so DashboardLdapLiveTests' GwAdmin-membership assumption
is reproducible.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The dashboard's API Keys page (list plus Create/Rotate/Revoke and the
create dialog) had no design-doc coverage even though Authorization.md
already documents the constraint model it exposes. Add an "API keys
page" section to GatewayDashboardDesign.md describing the table columns,
the LDAP-group-gated management actions, the one-time secret reveal, and
audit logging. Cross-link it from the constraint-enforcement section of
Authorization.md and the CLI section of Authentication.md so the two
key-management surfaces reference each other.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Skip-gated AlarmsLiveSmokeTests.Alarms_full_pipeline_round_trip ran
against the dev rig with the flip script firing
TestMachine_001.TestAlarm001 every 10s. Verified:
- Subscribe + 1st PollOnce yield real transition events
- Field-by-field decode correct (provider, group, tag, severity,
UTC timestamp, comment, type)
- SnapshotActiveAlarms reflects current state
- AcknowledgeByName(real identity) -> rc=0
- Pipeline keeps streaming transitions on the 10s cadence post-ack
Three production quirks surfaced and were fixed in
WnWrapAlarmConsumer:
1. SetXmlAlarmQuery is mandatory for reads. Skipping it (per the
earlier discovery-doc recommendation) makes the first
GetXmlCurrentAlarms2 fail with E_FAIL. The doc's claim that the
call is unnecessary because the round-trip echo is mangled was
wrong — mangled echo or not, the call is required.
2. SetXmlAlarmQuery breaks AlarmAckByName on the same consumer
instance (returns -55). Workaround: provision a parallel
"ack-only" wnwrap consumer that runs Initialize → Register →
Subscribe via the v1-prefixed methods, no SetXmlAlarmQuery.
Production WnWrapAlarmConsumer now holds two COM clients;
AcknowledgeByName always dispatches through the ack-only one.
3. AlarmAckByName has v2 (8-arg) and v1 (6-arg) overloads. The v2
8-arg overload returns -55 on this AVEVA build (apparently a
stub); the v1 6-arg overload works. Production now calls the
6-arg overload, discarding the proto's operator_domain and
operator_full_name fields. The proto contract keeps both for
forward-compat if AVEVA fixes the v2 method.
Bonus finding (not fixed here): AlarmAckByGUID throws
NotImplementedException on wnwrap. Reference→GUID lookup that we
initially planned to plumb is therefore not viable; all acks must
go through AlarmAckByName. WorkerAlarmRpcDispatcher.AcknowledgeAsync
already routes references through the by-name path, so this only
affects the GUID-input branch (which the worker tries first if the
input parses as a GUID — that branch will surface
NotImplementedException as MxaccessFailure if a client supplies one).
Threading caveat: wnwrap is ThreadingModel=Apartment, so the
consumer's internal Timer (firing on threadpool threads) blocks on
cross-apartment marshaling without an STA message pump. The smoke
test sidesteps this with pollIntervalMilliseconds=0 (Timer disabled)
+ manual PollOnce calls from the test STA. Production hosting will
route polls through the worker's StaRuntime in a follow-up; PollOnce
is now public so the wire-up is straightforward.
Test counts after this slice:
Worker: 195 pass / 4 skipped (live probes incl. new live smoke) /
1 pre-existing structure-fail (untouched)
Server: 308 pass / 0 fail
Solution builds clean.
docs/AlarmClientDiscovery.md "Live smoke-test discoveries" section
records all five findings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Switch the worker's alarm-consumer surface from `aaAlarmManagedClient.AlarmClient`
to `WNWRAPCONSUMERLib.wwAlarmConsumerClass` (CLSID 7AB52E5F-…) hosted by
`wnwrapConsumer.dll`. The new path returns alarm records as a BSTR XML
payload via `GetXmlCurrentAlarms2`, bypassing the FILETIME→DateTime
auto-marshaling that crashed `GetHighPriAlarm` with
ArgumentOutOfRangeException on every poll. Live captured 60/60 polls
clean against `\DESKTOP-6JL3KKO\Galaxy!DEV` while a System Platform
script flipped TestMachine_001.TestAlarm001 every 10s; the GUID,
priority, state (UNACK_ALM ↔ UNACK_RTN), and ASCII-formatted timestamps
arrived end-to-end.
Implementation:
- `Interop.WNWRAPCONSUMERLib.dll` generated via tlbimp, checked in under
`lib/` so dev boxes don't need the SDK to build.
- New `WnWrapAlarmConsumer` (replaces `AlarmClientConsumer`): owns a
500ms polling timer, parses `GetXmlCurrentAlarms2` output, diffs the
snapshot keyed by alarm GUID, and raises one
`MxAlarmTransitionEvent` per state change. Includes the
Initialize→Register-before-Subscribe ordering fix found during
Discovery probe runs.
- New library-agnostic types `MxAlarmSnapshotRecord` /
`MxAlarmStateKind` / `MxAlarmTransitionEvent` so the proto-build
path is testable without an AVEVA install.
- `AlarmRecordTransitionMapper` retired the COM-coupled
`MapTransitionKind(eAlmTransitions)`; new pure helpers
`ParseStateKind`, `MapTransition(prev, curr)`, and
`ParseTransitionTimestampUtc` cover XML decode + state-delta logic.
- `IMxAccessAlarmConsumer` event surface changed from
`EventHandler<AlarmRecord>` to `EventHandler<MxAlarmTransitionEvent>`
and `SnapshotActiveAlarms()` returns `MxAlarmSnapshotRecord` —
decoupling the interface from any specific COM library.
- Worker csproj drops `aaAlarmManagedClient` / `IAlarmMgrDataProvider`
refs; adds `Interop.WNWRAPCONSUMERLib`.
Tests:
- 36 new unit tests (state-string mapping, prev/current → proto kind
decision table, timestamp UTC reassembly, XML payload parser, 32-char
hex GUID round-trip) covering everything that doesn't touch the live
COM surface — all passing.
- Skip-gated `WnWrapConsumerProbeTests.ProbeWnWrapConsumer` archives
the live capture flow for regression / future probes.
Docs:
- `docs/AlarmClientDiscovery.md` "Option A — captured" section records
sample XML payloads, the mangled `SetXmlAlarmQuery` round-trip
(prefer `Subscribe` for filtering), the `GetStatistics`
AccessViolationException quirk, and the worker-integration outline.
Pre-existing failure noted (separate):
`MxAccessInteropReference_ExistsOnlyInWorkerProject` was already
failing on HEAD — the test project still references `ArchestrA.MxAccess`
for the Skip-gated discovery probes. Not regressed by this change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reflection on aaAlarmManagedClient.AlarmClient shows it implements
only IDisposable (no [ComImport] interface, no class GUID) and
has a single field "CwwAlarmConsumer* m_almUnmanaged". So
AlarmClient is a C++/CLI managed wrapper around a native C++
class -- NOT a COM-interop class. The DateTime conversion happens
INSIDE AVEVA's wrapper IL, not at the .NET-COM marshaling
boundary. There's no separate COM interface to QI to.
Revised approach (in docs/AlarmClientDiscovery.md):
A. wnwrapConsumer.dll -- separate standalone COM library AVEVA
ships at "C:\Program Files (x86)\Common Files\ArchestrA"
exposing WNWRAPCONSUMERLib.wwAlarmConsumerClass with
SetXmlAlarmQuery / GetXmlCurrentAlarms. XML-string output
bypasses FILETIME marshaling entirely. Best fit -- real COM,
self-contained, conventional production-grade approach.
B. Patch aaAlarmManagedClient.dll IL -- direct but modifies a
vendor binary, brittle to upgrades.
C. Reflect into m_almUnmanaged and call native vtable directly
-- requires reverse-engineering the C++ class layout.
Picking A. Probe restored to Skip; next commit starts the
wnwrapConsumer integration.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two findings that turn the alarm capture path on:
1. Subscription expression: \<MachineName>\Galaxy!<Area> is the
canonical AlarmClient subscription format per ArchestrA docs:
\Node\Provider!Area!Filter, with Provider literally "Galaxy"
(not the Galaxy name) and Node being the machine name. For
this rig: \DESKTOP-6JL3KKO\Galaxy!DEV catches alarms.
2. InitializeConsumer before RegisterConsumer — discovered
earlier; bug-fix for PR A.5's AlarmClientConsumer.
With these in place, GetHighPriAlarm returned a record on every
poll for 60s straight (117/117 calls). But every call throws
ArgumentOutOfRangeException: Not a valid Win32 FileTime, because
AlarmRecord has five DateTime fields (ar_Time / ar_OrigTime /
ar_AckTime / ar_RtnTime / ar_SubTime) and AVEVA writes sentinel
FILETIME values for unset ones (e.g., ar_AckTime on an
unacknowledged alarm). The aaAlarmManagedClient.dll auto-marshals
FILETIME -> DateTime and rejects out-of-range values.
GetStatistics still reports total=0 active=0 even with
GetHighPriAlarm returning records — those two APIs have
different views. The active read API for current alarms is
GetHighPriAlarm, not GetStatistics's change array.
So the consumer chain works. The blocking issue is now
extracting the payload past the AVEVA-shipped DateTime
auto-marshaling. Three approaches for the next PR:
1. Patch aaAlarmManagedClient.dll via ildasm/ilasm round-trip.
2. Define a custom [ComImport] interface with safe-blittable
types and Marshal.QueryInterface to it.
3. Use IDispatch late binding to bypass strong-typed marshaling.
Option 2 is cleanest; needs the AlarmClient COM IID.
Probe changes:
- Subscription expression set to \<MachineName>\Galaxy!DEV.
- GetHighPriAlarm tally counters (ok-with-record vs throw).
- 117 throws / 0 ok-with-record over 60s confirms alarms are
flowing continuously while the user's flip script runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sixth probe iteration with every consumer-side knob exhausted:
- Subscriptions tried (all rc=0): \Galaxy!, \Galaxy!*, \Galaxy!,
\Galaxy!TestArea, \.\Galaxy!.
- Read channels polled at 500ms: GetStatistics, GetHighPriAlarm,
SFCreateSnapshot + SFGetStatistics.
- Filters: priority 0..32767, qtSummary + qtHistory both tried,
asAlarmActiveNow.
- AlarmRecord pre-init to FILETIME epoch to dodge marshaler bug
on default(DateTime).
Result: every read API returns empty for the entire 60s window
even with TestMachine_001.TestAlarm001 firing every 10s and
aaObjectViewer confirming InAlarm transitions. The
aaAlarmManagedClient.AlarmClient is not the receive surface
AVEVA's alarm pipeline routes to in this Galaxy configuration.
The consumer chain is verified working end-to-end: Initialize +
Register + Subscribe all succeed, GetProviders finds the
provider, the WM 0xC275 heartbeat fires at 1Hz to AVEVA's
internal hwnd. There is simply no alarm data flowing through
this consumer surface.
Next investigation is not consumer-side: either find the SDK
aaObjectViewer's alarm panel uses, or query the historian
event storage directly. If alarms only flow via the historian
path on this customer's Galaxy, the worker's PR A.5 architecture
is a dead-end and A.2 needs a different transport.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tried every documented subscription knob with InitializeConsumer
present + provider visible at status 100:
- qtSummary AND qtHistory (the only eQueryType values).
- Priority 1..999 AND 0..32767.
- FilterMask/Spec asNone AND asAlarmActiveNow.
eAlarmFilterState is single-state-valued (asNone=0,
asAlarmActiveNow=1, asAlarmAcked=2, asShelved=3), not flag bits,
so the filter surface is exhausted.
GetStatistics continued to report total=0 active=0 codes=[7]
for every poll across all combinations.
User confirmation: the BoolAlarm extension on
TestMachine_001.TestAlarm001 is evaluating (the $Alarm.InAlarm
sub-attribute flips true/false in lockstep with the script
writes, visible in aaObjectViewer). So the consumer chain is
verified working end-to-end on our side. What's missing is
producer-side publication into the aaAlarmManagedClient stream.
Probable causes (config, not code):
- BoolAlarm extension's "publish to alarm manager" / "Active" /
"Enabled" flag may be off.
- Alarm-vs-event mode setting may have it routing to events,
not alarms.
- Platform alarm area may not match the consumer's subscription
scope.
Resolution path: check the BoolAlarm extension's config in System
Platform IDE; check aaObjectViewer's Active Alarms panel (not
attribute panel) to see if the alarm appears there.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
InitializeConsumer was the missing call. Adding it before
RegisterConsumer makes the \Galaxy! provider appear in
GetProviders (status 0 -> 100 within 500ms). Without Initialize,
GetProviders returns an empty list even though everything else
returns rc=0 (success).
Probe trace 2026-05-01:
InitializeConsumer -> 0
RegisterConsumer -> 0
GetProviders [after Register] -> count=0 list=[]
Subscribe('\Galaxy!') -> 0
GetProviders [after Subscribe] -> count=1 list=[ 0 \Galaxy!]
GetProviders [poll #1] -> count=1 list=[100 \Galaxy!]
Despite the provider being at "100% query complete" for the
entire 60s window, GetStatistics continued to report
total=0 active=0 codes=[7] -- no alarm transitions reached the
consumer even with a System Platform script flipping
TestMachine_001.TestAlarm001 every 10s during the run.
So the consumer chain works end-to-end. What's missing is alarm
traffic from the producer side. The next discriminator is
whether ObjectViewer (or another live consumer) sees the alarm
fire while the script runs.
API-ordering bug fix to apply to PR A.5's AlarmClientConsumer
regardless of how A.2 lands: AlarmClientConsumer.Subscribe
should call InitializeConsumer before RegisterConsumer (currently
omits Initialize entirely, which means the provider chain is
never visible from the worker either). That fix lifts a
fundamental bug independent of the polling-vs-callback question.
Probe changes:
- Added InitializeConsumer call before RegisterConsumer.
- Added LogProviders helper that logs only on change; called
after Register, after Subscribe, and on every poll. Easier
to spot when the provider chain transitions from empty to
populated.
- Restored Skip-gating after run.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extended AlarmClientWmProbeTests to call AlarmClient.GetProviders
after RegisterConsumer. Run 2026-05-01:
GetProviders -> rc=0 count=0 list=[]
Zero alarm providers visible to the consumer. This explains every
preceding probe run — no providers means no alarm events,
regardless of subscription expression or value writes upstream.
Even with a System Platform script flipping
TestMachine_001.TestAlarm001 every 10s during the run,
GetStatistics reported no transitions, no positions[] entries,
no field changes after t=0.85s.
Possible causes (dev-rig configuration, not code):
1. No $Alarm extension on the test bool — flipping the value
writes a value but doesn't fire an alarm.
2. AVEVA alarm-manager service (aaAlarmMgr or equivalent) not
running on this rig.
3. Process security context — providers registered under a
service account aren't visible to a consumer running under
a normal user account.
A.2 implementation is blocked on this until at least one provider
is visible. Once a provider exists, the polling-vs-callback
question is answerable in one probe run; without a provider both
paths return the same "nothing happening" answer.
Probe changes:
- Added in-process MxAccess Write attempt (TriggerWriteValue) —
hit TargetParameterCountException so the Write signature is
not (handle, item, value); reflection diag added but not
resolved. Now disabled in favor of external trigger.
- Added GetProviders enumeration after RegisterConsumer.
- Removed firePrint/clearPrint markers; probe is observe-only.
- Added ArchestrA.MxAccess reference to the test project.
Also updated docs/AlarmClientDiscovery.md with the
alarm-provider-visibility section explaining what's blocked
and why.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extended AlarmClientWmProbeTests.ProbeAlarmClientWmMessages to also
call GetStatistics every ~2s during the pump window. Re-ran on the
dev rig 2026-05-01:
- GetStatistics is safely callable from the same thread that did
RegisterConsumer + Subscribe. Every poll (9 calls / 20s window)
returned rc=0, no exceptions.
- Galaxy currently has zero active alarms. total=0 active=0
suppressed=0 newAlarms=0 across every poll. positions[] and
handles[] arrays were empty.
- changes=1 codes=[7] was constant across all polls, matching the
constant 1 Hz WM 0xC275 cadence — same heartbeat semantics
exposed through both the WM path and the pull API.
Confirms the polling design is mechanically viable: GetStatistics
threading-affinity is fine and the call is cheap. The remaining
unknown is whether GetStatistics populates positions[] / handles[]
with real entries when an alarm actually fires. Proving that
requires triggering an alarm — next probe is an MxAccess write to a
$Alarm-extended boolean tag (reference pending).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Added MxGateway.Worker.Tests/AlarmClientWmProbeTests.cs as a Skip-gated
runtime probe. Run on the dev rig 2026-05-01 against the live AVEVA
install (Galaxy reachable, no manual alarm fired). Findings:
- RegisterConsumer(hWnd, ...) and Subscribe("\Galaxy!", ...) both
return 0 (success). Calls are valid against the deployed assembly.
- A registered-message-class WM (ID 0xC275 in this OS session) fires
every ~1 second after Subscribe completes. Constant wParam=0x1100,
constant lParam=0x079E46D8 — looks like a heartbeat / keepalive,
not a per-change notification.
- Critically, this WM is delivered to AVEVA's own internal window
(hwnd=0x18032E), NOT to the consumer hWnd we registered. The
consumer window receives only the standard WM_CREATE / WM_DESTROY
sequence; no AVEVA traffic in between.
This invalidates the WM_APP-pump design previously documented. The
hWnd parameter to RegisterConsumer appears to be a registration
identity only — AVEVA's notification path runs entirely against
AVEVA's own internal window.
Two viable A.2 designs replace the previous one:
1. Polling. Call GetStatistics on a 500ms timer in the worker's STA
and react to whatever change set it reports. No window plumbing
needed. Latency floor = poll period. Matches AVEVA's own
internal heartbeat cadence.
2. Hook AVEVA's internal window. Discover AVEVA's own hwnd,
SetWindowSubclass on it, intercept WM 0xC275 on AVEVA's thread.
Higher fidelity, lower latency, but invasive and fragile across
AVEVA upgrades — likely a non-starter.
Recommendation in docs/AlarmClientDiscovery.md is option 1 (polling)
unless a follow-up probe with a real fired alarm shows AVEVA does
post change-specific WMs to a different hWnd.
Open follow-up probes documented:
- Fire a real Galaxy alarm during pump and check whether WM 0xC275
cadence changes or GetStatistics returns non-empty arrays.
- GetStatistics threading affinity test.
- Hook AVEVA's internal window 0x18032E.
- Decompile aaAlarmManagedClient IL for RegisterConsumer to find
whether WNAL_Register's callback surface is wrapped.
Test project changes:
- Added Reference to aaAlarmManagedClient + IAlarmMgrDataProvider
(Private=true so the DLL gets copied into bin for test load).
- Test-suite-wide: 127 real tests still pass; both alarm-related
Skip-gated tests skip cleanly.
Code change to the probe is additive — the worker is unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reflection probe of the deployed aaAlarmManagedClient.dll
(v1.0.7368.41290) on 2026-05-01 confirmed the public AlarmClient class
exposes zero public events. The PR A.5 design that AlarmClientConsumer
is built on (managed-event surface, no message pump) does not hold
against this assembly.
The actual notification mechanism is WM_APP messaging:
RegisterConsumer(hWnd, ...) takes a window handle because AVEVA's alarm
provider WM_APP-pokes the registered window, then GetStatistics +
GetAlarmExtendedRec pull the change set on each poke.
Practical impact:
- AlarmClientConsumer.AlarmRecordReceived has no production caller.
RaiseAlarmRecordReceived is invoked only from tests. Subscribe(...)
returns OK from RegisterConsumer + Subscribe but no notifications
reach the consumer at runtime because no window is attached.
- Until A.2 lands a hidden message-only window + WindowProc that routes
WM_APP into MxAccessAlarmEventSink.EnqueueTransition, the gateway's
MX_EVENT_FAMILY_ON_ALARM_TRANSITION family cannot carry events.
- AcknowledgeByGuid and SnapshotActiveAlarms are pull-style and remain
correct as written.
Changes:
- docs/AlarmClientDiscovery.md (new) — reflection probe summary, full
AlarmClient method list, open questions for A.2 implementation.
- AlarmClientConsumer.cs xmldoc — replaced the inaccurate "managed
event surface" claim with the WM_APP finding; flagged
AlarmRecordReceived as unreachable in production until the WM_APP
pump lands.
- MxAccessAlarmEventSink.cs xmldoc — replaced the "verify on dev rig"
hedge in the wiring plan with the resolved finding; expanded the
open-questions list (WM_APP message ID, wParam/lParam semantics, STA
affinity, subscription scope) so the next A.2 PR knows what the
dev-rig probe needs to answer.
Code-only no-op for the worker; worker builds clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Resolve 14 conflicts from popping local stash on top of origin's
eed1e88 + 8d3352f doc-comment additions (11 mechanical, plus
version.rs, DashboardAuthenticatorTests.cs, DashboardGalaxyProjector.cs)
- Fix 4 test files that used AGENTS.md as the repo-root sentinel
(now use CLAUDE.md, since AGENTS.md was removed in 4731ab5)
- Redirect 10 doc citations from AGENTS.md to the matching gateway.md
sections (Value Model, Status Model, Security, STA Worker Thread
Model, gRPC Layer rule, cancellation rule)
Verified: solution build clean, x86 worker build clean, 266/266
gateway tests passing, 121/121 worker tests passing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Rename 16 kebab-case docs to PascalCase per StyleGuide
- Move per-language client design docs from docs/ to clients/<lang>/
alongside their READMEs
- Add ## Related Documentation sections to 15 docs that lacked one
- Fix sentence-case violations in H3 headings (StyleGuide rule)
- Update cross-references in gateway.md, client READMEs, scripts,
and generate-proto.ps1 helpers to follow the new paths
- Add CLAUDE.md with build/test commands, the source-update
verification matrix, the parity-first contract, and pointers
to MXAccess and Galaxy Repository analysis sources
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Joseph Doherty