5e11b30507f3cc5f94997c0bd7434ff372221894
42 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Joseph Doherty
|
5e11b30507 |
[F56 resolved] subscribe paths now drive 0x33 DataUpdate frames
Root cause: `Session::subscribe` and `Session::subscribe_buffered_nmx`
were missing the `INmxService2::Connect` + `AddSubscriberEngine` RPC
pair that the .NET reference's `MxNativeSession.EnsurePublisherConnected`
(`cs:516-526`) issues before the first advise against a publishing
engine. Without those two RPCs, NmxSvc accepted the subscription
registration but the publishing engine never knew our engine was
subscribed — so it never dispatched DataUpdate frames back.
Diagnosis driven by wwtools/aalogcli reading
C:\ProgramData\ArchestrA\LogFiles. The user pointed at this tooling
which lit up the path.
Red herring: NmxSvc's `[Warning] NmxCallback->DataReceived ... failed
with error 0x{N}` log lines turned out to be normal log spam where N
is the bufferSize of the inbound call, not a real error code. The
.NET reference's own probe triggers identical entries while still
receiving DataUpdate frames successfully.
Fix:
- SessionInner::publisher_endpoints — per-session HashMap<(platform_id,
engine_id), ()> cache mirroring MxNativeSession._publisherEndpoints.
- Session::ensure_publisher_connected — issues Connect +
AddSubscriberEngine, once per publisher endpoint per session.
- Session::subscribe + subscribe_buffered_nmx — both call it before
the wire advise.
- subscribe_buffered_nmx — additionally issues AdviseSupervisory after
RegisterReference. The .NET reference's RegisterBufferedItemAsync
only calls RegisterReference, but on this AVEVA install
RegisterReference alone produces the registration result + heartbeat
callbacks without ever starting DataUpdate dispatch; AdviseSupervisory
unblocks the dispatch.
Live verification (`TestMachine_001.TestChangingInt`, a tag that
updates >1×/s):
cargo test -p mxaccess-compat --features live-windows-com \
--test plain_subscribe_live -- --ignored --nocapture
cargo test -p mxaccess-compat --features live-windows-com \
--test buffered_subscribe_live -- --ignored --nocapture
Both pass — `cmd=0x32` SubscriptionStatus + sequence of `cmd=0x33`
DataUpdate frames flow as expected. Tests assert on the raw
Session::callbacks() broadcast (not the typed Subscription::next
DataChange path) because the engine reports quality=Uncertain
value=null for this attribute on this Galaxy — the wire-level
subscription is what F56 was about, not the value content.
DcomCallbackSink reverted to S_OK return for both DataReceivedRaw
and StatusReceivedRaw (the bytes-processed / sentinel HRESULT
experiments during diagnosis turned out to be irrelevant — the
"failed with error 0xN" logs come from NmxSvc regardless of the
return value).
design/followups.md F49 + F56 + docs/M6-live-verification.md updated:
F56 resolved, F49 steps 1 + 4 + 5 pass live, steps 2 + 3 pending
(now executable on this fixture).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
df3457c54a |
[F56] subscribe / subscribe_buffered: split-form wire body + diagnose Galaxy fixture gap
Three real fixes + one architectural diagnosis:
1. Session::subscribe_buffered_nmx now sends the .NET-reference split
form on the wire:
item_definition = "<attr>.property(buffer)" (was: full reference)
item_context = "<object_tag_name>" (was: empty)
item_handle = SessionInner::next_item_handle.fetch_add(1)
(was: hardcoded 0)
Verified byte-identical against captures/082 + 094 by the existing
buffered_register_reference_parity unit tests. The
item_handle counter mirrors MxNativeCompatibilityServer's
_nextItemHandle++ at MxNativeSession.cs:613.
2. New live tests:
- tests/buffered_subscribe_live.rs (F49 step 1) — uses real Galaxy
metadata via SqlTagResolver + connect_nmx_auto, drives a
background writer at 500ms cadence to force value-changes,
drains DataChange events from Subscription.
- tests/plain_subscribe_live.rs — same harness over plain
Session::subscribe (NOT buffered), used to isolate whether
"no DataUpdate" is buffered-specific (it's not — both fail).
Both pull tracing-subscriber as a dev-dep so `RUST_LOG=trace`
surfaces dcom_sink + router activity.
3. mxaccess-galaxy/sql_resolver.rs: drop the inner-attribute
`#![cfg(feature = "galaxy-resolver")]` — the module-level cfg on
`pub mod sql_resolver` in lib.rs already handles this and Rust
1.85's clippy::duplicated_attributes lint flagged the duplicate
once mxaccess-compat dev-deps activated the feature.
4. F56 finding (diagnosis, NOT a bug fix): the engine on this Galaxy
install does not have an active value for TestChildObject.TestInt.
Confirmed by running the .NET reference's own probe:
dotnet run --project src/MxNativeClient.Probe -c Release \
-- --probe-session-subscribe --tag=TestChildObject.TestInt \
--subscribe-hold-seconds=10
...returns ONE 0x32 SubscriptionStatus (status=3 detail=3
quality=0x00C0 Uncertain value=null) and zero 0x33 DataUpdates —
matching the Rust port's symptom exactly. Not a Rust port bug,
not a wire-byte gap. F49 steps 1-3 need either an actively-
scanned tag or local Galaxy reconfiguration to scan
TestChildObject.TestInt.
Workspace tests + clippy clean under both feature configurations.
F56 entry in design/followups.md updated with the full diagnostic
chain so future-me / future-collaborators can pick it up without
re-tracing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
af15fe7587 |
[F49 step 1 + F56] callback router: peel envelope before parsing subscription / 0x11 frames
The router used to call NmxSubscriptionMessage::parse_inner directly on the COM-stub-delivered body, but the wire bytes arrive wrapped in a ProcessDataReceived envelope (46-byte header + optional 4-byte length prefix); parse_inner expects post-envelope bytes. Result: every 0x33 DataUpdate that ever arrived was silently dropped. Mirrors the .NET reference's MxNativeSession.OnCallbackReceived flow at cs:582-606 — three sequential parse attempts: 1. NmxOperationStatusMessage::try_parse_process_data_received_body (already wired) 2. NmxReferenceRegistrationResultMessage::try_parse_... (NEW — was missing) 3. NmxSubscriptionMessage::try_parse_process_data_received_body (NEW — was wrong) Adds: - NmxSubscriptionMessage::try_parse_process_data_received_body — peels envelope via NmxObservedEnvelope::parse_process_data_received_body_flexible, then dispatches to existing parse_inner. - NmxReferenceRegistrationResultMessage::try_parse_process_data_received_body — same shape, for the 0x11 registration-result frame. - Router branch for 0x11 — currently traces the assigned item_handle and drops the frame (matches the .NET reference, which fires a ReferenceRegistrationReceived event with no consumer in the codebase). - Router fall-through trace! when neither path matches, so future unparseable bodies surface in RUST_LOG=trace instead of vanishing. - DcomCallbackSink::forward — trace! per inbound callback so RUST_LOG=mxaccess_callback=trace surfaces opnum + size. - crates/mxaccess-compat/tests/buffered_subscribe_live.rs — F49 step 1 live test that drives subscribe_buffered + a 500ms-cadence writer. Also pulls tracing-subscriber as a dev-dep so the test can dump router activity. Existing router_task_decodes_callback_invoked_into_broadcast unit test updated to wrap its synthetic 0x32 body in an envelope so the new parse path actually accepts it. Live result: F56 — the buffered round-trip *registers* successfully (RegisterReference returns HRESULT 0; engine sends one 0x11 RegistrationResult + one 51-byte op-status per write, perfectly clocked) but the engine never sends a 0x33 DataUpdate. Rust-port- specific gap vs the .NET reference's working buffered path; root cause is likely a field-level difference in the RegisterReference body or a missing post-RegisterReference step. Captured as F56 in design/followups.md, blocking F49 step 1; F56's DoD is the same live test reporting >=3 DataChange arrivals. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
2fc327a8d5 |
[F55 Path A] DCOM-managed INmxSvcCallback sink
Replace the hand-rolled CallbackExporter (TCP listener + custom
OBJREF) with a real `windows-rs` `#[implement]` COM class for
INmxSvcCallback, marshalled via CoMarshalInterface. NmxSvc validates
the callback OBJREF by calling IObjectExporter::ResolveOxid against
the local RPCSS at 127.0.0.1:135; hand-rolled OXIDs aren't registered
there, which is why RegisterEngine2 returned RPC_S_SERVER_UNAVAILABLE
(1722) on every live attempt. CoMarshalInterface registers the OXID
with RPCSS automatically, so the SCM-side resolution succeeds.
Mirrors MxNativeSession.CreateRegisteredService (cs:624), which is
the .NET reference's working path:
ComObjRefProvider.MarshalInterfaceObjRef(callback,
INmxSvcCallback, DifferentMachine)
Layout:
- mxaccess-callback::dcom_sink — INmxSvcCallback + DcomCallbackSink
+ create_dcom_callback_sink_objref. Forwards inbound calls into
the same CallbackEvent::CallbackInvoked { opnum, body } shape the
legacy exporter produces, so callback_router stays path-agnostic.
- Session::from_nmx_client — branched on `windows-com`. Real DCOM
sink when on; legacy CallbackExporter when off (kept for unit
tests that run against an in-process fake NMX peer).
- SessionInner.dcom_sink_holder: Option<IUnknownHolder> — keeps the
COM ref alive for the session's lifetime; shutdown_nmx drops it.
- mxaccess-rpc + mxaccess-callback: windows-rs 0.59 → 0.62. The 0.59
#[implement] macro generates code that doesn't compile under
edition 2024; 0.62 is fixed.
Live result: cargo test -p mxaccess-compat --features
live-windows-com --test lmx_write_complete_live -- --ignored
--nocapture passes end-to-end. RegisterEngine2 OK, write
round-trips, OnWriteComplete fires with the captured MxStatus shape.
Unblocks F49 step 5; F55 marked Resolved in design/followups.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
c5d611d6fa |
[F12 partial + F55] hold IUnknown for client lifetime + diagnose RegisterEngine2 1722
**F12 partial improvement** (`mxaccess-rpc::IUnknownHolder` + `mxaccess-nmx`):
- New `IUnknownHolder` newtype that owns an MTA-resident COM proxy
with `unsafe impl Send + Sync`. Mirrors the .NET reference's
`ManagedNmxService2Client._activatedComObject` private field
(`cs:15`).
- New `activate_and_marshal_iunknown_objref(prog_id, ctx)` returns
`(Vec<u8>, IUnknownHolder)`. Existing
`marshal_activated_iunknown_objref` retained as a wrapper that
drops the holder (kept for inline-use callers).
- `NmxClient` gains an `activated_com_object: Option<IUnknownHolder>`
field, populated by `Self::create` from the new helper.
`Self::connect` / `Self::from_bound_transport` set it `None` (no
COM activation in those paths).
- Holding the IUnknown for the client's lifetime keeps the
SCM-tracked OXID valid; without it the COM ref count drops to
zero and the SCM may release the activated server-side instance,
making subsequent `ResolveOxid` / `RemQueryInterface` calls
return `RPC_S_SERVER_UNAVAILABLE`.
**F55 (new) — hand-rolled callback exporter rejected by RegisterEngine2**
Five-step instrumentation of `Session::connect_nmx_auto` proves all
six COM-activation / RemQI / final-bind steps succeed. The 1722
fault originates at `RegisterEngine2` itself:
```
from_nmx_client: callback hostname="DESKTOP-6JL3KKO" port=57886 obj_ref_len=162
from_nmx_client: callback obj_ref hex: 4d454f57010000...
from_nmx_client: RegisterEngine2 (31112, mxaccess.31112)
from_nmx_client: RegisterEngine2 FAIL: Transport(Fault { status: 2147944122 })
```
Status `0x800706BA` = `RPC_S_SERVER_UNAVAILABLE` wrapped as Win32
HRESULT.
**Critical finding: the .NET reference's `--probe-register-managed-callback`
(which uses the same hand-rolled `ManagedCallbackExporter` approach
as the Rust port) ALSO fails with the same `0x800706BA` fault.**
Only `--probe-session-write`, which uses
`ComObjRefProvider.MarshalInterfaceObjRef(callback, ...)` to build
the OBJREF via Windows DCOM proxy/stub marshalling, succeeds. So
this is an architectural artifact of the hand-rolled-callback
design, not a Rust port regression.
`design/followups.md` F55 entry documents the three resolution
paths (switch to DCOM-marshalled callback / hybrid / continue
investigating OBJREF rejection at NmxSvc).
F49 stays open with a refined diagnostic — the per-feature live
verification is gated on F55's resolution.
Workspace tests still 824 passing; clippy `-D warnings` clean
across both feature configurations.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
04c10babfb |
[F54 test] end-to-end smoke: write_with_handle ↔ callback_router boundary
Adds `write_handle_correlates_with_router_emitted_status` — the
closest-to-live test we can write without an AVEVA endpoint, pinning
the F54 boundary the C# `OnWriteComplete` callback ultimately depends
on.
The existing tests cover the layers individually:
- `write_value_with_handle_inserts_into_pending_ops` — write API
populates pending_ops with the right correlation id.
- `router_populates_operation_status_context_from_pending_ops_fifo`
— callback_router consumes a frame + the registry, emits a typed
OperationStatus with context attached.
- `drain_routes_write_status_to_on_write_complete` (mxaccess-compat)
— drain function routes Write op_kind to on_write_complete_tx.
What was missing: a test that combines the public `write_value_with_
handle` API with a real callback_router invocation against the SAME
`pending_ops` Arc the write populated. The new test:
1. Builds a Session via `connect_test_session`.
2. Calls `session.write_value_with_handle("TestObj.TestInt", ...)` —
gets a real `WriteHandle { correlation_id }` and a real entry in
`pending_ops` (no manual insertion).
3. Spins a parallel `callback_router` over the SESSION's
`pending_ops` Arc + a fake event_tx (the live exporter's
internal channel isn't reachable from tests; this is the
established workaround pattern from
`router_task_decodes_callback_invoked_into_broadcast`).
4. Injects the proven `WRITE_COMPLETE_OK` 5-byte frame.
5. Asserts the emitted `OperationStatus.context.correlation_id`
equals the cid the write returned, that op_kind is Write, that
reference is the original tag string, and that
`pending_ops` is now empty (one-shot popped).
This closes the integration-test gap the user flagged. Live AVEVA
verification still falls under F49.
Workspace 823 → 824 tests; clippy + rustdoc clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
4ff511bbed |
[F54] per-operation correlation + compat OnWriteComplete fan-out
Closes the residual that R3/R4 Path A's commit `c73a33e` deferred:
the OperationStatus.context field was always None because no
in-flight correlation map existed in SessionInner, and the
mxaccess-compat broadcast channels for OnWriteComplete /
OperationComplete were exposed on the public API but had no
fan-out task draining session events into them.
**mxaccess (Part 1 — per-operation correlation):**
- New `pending_ops: Mutex<HashMap<[u8; 16], OperationContext>>` on
SessionInner. Populated when `Session::write*` / `subscribe*`
dispatches an outstanding operation; entry removed when the
matching OperationStatus event fires (one-shot semantics).
- New `Session::write_with_handle` (and equivalents for the secured /
timestamped paths) returns a `WriteHandle { correlation_id }` so
consumers can correlate completions back to their originating
call. Existing `write` / `write_value` / etc. signatures unchanged
and delegate to the handle-returning variant.
- Callback router extended to look up `pending_ops` by correlation_id
on each operation-status event. When found, populates
`OperationStatus.context: Some(OperationContext { correlation_id,
op_kind, reference, retry_count: 0 })`. When not found, falls
through with `context: None` (verbatim-preserve per CLAUDE.md).
- New unit tests assert: matching correlation_id populates context,
unknown correlation_id leaves context None, the entry is removed
from `pending_ops` after one event fires.
**mxaccess-compat (Part 2 — compat-layer fan-out):**
- New `correlation_to_item: tokio::sync::Mutex<HashMap<[u8; 16], i32>>`
on LmxClientInner.
- `LmxClient::write` / `write_2` / `write_secured` / `write_secured_2`
call `Session::write_with_handle` (or equivalent) and insert
`correlation_id → item_handle` into the map before returning.
- `LmxClient::register` / `register_asb` spawn a background task that
drains `session.operation_status_stream()`. Per event, looks up
`correlation_to_item[event.context?.correlation_id]` to find the
item_handle, then routes:
- `OperationKind::Write` / `OperationKind::WriteSecured` →
`WriteCompleteEvent { server_handle, item_handle, statuses,
is_during_recovery }` into `on_write_complete_tx`.
- Other variants → `OperationCompleteEvent { ... }` into
`on_operation_complete_tx`.
- Removes the correlation_id from `correlation_to_item` after
firing (one-shot).
- Events with no matching item_handle (correlation_id not in map)
are dropped silently — no bogus item_handle=0 events.
- Task cancelled on LmxClient drop via `JoinHandle::abort` (matches
the existing `subscription_task` pattern).
- New unit tests cover: Write op routes to on_write_complete, Read
op routes to on_operation_complete, unknown correlation_id is
dropped.
Result: the C# `LMX_OnWriteComplete(int hLMXServerHandle, int
phItemHandle, ref MXSTATUS_PROXY[] pVars)` callback shape is now
end-to-end-achievable. A consumer calls `LmxClient::write(hServer,
hItem, value, userId)` and drains `client.on_write_complete()`; the
yielded `WriteCompleteEvent` carries the right `(server_handle,
item_handle, statuses, is_during_recovery)` tuple.
Public API: `Session::write_with_handle` + `WriteHandle` are new;
existing signatures unchanged. `cargo public-api` baselines
regenerated under `design/public-api/{mxaccess,mxaccess-compat}.txt`.
Workspace: 765 → 823 tests pass (~58 new tests from F54). Clippy
`-D warnings` clean. Rustdoc `-D warnings` clean.
F54 status in `design/followups.md` moved Open → Resolved.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
c73a33edd8 |
[R3/R4 Path A] mxaccess: port Lmx.dll FUN_10100ce0 synthesizer kernel
Path A landed for R3/R4. The byte->MxStatus synthesizer in Lmx.dll is
FUN_10100ce0 (`analysis/ghidra/exports/Lmx.dll.synthesizer-helpers2-decompile.md`),
a 4-byte u32 LE -> 4-tuple MxStatus decoder used by every NMX-frame
parser in Lmx.dll. The kernel is byte-deterministic and context-free,
so it ports as a pure function -- the operation-tracking state
machine the original verdict deferred is NOT required for synthesis.
Bit layout (per FUN_10100ce0 lines 21-24):
bit 31: success (-1 if set, 0 if clear)
bits 27..24: category (4 bits)
bits 23..20: detected_by (4 bits)
bits 15..0: detail (i16 -- low 16 bits, signed)
bits 30..28, 19..16: reserved/padding
Codec changes:
- MxStatus::from_packed_u32() / ::to_packed_u32() -- the kernel +
inverse for round-trip parity.
- MxStatus::from_nmx_response_code() -- the constructed-from-response-
code switch in FUN_1010bd10:741-770 (six proven mappings: 0x01, 0x02
-> CommunicationError + RequestingNmx; 0x03 -> ConfigurationError +
RequestingNmx; 0x04 -> ConfigurationError + RespondingNmx; 0x05 ->
CommunicationError + RespondingNmx; 0x1A -> CommunicationError +
RequestingNmx).
- MxStatusCategory / MxStatusSource: from_i16/to_i16 promoted to const
fn so MxStatus::from_packed_u32 can be const.
- NmxOperationStatusMessage::try_parse_process_data_received_body() --
thin wrapper that peels the outer NmxObservedEnvelope before
delegating to try_parse_inner. Mirrors
NmxOperationStatusMessage.TryParseProcessDataReceivedBody (.NET cs:20-32).
- NmxOperationStatusMessage::promote_to_typed() -- entry point that
returns the existing Status field. Documented as a no-op pass-through
for now (the 5-byte inner-body wire shape is NOT the same field as
the 4-byte packed-u32 the kernel decodes); kept for API symmetry.
- 22 new round-trip tests covering the kernel, the response-code
switch, the proven 0x00/0x41/0xEF completion bytes, and round-trip
for every canonical sentinel.
mxaccess (Session) changes:
- New OperationKind enum (Write/WriteSecured/Read/Subscribe/
Unsubscribe/Activate/Suspend/Other).
- New OperationContext struct (correlation_id, op_kind, reference,
retry_count) -- ground for the F54 follow-on per-operation
correlation work.
- New OperationStatus event type {raw, status, context,
is_during_recovery}, mirroring MxNativeOperationStatusEvent (cs:73-78)
with the typed-MxStatus addition.
- Session::operation_status_events() -> broadcast::Receiver<Arc<
OperationStatus>> + operation_status_stream() Stream variant.
- callback_router() now tries operation-status parsing first, falling
through to subscription messages -- matches MxNativeSession
.OnCallbackReceived dispatch order (cs:574,582,590).
- recover_connection() flips a recovery_active counter (Arc<AtomicU32>
shared with the router) so OperationStatus.is_during_recovery is
populated correctly. Mirrors MxNativeSession._recoveryActive
Volatile.Read at cs:573.
- 3 new router tests covering: status-word frame dispatch + typed
promotion to WriteCompleteOk; completion-only frames stay verbatim;
is_during_recovery is stamped from the live counter.
Per-operation context tracking (correlating completion frames back to
outstanding writes/subscribes via the correlation_id) is filed as F54
in design/followups.md. The synthesizer kernel itself is byte-
deterministic, so the kernel and the correlation work are decoupled.
Ghidra evidence (the next-ring xref walk beyond FUN_10114a90):
- analysis/ghidra/exports/Lmx.dll.set-attribute-result-xrefs.md --
xrefs to OnSetAttributeResult / CancelWithStatus / OperationComplete.
- analysis/ghidra/exports/Lmx.dll.vtable-data-xrefs.md -- vtable-slot
data xrefs for the virtual-dispatch path.
- analysis/ghidra/exports/Lmx.dll.synthesizer-decompile.md --
ScanOnDemandCallback::OperationComplete/MultipleOperationComplete
(FUN_1010b990), RemotePlatformResolver::OperationComplete
(FUN_1010dc80), and the constructed-from-responseCode synthesizer
in FUN_1010bd10 (lines 698-770). FUN_1010bd10 is the wire-frame
receiver that drives the synthesis.
- analysis/ghidra/exports/Lmx.dll.synthesizer-helpers-decompile.md --
FUN_10003fc0 (the <success %d category %d ...> formatter; confirms
the 4-tuple layout), FUN_1008f150 (dispatch helper).
- analysis/ghidra/exports/Lmx.dll.synthesizer-helpers2-decompile.md --
FUN_10100ce0 (the kernel itself), FUN_10100bc0 (3xu16 reader),
FUN_1005e580 (4-byte stream reader), FUN_1010ee00 (sister NMX-frame
parser using the same kernel).
- analysis/ghidra/exports/Lmx.dll.synthesizer-callers-xrefs.md --
caller graph; confirms the kernel is called from many wire-frame
parsers but each parser shares the single 4-byte decoder.
R3/R4 verdict updated in design/70-risks-and-open-questions.md from
"settled at verbatim-preserve" to "settled per Path A". F54 filed in
design/followups.md for the per-operation correlation work.
cargo build / test / clippy -D warnings / RUSTDOCFLAGS=-D warnings doc
all clean. cargo public-api baselines regenerated for mxaccess and
mxaccess-codec.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
1a1830f3bf |
[F47] mxaccess: unsubscribe skips UnAdvise for buffered subscriptions
Mirrors the .NET reference's `if (!subscription.IsBuffered)` guard
at `MxNativeSession.cs:361-381`. The Rust port previously emitted an
`UnAdvise` frame for both plain and buffered subscriptions; the
buffered server-side registration is unwound by the engine when the
`RegisterReference` handle goes away, so emitting an `UnAdvise` for
buffered entries is at best a no-op extra frame and at worst could
race with the engine's own teardown.
Fix: branch `Session::unsubscribe` on `SubscriptionEntry::mode` (the
discriminator F45 added). For `SubscriptionMode::Buffered { ... }`,
skip the `un_advise` call and proceed directly to registry cleanup.
For `SubscriptionMode::Plain`, retain the previous behaviour.
The registry-entry probe runs first (separate lock acquisition) so
the `is_buffered` decision doesn't hold the NMX-client mutex
unnecessarily — common case where the entry is plain still acquires
the NMX lock immediately after.
The metrics counter `record_unadvise()` still fires on every public
`unsubscribe` call regardless of mode — it tracks consumer-side
unsubscribe rate, not wire-frame rate. That matches what dashboards
expect from the public API.
New unit test `unsubscribe_skips_un_advise_for_buffered_subscription`
issues a plain subscribe (recorded as 1 RPC), mutates the registry
entry to `SubscriptionMode::Buffered`, calls unsubscribe, and
asserts the recorded RPC count stays at 1 (no UnAdvise emitted).
The existing `subscribe_populates_registry_unsubscribe_clears_it`
test serves as the negative control for the plain branch.
Workspace 794 → 795 tests; clippy clean; rustdoc clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
9b57cf8f3b |
[F45] mxaccess: recovery replay re-issues RegisterReference for buffered subs
`Session::recover_connection_core` previously walked
`SessionInner::subscriptions` and replayed every entry via
`AdviseSupervisory`, which lost the `.property(buffer)` registration
on buffered subscriptions — silently downgrading buffered → plain on
transport rebuild.
Fix:
- New `pub(crate) enum SubscriptionMode { Plain, Buffered { ... } }`
discriminator carried on each `SubscriptionEntry`. Buffered variant
retains the un-suffixed reference + the rounded interval (so the
re-issued buffered registration matches the original cadence) +
the empty `item_context` / zero `item_handle` matching the wire
send.
- `Session::subscribe` (plain path) records `SubscriptionMode::Plain`.
`subscribe_buffered_nmx` records `SubscriptionMode::Buffered { ... }`.
- `recover_connection_core` matches on `entry.mode`. Plain branch
unchanged. Buffered branch re-applies `.property(buffer)` via
`to_buffered_item_definition` (idempotent), rebuilds the original
`NmxReferenceRegistrationMessage` with the saved correlation id +
`subscribe = true`, and dispatches `register_reference` (kind=
ItemControl, inner command 0x10) against the replacement
transport. Mirrors `MxNativeSession.ReAdviseSubscription`
(`MxNativeSession.cs:538-569`).
New unit test `recover_connection_replays_buffered_subscription_via_
register_reference` synthesises a buffered registry entry, installs a
`RebuildFactory` pointing at a recording NMX server, drives
`recover_connection`, then asserts the recorded `TransferData` carries
inner command `0x10` (NOT `0x1f`) with the `.property(buffer)`-
suffixed item_definition + the saved correlation id + subscribe=true.
Side-finding worth filing separately: `Session::unsubscribe`
unconditionally calls `un_advise` for both plain and buffered
entries, but the .NET reference's `Unsubscribe`
(`MxNativeSession.cs:361-381`) skips `UnAdvise` for buffered
(`if (!subscription.IsBuffered)`). Out of scope for F45 (recovery-
only); will file as F47.
Public API unchanged. `SubscriptionMode` + `SubscriptionEntry` stay
`pub(crate)` — `cargo public-api -p mxaccess` baseline is unchanged.
Workspace 793 → 794 tests; clippy clean; rustdoc clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
f0c9dd2214 | rust: add version specifiers to workspace path deps for cargo publish | ||
|
Joseph Doherty
|
ad1cf2351c |
[F36 + F40 + F44] M6 wave 1: subscribe_buffered (NMX) + metrics + evidence
Three M6 sub-followups landed in this wave (sub-agent worktrees +
manual reconciliation in main):
**F36 — Session::subscribe_buffered (NMX) per R2 single-sample**
- `BufferedOptions::rounded_update_interval_ms()` — 100ms rounding
helper mirroring MxNativeCompatibilityServer.cs:638
((updateInterval + 99) / 100) * 100, saturating on overflow.
- `Session::subscribe_buffered` (public, lib.rs:604) delegates to
the new private `subscribe_buffered_nmx` which uses the buffered
RegisterReference path: item_definition suffixed with
`.property(buffer)`, subscribe=true (no separate
AdviseSupervisory follow-up — verified against capture 082).
- Per R2 verified at wwtools/mxaccesscli/docs/api-notes.md the wire
semantic is single-sample-per-event with a server-side cadence
knob; rounded_ms is held client-side only (native MXAccess does
not emit a separate SetBufferedUpdateInterval RPC, verified by
absence in 079/082 captures).
- New crates/mxaccess/examples/subscribe-buffered.rs.
- New crates/mxaccess-codec/tests/buffered_register_reference_parity.rs:
4 tests (capture 079/082 round-trip, suffix helper, constructive
forward-build vs capture 082).
**F40 — Optional metrics feature**
- New crates/mxaccess/src/metrics.rs (275 lines): `pub(crate)`
thin wrappers (`record_write_latency`, `record_read_latency`,
`inc_writes`, `inc_reads`, `inc_advises`, `inc_recovery_*`,
`set_active_subscriptions`, etc.) that compile to no-ops under
`#[cfg(not(feature = "metrics"))]`. Call sites in session.rs +
asb_session.rs invoke them unconditionally; the gate is inside
the wrapper.
- `metrics = { version = "0.24", optional = true }` added to
workspace + mxaccess crate Cargo.toml.
- Default build: zero metrics dep, zero runtime cost.
**F44 — Buffered batch + suspend capture decode evidence**
- New docs/M6-buffered-evidence.md: per-capture summary for
077, 079, 080, 081, 082, 094 — call sequence, key wire bytes,
R2/R5 verdict.
- R2 confirmed silently as "not a real risk" — single-sample
observed across 079/080/082/094.
- R5 trigger conditions documented from capture 077: AdviseSupervisory
+ Suspend pair, 1-second intervals, succeeds on enum attributes.
- design/70-risks-and-open-questions.md R2/R5 status updated.
Workspace: 759 → 792 tests, clippy clean, rustdoc -D warnings clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
e79e289743 |
[F42] cargo doc --workspace --no-deps clean (0 warnings)
Fix all 33 rustdoc warnings across the workspace: - Unresolved intra-doc links: rewrite [`name`] → either backtick text (when not actually a link) or fully-qualified `[Type::method]` / `[crate::module::name]` form. Affected: mxaccess-codec (asb_variant, item_control, metadata_query, observed_write_template, reference_handle, write_message), mxaccess-rpc (pdu), mxaccess-nmx (client), mxaccess-asb-nettcp (nmf), mxaccess-callback (exporter), mxaccess (asb_session, session, lib). - Bracket-text being interpreted as link refs (e.g. `body[17]` → `` `body[17]` ``). - Private-item references in public docs (CALLBACK_BROADCAST_CAPACITY, recover_connection_core, mxvalue_to_writevalue) reduced to backtick-text since they aren't part of the public API. `RUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-deps` now exits clean. Workspace 759 tests pass; clippy clean. Defers `#![warn(missing_docs)]` lint to a future pass — the cleanup target is the broken-link warnings, which are signal; missing-docs would surface hundreds of low-priority public-item gaps that are out of scope for this F-number. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
34045c2f6d |
[F37] mxaccess: AsbSession::subscribe_buffered returns Unsupported
ASB has no `SetBufferedUpdateInterval` analogue — the per-monitored-
item `MinimalMonitoredItem::sample_interval` plays the cadence-knob
role. Calling `subscribe_buffered` on an ASB session now returns
`Error::Unsupported { transport: TransportKind::Asb, operation: ... }`
synchronously, without touching the wire.
The error-construction logic is split into a free fn
`unsupported_subscribe_buffered_error()` so the gate's exact shape
is unit-testable without spinning up a live authenticator + transport
fake. New unit test asserts both the variant tag and that the
operation message names the unsupported method + hints at the
`sample_interval` analogue.
Workspace 758 → 759 tests, clippy clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
fb40e4c20b |
[F34 partial] mxaccess-asb: fix collect_asbidata_payloads + add Active flag
rust / build / test / clippy / fmt (push) Has been cancelled
Investigation via examples/asb-relay.rs middleman captured the full
S→C bytes of a working PublishResponse from the .NET probe against
MxDataProvider. Decoder fix verified by regression test against the
captured fixture; one further wire-format gap surfaced and is filed.
Closed in this commit:
1. collect_asbidata_payloads filtered out empty <ASBIData/> elements
so positional payload[N] indexing collapsed when Status was
empty-but-present. The wire form for PublishResponse is:
<Status><ASBIData/></Status> ← empty placeholder
<Values><ASBIData>{bytes}</ASBIData></Values>
Our decoder lost the positional info and read Values as Status,
then panicked on the malformed parse. Fix: always push every
<ASBIData> element (empty or not) so payloads[0]=Status and
payloads[1]=Values stay aligned. New regression test
tests/publish_capture.rs runs the full decode chain over the
captured wire bytes (305-byte frame at
tests/fixtures/publish-response-with-value.bin) and asserts
values.len() == 1.
2. MinimalMonitoredItem.active: Option<bool> + new with_active()
constructor. The .NET reference's MxAsbDataClient.AddMonitoredItems
defaults to active: true (cs:441). Without <Active>true</Active>
on the wire, MxDataProvider treats the subscription as inactive
and Publish polls return empty Values. Both binary build and
canonical XML emitters now conditionally emit <Active> when
active.is_some(). Shared push_monitored_item_body helper
eliminates the duplicate MonitoredItem encoder between
AddMonitoredItems and DeleteMonitoredItems builders.
3. SampleInterval unit: clarified as **milliseconds** in
MinimalMonitoredItem.sample_interval doc + the example
(sample_interval_ticks → sample_interval_ms = 1000). Matches the
.NET reference's `ulong sampleInterval = 1000` default.
Open: F34's deeper finding — `MonitoredItem`'s wire schema is
DataContract field-suffix names (`activeField`, `bufferedField`,
`itemField`, `sampleIntervalField`, etc., per the per-session NBFX
dictionary the .NET probe declares), NOT XmlSerializer property
names (`Active`, `Buffered`, `Item`, `SampleInterval`). Our binary
NBFX builder still uses the property names, so MxDataProvider
silently fails to register monitored items — successField=true with
a 0-length Status array. The fix needs a complete rebuild of
build_add_monitored_items_request_body and
build_delete_monitored_items_request_body to use the field-suffix
names plus emit the *Specified siblings (activeFieldSpecified,
idFieldSpecified, etc.) as their own elements. The HMAC canonical
XML side is unaffected (XmlSerializer naming is correct there;
verified byte-equal to .NET via the F28 fixtures). Detailed in
design/followups.md F34's "Open" section.
Live verification of the F34-partial bonus context:
- Read still returns 99 end-to-end via canonical XML signing.
- AddMonitoredItems still returns Status[0] = 0 items
(server doesn't recognize our DataContract-misnamed payload).
- Publish still returns 0 values (the F34-open consequence).
- All other 13 canonical-XML signed ops succeed at the request
level (no SOAP faults, no HMAC rejections).
Workspace: mxaccess-asb 95 → 96 (+1 capture-driven decoder test);
default-feature clippy clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
0771664092 |
asb: SampleInterval unit fix + F34 followup for Publish-decoder gap
rust / build / test / clippy / fmt (push) Has been cancelled
Investigation triggered by "Publish returns 0 values where .NET sees real
values" against the local AVEVA install.
Three findings:
1. SampleInterval unit: the wire field is **milliseconds**, not 100-ns
ticks. The .NET reference (MxAsbDataClient.cs:441) defaults to
`ulong sampleInterval = 1000` and the probe passes `subscribeSampleMs`
directly through that surface. Sending 10_000_000 (1s in 100-ns ticks)
makes MxDataProvider schedule the next sample ~2.8 hours out; Publish
polls always come back empty until the misinterpreted timer expires.
Fixed in `examples/asb-subscribe.rs` (sample_interval_ticks →
sample_interval_ms = 1000) and clarified in
`MinimalMonitoredItem.sample_interval`'s doc comment with the live-2026-05-06
evidence.
2. result_code=32 is `AsbErrorCode.PublishComplete`
(`AsbResultMapping.cs:37`) — informational, not a fatal error. .NET's
`ToResult` (cs:122-129) explicitly treats it like Success.
`ArchestrAResult.ErrorCode` and `ResultCode` are aliases for the same
`resultCodeField` (cs:424-434), so `publish[i]_error=0x00000020` in
the .NET probe trace = `result_code=Some(32)` in our trace = the same
thing. Already handled correctly via the F26 narrower-bail fix
(commit
|
||
|
Joseph Doherty
|
983f02921c |
asb-subscribe example: drive every canonical-XML signed op live
rust / build / test / clippy / fmt (push) Has been cancelled
Extends the example to exercise the full data-plane through the
new canonical-XML signing path (F28 step 2). Each op is announced
with a "[canonical XML <Op>]" tag in the trace so the lifecycle is
self-documenting:
Connect → Register → Read → Write → CreateSubscription
→ AddMonitoredItems → Publish × N → PublishWriteComplete
→ DeleteMonitoredItems → DeleteSubscription
→ UnregisterItems → Disconnect → SendEnd
Per-section errors are caught and logged but don't abort the
lifecycle — a failed Publish still reaches Disconnect cleanly so
the server-side pending-connection table doesn't fill up.
New env vars MX_RUN_WRITE / MX_RUN_SUBSCRIBE / MX_SUBSCRIBE_COUNT
(defaults: run, run, 3) for opting into / sizing the optional steps.
Live verification on this host (this turn, first run):
register status: 1 item(s); result_code=Some(0) success=Some(true)
TestChildObject.TestInt = AsbVariant{type_id:4,length:4,payload:[99]}
write status: 0 item(s); result_code=Some(0) success=Some(true)
subscription_id=2 result_code=Some(0) success=Some(true)
add status: 0 item(s); result_code=Some(0) success=Some(true)
publish: 0 value(s); result_code=Some(32) success=Some(false)
publish_write_complete: 0 write(s); result_code=Some(0)
delete_monitored_items ok
delete_subscription ok
unregistering ... disconnecting
All 13 canonical-XML-signed ops accepted by MxDataProvider — no SOAP
faults, no HMAC rejections, no decode errors. F28 step 2 verified
end-to-end against the live AVEVA install.
Bonus fix: F26 stream's publish_loop bail logic narrowed.
The original F33 bail-on-any-non-zero-result_code was over-aggressive:
.NET's MxAsbClient.Probe shows that result_code=32 (= 0x20) fires on
*every* Publish poll while values are still being delivered. Updated
publish_loop and the example's Publish loop to bail only on
RESULT_CODE_INVALID_CONNECTION_ID (1) — that one truly means the
session is desynced. Other non-zero result_codes are informational
and the loop continues draining.
New public re-export: mxaccess_asb::RESULT_CODE_INVALID_CONNECTION_ID
(was crate-private under the operations module).
The InvalidConnectionId transient still hits after many back-to-back
test runs against a long-running MxDataProvider — the pending-
connection table fills up — same well-documented behaviour from F32.
A 30-second cool-down restores reliability in our experience.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
ff4ea4d5a9 |
[F16] mxaccess: real Session::recover_connection (re-bind + re-advise)
rust / build / test / clippy / fmt (push) Has been cancelled
Closes F16. Replaces the wave-2 no-op recover_connection with the
full .NET-equivalent shape (MxNativeSession.cs:399-474). Three
pieces:
1. Subscription registry on SessionInner.
New subscriptions: Mutex<HashMap<[u8; 16], SubscriptionEntry>>
tracks every active advise. subscribe() inserts after a successful
AdviseSupervisory; unsubscribe() removes on the success path only
(failed UnAdvises stay registered so next recovery replays them).
The consumer's Subscription handle still holds the BroadcastStream;
the registry is purely for AdviseSupervisory replay.
2. Pluggable RebuildFactory.
New public typedef:
pub type RebuildFactory = Arc<
dyn Fn() -> Pin<Box<dyn Future<Output = Result<NmxClient,
NmxClientError>>
+ Send>>
+ Send + Sync,
>;
Installed via Session::set_recovery_factory(factory);
queryable via has_recovery_factory(). Kept separate from
connect_nmx / connect_nmx_auto so existing constructors stay
non-breaking — consumers opt in by calling the setter
after-the-fact.
3. Real recover_connection + recover_connection_core.
recover_connection is the retry loop (mirrors cs:399-440): for
attempt in 1..=policy.max_attempts, emit RecoveryEvent::Started
→ call recover_connection_core → on Ok emit Recovered + return,
on Err emit Failed{will_retry, error}, sleep policy.delay, retry,
or bubble the last error.
recover_connection_core mirrors cs:442-474: rebuild NMX via the
factory → RegisterEngine2 with the saved callback_obj_ref → optional
SetHeartbeatSendInterval → snapshot the registry under the lock,
replay AdviseSupervisory(correlation_id) for each entry → atomically
swap *nmx_lock = replacement. Old NmxClient drops at end of scope,
closing its TCP transport.
Subscription correlation ids are preserved across the swap so the
consumer's Subscription stream continues to receive on its existing
broadcast filter. The CallbackExporter stays bound across recoveries
— no TCP listener re-bind.
R15's "long-lived connection task" was listed as a hard prereq, but
the existing Mutex<NmxClient> already serialises concurrent ops
during the rebuild — recover_connection_core holds the inner mutex
during the swap, concurrent ops just wait. Functionally equivalent
to the long-lived-task design.
New ConfigError::RecoveryNotConfigured returned when
recover_connection is called without a factory installed. New
public re-export: RebuildFactory.
Tests (mxaccess 65 → 67):
- recover_connection_without_factory_returns_recovery_not_configured
- recover_connection_with_always_failing_factory_exhausts_attempts
(pins (Started, Failed)×3 + final will_retry=false + bubbled
TransportFailure)
- subscribe_populates_registry_unsubscribe_clears_it
- recovery_events_supports_multiple_subscribers (updated for the
new factory-required path)
connect_nmx_auto-side auto-population of the factory (capturing the
ntlm_factory + discovered (addr, service_ipid) so consumers don't
re-author the closure) is a future polish — not required to close
F16.
design/followups.md: F16 moved to Resolved.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
cfeb761092 |
[F33] mxaccess-asb: complete InvalidConnectionId tolerance propagation
rust / build / test / clippy / fmt (push) Has been cancelled
Closes F33. Final commit in the three-step F33 closure ( |
||
|
Joseph Doherty
|
f2f22dfcd1 |
[F26 stream] mxaccess: AsbSession::subscribe — Stream<Item = MonitoredItemValue>
rust / build / test / clippy / fmt (push) Has been cancelled
Closes the last F26 stub from the M5 status block. New
AsbSession::subscribe(subscription_id) returns an AsbSubscription
that impls Stream<Item = Result<MonitoredItemValue, Error>>. An
internal tokio::spawn'd publish-loop drains the subscription queue
via the existing AsbSession::publish() and fans each
PublishResponse's `values` array out as individual stream items.
Termination semantics:
- Drop of AsbSubscription calls JoinHandle::abort() — the publish
task stops draining the server-side queue (the .NET reference
pattern at MxAsbDataClient.cs uses the same task-cancellation
shape).
- Transport error from publish() is delivered as the final stream
item; the loop returns and the channel closes.
- Receiver-drop (consumer stops polling) is detected when
tx.send returns Err — the loop exits without making more
publish calls.
The inner publish_loop helper takes any FnMut() -> Future<Result<...>>
so it's testable in isolation (no live ASB endpoint required).
Per-item ItemStatus from the server is intentionally not surfaced
on the stream: the field is opaque per-item and rarely actionable
for the streaming consumer. A richer struct can wrap each value if
that need surfaces.
3 new tests pin:
- asb_subscription_is_stream_send_unpin (compile-time bounds);
- publish_loop_delivers_values_then_terminates_on_error
(3 Ok values from 2 batches, then 1 terminal Err);
- publish_loop_exits_when_consumer_drops_channel.
New deps used (already in mxaccess Cargo.toml): futures_util::Stream,
tokio::sync::mpsc, tokio_stream::wrappers::ReceiverStream,
tokio::task::JoinHandle.
Workspace: 718 → 721 tests. Default-feature clippy clean.
mxaccess crate-level doc updated to drop the "stubbed for next F26
iteration" note for the subscription stream.
design/followups.md F18 M5 status block updated: F26 stream
subscription marked resolved.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
8e695b9347 |
[F12 wrapper + F32 close] Session::connect_nmx_auto + close M5 type-matrix DoD
rust / build / test / clippy / fmt (push) Has been cancelled
Two related closures in one commit:
1. Session-level wrapper around F12: new
`mxaccess::Session::connect_nmx_auto(ntlm_factory, options,
resolver, recovery)` gated on a new `mxaccess/windows-com` feature
(which propagates `mxaccess-nmx/windows-com`). Drives
`NmxClient::create` (the F12 COM-activation factory) for the
`(host, port, service_ipid)` discovery, then funnels into the
shared post-NMX-bind orchestration. Refactored `connect_nmx` to
extract steps 1+2+4+5 into a private `from_nmx_client` helper —
both `connect_nmx` and `connect_nmx_auto` reuse it so the
`CallbackExporter` + router + `RegisterEngine2` + heartbeat policy
stays in one place. The .NET `MxNativeSession.Open` shape
(`MxNativeSession.cs:127-147`) is now reproduced end-to-end on
Windows with `windows-com` on — callers no longer pre-resolve
`(addr, service_ipid)` by hand.
`connect_nmx`'s doc comment updated to drop the stale "F12 not yet
wired" note. `parse_bracketed_host_port` in mxaccess-nmx gets a
`cfg_attr(not(...), allow(dead_code))` so the default-feature
build stays warning-clean.
2. F32 closed via option (b) of its own resolve criterion: the four
missing types (Float / Double / DateTime / Duration) are gated on
Galaxy-side template provisioning that's outside the Rust port's
scope. The deployed test Galaxy on this host only has
mx_data_type ∈ {1=Bool, 2=Int32, 5=String}; we cannot exercise
the missing types without authoring new template attributes in
the Aveva console (a manual platform-engineering task). The
three-type live verification at commit
|
||
|
Joseph Doherty
|
5845b5eb12 |
[M5] mxaccess-asb: F32 partial — Bool + String + Int32 live, longer retry budget
rust / build / test / clippy / fmt (push) Has been cancelled
Three of seven proven types now round-trip end-to-end against the live MxDataProvider: ✅ Int32 (type_id 4) — TestChildObject.TestInt = 99 ✅ String (type_id 10) — TestChildObject.TestString = "mxaccesscli verified 17778523775" (UTF-16LE on wire) ✅ Bool (type_id 17) — DelmiaReceiver_001.TestAttribute = 0 A SQL probe of the live Galaxy (`gobject ⨝ package ⨝ dynamic_attribute` grouped by `mx_data_type`) shows only types {1=Bool, 2=Int32, 5=String} have deployed instances. Float/Double/DateTime/ Duration/array shapes are not in this Galaxy, so the remaining four type-matrix bullets in F32 are gated on Galaxy-side provisioning that's outside the Rust port's scope. The M5 DoD #3 was always going to bottom out at "what types are deployed in the test environment." Code changes: - `register_items` retry budget bumped: 10 attempts (was 5) with `200 * attempt` ms backoff (was 100 * attempt). Worst-case wait ~11 s, well within user-perceived latency on a one-shot RPC. The .NET reference's 5×100 ms didn't always cover the live AVEVA install's auth-state-commit latency on this hardware. - `AsbClient::connect` adds a 250 ms `tokio::time::sleep` immediately after the one-way `AuthenticateMe` send. The server processes the request asynchronously; without an initial settle, the per-op retry loop frequently exhausts its budget on the InvalidConnectionId race even on the FIRST register attempt. 250 ms is short enough to be invisible and long enough to absorb the typical commit delay. - `examples/asb-subscribe.rs` now prints `result_code` and `success` alongside the status count so the user can see when register is hitting the retry-exhausted state. Live flakiness note: the AuthenticateMe race is not fully deterministic — after many back-to-back test runs the live server appears to degrade (presumably pending-connection table fills) and the retry budget exhausts on EVERY tag, not just one. A 30-second cool-down restores reliability. Production deployments with a single long-lived session are unlikely to hit this. F32 status doc captures the observation. Workspace: 711 unit tests pass. Clippy clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
f14580e0db |
[M5] mxaccess-asb: F28 canonical-XML signing wired + registry-driven DH params
Adds `xml_canonical` module that emits XmlSerializer-compatible canonical XML for the five primary `ConnectedRequest` shapes (AuthenticateMe, Disconnect, KeepAlive, RegisterItemsRequest, UnregisterItemsRequest). Six fixture-comparison tests verify byte-exact match against captured .NET output, including the empty-MAC-IV variant that the live signing flow uses (`authenticate-me-empty-mac-iv.xml`, 896 bytes; new `emit_data_ns_byte_array` helper picks self-closing form for empty byte[]). Plumbing: `AsbAuthenticator::peek_next_message_number` exposes the pre-allocated message number; `AsbClient::send_signed_envelope[_one_way]` gain an `xml_for_signing: Option<&[u8]>` parameter. `connect`, `disconnect`, `keep_alive`, `register_items`, `unregister_items` now build a pre-signing `ConnectionValidator` (empty MAC + IV) + emit the canonical XML + pass the bytes through to HMAC. Other ops (Read, Write, Subscription) keep the legacy NBFX-bytes path until F28 expands to cover their request shapes. Live-bring-up wiring: - `tools/Get-AsbPassphrase.ps1` now exports `MX_ASB_DH_PRIME`, `MX_ASB_DH_GENERATOR`, `MX_ASB_DH_HASH_ALGORITHM` (always — even when empty, so the example can distinguish "no env var" from "registry says empty"), and `MX_ASB_DH_KEY_SIZE`. - `examples/asb-subscribe.rs` honours those env vars to override `CryptoParameters::defaults()`. Each AVEVA install picks its own DH group at provisioning time (768-bit prime is typical, vs the .NET reference's 1024-bit fallback that we previously hardcoded). Empty hashAlgorithm in the registry maps to `HashAlgorithm::Unrecognised`, matching `AsbSystemAuthenticator.CreateHmac:84-93` semantics where empty + forceHmac=true → HMAC-SHA1. - `MxAsbClient.Probe --dump-signed-xml` flag (added in earlier commit) now traces the live HMAC inputs (`asb.sign.xml-utf8-len`, `asb.sign.xml-b64`, `asb.sign.hmac-b64`, etc.) so the Rust port can diff its canonical XML against .NET's byte-for-byte for any live scenario (env-driven via `Action<string>? sharedTrace`). Wire-format alignment for `XmlSerializer` parity: - `ItemIdentity::default()` and `absolute_by_name` now use `Some(String::new())` for null-able strings (matches .NET's `CreateAbsoluteItem` setting `ContextName = string.Empty` not null). - `read_unicode_string` returns `Some(String::new())` for length-0 rather than `None` — mirrors .NET's `AsbBinary.ReadUnicodeString: return string.Empty for byteLength == 0`. Wire format genuinely cannot distinguish null from empty (both encode as 4 bytes of zero); callers that need to preserve the distinction MUST track it in their domain types before encoding. Live status (post-fix): Connect handshake completes end-to-end. The canonical XML our emitter produces matches .NET's structure byte-for- byte (verified by fixture comparison). DH prime/generator/hash now match the live registry values. Despite all this, AuthenticateMe still produces a generic dispatcher fault on the server — there's at least one more subtle wire-byte or crypto mismatch that needs isolation. F28 stays open with that note. Workspace: 709 unit tests pass (was 702 + 7 new xml_canonical tests). Clippy: clean (`-D warnings`). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
2867310817 |
[M5] mxaccess-asb: WCF binary message header (action+to dict pre-pop)
Adds the binary header block that WCF prepends to SizedEnvelope
payloads. Reverse-engineered from .NET probe wire bytes captured via
asb-relay.
Wire form (per the .NET capture analysis in the previous commit):
```
[outer length, multibyte-int31]
[string-1 length, multibyte-int31] [UTF-8 bytes] ← dict id 1 (action)
[string-2 length, multibyte-int31] [UTF-8 bytes] ← dict id 3 (to)
[NBFX <s:Envelope>...]
```
Inside the NBFX envelope, `<a:Action>` and `<a:To>` reference the
pre-pop strings via `DictionaryText 0xAA {odd-id}` instead of inlining
their values. The header strings get assigned odd dict ids
(1, 3, 5, ...); even ids stay reserved for the [MC-NBFS] static dict.
Encode side:
* `encode_envelope` now emits header [action, to] before NBFX. `to_uri`
defaults to empty string when None — caller-supplied `with_to(uri)`
is the supported path.
* AsbClient's `send_envelope` and `send_envelope_one_way` auto-fill
`to_uri` from `self.via_uri` when not set.
* New private `encode_binary_header(strings)` helper.
Decode side:
* New `parse_binary_header_prefix(input)` heuristically detects + parses
the header (look for plausible NBFX element record byte 0x40-0x77 at
the offset implied by the outer length).
* New `resolve_with_header(text, dynamic, header)` resolves
`DictionaryText` with odd id by indexing into header.strings; even
ids fall through to static-dict lookup as before.
Tests pass (72) — round-trip envelope → bytes → envelope recovers
action through the new dict-id resolution path.
Live status: this commit gets us further but the connect SOAP
envelope still TCP-RSTs at SMSvcHost. The remaining delta vs the .NET
capture is structural NBFX optimisation: .NET uses single-letter
prefix-element/attribute records (0x44-0x77 PrefixDictionaryElement
_<a-z>, 0x0C-0x25 PrefixDictionaryAttribute_<a-z>, 0x0B
DictionaryXmlnsAttribute) while our F21 encoder always uses the long
forms (0x43 prefix-string + name-dict-id, etc.). Logically
equivalent but WCF's parser likely strict on which form it accepts.
Next iteration will add short-form encoding to F21 for single-letter
prefixes (s:, a:, h:, i:) which covers every namespace prefix in our
envelope.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
d4ee5f3a18 |
[M5] examples: asb-relay TCP middleman for live wire-byte capture
Listens on MX_RELAY_LISTEN (default 127.0.0.1:8088) and forwards to
MX_RELAY_UPSTREAM (default 127.0.0.1:808 — AVEVA's NetTcpPortSharing
SMSvcHost listener). Hex-dumps every byte both directions to stderr
with C->S / S->C tags + per-direction offset prefixes.
Usage:
$env:MX_RELAY_LISTEN = '0.0.0.0:8088'
.\rust\target\debug\examples\asb-relay.exe 2> relay.log
# then in another shell:
dotnet run --project src\MxAsbClient.Probe -c Release -- `
'--endpoint=net.tcp://desktop-6jl3kko:8088/ASBService/Default_ZB_MxDataProvider/IDataV2'
Tested against the live AVEVA install on this box — captured a
620-byte client→server exchange including the full .NET probe's
preamble, SizedEnvelope, and End record. The capture surfaced one
critical missing piece in our wire format:
**WCF binary message framing prepends Action + To strings out-of-band**
before the actual NBFX SOAP envelope. The .NET probe's envelope
payload begins:
74 27 [39 bytes "http://asb.contracts/20111111:connectIn"] ← Action
4b [75 bytes "net.tcp://desktop-6jl3kko:8088/.../IDataV2"] ← To
56 02 ... ← <s:Envelope>
The 0x74 / 0x4b prefix bytes appear to be WCF-internal framing that
stores Action and To headers OUT of the SOAP envelope as a binary
optimization. Our F25 envelope encoder doesn't emit this — it goes
straight to `<s:Envelope>` (which the probe captured as `56 02 ...`
PrefixDictionaryElement_s + dict id 2). This is likely why the
server fault'd at AddressFilter mismatch in the previous iteration.
Note: when going through the relay, the .NET probe's `:8088` port
appears in the To URL inside the binary header, which doesn't match
the registered service URL on SMSvcHost — so this exact relay setup
returns the AddressFilterMismatch fault. The capture is still
valuable (we see what bytes WCF emits for our action/header
structure). For a fault-free dispatch, we'd need to:
* rewrite the binary header's port (0x4b length / URL bytes) at
the relay, OR
* listen on port 808 directly (requires stopping SMSvcHost), OR
* run an admin-elevated Wireshark/Npcap loopback capture.
Cleanup: dotnet probe must use `--endpoint=URL` (single arg with `=`),
not space-separated; the probe's GetArg helper splits on `=`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
3b09297b27 |
[M5] live-probe iteration 1 — major wire-byte reconciliation fixes
First live-test cycle against AVEVA on this box. Comparing the .NET
probe's `--dump-messages` XML output against our NBFX-encoded
envelope surfaced six structural bugs in the F25 envelope/operations
layer. All fixed; tests passing (702 workspace).
Fixes (all backed by the .NET dump as ground truth):
1. **`mustUnderstand` attribute name** — NBFS dict id was 116
(`MustUnderstand`, capital-M, a different SOAP token); SOAP 1.2
spec uses lowercase `mustUnderstand` at id 0. Sending the wrong
one triggered a WCF parse fault that surfaced as TCP RST.
2. **Missing `<a:MessageID>` header** — WCF's default binding
requires MessageID for two-way operations. We now auto-generate
`urn:uuid:<v4>` per envelope via a small inline `make_random_uuid_v4`
helper (no `uuid` crate dep).
3. **Missing `<a:ReplyTo>` anonymous header** — WCF's
BinaryMessageEncoder always emits `<a:ReplyTo><a:Address>...
addressing/anonymous</a:Address></a:ReplyTo>` for two-way ops.
4. **ConnectionValidator field names + namespace** — we were
emitting PascalCase `<ConnectionId>` etc. .NET's WCF
DataContractSerializer uses the private backing-field names
(`<connectionIdField xmlns="...ASBContract">guid</connectionIdField>`)
per `[DataMember(Name = "fooField")]`. Added the
`xmlns:i="...XMLSchema-instance"` declaration WCF emits
alongside (even when no `i:nil` is used). Decoder now accepts
both PascalCase (legacy tests) and DataContract field names.
5. **`<ASBIData>` over-wrapping** — we were emitting
`<Items><ASBIData>{bytes}</ASBIData></Items>`. .NET's
`AsbDataCustomSerializer.WriteStartObject` (`AsbContracts.cs:
1561-1572`) REPLACES the field's outer element with `<ASBIData>`
directly — there's no `<Items>` wrapper on the wire. Fixed by
collapsing `BodyField::AsbiDataElement` to emit just `<ASBIData>`
without the named outer element. The `name` field is retained
for self-documentation.
6. **`collect_asbidata_payloads` API** — was keyed by field name
(`Status` / `Values`); now positional (`payloads[0]`,
`payloads.get(1)`) since the wrapper element is gone. All seven
response decoders updated.
Plus tooling for the live-probe loop:
* `tools/Get-AsbPassphrase.ps1` — DPAPI loader that auto-discovers
the solution name + reads the sharedsecret + decrypts it. Sets
$env:MX_ASB_PASSPHRASE / MX_ASB_HOST / MX_ASB_VIA / MX_LIVE.
Lowercase via-host (WCF SMSvcHost is case-sensitive on the URL
host segment).
* `examples/asb-preamble-probe.rs` — diagnostic that connects,
runs the preamble, captures the PreambleAck, then sends a
synthetic ConnectRequest and dumps both directions as hex. Used
to bisect the wire-byte deltas above.
* `examples/asb-subscribe.rs` port default fixed (5074 → 808 —
WCF's NetTcpPortSharing/SMSvcHost listener confirmed via
Get-NetTCPConnection).
**Status**: preamble + PreambleAck round-trip works end-to-end
against the live AVEVA install (verified via probe). The
post-preamble Connect SOAP envelope still gets TCP RST'd — the six
structural fixes above are necessary but not yet sufficient. Next
iteration needs binary wire capture (Wireshark + Npcap loopback,
or a TCP-relay middleman) to compare the .NET probe's BinaryMessageEncoder
output byte-for-byte with ours and find the remaining delta(s).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
e3baeb8803 |
[M5] mxaccess: F26 step 3 — AsbSession high-level cheap-clone async API
Adds the public high-level entry point for the ASB transport.
Parallel to the NMX-shaped `Session` (rather than unified) because
NMX's `Session` carries CallbackExporter / callback router task /
recovery broadcast / INmxService2 mutex orchestration that has no
ASB analogue — and ASB's request/response loop over a single TCP
stream maps naturally to `Mutex<AsbClient>` that would be foreign
to NMX. Two paths converge at the consumer-facing API but stay
distinct at the orchestration layer.
Struct shape:
```rust
pub struct AsbSession { inner: Arc<AsbSessionInner> }
struct AsbSessionInner {
transport: Mutex<AsbTransport<TcpStream>>,
connect_response: ConnectResponse,
}
```
`Clone + Send + Sync` — clones share state through `Arc`, lock
serialises operations. Compile-time `assert_clone_send_sync` test
guards the contract.
API:
* `connect(endpoint, passphrase, crypto_parameters, via_uri,
connection_id)` — full bring-up (TCP + preamble + DH handshake).
* `from_transport(transport, connect_response)` — build from an
existing transport (tests, custom transports).
* `connect_response()` — surface the negotiated lifetime /
Apollo flag.
Operation methods forward to AsbClient:
* `register_items` / `unregister_items` / `read` / `write`
* `keep_alive` / `disconnect`
* `create_subscription` / `add_monitored_items` / `publish` /
`delete_monitored_items` / `delete_subscription`
* `publish_write_complete`
ClientError → mxaccess::Error mapping via
`ConnectionError::TransportFailure` (consistent with F26 step 2).
1 new test:
* `asb_session_is_clone_send_sync` — compile-time trait-bound
assertion.
Workspace: 702 tests pass.
Stubbed for next F26 iteration:
* `Stream<Item = MonitoredItemValue>` subscription handle that
internally drives a publish-loop. Today consumers loop
`publish().await` themselves.
* Recovery / reconnect policy — needs a captured ASB-side
disconnect to inform the retry strategy.
* Live-probe wire-byte reconciliation against the WCF DataContract
XML serializer's actual output.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
c6570dcd06 |
[M5] mxaccess: asb-subscribe example exercises full F25+F26 stack
Replaces the M5 placeholder with an actual end-to-end demo:
AsbTransport::connect (TCP + preamble + DH handshake)
→ register_items
→ read
→ disconnect
→ send_end
Until F25 subscription ops (CreateSubscription / AddMonitoredItems
/ Publish-callback) land, the example is a Read-loop demo. Once
subscription ops arrive, it gains a Publish-loop and lives up to
its name.
Env vars (analogous to the NMX `connect-write-read` example):
MX_LIVE — non-empty enables the live path
MX_ASB_HOST — endpoint host[:port]; defaults port 5074
MX_ASB_PASSPHRASE — solution shared secret
MX_ASB_VIA — `net.tcp://...` URI (optional; derived from MX_ASB_HOST
when omitted)
MX_TEST_TAG — tag reference (default `TestChildObject.TestInt`)
Without MX_LIVE: prints the `Setup-LiveProbeEnv.ps1` hint and exits
cleanly with status 0 — the same pattern every other live example
follows.
Connection-id is a fresh 16-byte random buffer (matches .NET's
`Guid.NewGuid()` at `MxAsbDataClient.cs:36`).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
14bb5297a8 |
[M5] mxaccess: F26 step 2 — AsbTransport::connect TCP+preamble+handshake
Adds the `tokio::net::TcpStream`-specialised async constructor that
owns the full transport-bring-up sequence:
TCP connect → NMF preamble → DH Connect → AuthenticateMe (one-way)
Signature:
```
async fn connect(
endpoint: SocketAddr,
passphrase: &str,
crypto_parameters: &CryptoParameters,
via_uri: impl Into<String>,
connection_id: [u8; 16],
) -> Result<(AsbTransport<TcpStream>, ConnectResponse), Error>
```
Returns the `ConnectResponse` alongside the transport so callers can
inspect the negotiated `connection_lifetime` (the `:V2` suffix
toggles Apollo vs Baktun encryption — see F23).
New error variant: `ConnectionError::TransportFailure { detail }`
covers all transport-bring-up failure modes (NMF / NBFX / auth /
peer Fault). The underlying error type is intentionally erased to
keep the public taxonomy small; `detail` carries the Display
representation.
Errors are mapped at the AsbClient / AuthError boundary via private
`map_client_error` / `map_auth_error` helpers.
1 new test:
* `connect_to_unreachable_endpoint_surfaces_connection_error` — TCP
connect to 127.0.0.1:1 (TCPMUX-reserved) cleanly errors without
panicking. Smoke test for the constructor signature + error path.
Stubbed for F26 step 3:
* `Session::connect_asb` constructor — the SessionInner refactor to
host both NMX + ASB transports under one struct is heavier than
this iteration's scope.
* Operation-routing layer that maps ASB result types (ItemStatus,
RuntimeValue) back to mxaccess types (MxStatus, DataChange,
MxValue).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
8a0f92b6bc |
[M5] mxaccess: F26 step 1 — AsbTransport bridges AsbClient into Transport trait
First slice of F26. Bridges F25's working AsbClient into the M0
`mxaccess::Transport` trait that Session uses to discriminate
operations across NMX and ASB transports.
API additions:
* `mxaccess::AsbTransport<T>` — generic over the same
AsyncRead+AsyncWrite+Unpin+Send+Sync+'static bound that AsbClient
takes. Owns an AsbClient and exposes it via `client_mut()` /
`into_client()`.
* `impl Transport for AsbTransport<T>`:
- `capabilities()` — `buffered_subscribe = false`,
`activate_suspend = false`, `operation_complete_frame = false`
per `design/60-roadmap.md` M5 (no NMX-specific extensions on
ASB).
- `kind()` — `TransportKind::Asb`.
Path-dep wiring: `mxaccess` now imports `mxaccess-asb` +
`mxaccess-asb-nettcp` directly.
Compile-time `Send + Sync + 'static` assertion guards the
trait-bound contract.
2 new tests:
* `asb_transport_kind_is_asb`.
* `asb_transport_capabilities_disable_buffered_and_activate_suspend`.
Stubbed for F26 step 2:
* `Session::connect_asb` constructor that owns TCP open +
preamble + DH handshake orchestration.
* Operation routing that maps ASB types (ItemStatus, RuntimeValue)
back to mxaccess types (MxStatus, DataChange, MxValue).
Stubbed for F26 step 3:
* Subscription routing — Session::subscribe on ASB needs F25
subscription operations (CreateSubscription / AddMonitoredItems
/ Publish), which are not yet implemented.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
a5d31cc2e1 |
[M4] mxaccess: wire MxValue overloads + shutdown(timeout) shim
rust / build / test / clippy / fmt (push) Has been cancelled
Replaces the lib.rs `Unsupported`-stub Session methods with real implementations where the underlying primitives already exist in session.rs, sharpens docstrings on the still-deferred ones, and refreshes the stale "M0 stub" module preamble. Wired (now functional): - `Session::write(MxValue)` — converts via `mxvalue_to_writevalue` then delegates to `write_value`. - `Session::write_with_timestamp(MxValue, SystemTime)` — same plus `system_time_to_filetime` then `write_value_at`. - `Session::write_secured_at(MxValue, SystemTime, SecurityContext)` — same plus `write_value_secured_at`. - `Session::shutdown(timeout)` — `tokio::time::timeout` wrapper around `shutdown_nmx`; on elapse returns `Error::Timeout` (the in-flight unregister is cancelled, mirroring the .NET `IDisposable` semantics at `MxNativeSession.cs:481`). Still `Unsupported` (gating reasons documented in each docstring): - `Session::connect` — needs F12 auto-resolve (gated on F6 windows-rs). - `Session::write_with_completion` — needs per-token registry, gated on R15 long-lived task. - `Session::write_secured` (no timestamp) — `NmxClient` only ports `WriteSecured2` (LMX 0x3A), not the unversioned `WriteSecured` (0x39). - `Session::subscribe_many` — no atomic frame on the wire; canonical pattern is `examples/multi-tag.rs`. - `Session::subscribe_buffered` — M6 `SetBufferedUpdateInterval` RPC. `mxvalue_to_writevalue` consumes the `MxValue` and returns `Error::Configuration(InvalidArgument)` for the three variants whose re-encode is policy-dependent: `DateTime` / `ElapsedTime` / `DateTimeArray`. The `non_exhaustive` MxValue catch-all preserves forward compat. Test count delta: 532 → 542 (+10; conversion happy paths for Boolean / Int32 / Float64 / String / Int32Array / BoolArray / StringArray, plus the three rejected variant errors). Open followups touched: none resolved (F12, F16 still gating). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
48d3a9d6da |
[M2/M4] mxaccess-rpc: Guid::parse_str + dedupe examples (resolves F17)
Adds `Guid::parse_str(&str) -> Result<Guid, RpcError>` to `crates/mxaccess-rpc/src/guid.rs` as the inverse of the existing `Display` impl. Accepts the canonical dashed-hex form, optionally braced (.NET `B` format), case-insensitive, and tolerant of bare 32-char hex without dashes. Single-pass char-by-char nibble accumulator avoids per-byte string allocation; applies the same byte-swap of groups 1-3 that the `Display` impl reads. Eight new tests cover round-trip against the existing `Display` fixture (`crates/mxaccess-rpc/src/guid.rs:111-119`, `b49f92f7-c748-4169-8eca-a0670b012746`), braces, uppercase, no-dashes, zero-GUID, too-short, too-long, and non-hex rejection. The five live-NMX examples (`connect-write-read`, `subscribe`, `recovery`, `multi-tag`, `secured-write`) lose their per-file 15-line `parse_guid` helpers in favour of the canonical implementation. `asb-subscribe` and `subscribe-buffered` are unaffected — they don't parse GUIDs. Test count delta: 524 → 532 (+8) Open followups touched: F17 resolved. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
af939730b1 |
[M4] mxaccess: examples wave 3 — 7 example programs (M4 wave 3)
Replaces the M0 stub bodies in `crates/mxaccess/examples/` with real
consumer-facing demos against the M4 NMX `Session` surface. Each example
gates on `MX_LIVE` and prints a friendly bypass message when the live
env vars aren't populated, so `cargo build --workspace --all-targets`
stays green in CI without an AVEVA install.
Five examples target the proven NMX path (build + connect + demo +
shutdown):
- `connect-write-read` — `Session::write_value` + `read` round-trip; the
30-line consumer-experience target from `design/60-roadmap.md` M4 DoD.
- `subscribe` — single-tag `Subscription` stream; drains 5 updates or
10s timeout, then `unsubscribe` cleanly.
- `recovery` — `RecoveryPolicy { max_attempts: 3, delay: 250ms }`
+ spawned `recovery_events()` listener consuming the broadcast.
- `multi-tag` — per-tag `subscribe` loop merged via
`futures_util::stream::select_all`; matches the .NET cs:250-270 shape
(no atomic subscribe-many RPC on the wire).
- `secured-write` — `write_value_secured_at` exercising both single-user
(`current_user_id == verifier_user_id`) and two-person paths per
`wwtools/mxaccesscli/src/MxAccess.Cli/Commands/WriteCommand.cs:151-155,196-199`.
Two examples hold the place for downstream milestones:
- `subscribe-buffered` — pattern-matches on `Error::Unsupported` from
`Session::subscribe_buffered` (M6) and exits 0 with an explanation.
- `asb-subscribe` — same shape against `Session::connect` (M5 ASB).
All five live examples share an inline `LiveEnv::from_process` helper,
a dashed-hex `parse_guid`, and a `StaticResolver` that returns canned
metadata for the configured `MX_TEST_TAG`. The duplication is
intentional — Cargo examples are meant to be self-contained and read
top-to-bottom; consumers swap `StaticResolver` for a tiberius-backed
Galaxy resolver (followup F14) without touching any other example.
Test count delta: 524 → 524 (+0; examples are demos, not tests)
Open followups touched: F17 logged (Guid::parse_str helper to dedupe
the per-example dashed-hex parser).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
4863c6dc1f |
[M4] mxaccess: Session::recover_connection + RecoveryEvent broadcast
Wires the recovery API surface and event channel. Recovery is currently a no-op (validates policy + emits Started/Recovered events); the real teardown + re-bind + re-advise loop is wave-3 work tracked as F16. New - Session::recover_connection(policy) — port of MxNativeSession.RecoverConnectionAsync (cs:399-440). Validates policy.max_attempts >= 1 (mirrors cs:33-36 via RecoveryPolicy::validate). Emits RecoveryEvent::Started + Recovered through the broadcast channel. Returns Ok(()) immediately — actual reconnect work is F16. - Session::recovery_events() -> broadcast::Receiver<Arc<RecoveryEvent>> — typed observable for consumers that want to wire monitoring or state-machine handling. Same Arc-broadcast pattern as Session::callbacks(). Multi-subscriber safe (Arc::ptr_eq verified in tests). - SessionInner.recovery_tx: broadcast::Sender<Arc<RecoveryEvent>> initialized in connect_nmx + connect_test_session. Removed lib.rs stub (was Err(Unsupported)). design/followups.md: F16 added (P1) covering the actual reconnect loop. Resolves when R15's long-lived connection task lands and SessionInner gains a subscription registry — at that point the recover loop becomes ~50 lines slotting RecoverConnectionCore-style work between the Started and Recovered events. Tests (4 new in mxaccess; total 48) - recover_connection emits Started + Recovered for the default single-attempt policy. - recover_connection rejects max_attempts == 0 with InvalidArgument. - recover_connection after shutdown returns EngineNotRegistered. - recovery_events supports multiple subscribers (Arc::ptr_eq verifies the same allocation reaches both). Test count delta: 520 -> 524 (+4). All four DoD gates green. Open followups: 9 -> 10 (added F16). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
2dc091d0be |
[M4] mxaccess: Session::read (read-as-subscribe pattern)
Now that Subscription impls Stream<Item = Result<DataChange, Error>>,
the read-as-subscribe pattern is a thin wrapper over subscribe +
timeout + best-effort unsubscribe.
New
- Session::read(reference, timeout) -> Result<DataChange, Error> —
port of MxNativeSession.ReadAsync (cs:312-359). Validates timeout
> 0, subscribes, awaits the first DataChange under
tokio::time::timeout, then issues UnAdvise (best-effort, mirrors
the .NET finally block at cs:351-358 which discards the
unsubscribe return).
Error mapping
- timeout=0: Configuration::InvalidArgument ("Read timeout must be
positive") matching ArgumentOutOfRangeException at cs:318-321.
- timeout elapsed: Error::Timeout(timeout).
- subscribe failure (resolver / transport): propagated unchanged.
- stream ends before any value: Connection::EngineNotRegistered
(broadcast sender dropped during shutdown).
- unsubscribe failure: tracing::warn! with the error; doesn't
override the read result.
Removed the placeholder stub in lib.rs that returned
Error::Unsupported.
Tests (4 new in mxaccess; total 44)
- read_returns_first_data_change_within_timeout: spawn read,
inject a 0x33 DataUpdate via test_inject_sender (which fans out
to all subscriptions), assert the DataChange comes back with the
right value.
- read_returns_timeout_when_no_data_arrives: read times out cleanly
with Error::Timeout when no callback fires.
- read_zero_timeout_returns_invalid_argument_without_subscribing:
validates the early-reject path before any RPC is issued.
- read_propagates_resolver_not_found: subscribe-side error
surfaces through read unchanged.
Test count delta: 516 -> 520 (+4). All four DoD gates green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
a31237d1d0 |
[M4] mxaccess: Subscription impls Stream<Item = DataChange> (resolves F15)
F15 step 2/2 lands the per-Subscription routing on top of step 1's broadcast layer. Subscription is now a working data-change stream. Subscription type - Now impls futures_util::Stream<Item = Result<DataChange, Error>> via tokio_stream::wrappers::BroadcastStream + a per-message filter. - No longer Clone (broadcast::Receiver isn't Clone). Consumers that want fanout subscribe twice or share via Arc<Mutex<...>>. - Holds the broadcast::Receiver subscribed BEFORE AdviseSupervisory fires — guarantees no updates between advise and stream-creation are dropped. - pending VecDeque buffers records from the current message so each poll_next yields at most one DataChange (Stream contract). Filter logic (records_to_data_changes, mirrors cs:333-343) - 0x32 SubscriptionStatus: keep when msg.item_correlation_id == subscription.correlation_id; drop otherwise. - 0x33 DataUpdate: keep ALL — codec exposes no per-record correlation field, and the .NET filter only checks item_correlation_id (which 0x33 doesn't carry), so DataUpdates fan out to every active subscription. Matches .NET behavior verbatim. - Records with value: None drop silently (mirrors evt.Record.Value is null filter at cs:337). - BroadcastStream Lagged(n) maps to Error::Configuration with the lag count in the detail string. Helpers - filetime_to_system_time(i64) -> SystemTime: inverse of system_time_to_filetime; saturates at Unix epoch for FILETIMEs before 1970 since SystemTime can't portably represent pre-epoch. - record_to_data_change(record, reference) -> Option<DataChange>: builds DataChange from one record, returns None for unparseable value (the codec couldn't decode the wire kind). - Status currently hardcoded to MxStatus::DATA_CHANGE_OK (mirrors NmxSubscriptionRecord.ToDataChangeStatus at NmxSubscriptionMessage.cs:22-25 which the .NET reference itself stubs to the OK constant). Cargo.toml additions: futures-util (workspace) + tokio-stream (0.1 with sync feature for BroadcastStream). Tests (5 new in mxaccess; total 40) - subscription_stream_yields_data_change_for_matching_correlation: build a 0x32 SubscriptionStatus with one Int32 record and the subscription's correlation id, inject through test_inject_sender, observe the DataChange (reference, value, quality match) on the Stream. - subscription_stream_filters_out_mismatched_correlation_for_status: inject 0x32 with wrong correlation id, assert the stream stays pending (timeout-as-success). - subscription_stream_keeps_data_update_regardless_of_correlation: inject 0x33 DataUpdate with one Int32 record (no correlation field on the message); stream still yields the DataChange. - filetime_to_system_time_round_trip: build a SystemTime with .005s precision, round-trip through both helpers, assert equality. - filetime_to_system_time_pre_unix_epoch_saturates: FILETIME 0 (year 1601) → SystemTime::UNIX_EPOCH (saturating clamp). design/followups.md: F15 moved to Resolved with both step commits referenced. Open list: 9 items (was 10). Test count delta: 511 -> 516 (+5). All four DoD gates green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
2b849aed7a |
[M4] mxaccess: wire CallbackExporter + spawn callback router (F15 step 1/2)
Lands the broadcast layer of F15. Session::connect_nmx now starts a local CallbackExporter on an OS-assigned ephemeral port, builds a callback OBJREF advertising it (using local_hostname() with a 127.0.0.1 fallback), and registers that OBJREF with NmxClient::register_engine_2 (was register_engine_2_without_callback). A router task drains the exporter's CallbackEvent stream, decodes each CallbackInvoked body as NmxSubscriptionMessage, and broadcasts parsed messages on a tokio::sync::broadcast channel. Per-subscription correlation routing — turning these raw messages into per-Subscription DataChange streams — is the next iteration's work. F15 stays open until that lands. New Session API - Session::callbacks() -> broadcast::Receiver<Arc<NmxSubscriptionMessage>>: raw observable of every parsed callback message. Test seam + escape hatch for consumers that need raw access today. - Session::callback_exporter_addr() -> Option<SocketAddr>: returns the exporter's local addr (Some until shutdown_nmx, None after). SessionInner additions - callback_exporter: Mutex<Option<CallbackExporter>> — taken in shutdown. - callback_tx: broadcast::Sender<Arc<NmxSubscriptionMessage>>. - router_handle: std::sync::Mutex<Option<JoinHandle<()>>>. shutdown_nmx now performs the full cleanup chain: 1. UnregisterEngine over the live NMX transport. 2. CallbackExporter::shutdown (cancels accept loop). 3. Wait for router task — exits naturally once exporter's mpsc sender side closes. Std::sync::Mutex guard taken-out-then-dropped before await to avoid clippy::await_holding_lock. Routing rationale (callback_router fn) - CallbackEvent::CallbackInvoked → parse via NmxSubscriptionMessage::parse_inner → broadcast Arc<msg>. - Other event variants (Bind / Auth3Ignored / ProtocolError / etc.) silently dropped at this layer; consumers needing them can listen to a future diagnostic-channel hook (no followup yet). - Parse failures silent — the .NET reference fires a separate UnparsedCallbackReceived event we don't model yet. Cargo.toml: added mxaccess-callback as a direct dep on mxaccess. Tests (5 new in mxaccess; total 35) - callbacks receiver observes injected NmxSubscriptionMessage. - multi-subscriber broadcast hands out the same Arc to each receiver. - callback_exporter_addr is Some before shutdown, None after. - router_task end-to-end: feed a hand-built CallbackInvoked event with a 39-byte 0x32 SubscriptionStatus body, observe the parsed message on the broadcast. - router silently drops non-CallbackInvoked events (e.g. Bind). Test count delta: 506 -> 511 (+5). All four DoD gates green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
70feb63ea5 |
[M4] mxaccess: Session::subscribe + unsubscribe + Subscription handle
Lands the subscribe-path lifecycle: AdviseSupervisory + UnAdvise
round-trip via a Subscription handle. The actual DataChange stream
routing is deferred to F15.
New
- Session::subscribe(reference) -> Result<Subscription, Error> —
resolves the tag, generates a 16-byte correlation_id via
rand::random(), calls NmxClient::advise_supervisory. Mirrors
MxNativeSession.SubscribeAsync (cs:250-270) minus the publisher
Connect dance (will land alongside F15's callback routing).
- Session::unsubscribe(subscription) -> Result<(), Error> — consumes
the handle and calls NmxClient::un_advise. Mirrors
MxNativeSession.Unsubscribe (cs:361-381).
- Subscription { correlation_id, reference, metadata } public type
with accessor methods. Currently a pure lifecycle handle — no
Stream impl yet; the Stream<Item=DataChange> shape lands when F15
wires CallbackExporter routing.
- Removed the old subscribe stub from lib.rs (was Err(Unsupported)).
Drop hazard note
- Subscription deliberately does NOT impl Drop to fire UnAdvise. The
spawn-from-Drop pattern is the R15 hazard tracked in
design/70-risks-and-open-questions.md. Callers must call
Session::unsubscribe(sub).await explicitly. F15's wave-2 long-lived
connection task will support best-effort drop-time cleanup without
the spawn-from-Drop hazard.
Cargo.toml: added rand (for correlation_id generation).
design/followups.md: F15 added (P1, M4 wave 2 callback router).
Open followups now at 11 — slightly over the soft 10-item threshold
but no drift (F13 just resolved last iteration). Next iteration's
Step 0 triage will check whether F15 is actionable.
Tests (4 new in mxaccess; total 30)
- subscribe_then_unsubscribe round-trip via in-memory resolver +
hand-rolled server (2 RPCs: AdviseSupervisory + UnAdvise).
- subscribe propagates non-zero AdviseSupervisory HRESULT.
- subscribe after shutdown returns EngineNotRegistered.
- two_subscribes_produce_distinct_correlation_ids — verifies the
rand::random() correlation id generation differentiates handles.
Test count delta: 494 -> 498 (+4). All four DoD gates green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
bf95995573 |
[M4] mxaccess: Session::write_value_at + write_value_secured_at
Adds the timestamped + verified-write paths on top of the wave 1 write path. Plus a SystemTime → FILETIME helper so callers don't have to do the 1970→1601 epoch arithmetic by hand. New - Session::write_value_at(reference, value, timestamp_filetime) — port of MxNativeSession.Write2Async (cs:187-209). Delegates to NmxClient::write2 with the same routing as write_value. - Session::write_value_secured_at(reference, value, ts, security) — port of MxNativeSession.WriteSecured2Async (cs:223-248). Uses the session's options.engine_name as the client name (matches cs:239's _options.EngineName convention). Single-user secured writes pass current_user_id == verifier_user_id per R6 verification. - system_time_to_filetime(SystemTime) -> Result<i64, Error>: converts via the canonical 11_644_473_600s offset between 1970-01-01 and 1601-01-01. Pre-1970 values map to Configuration::InvalidArgument. Tests (7 new in mxaccess; total 26) - write_value_at round-trip via in-memory resolver + hand-rolled server. - write_value_secured_at round-trip with single-user (same id twice). - write_value_at propagates non-zero HRESULT as InvalidArgument. - system_time_to_filetime: Unix-epoch known value (11_644_473_600 * 10_000_000), +1s offset, +500ms subsecond conversion, pre-1970 rejection. One targeted fix: rewrote a doc comment that started a continuation line with `+ verifier user pair` — clippy parsed `+` as a markdown list bullet (clippy::doc_lazy_continuation). Test count delta: 487 -> 494 (+7). All four DoD gates green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
12cb10c3a1 |
[M4] mxaccess: Session::connect_nmx + write_value + shutdown (wave 1 main)
First working M4 wave 1 slice. Adds session.rs with the connect /
write / shutdown path on top of NmxClient + Resolver, plus a tokio
test that exercises a full round-trip against a hand-rolled server.
Read, subscribe, recovery, and the long-lived connection task land
in wave 2.
Architecture
- Session holds Arc<SessionInner>; SessionInner wraps NmxClient
behind a tokio::sync::Mutex. All RPC ops serialize on that mutex.
Wave 2 will replace it with an mpsc::channel<Op> + dispatcher task
per design/70-risks-and-open-questions.md R15 (drop-time async
cleanup hazards).
- ensure_connected gate stops post-shutdown ops with
Connection::EngineNotRegistered. Shutdown is idempotent via
AtomicBool::swap.
- Manual Debug impl on SessionInner — neither dyn Resolver nor
NmxClient impl Debug.
Public API
- Session::connect_nmx(addr, options, ntlm, service_ipid, resolver,
recovery): validates the policy, opens NmxClient, runs
RegisterEngine2 (no callback yet — wave 2), optionally configures
heartbeat. Returns Error::Connection on non-zero HRESULT.
- Session::write_value(reference, value: WriteValue): resolves the
tag through the configured Resolver, dispatches NmxClient::write.
- Session::resolve_write_kind / resolve_tag: convenience accessors.
- Session::shutdown_nmx: calls UnregisterEngine, idempotent.
Error mapping
- map_nmx / map_transport / map_resolver bridge the inner crate
errors into the public Error enum. NonZeroHresult → InvalidArgument
with the hex code; transport Fault → Status-shaped error;
ResolverError::NotFound → Galaxy { reason: "tag not found: ..." }.
- All three matchers handle their #[non_exhaustive] sources with a
generic catch-all so future variants don't silently break the map.
Tests (8 new in mxaccess; total mxaccess: 19)
- write_value round-trip via in-memory StaticResolver + hand-rolled
unauthenticated DCE/RPC server.
- write_value propagates resolver not-found → Galaxy error.
- write_value propagates non-zero HRESULT → InvalidArgument.
- shutdown is idempotent (second call is a no-op).
- write after shutdown returns EngineNotRegistered.
- resolve_tag and resolve_write_kind work without RPC.
- envelope-kind constants used by Session match codec exports
(sanity guard against codec rename).
mxaccess-nmx: WriteValue now re-exported at crate root.
mxaccess: deps gained mxaccess-nmx/galaxy/rpc + tokio + tracing,
plus async-trait as a dev-dep for the test resolver impl.
Test count delta: 479 -> 487 (+8). All four DoD gates green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
5cbc330f82 |
[M4] mxaccess: RecoveryPolicy fields + SessionOptions config
M4 wave 1 prep — the design-pivotal small types per dependencies.md
("(b) is small but design-pivotal — agree the event shape before
consumers depend on it"). The actual Session implementation lands
next iteration as wave 1 main (the .NET MxNativeSession.cs is ~24 KB).
RecoveryPolicy
- Was a unit struct; now carries max_attempts: u32 + delay: Duration
(port of MxNativeRecoveryPolicy at MxNativeSession.cs:24-43).
- SINGLE_ATTEMPT associated const matches the .NET static at cs:26.
- validate() rejects max_attempts == 0 (cs:33-36); the negative-Delay
branch (cs:38-41) is unreachable in Rust because Duration is
unsigned, so it's elided with a doc note.
- Default impl now returns SINGLE_ATTEMPT (was derive Default which
zero-initialised).
SessionOptions (new — port of MxNativeClientOptions at cs:7-22)
- local_engine_id, engine_name, partner_version, galaxy_id,
source_platform_id, heartbeat_ticks_per_beat: Option<i32>,
heartbeat_max_missed_ticks.
- default_local_engine_id() constructor: 0x7000 + (process_id & 0x0FFF)
per GenerateDefaultLocalEngineId at cs:18-21.
- default_engine_name(): "mxaccess.<pid>" mirroring the .NET
"MxNativeClient.{ProcessId}" at cs:10.
- partner_version=6 default matches design/60-roadmap.md:54 DoD #1.
Test count delta: 468 -> 479 (+11). All four DoD gates green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
fe2a6db786 |
Initial project state: .NET reference, design, Rust port (M0+M1), evidence
rust / build / test / clippy / fmt (push) Has been cancelled
Layout:
- src/ .NET 10 x64 reference: MxNativeCodec, MxNativeClient,
MxAsbClient, probes, tests, harnesses. Executable spec.
- design/ Architectural plan for the Rust port (M0–M6), error
model, protocol invariants, risks (R1–R16), adversarial
review log (review.md).
- rust/ Rust workspace. M0 skeleton + M1 codec parity.
mxaccess-codec: 215 unit tests + 2 cross-implementation
parity tests (byte-identical against .NET reference).
Other crates are M0 stubs awaiting M2+.
- captures/ Frida + netsh + pcap evidence per CLAUDE.md
("captures are evidence, not throwaway logs").
- analysis/ Decompiled C# (frida/proxy/decompiled-*),
Ghidra exports for native DLLs (`exports/` only —
working state at `projects/` and AVEVA's input
binaries at `input/` are gitignored).
- docs/ Reverse-engineering reference docs.
- tools/ Setup-LiveProbeEnv.ps1 (Infisical credential fetcher),
Compute-Crc.ps1 (.NET parity helper).
- .github/workflows/ Rust CI: fmt + build + test + clippy on Windows.
- LICENSE MIT (Joseph Doherty, 2026).
Verified:
- cargo test --workspace → 217 passed (215 unit + 2 .NET parity), 0 failed
- cargo clippy --workspace -- -D warnings → clean
- cargo fmt --all -- --check → clean
- cargo publish --dry-run -p mxaccess-codec → packages cleanly
Excluded from history (see .gitignore):
- **/bin, **/obj, **/target — build artifacts
- analysis/ghidra/projects/ — Ghidra working state (regenerable)
- analysis/ghidra/input/ — AVEVA proprietary DLLs (vendor IP)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|