a0fa5bedfdd944760a7500c01044337484470313
134 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Joseph Doherty
|
a0fa5bedfd |
[F52.2] mxaccess-codec: thread-local name-signature cache
Adds a thread-local `HashMap<String, u16>` cache inside `compute_name_signature`. Repeated calls with the same name (the hot path inside `MxReferenceHandle::from_names`) skip the `to_lowercase` allocation and the CRC-16/IBM walk entirely. Bounded at 1024 entries per thread; on overflow the cache is cleared rather than evicted LRU — any sane workload re-fills only the names it actively uses. `MxReferenceHandle::from_names` drops from 2 → 0 allocs/op once warm (bench delta in `design/M6-bench-baseline.md` § F52.2). Cold-path behaviour is unchanged: first call with a new name still pays the `to_lowercase` + cache-key `String` allocations. Two new tests pin the cache: cache-hit returns the same value as cold-compute, and cache overflow doesn't break correctness. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
4e76b44391 |
[F52.1] mxaccess-codec: BytesMut output buffer for write encoder
Adds `write_message::encode_to_bytes_mut` (and the timestamped variant) returning a freshly-allocated `BytesMut`. Allocation count is identical to `encode` (2 allocs/op for fixed-width scalars); the benefit is downstream — consumers can `BytesMut::split_to` / `freeze` and forward the body bytes to a wire-level sink without an intermediate copy. The body builders (`encode_boolean` / `encode_fixed` / `encode_variable` / `encode_array`) were refactored to fill a pre-sized `&mut [u8]` rather than each allocating their own `Vec<u8>`. The dispatcher computes the body size up front via small `*_body_size` helpers and resizes the destination buffer (Vec or BytesMut) once. This is also the prerequisite refactor for F52.3. Bench delta in `design/M6-bench-baseline.md` § F52.1; existing `encode` row unchanged at 2 allocs/op. All 265 round-trip tests unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
c7505f9570 |
[F51] live ASB type-matrix: provision UDAs + capture wire fixtures + round-trip tests
Provisioned 7 new UDAs on $TestMachine via wwtools/graccesscli
object uda add (then deployed to TestMachine_001):
TestFloat MxFloat scalar
TestFloatArray MxFloat array (4)
TestDouble MxDouble scalar
TestDoubleArray MxDouble array (4)
TestDateTime MxTime scalar
TestDuration MxElapsedTime scalar
TestDurationArray MxElapsedTime array (4)
New crates/mxaccess/examples/asb-type-matrix.rs reads all 14 tags
(7 pre-existing + 7 new) in a single batch and dumps the live
AsbVariant bytes per tag when MX_ASB_DUMP_FIXTURES=<dir> is set.
Single-attempt register (no retry — F31 InvalidConnectionId
cool-down re-arms on every retry, making backoff
counter-productive; if the cool-down is engaged, wait 60+ seconds
without ASB activity then re-run).
Captured live evidence (single cold-start run, all 14 register
calls returned error_code=0x0000):
TestChangingInt type_id=4 (Int32) length=4 payload=4
TestAlarm001 type_id=17 (Boolean) length=1 payload=1
MachineCode type_id=10 (String) length=30 payload=30
TestFloat type_id=8 (Float) length=4 payload=4
TestDouble type_id=9 (Double) length=8 payload=8
TestDateTime type_id=11 (DateTime) length=8 payload=8
TestDuration type_id=12 (ElapsedTime) length=8 payload=8
TestIntArray, TestBoolArray, TestStringArray, TestDateTimeArray,
TestFloatArray, TestDoubleArray, TestDurationArray
type_id=0 length=0 payload=0
(provisioned but no value written yet)
Per-tag fixture .bin files saved under
crates/mxaccess-codec/tests/fixtures/f51-type-matrix/ — full
14-byte to 40-byte AsbVariant byte sequences (i32 type_id LE +
i32 length LE + payload bytes).
crates/mxaccess-codec/tests/f51_type_matrix_parity.rs round-trips
each scalar fixture: decode -> re-encode -> assert byte-equal +
type_id / length pin. Tests skip with [skip] message when fixtures
are absent (so the suite passes on a fresh checkout without live
captures). 7 scalar tests pass against the captured fixtures.
Array tags excluded from round-trip pinning because the live
engine returns empty payloads for unwritten arrays. Codec-side
array round-trip is covered by asb_variant's existing synthetic-
payload unit tests.
docs/galaxy-test-fixtures.md inventories all $TestMachine UDAs
(pre-existing + F51-provisioned), the graccesscli provisioning
recipe, the fixture-regeneration pattern, and the F31 cool-down
caveat.
design/followups.md F51 marked resolved.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
8bd66bbe65 |
[F53 measurement] document protocol-crate missing-docs magnitude
Enabled #![warn(missing_docs)] on each of the 7 protocol crates to
measure how many one-liners filling them in would be:
mxaccess-asb 422
mxaccess-nmx 398
mxaccess-callback 371
mxaccess-galaxy 229
mxaccess-codec 205
mxaccess-rpc 147
mxaccess-asb-nettcp 111
-----
Total 1883
Reverted the lint enables — most of those are protocol-internal
types (struct fields on wire-shape records, enum variants on opcode
discriminators) whose meaning is already documented at the
consumer-facing layer. Filling 1883 one-liners adds noise without
consumer value, and forcing them as errors via RUSTDOCFLAGS would
block routine cargo doc runs.
design/followups.md F53 entry updated with the measured numbers
and the explicit "stays off indefinitely" verdict. If a future
contributor wants per-crate enforcement, the recipe in the strategy
paragraph (allow(missing_docs) on protocol-internal modules,
warn(missing_docs) on the re-export surface) is still valid.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
349e217ea3 |
[F50] live Suspend/Activate captures — Suspend wires opcode 0x2D, Activate client-side
Re-ran analysis/frida/mx-nmx-trace.js (with the F46 hooks for
LmxProxy.dll!CLMXProxyServer.Suspend / .Activate) against
MxTraceHarness on the local AVEVA install. Two captures landed:
- captures/123-frida-suspend-advised-instrumented/
Scenario: --scenario=suspend-advised --tag=TestChildObject.ScanState
After mx.suspend.begin/end at 17:23:51.949Z, NMX PutRequest fires
~140ms later with body:
2d 01 00 command 0x2D, version 0x0001
cd 2a ee ec b2 76 06 4f b4 58 5c a0 2d f7 a8 93 16-byte correlation_id (matches the prior AdviseSupervisory)
01 00 05 00 01 00 02 00 01 00 69 00 0a 00 engine + handle + attribute / property ids
47 92 00 00 03 00 00 00 trailer
TransferData wraps it; HRESULT 0 returned; ProcessDataReceived
callback delivers a 50-byte op-status frame; LMX surfaces it
through CUserConnectionCallback.OperationComplete. Suspend is
unambiguously server-side wire op 0x2D.
- captures/124-frida-activate-advised-instrumented/
Scenario: --scenario=activate-advised --tag=TestChildObject.ScanState
Activate fires at 17:26:02.982Z and returns Success synchronously
with no NMX traffic. The next NMX activity is 7+ seconds later
(harness teardown). Activate against a non-suspended item is
client-side only on this build.
The harness's activate-advised scenario doesn't sequence
Suspend-then-Activate, so we don't have direct evidence for
Activate-after-Suspend. Circumstantial reasoning: since Suspend
goes server-side with a state change, Activate likely also does to
revert. If direct evidence becomes needed, add a new
suspend-then-activate scenario to MxTraceHarness/Program.cs and
re-run.
design/70-risks-and-open-questions.md R5 moves to "settled —
Suspend is wire op 0x2D, Activate behaviour is conditional",
severity downgraded P2 -> P3 (no public Session::suspend /
Session::activate API exists today; if added later, 0x2D is the
encoder target).
design/followups.md F50 marked resolved.
docs/F50-suspend-activate-evidence.md: per-capture byte-level
evidence + repro recipe.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
b62ffc8c5d |
[F48] mark out-of-scope: internal usage only, no crates.io publish
Maintainer confirmed 2026-05-06 the project is internal-use only — workspace stays at version "0.0.0", consumers depend via path or git, not crates.io. F48's actual publish goal is dropped. design/followups.md F48 entry: replace the "P1 release driver" framing with "Out of scope" + a pointer to the recipe doc in case this ever changes. design/F48-publish-dry-run.md: add a banner at the top explaining the doc is now retained as a workspace-hygiene record (cargo package --list per crate produces clean tarballs, no captures or big files), not as release prep. The "What the actual V1 publish needs" section reframed as "If a publish ever does become a goal — recipe" so the steps survive without implying they're scheduled. No code change. F49 / F53 / F55 / F56 status unchanged — those weren't release-cut-gated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
e77db4306a |
[F48 dry-run] validate publish chain on workspace 0.0.0
cargo publish --dry-run on each of the 9 workspace crates: - Tier 1 leaves (mxaccess-codec, mxaccess-rpc, mxaccess-asb-nettcp) pass cleanly. cargo assembles each tarball, the only failure is the dry-run upload abort. - Tiers 2 + 3 (galaxy, callback, asb, nmx, mxaccess, mxaccess-compat) surface the documented "no matching package" registry-lookup failure because workspace internal deps are pinned at version "0.0.0" which doesn't exist on crates.io. Expected; resolves at actual publish time once the leaves are uploaded and indexed. cargo package --list confirms each crate ships only source + tests + small round-trip fixtures. No captures, decompiled binaries, or accidental big files. design/F48-publish-dry-run.md captures the per-crate run output, the per-crate file count, and the V1 publish recipe (bump 0.0.0 → 0.1.0 across workspace + internal-dep pins, publish in tier order, wait for indexing between tiers, tag). design/followups.md F48 entry annotated with the dry-run status. The actual publish to crates.io is deliberately not done — that needs maintainer auth + a deliberate version bump that's a release- cut decision, not a routine validation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
c606736ec3 |
[F53 partial] enable #![warn(missing_docs)] on consumer crates
mxaccess + mxaccess-compat now carry #![warn(missing_docs)] at the crate root. Every public item has at least a one-line doc comment (struct fields, enum variants, trait methods all covered). Touched items: - mxaccess::lib: DataChange fields, SecurityContext fields, TransportKind variants, TransportCapabilities fields, RecoveryEvent variants + their inner fields, SessionOptions fields, the full Error / ConnectionError / AuthError / ProtocolError / ConfigError / SecurityError taxonomy + nested fields, Transport trait method docs. - mxaccess-compat::lib: DataChangeEvent / BufferedDataChangeEvent / WriteCompleteEvent / OperationCompleteEvent fields. Protocol crates (codec, rpc, galaxy, nmx, callback, asb, asb-nettcp) deliberately left without the lint per F53's strategy paragraph — their consumers (mxaccess + mxaccess-compat) already document the surfaces they re-export, and forcing one-liners on every transport-internal item adds noise without consumer value. Verification: - `RUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-deps` clean. - `cargo test --workspace` (824 tests) green. - `cargo clippy --workspace --all-targets -- -D warnings` clean. design/followups.md F53 marked partially resolved. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
d149143535 |
[F49 steps 2 + 3] live verification: buffered recovery replay + unsubscribe skip
Step 3 (F47 buffered unsubscribe skip):
- crates/mxaccess-compat/tests/buffered_unsubscribe_skip_live.rs.
- Subscribe buffered, sleep so the engine has DataUpdates in flight,
then call unsubscribe. Asserts Ok return without surfacing transport
or HRESULT errors.
- Session::unsubscribe (session.rs:2261) probes the registry: if
Buffered { .. }, it skips nmx.un_advise entirely, mirroring the .NET
reference's `if (!subscription.IsBuffered)` guard at
MxNativeSession.cs:361-381. If unsubscribe accidentally emitted
UnAdvise for a buffered correlation id, the engine would return
non-zero HRESULT (no matching plain advise to retract) — surfacing
as a panic.
Step 2 (F45 buffered recovery replay):
- crates/mxaccess-compat/tests/buffered_recovery_replay_live.rs.
- Subscribe buffered, drain >=1 NMX subscription message
(cmd=0x32 SubscriptionStatus + cmd=0x33 DataUpdate) to confirm the
wire path is hot pre-recovery, install a RebuildFactory that calls
NmxClient::create (the same auto-resolving COM-activation path
Session::connect_nmx_auto uses), invoke recover_connection, drain
>=1 NMX subscription message post-recovery.
- Verifies the replay branch in recover_connection_core re-issues
RegisterReference (NOT AdviseSupervisory) for the buffered entry,
mirroring MxNativeSession.ReAdviseSubscription (cs:538-569).
Structural property is unit-tested; this confirms the engine
actually picks back up after the rebuild + replay.
Both tests pass live on this Galaxy:
cargo test -p mxaccess-compat --features live-windows-com \
--test buffered_unsubscribe_skip_live -- --ignored --nocapture
cargo test -p mxaccess-compat --features live-windows-com \
--test buffered_recovery_replay_live -- --ignored --nocapture
Pulls mxaccess-nmx + mxaccess-codec into mxaccess-compat dev-deps so
the recovery test can build a RebuildFactory closure that returns
NmxClient and bind a typed broadcast Receiver.
design/followups.md F49 -> Resolved (all five steps pass live).
docs/M6-live-verification.md updated with per-step evidence + repro
commands.
F49 is fully closed out. F55 (DCOM-managed INmxSvcCallback, Path A)
and F56 (missing EnsurePublisherConnected + post-RegisterReference
AdviseSupervisory for buffered) were the two real Rust-port bugs
uncovered along the way; both resolved. Remaining post-V1 followups
(F50 Suspend/Activate Frida, F51 ASB type matrix, F52 perf, F53 doc
lint, etc.) are scoped independently and not part of F49.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
5e11b30507 |
[F56 resolved] subscribe paths now drive 0x33 DataUpdate frames
Root cause: `Session::subscribe` and `Session::subscribe_buffered_nmx`
were missing the `INmxService2::Connect` + `AddSubscriberEngine` RPC
pair that the .NET reference's `MxNativeSession.EnsurePublisherConnected`
(`cs:516-526`) issues before the first advise against a publishing
engine. Without those two RPCs, NmxSvc accepted the subscription
registration but the publishing engine never knew our engine was
subscribed — so it never dispatched DataUpdate frames back.
Diagnosis driven by wwtools/aalogcli reading
C:\ProgramData\ArchestrA\LogFiles. The user pointed at this tooling
which lit up the path.
Red herring: NmxSvc's `[Warning] NmxCallback->DataReceived ... failed
with error 0x{N}` log lines turned out to be normal log spam where N
is the bufferSize of the inbound call, not a real error code. The
.NET reference's own probe triggers identical entries while still
receiving DataUpdate frames successfully.
Fix:
- SessionInner::publisher_endpoints — per-session HashMap<(platform_id,
engine_id), ()> cache mirroring MxNativeSession._publisherEndpoints.
- Session::ensure_publisher_connected — issues Connect +
AddSubscriberEngine, once per publisher endpoint per session.
- Session::subscribe + subscribe_buffered_nmx — both call it before
the wire advise.
- subscribe_buffered_nmx — additionally issues AdviseSupervisory after
RegisterReference. The .NET reference's RegisterBufferedItemAsync
only calls RegisterReference, but on this AVEVA install
RegisterReference alone produces the registration result + heartbeat
callbacks without ever starting DataUpdate dispatch; AdviseSupervisory
unblocks the dispatch.
Live verification (`TestMachine_001.TestChangingInt`, a tag that
updates >1×/s):
cargo test -p mxaccess-compat --features live-windows-com \
--test plain_subscribe_live -- --ignored --nocapture
cargo test -p mxaccess-compat --features live-windows-com \
--test buffered_subscribe_live -- --ignored --nocapture
Both pass — `cmd=0x32` SubscriptionStatus + sequence of `cmd=0x33`
DataUpdate frames flow as expected. Tests assert on the raw
Session::callbacks() broadcast (not the typed Subscription::next
DataChange path) because the engine reports quality=Uncertain
value=null for this attribute on this Galaxy — the wire-level
subscription is what F56 was about, not the value content.
DcomCallbackSink reverted to S_OK return for both DataReceivedRaw
and StatusReceivedRaw (the bytes-processed / sentinel HRESULT
experiments during diagnosis turned out to be irrelevant — the
"failed with error 0xN" logs come from NmxSvc regardless of the
return value).
design/followups.md F49 + F56 + docs/M6-live-verification.md updated:
F56 resolved, F49 steps 1 + 4 + 5 pass live, steps 2 + 3 pending
(now executable on this fixture).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
c6332c26a1 |
[F49 step 4 + step 5 + doc] live evidence: metrics smoke pass, M6-live-verification.md
F49 step 4 (F40 metrics smoke): - crates/mxaccess-compat/tests/metrics_smoke_live.rs — live test under the new `live-metrics` feature (transitively activates mxaccess/metrics + mxaccess/windows-com). Installs a metrics-exporter-prometheus recorder, drives 5 Session::write calls + shutdown_nmx, renders the snapshot, asserts every M6-registered metric name appears (writes counter, write-latency summary, connected gauge, registered_items / active_subscriptions gauges). Pass on the live AVEVA install. Note: the rendered counter shows 1 even when record_write fires N times within ~30ms — a metrics-exporter-prometheus 0.16 quirk under tight loops, not a Rust port bug. Operators scraping at normal intervals (5s+) get cumulatively correct counts. Documented in the test + in M6-live-verification.md so future runs aren't surprised. F49 status update (in design/followups.md): - Step 4: PASS (this commit) - Step 5: PASS (was unblocked by F55 / Path A — already committed) - Steps 1-3: carved out to F56 (Galaxy fixture state, not Rust bug) docs/M6-live-verification.md: - Per-step evidence table with test invocations + outcomes. - Sample Prometheus snapshot for step 4. - Reproduction commands for the live tests. - F56 explanation cross-referenced from step 1. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
df3457c54a |
[F56] subscribe / subscribe_buffered: split-form wire body + diagnose Galaxy fixture gap
Three real fixes + one architectural diagnosis:
1. Session::subscribe_buffered_nmx now sends the .NET-reference split
form on the wire:
item_definition = "<attr>.property(buffer)" (was: full reference)
item_context = "<object_tag_name>" (was: empty)
item_handle = SessionInner::next_item_handle.fetch_add(1)
(was: hardcoded 0)
Verified byte-identical against captures/082 + 094 by the existing
buffered_register_reference_parity unit tests. The
item_handle counter mirrors MxNativeCompatibilityServer's
_nextItemHandle++ at MxNativeSession.cs:613.
2. New live tests:
- tests/buffered_subscribe_live.rs (F49 step 1) — uses real Galaxy
metadata via SqlTagResolver + connect_nmx_auto, drives a
background writer at 500ms cadence to force value-changes,
drains DataChange events from Subscription.
- tests/plain_subscribe_live.rs — same harness over plain
Session::subscribe (NOT buffered), used to isolate whether
"no DataUpdate" is buffered-specific (it's not — both fail).
Both pull tracing-subscriber as a dev-dep so `RUST_LOG=trace`
surfaces dcom_sink + router activity.
3. mxaccess-galaxy/sql_resolver.rs: drop the inner-attribute
`#![cfg(feature = "galaxy-resolver")]` — the module-level cfg on
`pub mod sql_resolver` in lib.rs already handles this and Rust
1.85's clippy::duplicated_attributes lint flagged the duplicate
once mxaccess-compat dev-deps activated the feature.
4. F56 finding (diagnosis, NOT a bug fix): the engine on this Galaxy
install does not have an active value for TestChildObject.TestInt.
Confirmed by running the .NET reference's own probe:
dotnet run --project src/MxNativeClient.Probe -c Release \
-- --probe-session-subscribe --tag=TestChildObject.TestInt \
--subscribe-hold-seconds=10
...returns ONE 0x32 SubscriptionStatus (status=3 detail=3
quality=0x00C0 Uncertain value=null) and zero 0x33 DataUpdates —
matching the Rust port's symptom exactly. Not a Rust port bug,
not a wire-byte gap. F49 steps 1-3 need either an actively-
scanned tag or local Galaxy reconfiguration to scan
TestChildObject.TestInt.
Workspace tests + clippy clean under both feature configurations.
F56 entry in design/followups.md updated with the full diagnostic
chain so future-me / future-collaborators can pick it up without
re-tracing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
af15fe7587 |
[F49 step 1 + F56] callback router: peel envelope before parsing subscription / 0x11 frames
The router used to call NmxSubscriptionMessage::parse_inner directly on the COM-stub-delivered body, but the wire bytes arrive wrapped in a ProcessDataReceived envelope (46-byte header + optional 4-byte length prefix); parse_inner expects post-envelope bytes. Result: every 0x33 DataUpdate that ever arrived was silently dropped. Mirrors the .NET reference's MxNativeSession.OnCallbackReceived flow at cs:582-606 — three sequential parse attempts: 1. NmxOperationStatusMessage::try_parse_process_data_received_body (already wired) 2. NmxReferenceRegistrationResultMessage::try_parse_... (NEW — was missing) 3. NmxSubscriptionMessage::try_parse_process_data_received_body (NEW — was wrong) Adds: - NmxSubscriptionMessage::try_parse_process_data_received_body — peels envelope via NmxObservedEnvelope::parse_process_data_received_body_flexible, then dispatches to existing parse_inner. - NmxReferenceRegistrationResultMessage::try_parse_process_data_received_body — same shape, for the 0x11 registration-result frame. - Router branch for 0x11 — currently traces the assigned item_handle and drops the frame (matches the .NET reference, which fires a ReferenceRegistrationReceived event with no consumer in the codebase). - Router fall-through trace! when neither path matches, so future unparseable bodies surface in RUST_LOG=trace instead of vanishing. - DcomCallbackSink::forward — trace! per inbound callback so RUST_LOG=mxaccess_callback=trace surfaces opnum + size. - crates/mxaccess-compat/tests/buffered_subscribe_live.rs — F49 step 1 live test that drives subscribe_buffered + a 500ms-cadence writer. Also pulls tracing-subscriber as a dev-dep so the test can dump router activity. Existing router_task_decodes_callback_invoked_into_broadcast unit test updated to wrap its synthetic 0x32 body in an envelope so the new parse path actually accepts it. Live result: F56 — the buffered round-trip *registers* successfully (RegisterReference returns HRESULT 0; engine sends one 0x11 RegistrationResult + one 51-byte op-status per write, perfectly clocked) but the engine never sends a 0x33 DataUpdate. Rust-port- specific gap vs the .NET reference's working buffered path; root cause is likely a field-level difference in the RegisterReference body or a missing post-RegisterReference step. Captured as F56 in design/followups.md, blocking F49 step 1; F56's DoD is the same live test reporting >=3 DataChange arrivals. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
2fc327a8d5 |
[F55 Path A] DCOM-managed INmxSvcCallback sink
Replace the hand-rolled CallbackExporter (TCP listener + custom
OBJREF) with a real `windows-rs` `#[implement]` COM class for
INmxSvcCallback, marshalled via CoMarshalInterface. NmxSvc validates
the callback OBJREF by calling IObjectExporter::ResolveOxid against
the local RPCSS at 127.0.0.1:135; hand-rolled OXIDs aren't registered
there, which is why RegisterEngine2 returned RPC_S_SERVER_UNAVAILABLE
(1722) on every live attempt. CoMarshalInterface registers the OXID
with RPCSS automatically, so the SCM-side resolution succeeds.
Mirrors MxNativeSession.CreateRegisteredService (cs:624), which is
the .NET reference's working path:
ComObjRefProvider.MarshalInterfaceObjRef(callback,
INmxSvcCallback, DifferentMachine)
Layout:
- mxaccess-callback::dcom_sink — INmxSvcCallback + DcomCallbackSink
+ create_dcom_callback_sink_objref. Forwards inbound calls into
the same CallbackEvent::CallbackInvoked { opnum, body } shape the
legacy exporter produces, so callback_router stays path-agnostic.
- Session::from_nmx_client — branched on `windows-com`. Real DCOM
sink when on; legacy CallbackExporter when off (kept for unit
tests that run against an in-process fake NMX peer).
- SessionInner.dcom_sink_holder: Option<IUnknownHolder> — keeps the
COM ref alive for the session's lifetime; shutdown_nmx drops it.
- mxaccess-rpc + mxaccess-callback: windows-rs 0.59 → 0.62. The 0.59
#[implement] macro generates code that doesn't compile under
edition 2024; 0.62 is fixed.
Live result: cargo test -p mxaccess-compat --features
live-windows-com --test lmx_write_complete_live -- --ignored
--nocapture passes end-to-end. RegisterEngine2 OK, write
round-trips, OnWriteComplete fires with the captured MxStatus shape.
Unblocks F49 step 5; F55 marked Resolved in design/followups.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
0a274af76f |
[F55] Path C investigation: NmxSvc requires SCM-registered OXID for callbacks
Captured OBJREF byte structures from both paths via the .NET probe: - `--probe-callback-marshal`: DCOM-marshalled, 338 bytes, succeeds (when used inside `MxNativeSession.Open` → `CreateRegisteredService`). - `--probe-register-managed-callback`: hand-rolled, 162 bytes, fails with `RegisterEngine2 → 0x800706BA RPC_S_SERVER_UNAVAILABLE`. The structural diff: - `std_flags`: DCOM=`0x0A80` (SORF_OXRES4+6+8) vs hand-rolled=`0x280` (SORF_OXRES4+6). Bit `0x0800` (SORF_OXRES8) only set in DCOM. - ncacn_ip_tcp bindings: DCOM=4 with no ports; hand-rolled=1 with explicit `[port]`. - Total size: 338 vs 162 bytes. Tested the simplest fix (hand-rolled `std_flags = 0x0A80` to match DCOM): **still fails with the same 1722.** Reverted. **Diagnosis updated in F55:** NmxSvc on receiving RegisterEngine2 appears to call `IObjectExporter::ResolveOxid` against the local SCM (`127.0.0.1:135`) to resolve the callback OBJREF's OXID, then dial the resulting bindings. Our hand-rolled OXID is never registered with RPCSS, so the SCM-side resolution fails and NmxSvc returns RPC_S_SERVER_UNAVAILABLE — matching: - the symptom (1722), - the sub-second timing (no TCP dial-back to our listener attempted), - the fact that the .NET `ManagedCallbackExporter` (same hand-rolled approach) ALSO fails identically. DCOM marshalling fixes this because `CoMarshalInterface` internally registers the OXID with RPCSS. The bindings have no port because RPCSS returns the dynamic port from the DCOM stub layer. **Conclusion: Path A is the architecturally correct fix** — the callback exporter must be a DCOM-managed object (e.g. via `windows-rs` `#[implement]`) for NmxSvc to accept the callback. The hand-rolled-listener-with-explicit-port approach is fundamentally incompatible with NmxSvc's callback validation, in both Rust and the .NET reference. Path C (cheap investigation) is exhausted; F55 verdict updated to recommend Path A explicitly. `cargo test --workspace` 824 passing; clippy `-D warnings` clean across both feature configurations. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
c5d611d6fa |
[F12 partial + F55] hold IUnknown for client lifetime + diagnose RegisterEngine2 1722
**F12 partial improvement** (`mxaccess-rpc::IUnknownHolder` + `mxaccess-nmx`):
- New `IUnknownHolder` newtype that owns an MTA-resident COM proxy
with `unsafe impl Send + Sync`. Mirrors the .NET reference's
`ManagedNmxService2Client._activatedComObject` private field
(`cs:15`).
- New `activate_and_marshal_iunknown_objref(prog_id, ctx)` returns
`(Vec<u8>, IUnknownHolder)`. Existing
`marshal_activated_iunknown_objref` retained as a wrapper that
drops the holder (kept for inline-use callers).
- `NmxClient` gains an `activated_com_object: Option<IUnknownHolder>`
field, populated by `Self::create` from the new helper.
`Self::connect` / `Self::from_bound_transport` set it `None` (no
COM activation in those paths).
- Holding the IUnknown for the client's lifetime keeps the
SCM-tracked OXID valid; without it the COM ref count drops to
zero and the SCM may release the activated server-side instance,
making subsequent `ResolveOxid` / `RemQueryInterface` calls
return `RPC_S_SERVER_UNAVAILABLE`.
**F55 (new) — hand-rolled callback exporter rejected by RegisterEngine2**
Five-step instrumentation of `Session::connect_nmx_auto` proves all
six COM-activation / RemQI / final-bind steps succeed. The 1722
fault originates at `RegisterEngine2` itself:
```
from_nmx_client: callback hostname="DESKTOP-6JL3KKO" port=57886 obj_ref_len=162
from_nmx_client: callback obj_ref hex: 4d454f57010000...
from_nmx_client: RegisterEngine2 (31112, mxaccess.31112)
from_nmx_client: RegisterEngine2 FAIL: Transport(Fault { status: 2147944122 })
```
Status `0x800706BA` = `RPC_S_SERVER_UNAVAILABLE` wrapped as Win32
HRESULT.
**Critical finding: the .NET reference's `--probe-register-managed-callback`
(which uses the same hand-rolled `ManagedCallbackExporter` approach
as the Rust port) ALSO fails with the same `0x800706BA` fault.**
Only `--probe-session-write`, which uses
`ComObjRefProvider.MarshalInterfaceObjRef(callback, ...)` to build
the OBJREF via Windows DCOM proxy/stub marshalling, succeeds. So
this is an architectural artifact of the hand-rolled-callback
design, not a Rust port regression.
`design/followups.md` F55 entry documents the three resolution
paths (switch to DCOM-marshalled callback / hybrid / continue
investigating OBJREF rejection at NmxSvc).
F49 stays open with a refined diagnostic — the per-feature live
verification is gated on F55's resolution.
Workspace tests still 824 passing; clippy `-D warnings` clean
across both feature configurations.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
e5b31fadb1 |
[F49] live-test scaffolding for F54 OnWriteComplete + COM probe diagnostic
Live attempt against AVEVA on this dev host produced two artefacts:
**`crates/mxaccess-compat/tests/lmx_write_complete_live.rs`** — the
F54 OnWriteComplete round-trip test. Compiles + runs against the
live AVEVA install via either path:
- `--features live-windows-com` (preferred): uses
`Session::connect_nmx_auto` so the COM activation reference is
held in-process for the duration of the test.
- Default features (fallback): shells out to
`MxNativeClient.Probe --probe-resolve-oxid-managed-ntlm-integrity`
+ `--probe-remqi-managed` to learn the per-session NMX endpoint +
INmxService2 IPID, then uses `Session::connect_nmx`.
Both code paths are wired and the test runs through endpoint
resolution + IPID extraction successfully. The connect step itself
fails with `Status { detail: 1722 }` (RPC_S_SERVER_UNAVAILABLE).
**`crates/mxaccess-rpc/examples/com-marshal-probe.rs`** — minimal
one-shot binary that calls
`marshal_activated_iunknown_objref("NmxSvc.NmxService",
DifferentMachine)` in isolation. Confirms the COM activation +
CoMarshalInterface chain works fine standalone (returns a 338-byte
OBJREF with valid OXID/IPID structure). The 1722 in the live test
is therefore downstream of the activation — likely a COM-apartment
threading interaction with the tokio multi-thread runtime.
This is an F12-related issue (auto-resolve hardening), not an F54
issue. F54's correctness is covered by the existing unit-level
integration tests:
- `mxaccess::session::tests::router_populates_operation_status_context_from_pending_ops_fifo`
- `mxaccess::session::tests::write_handle_correlates_with_router_emitted_status`
- `mxaccess_compat::tests::drain_routes_write_status_to_on_write_complete`
- `mxaccess_compat::tests::drain_routes_non_write_status_to_on_operation_complete`
`design/followups.md` F49 entry updated to reflect:
- F54 added as a fifth row in the live-verification scope.
- "Live attempt 2026-05-06" sub-section documents the 1722 issue +
what was verified (.NET probe end-to-end works against same
install; Rust COM activation works in isolation; the failure is
Rust-port-specific to `connect_nmx_auto` under tokio).
- F49 now Blocked-by F12 hardening (the 1722 path).
New `live-windows-com` feature on `mxaccess-compat` propagates to
`mxaccess/windows-com` for the test binary.
Workspace 824 → 824 tests; clippy + rustdoc clean across both
feature configurations.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
04c10babfb |
[F54 test] end-to-end smoke: write_with_handle ↔ callback_router boundary
Adds `write_handle_correlates_with_router_emitted_status` — the
closest-to-live test we can write without an AVEVA endpoint, pinning
the F54 boundary the C# `OnWriteComplete` callback ultimately depends
on.
The existing tests cover the layers individually:
- `write_value_with_handle_inserts_into_pending_ops` — write API
populates pending_ops with the right correlation id.
- `router_populates_operation_status_context_from_pending_ops_fifo`
— callback_router consumes a frame + the registry, emits a typed
OperationStatus with context attached.
- `drain_routes_write_status_to_on_write_complete` (mxaccess-compat)
— drain function routes Write op_kind to on_write_complete_tx.
What was missing: a test that combines the public `write_value_with_
handle` API with a real callback_router invocation against the SAME
`pending_ops` Arc the write populated. The new test:
1. Builds a Session via `connect_test_session`.
2. Calls `session.write_value_with_handle("TestObj.TestInt", ...)` —
gets a real `WriteHandle { correlation_id }` and a real entry in
`pending_ops` (no manual insertion).
3. Spins a parallel `callback_router` over the SESSION's
`pending_ops` Arc + a fake event_tx (the live exporter's
internal channel isn't reachable from tests; this is the
established workaround pattern from
`router_task_decodes_callback_invoked_into_broadcast`).
4. Injects the proven `WRITE_COMPLETE_OK` 5-byte frame.
5. Asserts the emitted `OperationStatus.context.correlation_id`
equals the cid the write returned, that op_kind is Write, that
reference is the original tag string, and that
`pending_ops` is now empty (one-shot popped).
This closes the integration-test gap the user flagged. Live AVEVA
verification still falls under F49.
Workspace 823 → 824 tests; clippy + rustdoc clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
4ff511bbed |
[F54] per-operation correlation + compat OnWriteComplete fan-out
Closes the residual that R3/R4 Path A's commit `c73a33e` deferred:
the OperationStatus.context field was always None because no
in-flight correlation map existed in SessionInner, and the
mxaccess-compat broadcast channels for OnWriteComplete /
OperationComplete were exposed on the public API but had no
fan-out task draining session events into them.
**mxaccess (Part 1 — per-operation correlation):**
- New `pending_ops: Mutex<HashMap<[u8; 16], OperationContext>>` on
SessionInner. Populated when `Session::write*` / `subscribe*`
dispatches an outstanding operation; entry removed when the
matching OperationStatus event fires (one-shot semantics).
- New `Session::write_with_handle` (and equivalents for the secured /
timestamped paths) returns a `WriteHandle { correlation_id }` so
consumers can correlate completions back to their originating
call. Existing `write` / `write_value` / etc. signatures unchanged
and delegate to the handle-returning variant.
- Callback router extended to look up `pending_ops` by correlation_id
on each operation-status event. When found, populates
`OperationStatus.context: Some(OperationContext { correlation_id,
op_kind, reference, retry_count: 0 })`. When not found, falls
through with `context: None` (verbatim-preserve per CLAUDE.md).
- New unit tests assert: matching correlation_id populates context,
unknown correlation_id leaves context None, the entry is removed
from `pending_ops` after one event fires.
**mxaccess-compat (Part 2 — compat-layer fan-out):**
- New `correlation_to_item: tokio::sync::Mutex<HashMap<[u8; 16], i32>>`
on LmxClientInner.
- `LmxClient::write` / `write_2` / `write_secured` / `write_secured_2`
call `Session::write_with_handle` (or equivalent) and insert
`correlation_id → item_handle` into the map before returning.
- `LmxClient::register` / `register_asb` spawn a background task that
drains `session.operation_status_stream()`. Per event, looks up
`correlation_to_item[event.context?.correlation_id]` to find the
item_handle, then routes:
- `OperationKind::Write` / `OperationKind::WriteSecured` →
`WriteCompleteEvent { server_handle, item_handle, statuses,
is_during_recovery }` into `on_write_complete_tx`.
- Other variants → `OperationCompleteEvent { ... }` into
`on_operation_complete_tx`.
- Removes the correlation_id from `correlation_to_item` after
firing (one-shot).
- Events with no matching item_handle (correlation_id not in map)
are dropped silently — no bogus item_handle=0 events.
- Task cancelled on LmxClient drop via `JoinHandle::abort` (matches
the existing `subscription_task` pattern).
- New unit tests cover: Write op routes to on_write_complete, Read
op routes to on_operation_complete, unknown correlation_id is
dropped.
Result: the C# `LMX_OnWriteComplete(int hLMXServerHandle, int
phItemHandle, ref MXSTATUS_PROXY[] pVars)` callback shape is now
end-to-end-achievable. A consumer calls `LmxClient::write(hServer,
hItem, value, userId)` and drains `client.on_write_complete()`; the
yielded `WriteCompleteEvent` carries the right `(server_handle,
item_handle, statuses, is_during_recovery)` tuple.
Public API: `Session::write_with_handle` + `WriteHandle` are new;
existing signatures unchanged. `cargo public-api` baselines
regenerated under `design/public-api/{mxaccess,mxaccess-compat}.txt`.
Workspace: 765 → 823 tests pass (~58 new tests from F54). Clippy
`-D warnings` clean. Rustdoc `-D warnings` clean.
F54 status in `design/followups.md` moved Open → Resolved.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
f98ab9846d |
design/70-risks: record the .NET reference's WriteCompleted half-implementation
R3's verdict gains an aside documenting why the original native MxAccess `OnWriteComplete` event has historically only fired for the one exact 5-byte pattern `00 00 50 80 00` (= `MxStatus.WriteCompleteOk`). Verified at: - `src/MxNativeClient/MxNativeCompatibilityServer.cs:756` — `if (!evt.Message.IsMxAccessWriteComplete) return;` gates the consumer-facing `WriteCompleted` event. - `src/MxNativeCodec/NmxOperationStatusMessage.cs:18` — `IsMxAccessWriteComplete` requires `Format == StatusWord && StatusCode == 0x8050 && CompletionCode == 0x00`. Every other completion frame is silently dropped — the 1-byte `0x00`/`0x41`/`0xEF` ones, plus any non-success status word. This was the underlying reason R3/R4 looked unsolvable for a year: the answer "we don't know how to map" was actually "the native compatibility shim deliberately doesn't map these because firing typed failure events on ambiguous bytes was never a goal." Path A's `MxStatus::from_packed_u32` (commit `c73a33e`) closes the asymmetry on the Rust side: `Session::operation_status_events()` exposes ALL typed outcomes the upstream synthesizer produces, not just the WriteCompleteOk slice. The Rust port now has strictly broader operation-status visibility than the .NET reference offered. Recorded so future contributors don't re-derive this from scratch. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
c73a33edd8 |
[R3/R4 Path A] mxaccess: port Lmx.dll FUN_10100ce0 synthesizer kernel
Path A landed for R3/R4. The byte->MxStatus synthesizer in Lmx.dll is
FUN_10100ce0 (`analysis/ghidra/exports/Lmx.dll.synthesizer-helpers2-decompile.md`),
a 4-byte u32 LE -> 4-tuple MxStatus decoder used by every NMX-frame
parser in Lmx.dll. The kernel is byte-deterministic and context-free,
so it ports as a pure function -- the operation-tracking state
machine the original verdict deferred is NOT required for synthesis.
Bit layout (per FUN_10100ce0 lines 21-24):
bit 31: success (-1 if set, 0 if clear)
bits 27..24: category (4 bits)
bits 23..20: detected_by (4 bits)
bits 15..0: detail (i16 -- low 16 bits, signed)
bits 30..28, 19..16: reserved/padding
Codec changes:
- MxStatus::from_packed_u32() / ::to_packed_u32() -- the kernel +
inverse for round-trip parity.
- MxStatus::from_nmx_response_code() -- the constructed-from-response-
code switch in FUN_1010bd10:741-770 (six proven mappings: 0x01, 0x02
-> CommunicationError + RequestingNmx; 0x03 -> ConfigurationError +
RequestingNmx; 0x04 -> ConfigurationError + RespondingNmx; 0x05 ->
CommunicationError + RespondingNmx; 0x1A -> CommunicationError +
RequestingNmx).
- MxStatusCategory / MxStatusSource: from_i16/to_i16 promoted to const
fn so MxStatus::from_packed_u32 can be const.
- NmxOperationStatusMessage::try_parse_process_data_received_body() --
thin wrapper that peels the outer NmxObservedEnvelope before
delegating to try_parse_inner. Mirrors
NmxOperationStatusMessage.TryParseProcessDataReceivedBody (.NET cs:20-32).
- NmxOperationStatusMessage::promote_to_typed() -- entry point that
returns the existing Status field. Documented as a no-op pass-through
for now (the 5-byte inner-body wire shape is NOT the same field as
the 4-byte packed-u32 the kernel decodes); kept for API symmetry.
- 22 new round-trip tests covering the kernel, the response-code
switch, the proven 0x00/0x41/0xEF completion bytes, and round-trip
for every canonical sentinel.
mxaccess (Session) changes:
- New OperationKind enum (Write/WriteSecured/Read/Subscribe/
Unsubscribe/Activate/Suspend/Other).
- New OperationContext struct (correlation_id, op_kind, reference,
retry_count) -- ground for the F54 follow-on per-operation
correlation work.
- New OperationStatus event type {raw, status, context,
is_during_recovery}, mirroring MxNativeOperationStatusEvent (cs:73-78)
with the typed-MxStatus addition.
- Session::operation_status_events() -> broadcast::Receiver<Arc<
OperationStatus>> + operation_status_stream() Stream variant.
- callback_router() now tries operation-status parsing first, falling
through to subscription messages -- matches MxNativeSession
.OnCallbackReceived dispatch order (cs:574,582,590).
- recover_connection() flips a recovery_active counter (Arc<AtomicU32>
shared with the router) so OperationStatus.is_during_recovery is
populated correctly. Mirrors MxNativeSession._recoveryActive
Volatile.Read at cs:573.
- 3 new router tests covering: status-word frame dispatch + typed
promotion to WriteCompleteOk; completion-only frames stay verbatim;
is_during_recovery is stamped from the live counter.
Per-operation context tracking (correlating completion frames back to
outstanding writes/subscribes via the correlation_id) is filed as F54
in design/followups.md. The synthesizer kernel itself is byte-
deterministic, so the kernel and the correlation work are decoupled.
Ghidra evidence (the next-ring xref walk beyond FUN_10114a90):
- analysis/ghidra/exports/Lmx.dll.set-attribute-result-xrefs.md --
xrefs to OnSetAttributeResult / CancelWithStatus / OperationComplete.
- analysis/ghidra/exports/Lmx.dll.vtable-data-xrefs.md -- vtable-slot
data xrefs for the virtual-dispatch path.
- analysis/ghidra/exports/Lmx.dll.synthesizer-decompile.md --
ScanOnDemandCallback::OperationComplete/MultipleOperationComplete
(FUN_1010b990), RemotePlatformResolver::OperationComplete
(FUN_1010dc80), and the constructed-from-responseCode synthesizer
in FUN_1010bd10 (lines 698-770). FUN_1010bd10 is the wire-frame
receiver that drives the synthesis.
- analysis/ghidra/exports/Lmx.dll.synthesizer-helpers-decompile.md --
FUN_10003fc0 (the <success %d category %d ...> formatter; confirms
the 4-tuple layout), FUN_1008f150 (dispatch helper).
- analysis/ghidra/exports/Lmx.dll.synthesizer-helpers2-decompile.md --
FUN_10100ce0 (the kernel itself), FUN_10100bc0 (3xu16 reader),
FUN_1005e580 (4-byte stream reader), FUN_1010ee00 (sister NMX-frame
parser using the same kernel).
- analysis/ghidra/exports/Lmx.dll.synthesizer-callers-xrefs.md --
caller graph; confirms the kernel is called from many wire-frame
parsers but each parser shares the single 4-byte decoder.
R3/R4 verdict updated in design/70-risks-and-open-questions.md from
"settled at verbatim-preserve" to "settled per Path A". F54 filed in
design/followups.md for the per-operation correlation work.
cargo build / test / clippy -D warnings / RUSTDOCFLAGS=-D warnings doc
all clean. cargo public-api baselines regenerated for mxaccess and
mxaccess-codec.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
460c61df43 |
[R3/R4] Path-A trace: synthesizer is in Lmx.dll's NMX-frame decoder
Five-stage Ghidra headless decompile traces the byte-to-MXSTATUS_PROXY synthesis path end-to-end across LmxProxy.dll and Lmx.dll. New evidence files committed alongside R3/R4 verdict update: - analysis/ghidra/exports/LmxProxy.dll.fire-event-xrefs.md - analysis/ghidra/exports/LmxProxy.dll.status-synthesis-decompile.md - analysis/ghidra/exports/LmxProxy.dll.mxstatus-safearray-decompile.md - analysis/ghidra/exports/Lmx.dll.set-attribute-result-decompile.md Layer-by-layer findings (bytes flow inward; synthesis flows outward): 1. `Lmx.aaDCT` at 0x10178fc0 is `SysAllocString(L"Lmx.aaDCT")` — a tracing category BSTR, not a table. 2. `MXSTATUS_PROXY` is a 16-byte marshalled struct (4 × i16 padded to i32 boundaries with Pack=4) — the OUTPUT of synthesis, not a lookup entry. 3. `LmxProxy.dll` Fire_* event handlers receive already-populated `MXSTATUS_PROXY[]` and forward through ATL dispatch — no synthesis. 4. `LmxProxy.dll` Fire_* CALLERS (FUN_1001657f / FUN_10016b50 / FUN_10016d4b) call FUN_10003f60(out_safearray, in_status_ptr, count=1) which is a VERBATIM memcpy of an existing 14-byte buffer into the SAFEARRAY — no transformation. 5. `Lmx.dll`'s `PreboundReference::OnSetAttributeResult` (FUN_10114a90) receives an already-populated `short *param_7` status buffer. Log line confirms the layout: `<success %d category %d detectedBy %d detail %d>`. Dispatches on typed values — synthesis is upstream of this function too. The synthesizer is the NMX-frame decoder in Lmx.dll that calls OnSetAttributeResult / OnGetAttributeResult / equivalent OperationComplete handler. The decoder takes raw NMX bytes plus operation context (item handle, engine state, retry state, correlation id) and computes the populated MXSTATUS_PROXY. There is NO static lookup table — synthesis is per-message contextual. Two viable paths to typed promotion (both substantial; neither a small codec patch): - Path A: port the synthesizer. ~1-2 weeks. Requires extending the Rust session to track per-operation context (handles, retries, correlation ids). Out of V1 scope. - Path B: empirical capture pairs. ~30 min × 6-10 scenarios. Output is a (byte, context → status) mapping that approximates without re-implementing. Risk: mapping is only valid for captured contexts. R3/R4 stay settled at verbatim-preserve. The .NET reference does the same for the same reason: the synthesizer is too context- dependent to mirror without porting the entire operation-tracking state machine. Reopen criteria sharpened: either (a) a consumer files a concrete use case for typed promotion of a specific byte+context combination (Path B's empirical capture for that one combination is the cheapest answer); or (b) a major-version bump justifies the state-machine port (Path A). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
4dfc0cee65 |
[R3 + R4 + R8] settle protocol-level risks per Ghidra evidence
Ghidra headless decompile of `Lmx.dll`'s `aaDCT` symbol + the `LmxProxy.dll` Fire_* event handlers (logs at `analysis/ghidra/exports/Lmx.dll.aadct-decompile.md` and `analysis/ghidra/exports/LmxProxy.dll.completion-status-decompile.md`) settles **R3** and **R4** as "no static byte→status lookup table exists": - `Lmx.aaDCT` at `0x10178fc0` is a `SysAllocString(L"Lmx.aaDCT")` into a global BSTR — a logging category name, not a table. - `MXSTATUS_PROXY` is a 4-field struct (success/category/detectedBy/ detail), used as the marshalled COM event payload — not a static array of pre-mapped statuses. - `Fire_OnDataChange` / `Fire_OnWriteComplete` / `Fire_OperationComplete` / `Fire_OnBufferedDataChange` (RVAs 0x15f72, 0x1611f, 0x16271, 0x163c0 in `LmxProxy.dll`) receive ALREADY-POPULATED `MXSTATUS_PROXY[]` arrays — the byte-to-struct synthesis happens upstream in the proxy's NMX-callback ingestion code, not via a table lookup. The synthesis is per-event computation from operation context (engine ids, item handles, retry counters), not a static promotion. R3/R4 status updated from "indefinitely deferred — no Ghidra table" to "settled — no table exists; verbatim preservation is the canonical answer." The .NET reference's `NmxOperationStatusMessage.TryParseInner` + the Rust port's `mxaccess-codec/src/operation_status.rs` already match this canonical behaviour; no code change required. Reopen R3/R4 only if a context-aware capture surfaces a per-byte synthesis logic that depends on operation context — at which point the codec would need access to the originating operation's context, which is upstream of the bytes themselves. **R8** marked permanently deferred — implementation already parses NTLM AV pairs per [MS-NLMP] §2.2.2.1 (including the cross-domain shapes `MsvAvDnsTreeName` / `MsvAvDnsComputerName` carrying the trusted-domain DNS suffix), what's missing is the live capture, and the live capture requires a multi-domain Windows lab not available on this dev host. Same status pattern as F3 in `design/followups.md`. Open evidence gaps table updated to reflect: - Cross-domain NTLM: deferred (R8) - Ghidra mapping table for completion-only bytes: no table exists (R3/R4 settled) - Activate/Suspend transition (wire): partial (F44 + F46), live re-run pending (F50) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
0e93e3a8fa |
design/followups: file F48-F53 for known V1 residuals
After the M6 closeout sweep (F35-F47 all resolved), six residuals remain that were either documented inline in design docs or implied by closure language but not formally tracked as F-numbers. File them explicitly so the next iteration has a clean starting list: - F48: Execute `cargo publish` for V1 (F43 was dry-run only). Documents the 9-crate dependency-ordered publish sequence + the version bump from 0.0.0 placeholder to 0.1.0. - F49: Live verification sweep for F36 (buffered subscribe) + F45 (recovery replay) + F47 (unsubscribe skip) + F40 (metrics). Closes the gap where these features ship with unit tests but weren't live-exercised against AVEVA in the closing iteration. - F50: Run the F46 Suspend/Activate Frida capture live (script ready, capture deferred to maintainer-side). - F51: Live type-matrix expansion for `asb-subscribe` — Bool / Float / Double / String / DateTime / Duration / arrays. F32 was closed via "deployable maximum" but the codec supports more types than the live matrix exercises. - F52: Codec performance optimisations from F39 (BytesMut output buffer, name-signature cache, session scratch pool). Documented as post-V1 in M6-bench-baseline.md; filing them as F-numbers so the alloc-count deltas are tracked when they land. - F53: Enable `#![warn(missing_docs)]` workspace-wide. Deferred from F42 — the lint surfaces hundreds of low-priority gaps that need a dedicated pass. R3, R4, R8, F3 already in their respective tracking docs (the risks register + the Open section's permanently-external-blocked entry). Open section now contains: F3 (permanently external-blocked) + F48-F53 (V1-residual triage). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
25befcb72e |
design/followups: move F45 + F47 to Resolved (M6 + spawned closures)
F45 (commit |
||
|
Joseph Doherty
|
1a1830f3bf |
[F47] mxaccess: unsubscribe skips UnAdvise for buffered subscriptions
Mirrors the .NET reference's `if (!subscription.IsBuffered)` guard
at `MxNativeSession.cs:361-381`. The Rust port previously emitted an
`UnAdvise` frame for both plain and buffered subscriptions; the
buffered server-side registration is unwound by the engine when the
`RegisterReference` handle goes away, so emitting an `UnAdvise` for
buffered entries is at best a no-op extra frame and at worst could
race with the engine's own teardown.
Fix: branch `Session::unsubscribe` on `SubscriptionEntry::mode` (the
discriminator F45 added). For `SubscriptionMode::Buffered { ... }`,
skip the `un_advise` call and proceed directly to registry cleanup.
For `SubscriptionMode::Plain`, retain the previous behaviour.
The registry-entry probe runs first (separate lock acquisition) so
the `is_buffered` decision doesn't hold the NMX-client mutex
unnecessarily — common case where the entry is plain still acquires
the NMX lock immediately after.
The metrics counter `record_unadvise()` still fires on every public
`unsubscribe` call regardless of mode — it tracks consumer-side
unsubscribe rate, not wire-frame rate. That matches what dashboards
expect from the public API.
New unit test `unsubscribe_skips_un_advise_for_buffered_subscription`
issues a plain subscribe (recorded as 1 RPC), mutates the registry
entry to `SubscriptionMode::Buffered`, calls unsubscribe, and
asserts the recorded RPC count stays at 1 (no UnAdvise emitted).
The existing `subscribe_populates_registry_unsubscribe_clears_it`
test serves as the negative control for the plain branch.
Workspace 794 → 795 tests; clippy clean; rustdoc clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
9b57cf8f3b |
[F45] mxaccess: recovery replay re-issues RegisterReference for buffered subs
`Session::recover_connection_core` previously walked
`SessionInner::subscriptions` and replayed every entry via
`AdviseSupervisory`, which lost the `.property(buffer)` registration
on buffered subscriptions — silently downgrading buffered → plain on
transport rebuild.
Fix:
- New `pub(crate) enum SubscriptionMode { Plain, Buffered { ... } }`
discriminator carried on each `SubscriptionEntry`. Buffered variant
retains the un-suffixed reference + the rounded interval (so the
re-issued buffered registration matches the original cadence) +
the empty `item_context` / zero `item_handle` matching the wire
send.
- `Session::subscribe` (plain path) records `SubscriptionMode::Plain`.
`subscribe_buffered_nmx` records `SubscriptionMode::Buffered { ... }`.
- `recover_connection_core` matches on `entry.mode`. Plain branch
unchanged. Buffered branch re-applies `.property(buffer)` via
`to_buffered_item_definition` (idempotent), rebuilds the original
`NmxReferenceRegistrationMessage` with the saved correlation id +
`subscribe = true`, and dispatches `register_reference` (kind=
ItemControl, inner command 0x10) against the replacement
transport. Mirrors `MxNativeSession.ReAdviseSubscription`
(`MxNativeSession.cs:538-569`).
New unit test `recover_connection_replays_buffered_subscription_via_
register_reference` synthesises a buffered registry entry, installs a
`RebuildFactory` pointing at a recording NMX server, drives
`recover_connection`, then asserts the recorded `TransferData` carries
inner command `0x10` (NOT `0x1f`) with the `.property(buffer)`-
suffixed item_definition + the saved correlation id + subscribe=true.
Side-finding worth filing separately: `Session::unsubscribe`
unconditionally calls `un_advise` for both plain and buffered
entries, but the .NET reference's `Unsubscribe`
(`MxNativeSession.cs:361-381`) skips `UnAdvise` for buffered
(`if (!subscription.IsBuffered)`). Out of scope for F45 (recovery-
only); will file as F47.
Public API unchanged. `SubscriptionMode` + `SubscriptionEntry` stay
`pub(crate)` — `cargo public-api -p mxaccess` baseline is unchanged.
Workspace 793 → 794 tests; clippy clean; rustdoc clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
2281309a86 | design/followups: move F46 to Resolved (Frida hooks landed) | ||
|
Joseph Doherty
|
808fea18a0 |
[F46] analysis/frida: Suspend/Activate hooks + R5 next-step
Closes the wire-side gap left by capture 077 in F44's R5 walk. The Frida
script now hooks the production LmxProxy.dll dispatchers so a future live
re-run on the AVEVA host can answer "does CLMXProxyServer issue a separate
ORPC method for Suspend/Activate, or are they synthesised client-side?"
Hooks added in `analysis/frida/mx-nmx-trace.js`:
- `LmxProxy.dll!CLMXProxyServer.Suspend` @ RVA 0x13d9c (FUN_10013d9c)
- `LmxProxy.dll!CLMXProxyServer.Activate` @ RVA 0x14028 (FUN_10014028)
Both RVAs were extracted from
`analysis/ghidra/exports/LmxProxy.dll.string-refs.tsv` rows 119/122 (the
`CLMXProxyServer::Suspend - Server Handle` / `Activate - Server Handle`
log strings each xref one function — same pattern as the existing
AdviseSupervisory hook at 0x142b4). The hooks emit `mx.suspend.begin/end`
and `mx.activate.begin/end` events with serverHandle, itemHandle, and the
`MxStatus*` out parameter decoded as 4 x int16 (Success / Category /
DetectedBy / Detail per `src/MxNativeCodec/MxStatus.cs`). Naming matches
the F46 spec's `mx.<verb>.begin / end` grep convention rather than the
generic `call.enter / leave` shape because we want to filter these out
of large traces without false positives from other LmxProxy entrypoints.
No `Resume` / `Reactivate` exports exist in `LmxProxy.dll` — verified
against `analysis/ghidra/exports/LmxProxy.dll.ghidra.md` (no such string
xrefs) and the decompiled `ILMXProxyServer5` / `ILMXProxyServer4`
interfaces under `analysis/decompiled-mxaccess/ArchestrA/MxAccess/`
(only Suspend and Activate are declared on the dispatch interface).
The script's top-of-file comment now carries the live re-run procedure
(rebuild MxTraceHarness x86, attach Frida with `--scenario=suspend-advised`
then `--scenario=activate-advised`, save under
`captures/NNN-frida-suspend-activate-instrumented/`, grep the new TSV for
`mx.suspend.*` / `mx.activate.*` and correlate with `nmx.enter` events
in the same time window). Live capture is intentionally deferred to the
maintainer per the F46 spec — this dev box has no AVEVA install.
`design/70-risks-and-open-questions.md` R5 status updated:
- Title flag `(filed as F45)` -> `(filed as F46, hook landed pending live re-run)`
(the docs/M6-buffered-evidence.md footnote referenced F45 from before
F45 / F46 were de-conflicted by commit
|
||
|
Joseph Doherty
|
c7e71e4424 |
design/followups: move F41 + F43 to Resolved (M6 complete)
All 10 M6 sub-followups (F35-F44 minus the ones absorbed into F44) plus F41 + F43 are now resolved. Open section narrows to: - F45: buffered recovery replay (sub-followup of F36) - F46: Suspend/Activate wire emission (sub-followup of F44) - F3: cross-domain NTLM fixture (permanently external-blocked) M6 closeout: see CHANGELOG.md for the V1 release notes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
7b15c853d1 |
[F43] release prep: CHANGELOG + cargo publish --dry-run validation
V1 release prep per M6 DoD bullet 6: **`CHANGELOG.md`** — V1 release notes covering all 9 workspace crates (`mxaccess-codec`, `mxaccess-rpc`, `mxaccess-asb-nettcp`, `mxaccess-asb`, `mxaccess-galaxy`, `mxaccess-callback`, `mxaccess-nmx`, `mxaccess`, `mxaccess-compat`), the M0 → M6 milestone closeouts, deliberate divergences from the .NET reference (multi-record DataUpdate codec relaxation per F44; buffered single- sample stream per R2), and known limitations (F3 cross-domain NTLM, F45 buffered recovery replay, F46 Suspend/Activate wire instrumentation, R3/R4 OperationComplete trigger). Documents the dependency-ordered publish sequence (leaf crates first; dependent crates require their deps to exist on crates.io before their dry-runs can run). **`cargo publish --dry-run` validation:** - Leaf crates (mxaccess-codec, mxaccess-rpc, mxaccess-asb-nettcp): dry-run passes — tarball builds, metadata complete, license/ description/repository/rust-version all present via `workspace.package`. - Dependent crates (mxaccess-asb, mxaccess-galaxy, mxaccess-callback, mxaccess-nmx, mxaccess, mxaccess-compat): dry-run fails with "no matching package" against crates.io — expected behaviour, the registry lookup happens even with `--no-verify`. Validation of these crates falls to the build-test-clippy-public_api matrix rather than dry-run. `design/followups.md`: F43 moved to Resolved with a verdict pointing at this commit + the CHANGELOG. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
f0c9dd2214 | rust: add version specifiers to workspace path deps for cargo publish | ||
|
Joseph Doherty
|
9e57bfd451 |
[F41 + F44 reconciliation] cargo public-api baselines + multi-record DataUpdate codec
**F41 — public-api baselines (M6 DoD bullet 5)**
`design/public-api/{crate}.txt` for all 9 workspace crates, generated
via `cargo +nightly public-api --simplified -p <crate>`. Per-crate
baseline sizes:
- mxaccess-codec: 2516 lines
- mxaccess-asb: 1258 lines
- mxaccess-rpc: 1273 lines
- mxaccess-asb-nettcp: 708 lines
- mxaccess: 542 lines
- mxaccess-galaxy: 374 lines
- mxaccess-callback: 170 lines
- mxaccess-compat: 123 lines
- mxaccess-nmx: 118 lines
`design/public-api/README.md` documents the update procedure
(install nightly + cargo-public-api, regenerate the affected baseline
on intentional API changes, commit alongside).
`.github/workflows/rust.yml` gains a `public-api` job that runs the
same diff against the committed baseline; drift fails CI with a
unified diff in the log so the PR author can either revert or
update the baseline.
**F44 reconciliation — multi-record DataUpdate codec**
Cherry-picked from the F44 sub-agent's worktree (commit `aec6a0c`):
`subscription_message.rs::parse_data_update` now loops over
`record_count` like `parse_subscription_status` does, accepting any
positive count. The .NET reference still hard-throws on
`record_count != 1`; the Rust codec deliberately diverges per the F44
evidence walk against `captures/094-frida-buffered-separate-writer/
frida-events.tsv:145` (a `0x33` DataUpdate body with `record_count = 2`,
inner_length = 23 (preamble) + 2 * 19 (records) = 61, post a
separate-session writer triggering two value changes inside one
`SetBufferedUpdateInterval(1000)` window).
Two new round-trip tests:
- `data_update_multi_record_round_trip` — synthesises a 2-record body,
parses, asserts both records decode to expected Int32 values.
- `data_update_capture_094_truncated_record_errors` — truncates the
capture-094 fixture mid-second-record, asserts CodecError::Decode.
New wire-byte fixtures under `crates/mxaccess-codec/tests/fixtures/m6-buffered/`:
- `094-line145-dataupdate-recordcount2.bin` (57 bytes, `0x33` multi-record)
- `094-line48-substatus-recordcount2.bin` (101 bytes, `0x32` multi-record)
R2 in `design/70-risks-and-open-questions.md` updated from
"single-sample (settled silently)" to "settled per option (a) — codec
relaxed; multi-record observed in production-stack tracing."
`design/followups.md`: F44's verdict updated to reflect the
contradiction-then-relaxation, with reference to the new tests +
fixtures.
Workspace 792 → 794 tests pass; clippy clean; rustdoc clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
2120dfa965 |
design/followups: move F35/F40/F44 to Resolved + de-conflict F45/F46
rust / build / test / clippy / fmt (push) Has been cancelled
After commits |
||
|
Joseph Doherty
|
ad1cf2351c |
[F36 + F40 + F44] M6 wave 1: subscribe_buffered (NMX) + metrics + evidence
Three M6 sub-followups landed in this wave (sub-agent worktrees +
manual reconciliation in main):
**F36 — Session::subscribe_buffered (NMX) per R2 single-sample**
- `BufferedOptions::rounded_update_interval_ms()` — 100ms rounding
helper mirroring MxNativeCompatibilityServer.cs:638
((updateInterval + 99) / 100) * 100, saturating on overflow.
- `Session::subscribe_buffered` (public, lib.rs:604) delegates to
the new private `subscribe_buffered_nmx` which uses the buffered
RegisterReference path: item_definition suffixed with
`.property(buffer)`, subscribe=true (no separate
AdviseSupervisory follow-up — verified against capture 082).
- Per R2 verified at wwtools/mxaccesscli/docs/api-notes.md the wire
semantic is single-sample-per-event with a server-side cadence
knob; rounded_ms is held client-side only (native MXAccess does
not emit a separate SetBufferedUpdateInterval RPC, verified by
absence in 079/082 captures).
- New crates/mxaccess/examples/subscribe-buffered.rs.
- New crates/mxaccess-codec/tests/buffered_register_reference_parity.rs:
4 tests (capture 079/082 round-trip, suffix helper, constructive
forward-build vs capture 082).
**F40 — Optional metrics feature**
- New crates/mxaccess/src/metrics.rs (275 lines): `pub(crate)`
thin wrappers (`record_write_latency`, `record_read_latency`,
`inc_writes`, `inc_reads`, `inc_advises`, `inc_recovery_*`,
`set_active_subscriptions`, etc.) that compile to no-ops under
`#[cfg(not(feature = "metrics"))]`. Call sites in session.rs +
asb_session.rs invoke them unconditionally; the gate is inside
the wrapper.
- `metrics = { version = "0.24", optional = true }` added to
workspace + mxaccess crate Cargo.toml.
- Default build: zero metrics dep, zero runtime cost.
**F44 — Buffered batch + suspend capture decode evidence**
- New docs/M6-buffered-evidence.md: per-capture summary for
077, 079, 080, 081, 082, 094 — call sequence, key wire bytes,
R2/R5 verdict.
- R2 confirmed silently as "not a real risk" — single-sample
observed across 079/080/082/094.
- R5 trigger conditions documented from capture 077: AdviseSupervisory
+ Suspend pair, 1-second intervals, succeeds on enum attributes.
- design/70-risks-and-open-questions.md R2/R5 status updated.
Workspace: 759 → 792 tests, clippy clean, rustdoc -D warnings clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
d5aa152b1f |
[F35] mxaccess-compat: LMXProxyServer-shaped facade (18 methods)
Replace the 8-line `mxaccess-compat` stub with a real `LmxClient` struct exposing the 18 `ILMXProxyServer5` methods as Rust async fns on top of `mxaccess::Session` (NMX) and `mxaccess::AsbSession` (ASB). Handle-table approach * `Mutex<HashMap<i32, ItemRef>>` for item handles, populated by `add_item` / `add_item_2` / `add_buffered_item`, drained by `remove_item` / `unregister`. * `Mutex<HashMap<i32, UserRef>>` for user handles allocated by `authenticate_user` / `archestra_user_to_id`. * `AtomicI32` monotonic counters for both, matching the .NET reference's `_nextItemHandle` / `_nextUserHandles` per `MxNativeCompatibilityServer.cs:62-63`. Stream-based event surface (per Q4) * `OnDataChange` / `OnBufferedDataChange` / `OnWriteComplete` / `OperationComplete` exposed as `EventStream<T>: Stream<Item=T>`, backed by `tokio::sync::broadcast` channels. Lag silently skips past `BroadcastStream::Lagged` to keep the public `Item` shape ergonomic. NOT COM events — that's the post-V1 `mxaccess-compat-com` crate per design/70-risks-and-open-questions.md Q4. The `OperationComplete` channel is wired but no firing path is modelled (R3 deferred — no captured byte mapping yet). * `Advise` / `AdviseSupervisory` spawn a background fan-out task that drains the `Subscription` stream and routes each `DataChange` to either `on_data_change` or `on_buffered_data_change` based on the item's `is_buffered` flag. `UnAdvise` / `RemoveItem` abort the task. Pass-through methods * `Write` / `Write2` -> `Session::write` / `write_with_timestamp` (`userId` ignored — the underlying surface uses engine identity). * `WriteSecured2` -> `Session::write_secured_at` with both user ids always passed (R6: single-user secured = same id twice; never gated). * `AdviseSupervisory` collapses onto `Session::subscribe` because the wire path is `AdviseSupervisory` already (`session.rs:1057`), matching the .NET reference's `cs:251-259` identical collapse. * `SetBufferedUpdateInterval` rounds up to nearest 100 ms per `MxNativeCompatibilityServer.cs:638`. Stubbed pass-throughs (mirror upstream `Error::Unsupported`) * `WriteSecured` (no timestamp) — `Session::write_secured` is stubbed at `crates/mxaccess/src/lib.rs:472` (only `WriteSecured2`/`0x3A` is ported); workaround documented inline. * `AddBufferedItem` allocates the handle but `Advise` for buffered items does not yet drive `Session::subscribe_buffered` cadence knob — TODO(F36) flagged inline at `add_buffered_item` and `set_buffered_update_interval`. Tests (25 new, all green) * Handle-table lifecycle: Add -> Advise -> UnAdvise -> Remove with a mocked subscription task. * Monotonic handle allocation; context-prefix combination. * `SetBufferedUpdateInterval` rounding (50 -> 100, 101 -> 200, etc.) + zero-rejection. * Compile-time check that all 18 LMX methods are reachable on `LmxClient`. * Each event stream yields published items; lag silently dropped. * GUID-shape validation; server-handle mismatch errors. Build hygiene * `cargo build -p mxaccess-compat` clean. * `cargo test -p mxaccess-compat` -> 25 passed. * `cargo clippy -p mxaccess-compat --all-targets -- -D warnings` clean. * `RUSTDOCFLAGS=-D warnings cargo doc -p mxaccess-compat --no-deps` clean. Deferred / TODOs * TODO(F36): wire `set_buffered_update_interval` cadence into the `advise` path for buffered items. * TODO(R3): plumb a real trigger into `on_operation_complete` once the byte mapping lands. * TODO(wave 2): live integration tests against AVEVA. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
a1c4c6203e |
design/followups: move F37/F38/F39/F42 to Resolved
rust / build / test / clippy / fmt (push) Has been cancelled
Four M6 sub-followups closed in this session — moved to Resolved section with concise verdicts referencing the matching commits: - F37 (commit |
||
|
Joseph Doherty
|
71c69b80c6 |
[F38] mxaccess-codec: counting-allocator bench harness + R12 baseline
Hand-rolled GlobalAlloc wrapper around System that tracks allocs +
bytes + deallocs via two atomics. Each scenario runs 10k iterations
after a 1k warm-up; output is a markdown table with allocs/op,
bytes/op, deallocs/op.
Why hand-rolled (not dhat/criterion): R12 gates on a single number
("< 5 allocs/write"). dhat is heap-profiling-oriented (call-stack
attribution, JSON snapshots); criterion measures wall-clock latency
which is reported-but-not-gated per 60-roadmap.md:104. A 50-line
GlobalAlloc + atomic counters is the simplest thing that answers
the gate.
Run: `cargo bench -p mxaccess-codec`
Baseline numbers (release, Windows x64):
- Bool write: 1.00 allocs/op
- Int32 write: 2.00 allocs/op
- Float32 write: 2.00 allocs/op
- Float64 write: 2.00 allocs/op
- String write: 4.00 allocs/op (5-char string)
- Handle from_names: 2.00 allocs/op
- DataUpdate decode: 1.00 alloc/op
R12's < 5 allocs/write target is **already met** across the proven
matrix without any zero-copy work. The bench gates on this — any
write_message::encode scenario at >= 5 allocs/op exits the harness
with code 1.
Companion: `design/M6-bench-baseline.md` documents the numbers,
explains the per-scenario breakdown, and tightens F39's scope from
"hit the target" to "nice-to-have optimisations" (BytesMut output
buffer, name-signature cache, session-level scratch pool).
Workspace: 759 tests still pass; clippy --benches clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
e79e289743 |
[F42] cargo doc --workspace --no-deps clean (0 warnings)
Fix all 33 rustdoc warnings across the workspace: - Unresolved intra-doc links: rewrite [`name`] → either backtick text (when not actually a link) or fully-qualified `[Type::method]` / `[crate::module::name]` form. Affected: mxaccess-codec (asb_variant, item_control, metadata_query, observed_write_template, reference_handle, write_message), mxaccess-rpc (pdu), mxaccess-nmx (client), mxaccess-asb-nettcp (nmf), mxaccess-callback (exporter), mxaccess (asb_session, session, lib). - Bracket-text being interpreted as link refs (e.g. `body[17]` → `` `body[17]` ``). - Private-item references in public docs (CALLBACK_BROADCAST_CAPACITY, recover_connection_core, mxvalue_to_writevalue) reduced to backtick-text since they aren't part of the public API. `RUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-deps` now exits clean. Workspace 759 tests pass; clippy clean. Defers `#![warn(missing_docs)]` lint to a future pass — the cleanup target is the broken-link warnings, which are signal; missing-docs would surface hundreds of low-priority public-item gaps that are out of scope for this F-number. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
34045c2f6d |
[F37] mxaccess: AsbSession::subscribe_buffered returns Unsupported
ASB has no `SetBufferedUpdateInterval` analogue — the per-monitored-
item `MinimalMonitoredItem::sample_interval` plays the cadence-knob
role. Calling `subscribe_buffered` on an ASB session now returns
`Error::Unsupported { transport: TransportKind::Asb, operation: ... }`
synchronously, without touching the wire.
The error-construction logic is split into a free fn
`unsupported_subscribe_buffered_error()` so the gate's exact shape
is unit-testable without spinning up a live authenticator + transport
fake. New unit test asserts both the variant tag and that the
operation message names the unsupported method + hints at the
`sample_interval` analogue.
Workspace 758 → 759 tests, clippy clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
2546710604 |
design/followups: add F35-F44 for M6 implementation plan
10 new followups decompose M6 (compatibility shim + production hardening) into parallel-safe sub-streams: - F35: mxaccess-compat LMXProxyServer-shaped facade (18 methods over Session/AsbSession) - F36: Session::subscribe_buffered NMX path per R2 single-sample - F37: ASB subscribe_buffered capability gate - F38: counting-allocator cargo bench harness for R12 target - F39: zero-copy codec pass (depends on F38) - F40: optional metrics feature - F41: cargo public-api baseline (depends on F35/F36/F37/F39/F40) - F42: cargo doc cleanup pass - F43: cargo publish --dry-run all crates (depends on F41) - F44: decode buffered batch + suspend captures (077, 079-082, 094) for R2/R5 evidence Parallelization: Wave 1 = F35/F36/F37/F38/F40/F42/F44 (different crates / different concerns); Wave 2 = F39 (needs F38's bench); Wave 3 = F41 (needs API stable); Wave 4 = F43 (release). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
bedad57b4e |
design/followups: move F18 (M5 meta-tracker) to Resolved
rust / build / test / clippy / fmt (push) Has been cancelled
Trim the planning content (sub-stream breakdown table, parallel-safety analysis, risk-driven sequencing, "Resolves when" gate) since M5 is done. Keep the closure verdict, M5 DoD checklist showing the actual state at close, sub-followup closeout list (F19-F26 + F28/F29/F30/ F31/F32/F33/F34), cumulative execution log, and the architectural note explaining why AsbSession stays parallel to the NMX Session rather than unified — that's load-bearing for future maintenance. Open section now contains only F3 (cross-domain NTLM Type1/2/3 fixture, permanently external-blocked on this single-domain dev host — resolution requires multi-domain Windows lab not available here). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
b1a5f5ff1e |
design/followups: move F34 to Resolved (live-verified closure)
The F34 entry's body had grown into a debugging notebook with five
"Open hypotheses" and a "Resolves when" speculation block — all of
which are now moot since the actual fix landed. Trim to the closure
verdict, the technical evidence (captured fixture dictionary, the
dual-format insight), and the bonus context discovered while
debugging (Active/SampleInterval/result_code 32 quirks). Move from
"## Open" to "## Resolved" with date + commit
|
||
|
Joseph Doherty
|
101a8b13f5 |
[F34] mxaccess-asb: AddMonitoredItems body uses DataContract field names
Rewrite push_monitored_item_body to emit the DataContract field-suffix names from AsbContracts.cs:940-965 (activeField, bufferedField, itemField, sampleIntervalField, timeDeadbandField, userDataField, valueDeadbandField) under prefix `b` bound to the http://schemas.datacontract.org/2004/07/ArchestrAServices.ASBIDataV2Contract namespace. The <Items> wrapper now declares xmlns:b + xmlns:i. The legacy XmlSerializer property names (<Active>, <Item>, <SampleInterval>, <Buffered>) only matter for the canonical-XML HMAC signing input — that emitter at xml_canonical::emit_monitored_item is unchanged and F28 fixture byte-equality still holds for all 13 ops. On the binary NBFX wire MxDataProvider's DataContractSerializer expects the field-suffix form. Wire-byte type encoding matches the captured fixture (add-monitored-items-request-wire.bin): bool → Bool record, ulong → Zero/One/Chars (XmlConvert decimal text), ushort → Zero/One/Int8/Int16/Int32 (smallest-fit binary). Empty string? + null byte[]? emit as empty elements with no <i:nil> attribute (matching the wire). Field order follows the explicit [DataMember(Order = N)] sequence. Adjacent: ItemIdentity is nested via DataContract field names too — NOT the binary <ASBIData> fast-path, which only kicks in at top-level message body members. Verified live against AVEVA MxDataProvider: AddMonitoredItems now returns 1 status item with error_code=0x0000 (previously 0 items; the silent failure was the deliberate DC-schema mismatch); Publish poll #4 delivers the actual tag value as AsbVariant { type_id: 4, length: 4, payload: [99,0,0,0] } through the F26 stream. Pre-existing clippy::format_collect errors in auth.rs:339,342 and client.rs:952 fixed in passing — they were blocking workspace clippy otherwise. Workspace: 757 → 758 tests, clippy -D warnings clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
6762526f09 |
design/followups: mark F18 (M5 meta-tracker) resolved
All sub-followups F19-F26 closed; M5 is functionally LIVE end-to-end (asb-subscribe returns the real tag value over the wire). The structural-port followups F18 spawned (F2/F10/F11/F27 for NTLM / DCOM / RemUnknown / DH) all resolved separately. F18 stays under "## Open" as the cumulative-execution-log anchor; status line at the top now reflects the closed state so the open/resolved structure matches reality. Remaining open items: F34 (MonitoredItem wire format, P2 — needs nbfx auto-intern fix + DataContract field-suffix body builders) and F3 (cross-domain NTLM fixture, P2 — permanently external-blocked on this single-domain dev host). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
1de049e114 |
[F2] mxaccess-rpc: NTLM verify_signature (server-to-client) with constant-time MAC compare
rust / build / test / clippy / fmt (push) Has been cancelled
Closes F2. Structural port from [MS-NLMP] §3.4.4 — same shape as
the existing sign path but uses the server-to-client sub-keys
(`SealKey_S→C` / `SignKey_S→C`) derived alongside the client-to-
server pair at the end of create_type3.
NtlmClientContext gained four new fields populated during
create_type3:
- server_signing_key
- server_sealing_key
- server_sealing_state (independent RC4 stream)
- server_sequence (independent counter)
The S→C key derivation already existed in auth.rs (the seal_key /
sign_key helpers take a client_mode flag); F2 plumbs them into a
new verify_signature(message, signature) method.
The verify path:
1. Validates signature.len() == 16 + leading version word 0x01.
2. Reads trailing seq num, compares against self.server_sequence
(mismatch ⇒ InvalidSignature, no state change).
3. Computes expected_mac = HMAC_MD5(server_signing_key,
seq || message)[0..8] then RC4 transform.
4. Constant-time compares expected_mac against wire bytes 4..12
via subtle::ConstantTimeEq.
5. On success: commits cipher-state advance + ++server_sequence.
On failure: re-derives RC4 from server_sealing_key and skips
past server_sequence × 8 keystream bytes to restore the
pre-verify position — caller can retry.
New dep `subtle = "2"` (workspace-internal to mxaccess-rpc) for
the timing-oracle-safe MAC compare.
6 new tests:
- verify_signature_round_trip_against_sign (3-message sequence
via paired_authed_context helper that aliases server-side keys
onto client-side for self-validating round-trip)
- verify_signature_rejects_corrupted_mac (with
server_sequence-non-advance assertion)
- verify_signature_rejects_wrong_sequence_number
- verify_signature_rejects_wrong_version_field
- verify_signature_rejects_wrong_length
- verify_signature_before_authenticate_errors
mxaccess-rpc 188 → 194 tests; default-feature clippy clean.
The "awaiting wire-fixture capture" step listed in F2's prior
status note is no longer a hard prerequisite — [MS-NLMP] §3.4.4
fully defines the algorithm and the round-trip tests prove the
encoder/decoder pair is internally consistent. A captured
StatusReceived frame would still validate byte-parity vs a real
NmxSvc.exe signer, but that's future verification work; the
structural port ships unblocked.
design/followups.md F2 moved to Resolved.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
161b0cedfa |
[F10 + F11] mxaccess-rpc: structural ports for ResolveOxid2 + RemAddRef/RemRelease
rust / build / test / clippy / fmt (push) Has been cancelled
Closes F10 and F11 per option (b) of each followup's resolve
criterion: hand-rolled body codecs derived from the [MS-DCOM]
spec, ship structurally with no live fixture (the .NET reference
doesn't call these opnums), validate against captured frames when
they become available.
F10 — IObjectExporter::ResolveOxid2 (opnum 4):
Per [MS-DCOM] §3.1.2.5.1.4. New parse_resolve_oxid2_result
mirrors parse_resolve_oxid_result exactly except for the extra
4-byte COMVERSION slot (u16 major + u16 minor) between
authn_hint and error_status. Trailing-fields check tightens
from 24 bytes (opnum 0) to 28 bytes (opnum 4). New ComVersion +
ResolveOxid2Result types. referent_id=0 short-circuit returns
empty bindings + default ComVersion + tail-status, matching
opnum 0's pattern.
F11 — IRemUnknown::RemAddRef + RemRelease (opnums 4 and 5):
Per [MS-DCOM] §3.1.1.5.6 + §2.2.19 (REMINTERFACEREF). Both
opnums share the wire shape, so:
- encode_rem_add_ref_request + encode_rem_release_request
both delegate to a shared encode_remref_array_request
helper.
- parse_remref_response is shared between them too — they
have an identical OrpcThat + referent_id + max_count +
N×HRESULT + error_code layout.
New RemInterfaceRef struct (ipid + public_refs + private_refs,
ENCODED_LEN = 24) + RemRefResponse decoded shape.
8 new structural tests across both modules pin every documented
edge of each shape — short stubs, referent-zero short-circuits,
truncated-trailing detection, full multi-element round-trips.
mxaccess-rpc 183 → 188 tests; default-feature clippy clean.
Both followups documented "**Status:** Awaiting wire-fixture
capture" prior to this commit; the structural-port option was
explicitly listed as resolution path (b) in each. Future captured
frames will validate the byte-by-byte match — until then the
port is byte-faithful to the spec but unverified against a live
peer (which is fine for shipping since neither opnum is on the
NMX session's hot path).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
4ed1355761 |
design/followups: rewrite F2/F3/F10/F11 with concrete next-step recipes
Each remaining open followup now lists the precise "Concrete next step" to close it — what to capture, where to write the fixture, which file to edit. Future sessions (or anyone without the project context) can pick up any of these and execute without guessing. F2 (NTLM verify_signature server→client): Status: awaiting wire-fixture capture. M2 wave 3 (callback exporter) is closed under F15, so the path is wired — instrument CallbackExporter to hex-dump inbound StatusReceived bytes during a live subscribe, save under tests/fixtures/m2-status-frame/, port verify_signature mirroring `sign` but using the server-to-client sub-keys per [MS-NLMP] §3.4.4, add `subtle = "2"` for constant-time MAC compare. F3 (cross-domain NTLM Type1/2/3 fixture): Status: permanently out-of-scope on this host (no second AD domain). Documented the lab-environment requirements and the capture procedure for whoever provisions the two-domain harness. F10 (IObjectExporter::ResolveOxid2 opnum 4): Status: awaiting capture or .NET helper. Two paths documented — extend MxNativeClient.Probe with --probe-resolve-oxid2 OR hand-roll the layout from [MS-DCOM] §3.1.2.5.1.4 and validate later. F11 (IRemUnknown::RemAddRef + RemRelease): Status: same shape as F10. Document either probe extension or structural port from [MS-DCOM] §3.1.1.5.6 (REMINTERFACEREF[]). No code changes in this commit — purely sharpening the followup specs so each one's resolution recipe is self-contained. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
9496322712 |
[F27] mxaccess-asb-nettcp: constant-time DH mod_exp via crypto-bigint::DynResidue
rust / build / test / clippy / fmt (push) Has been cancelled
Closes F27 per option (b) of its resolve criterion: fixed-width
U2048 DH backend using crypto-bigint's Montgomery-form residue
arithmetic.
New auth.rs::constant_time_mod_exp(base, exp, modulus) wrapper
preserves the BigUint-in-BigUint-out API of the existing byte
helpers; the actual square-and-multiply chain runs in Montgomery
form. Both DH call sites swap to the wrapper:
- AsbAuthenticator::new line 179 (public-key generation)
- crypto_key line 354 (shared-secret derivation)
DH private exponent timing-leak resistance is the goal: the .NET
reference's BigInteger.ModPow is also non-constant-time, so we
were at parity but not at the long-term Rust target. With this
fix the production path no longer leaks the bit-pattern of the
long-lived DH private key through power/timing side channels.
DynResidueParams::new requires an odd modulus (Montgomery form's
only restriction). Production DH primes are always odd
(`MX_ASB_DH_PRIME = 1552...7919` on this host's registry).
CryptoParameters::DEFAULT_PRIME_TEXT — the test-fixture default
inherited from AsbRegistry.cs:66 — ends in 4 (even), which is
mathematically unsound for DH but kept for parity with the .NET
default. For that case the wrapper falls back to BigUint::modpow,
preserving the wire bytes (modular exp is a pure function of
inputs).
Wire-byte parity verified two ways:
1. Unit fixture test
`auth.rs::deterministic_hmac_matches_dotnet_fixture` — byte-equal
to captured .NET output for the full DH → PBKDF2 → AES-CBC chain.
Continues to pass.
2. Live: Connect handshake against the local AVEVA install
completes with apollo:V2 lifetime, proving MxDataProvider
accepts the constant-time-derived public key and the
shared-secret-based AuthenticateMe.
Workspace deps:
- crypto-bigint = "0.5" added to [workspace.dependencies] and
mxaccess-asb-nettcp/Cargo.toml.
- num-bigint retained for decimal-string parsing + .NET-LE byte
conversion (crypto-bigint has neither).
Closes the "review.md MAJOR finding" originally flagged at
design/30-crate-topology.md:269-274.
design/followups.md: F27 moved to Resolved.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
d03bd04ef5 |
[F34 evidence] dump WCF binary-header dictionary for AddMonitoredItems
rust / build / test / clippy / fmt (push) Has been cancelled
Extends tests/add_monitored_items_request_capture.rs with a manual binary-header walk that prints every pre-interned string + its wire id. The captured request's binary header pre-declares **23 strings** covering the entire DataContract field set: wire-id 1 http://ASB.IDataV2:addMonitoredItemsIn wire-id 3 AddMonitoredItemsRequest wire-id 5 SubscriptionId wire-id 7 Items wire-id 9 http://schemas.datacontract.org/.../ASBIDataV2Contract wire-id 11 MonitoredItem wire-id 13 activeField wire-id 15 activeFieldSpecified wire-id 17 bufferedField wire-id 19 itemField wire-id 21 contextNameField wire-id 23 idField wire-id 25 idFieldSpecified wire-id 27 nameField wire-id 29 referenceTypeField wire-id 31 typeField wire-id 33 sampleIntervalField wire-id 35 timeDeadbandField wire-id 37 timeDeadbandFieldSpecified wire-id 39 userDataField wire-id 41 lengthField wire-id 43 payloadField wire-id 45 valueDeadbandField That gives F34's binary-builder rewrite the exact dict-id mapping to target — every MonitoredItem child can be emitted as a DictionaryStatic(odd-id) reference instead of an inline string, matching WCF's compression. The "RequireId" mystery from the earlier inline-name decode is also resolved: the wire body has NO `RequireId` element at the bottom — the trailing `Inline("referenceTypeField")` was a dict-id wraparound or auto-intern artifact, not actual content. design/followups.md F34 updated with the full ground-truth header, plus a refined "Resolves when" pointing at the underlying `nbfx.rs::decode_tokens` auto-intern semantics. The current codec's doc comment ("the codec doesn't auto-intern") is correct for raw [MC-NBFX] but wrong for WCF binary messages where the writer auto-interns by convention; that's the structural fix the F34 binary rewrite depends on. No code-path change in this commit beyond the test improvements. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |