dohertj2/mxaccess
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
124 Commits 1 Branch 0 Tags
c6332c26a1a62ae87d4ce8c23ba8c78a7e4ceeda
Commit Graph

9 Commits

Author SHA1 Message Date
Joseph Doherty af15fe7587 [F49 step 1 + F56] callback router: peel envelope before parsing subscription / 0x11 frames 
The router used to call NmxSubscriptionMessage::parse_inner directly
on the COM-stub-delivered body, but the wire bytes arrive wrapped in
a ProcessDataReceived envelope (46-byte header + optional 4-byte
length prefix); parse_inner expects post-envelope bytes. Result:
every 0x33 DataUpdate that ever arrived was silently dropped.

Mirrors the .NET reference's MxNativeSession.OnCallbackReceived flow
at cs:582-606 — three sequential parse attempts:
  1. NmxOperationStatusMessage::try_parse_process_data_received_body  (already wired)
  2. NmxReferenceRegistrationResultMessage::try_parse_...              (NEW — was missing)
  3. NmxSubscriptionMessage::try_parse_process_data_received_body      (NEW — was wrong)

Adds:
- NmxSubscriptionMessage::try_parse_process_data_received_body — peels
  envelope via NmxObservedEnvelope::parse_process_data_received_body_flexible,
  then dispatches to existing parse_inner.
- NmxReferenceRegistrationResultMessage::try_parse_process_data_received_body —
  same shape, for the 0x11 registration-result frame.
- Router branch for 0x11 — currently traces the assigned item_handle and
  drops the frame (matches the .NET reference, which fires a
  ReferenceRegistrationReceived event with no consumer in the codebase).
- Router fall-through trace! when neither path matches, so future
  unparseable bodies surface in RUST_LOG=trace instead of vanishing.
- DcomCallbackSink::forward — trace! per inbound callback so
  RUST_LOG=mxaccess_callback=trace surfaces opnum + size.
- crates/mxaccess-compat/tests/buffered_subscribe_live.rs — F49 step 1
  live test that drives subscribe_buffered + a 500ms-cadence writer.
  Also pulls tracing-subscriber as a dev-dep so the test can dump
  router activity.

Existing router_task_decodes_callback_invoked_into_broadcast unit test
updated to wrap its synthetic 0x32 body in an envelope so the new
parse path actually accepts it.

Live result: F56 — the buffered round-trip *registers* successfully
(RegisterReference returns HRESULT 0; engine sends one 0x11
RegistrationResult + one 51-byte op-status per write, perfectly
clocked) but the engine never sends a 0x33 DataUpdate. Rust-port-
specific gap vs the .NET reference's working buffered path; root
cause is likely a field-level difference in the RegisterReference
body or a missing post-RegisterReference step. Captured as F56 in
design/followups.md, blocking F49 step 1; F56's DoD is the same
live test reporting >=3 DataChange arrivals.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-06 09:50:57 -04:00
Joseph Doherty c73a33edd8 [R3/R4 Path A] mxaccess: port Lmx.dll FUN_10100ce0 synthesizer kernel
rust / build / test / clippy / fmt (push) Has been cancelled
Details
rust / cargo public-api drift check (F41) (push) Has been cancelled
Details 
Path A landed for R3/R4. The byte->MxStatus synthesizer in Lmx.dll is
FUN_10100ce0 (`analysis/ghidra/exports/Lmx.dll.synthesizer-helpers2-decompile.md`),
a 4-byte u32 LE -> 4-tuple MxStatus decoder used by every NMX-frame
parser in Lmx.dll. The kernel is byte-deterministic and context-free,
so it ports as a pure function -- the operation-tracking state
machine the original verdict deferred is NOT required for synthesis.

Bit layout (per FUN_10100ce0 lines 21-24):
  bit 31:        success    (-1 if set, 0 if clear)
  bits 27..24:   category   (4 bits)
  bits 23..20:   detected_by (4 bits)
  bits 15..0:    detail     (i16 -- low 16 bits, signed)
  bits 30..28, 19..16: reserved/padding

Codec changes:
- MxStatus::from_packed_u32() / ::to_packed_u32() -- the kernel +
  inverse for round-trip parity.
- MxStatus::from_nmx_response_code() -- the constructed-from-response-
  code switch in FUN_1010bd10:741-770 (six proven mappings: 0x01, 0x02
  -> CommunicationError + RequestingNmx; 0x03 -> ConfigurationError +
  RequestingNmx; 0x04 -> ConfigurationError + RespondingNmx; 0x05 ->
  CommunicationError + RespondingNmx; 0x1A -> CommunicationError +
  RequestingNmx).
- MxStatusCategory / MxStatusSource: from_i16/to_i16 promoted to const
  fn so MxStatus::from_packed_u32 can be const.
- NmxOperationStatusMessage::try_parse_process_data_received_body() --
  thin wrapper that peels the outer NmxObservedEnvelope before
  delegating to try_parse_inner. Mirrors
  NmxOperationStatusMessage.TryParseProcessDataReceivedBody (.NET cs:20-32).
- NmxOperationStatusMessage::promote_to_typed() -- entry point that
  returns the existing Status field. Documented as a no-op pass-through
  for now (the 5-byte inner-body wire shape is NOT the same field as
  the 4-byte packed-u32 the kernel decodes); kept for API symmetry.
- 22 new round-trip tests covering the kernel, the response-code
  switch, the proven 0x00/0x41/0xEF completion bytes, and round-trip
  for every canonical sentinel.

mxaccess (Session) changes:
- New OperationKind enum (Write/WriteSecured/Read/Subscribe/
  Unsubscribe/Activate/Suspend/Other).
- New OperationContext struct (correlation_id, op_kind, reference,
  retry_count) -- ground for the F54 follow-on per-operation
  correlation work.
- New OperationStatus event type {raw, status, context,
  is_during_recovery}, mirroring MxNativeOperationStatusEvent (cs:73-78)
  with the typed-MxStatus addition.
- Session::operation_status_events() -> broadcast::Receiver<Arc<
  OperationStatus>> + operation_status_stream() Stream variant.
- callback_router() now tries operation-status parsing first, falling
  through to subscription messages -- matches MxNativeSession
  .OnCallbackReceived dispatch order (cs:574,582,590).
- recover_connection() flips a recovery_active counter (Arc<AtomicU32>
  shared with the router) so OperationStatus.is_during_recovery is
  populated correctly. Mirrors MxNativeSession._recoveryActive
  Volatile.Read at cs:573.
- 3 new router tests covering: status-word frame dispatch + typed
  promotion to WriteCompleteOk; completion-only frames stay verbatim;
  is_during_recovery is stamped from the live counter.

Per-operation context tracking (correlating completion frames back to
outstanding writes/subscribes via the correlation_id) is filed as F54
in design/followups.md. The synthesizer kernel itself is byte-
deterministic, so the kernel and the correlation work are decoupled.

Ghidra evidence (the next-ring xref walk beyond FUN_10114a90):
- analysis/ghidra/exports/Lmx.dll.set-attribute-result-xrefs.md --
  xrefs to OnSetAttributeResult / CancelWithStatus / OperationComplete.
- analysis/ghidra/exports/Lmx.dll.vtable-data-xrefs.md -- vtable-slot
  data xrefs for the virtual-dispatch path.
- analysis/ghidra/exports/Lmx.dll.synthesizer-decompile.md --
  ScanOnDemandCallback::OperationComplete/MultipleOperationComplete
  (FUN_1010b990), RemotePlatformResolver::OperationComplete
  (FUN_1010dc80), and the constructed-from-responseCode synthesizer
  in FUN_1010bd10 (lines 698-770). FUN_1010bd10 is the wire-frame
  receiver that drives the synthesis.
- analysis/ghidra/exports/Lmx.dll.synthesizer-helpers-decompile.md --
  FUN_10003fc0 (the <success %d category %d ...> formatter; confirms
  the 4-tuple layout), FUN_1008f150 (dispatch helper).
- analysis/ghidra/exports/Lmx.dll.synthesizer-helpers2-decompile.md --
  FUN_10100ce0 (the kernel itself), FUN_10100bc0 (3xu16 reader),
  FUN_1005e580 (4-byte stream reader), FUN_1010ee00 (sister NMX-frame
  parser using the same kernel).
- analysis/ghidra/exports/Lmx.dll.synthesizer-callers-xrefs.md --
  caller graph; confirms the kernel is called from many wire-frame
  parsers but each parser shares the single 4-byte decoder.

R3/R4 verdict updated in design/70-risks-and-open-questions.md from
"settled at verbatim-preserve" to "settled per Path A". F54 filed in
design/followups.md for the per-operation correlation work.

cargo build / test / clippy -D warnings / RUSTDOCFLAGS=-D warnings doc
all clean. cargo public-api baselines regenerated for mxaccess and
mxaccess-codec.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-06 07:08:36 -04:00
Joseph Doherty 9e57bfd451 [F41 + F44 reconciliation] cargo public-api baselines + multi-record DataUpdate codec 
**F41 — public-api baselines (M6 DoD bullet 5)**

`design/public-api/{crate}.txt` for all 9 workspace crates, generated
via `cargo +nightly public-api --simplified -p <crate>`. Per-crate
baseline sizes:
- mxaccess-codec: 2516 lines
- mxaccess-asb:   1258 lines
- mxaccess-rpc:   1273 lines
- mxaccess-asb-nettcp: 708 lines
- mxaccess: 542 lines
- mxaccess-galaxy: 374 lines
- mxaccess-callback: 170 lines
- mxaccess-compat: 123 lines
- mxaccess-nmx: 118 lines

`design/public-api/README.md` documents the update procedure
(install nightly + cargo-public-api, regenerate the affected baseline
on intentional API changes, commit alongside).

`.github/workflows/rust.yml` gains a `public-api` job that runs the
same diff against the committed baseline; drift fails CI with a
unified diff in the log so the PR author can either revert or
update the baseline.

**F44 reconciliation — multi-record DataUpdate codec**

Cherry-picked from the F44 sub-agent's worktree (commit `aec6a0c`):
`subscription_message.rs::parse_data_update` now loops over
`record_count` like `parse_subscription_status` does, accepting any
positive count. The .NET reference still hard-throws on
`record_count != 1`; the Rust codec deliberately diverges per the F44
evidence walk against `captures/094-frida-buffered-separate-writer/
frida-events.tsv:145` (a `0x33` DataUpdate body with `record_count = 2`,
inner_length = 23 (preamble) + 2 * 19 (records) = 61, post a
separate-session writer triggering two value changes inside one
`SetBufferedUpdateInterval(1000)` window).

Two new round-trip tests:
- `data_update_multi_record_round_trip` — synthesises a 2-record body,
  parses, asserts both records decode to expected Int32 values.
- `data_update_capture_094_truncated_record_errors` — truncates the
  capture-094 fixture mid-second-record, asserts CodecError::Decode.

New wire-byte fixtures under `crates/mxaccess-codec/tests/fixtures/m6-buffered/`:
- `094-line145-dataupdate-recordcount2.bin` (57 bytes, `0x33` multi-record)
- `094-line48-substatus-recordcount2.bin` (101 bytes, `0x32` multi-record)

R2 in `design/70-risks-and-open-questions.md` updated from
"single-sample (settled silently)" to "settled per option (a) — codec
relaxed; multi-record observed in production-stack tracing."

`design/followups.md`: F44's verdict updated to reflect the
contradiction-then-relaxation, with reference to the new tests +
fixtures.

Workspace 792 → 794 tests pass; clippy clean; rustdoc clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-06 05:27:11 -04:00
Joseph Doherty ad1cf2351c [F36 + F40 + F44] M6 wave 1: subscribe_buffered (NMX) + metrics + evidence 
Three M6 sub-followups landed in this wave (sub-agent worktrees +
manual reconciliation in main):

**F36 — Session::subscribe_buffered (NMX) per R2 single-sample**
- `BufferedOptions::rounded_update_interval_ms()` — 100ms rounding
  helper mirroring MxNativeCompatibilityServer.cs:638
  ((updateInterval + 99) / 100) * 100, saturating on overflow.
- `Session::subscribe_buffered` (public, lib.rs:604) delegates to
  the new private `subscribe_buffered_nmx` which uses the buffered
  RegisterReference path: item_definition suffixed with
  `.property(buffer)`, subscribe=true (no separate
  AdviseSupervisory follow-up — verified against capture 082).
- Per R2 verified at wwtools/mxaccesscli/docs/api-notes.md the wire
  semantic is single-sample-per-event with a server-side cadence
  knob; rounded_ms is held client-side only (native MXAccess does
  not emit a separate SetBufferedUpdateInterval RPC, verified by
  absence in 079/082 captures).
- New crates/mxaccess/examples/subscribe-buffered.rs.
- New crates/mxaccess-codec/tests/buffered_register_reference_parity.rs:
  4 tests (capture 079/082 round-trip, suffix helper, constructive
  forward-build vs capture 082).

**F40 — Optional metrics feature**
- New crates/mxaccess/src/metrics.rs (275 lines): `pub(crate)`
  thin wrappers (`record_write_latency`, `record_read_latency`,
  `inc_writes`, `inc_reads`, `inc_advises`, `inc_recovery_*`,
  `set_active_subscriptions`, etc.) that compile to no-ops under
  `#[cfg(not(feature = "metrics"))]`. Call sites in session.rs +
  asb_session.rs invoke them unconditionally; the gate is inside
  the wrapper.
- `metrics = { version = "0.24", optional = true }` added to
  workspace + mxaccess crate Cargo.toml.
- Default build: zero metrics dep, zero runtime cost.

**F44 — Buffered batch + suspend capture decode evidence**
- New docs/M6-buffered-evidence.md: per-capture summary for
  077, 079, 080, 081, 082, 094 — call sequence, key wire bytes,
  R2/R5 verdict.
- R2 confirmed silently as "not a real risk" — single-sample
  observed across 079/080/082/094.
- R5 trigger conditions documented from capture 077: AdviseSupervisory
  + Suspend pair, 1-second intervals, succeeds on enum attributes.
- design/70-risks-and-open-questions.md R2/R5 status updated.

Workspace: 759 → 792 tests, clippy clean, rustdoc -D warnings clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-06 05:12:17 -04:00
Joseph Doherty 71c69b80c6 [F38] mxaccess-codec: counting-allocator bench harness + R12 baseline 
Hand-rolled GlobalAlloc wrapper around System that tracks allocs +
bytes + deallocs via two atomics. Each scenario runs 10k iterations
after a 1k warm-up; output is a markdown table with allocs/op,
bytes/op, deallocs/op.

Why hand-rolled (not dhat/criterion): R12 gates on a single number
("< 5 allocs/write"). dhat is heap-profiling-oriented (call-stack
attribution, JSON snapshots); criterion measures wall-clock latency
which is reported-but-not-gated per 60-roadmap.md:104. A 50-line
GlobalAlloc + atomic counters is the simplest thing that answers
the gate.

Run: `cargo bench -p mxaccess-codec`

Baseline numbers (release, Windows x64):
- Bool write:    1.00 allocs/op
- Int32 write:   2.00 allocs/op
- Float32 write: 2.00 allocs/op
- Float64 write: 2.00 allocs/op
- String write:  4.00 allocs/op (5-char string)
- Handle from_names: 2.00 allocs/op
- DataUpdate decode: 1.00 alloc/op

R12's < 5 allocs/write target is **already met** across the proven
matrix without any zero-copy work. The bench gates on this — any
write_message::encode scenario at >= 5 allocs/op exits the harness
with code 1.

Companion: `design/M6-bench-baseline.md` documents the numbers,
explains the per-scenario breakdown, and tightens F39's scope from
"hit the target" to "nice-to-have optimisations" (BytesMut output
buffer, name-signature cache, session-level scratch pool).

Workspace: 759 tests still pass; clippy --benches clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-06 04:45:33 -04:00
Joseph Doherty e79e289743 [F42] cargo doc --workspace --no-deps clean (0 warnings) 
Fix all 33 rustdoc warnings across the workspace:

- Unresolved intra-doc links: rewrite [`name`] → either backtick text
  (when not actually a link) or fully-qualified `[Type::method]` /
  `[crate::module::name]` form. Affected: mxaccess-codec
  (asb_variant, item_control, metadata_query, observed_write_template,
  reference_handle, write_message), mxaccess-rpc (pdu), mxaccess-nmx
  (client), mxaccess-asb-nettcp (nmf), mxaccess-callback (exporter),
  mxaccess (asb_session, session, lib).
- Bracket-text being interpreted as link refs (e.g. `body[17]` →
  `` `body[17]` ``).
- Private-item references in public docs (CALLBACK_BROADCAST_CAPACITY,
  recover_connection_core, mxvalue_to_writevalue) reduced to
  backtick-text since they aren't part of the public API.

`RUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-deps` now
exits clean. Workspace 759 tests pass; clippy clean.

Defers `#![warn(missing_docs)]` lint to a future pass — the cleanup
target is the broken-link warnings, which are signal; missing-docs
would surface hundreds of low-priority public-item gaps that are out
of scope for this F-number.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-06 04:39:51 -04:00
Joseph Doherty 7611d9e215 [M5] mxaccess-codec: F24 ASB Variant + AsbStatus + RuntimeValue codec 
Ports `Variant` (cs:1170-1241), `AsbStatus` (cs:1109-1167), `RuntimeValue`
(cs:741-791), `AsbVariantFactory.From*` (cs:1310-1429), and
`MxAsbDataClient.DecodeVariant` (cs:713-825) into `mxaccess-codec::asb_variant`.

Three layers per `docs/ASB-Variant-Wire-Format.md`:
1. `AsbVariant` — raw 2/4/4/payload header + bytes; round-trips byte-identical.
2. `DecodedVariant` — typed view with one variant per proven ASB scalar / array
   (`Bool`, `Int32`, `Float`, `Double`, `String`, `DateTime`, `Duration` plus
   array forms). Type ids outside the proven matrix surface as
   `Unsupported { type_id, payload }` — same fallback as .NET's `_ => payload`.
3. `from_*` factories — mirror `AsbVariantFactory.FromX` exactly, setting
   `length` to `payload.len()` per `cs:1431-1438`.

`AsbStatus` and `RuntimeValue` round-trip the wire layout verbatim.
Status-element walking (marker bit 7 = implicit zero, etc., per
`docs/ASB-Variant-Wire-Format.md:180-205`) is deferred to a follow-up; the
codec exposes the raw status payload bytes for now, matching .NET's
`AsbStatus.Payload = byte[]` shape.

The lib.rs `AsbVariant` / `AsbStatus` / `RuntimeValue` stubs are replaced by
the real types via `pub use`. 25 new unit tests cover the proven matrix:
scalar + array round-trip, byte layout (2/4/4/payload), `Unsupported`
fallback for declared-but-unproven types, short-frame rejection,
malformed `string[]` partial-decode preservation matching .NET behavior.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 10:47:11 -04:00
Joseph Doherty 68aa2e30ab [M3] codec+galaxy: MxValueKind::for_data_type + GalaxyTagMetadata::resolve_write_kind 
Last codec-side prerequisite before F13 (NmxClient high-level write
wrappers) can land. Two small additions, both wire-byte-direct ports
of the .NET reference's MxDataType → MxValueKind lookup logic.

mxaccess-codec
- MxValueKind::for_data_type(MxDataType, is_array) -> Option<MxValueKind>:
  fuses NmxWriteMessage.cs:58-86 (TryGetValueKind's 12 base mappings
  for data types 1..=6 scalar+array) with the two scalar fallbacks the
  .NET GalaxyTagMetadata.ProjectWriteValue layers on top
  (GalaxyRepositoryTagResolver.cs:65-69): ElapsedTime → Int32,
  InternationalizedString → String. Returns None for any other
  combination — including arrays of those two types and unsupported
  scalars (ReferenceType, StatusType, Enum, etc.).
- 6 new tests covering the base table, both fallbacks, the array-of-
  unsupported rejection, and the no-mapping branch for ReferenceType /
  StatusType / Enum / DataQualityType / BigString / Unknown / NoData /
  End sentinels.

mxaccess-galaxy
- GalaxyTagMetadata::resolve_write_kind() -> Result<MxValueKind,
  UnsupportedDataType>: pure delegation to MxValueKind::for_data_type
  + a typed error carrying (mx_data_type, is_array) for diagnostics.
- GalaxyTagMetadata::is_writable() — Ok-side accessor for browse UIs.
- UnsupportedDataType public error type (re-exported from lib.rs).
- 7 new tests: Double scalar → Float64, Boolean array → BoolArray,
  ElapsedTime scalar → Int32 (the fallback path), array-of-ElapsedTime
  rejected, InternationalizedString → String, ReferenceType rejected,
  Unknown sentinel rejected.

Test count delta: 446 -> 459 (+13; codec 215 -> 221, galaxy 49 -> 56).
All four DoD gates green.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 08:33:42 -04:00
Joseph Doherty fe2a6db786 Initial project state: .NET reference, design, Rust port (M0+M1), evidence
rust / build / test / clippy / fmt (push) Has been cancelled
Details 
Layout:
- src/                    .NET 10 x64 reference: MxNativeCodec, MxNativeClient,
                          MxAsbClient, probes, tests, harnesses. Executable spec.
- design/                 Architectural plan for the Rust port (M0–M6), error
                          model, protocol invariants, risks (R1–R16), adversarial
                          review log (review.md).
- rust/                   Rust workspace. M0 skeleton + M1 codec parity.
                          mxaccess-codec: 215 unit tests + 2 cross-implementation
                          parity tests (byte-identical against .NET reference).
                          Other crates are M0 stubs awaiting M2+.
- captures/               Frida + netsh + pcap evidence per CLAUDE.md
                          ("captures are evidence, not throwaway logs").
- analysis/               Decompiled C# (frida/proxy/decompiled-*),
                          Ghidra exports for native DLLs (`exports/` only —
                          working state at `projects/` and AVEVA's input
                          binaries at `input/` are gitignored).
- docs/                   Reverse-engineering reference docs.
- tools/                  Setup-LiveProbeEnv.ps1 (Infisical credential fetcher),
                          Compute-Crc.ps1 (.NET parity helper).
- .github/workflows/      Rust CI: fmt + build + test + clippy on Windows.
- LICENSE                 MIT (Joseph Doherty, 2026).

Verified:
- cargo test --workspace → 217 passed (215 unit + 2 .NET parity), 0 failed
- cargo clippy --workspace -- -D warnings → clean
- cargo fmt --all -- --check → clean
- cargo publish --dry-run -p mxaccess-codec → packages cleanly

Excluded from history (see .gitignore):
- **/bin, **/obj, **/target — build artifacts
- analysis/ghidra/projects/ — Ghidra working state (regenerable)
- analysis/ghidra/input/ — AVEVA proprietary DLLs (vendor IP)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 06:21:00 -04:00