Initial project state: .NET reference, design, Rust port (M0+M1), evidence

Layout: - src/ .NET 10 x64 reference: MxNativeCodec, MxNativeClient, MxAsbClient, probes, tests, harnesses. Executable spec. - design/ Architectural plan for the Rust port (M0–M6), error model, protocol invariants, risks (R1–R16), adversarial review log (review.md). - rust/ Rust workspace. M0 skeleton + M1 codec parity. mxaccess-codec: 215 unit tests + 2 cross-implementation parity tests (byte-identical against .NET reference). Other crates are M0 stubs awaiting M2+. - captures/ Frida + netsh + pcap evidence per CLAUDE.md ("captures are evidence, not throwaway logs"). - analysis/ Decompiled C# (frida/proxy/decompiled-*), Ghidra exports for native DLLs (`exports/` only — working state at `projects/` and AVEVA's input binaries at `input/` are gitignored). - docs/ Reverse-engineering reference docs. - tools/ Setup-LiveProbeEnv.ps1 (Infisical credential fetcher), Compute-Crc.ps1 (.NET parity helper). - .github/workflows/ Rust CI: fmt + build + test + clippy on Windows. - LICENSE MIT (Joseph Doherty, 2026). Verified: - cargo test --workspace → 217 passed (215 unit + 2 .NET parity), 0 failed - cargo clippy --workspace -- -D warnings → clean - cargo fmt --all -- --check → clean - cargo publish --dry-run -p mxaccess-codec → packages cleanly Excluded from history (see .gitignore): - **/bin, **/obj, **/target — build artifacts - analysis/ghidra/projects/ — Ghidra working state (regenerable) - analysis/ghidra/input/ — AVEVA proprietary DLLs (vendor IP) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 06:21:00 -04:00
parent 43733699b0
commit fe2a6db786
3849 changed files with 352975 additions and 0 deletions
@@ -0,0 +1,426 @@
+# MxNativeSession API
+
+`MxNativeSession` is the first high-level .NET 10 x64 facade over the managed
+NMX protocol work in this repository. It is intended to be the consumer-facing
+entry point while the lower layers remain available for protocol tests and
+diagnostics.
+
+## Runtime prerequisites
+
+- AVEVA System Platform/NMX installed locally so `NmxSvc.NmxService` can be
+  activated.
+- The Galaxy Repository SQL database available to the current Windows session.
+- Managed NTLM runtime credentials supplied through the existing environment
+  variable path used by `ManagedNtlmClientContext`.
+- The x64 callback COM registration/type-library setup already documented in
+  `DotNet10-Native-Library-Plan.md`.
+
+The session facade does not store credentials. It relies on the process
+environment and the already-authenticated Windows/SQL context.
+
+## Basic write
+
+```csharp
+using MxNativeClient;
+
+using var session = MxNativeSession.Open(new MxNativeClientOptions
+{
+    LocalEngineId = 0x7fee,
+    EngineName = "MxNativeClient.Sample",
+    // Optional. Leave null unless a deployment-specific heartbeat cadence is known.
+    HeartbeatTicksPerBeat = null,
+});
+
+await session.WriteAsync("TestChildObject.TestInt", 123);
+await session.Write2Async(
+    "TestChildObject.TestInt",
+    124,
+    DateTime.UtcNow);
+```
+
+`HeartbeatTicksPerBeat` and `HeartbeatMaxMissedTicks` expose
+`INmxService2.SetHeartbeatSendInterval`. A .NET 10 x64 probe validated
+`SetHeartbeatSendInterval(5, 3)` against local NmxSvc. The setting remains
+opt-in because the exact MXAccess heartbeat cadence has not been captured on
+this VM; leaving `HeartbeatTicksPerBeat` null preserves the previously
+validated connection behavior.
+
+## Recovery
+
+`MxNativeSession.RecoverConnection()` explicitly rebuilds the managed NMX
+service client, re-registers the local engine and callback, reapplies optional
+heartbeat settings, reconnects known publisher engines, and replays current
+normal or buffered subscriptions with their existing correlation IDs. The
+compatibility facade exposes the same operation as
+`MxNativeCompatibilityServer.RecoverConnection(serverHandle)`.
+
+`RecoverConnectionAsync(policy, cancellationToken)` adds an explicit retry loop
+around that primitive. `MxNativeRecoveryPolicy.MaxAttempts` defaults to `1`;
+`Delay` defaults to zero. The compatibility facade exposes the same retrying
+operation as `RecoverConnectionAsync(serverHandle, policy, cancellationToken)`.
+
+Recovery progress is observable:
+
+- `RecoveryAttemptStarted` fires before each reconnect/replay attempt.
+- `RecoveryAttemptFailed` fires after a recoverable attempt failure and reports
+  the exception plus whether the retry loop will continue.
+- `RecoveryCompleted` fires after the replacement service has been registered,
+  publisher endpoints have been reconnected, subscriptions have been replayed,
+  and the session has swapped to the recovered service.
+
+The compatibility facade exposes the same events with the server handle added
+to each event payload. A live .NET 10 x64 probe with two allowed attempts and a
+100 ms delay observed one started event, zero failed events, one completed
+event, preserved the existing subscription count, and wrote through the
+recovered session.
+
+Callbacks are passed through during the recovery window instead of being
+suppressed or buffered. `MxNativeCallbackEvent`,
+`MxNativeOperationStatusEvent`, `MxNativeReferenceRegistrationEvent`, and
+`MxNativeUnparsedCallbackEvent` include `IsDuringRecovery` so callers can
+separate replay-window events from steady-state events. Compatibility
+`DataChanged`, `BufferedDataChanged`, and `WriteCompleted` payloads carry the
+same marker.
+
+`MxNativeClient.Probe --probe-session-recover-multi` subscribes to multiple
+tags, runs explicit recovery, and reports total callbacks plus
+`IsDuringRecovery` counts for data, operation-status, reference-registration,
+and unparsed callback families. The first live run replayed four subscriptions
+and preserved all four, with zero callback events marked as recovery-window
+events on this VM.
+
+Adding `--recover-concurrent-writes` starts a separate writer session during
+the same recovery window. A live run wrote `TestChildObject.TestInt` values
+`330`-`334`, preserved all four subscriptions, and observed two data callbacks
+with `IsDuringRecovery=true`. This confirms that callbacks are not quiesced by
+the library during recovery; consumers that require strict post-recovery
+ordering should ignore or queue marked events at their boundary.
+
+Automatic background recovery is intentionally not enabled yet. Retrying writes
+inside normal write calls can duplicate writes, so callers should choose where
+recovery belongs in their own operation model. A .NET 10 x64 probe validated the
+current explicit path by subscribing to `TestChildObject.TestInt`, recovering
+the connection, preserving the subscription count, and writing through the
+recovered session.
+
+`WriteSecured2Async` is implemented for the observed boolean secured/verified
+payload shape:
+
+```csharp
+await session.WriteSecured2Async(
+    "TestMachine_001.ProtectedValue",
+    true,
+    DateTime.Now,
+    currentUserId: 1,
+    verifierUserId: 0);
+```
+
+The encoder emits native command `0x38` and has been live validated from .NET
+10 x64 against the secured and verified boolean tags deployed on this node, as
+well as authenticated bool, int, float, double, string, datetime, and
+scalar-array calls against `TestChildObject` attributes. The handle-based
+compatibility facade also supports `AuthenticateUser` followed by
+`WriteSecured2`, and intentionally does not synthesize `OnWriteComplete` for
+this path because the successful native captures did not show that event.
+`WriteSecuredAsync` remains unsupported because native MXAccess still returns
+`0x80004021` before emitting a value-bearing body in every captured scenario.
+Additional native captures beyond bool and int would improve fixture coverage,
+but the managed encoder is now generic over the timestamped write-body support.
+
+## Browse
+
+```csharp
+IReadOnlyList<GalaxyTagMetadata> tags =
+    await session.BrowseAsync("TestChildObject", "Test%", maxRows: 25);
+```
+
+`BrowseAsync` uses Galaxy Repository metadata directly. It does not call the
+x86 LMX resolver and returns the same metadata needed by the managed handle and
+wire encoders.
+
+`ResolveAsync` accepts full references in the observed MXAccess forms:
+`Object.Attribute`, `Object.Primitive.Attribute`, and
+`Object.Primitive.Dotted.Attribute` for primitive attributes such as
+`TestMachine_001.TestAlarm001.Alarm.TimeDeadband`. It also recognizes the
+captured literal property form `Object.Attribute.property(buffer)`, resolving it
+to the base attribute handle with property id `0x32`.
+
+Each `GalaxyTagMetadata` exposes `IsSupportedValueKind` and
+`TryGetValueKind(out MxValueKind valueKind)`. Use those before write/read
+projection when browsing broad GR metadata. The current wire codec supports the
+live OPC-UA-critical GR types `Boolean`, `Integer`, `Float`, `Double`, `String`,
+`Time`, and array forms captured so far. Live GR inspection also found
+`ElapsedTime` and `InternationalizedString`. Subscribe/read callback decoding now
+handles the observed `ElapsedTime` wire kind `0x07` as `TimeSpan` and the compact
+empty `InternationalizedString`/string payload form `0x05 04 00 00 00` as
+`string.Empty`. They are still reported as unsupported by `TryGetValueKind`
+because they are not core one-to-one data kinds, but write projection now follows
+captured MXAccess caller-variant behavior: `ElapsedTime` values supplied as
+`TimeSpan` or integer project to integer milliseconds/wire kind `0x02`, and
+`InternationalizedString` values project to normal string wire kind `0x05`.
+
+`WriteAsync` and `Write2Async` resolve the tag from Galaxy Repository metadata,
+build the managed `MxReferenceHandle`, encode the NMX write body, wrap it in
+the `TransferData` envelope, and call `INmxService2::TransferData` through the
+managed DCOM path.
+
+## Subscribe
+
+```csharp
+using MxNativeClient;
+
+using var session = MxNativeSession.Open();
+
+session.CallbackReceived += (_, evt) =>
+{
+    Console.WriteLine(
+        $"{evt.Record.TimestampUtc:O} status={evt.Status.Category} quality=0x{evt.Record.Quality:X4} value={evt.Record.Value}");
+};
+
+session.OperationStatusReceived += (_, evt) =>
+{
+    Console.WriteLine(
+        $"operation_status=0x{evt.Message.StatusCode:X4} status={evt.Message.Status.Category}");
+};
+
+MxNativeSubscription sub = await session.SubscribeAsync("TestChildObject.TestInt");
+await Task.Delay(TimeSpan.FromSeconds(10));
+session.Unsubscribe(sub.CorrelationId);
+```
+
+## Read
+
+```csharp
+object? value = await session.ReadAsync(
+    "TestChildObject.TestInt",
+    timeout: TimeSpan.FromSeconds(10));
+```
+
+`ReadAsync` is implemented as a transient subscription read. It subscribes,
+waits for the first typed value callback matching that item correlation, then
+unsubscribes.
+
+`SubscribeAsync` performs the managed equivalent of the observed MXAccess path:
+
+1. Resolve tag metadata from the Galaxy Repository.
+2. Connect the local engine to the tag's owning engine.
+3. Add the local engine as a subscriber.
+4. Send the generated item-control `0x1f` body using the native value handle.
+5. Decode incoming `INmxSvcCallback` bodies with `NmxSubscriptionMessage`.
+
+Capture `099-frida-plain-advise-testint` showed public `Advise` and the earlier
+`AdviseSupervisory` scalar path using the same `0x1f` body shape for
+`TestChildObject.TestInt`, so the compatibility wrapper maps both methods onto
+this subscription path.
+
+`CallbackReceived` is raised for typed `0x32` subscription-status and `0x33`
+data-update records. Each callback exposes the raw NMX record fields plus a
+first-pass `MxStatus` projection. `MxStatus.DetailText` returns the installed
+English `Lmx.aaDCT` detail text for known LMX/NMX status details, including
+reference, conversion, security, alarm, initializing, and secured/verified
+write conditions. `OperationStatusReceived`
+is raised for both observed operation-status frame forms. The non-length-prefixed
+5-byte status-word form with status word `0x8050` and completion byte `0x00`
+maps to `MxStatus.WriteCompleteOk` and is the captured public
+`OnWriteComplete` success notification. Length-prefixed completion-only frames
+are decoded separately and expose `NmxOperationStatusFormat.CompletionOnly`.
+Captures `089`, `092`, and `093` produced completion byte `0x41` after
+wrong-type string writes, and capture `091` produced completion byte `0x00`
+after double-to-int coercion; MXAccess did not raise `OnWriteComplete` for
+either completion-only form.
+
+The managed x64 path now has a proven value-bearing subscription fixture:
+`SubscribeAsync("TestChildObject.ShortDesc")` receives the same compact empty
+string callback observed from x86 MXAccess (`0x32`, wire kind `0x05`, quality
+`0x00C0`). The key handle detail is that value references use property id `10`;
+GR `mx_attribute_category` is metadata and is not the NMX value-handle property
+field.
+
+String-array callback handling is intentionally conservative. Captures `100`
+and `101` showed `0x45` string-array callback records whose buffers stopped
+inside the final element, and MXAccess did not raise a public data-change event
+for those runs. The decoder does not fabricate a value for malformed array
+payloads; callers should treat a `null` value with a known array wire kind as an
+incomplete callback until a complete/public string-array callback is captured.
+
+`ReferenceRegistrationReceived` is raised for decoded `0x11`
+reference-registration result frames. Those frames are emitted after the `0x10`
+normal/buffered registration request and include the item handle, correlation
+ID, reference text, context, and MX data type.
+
+`UnparsedCallbackReceived` is a diagnostic event for callback bodies that are
+not yet part of the high-level API surface. It is currently used to expose
+frames such as the 92-byte operation/status-shaped body seen before scalar
+subscription status callbacks, rather than silently swallowing unknown protocol
+bodies during reverse engineering.
+
+## Probe commands
+
+The probe now has facade-level entry points:
+
+```powershell
+dotnet run --project .\src\MxNativeClient.Probe\MxNativeClient.Probe.csproj -c Release -- --probe-session-write --tag=TestChildObject.TestInt --value=123 --objref-only
+dotnet run --project .\src\MxNativeClient.Probe\MxNativeClient.Probe.csproj -c Release -- --probe-session-read --tag=TestChildObject.TestInt --read-timeout-seconds=10 --objref-only
+dotnet run --project .\src\MxNativeClient.Probe\MxNativeClient.Probe.csproj -c Release -- --probe-session-subscribe --tag=TestChildObject.TestInt --subscribe-hold-seconds=5 --objref-only
+dotnet run --project .\src\MxNativeClient.Probe\MxNativeClient.Probe.csproj -c Release -- --probe-session-subscribe --tag=TestChildObject.ShortDesc --subscribe-hold-seconds=8 --objref-only
+dotnet run --project .\src\MxNativeClient.Probe\MxNativeClient.Probe.csproj -c Release -- --probe-compatibility-subscribe --tag=TestChildObject.ShortDesc --subscribe-hold-seconds=8 --objref-only
+dotnet run --project .\src\MxNativeClient.Probe\MxNativeClient.Probe.csproj -c Release -- --probe-compatibility-subscribe-write --tag=TestChildObject.TestInt --value=793 --subscribe-hold-seconds=8 --objref-only
+dotnet run --project .\src\MxNativeClient.Probe\MxNativeClient.Probe.csproj -c Release -- --probe-compatibility-subscribe --tag=NoSuchObject_999.NoSuchAttr --subscribe-hold-seconds=2 --objref-only
+dotnet run --project .\src\MxNativeClient.Probe\MxNativeClient.Probe.csproj -c Release -- --probe-compatibility-subscribe-multi --tag=TestChildObject.ShortDesc --tag=TestChildObject.TestInt --tag=NoSuchObject_999.NoSuchAttr --subscribe-hold-seconds=4 --objref-only
+dotnet run --project .\src\MxNativeClient.Probe\MxNativeClient.Probe.csproj -c Release -- --probe-compatibility-write-multi --subscribe-hold-seconds=4 --objref-only
+dotnet run --project .\src\MxNativeClient.Probe\MxNativeClient.Probe.csproj -c Release -- --probe-compatibility-write-unadvised --tag=TestChildObject.TestInt --value=99 --objref-only
+dotnet run --project .\src\MxNativeClient.Probe\MxNativeClient.Probe.csproj -c Release -- --probe-managed-subscribe --tag=TestChildObject.TestInt --subscribe-hold-seconds=5 --send-observed-preadvise-metadata --objref-only
+```
+
+These commands require the same runtime managed NTLM environment as the lower
+level managed probes.
+
+## MXAccess-style compatibility wrapper
+
+`MxNativeCompatibilityServer` provides an MXAccess-like handle model for code
+that expects integer server and item handles:
+
+```csharp
+using MxNativeClient;
+
+using var mx = new MxNativeCompatibilityServer();
+
+mx.DataChanged += (_, evt) =>
+{
+    Console.WriteLine(
+        $"server={evt.ServerHandle} item={evt.ItemHandle} value={evt.Value} quality=0x{evt.Quality:X4}");
+};
+
+mx.WriteCompleted += (_, evt) =>
+{
+    Console.WriteLine(
+        $"server={evt.ServerHandle} item={evt.ItemHandle} status={evt.Statuses[0].Category}");
+};
+
+int server = mx.Register("OpcUaBridge.Native");
+int item = mx.AddItem(server, "TestChildObject.TestInt");
+mx.AdviseSupervisory(server, item);
+mx.Write(server, item, 123);
+mx.UnAdvise(server, item);
+mx.RemoveItem(server, item);
+mx.Unregister(server);
+```
+
+The compatibility wrapper maps `Register`, `Unregister`, `AddItem`,
+`RemoveItem`, `Advise`, `AdviseSupervisory`, `UnAdvise`, `Write`, and `Write2`
+onto `MxNativeSession`. `Suspend` and `Activate` mirror observed MXAccess local
+behavior for advised items: unadvised calls throw, `Suspend` returns
+pending/requesting-LMX status, and `Activate` returns ok/requesting-LMX status.
+After `RemoveItem`, the wrapper mirrors native x86 stale-item behavior by
+returning `ArgumentException` with `HResult=0x80070057` for advise, unadvise,
+write, write2, suspend, activate, and repeated remove attempts.
+Invalid server handles and cross-server item handles use the same
+`ArgumentException`/`0x80070057` shape observed from native MXAccess.
+Literal `property(buffer)` items added with `AddItem` are not treated as
+`AddBufferedItem` handles. They follow the captured MXAccess literal-reference
+path: add/advise succeeds, normal write returns without a public write-complete,
+and no public data-change is promoted on the current VM state.
+`AddItem2` resolves the item definition directly first and then retries with the
+supplied context. This covers the captured simple form
+`AddItem2("TestInt", "TestChildObject")` plus dotted relative forms such as
+`AddItem2("Alarm.TimeDeadband", "TestMachine_001.TestAlarm001")` and
+`AddItem2("TestInt.property(buffer)", "TestChildObject")`.
+`AddBufferedItem` and `SetBufferedUpdateInterval` now cover the decoded public
+API path: buffered add creates a handle, the interval is stored as local session
+state, and advising the buffered handle sends the observed NMX `0x10`
+registration for `itemDefinition.property(buffer)` in the supplied item
+context. Headless Ghidra analysis shows native MXAccess routes buffered item
+callbacks through the same `OnDataChange` callback method as normal items, then
+branches to `_ILMXProxyServerEvents2::OnBufferedDataChange` when the item record
+is marked buffered. `MxNativeCompatibilityServer.BufferedDataChanged` mirrors
+that separation for any parsed buffered callback and does not promote buffered
+items through `DataChanged`. Captures against `TestChildObject.TestInt` and
+GR-confirmed historized `TestMachine_001.TestHistoryValue` prove the outbound
+context-bearing registration/result bodies, but this VM has not emitted a live
+`OnBufferedDataChange` payload. True multi-sample buffered payload decoding
+still needs that runtime condition.
+`ArchestrAUserToId` follows the observed MXAccess x86
+behavior on this node: known user GUIDs and an invalid zero GUID all return `1`.
+`AuthenticateUser` follows the same session-local handle model for the observed
+dev-node behavior: MXAccess returned `S_OK` and user ID `1` for both
+`Administrator` and an invalid user name with an empty password. The managed
+compatibility method validates the server handle and user string, returns a
+session-local handle, and does not store or compare password material.
+`WriteCompleted` is raised only for the captured MXAccess-visible 5-byte
+`0x8050/0x00` operation-status frame. Completion-only frames are still available
+through `MxNativeSession.OperationStatusReceived`, but the compatibility wrapper
+does not convert them into `OnWriteComplete` events because x86 MXAccess did not
+fire that event in those captures. Pending write handles are queued per server
+session, so concurrent compatibility sessions do not consume each other's
+completion callbacks. The mixed `--probe-compatibility-write-multi` path
+validates that normal int, internationalized string, literal buffer-property,
+and invalid-reference items in one server session do not leak public data-change
+or write-complete events across item handles; only the invalid reference emits
+the expected configuration-error data-change. `OperationCompleted` keeps the public event
+shape, but no trigger has been modeled yet because the capture set contains no
+`mx.event.operation-complete` events.
+`DataChanged` is also suppressed when the lower-level decoder reports a known
+wire kind with a `null` value, which is how malformed/incomplete callback
+payloads such as the observed string-array records from captures `100` and
+`101` are represented.
+`GalaxyRepositoryUserResolver` is separate metadata support: it resolves
+`dbo.user_profile` rows, profile IDs, default security group, InTouch access
+level, and decoded role names from the GR role blob. `AddItem2` currently
+combines context and item text only for simple relative references; the real
+MXAccess context behavior still needs captures. Unsupported MXAccess methods
+throw `NotSupportedException` with the missing capture/protocol reason.
+
+For lower-level use, `GalaxyRepositoryUserResolver` exposes:
+
+```csharp
+var users = new GalaxyRepositoryUserResolver();
+GalaxyUserProfile profile = await users.ResolveByNameAsync("Administrator");
+int profileId = await users.ResolveUserProfileIdByGuidAsync(profile.UserGuid);
+IReadOnlyList<string> roles = profile.Roles;
+```
+
+Security-enabled `AuthenticateUser` password verification remains intentionally
+out of scope until successful and failed captures from a security-enabled Galaxy
+show whether MXAccess delegates to OS/domain auth, GR hashes, or another token
+provider.
+
+## Current limitations
+
+- Managed NTLM live probes now validate .NET 10 x64 activation, RemQI,
+  `GetPartnerVersion`, direct `WriteAsync` transport, callback delivery, and
+  unsubscribe cleanup with unique local engine IDs.
+- Value-bearing managed `OnDataChange` is proven for
+  `TestChildObject.ShortDesc` after matching native advise transfer kind and
+  value-handle property id. The lower-level session exposes that callback, but
+  the MXAccess-style compatibility wrapper suppresses the empty
+  `InternationalizedString` public `DataChanged` event because fresh x86
+  MXAccess capture `108` also raises no public event for `ShortDesc`. Fresh
+  current-state x86 MXAccess captures also raise no public `TestInt`
+  data-change for subscribe-only or subscribe-then-write, matching the managed
+  status-only `TestInt` callbacks on this VM. Broader scalar/array parity still
+  needs live validation against a runtime state that emits public values.
+  Replaying the captured
+  adapter-visible `0x17` metadata body is accepted by `NmxSvc` and produces
+  decoded `0x40` metadata-response plus `0x32` metadata-status callbacks, but
+  the response still contains an internal base-object error and does not appear
+  to be the gate for the `ShortDesc` value callback.
+- `ReadAsync` currently uses a transient subscription. A lower-latency direct
+  read path can be added if a distinct NMX request/response read body is found;
+  tags with no initial value can still time out after receiving only status
+  callbacks.
+- String-array callbacks are partially modeled from the element format, but the
+  available Frida string-array callback line is truncated before all values.
+- Error/failure callback bodies still need more captures for exact
+  `MXSTATUS_PROXY[]` parity.
+- `ElapsedTime` and `InternationalizedString` writes are value-projected from
+  captures `095` and `096`. The managed `ShortDesc` write body now matches
+  capture `096` byte-for-byte and returns the same completion-only `0xef` as
+  native MXAccess on this node, while `TestInt` returns completion-only `0x00`
+  and a status-only update callback. Public write-complete semantics for these
+  completion-only frames still need more captures.
+- `MxNativeCompatibilityServer` provides source-level migration help but is not
+  a binary COM replacement for `ArchestrA.MxAccess`.
+- For invalid references, the compatibility wrapper intentionally differs from
+  strict `MxNativeSession`: it returns an item handle and raises the observed
+  public configuration-error data-change (`null`, quality `0`, detail `6`) on
+  advise, matching x86 MXAccess captures.