dohertj2/mxaccess
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
59 Commits 1 Branch 0 Tags
104efc4e9baba5caa8ac1fb73f19bbc832385180
Commit Graph

8 Commits

Author SHA1 Message Date
Joseph Doherty 104efc4e9b [M5] mxaccess-asb: F28 wire-format fixes — AuthenticateMe accepted live 
Three wire-level bugs surfaced by side-by-side relay capture against
the .NET probe routed via the new --via flag:

1. **Dynamic-dictionary id drift**. Our `encode_envelope` hardcoded
   action_dict_id=1 / to_dict_id=3, which is correct for the FIRST
   message in a session but wrong for every subsequent one. The
   per-session dynamic dict accumulates across messages: Connect's
   binary header pre-pops [action,to] at ids 1,3; AuthenticateMe must
   reference the new action at id 5 (continuing the odd sequence) and
   the To URL at id 3 (still in the dict from Connect). Fix uses
   `DynamicDictionary::position_of` + `intern` to compute the right
   wire id, only pre-popping strings that are NEW to the session.
   Captured against .NET probe via asb-relay: AuthenticateMe binary
   header has only one string (action) at offset 0x260 (`06 de 08 2f
   2e ...`), and `<a:Action>` value `ab 05` references the new id 5.

2. **ConnectionValidator wire format depends on operation**. .NET's
   `IAsbDataV2` declares `[XmlSerializerFormat]` on AuthenticateMe,
   Disconnect, KeepAlive (one-way ops) — those use XmlSerializer for
   the ENTIRE message including the [MessageHeader] ConnectionValid-
   ator. Other ops use the default DataContractSerializer. The wire
   shapes differ:
     XmlSerializer: `<ConnectionId xmlns="http://asb.contracts.data/
       20111111">guid</ConnectionId>` (PascalCase property name in
       data namespace)
     DataContract: `<connectionIdField xmlns="http://schemas.data
       contract.org/2004/07/ArchestrAServices.ASBContract">guid</…>`
       (private "fooField" name in datacontract namespace)
   New `ValidatorWireFormat::for_action` picks the right shape per
   action; `encode_validator` now branches on it. New helpers
   `push_xml_text_field` / `push_xml_byte_array_field` for the
   XmlSerializer form. The DataContract form is preserved verbatim
   for Register/Read/Write/etc.

3. **Decoder missing 0x0A** (`ShortDictionaryXmlnsAttribute`). The
   server's RegisterItemsResponse uses `0x0A {dict-id}` to set the
   default namespace from the static dict; our decoder bailed out
   with `UnknownRecord(10)`. New decode arm produces a
   `DefaultNamespace` token with `DictionaryStatic` value.

**.NET probe gains a `--via` flag** (`AsbConnectionOptions.Via` →
`ChannelFactory.CreateChannel(addr, viaUri)`) so the probe can be
routed through asb-relay for byte-level capture without triggering
an `AddressFilterMismatch` fault. CoreWCF / .NET 10 dropped
`ClientViaBehavior`; the `CreateChannel(addr, via)` overload is the
modern equivalent.

Live status (this commit): Connect handshake works, AuthenticateMe
no longer faults (canonical XML + crypto + wire-format all match
.NET now), RegisterItemsResponse comes back from the server (a real
response, not a dispatcher fault). One remaining issue: our response
decoder hits `MissingField { field: "Status" }` — the server's
RegisterItemsResponse uses a slightly different element naming or
encoding than `collect_asbidata_payloads` expects. Next iteration
hunts that.

Workspace: 710 unit tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 19:29:48 -04:00
Joseph Doherty f14580e0db [M5] mxaccess-asb: F28 canonical-XML signing wired + registry-driven DH params 
Adds `xml_canonical` module that emits XmlSerializer-compatible canonical
XML for the five primary `ConnectedRequest` shapes (AuthenticateMe,
Disconnect, KeepAlive, RegisterItemsRequest, UnregisterItemsRequest).
Six fixture-comparison tests verify byte-exact match against captured
.NET output, including the empty-MAC-IV variant that the live signing
flow uses (`authenticate-me-empty-mac-iv.xml`, 896 bytes; new
`emit_data_ns_byte_array` helper picks self-closing form for empty
byte[]).

Plumbing: `AsbAuthenticator::peek_next_message_number` exposes the
pre-allocated message number; `AsbClient::send_signed_envelope[_one_way]`
gain an `xml_for_signing: Option<&[u8]>` parameter. `connect`,
`disconnect`, `keep_alive`, `register_items`, `unregister_items` now
build a pre-signing `ConnectionValidator` (empty MAC + IV) + emit the
canonical XML + pass the bytes through to HMAC. Other ops (Read, Write,
Subscription) keep the legacy NBFX-bytes path until F28 expands to
cover their request shapes.

Live-bring-up wiring:
- `tools/Get-AsbPassphrase.ps1` now exports `MX_ASB_DH_PRIME`,
  `MX_ASB_DH_GENERATOR`, `MX_ASB_DH_HASH_ALGORITHM` (always — even when
  empty, so the example can distinguish "no env var" from "registry
  says empty"), and `MX_ASB_DH_KEY_SIZE`.
- `examples/asb-subscribe.rs` honours those env vars to override
  `CryptoParameters::defaults()`. Each AVEVA install picks its own DH
  group at provisioning time (768-bit prime is typical, vs the .NET
  reference's 1024-bit fallback that we previously hardcoded). Empty
  hashAlgorithm in the registry maps to `HashAlgorithm::Unrecognised`,
  matching `AsbSystemAuthenticator.CreateHmac:84-93` semantics where
  empty + forceHmac=true → HMAC-SHA1.
- `MxAsbClient.Probe --dump-signed-xml` flag (added in earlier commit)
  now traces the live HMAC inputs (`asb.sign.xml-utf8-len`,
  `asb.sign.xml-b64`, `asb.sign.hmac-b64`, etc.) so the Rust port can
  diff its canonical XML against .NET's byte-for-byte for any live
  scenario (env-driven via `Action<string>? sharedTrace`).

Wire-format alignment for `XmlSerializer` parity:
- `ItemIdentity::default()` and `absolute_by_name` now use
  `Some(String::new())` for null-able strings (matches .NET's
  `CreateAbsoluteItem` setting `ContextName = string.Empty` not null).
- `read_unicode_string` returns `Some(String::new())` for length-0
  rather than `None` — mirrors .NET's `AsbBinary.ReadUnicodeString:
  return string.Empty for byteLength == 0`. Wire format genuinely
  cannot distinguish null from empty (both encode as 4 bytes of zero);
  callers that need to preserve the distinction MUST track it in their
  domain types before encoding.

Live status (post-fix): Connect handshake completes end-to-end. The
canonical XML our emitter produces matches .NET's structure byte-for-
byte (verified by fixture comparison). DH prime/generator/hash now
match the live registry values. Despite all this, AuthenticateMe
still produces a generic dispatcher fault on the server — there's at
least one more subtle wire-byte or crypto mismatch that needs
isolation. F28 stays open with that note.

Workspace: 709 unit tests pass (was 702 + 7 new xml_canonical tests).
Clippy: clean (`-D warnings`).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 17:31:31 -04:00
Joseph Doherty 4c4177050c [M5] mxaccess-asb-nettcp/asb: xmlns raw-string + xsi/xsd on Body 
WIRE-FORMAT BREAKTHROUGH — our envelope is now valid NBFX/WCF.
ConnectRequest reaches the server's operation handler. Direct-port-808
Connect now returns a server-side fault (operation invocation
error from the placeholder DH key) rather than TCP RST. With a real
DH key, asb-subscribe gets all the way to "response is missing
required field ServicePublicKey/Data" — meaning our Connect request
processed end-to-end, the server returned a ConnectResponse, and
the only remaining issue is in our response-body decoder.

Fixes:

1. **`XmlnsAttribute` (0x09) value is a RAW length-prefixed string,
   not a text record**. Per `[MC-NBFX]` §2.2.3, xmlns attribute
   values are `[length][bytes]`, NOT `[text-record-byte][value]`.
   Our F21 was emitting `aa <id>` for dict-static values which the
   receiver misparsed as a 0xAA-length string. Same fix applies to
   `ShortXmlnsAttribute` (0x08). Encoder now picks raw-string for
   `Chars` value, raw-int31 (via 0x0B) for `DictionaryStatic` value;
   decoder reads raw string in both code paths.

2. **xmlns:xsi + xmlns:xsd on `<s:Body>`**. WCF declares these
   namespaces on Body before opening the operation request element.
   Our envelope encoder now emits both as raw-string xmlns attrs
   right after `<s:Body>` opens. Required for `xsi:type` annotations
   that appear inside DataContract-serialised body fields.

Combined wire-byte impact (verified via asb-relay side-by-side
diff):

* All header bytes match .NET byte-for-byte through the entire
  `<s:Header>` section (Action / ConnectionValidator / MessageID
  via UniqueIdText / ReplyTo / To).
* `<s:Body>` xmlns:xsi + xmlns:xsd declarations match .NET.
* `<ConnectRequest>` opens identically.
* `<ConnectionId>` / `<ConsumerPublicKey>` / `<Data>` element names
  match.
* The only known remaining diff in the request: .NET emits
  `xmlns="http://asb.contracts.data/20111111"` on the inner
  `<Data>` element (the PublicKey class's XmlType namespace);
  we don't. Likely an issue but apparently non-fatal — the server
  processed our request successfully past this point.

Live status:
* Direct port-808 connect with real DH key: server returns
  "response is missing required field ServicePublicKey/Data" —
  meaning we sent a valid Connect, server replied with a
  ConnectResponse, but our decoder can't find the field. Next
  iteration is response-side decode work.

Workspace: 702 tests pass; clippy + fmt clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 15:57:31 -04:00
Joseph Doherty c2222b16b0 [M5] mxaccess-asb-nettcp/asb: F21 short forms + EndElement fix + UniqueIdText 
Three NBFX-spec corrections discovered by diffing our wire output
against the .NET probe's capture:

1. **EndElement is 0x01, NOT 0x00**. Our F21 had this wrong since the
   first iteration. Our round-trip tests passed because encode and
   decode used the same wrong value, but interop with WCF's parser
   silently failed (TCP RST on every request). Fixed by changing
   `REC_END_ELEMENT` to 0x01 — all 702 tests pass on the new value.

2. **Single-letter prefix short forms**. WCF uses
   `PrefixDictionaryElement_<a-z>` (records 0x44-0x5D) and
   `PrefixDictionaryAttribute_<a-z>` (records 0x0C-0x25) for
   single-character prefixes. Our F21 always used the long forms
   (0x43 prefix-string + dict-id, etc.). The encoder now emits the
   short form when the prefix is a single ASCII lowercase letter; the
   decoder accepts both. New `prefix_letter_offset(prefix)` helper.

3. **`DictionaryXmlnsAttribute` (0x0B)** for xmlns:prefix declarations
   whose value is a static-dict id. The long form (0x09 +
   prefix-string + text-record) is still emitted when the value is an
   inline string, but for `xmlns:s="...soap-envelope"` (dict id 4) we
   now emit the short `0b 01 73 04` form WCF uses.

4. **UniqueIdText (0xAC)** added to `NbfxText` enum + encode/decode.
   WCF emits `<a:MessageID>` as a UniqueIdText carrying the 16 raw
   UUID bytes (NOT the `urn:uuid:...` text form). Updated
   `encode_envelope` to use this for MessageID.

Combined wire-byte impact: our envelope body section now matches the
.NET probe byte-for-byte through `<a:Action>`, `<h:ConnectionValidator>`,
`<a:MessageID>` (UniqueId), `<a:ReplyTo>`, `<a:To>`, and `<s:Body>`.
The trailing `01 01 01 01` = 4 EndElements is now the correct
record byte. Tests pass (702 total).

Live status: still TCP RST after the SizedEnvelope. Remaining
unknown is in the body section — the .NET capture shows xmlns:xsi /
xmlns:xsd declarations on the operation-specific request element
(ConnectRequest etc.) that we don't emit, plus possibly different
field encoding inside ConnectRequest. Next iteration will re-capture
through the relay and diff our body bytes against the new
.NET-byte-equivalent we now produce.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 15:48:03 -04:00
Joseph Doherty 2867310817 [M5] mxaccess-asb: WCF binary message header (action+to dict pre-pop) 
Adds the binary header block that WCF prepends to SizedEnvelope
payloads. Reverse-engineered from .NET probe wire bytes captured via
asb-relay.

Wire form (per the .NET capture analysis in the previous commit):
```
[outer length, multibyte-int31]
  [string-1 length, multibyte-int31] [UTF-8 bytes]   ← dict id 1 (action)
  [string-2 length, multibyte-int31] [UTF-8 bytes]   ← dict id 3 (to)
[NBFX <s:Envelope>...]
```

Inside the NBFX envelope, `<a:Action>` and `<a:To>` reference the
pre-pop strings via `DictionaryText 0xAA {odd-id}` instead of inlining
their values. The header strings get assigned odd dict ids
(1, 3, 5, ...); even ids stay reserved for the [MC-NBFS] static dict.

Encode side:
* `encode_envelope` now emits header [action, to] before NBFX. `to_uri`
  defaults to empty string when None — caller-supplied `with_to(uri)`
  is the supported path.
* AsbClient's `send_envelope` and `send_envelope_one_way` auto-fill
  `to_uri` from `self.via_uri` when not set.
* New private `encode_binary_header(strings)` helper.

Decode side:
* New `parse_binary_header_prefix(input)` heuristically detects + parses
  the header (look for plausible NBFX element record byte 0x40-0x77 at
  the offset implied by the outer length).
* New `resolve_with_header(text, dynamic, header)` resolves
  `DictionaryText` with odd id by indexing into header.strings; even
  ids fall through to static-dict lookup as before.

Tests pass (72) — round-trip envelope → bytes → envelope recovers
action through the new dict-id resolution path.

Live status: this commit gets us further but the connect SOAP
envelope still TCP-RSTs at SMSvcHost. The remaining delta vs the .NET
capture is structural NBFX optimisation: .NET uses single-letter
prefix-element/attribute records (0x44-0x77 PrefixDictionaryElement
_<a-z>, 0x0C-0x25 PrefixDictionaryAttribute_<a-z>, 0x0B
DictionaryXmlnsAttribute) while our F21 encoder always uses the long
forms (0x43 prefix-string + name-dict-id, etc.). Logically
equivalent but WCF's parser likely strict on which form it accepts.

Next iteration will add short-form encoding to F21 for single-letter
prefixes (s:, a:, h:, i:) which covers every namespace prefix in our
envelope.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 15:40:59 -04:00
Joseph Doherty 3b09297b27 [M5] live-probe iteration 1 — major wire-byte reconciliation fixes 
First live-test cycle against AVEVA on this box. Comparing the .NET
probe's `--dump-messages` XML output against our NBFX-encoded
envelope surfaced six structural bugs in the F25 envelope/operations
layer. All fixed; tests passing (702 workspace).

Fixes (all backed by the .NET dump as ground truth):

1. **`mustUnderstand` attribute name** — NBFS dict id was 116
   (`MustUnderstand`, capital-M, a different SOAP token); SOAP 1.2
   spec uses lowercase `mustUnderstand` at id 0. Sending the wrong
   one triggered a WCF parse fault that surfaced as TCP RST.

2. **Missing `<a:MessageID>` header** — WCF's default binding
   requires MessageID for two-way operations. We now auto-generate
   `urn:uuid:<v4>` per envelope via a small inline `make_random_uuid_v4`
   helper (no `uuid` crate dep).

3. **Missing `<a:ReplyTo>` anonymous header** — WCF's
   BinaryMessageEncoder always emits `<a:ReplyTo><a:Address>...
   addressing/anonymous</a:Address></a:ReplyTo>` for two-way ops.

4. **ConnectionValidator field names + namespace** — we were
   emitting PascalCase `<ConnectionId>` etc. .NET's WCF
   DataContractSerializer uses the private backing-field names
   (`<connectionIdField xmlns="...ASBContract">guid</connectionIdField>`)
   per `[DataMember(Name = "fooField")]`. Added the
   `xmlns:i="...XMLSchema-instance"` declaration WCF emits
   alongside (even when no `i:nil` is used). Decoder now accepts
   both PascalCase (legacy tests) and DataContract field names.

5. **`<ASBIData>` over-wrapping** — we were emitting
   `<Items><ASBIData>{bytes}</ASBIData></Items>`. .NET's
   `AsbDataCustomSerializer.WriteStartObject` (`AsbContracts.cs:
   1561-1572`) REPLACES the field's outer element with `<ASBIData>`
   directly — there's no `<Items>` wrapper on the wire. Fixed by
   collapsing `BodyField::AsbiDataElement` to emit just `<ASBIData>`
   without the named outer element. The `name` field is retained
   for self-documentation.

6. **`collect_asbidata_payloads` API** — was keyed by field name
   (`Status` / `Values`); now positional (`payloads[0]`,
   `payloads.get(1)`) since the wrapper element is gone. All seven
   response decoders updated.

Plus tooling for the live-probe loop:
* `tools/Get-AsbPassphrase.ps1` — DPAPI loader that auto-discovers
  the solution name + reads the sharedsecret + decrypts it. Sets
  $env:MX_ASB_PASSPHRASE / MX_ASB_HOST / MX_ASB_VIA / MX_LIVE.
  Lowercase via-host (WCF SMSvcHost is case-sensitive on the URL
  host segment).
* `examples/asb-preamble-probe.rs` — diagnostic that connects,
  runs the preamble, captures the PreambleAck, then sends a
  synthetic ConnectRequest and dumps both directions as hex. Used
  to bisect the wire-byte deltas above.
* `examples/asb-subscribe.rs` port default fixed (5074 → 808 —
  WCF's NetTcpPortSharing/SMSvcHost listener confirmed via
  Get-NetTCPConnection).

**Status**: preamble + PreambleAck round-trip works end-to-end
against the live AVEVA install (verified via probe). The
post-preamble Connect SOAP envelope still gets TCP RST'd — the six
structural fixes above are necessary but not yet sufficient. Next
iteration needs binary wire capture (Wireshark + Npcap loopback,
or a TCP-relay middleman) to compare the .NET probe's BinaryMessageEncoder
output byte-for-byte with ours and find the remaining delta(s).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 15:06:48 -04:00
Joseph Doherty 321b7963a4 [M5] mxaccess-asb: F25 step 6 — Connect/AuthenticateMe handshake 
Critical-path piece that turns a fresh TCP stream into an
authenticated session. With this slice landed, an `AsbClient` can
now do `send_preamble().await? -> connect().await? -> register_items()`
end-to-end against a peer.

Operations API additions:
* `build_connect_request_body(connection_id, public_key)` — first op
  on a fresh session. **Unsigned** (no ConnectionValidator header)
  because the authenticator hasn't received the service key yet.
  Wire shape: `<ConnectRequest xmlns="…messages/20111111">
    <ConnectionId>{guid-text}</ConnectionId>
    <ConsumerPublicKey><Data>{pubkey-bytes}</Data></ConsumerPublicKey>
  </ConnectRequest>` per `AsbContracts.cs:78-86`.
* `build_authenticate_me_request_body(data, iv)` — second op,
  **one-way + signed with `forceHmac=true`** per `MxAsbDataClient.cs
  :106-111`. Carries the encrypted `local_pub || remote_pub` blob
  produced by F23's `create_authentication_data()`.
* `ConnectResponse { service_public_key, service_authentication_data,
  connection_lifetime }` + `AuthenticationDataBytes { data, iv }`.
* `decode_connect_response(body, dict)` — extracts ServicePublicKey
  (required), optional ServiceAuthenticationData, optional
  ConnectionLifetime. The lifetime's `:V2` suffix is what F23
  inspects to toggle Apollo (raw AES) vs Baktun (deflate-then-AES)
  encryption.

Client API addition:
* `AsbClient::connect()` — orchestrates the full handshake:
  1. Build + send ConnectRequest (unsigned) carrying our DH public
     key + connection-id GUID.
  2. Decode ConnectResponse.
  3. `authenticator.accept_connect_response(...)` — feeds the
     service public key + lifetime into F23 so it derives the
     shared secret and picks Apollo/Baktun.
  4. `authenticator.create_authentication_data()` — encrypts
     `local_pub || remote_pub` under the derived AES key.
  5. Send AuthenticateMeRequest (one-way, signed with HMAC-SHA1
     forced).
  Returns the `ConnectResponse` so callers can inspect the
  negotiated connection lifetime.

6 new tests:
* ConnectRequest carries hyphenated GUID + raw public-key bytes.
* AuthenticateMe carries Data + IV bytes in order.
* ConnectResponse round-trip with all optional fields populated.
* ConnectResponse round-trip without optional fields.
* ConnectResponse decoder surfaces MissingField when
  ServicePublicKey is absent.
* End-to-end client::connect handshake via `tokio::io::duplex`
  peer that synthesises a ConnectResponse using bob's public key
  (so DH shared-secret derivation actually works) and drains the
  AuthenticateMe one-way SizedEnvelope.

Wire-byte caveat documented inline: WCF XML serialization may add
`xsi:type` attributes / distinct namespaces around <PublicKey> /
<AuthenticationData>; this builder ships the simplest plausible
shape and the live-probe iteration will reconcile.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 11:47:35 -04:00
Joseph Doherty 25dbd8d3bd [M5] mxaccess-asb: F25 step 1 — SOAP envelope codec 
First slice of F25. Provides the building blocks the per-operation
request/response codecs and the network loop will compose:

* `actions` module — IASBIDataV2 action strings (all 14 operations,
  verbatim from `AsbContracts.cs:14-58`).
* `ConnectionValidator` — SOAP header struct mirroring
  `AsbContracts.cs:65-117`. `from_signed(&SignedValidator)` converts
  F23's MAC + IV to base64 for the wire, matching .NET's
  `BinaryWriter`-via-`XmlSerializer` shape.
* `SoapEnvelope` + `encode_envelope` — assembles the NBFX token
  stream: `s:Envelope` → `s:Header` → `a:Action s:mustUnderstand="1"`
  → optional `h:ConnectionValidator` → `s:Body` → caller-supplied
  body tokens. Uses static-dictionary IDs for the SOAP/WS-Addressing
  tokens via F22's `lookup_static`.
* `decode_envelope` — pulls action + validator + body tokens back
  out of received bytes. Tolerant of header ordering.
* Mixed-endian GUID format/parse (`format_uuid` / `parse_uuid`) that
  mirrors .NET's `Guid.ToString("D")` byte order so connection-id
  round-trip matches the wire exactly.

9 new unit tests cover:
* Round-trip with and without validator.
* `from_signed` base64 encoding of MAC + IV.
* `format_uuid` produces the correct .NET-mixed-endian hex string.
* GUID round-trip through string formatter.
* Action string presence in the encoded byte stream.
* Decoder tolerance of envelopes without an Action header.
* Validator round-trip through full encode → decode.
* Lint-style guard that all 14 action constants are URIs ending `In`.

Stubbed for next F25 iteration: per-operation request/response
struct codecs (`ConnectRequest`, `RegisterItemsRequest`, etc.) +
`AsbClient` network loop.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 11:16:22 -04:00