104efc4e9baba5caa8ac1fb73f19bbc832385180
19 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Joseph Doherty
|
104efc4e9b |
[M5] mxaccess-asb: F28 wire-format fixes — AuthenticateMe accepted live
Three wire-level bugs surfaced by side-by-side relay capture against
the .NET probe routed via the new --via flag:
1. **Dynamic-dictionary id drift**. Our `encode_envelope` hardcoded
action_dict_id=1 / to_dict_id=3, which is correct for the FIRST
message in a session but wrong for every subsequent one. The
per-session dynamic dict accumulates across messages: Connect's
binary header pre-pops [action,to] at ids 1,3; AuthenticateMe must
reference the new action at id 5 (continuing the odd sequence) and
the To URL at id 3 (still in the dict from Connect). Fix uses
`DynamicDictionary::position_of` + `intern` to compute the right
wire id, only pre-popping strings that are NEW to the session.
Captured against .NET probe via asb-relay: AuthenticateMe binary
header has only one string (action) at offset 0x260 (`06 de 08 2f
2e ...`), and `<a:Action>` value `ab 05` references the new id 5.
2. **ConnectionValidator wire format depends on operation**. .NET's
`IAsbDataV2` declares `[XmlSerializerFormat]` on AuthenticateMe,
Disconnect, KeepAlive (one-way ops) — those use XmlSerializer for
the ENTIRE message including the [MessageHeader] ConnectionValid-
ator. Other ops use the default DataContractSerializer. The wire
shapes differ:
XmlSerializer: `<ConnectionId xmlns="http://asb.contracts.data/
20111111">guid</ConnectionId>` (PascalCase property name in
data namespace)
DataContract: `<connectionIdField xmlns="http://schemas.data
contract.org/2004/07/ArchestrAServices.ASBContract">guid</…>`
(private "fooField" name in datacontract namespace)
New `ValidatorWireFormat::for_action` picks the right shape per
action; `encode_validator` now branches on it. New helpers
`push_xml_text_field` / `push_xml_byte_array_field` for the
XmlSerializer form. The DataContract form is preserved verbatim
for Register/Read/Write/etc.
3. **Decoder missing 0x0A** (`ShortDictionaryXmlnsAttribute`). The
server's RegisterItemsResponse uses `0x0A {dict-id}` to set the
default namespace from the static dict; our decoder bailed out
with `UnknownRecord(10)`. New decode arm produces a
`DefaultNamespace` token with `DictionaryStatic` value.
**.NET probe gains a `--via` flag** (`AsbConnectionOptions.Via` →
`ChannelFactory.CreateChannel(addr, viaUri)`) so the probe can be
routed through asb-relay for byte-level capture without triggering
an `AddressFilterMismatch` fault. CoreWCF / .NET 10 dropped
`ClientViaBehavior`; the `CreateChannel(addr, via)` overload is the
modern equivalent.
Live status (this commit): Connect handshake works, AuthenticateMe
no longer faults (canonical XML + crypto + wire-format all match
.NET now), RegisterItemsResponse comes back from the server (a real
response, not a dispatcher fault). One remaining issue: our response
decoder hits `MissingField { field: "Status" }` — the server's
RegisterItemsResponse uses a slightly different element naming or
encoding than `collect_asbidata_payloads` expects. Next iteration
hunts that.
Workspace: 710 unit tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
f14580e0db |
[M5] mxaccess-asb: F28 canonical-XML signing wired + registry-driven DH params
Adds `xml_canonical` module that emits XmlSerializer-compatible canonical XML for the five primary `ConnectedRequest` shapes (AuthenticateMe, Disconnect, KeepAlive, RegisterItemsRequest, UnregisterItemsRequest). Six fixture-comparison tests verify byte-exact match against captured .NET output, including the empty-MAC-IV variant that the live signing flow uses (`authenticate-me-empty-mac-iv.xml`, 896 bytes; new `emit_data_ns_byte_array` helper picks self-closing form for empty byte[]). Plumbing: `AsbAuthenticator::peek_next_message_number` exposes the pre-allocated message number; `AsbClient::send_signed_envelope[_one_way]` gain an `xml_for_signing: Option<&[u8]>` parameter. `connect`, `disconnect`, `keep_alive`, `register_items`, `unregister_items` now build a pre-signing `ConnectionValidator` (empty MAC + IV) + emit the canonical XML + pass the bytes through to HMAC. Other ops (Read, Write, Subscription) keep the legacy NBFX-bytes path until F28 expands to cover their request shapes. Live-bring-up wiring: - `tools/Get-AsbPassphrase.ps1` now exports `MX_ASB_DH_PRIME`, `MX_ASB_DH_GENERATOR`, `MX_ASB_DH_HASH_ALGORITHM` (always — even when empty, so the example can distinguish "no env var" from "registry says empty"), and `MX_ASB_DH_KEY_SIZE`. - `examples/asb-subscribe.rs` honours those env vars to override `CryptoParameters::defaults()`. Each AVEVA install picks its own DH group at provisioning time (768-bit prime is typical, vs the .NET reference's 1024-bit fallback that we previously hardcoded). Empty hashAlgorithm in the registry maps to `HashAlgorithm::Unrecognised`, matching `AsbSystemAuthenticator.CreateHmac:84-93` semantics where empty + forceHmac=true → HMAC-SHA1. - `MxAsbClient.Probe --dump-signed-xml` flag (added in earlier commit) now traces the live HMAC inputs (`asb.sign.xml-utf8-len`, `asb.sign.xml-b64`, `asb.sign.hmac-b64`, etc.) so the Rust port can diff its canonical XML against .NET's byte-for-byte for any live scenario (env-driven via `Action<string>? sharedTrace`). Wire-format alignment for `XmlSerializer` parity: - `ItemIdentity::default()` and `absolute_by_name` now use `Some(String::new())` for null-able strings (matches .NET's `CreateAbsoluteItem` setting `ContextName = string.Empty` not null). - `read_unicode_string` returns `Some(String::new())` for length-0 rather than `None` — mirrors .NET's `AsbBinary.ReadUnicodeString: return string.Empty for byteLength == 0`. Wire format genuinely cannot distinguish null from empty (both encode as 4 bytes of zero); callers that need to preserve the distinction MUST track it in their domain types before encoding. Live status (post-fix): Connect handshake completes end-to-end. The canonical XML our emitter produces matches .NET's structure byte-for- byte (verified by fixture comparison). DH prime/generator/hash now match the live registry values. Despite all this, AuthenticateMe still produces a generic dispatcher fault on the server — there's at least one more subtle wire-byte or crypto mismatch that needs isolation. F28 stays open with that note. Workspace: 709 unit tests pass (was 702 + 7 new xml_canonical tests). Clippy: clean (`-D warnings`). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
dbb580b2c8 |
[M5] tools+fixtures: F28 canonical-XML signing target captured from .NET
Adds `MxAsbClient.Probe --dump-signed-xml` flag that builds five ConnectedRequest shapes (AuthenticateMe, Disconnect, KeepAlive, RegisterItemsRequest, UnregisterItemsRequest) with deterministic field values and prints `AsbSerialization.ToXml(...)` output. The output is exactly what `AsbSystemAuthenticator.Sign` HMACs (`AsbSystemAuthenticator.cs:79`), so the Rust port's canonical-XML emitter must produce byte-identical bytes for HMAC parity. Captured fixtures land under `rust/crates/mxaccess-asb/tests/fixtures/signed-xml/`: - `authenticate-me.xml` — 1000 bytes - `disconnect.xml` — 980 bytes - `keep-alive.xml` — 705 bytes - `register-items.xml` — 1068 bytes - `unregister-items.xml` — 1072 bytes Plus a `README.md` documenting 10 inferred XmlSerializer rules (element name = class name not WrapperName, field order = declaration order not [MessageBodyMember.Order], `[XmlType.Namespace]` on field type causes per-child xmlns redeclaration on the children not the wrapper, `*Specified` pattern controls Xxx emission, CRLF + 2-space indent + utf-16 declaration but UTF-8 bytes fed to HMAC). `.gitattributes` marks the XML fixtures as binary (`*.xml -text`) so neither `core.autocrlf` nor `text` filters can rewrite the byte content — CRLF is part of the canonical form and must survive round-trip through Git untouched. `MxAsbClient.csproj` gains `<InternalsVisibleTo Include="MxAsbClient .Probe" />` so the probe can reach the internal `AsbSerialization` helper without making it public. Workspace: 702 tests pass (no Rust changes — fixtures only). F28 follow-up updated with the captured fixtures + the inferred rules. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
d1e887b91b |
[M5] mxaccess-asb-nettcp/asb: Connect handshake live + SOAP fault detection
Live-bring-up reconciliation against AVEVA's MxDataProvider on Windows.
Connect now completes end-to-end (real DH key exchange, apollo:V2
encryption, ServicePublicKey/ServiceAuthenticationData populated). Five
fixes land:
1. NBFX `PrefixElement_a..z` (0x5E-0x77) and `PrefixAttribute_a..z`
(0x26-0x3F) decode + encode arms. The server's ConnectResponse hit
`0x65 = PrefixElement_h` for a dynamically-named element and our
decoder bailed with `unknown NBFX record byte 0x65`. Both directions
now round-trip; encoder picks short-form when prefix is a single
lowercase ASCII letter.
2. xmlns redeclaration on `<Data>` AND `<InitializationVector>` inside
`AuthenticationData` / `PublicKey`. `[XmlType(Namespace = ...)]` on
AuthenticationData / PublicKey (`AsbContracts.cs:350-381`) means
XmlSerializer emits `xmlns="..."` on each direct child. The default-
ns scope ends at `</Data>`, so `<InitializationVector>` needs its own
redeclaration to stay in the data namespace; without it the server
fell back to messages-namespace and the deserialiser threw an
`InternalServiceFault`.
3. SOAP-fault detection in `AsbClient::send_envelope`. New
`ClientError::SoapFault { action, code, reason }` surfaces when the
response Action header matches the canonical `dispatcher/fault`
template; previously body decoders blindly ran and surfaced
`MissingField { field: "Status" }` masking the actual fault. Reason
text is extracted as the longest `NbfxText::Chars` in the body —
robust against the `nbfs.rs` static-dictionary id mismatches.
4. Identified blocker (filed as F28): signed-request HMAC currently
covers the NBFX wire bytes, but .NET's `AsbSystemAuthenticator.Sign`
HMACs `Encoding.UTF8.GetBytes(request.ToXml())` — the canonical XML
serialisation via `XmlSerializer` with namespace
`urn:invensys.schemas` (`AsbSerialization.cs:12-48`). Until the Rust
port emits identical XML bytes for `ConnectedRequest` subclasses,
AuthenticateMe / RegisterItems / every signed RPC fault on the
server. Connect itself is unsigned (`ServiceMessage` not
`ConnectedRequest`) which is why it works today.
5. Identified `nbfs.rs` static-dictionary id drift (filed as F29): wire
uses Fault=134/Code=142/Reason=144/Text=146/Value=154/Subcode=156
but our table has them at 114/122/124/126/134/136. Off by 20 from
id 114+ — 10 missing entries between `s` (id 112) and `Fault`. No
request-side impact (we only encode IDs ≤44, all correct); the SOAP
fault decode walks text records directly so it sidesteps the issue.
Workspace: 702 tests pass (no test count delta — wire-only fixes).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
4c4177050c |
[M5] mxaccess-asb-nettcp/asb: xmlns raw-string + xsi/xsd on Body
WIRE-FORMAT BREAKTHROUGH — our envelope is now valid NBFX/WCF. ConnectRequest reaches the server's operation handler. Direct-port-808 Connect now returns a server-side fault (operation invocation error from the placeholder DH key) rather than TCP RST. With a real DH key, asb-subscribe gets all the way to "response is missing required field ServicePublicKey/Data" — meaning our Connect request processed end-to-end, the server returned a ConnectResponse, and the only remaining issue is in our response-body decoder. Fixes: 1. **`XmlnsAttribute` (0x09) value is a RAW length-prefixed string, not a text record**. Per `[MC-NBFX]` §2.2.3, xmlns attribute values are `[length][bytes]`, NOT `[text-record-byte][value]`. Our F21 was emitting `aa <id>` for dict-static values which the receiver misparsed as a 0xAA-length string. Same fix applies to `ShortXmlnsAttribute` (0x08). Encoder now picks raw-string for `Chars` value, raw-int31 (via 0x0B) for `DictionaryStatic` value; decoder reads raw string in both code paths. 2. **xmlns:xsi + xmlns:xsd on `<s:Body>`**. WCF declares these namespaces on Body before opening the operation request element. Our envelope encoder now emits both as raw-string xmlns attrs right after `<s:Body>` opens. Required for `xsi:type` annotations that appear inside DataContract-serialised body fields. Combined wire-byte impact (verified via asb-relay side-by-side diff): * All header bytes match .NET byte-for-byte through the entire `<s:Header>` section (Action / ConnectionValidator / MessageID via UniqueIdText / ReplyTo / To). * `<s:Body>` xmlns:xsi + xmlns:xsd declarations match .NET. * `<ConnectRequest>` opens identically. * `<ConnectionId>` / `<ConsumerPublicKey>` / `<Data>` element names match. * The only known remaining diff in the request: .NET emits `xmlns="http://asb.contracts.data/20111111"` on the inner `<Data>` element (the PublicKey class's XmlType namespace); we don't. Likely an issue but apparently non-fatal — the server processed our request successfully past this point. Live status: * Direct port-808 connect with real DH key: server returns "response is missing required field ServicePublicKey/Data" — meaning we sent a valid Connect, server replied with a ConnectResponse, but our decoder can't find the field. Next iteration is response-side decode work. Workspace: 702 tests pass; clippy + fmt clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
c2222b16b0 |
[M5] mxaccess-asb-nettcp/asb: F21 short forms + EndElement fix + UniqueIdText
Three NBFX-spec corrections discovered by diffing our wire output against the .NET probe's capture: 1. **EndElement is 0x01, NOT 0x00**. Our F21 had this wrong since the first iteration. Our round-trip tests passed because encode and decode used the same wrong value, but interop with WCF's parser silently failed (TCP RST on every request). Fixed by changing `REC_END_ELEMENT` to 0x01 — all 702 tests pass on the new value. 2. **Single-letter prefix short forms**. WCF uses `PrefixDictionaryElement_<a-z>` (records 0x44-0x5D) and `PrefixDictionaryAttribute_<a-z>` (records 0x0C-0x25) for single-character prefixes. Our F21 always used the long forms (0x43 prefix-string + dict-id, etc.). The encoder now emits the short form when the prefix is a single ASCII lowercase letter; the decoder accepts both. New `prefix_letter_offset(prefix)` helper. 3. **`DictionaryXmlnsAttribute` (0x0B)** for xmlns:prefix declarations whose value is a static-dict id. The long form (0x09 + prefix-string + text-record) is still emitted when the value is an inline string, but for `xmlns:s="...soap-envelope"` (dict id 4) we now emit the short `0b 01 73 04` form WCF uses. 4. **UniqueIdText (0xAC)** added to `NbfxText` enum + encode/decode. WCF emits `<a:MessageID>` as a UniqueIdText carrying the 16 raw UUID bytes (NOT the `urn:uuid:...` text form). Updated `encode_envelope` to use this for MessageID. Combined wire-byte impact: our envelope body section now matches the .NET probe byte-for-byte through `<a:Action>`, `<h:ConnectionValidator>`, `<a:MessageID>` (UniqueId), `<a:ReplyTo>`, `<a:To>`, and `<s:Body>`. The trailing `01 01 01 01` = 4 EndElements is now the correct record byte. Tests pass (702 total). Live status: still TCP RST after the SizedEnvelope. Remaining unknown is in the body section — the .NET capture shows xmlns:xsi / xmlns:xsd declarations on the operation-specific request element (ConnectRequest etc.) that we don't emit, plus possibly different field encoding inside ConnectRequest. Next iteration will re-capture through the relay and diff our body bytes against the new .NET-byte-equivalent we now produce. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
2867310817 |
[M5] mxaccess-asb: WCF binary message header (action+to dict pre-pop)
Adds the binary header block that WCF prepends to SizedEnvelope
payloads. Reverse-engineered from .NET probe wire bytes captured via
asb-relay.
Wire form (per the .NET capture analysis in the previous commit):
```
[outer length, multibyte-int31]
[string-1 length, multibyte-int31] [UTF-8 bytes] ← dict id 1 (action)
[string-2 length, multibyte-int31] [UTF-8 bytes] ← dict id 3 (to)
[NBFX <s:Envelope>...]
```
Inside the NBFX envelope, `<a:Action>` and `<a:To>` reference the
pre-pop strings via `DictionaryText 0xAA {odd-id}` instead of inlining
their values. The header strings get assigned odd dict ids
(1, 3, 5, ...); even ids stay reserved for the [MC-NBFS] static dict.
Encode side:
* `encode_envelope` now emits header [action, to] before NBFX. `to_uri`
defaults to empty string when None — caller-supplied `with_to(uri)`
is the supported path.
* AsbClient's `send_envelope` and `send_envelope_one_way` auto-fill
`to_uri` from `self.via_uri` when not set.
* New private `encode_binary_header(strings)` helper.
Decode side:
* New `parse_binary_header_prefix(input)` heuristically detects + parses
the header (look for plausible NBFX element record byte 0x40-0x77 at
the offset implied by the outer length).
* New `resolve_with_header(text, dynamic, header)` resolves
`DictionaryText` with odd id by indexing into header.strings; even
ids fall through to static-dict lookup as before.
Tests pass (72) — round-trip envelope → bytes → envelope recovers
action through the new dict-id resolution path.
Live status: this commit gets us further but the connect SOAP
envelope still TCP-RSTs at SMSvcHost. The remaining delta vs the .NET
capture is structural NBFX optimisation: .NET uses single-letter
prefix-element/attribute records (0x44-0x77 PrefixDictionaryElement
_<a-z>, 0x0C-0x25 PrefixDictionaryAttribute_<a-z>, 0x0B
DictionaryXmlnsAttribute) while our F21 encoder always uses the long
forms (0x43 prefix-string + name-dict-id, etc.). Logically
equivalent but WCF's parser likely strict on which form it accepts.
Next iteration will add short-form encoding to F21 for single-letter
prefixes (s:, a:, h:, i:) which covers every namespace prefix in our
envelope.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
3b09297b27 |
[M5] live-probe iteration 1 — major wire-byte reconciliation fixes
First live-test cycle against AVEVA on this box. Comparing the .NET
probe's `--dump-messages` XML output against our NBFX-encoded
envelope surfaced six structural bugs in the F25 envelope/operations
layer. All fixed; tests passing (702 workspace).
Fixes (all backed by the .NET dump as ground truth):
1. **`mustUnderstand` attribute name** — NBFS dict id was 116
(`MustUnderstand`, capital-M, a different SOAP token); SOAP 1.2
spec uses lowercase `mustUnderstand` at id 0. Sending the wrong
one triggered a WCF parse fault that surfaced as TCP RST.
2. **Missing `<a:MessageID>` header** — WCF's default binding
requires MessageID for two-way operations. We now auto-generate
`urn:uuid:<v4>` per envelope via a small inline `make_random_uuid_v4`
helper (no `uuid` crate dep).
3. **Missing `<a:ReplyTo>` anonymous header** — WCF's
BinaryMessageEncoder always emits `<a:ReplyTo><a:Address>...
addressing/anonymous</a:Address></a:ReplyTo>` for two-way ops.
4. **ConnectionValidator field names + namespace** — we were
emitting PascalCase `<ConnectionId>` etc. .NET's WCF
DataContractSerializer uses the private backing-field names
(`<connectionIdField xmlns="...ASBContract">guid</connectionIdField>`)
per `[DataMember(Name = "fooField")]`. Added the
`xmlns:i="...XMLSchema-instance"` declaration WCF emits
alongside (even when no `i:nil` is used). Decoder now accepts
both PascalCase (legacy tests) and DataContract field names.
5. **`<ASBIData>` over-wrapping** — we were emitting
`<Items><ASBIData>{bytes}</ASBIData></Items>`. .NET's
`AsbDataCustomSerializer.WriteStartObject` (`AsbContracts.cs:
1561-1572`) REPLACES the field's outer element with `<ASBIData>`
directly — there's no `<Items>` wrapper on the wire. Fixed by
collapsing `BodyField::AsbiDataElement` to emit just `<ASBIData>`
without the named outer element. The `name` field is retained
for self-documentation.
6. **`collect_asbidata_payloads` API** — was keyed by field name
(`Status` / `Values`); now positional (`payloads[0]`,
`payloads.get(1)`) since the wrapper element is gone. All seven
response decoders updated.
Plus tooling for the live-probe loop:
* `tools/Get-AsbPassphrase.ps1` — DPAPI loader that auto-discovers
the solution name + reads the sharedsecret + decrypts it. Sets
$env:MX_ASB_PASSPHRASE / MX_ASB_HOST / MX_ASB_VIA / MX_LIVE.
Lowercase via-host (WCF SMSvcHost is case-sensitive on the URL
host segment).
* `examples/asb-preamble-probe.rs` — diagnostic that connects,
runs the preamble, captures the PreambleAck, then sends a
synthetic ConnectRequest and dumps both directions as hex. Used
to bisect the wire-byte deltas above.
* `examples/asb-subscribe.rs` port default fixed (5074 → 808 —
WCF's NetTcpPortSharing/SMSvcHost listener confirmed via
Get-NetTCPConnection).
**Status**: preamble + PreambleAck round-trip works end-to-end
against the live AVEVA install (verified via probe). The
post-preamble Connect SOAP envelope still gets TCP RST'd — the six
structural fixes above are necessary but not yet sufficient. Next
iteration needs binary wire capture (Wireshark + Npcap loopback,
or a TCP-relay middleman) to compare the .NET probe's BinaryMessageEncoder
output byte-for-byte with ours and find the remaining delta(s).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
9876b4ebb4 |
[M5] mxaccess-asb: F25 step 10 — PublishWriteComplete + DeleteMonitoredItems
Closes out the F25 operation matrix. AsbClient now wraps every
IASBIDataV2 operation:
Lifecycle: connect / disconnect / send_end / send_preamble / keep_alive
Items: register_items / unregister_items / read / write
Subscriptions:create_subscription / add_monitored_items / publish
/ delete_monitored_items / delete_subscription
Write cb: publish_write_complete
API additions:
* `build_publish_write_complete_request_body()` — empty wrapper
per `AsbContracts.cs:204-205`. No body fields beyond inherited
ConnectionValidator.
* `decode_publish_write_complete_response` — returns count of
`<ItemWriteComplete>` elements observed. Per-element decode
(Status + WriteHandle) deferred to a later iteration since
ItemWriteComplete is regular WCF DataContract rather than the
binary fast-path.
* `build_delete_monitored_items_request_body` — same MonitoredItem
shape as AddMonitoredItems but omits RequireId per `cs:268-277`.
* `decode_delete_monitored_items_response` — per-item Status array.
* Client wrappers: `publish_write_complete()`,
`delete_monitored_items(subscription_id, items)`.
6 new tests:
* `publish_write_complete_body_is_empty_wrapper` — body shape.
* `publish_write_complete_response_counts_item_write_complete_elements`
— counts 2 / 0 elements correctly.
* `publish_write_complete_response_zero_when_no_callbacks`.
* `delete_monitored_items_body_carries_subscription_id_and_items`.
* `delete_monitored_items_body_omits_require_id_field`.
* `delete_monitored_items_response_round_trip`.
Workspace: 701 tests pass (was 695, +6).
Stubbed for future iterations:
* ItemWriteComplete per-element decode (Status + WriteHandle) once
a live capture confirms the WCF DataContract XML wire form.
* Optional MonitoredItem fields (Active / TimeDeadband /
ValueDeadband / UserData) — same wire-byte uncertainty.
* Optional WriteValue fields (Comment / Timestamp / etc.).
All wire-byte caveats trace back to live-probe reconciliation
against an actual AVEVA VM.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
0441a2e693 |
[M5] mxaccess-asb: F25 step 9 — Write operation
Closes the highest-value remaining IASBIDataV2 op. With Write landed,
the read+write+subscribe path is functionally complete in-memory.
API additions:
* `MinimalWriteValue { value: AsbVariant }` — carries just the Value
payload. Optional ArrayElementIndex / Comment / HasQT / Status /
Timestamp fields are deferred to a later iteration once a live
capture confirms the WCF DataContract XML form.
* `build_write_request_body(items, values, write_handle)` per
`AsbContracts.cs:181-194`:
```xml
<WriteBasicRequest xmlns="urn:msg.data.asb.iom:2">
<Items><ASBIData>{ItemIdentity[] binary}</ASBIData></Items>
<Values>
<WriteValue><Value><ASBIData>{Variant binary}</ASBIData></Value></WriteValue>
...
</Values>
<WriteHandle>{i32}</WriteHandle>
</WriteBasicRequest>
```
Items array uses the IAsbCustomSerializableType binary fast-path;
each Value's inner Variant also uses the fast-path. WriteHandle is
an Int32 (opaque correlation echoed in PublishWriteComplete).
* `decode_write_response` — per-item Status array (mirrors the
unregister/register pattern).
* `AsbClient::write(items, values, write_handle)` — thin wrapper.
4 new tests:
* `write_request_body_carries_items_values_and_write_handle` — body
shape sanity (WriteHandle = 7 Int32, WriteValue element present).
* `write_request_body_pairs_items_and_values_arrays` — 2 items + 2
values produces 2 WriteValue elements.
* `write_response_round_trips_status_array` — Status decode.
* `write_response_missing_status_fails` — graceful MissingField
error.
Workspace: 695 tests pass (was 691, +4).
Stubbed for next F25 iterations:
* `PublishWriteComplete` — empty request, `ItemWriteComplete[]`
response.
* `DeleteMonitoredItems` — mirrors AddMonitoredItems pattern.
* Optional WriteValue fields (Comment / Timestamp / etc.) once a
live capture confirms the wire-byte layout.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
b543eb1f84 |
[M5] mxaccess-asb: F25 step 8 — subscription operations
CreateSubscription / AddMonitoredItems / Publish / DeleteSubscription. Completes the IASBIDataV2 read-and-subscribe path; remaining ops (Write/PublishWriteComplete/DeleteMonitoredItems) are mechanical extensions of the same pattern. Contracts: * `MonitoredItemValue` codec (IAsbCustomSerializableType binary fast-path: ItemIdentity + RuntimeValue + AsbVariant per `AsbContracts.cs:1064-1068`) with array codec (4-byte int32 count + per-element body, mirrors `WriteArrayToStream` at `cs:1095-1103`). Request builders: * `build_create_subscription_request_body(max_queue_size, sample_interval)` — primitive fields per `cs:215-223`. * `build_delete_subscription_request_body(subscription_id)` — primitive field per `cs:232-237`. * `build_publish_request_body(subscription_id)` — primitive field per `cs:287-292`. * `build_add_monitored_items_request_body(subscription_id, items, require_id)` — minimal MonitoredItem shape (Item + SampleInterval + Buffered). Full optional-field set (Active/TimeDeadband/ValueDeadband/UserData) deferred to a later iteration once a live capture confirms the WCF DataContract XML wire form. Response decoders: * `decode_create_subscription_response` — single int64 SubscriptionId field. Decoder accepts Int64Text, Int32Text, Zero/One, or numeric-string Chars (covers all WCF binary numeric encodings). * `decode_add_monitored_items_response` — Status array + ItemCapabilities-presence flag (mirrors RegisterItemsResponse). * `decode_publish_response` — Status array + Values (MonitoredItemValue) array. `BodyField::Int64Element` variant added for the primitive SubscriptionId / MaxQueueSize / SampleInterval fields. `uint64` helper casts to i64 (covers proven value range; if ulong > i64::MAX ever appears we'll add UInt64Text to F21's NbfxText enum). Client wrappers (4 new methods on AsbClient): * `create_subscription(max_queue_size, sample_interval)` * `add_monitored_items(subscription_id, items, require_id)` * `publish(subscription_id)` * `delete_subscription(subscription_id)` 11 new tests cover: * MonitoredItemValue round-trip + array round-trip. * CreateSubscription request body shape (Int64 payloads). * CreateSubscription response decoder via Int64Text. * CreateSubscription response decoder via Chars text fallback. * CreateSubscription response missing-field error. * AddMonitoredItems body carries SubscriptionId + MonitoredItem elements. * AddMonitoredItems response Status round-trip. * DeleteSubscription body carries SubscriptionId. * Publish request body shape. * Publish response Status + Values round-trip. Workspace: 691 tests pass (was 680, +11). The asb-subscribe example can now do create_subscription → add_monitored_items → publish-loop → delete_subscription once wire-byte reconciliation against a live capture confirms the MonitoredItem XML shape. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
1b1ee1e0b7 |
[M5] mxaccess-asb: F25 step 7 — Disconnect closes the session lifecycle
Mirrors `AsbContracts.cs:109-114` — same payload shape as AuthenticateMe (Data + InitializationVector under ConsumerAuthenticationData) but under the `<DisconnectRequest>` wrapper. Sent one-way + signed (regular HMAC, no force) per `AsbContracts.cs:22` (`IsOneWay = true`). API additions: * `build_disconnect_request_body(data, iv)` — NBFX token stream for the DisconnectRequest body. * `AsbClient::disconnect()` — builds a fresh encrypted authentication-data blob via F23's `create_authentication_data()` (encrypts `local_pub || remote_pub` under the derived AES key with a fresh IV), wraps it in a DisconnectRequest, sends one-way signed. 2 new tests: * `disconnect_request_carries_data_and_iv_under_correct_wrapper` — outer element name + Data/IV byte-payload order. * `disconnect_writes_signed_one_way_envelope` — end-to-end via `tokio::io::duplex` peer; verifies the SizedEnvelope payload contains the `:disconnectIn` action string. With Disconnect landed, AsbClient now covers the full session lifecycle: send_preamble → connect → register_items / read / keep_alive / unregister_items → disconnect → send_end → stream shutdown Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
321b7963a4 |
[M5] mxaccess-asb: F25 step 6 — Connect/AuthenticateMe handshake
Critical-path piece that turns a fresh TCP stream into an
authenticated session. With this slice landed, an `AsbClient` can
now do `send_preamble().await? -> connect().await? -> register_items()`
end-to-end against a peer.
Operations API additions:
* `build_connect_request_body(connection_id, public_key)` — first op
on a fresh session. **Unsigned** (no ConnectionValidator header)
because the authenticator hasn't received the service key yet.
Wire shape: `<ConnectRequest xmlns="…messages/20111111">
<ConnectionId>{guid-text}</ConnectionId>
<ConsumerPublicKey><Data>{pubkey-bytes}</Data></ConsumerPublicKey>
</ConnectRequest>` per `AsbContracts.cs:78-86`.
* `build_authenticate_me_request_body(data, iv)` — second op,
**one-way + signed with `forceHmac=true`** per `MxAsbDataClient.cs
:106-111`. Carries the encrypted `local_pub || remote_pub` blob
produced by F23's `create_authentication_data()`.
* `ConnectResponse { service_public_key, service_authentication_data,
connection_lifetime }` + `AuthenticationDataBytes { data, iv }`.
* `decode_connect_response(body, dict)` — extracts ServicePublicKey
(required), optional ServiceAuthenticationData, optional
ConnectionLifetime. The lifetime's `:V2` suffix is what F23
inspects to toggle Apollo (raw AES) vs Baktun (deflate-then-AES)
encryption.
Client API addition:
* `AsbClient::connect()` — orchestrates the full handshake:
1. Build + send ConnectRequest (unsigned) carrying our DH public
key + connection-id GUID.
2. Decode ConnectResponse.
3. `authenticator.accept_connect_response(...)` — feeds the
service public key + lifetime into F23 so it derives the
shared secret and picks Apollo/Baktun.
4. `authenticator.create_authentication_data()` — encrypts
`local_pub || remote_pub` under the derived AES key.
5. Send AuthenticateMeRequest (one-way, signed with HMAC-SHA1
forced).
Returns the `ConnectResponse` so callers can inspect the
negotiated connection lifetime.
6 new tests:
* ConnectRequest carries hyphenated GUID + raw public-key bytes.
* AuthenticateMe carries Data + IV bytes in order.
* ConnectResponse round-trip with all optional fields populated.
* ConnectResponse round-trip without optional fields.
* ConnectResponse decoder surfaces MissingField when
ServicePublicKey is absent.
* End-to-end client::connect handshake via `tokio::io::duplex`
peer that synthesises a ConnectResponse using bob's public key
(so DH shared-secret derivation actually works) and drains the
AuthenticateMe one-way SizedEnvelope.
Wire-byte caveat documented inline: WCF XML serialization may add
`xsi:type` attributes / distinct namespaces around <PublicKey> /
<AuthenticationData>; this builder ships the simplest plausible
shape and the live-probe iteration will reconcile.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
9b8133f725 |
[M5] mxaccess-asb: F25 step 5 — KeepAlive + Read + one-way client ops
Extends AsbClient with one-way operation support (`IsOneWay = true`
in IASBIDataV2) plus the KeepAlive and Read operations.
Client API additions:
* `send_envelope_one_way(env)` — frames in SizedEnvelope, writes,
returns immediately. No response read. Mirrors WCF's IsOneWay
semantics for KeepAlive / Disconnect / AuthenticateMe.
* `send_signed_envelope_one_way(action, body, force_hmac)` —
one-way variant that runs the body through F23's authenticator
signing path so the ConnectionValidator header is attached.
* `keep_alive()` — sends an empty `KeepAliveRequest` with default
signing. Used to keep the channel alive past the WCF inactivity
timeout (30s default at `MxAsbDataClient.cs:683`).
* `read(items)` — sends a signed Read envelope, decodes
ReadResponse with both Status and Values arrays.
Operations API additions:
* `build_keep_alive_request_body()` — empty wrapper element +
asb.contracts.messages namespace. Mirror of `AsbContracts.cs:117`
(`public sealed class KeepAlive : ConnectedRequest;`).
* `ReadResponse { status: Vec<ItemStatus>, values: Vec<RuntimeValue> }`
per `AsbContracts.cs:169-179`.
* `decode_read_response(body_tokens)` — pulls both ASBIData
payloads, decodes Status as ItemStatus[], decodes Values via
`decode_runtime_value_array` (4-byte int32 count + per-element
`RuntimeValue::decode` from F24).
5 new tests:
* KeepAlive body shape (empty wrapper, correct namespace).
* ReadResponse decoder round-trip with both Status and Values.
* ReadResponse decoder graceful handling when Values is absent
(returns empty vec).
* End-to-end client::keep_alive — peer drains SizedEnvelope but
doesn't respond; client returns Ok().
* End-to-end client::read — peer responds with synthetic
ReadResponse, client recovers Values[0].timestamp_binary == 1234
and Values[0].status round-trip.
Stubbed for next F25 iterations:
* AsbClient::connect — DH Connect + AuthenticateMe handshake. Needs
ConnectRequest / ConnectResponse builders (regular WCF XML, not
the IAsbCustomSerializableType fast-path).
* Write / PublishWriteComplete / CreateSubscription /
AddMonitoredItems / Publish / Disconnect operation wrappers.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
1e59249662 |
[M5] mxaccess-asb: F25 step 4 — AsbClient async network loop
The first slice of F25 that actually moves bytes across a transport. Wraps every M5 framing layer (F19-F25.3) into a single async client generic over `AsyncRead + AsyncWrite + Unpin + Send`. Tested in-memory via `tokio::io::duplex` — no live ASB endpoint required. API: * `AsbClient::new(stream, authenticator, via_uri)` — wraps a Tokio transport + F23 authenticator into a ready client. * `send_preamble()` — writes the canonical preamble (Version 1.0 → Duplex → Via → BinaryWithDictionary → PreambleEnd) and reads the peer's PreambleAck. Surfaces Fault as `ClientError::Fault(msg)`. * `send_envelope(env)` — frames `SoapEnvelope` in a SizedEnvelope NMF record, writes, reads the response SizedEnvelope, decodes back to `DecodedEnvelope`. * `send_signed_envelope(action, body, force_hmac)` — calls F23 authenticator's `sign` on the unsigned body bytes, attaches a ConnectionValidator header (base64'd MAC + IV), sends. * `register_items` / `unregister_items` — thin per-operation wrappers threading body builder + response decoder. * `send_end()` — writes record 0x07 + shutdowns the stream. Async record reader: streaming decode of the multibyte-int31 length prefix for SizedEnvelope (0x06) / Fault (0x08), plus a fallback path for Version / Mode / KnownEncoding / etc. `ClientError` covers I/O, NMF, NBFX, Envelope, Operation, Auth, plus PreambleNotSent / AlreadyClosed / Fault / PeerClosed / UnexpectedRecord guards. 6 new tests via in-memory `tokio::io::duplex`: * Preamble round-trip with synthetic peer returning PreambleAck. * Fault propagation through preamble exchange. * End-to-end RegisterItems request → response with a peer that drains preamble, replies PreambleAck, drains the SizedEnvelope, responds with a synthetic RegisterItemsResponse body containing a binary-encoded ItemStatus array. Client decodes and asserts the recovered ItemIdentity name. * `send_envelope` before preamble fails with PreambleNotSent. * `send_end` writes record 0x07 to the wire. * PreambleMode re-export keeps shape parity with `nmf::NmfMode`. Known limitation: the signing path currently hashes the NBFX-encoded body; .NET hashes the XML-text `request.ToXml()`. Functionally present (validator built and attached) but MAC bytes won't match .NET's MAC for the same payload until the live-probe iteration reconciles which canonical form to sign. Stubbed for next F25 iteration: * `AsbClient::connect` — DH `Connect` + `AuthenticateMe` handshake flow. Needs ConnectRequest/Response builders (regular WCF XML, not the IAsbCustomSerializableType fast-path) and the `AsbAuthenticator::create_authentication_data` integration. * Read / Write / Subscription operation wrappers. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
c4bf0a0a04 |
[M5] mxaccess-asb: F25 step 3 — response decoders + Read request body
Foundation for response decoding. Adds:
* `contracts::ItemStatus` — ports `AsbContracts.cs:639-722`. Wire
layout matches `WriteToStream` exactly: Item (ItemIdentity binary)
→ Status (AsbStatus binary, from F24) → ErrorCode (u16) →
ErrorCodeSpecified (u8 bool). Note this is NOT the DataMember
declaration order — the binary serialiser hand-picks Item-first.
* `encode_item_status_array` / `decode_item_status_array` — same
4-byte int32 count + per-element WriteToStream pattern as the
ItemIdentity array codec.
* `operations::collect_asbidata_payloads(tokens, field_name)` — walks
an NBFX token stream and pulls out `<{field}><ASBIData>{Bytes}
</ASBIData></{field}>` payload bytes. Returns Vec<Vec<u8>> because
some response shapes (ReadResponse) carry multiple ASBIData
payloads (Status + Values).
* `decode_register_items_response` / `decode_unregister_items_response`
— parse SOAP body NBFX tokens into typed RegisterItemsResponse /
UnregisterItemsResponse. The optional ItemCapabilities array (XML-
serialised, not binary) is recorded as a presence flag for now;
decoding the individual ItemRegistration records is a follow-up.
* `build_read_request_body(items)` — simplest unary IASBIDataV2
request, just `<ReadRequest xmlns="..."><Items><ASBIData>...
</ASBIData></Items></ReadRequest>`.
* `OperationError` — typed error for response-decode failures
(`MissingField { field }` and codec wraps).
9 new tests: ItemStatus round-trip (default + with id + with status
payload), ItemStatus array round-trip, RegisterItemsResponse
round-trip via synthetic body, ItemCapabilities presence detection,
UnregisterItemsResponse round-trip, multi-payload extraction (ReadResponse-
shape Status + Values), Read body shape correctness, MissingField
error when Status is absent.
Stubbed for next F25 iteration: Write / PublishWriteComplete /
CreateSubscription / AddMonitoredItems / DeleteMonitoredItems /
Publish builders, ReadResponse + WriteResponse decoders (need
WriteValue / RuntimeValue contract codecs), and the AsbClient
network loop.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
a2b8989cbf |
[M5] mxaccess-asb: F25 step 2 — per-operation request body codecs
Adds the IAsbCustomSerializableType binary fast-path + per-operation
request-body NBFX-token builders. RegisterItems and UnregisterItems
now compose end-to-end through SoapEnvelope + encode_envelope to a
byte stream that round-trips back to the original ItemIdentity array.
Three pieces:
1. F21 NBFX gains `Bytes8/16/32` text records (records 0x9E/0xA0/0xA2
plus +1 WithEndElement variants). WCF's `XmlDictionaryWriter.
WriteBase64` emits these in binary form — not actual base64 text —
so they're required for the `<ASBIData>` content.
2. `mxaccess-asb::contracts::ItemIdentity` ports `AsbContracts.cs:533-633`:
* Wire layout: u16 kind + u16 reference_type +
AsbBinary.WriteUnicodeString(Name) + AsbBinary.WriteUnicodeString
(ContextName) + u64 Id + u8 IdSpecified.
* `AsbBinary.WriteUnicodeString` per cs:1622-1633: u32 byte-length
+ UTF-16LE bytes; null/empty collapse to a 4-byte zero header.
* `encode_item_identity_array` / `decode_item_identity_array`
mirror `WriteArrayToStream` — 4-byte int32 count + each
element's `WriteToStream` output. Per `AsbDataCustomSerializer`
at cs:1583-1591.
* `absolute_by_name(...)` convenience constructor matching
`MxAsbDataClient.CreateAbsoluteItem` at cs:172-194.
3. `mxaccess-asb::operations` builds SOAP body NBFX token streams:
* `build_register_items_request_body(items, require_id, register_only)`
— RegisterItems contract per cs:119-143.
* `build_unregister_items_request_body(items)` — UnregisterItems
per cs:145-159.
* Internal `BodyField` helper assembles the wire shape:
`<RegisterItemsRequest xmlns="urn:msg.data.asb.iom:2">
<Items><ASBIData>{Bytes(payload)}</ASBIData></Items>
<RequireId>true|false</RequireId>
<RegisterOnly>true|false</RegisterOnly>
</RegisterItemsRequest>`
15 new tests cover:
* ItemIdentity round-trip (default, with id, unicode name).
* AsbBinary unicode-string null/empty/value semantics.
* Byte-layout pinning (21 bytes for default ItemIdentity, le-int32
array count).
* ItemIdentity array round-trip.
* `<ASBIData>` Bytes record round-trip across NBFX widths
(Bytes8/16/32 selected by length).
* RegisterItems body → SoapEnvelope → encode → decode → recover the
ItemIdentity array end-to-end.
* RequireId / RegisterOnly Bool wire form.
* UnregisterItems body uses correct outer element name and omits
the RegisterItems-only fields.
Stubbed for next F25 iteration: per-operation Read / Write /
PublishWriteComplete / CreateSubscription / AddMonitoredItems /
DeleteMonitoredItems / Publish builders, response decoders, and the
`AsbClient` network loop.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
25dbd8d3bd |
[M5] mxaccess-asb: F25 step 1 — SOAP envelope codec
First slice of F25. Provides the building blocks the per-operation
request/response codecs and the network loop will compose:
* `actions` module — IASBIDataV2 action strings (all 14 operations,
verbatim from `AsbContracts.cs:14-58`).
* `ConnectionValidator` — SOAP header struct mirroring
`AsbContracts.cs:65-117`. `from_signed(&SignedValidator)` converts
F23's MAC + IV to base64 for the wire, matching .NET's
`BinaryWriter`-via-`XmlSerializer` shape.
* `SoapEnvelope` + `encode_envelope` — assembles the NBFX token
stream: `s:Envelope` → `s:Header` → `a:Action s:mustUnderstand="1"`
→ optional `h:ConnectionValidator` → `s:Body` → caller-supplied
body tokens. Uses static-dictionary IDs for the SOAP/WS-Addressing
tokens via F22's `lookup_static`.
* `decode_envelope` — pulls action + validator + body tokens back
out of received bytes. Tolerant of header ordering.
* Mixed-endian GUID format/parse (`format_uuid` / `parse_uuid`) that
mirrors .NET's `Guid.ToString("D")` byte order so connection-id
round-trip matches the wire exactly.
9 new unit tests cover:
* Round-trip with and without validator.
* `from_signed` base64 encoding of MAC + IV.
* `format_uuid` produces the correct .NET-mixed-endian hex string.
* GUID round-trip through string formatter.
* Action string presence in the encoded byte stream.
* Decoder tolerance of envelopes without an Action header.
* Validator round-trip through full encode → decode.
* Lint-style guard that all 14 action constants are URIs ending `In`.
Stubbed for next F25 iteration: per-operation request/response
struct codecs (`ConnectRequest`, `RegisterItemsRequest`, etc.) +
`AsbClient` network loop.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
fe2a6db786 |
Initial project state: .NET reference, design, Rust port (M0+M1), evidence
rust / build / test / clippy / fmt (push) Has been cancelled
Layout:
- src/ .NET 10 x64 reference: MxNativeCodec, MxNativeClient,
MxAsbClient, probes, tests, harnesses. Executable spec.
- design/ Architectural plan for the Rust port (M0–M6), error
model, protocol invariants, risks (R1–R16), adversarial
review log (review.md).
- rust/ Rust workspace. M0 skeleton + M1 codec parity.
mxaccess-codec: 215 unit tests + 2 cross-implementation
parity tests (byte-identical against .NET reference).
Other crates are M0 stubs awaiting M2+.
- captures/ Frida + netsh + pcap evidence per CLAUDE.md
("captures are evidence, not throwaway logs").
- analysis/ Decompiled C# (frida/proxy/decompiled-*),
Ghidra exports for native DLLs (`exports/` only —
working state at `projects/` and AVEVA's input
binaries at `input/` are gitignored).
- docs/ Reverse-engineering reference docs.
- tools/ Setup-LiveProbeEnv.ps1 (Infisical credential fetcher),
Compute-Crc.ps1 (.NET parity helper).
- .github/workflows/ Rust CI: fmt + build + test + clippy on Windows.
- LICENSE MIT (Joseph Doherty, 2026).
Verified:
- cargo test --workspace → 217 passed (215 unit + 2 .NET parity), 0 failed
- cargo clippy --workspace -- -D warnings → clean
- cargo fmt --all -- --check → clean
- cargo publish --dry-run -p mxaccess-codec → packages cleanly
Excluded from history (see .gitignore):
- **/bin, **/obj, **/target — build artifacts
- analysis/ghidra/projects/ — Ghidra working state (regenerable)
- analysis/ghidra/input/ — AVEVA proprietary DLLs (vendor IP)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|