diff --git a/design/followups.md b/design/followups.md index 713720b..2d09dc5 100644 --- a/design/followups.md +++ b/design/followups.md @@ -141,12 +141,20 @@ F25 (`mxaccess-asb` IASBIDataV2 client) and F26 (`mxaccess::Session` over `AsbTr **Resolves when:** `decode_tokens` (or a post-pass over the token stream) substitutes `NbfxName::Static(id)` with `NbfxName::Inline(name)` whenever the dict id resolves to a known string. The dynamic dict (`read_dictionary`) accumulates session strings via `intern`; the read-path needs the parallel session counter to map wire ids to slots — wire ids are odd and session-cumulative across messages, mirroring the F28 fix on the write side. **Resolves**: F25 live data path (Read/Write/Subscribe responses are all dict-encoded too). -### F31 — Wire-byte server response says `successField = false` for AuthenticateMe-then-Register -**Severity:** P1 — visible failure mode. -**Source:** `relay-rust-decode.log` + `MX_ASB_TRACE_REPLY` dump. -**Why deferred:** With every wire-format fix from F28 landed (canonical XML signing, registry-driven DH params, dynamic-dict id management, ConnectionValidator wire-format-per-action, chunked ASBIData decode, `0x0A ShortDictionaryXmlnsAttribute` decode), `AuthenticateMe` is accepted by the server and a real `RegisterItemsResponse` returns. The response body decodes structurally but the visible `false` element matches the `successField` slot in the response binary header pre-pop, so Register returned success=false with empty Status array. Hypothesis (highest-likelihood first): (a) `AuthenticateMe` reaches the server but its HMAC is silently invalid → server treats subsequent requests as unauthenticated → register returns "no items processed"; (b) wire-byte difference between our Register and .NET's that the server still accepts but interprets as a 0-item registration; (c) tag `TestChildObject.TestInt` doesn't resolve in the live Galaxy state during this run. +### F31 — `AuthenticateMe` HMAC silently invalid on the server (resultCode = `InvalidConnectionId`) +**Severity:** P1 — gates every signed and unsigned operation after Connect. +**Source:** Live capture + F30 dict-id resolution exposing the response `1` (= `AsbErrorCode.InvalidConnectionId` per `AsbResultMapping.cs:6`) plus `false`. -**Resolves when:** F30 lands (so we can read the actual Status array + error codes from the response), AND we confirm against a side-by-side .NET probe wire diff which interpretation applies. If (a), AuthenticateMe HMAC needs further investigation despite the deterministic-HMAC fixture parity (commit `ce27b63`) — a session-state mismatch between client+server view of next-message-number could explain it. If (b), expect a structural delta in the request bytes the server tolerates but interprets differently. If (c), pick a different known-resolvable tag. +**Why this is mysterious:** the entire crypto stack is proven byte-equal to .NET (commit `ce27b63` deterministic HMAC fixture covers DH, crypto_key, HMAC-SHA1, PBKDF2-SHA1, AES-CBC PKCS7), the canonical XML emitter is fixture-validated against `request.ToXml()` (commit `f14580e`), the registry DH params are honoured (commit `f14580e`), and the wire-level `` now carries the same four xmlns declarations .NET emits (`xmlns:h`, default `xmlns`, `xmlns:xsi`, `xmlns:xsd` all in this commit). Yet the server reports `InvalidConnectionId` on Register, indicating that AuthenticateMe's HMAC failed to verify and the server discarded the connection state. + +**Investigation done:** side-by-side `MX_ASB_TRACE_DERIVE` confirms passphrase bytes [96..176] of the crypto_key match .NET (commit `fd38189`); shared_secret bytes diverge per session because each peer chooses its own DH random, but the client+server pair derives the same value by construction. + +**Hypotheses still standing:** +- The server's canonical-XML reconstruction uses `new XmlSerializer(type)` without the `"urn:invensys.schemas"` default namespace that the client passes in `AsbSerialization.cs:27` — would produce different bytes, mismatching HMAC. Untestable from outside the server. +- A subtle byte-level wire difference that affects deserialization (e.g. an attribute the server's XmlSerializer requires but XmlBinaryReader normalizes differently). Hard to find without server logs. +- Some other state the server tracks per-connection that we're not setting (e.g. a session token from `ServiceAuthenticationData` we ignore). The `ConnectResponse.ServiceAuthenticationData` is currently parsed but not fed back into anything; .NET's `AsbSystemAuthenticator` may use it for a downstream verification we're missing. + +**Resolves when:** Either (a) the server is instrumented (`IncludeExceptionDetailInFaults` on the WCF service config, or a TraceListener on `System.ServiceModel.MessageLogging`) to surface the actual deserialization / HMAC mismatch reason; or (b) we capture .NET probe HMAC bytes alongside Rust HMAC bytes for a controlled scenario (fixed DH private key on both ends) and identify the byte-level divergence. ### F28 — Canonical XML serialiser for `ConnectedRequest` signing (matches `XmlSerializer.Serialize` byte-for-byte) **Severity:** P0 — blocks every signed ASB operation (AuthenticateMe, RegisterItems, all data-plane RPCs). diff --git a/rust/crates/mxaccess-asb/src/envelope.rs b/rust/crates/mxaccess-asb/src/envelope.rs index 252f3ae..9211916 100644 --- a/rust/crates/mxaccess-asb/src/envelope.rs +++ b/rust/crates/mxaccess-asb/src/envelope.rs @@ -589,6 +589,19 @@ pub fn decode_envelope( } } + // F30: resolve dict-encoded element/attribute names + namespace + // values in the body. WCF's response side encodes most field names + // as `NbfxName::Static(id)` against the response's binary header + // pre-pop (e.g. `false` is `false` + // for `RegisterItemsResponse.Result`). Without this pass, + // downstream consumers like `collect_asbidata_payloads` and + // `find_element_named` only match `NbfxName::Inline(local)` and + // miss every dict-named element. We resolve via the per-message + // binary header strings (per-message-relative ids) first; if the + // slot is out of range we fall back to the cumulative dynamic + // dict and finally to the static `[MC-NBFS]` table. + resolve_dict_names_in_tokens(&mut body_tokens, header.as_ref(), dynamic); + Ok(DecodedEnvelope { action, validator, @@ -596,6 +609,80 @@ pub fn decode_envelope( }) } +/// Walk the token stream and substitute `NbfxName::Static(id)` / +/// `NbfxText::DictionaryStatic(id)` with their resolved string forms +/// where possible. Leaves unresolved ids in their original form so +/// callers can still see them in trace output. +fn resolve_dict_names_in_tokens( + tokens: &mut [NbfxToken], + header: Option<&ParsedBinaryHeader>, + dynamic: &DynamicDictionary, +) { + for tok in tokens.iter_mut() { + match tok { + NbfxToken::Element { name, .. } => resolve_name_in_place(name, header, dynamic), + NbfxToken::Attribute { name, value, .. } => { + resolve_name_in_place(name, header, dynamic); + resolve_text_in_place(value, header, dynamic); + } + NbfxToken::Text(value) + | NbfxToken::DefaultNamespace { value } + | NbfxToken::NamespaceDeclaration { value, .. } => { + resolve_text_in_place(value, header, dynamic); + } + NbfxToken::EndElement => {} + } + } +} + +fn resolve_name_in_place( + name: &mut NbfxName, + header: Option<&ParsedBinaryHeader>, + dynamic: &DynamicDictionary, +) { + if let NbfxName::Static(id) = name { + if let Some(s) = resolve_dict_id(*id, header, dynamic) { + *name = NbfxName::Inline(s); + } + } +} + +fn resolve_text_in_place( + text: &mut NbfxText, + header: Option<&ParsedBinaryHeader>, + dynamic: &DynamicDictionary, +) { + if let NbfxText::DictionaryStatic(id) = text { + if let Some(s) = resolve_dict_id(*id, header, dynamic) { + *text = NbfxText::Chars(s); + } + } +} + +/// Resolve a wire dict id to a string. Even ids hit the `[MC-NBFS]` +/// static table; odd ids hit the per-session dynamic dict (with +/// fallback to the per-message binary header for the rare cases +/// where the dynamic dict hasn't yet caught up). +fn resolve_dict_id( + id: u32, + header: Option<&ParsedBinaryHeader>, + dynamic: &DynamicDictionary, +) -> Option { + if id % 2 == 0 { + return mxaccess_asb_nettcp::nbfs::lookup_static(id).map(String::from); + } + let slot = ((id - 1) / 2) as usize; + if let Some(h) = header { + if let Some(s) = h.strings.get(slot) { + return Some(s.clone()); + } + } + if let Some(s) = dynamic.lookup(slot as u32) { + return Some(s.to_string()); + } + None +} + // ---- helpers ------------------------------------------------------------- fn encode_validator( @@ -650,18 +737,38 @@ fn encode_validator( } ValidatorWireFormat::XmlSerializer => { // XmlSerializer form: public property names (PascalCase) in - // the data namespace from `[XmlType]` on the class. No - // `xmlns:i` declaration here — XmlSerializer doesn't emit - // `xsi:nil` for empty byte[] (it uses self-closing - // elements instead, but byte[] in the wire format actually - // shows a Chars text record holding the base64-encoded - // empty content per the .NET probe capture; see - // push_xml_byte_array_field). Captured via - // `MxAsbClient.Probe --via …` for AuthenticateMe. + // the data namespace from `[XmlType]` on the class. + // + // Captured against `.NET probe --via` through `asb-relay`: + // the wire `` carries FOUR xmlns + // declarations in this exact order: + // 1. xmlns:h="http://asb.contracts.headers/20111111" + // 2. xmlns="http://asb.contracts.headers/20111111" (same value, default) + // 3. xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + // 4. xmlns:xsd="http://www.w3.org/2001/XMLSchema" + // .NET emits the default xmlns redundantly even though the + // h prefix is bound to the same URL — without it the + // server's XmlSerializer deserialises the validator into + // a default-constructed `ConnectionValidator` with + // ConnectionId = Guid.Empty, leading to InvalidConnection + // Id (= 1) on every subsequent op. Confirmed by reading + // `1` in the live + // RegisterItemsResponse. out.push(NbfxToken::NamespaceDeclaration { prefix: "h".to_string(), value: NbfxText::Chars(asb_ns::HEADERS.to_string()), }); + out.push(NbfxToken::DefaultNamespace { + value: NbfxText::Chars(asb_ns::HEADERS.to_string()), + }); + out.push(NbfxToken::NamespaceDeclaration { + prefix: "xsi".to_string(), + value: NbfxText::Chars("http://www.w3.org/2001/XMLSchema-instance".to_string()), + }); + out.push(NbfxToken::NamespaceDeclaration { + prefix: "xsd".to_string(), + value: NbfxText::Chars("http://www.w3.org/2001/XMLSchema".to_string()), + }); let data_ns = "http://asb.contracts.data/20111111"; push_xml_text_field(out, "ConnectionId", data_ns, &format_uuid(&v.connection_id)); push_xml_text_field(