Joseph Doherty
a31237d1d0
F15 step 2/2 lands the per-Subscription routing on top of step 1's broadcast layer. Subscription is now a working data-change stream. Subscription type - Now impls futures_util::Stream<Item = Result<DataChange, Error>> via tokio_stream::wrappers::BroadcastStream + a per-message filter. - No longer Clone (broadcast::Receiver isn't Clone). Consumers that want fanout subscribe twice or share via Arc<Mutex<...>>. - Holds the broadcast::Receiver subscribed BEFORE AdviseSupervisory fires — guarantees no updates between advise and stream-creation are dropped. - pending VecDeque buffers records from the current message so each poll_next yields at most one DataChange (Stream contract). Filter logic (records_to_data_changes, mirrors cs:333-343) - 0x32 SubscriptionStatus: keep when msg.item_correlation_id == subscription.correlation_id; drop otherwise. - 0x33 DataUpdate: keep ALL — codec exposes no per-record correlation field, and the .NET filter only checks item_correlation_id (which 0x33 doesn't carry), so DataUpdates fan out to every active subscription. Matches .NET behavior verbatim. - Records with value: None drop silently (mirrors evt.Record.Value is null filter at cs:337). - BroadcastStream Lagged(n) maps to Error::Configuration with the lag count in the detail string. Helpers - filetime_to_system_time(i64) -> SystemTime: inverse of system_time_to_filetime; saturates at Unix epoch for FILETIMEs before 1970 since SystemTime can't portably represent pre-epoch. - record_to_data_change(record, reference) -> Option<DataChange>: builds DataChange from one record, returns None for unparseable value (the codec couldn't decode the wire kind). - Status currently hardcoded to MxStatus::DATA_CHANGE_OK (mirrors NmxSubscriptionRecord.ToDataChangeStatus at NmxSubscriptionMessage.cs:22-25 which the .NET reference itself stubs to the OK constant). Cargo.toml additions: futures-util (workspace) + tokio-stream (0.1 with sync feature for BroadcastStream). Tests (5 new in mxaccess; total 40) - subscription_stream_yields_data_change_for_matching_correlation: build a 0x32 SubscriptionStatus with one Int32 record and the subscription's correlation id, inject through test_inject_sender, observe the DataChange (reference, value, quality match) on the Stream. - subscription_stream_filters_out_mismatched_correlation_for_status: inject 0x32 with wrong correlation id, assert the stream stays pending (timeout-as-success). - subscription_stream_keeps_data_update_regardless_of_correlation: inject 0x33 DataUpdate with one Int32 record (no correlation field on the message); stream still yields the DataChange. - filetime_to_system_time_round_trip: build a SystemTime with .005s precision, round-trip through both helpers, assert equality. - filetime_to_system_time_pre_unix_epoch_saturates: FILETIME 0 (year 1601) → SystemTime::UNIX_EPOCH (saturating clamp). design/followups.md: F15 moved to Resolved with both step commits referenced. Open list: 9 items (was 10). Test count delta: 511 -> 516 (+5). All four DoD gates green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
85 lines
12 KiB
Markdown
85 lines
12 KiB
Markdown
# Followups
|
|
|
|
Open work items deferred during /loop iterations. Triaged at the top of
|
|
every iteration. New items are appended under `## Open`; resolved items
|
|
move to `## Resolved` with a date + commit hash.
|
|
|
|
## Open
|
|
|
|
### F2 — NTLM verify_signature path + constant-time MAC compare (server-to-client direction)
|
|
**Severity:** P2
|
|
**Source:** M2 wave 1, `crates/mxaccess-rpc/src/ntlm.rs`
|
|
**Why deferred:** The .NET `ManagedNtlmClientContext` only implements client-to-server signing (`cs:30,124`); there is no implementation of server-to-client sign/seal keys or `verify_signature`. Both are needed when the callback exporter receives a signed inbound frame from `NmxSvc.exe`, but no such fixture exists yet.
|
|
**Resolves when:** M2 wave 3 (callback exporter) captures an `INmxSvcCallback::StatusReceived` frame with an `auth_value` trailer per `design/60-roadmap.md:56` (DoD #3) and a fixture lands under `tests/fixtures/m2-status-frame/`. Add `subtle = "2"` and gate the byte compare behind `ConstantTimeEq` at the same time.
|
|
|
|
### F3 — Cross-domain NTLM Type1/2/3 fixture
|
|
**Severity:** P2
|
|
**Source:** M2 wave 1, `crates/mxaccess-rpc/src/ntlm.rs`
|
|
**Why deferred:** All current NTLM fixtures are single-domain (the local AVEVA install). Tracked separately in `design/70-risks-and-open-questions.md` R8 (P1 risk) and the open-evidence-gaps table.
|
|
**Resolves when:** A multi-domain AVEVA test harness lands and a successful cross-domain authenticate round-trip captures Type1/2/3 bytes. Notes: this clears R8.
|
|
|
|
### F4 — BindAck / AlterContextResponse body parser
|
|
**Severity:** P2
|
|
**Source:** M2 wave 1, `crates/mxaccess-rpc/src/pdu.rs`
|
|
**Why deferred:** The .NET reference (`DceRpcPdu.cs:217-262`) parses Bind and AlterContext into the same struct but does not decode the corresponding *response* body (result list + secondary address). The Rust port's `BindPdu::decode` accepts `BindAck` packet type but does not interpret the body. The negotiated transfer syntax — needed before opnum dispatch — is currently inferred from request-side context.
|
|
**Resolves when:** A captured BindAck frame from `captures/013-loopback-subscribe-scalars/nmx-stream-*.bin` is decoded and the body shape is documented in `docs/Loopback-Protocol-Findings.md`.
|
|
|
|
### F5 — Captured DCE/RPC bind-frame fixture round-trip
|
|
**Severity:** P2
|
|
**Source:** M2 wave 1, `crates/mxaccess-rpc/src/pdu.rs`
|
|
**Why deferred:** Existing PDU tests build hand-constructed `[C706]`-conformant frames. A capture-driven round-trip (extract bind/alter PDUs from `captures/013-loopback-subscribe-scalars/nmx-stream-*.bin`, decode → encode → assert byte-identical) would be stronger evidence of parity with the live wire.
|
|
**Resolves when:** Bytes from that capture are extracted into `tests/fixtures/m2-pdu/` and the round-trip test lands.
|
|
|
|
### F6 — Port `ComObjRefProvider.cs` (OBJREF emitter via Win32 CoMarshalInterface)
|
|
**Severity:** P2
|
|
**Source:** M2 wave 1, `crates/mxaccess-rpc/src/objref.rs`
|
|
**Why deferred:** The provider is a wrapper around `ole32::CoMarshalInterface` / `IStream` / `GlobalLock` / `GlobalSize`. It needs `windows-rs`, which is currently behind the `windows-com` feature in `mxaccess-rpc/Cargo.toml`. The pure-Rust parser stands alone for the inbound activation-response path that M2 wave 1 needs.
|
|
**Resolves when:** `windows-rs` is wired into `mxaccess-rpc` (M2 wave 3 callback exporter needs to publish its own OBJREF for `IRemUnknown` / `INmxSvcCallback` registration) and an emitter port lands behind the `windows-com` feature.
|
|
|
|
### F10 — `IObjectExporter::ResolveOxid2` (opnum 4) body codec
|
|
**Severity:** P2
|
|
**Source:** M2 wave 2, `crates/mxaccess-rpc/src/object_exporter.rs`
|
|
**Why deferred:** `ObjectExporterMessages.cs` only models opnum 0 (`ResolveOxid`). Opnum 4 (`ResolveOxid2`) has a different response shape — it adds a `COMVERSION` plus an `AuthnHnt[]` array. The .NET reference does not exercise this path, so there's no executable spec to mirror.
|
|
**Resolves when:** Either a `[MS-DCOM]` §3.1.2.5.1.4-derived layout is verified against a captured `ResolveOxid2` exchange, or the .NET reference grows a `ParseResolveOxid2*` helper.
|
|
|
|
### F11 — `IRemUnknown::RemAddRef` and `RemRelease` body codecs
|
|
**Severity:** P2
|
|
**Source:** M2 wave 2, `crates/mxaccess-rpc/src/rem_unknown.rs`
|
|
**Why deferred:** `RemUnknownMessages.cs` declares the opnums (`:9-10`) but does not implement encoders/decoders. The Rust port matches that exactly per "port what is already proven."
|
|
**Resolves when:** The .NET reference adds bodies for opnums 4 / 5 (or a captured frame establishes the on-wire shape). At that point port them into `rem_unknown.rs` alongside the existing `RemQueryInterface` codec.
|
|
|
|
### F12 — `NmxClient::create` (auto-resolving COM-activation factory)
|
|
**Severity:** P1
|
|
**Source:** M3 stream B, `crates/mxaccess-nmx/src/client.rs`
|
|
**Why deferred:** `ManagedNmxService2Client.Create()` (`ManagedNmxService2Client.cs:30-64`) auto-discovers `(host, port, service_ipid)` by activating the `NmxSvc.NmxService` COM ProgID, marshalling the resulting `IUnknown` to an OBJREF, calling `IObjectExporter::ResolveOxid` against the OXID inside, then `IRemUnknown::RemQueryInterface` to get the `INmxService2` IPID. This requires `windows-rs` for `CoCreateInstance` / `CLSIDFromProgID` (the same gating dep as F6), plus the `ComObjRefProvider.MarshalIUnknownObjRef` port (also F6).
|
|
**Resolves when:** F6 lands (windows-rs wired in + `ComObjRefProvider` port). At that point `NmxClient::create()` becomes ~30 lines that chain the existing primitives: COM activation → `MarshalIUnknownObjRef` → `ComObjRef::parse` → `object_exporter_client::resolve_oxid_with_managed_ntlm_packet_integrity` → `rem_unknown::encode_rem_query_interface_request` over a temporary transport → `NmxClient::connect`.
|
|
|
|
### F14 — `tiberius`-backed SQL implementation of `Resolver` + `UserResolver`
|
|
**Severity:** P2
|
|
**Source:** M3 stream A, `crates/mxaccess-galaxy/src/sql.rs` (constants present, no client wiring yet)
|
|
**Why deferred:** `tiberius` is the recommended Rust SQL Server client; pulling it as a non-default dep means the `mxaccess-galaxy` crate keeps a slim default footprint (consumers can plug their own `Resolver` / `UserResolver` impl without dragging in TDS / native-tls / winauth). The actual `GalaxyRepositoryTagResolver` and `GalaxyRepositoryUserResolver` impls are short — they just bind the canonical SQL constants in `crate::sql` (`RESOLVE_SQL`, `BROWSE_SQL`, `USER_BY_GUID_SQL`, `USER_BY_NAME_SQL`) and translate `tiberius::Row` → typed `GalaxyTagMetadata` / `GalaxyUserProfile`.
|
|
**Resolves when:** A `tiberius`-backed module lands behind the existing `galaxy-resolver` Cargo feature flag in `mxaccess-galaxy/Cargo.toml`. Live-probe gating: needs a Galaxy DB to verify against (`MX_GALAXY_DB` env var, populated by `tools/Setup-LiveProbeEnv.ps1`). The pure-Rust foundation (data types, parser, trait, SQL strings) is already in place — this is "fill in the backend" rather than "design the surface."
|
|
|
|
|
|
## Resolved
|
|
|
|
### F7 — Consolidate `Guid` type across `mxaccess-rpc`
|
|
**Resolved:** 2026-05-05 in this iteration's commit. `Guid` was hoisted from `objref::Guid` into the new shared `crate::guid::Guid` module. `objref` and `pdu` now re-export from there; M2 wave 2's `orpc`, `object_exporter`, and `rem_unknown` import it directly. The OXID-resolve dual-string decoder additionally needs an owned protocol label (`format!("protseq_0x{:04x}", tower_id)` per `ObjectExporterMessages.cs:120`) — `ComDualStringEntry::protocol` was upgraded from `&'static str` to `Cow<'static, str>` to support both decoders without the agent's interim `Box::leak` workaround.
|
|
|
|
### F8 — `RpcError` is duplicated across `objref` and `pdu` modules
|
|
**Resolved:** 2026-05-05 in this iteration's commit. `RpcError` was hoisted into the new shared `crate::error::RpcError` module as a single union of all wave 1 variants plus a generic `Decode { offset, reason: &'static str, buffer_len }` variant for the wave 2 ORPC parsers' one-off failures. `objref` and `pdu` re-export from there; M2 wave 2's `orpc`, `object_exporter`, and `rem_unknown` use it directly.
|
|
|
|
### F13 — `NmxClient` high-level write/advise/subscribe wrappers
|
|
**Resolved:** 2026-05-05. All seven wrappers landed in `crates/mxaccess-nmx/src/client.rs`: `write`, `write2`, `write_secured2`, `advise_supervisory`, `send_observed_pre_advise_metadata`, `register_reference`, `un_advise`. Each takes a `GalaxyTagMetadata` + a typed `WriteValue` (re-exported from `mxaccess-codec`), builds the inner NMX body via `mxaccess-codec` (`write_message::encode` / `encode_timestamped` / `secured_write::encode` / `NmxItemControlMessage` / `NmxMetadataQueryMessage` / `NmxReferenceRegistrationMessage`), wraps in `NmxTransferEnvelope`, and routes through `transfer_data`. The pure-codec `encode_*_transfer_body` helpers are extracted as `pub(crate) fn` for testability, mirroring the .NET reference's `internal static` shape. `un_advise` preserves the .NET reference's quirky `NmxTransferMessageKind::Write` envelope (not `ItemControl`) per `cs:457`.
|
|
|
|
### F15 — Callback router wires `CallbackExporter` events into `Subscription` stream
|
|
**Resolved:** 2026-05-05 across two commits.
|
|
- Step 1/2 (`2b849ae`): `Session::connect_nmx` now starts a `CallbackExporter` on a 127.0.0.1 ephemeral port, builds the OBJREF via `local_hostname()` + `127.0.0.1` fallback, registers it through `NmxClient::register_engine_2` (was `..._without_callback`). A `callback_router` task drains `CallbackEvent`s, decodes each `CallbackInvoked` body via `NmxSubscriptionMessage::parse_inner`, and broadcasts parsed messages on a `tokio::sync::broadcast` channel exposed via `Session::callbacks()`. Shutdown chains: UnregisterEngine → CallbackExporter::shutdown → wait for router task.
|
|
- Step 2/2 (this commit): `Subscription` now impls `Stream<Item = Result<DataChange, Error>>`. Filtering follows the .NET reference at `cs:333-343` exactly — `0x32` SubscriptionStatus messages are kept only when `message.item_correlation_id == subscription.correlation_id`; `0x33` DataUpdate messages pass through to ALL subscriptions because the codec exposes no per-record correlation field (matches the .NET `MxNativeCallbackEvent` filter behavior verbatim). Each `NmxSubscriptionRecord` with a parseable `value` becomes one `DataChange`. Records with `value: None` are dropped silently (mirrors the .NET `evt.Record.Value is null` filter at `cs:337`). Lag-loss surfaces as `Error::Configuration(InvalidArgument)` carrying the lag count. Stream-end (broadcast sender dropped) yields `None`. New helper: `filetime_to_system_time` (inverse of the existing `system_time_to_filetime`); saturates at Unix epoch for pre-1970 FILETIMEs. Tests cover correlation match/mismatch for `0x32`, `0x33` pass-through for any correlation, and FILETIME round-trip.
|
|
|
|
### F1 — NTLM consumer-layer helpers (workstation default + from_env constructor)
|
|
**Resolved:** 2026-05-05. `NtlmClientContext::from_env()` reads `MX_RPC_USER` / `MX_RPC_PASSWORD` / `MX_RPC_DOMAIN` (mirrors `ManagedNtlmClientContext.FromEnvironment` at `cs:41-49`); empty `MX_RPC_DOMAIN` is permitted. `local_hostname()` checks `COMPUTERNAME` then `HOSTNAME` and returns the empty string when neither is set — same "unavailable" semantics as `Environment.MachineName` returning null. Lives in `mxaccess-rpc/src/ntlm.rs`; deliberately doesn't pull `gethostname` (no native-libc deps, no `unsafe` for hostname lookup). Added `NtlmError::MissingEnvVar { name }` for the env-var-unset case. Test mod gained an `EnvScope` + `ENV_LOCK` mutex pattern for serializing process-global env mutation across parallel tests.
|
|
|
|
### F9 — `ObjectExporterClient.cs` ResolveOxid wrapper methods
|
|
**Resolved:** 2026-05-05. Both portable methods land in `crates/mxaccess-rpc/src/object_exporter_client.rs`: `resolve_oxid_unauthenticated` (mirrors `cs:14-30`) and `resolve_oxid_with_managed_ntlm_packet_integrity` (mirrors `cs:66-81`). Each opens a TCP connection, binds to `IObjectExporter`, calls opnum 0 with the encoded request, and decodes the response — preferring `parse_resolve_oxid_result` then falling back to `parse_resolve_oxid_failure` for short stubs. The two SSPI flavours (`ResolveOxidWithNtlmConnect`, `ResolveOxidWithNtlmPacketIntegrity`) wrap .NET's `System.Net.Security.SspiClientContext` and are explicitly out of scope for the Rust port — that's a permanent skip, not a deferral.
|