diff --git a/docs/WorkerConversion.md b/docs/WorkerConversion.md index 18691fe..9be993f 100644 --- a/docs/WorkerConversion.md +++ b/docs/WorkerConversion.md @@ -75,6 +75,20 @@ private static MxValue CreateNullValue( } ``` +### Sparse array expansion (write path, gateway only) + +`MxSparseArray` — the `sparse_array_value` arm on `MxValue` — is a write-only +shorthand. The worker never produces or receives it; the gateway expands it into +a full `MxArray` before the command reaches the named pipe. Expansion allocates +a complete array of `total_length` slots, initializes every slot to the element +type's default (bool → `false`; numeric → `0`; string → `""`; time/timestamp → +Unix epoch), then writes each `MxSparseElement` at its declared index. The +resulting `array_value` is an ordinary `MxArray` that passes through the +conversion layer unchanged. The worker therefore still performs a single +whole-array COM write, preserving MXAccess parity. Unmentioned indices are +**reset** to their type default, not preserved from prior state — there is no +read-modify-write merge. + ### Array projection `ConvertArray` records the rank and per-dimension lengths so multi-dimensional `SAFEARRAY` shapes survive the round trip. The element type is resolved from the caller-supplied hint or the CLR element type via `ResolveArrayElementDataType`, then dispatched to the matching typed builder (`ConvertBoolArray`, `ConvertInt64Array`, `ConvertTimestampArray`, and so on). diff --git a/gateway.md b/gateway.md index 894885f..156aa1d 100644 --- a/gateway.md +++ b/gateway.md @@ -481,6 +481,68 @@ metadata rather than dropped. If a value cannot be losslessly converted, the worker should return both the best typed projection and enough diagnostic metadata to reproduce the case. +### MxSparseArray — default-fill partial array writes (write-only) + +`MxSparseArray` is a write-only `oneof kind` arm on `MxValue` that lets clients +send only the indices they want to change plus a total length, rather than +marshalling the entire array every write. The worker never produces or receives +it; expansion happens entirely in the gateway before the command reaches the pipe. + +```protobuf +message MxSparseArray { + MxDataType element_data_type = 1; + uint32 total_length = 2; + repeated MxSparseElement elements = 3; +} + +message MxSparseElement { + uint32 index = 1; + MxValue value = 2; // scalar +} +``` + +**Expansion.** Before forwarding any write command to the worker the gateway +allocates a full array of `total_length` slots, initializes every slot to the +element type's default, places each `MxSparseElement` at its index, then +replaces the `MxValue` with a normal `array_value` (`MxArray`). The worker +receives an ordinary whole-array write — parity is preserved. + +Default values by element type: + +| Element type | Default | +|---|---| +| `Boolean` | `false` | +| `Integer` / `LongInteger` | `0` | +| `Float` / `Double` | `0` | +| `String` | `""` | +| `Time` / `Timestamp` | Unix epoch (1970-01-01T00:00:00Z) | + +Unmentioned indices take the element type's default — this is a **reset**, not a +preserve. There is no read-modify-write merge: adding that would introduce cache +staleness, a race window against other writers, and the latency of a round-trip +read, all of which contradict MXAccess semantics. + +**Validation.** The gateway rejects the following with `InvalidArgument`: + +- `total_length == 0` +- any `index >= total_length` +- duplicate indices +- `element_data_type` that is `Raw` or `Unspecified` +- an element `value` whose kind does not match `element_data_type` +- `total_length` exceeds the gateway-configured maximum array length + +An empty `elements` list with a non-zero `total_length` is valid — it writes an +all-defaults array of length `total_length` (explicit reset). A `sparse_array_value` +arriving on any read or event path is rejected as a guard; the worker never +produces one. + +**Non-goals.** There is no preserve-unchanged read-modify-write merge, no +element-wise COM write (MXAccess has no such API), and no change to `ReadBulk` +string addressing. + +`sparse_array_value` is accepted by every write variant: `Write`, `Write2`, +`WriteSecured`, `WriteSecured2`, and each `*BulkEntry` entry. + ## Status Model Represent `MXSTATUS_PROXY` explicitly: @@ -1057,7 +1119,19 @@ Known important parity areas from existing captures: - Writing an array attribute replaces the whole array — it is not an element-wise patch. To change a subset of elements the caller must send the full array (unchanged elements included); sending only the changed elements - resizes the attribute. + resizes the attribute. `MxSparseArray` provides a default-fill shorthand for + this: the gateway reconstructs the full array from the supplied sparse + representation (unmentioned indices → type default) before sending the + whole-array write to the worker. +- Array attribute addresses require the `[]` body suffix to be write-capable. + The gateway normalizes bare-name addresses at `AddItem` time: when Galaxy + metadata confirms `is_array`, the gateway appends `[]` before registering the + handle with the worker. When metadata is unavailable or the address is not + recognized as an array, the address is forwarded unchanged so existing + behavior is not regressed. The normalized address is stored in + `SessionItemRegistration.TagAddress` and applies consistently to all + subsequent writes on that handle. `ReadBulk` is unaffected — it uses raw + address strings with its own ephemeral registration. The gateway should not "fix" these behaviors unless the client explicitly opts into a non-parity mode.