mxaccessgw/docs/WorkerConversion.md

# Worker Conversion Layer

The conversion layer in `MxGateway.Worker.Conversion` projects COM `VARIANT` payloads, `HRESULT` codes, and `MXSTATUS_PROXY` records into the protobuf wire types in `MxGateway.Contracts.Proto`. The design is parity-first: every projection preserves enough raw metadata that the original COM observation can be reconstructed downstream.

## Overview

`AGENTS.md` (section "Value And Status Rules") requires that the wire format use a value union capable of representing COM `VARIANT` values and arrays, that lossy conversions retain both the typed projection and raw diagnostic metadata, and that `MXSTATUS_PROXY` arrays never collapse to a single success flag. The types in `src/MxGateway.Worker/Conversion/` are the worker-side enforcement of those rules.

The layer is split into three concerns:

- Value projection: `VariantConverter`
- HRESULT projection: `HResultConverter`, `HResultConversion`
- Status projection: `MxStatusProxyConverter`, `MxStatusDetailText`, `MxStatusConversionException`

## VariantConverter

`VariantConverter` projects scalar and array `VARIANT` payloads delivered by COM interop into `MxValue` and `MxArray`. It accepts an optional `expectedDataType` so that an MXAccess attribute hint (for example `MxDataType.Time` for a 64-bit FILETIME) overrides the default CLR-driven projection.

### Scalar projection

Scalars dispatch on `Type.GetTypeCode` and populate the matching field in the `MxValue` `oneof`. Each branch records both the typed value and the source `VARIANT` tag in `VariantType`, so consumers can distinguish a `short` from an `int` even after both project to `MxDataType.Integer`.

```csharp
case TypeCode.Boolean:
    return new MxValue
    {
        DataType = MxDataType.Boolean,
        VariantType = variantType,
        BoolValue = (bool)value,
    };

case TypeCode.Byte:
case TypeCode.SByte:
case TypeCode.Int16:
case TypeCode.UInt16:
case TypeCode.Int32:
    return new MxValue
    {
        DataType = MxDataType.Integer,
        VariantType = variantType,
        Int32Value = System.Convert.ToInt32(value, CultureInfo.InvariantCulture),
    };
```

`Decimal` and 64-bit unsigned integers that exceed `long.MaxValue` cannot project losslessly. The converter still emits a typed best-effort value but attaches a `RawDiagnostic` so the loss is recorded on the wire rather than silently absorbed.

```csharp
case TypeCode.Decimal:
    return new MxValue
    {
        DataType = MxDataType.Double,
        VariantType = variantType,
        DoubleValue = System.Convert.ToDouble(value, CultureInfo.InvariantCulture),
        RawDiagnostic = "Decimal value projected to double.",
    };
```

Time-typed integers use the `expectedDataType` hint. `ConvertInt64Scalar` interprets a `long` as a Windows FILETIME when the caller asks for `MxDataType.Time`, otherwise it stays an integer. `ConvertUInt64Scalar` falls through to a raw projection if the value exceeds `long.MaxValue`.

### Null and missing values

`VARIANT` distinguishes `VT_EMPTY` from `VT_NULL`. The converter preserves both by inspecting whether the input is `DBNull` or a CLR null reference, and tags the result with `IsNull = true` so consumers do not misread a default value as data.

```csharp
private static MxValue CreateNullValue(
    object? value,
    MxDataType expectedDataType)
{
    return new MxValue
    {
        DataType = expectedDataType == MxDataType.Unspecified ? MxDataType.NoData : expectedDataType,
        VariantType = value is DBNull ? "VT_NULL" : "VT_EMPTY",
        IsNull = true,
    };
}
```

### Array projection

`ConvertArray` records the rank and per-dimension lengths so multi-dimensional `SAFEARRAY` shapes survive the round trip. The element type is resolved from the caller-supplied hint or the CLR element type via `ResolveArrayElementDataType`, then dispatched to the matching typed builder (`ConvertBoolArray`, `ConvertInt64Array`, `ConvertTimestampArray`, and so on).

```csharp
for (int dimension = 0; dimension < array.Rank; dimension++)
{
    mxArray.Dimensions.Add((uint)array.GetLength(dimension));
}

System.Type? elementType = array.GetType().GetElementType();
MxDataType elementDataType = ResolveArrayElementDataType(elementType, expectedElementDataType);
mxArray.ElementDataType = elementDataType;
```

When the element type cannot be classified, `ConvertArray` does not throw. It downgrades the result to `MxDataType.Unknown`, records the original expected type in `RawElementDataType`, and serializes each element via `ConvertRawArray` as a UTF-8 byte string. This satisfies the AGENTS.md requirement to keep both the best typed projection and the raw diagnostic metadata.

```csharp
default:
    mxArray.ElementDataType = MxDataType.Unknown;
    mxArray.RawElementDataType = (int)expectedElementDataType;
    mxArray.RawDiagnostic = CreateRawDiagnostic(array);
    mxArray.RawValues = ConvertRawArray(array);
    return mxArray;
```

### Variant type names

`GetVariantTypeName` maps CLR types to canonical `VT_*` strings (`VT_BOOL`, `VT_I4`, `VT_BSTR`, `VT_DATE`, and so on). Unmapped CLR types fall back to `CLR:<FullName>` so the wire format never silently invents a `VT_*` tag it cannot justify. Array element tags are wrapped as `SAFEARRAY(<element>)` by `CreateArrayVariantType`.

### Why lossless preservation matters

The MXAccess engine returns values whose semantic type only fully resolves after consulting the engine's own attribute metadata. Clients that round-trip these values through the gateway (replay, parity fixtures, diagnostics) need the original `VT_*` tag, the engine-declared `MxDataType`, and any conversion diagnostic; otherwise edge cases such as decimal-to-double rounding, ulong overflow, or an unknown SAFEARRAY element type become invisible bugs. Storing both the typed projection and the raw fields in the same `MxValue`/`MxArray` lets cross-language clients recover the original observation byte-for-byte where possible and detect lossy cases where it is not.

## HResultConverter and HResultConversion

`HResultConverter.Convert` wraps any `Exception` thrown across the COM boundary. It prefers `COMException.ErrorCode` over `Exception.HResult` because the runtime sometimes overwrites `Exception.HResult` while marshalling, and the `ErrorCode` field is the value the COM call actually returned.

```csharp
public HResultConversion Convert(Exception exception)
{
    if (exception is null)
    {
        throw new ArgumentNullException(nameof(exception));
    }

    int hresult = exception is COMException comException
        ? comException.ErrorCode
        : exception.HResult;

    return new HResultConversion(
        hresult,
        CreateProtocolStatus(hresult, exception),
        CreateSafeDiagnosticMessage(exception));
}
```

`CreateProtocolStatus` maps `0` (`S_OK`) to `ProtocolStatusCode.Ok` and any non-zero HRESULT to `ProtocolStatusCode.MxaccessFailure`. The status `Message` includes the exception type name and the formatted HRESULT (`HRESULT 0x<8 hex digits>`), giving downstream operators the same identifier they would see in COM event logs.

`HResultConversion` is an immutable carrier with three fields:

| Field | Purpose |
|-------|---------|
| `HResult` | Raw signed 32-bit HRESULT, suitable for inclusion in a wire reply |
| `ProtocolStatus` | The `ProtocolStatus` projection used in command replies |
| `DiagnosticMessage` | A safe, fully-qualified diagnostic string for logs |

The diagnostic message is built from the exception type's `FullName` and the formatted HRESULT only. It deliberately omits `Exception.Message` so user-supplied or localized strings do not leak into worker log channels that may be shipped to less trusted destinations.

## MxStatusProxyConverter

`MxStatusProxyConverter` projects MXAccess `MXSTATUS_PROXY` records into the `MxStatusProxy` proto message. Because the COM struct is exposed via interop reflection rather than a typed binding, the converter reads the `success`, `category`, `detectedBy`, and `detail` fields by name through `FieldInfo`.

```csharp
public MxStatusProxy Convert(object status)
{
    if (status is null)
    {
        throw new ArgumentNullException(nameof(status));
    }

    Type statusType = status.GetType();
    int success = ReadInt32Field(status, statusType, "success");
    int rawCategory = ReadInt32Field(status, statusType, "category");
    int rawDetectedBy = ReadInt32Field(status, statusType, "detectedBy");
    int detail = ReadInt32Field(status, statusType, "detail");

    return new MxStatusProxy
    {
        Success = success,
        Category = MapCategory(rawCategory),
        DetectedBy = MapSource(rawDetectedBy),
        Detail = detail,
        RawCategory = rawCategory,
        RawDetectedBy = rawDetectedBy,
        DiagnosticText = MxStatusDetailText.Lookup(detail),
    };
}
```

`MapCategory` and `MapSource` translate the integer codes documented for `MXSTATUS_PROXY` (for example `0 = Ok`, `3 = CommunicationError`, `0 = RequestingLmx`, `5 = RespondingAutomationObject`) into typed enums. The original integers are preserved alongside the typed projection in `RawCategory` and `RawDetectedBy`, so any code the runtime emits outside the documented range is still observable.

### Status arrays

`ConvertMany` walks an inbound `Array` of status structs and emits `IReadOnlyList<MxStatusProxy>`. Null entries become explicit `Unknown`/`Unknown` placeholders with a diagnostic text rather than being dropped, so the index of each status still aligns with the index of the value or item handle it describes.

```csharp
foreach (object? status in statuses)
{
    if (status is null)
    {
        converted.Add(new MxStatusProxy
        {
            Category = MxStatusCategory.Unknown,
            DetectedBy = MxStatusSource.Unknown,
            DiagnosticText = "Null MXSTATUS_PROXY entry.",
        });
        continue;
    }

    converted.Add(Convert(status));
}
```

### Why arrays are not collapsed

A single MXAccess command (notably `Read`, `Write`, and event callbacks) can return one status per item handle. AGENTS.md requires that the wire format represent each entry independently, because collapsing them to a Boolean success flag hides partial failures: a 50-item write where one item fails would be indistinguishable from a 50-item write where every item failed. Preserving the array per-position lets clients correlate each `MxStatusProxy` with its item handle and `MxValue`.

### Completion-only status fallback

When MXAccess returns a status payload that is not a recognizable `MXSTATUS_PROXY` struct (for example a completion-only byte buffer from older runtimes), `PreserveCompletionOnlyStatusBytes` hex-encodes the raw bytes so they survive transport even though they cannot be typed.

```csharp
public string PreserveCompletionOnlyStatusBytes(byte[] statusBytes)
{
    if (statusBytes is null)
    {
        throw new ArgumentNullException(nameof(statusBytes));
    }

    return $"completion_only_status_hex={BitConverter.ToString(statusBytes).Replace("-", string.Empty)}";
}
```

## MxStatusDetailText

`MxStatusDetailText` is an internal lookup that maps known `MXSTATUS_PROXY.detail` codes to short human-readable strings (for example `28 = "Index out of range"`, `42 = "Unable to convert string"`, `8017 = "Object must be offscan to modify attributes that have an MxSecurityConfigure security classification"`). `MxStatusProxyConverter.Convert` calls `Lookup` and writes the result to `DiagnosticText`. Unknown codes return `string.Empty`, leaving the numeric `Detail` field as the authoritative identifier.

The mapping covers the engine-error range documented for MXAccess (16-50, 56-61, 541-542, 8017). Adding entries here is the supported way to enrich wire-level diagnostics without changing the proto schema.

## MxStatusConversionException

`MxStatusConversionException` is the worker-internal signal that a status struct could not be projected at all - for example, the interop type is missing one of the required fields, or the field is null. `MxStatusProxyConverter.ReadInt32Field` raises it with a message that names both the offending CLR type and the missing field.

```csharp
private static int ReadInt32Field(
    object value,
    Type valueType,
    string fieldName)
{
    FieldInfo? field = valueType.GetField(fieldName, BindingFlags.Instance | BindingFlags.Public);
    if (field is null)
    {
        throw new MxStatusConversionException(
            $"Status object type '{valueType.FullName}' does not expose required field '{fieldName}'.");
    }

    object? fieldValue = field.GetValue(value);
    if (fieldValue is null)
    {
        throw new MxStatusConversionException(
            $"Status object field '{fieldName}' on type '{valueType.FullName}' is null.");
    }

    return System.Convert.ToInt32(fieldValue, CultureInfo.InvariantCulture);
}
```

The exception is distinct from `COMException` so the worker's command pipeline can route it through a dedicated handler. Callers typically translate it into a `ProtocolStatus` with `ProtocolStatusCode.MxaccessFailure` and a synthetic HRESULT, then attach the exception message as a `RawDiagnostic` rather than letting the failure surface as a generic worker error.

## Related Documentation

- [MXAccess Worker Instance Design](./MxAccessWorkerInstanceDesign.md)
- [Contracts](./Contracts.md)
- [Worker STA Threading](./WorkerSta.md)