Compare commits

...

18 Commits

Author SHA1 Message Date
Joseph Doherty e7b8aa6114 feat(client-java): add writeArrayElements default-fill helper and document semantics 2026-06-18 03:32:20 -04:00
Joseph Doherty 8a1f037d5a fix(gateway): resolve array attribute constraints by bare name via [] fallback 2026-06-18 03:25:48 -04:00
Joseph Doherty e328758c53 fix(client-python): regenerate only MxSparseArray delta, drop spurious grpcio version churn
Commit 0702551 regenerated all three protos with grpcio-tools 1.81.1 /
protobuf 6.33.5, stamping every _pb2_grpc.py with GRPC_GENERATED_VERSION
'1.81.1' and every _pb2.py with 'Protobuf Python Version: 6.33.5'.  The
project pins grpcio>=1.80,<2 and the installed runtime is 1.80.0, so the
1.81.1 stamp caused a version error at import time, breaking pytest.

Fix: restore all generated files to the main baseline (grpcio 1.80.0,
protobuf 6.31.1 stamps), then regenerate only mxaccess_gateway.proto
using an isolated venv pinned to grpcio==1.80.0, grpcio-tools==1.80.0,
protobuf==6.31.1.  The net diff vs main is a single-file change to
mxaccess_gateway_pb2.py whose DESCRIPTOR blob now encodes the new
MxSparseArray / MxSparseElement messages and sparse_array_value field.
No _pb2_grpc.py and no galaxy/worker _pb2.py files change.
2026-06-18 03:19:12 -04:00
Joseph Doherty 72cf2f4091 fix(client-rust): correct outer MxValue data_type and add end-to-end write_array_elements test 2026-06-18 03:14:14 -04:00
Joseph Doherty 474b7bd0ff test(client-go): strengthen WriteArrayElements assertions and add README example 2026-06-18 03:12:57 -04:00
Joseph Doherty 437d29f19e docs: correct MxDataType default-table labels and note non-negative sparse indices
Fix the MxSparseArray type-defaults table in gateway.md to use the real MxDataType
enum member names (Integer, not Integer/LongInteger; Time, not Time/Timestamp) and
clarify the int64 note. Add a docstring sentence to write_array_elements in the
Python client noting that indices and total_length must be non-negative.
2026-06-18 03:12:56 -04:00
Joseph Doherty f0ef7ea0a8 feat(gateway): normalize array AddItem suffix and expand sparse writes at the worker boundary 2026-06-18 03:10:13 -04:00
Joseph Doherty 3a8f2bed4e feat(client-rust): add write_array_elements default-fill helper and document semantics
Handles the new MxSparseArray wire type (proto field 19 on MxValue::Kind):
- value.rs: map SparseArrayValue to MxValueProjection::Unset (write-only; never emitted on read path)
- session.rs: add write_array_elements() that builds the sparse proto value and delegates to write()
- tests: three unit tests asserting proto shape, empty-elements case, and read-path Unset projection
- README: document write_array_elements default-fill semantics and bare-name [] normalisation
2026-06-18 03:02:15 -04:00
Joseph Doherty b7f29f3048 feat(client-go): add WriteArrayElements default-fill helper and document semantics
Regenerate Go proto types from mxaccess_gateway.proto so MxSparseArray,
MxSparseElement, and MxValue_SparseArrayValue appear in the generated
package; add MxSparseArray/MxSparseElement type aliases to types.go;
add Session.WriteArrayElements and the unexported buildSparseArrayValue
builder; add three unit tests covering the sparse oneof structure,
empty-map case, and the round-trip through WriteArrayElements; update
README with default-fill reset semantics and auto-normalize note.
2026-06-18 03:01:55 -04:00
Joseph Doherty 0702551c25 feat(client-python): add write_array_elements default-fill helper and document semantics
Regenerate Python proto bindings to pick up MxSparseArray/MxSparseElement/
sparse_array_value from the shared mxaccess_gateway.proto. Add
Session.write_array_elements which builds an MxValue(sparse_array_value=…)
from a {index→scalar} dict and delegates to the existing write(). Add 8 pytest
tests covering builder correctness and full round-trip wire shape. Update
README with a default-fill semantics paragraph and bare-name array-write note.
2026-06-18 03:01:45 -04:00
Joseph Doherty db9c68ca9c docs: document MxSparseArray default-fill writes and bare-name array AddItem 2026-06-18 03:00:19 -04:00
Joseph Doherty 95b5b09a67 feat(client-dotnet): add WriteArrayElementsAsync default-fill helper and document semantics
Adds a public WriteArrayElementsAsync helper on MxGatewaySession that builds
an MxValue{SparseArrayValue} and delegates to the existing WriteAsync. Extracts
the MxValue construction into an internal static BuildSparseArray builder for
unit-testability. Two new tests cover builder output shape and the full write
command path. README documents the reset (not preserve) semantics alongside
the existing whole-array guidance.
2026-06-18 02:59:43 -04:00
Joseph Doherty 627c17fae1 fix(gateway): reject oversized sparse array total_length with InvalidArgument
Guard against proto uint32 total_length values that exceed Array.MaxLength
before casting; the previous checked cast threw OverflowException (gRPC
Internal) instead of the intended InvalidArgument. Adds tests for the new
guard, for the null-value ArgumentNullException path, and removes the
checked keyword (redundant after the guard).
2026-06-18 02:58:03 -04:00
Joseph Doherty 34a99c783b feat(gateway): add SparseArrayExpander for default-fill partial array writes 2026-06-18 02:52:33 -04:00
Joseph Doherty 52cd0da9f5 feat(gateway): add ArrayAddressNormalizer for bare-name array AddItem 2026-06-18 02:51:37 -04:00
Joseph Doherty 8ac9a33d91 feat(contracts): add MxSparseArray write-only value for default-fill partial writes 2026-06-18 02:48:18 -04:00
Joseph Doherty dd35ae1fe6 docs(plans): implementation plan for array write ergonomics + default-fill partial writes 2026-06-18 02:46:09 -04:00
Joseph Doherty 4a6a79d02e docs(plans): design for array write ergonomics and default-fill partial writes 2026-06-18 02:39:00 -04:00
38 changed files with 14588 additions and 1277 deletions
+24
View File
@@ -168,6 +168,30 @@ the unchanged elements included. For example, to change 2 elements of a
the 2 new ones). Sending only the 2 changed values overwrites the attribute
with a 2-element array.
When only a few indices need changing and the rest should be reset to the
element type's default, use `WriteArrayElementsAsync` instead of building the
full array manually:
```csharp
await session.WriteArrayElementsAsync(
serverHandle, itemHandle,
elementDataType: MxDataType.Integer,
totalLength: 20,
elements: new Dictionary<uint, MxValue>
{
[2] = 42.ToMxValue(),
[7] = 99.ToMxValue(),
});
```
The gateway expands the sparse descriptor into a full `totalLength`-element
array before forwarding to the worker. Indices not listed in `elements` are
written as the element type's default — this is a **reset**, not a preserve;
current values at those positions are discarded. `totalLength` is required and
must match the declared length of the array attribute. Bare-name array items
(`Area001.Pump001.Speed`) are auto-normalized to the `[]` form at `AddItem`
so the array attribute accepts the write.
## CLI Usage
The test CLI supports deterministic JSON output for automation:
@@ -303,6 +303,69 @@ public sealed class MxGatewayClientSessionTests
Assert.Equal(cancellation.Token, Assert.Single(transport.InvokeCalls).CallOptions.CancellationToken);
}
/// <summary>Verifies that BuildSparseArray produces a SparseArrayValue MxValue with the correct total length and elements.</summary>
[Fact]
public void BuildSparseArray_ProducesSparseArrayValueWithCorrectTotalLengthAndElements()
{
MxValue element0 = 42.ToMxValue();
MxValue element3 = 99.ToMxValue();
Dictionary<uint, MxValue> elements = new()
{
[0u] = element0,
[3u] = element3,
};
MxValue result = MxGatewaySession.BuildSparseArray(MxDataType.Integer, totalLength: 10, elements);
Assert.Equal(MxValue.KindOneofCase.SparseArrayValue, result.KindCase);
Assert.Equal(10u, result.SparseArrayValue.TotalLength);
Assert.Equal(MxDataType.Integer, result.SparseArrayValue.ElementDataType);
Assert.Equal(2, result.SparseArrayValue.Elements.Count);
MxSparseElement el0 = Assert.Single(result.SparseArrayValue.Elements, e => e.Index == 0u);
Assert.Same(element0, el0.Value);
MxSparseElement el3 = Assert.Single(result.SparseArrayValue.Elements, e => e.Index == 3u);
Assert.Same(element3, el3.Value);
}
/// <summary>Verifies that WriteArrayElementsAsync builds a write command whose value is a SparseArrayValue.</summary>
[Fact]
public async Task WriteArrayElementsAsync_BuildsWriteCommandWithSparseArrayValue()
{
FakeGatewayTransport transport = CreateTransport();
transport.AddInvokeReply(new MxCommandReply
{
SessionId = "session-fixture",
Kind = MxCommandKind.Write,
ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok },
});
await using MxGatewayClient client = CreateClient(transport);
MxGatewaySession session = await client.OpenSessionAsync();
Dictionary<uint, MxValue> elements = new() { [1u] = 7.ToMxValue() };
await session.WriteArrayElementsAsync(
serverHandle: 12,
itemHandle: 34,
elementDataType: MxDataType.Integer,
totalLength: 5,
elements: elements,
userId: 56);
MxCommandRequest request = Assert.Single(transport.InvokeCalls).Request;
Assert.Equal(MxCommandKind.Write, request.Command.Kind);
Assert.Equal(12, request.Command.Write.ServerHandle);
Assert.Equal(34, request.Command.Write.ItemHandle);
Assert.Equal(56, request.Command.Write.UserId);
MxValue written = request.Command.Write.Value;
Assert.Equal(MxValue.KindOneofCase.SparseArrayValue, written.KindCase);
Assert.Equal(5u, written.SparseArrayValue.TotalLength);
Assert.Equal(MxDataType.Integer, written.SparseArrayValue.ElementDataType);
MxSparseElement el = Assert.Single(written.SparseArrayValue.Elements);
Assert.Equal(1u, el.Index);
Assert.Equal(7, el.Value.Int32Value);
}
private static MxGatewayClient CreateClient(FakeGatewayTransport transport)
{
return new MxGatewayClient(transport.Options, transport);
@@ -687,6 +687,63 @@ public sealed class MxGatewaySession : IAsyncDisposable
reply.EnsureProtocolSuccess().EnsureMxAccessSuccess();
}
/// <summary>
/// Writes specific array indices to an item using default-fill semantics.
/// </summary>
/// <remarks>
/// The gateway expands the sparse descriptor into a full <c>totalLength</c>-element array
/// before forwarding to the worker. Indices not listed in <paramref name="elements"/> are
/// written as the element type's default value — this is a RESET, not a preserve. The
/// current values at those positions are discarded. <paramref name="totalLength"/> is
/// required and must match the declared length of the array attribute.
/// </remarks>
/// <param name="serverHandle">The ServerHandle from register.</param>
/// <param name="itemHandle">The ItemHandle from add-item.</param>
/// <param name="elementDataType">The MXAccess data type of each element.</param>
/// <param name="totalLength">The total declared length of the target array attribute.</param>
/// <param name="elements">Map of zero-based array index to scalar <see cref="MxValue"/>.</param>
/// <param name="userId">User ID context for the write.</param>
/// <param name="cancellationToken">Cancellation token.</param>
public Task WriteArrayElementsAsync(
int serverHandle,
int itemHandle,
MxDataType elementDataType,
uint totalLength,
IReadOnlyDictionary<uint, MxValue> elements,
int userId = 0,
CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(elements);
MxValue value = BuildSparseArray(elementDataType, totalLength, elements);
return WriteAsync(serverHandle, itemHandle, value, userId, cancellationToken);
}
/// <summary>
/// Builds an <see cref="MxValue"/> whose <see cref="MxValue.SparseArrayValue"/> describes a
/// default-fill partial array write.
/// </summary>
/// <param name="elementDataType">The MXAccess data type of each element.</param>
/// <param name="totalLength">The total declared length of the target array attribute.</param>
/// <param name="elements">Map of zero-based array index to scalar <see cref="MxValue"/>.</param>
/// <returns>An <see cref="MxValue"/> with <see cref="MxValue.KindOneofCase.SparseArrayValue"/> set.</returns>
internal static MxValue BuildSparseArray(
MxDataType elementDataType,
uint totalLength,
IReadOnlyDictionary<uint, MxValue> elements)
{
MxSparseArray sparse = new()
{
ElementDataType = elementDataType,
TotalLength = totalLength,
};
foreach (KeyValuePair<uint, MxValue> kv in elements)
{
sparse.Elements.Add(new MxSparseElement { Index = kv.Key, Value = kv.Value });
}
return new MxValue { SparseArrayValue = sparse };
}
/// <summary>
/// Writes a value to an item on the MXAccess server without error checking.
/// </summary>
+26
View File
@@ -145,6 +145,32 @@ the unchanged elements included. For example, to change 2 elements of a
the 2 new ones). Sending only the 2 changed values overwrites the attribute
with a 2-element array.
`Session.WriteArrayElements` offers a default-fill shorthand: pass only the
indices you want to set along with a `totalLength`. The gateway expands the
sparse representation into a full array before forwarding to MXAccess — every
unmentioned index receives the element type's zero value (boolean `false`,
integer `0`, float `0.0`, string `""`, time = Unix epoch). This is a **RESET**
of unmentioned indices, not a preserve of existing values. Use the full-array
form (read-modify-write) when existing element values must be preserved.
```go
// Set element [3] of a 10-element float array; all other indices reset to 0.0.
err = session.WriteArrayElements(
ctx,
serverHandle, itemHandle,
mxgateway.DataTypeFloat,
10, // totalLength
map[uint32]*mxgateway.MxValue{
3: mxgateway.FloatValue(1.5),
},
userID,
)
```
`AddItem` (and `AddItem2`) now auto-normalize a bare attribute name to the `[]`
array address form expected by MXAccess, so callers do not need to append `[]`
themselves. Both forms are accepted; duplicates are deduplicated by the gateway.
## Galaxy Repository browse
The `GalaxyRepository` service (proto package `galaxy_repository.v1`) is a
File diff suppressed because it is too large Load Diff
+121
View File
@@ -666,3 +666,124 @@ func authorizationFromContext(ctx context.Context) string {
}
return values[0]
}
// ---------------------------------------------------------------------------
// WriteArrayElements / buildSparseArrayValue unit tests
// ---------------------------------------------------------------------------
func TestBuildSparseArrayValueSetsSparseOneof(t *testing.T) {
elements := map[uint32]*MxValue{
2: Int32Value(99),
0: Int32Value(10),
}
v := buildSparseArrayValue(DataTypeInteger, 5, elements)
sa, ok := v.Kind.(*pb.MxValue_SparseArrayValue)
if !ok {
t.Fatalf("Kind is %T, want *pb.MxValue_SparseArrayValue", v.Kind)
}
got := sa.SparseArrayValue
if got.GetElementDataType() != DataTypeInteger {
t.Errorf("ElementDataType = %v, want DataTypeInteger", got.GetElementDataType())
}
if got.GetTotalLength() != 5 {
t.Errorf("TotalLength = %d, want 5", got.GetTotalLength())
}
if len(got.GetElements()) != 2 {
t.Fatalf("len(Elements) = %d, want 2", len(got.GetElements()))
}
// Elements must be sorted by index (ascending).
if got.GetElements()[0].GetIndex() != 0 {
t.Errorf("Elements[0].Index = %d, want 0", got.GetElements()[0].GetIndex())
}
if got.GetElements()[0].GetValue().GetInt32Value() != 10 {
t.Errorf("Elements[0].Value = %v, want 10", got.GetElements()[0].GetValue())
}
if got.GetElements()[1].GetIndex() != 2 {
t.Errorf("Elements[1].Index = %d, want 2", got.GetElements()[1].GetIndex())
}
if got.GetElements()[1].GetValue().GetInt32Value() != 99 {
t.Errorf("Elements[1].Value = %v, want 99", got.GetElements()[1].GetValue())
}
}
func TestBuildSparseArrayValueEmptyMapProducesEmptyElements(t *testing.T) {
v := buildSparseArrayValue(DataTypeBoolean, 4, map[uint32]*MxValue{})
sa, ok := v.Kind.(*pb.MxValue_SparseArrayValue)
if !ok {
t.Fatalf("Kind is %T, want *pb.MxValue_SparseArrayValue", v.Kind)
}
if len(sa.SparseArrayValue.GetElements()) != 0 {
t.Errorf("len(Elements) = %d, want 0", len(sa.SparseArrayValue.GetElements()))
}
if sa.SparseArrayValue.GetTotalLength() != 4 {
t.Errorf("TotalLength = %d, want 4", sa.SparseArrayValue.GetTotalLength())
}
}
func TestWriteArrayElementsSendsWriteCommandWithSparseOneof(t *testing.T) {
fake := &fakeGatewayServer{
invokeReply: &pb.MxCommandReply{
SessionId: "session-1",
Kind: pb.MxCommandKind_MX_COMMAND_KIND_WRITE,
ProtocolStatus: &pb.ProtocolStatus{
Code: pb.ProtocolStatusCode_PROTOCOL_STATUS_CODE_OK,
},
},
}
client, cleanup := newBufconnClient(t, fake)
defer cleanup()
session := NewSessionForID(client, "session-1")
err := session.WriteArrayElements(
context.Background(),
1, 2,
DataTypeFloat,
10,
map[uint32]*MxValue{3: FloatValue(1.5)},
42,
)
if err != nil {
t.Fatalf("WriteArrayElements() error = %v", err)
}
cmd := fake.invokeRequest.GetCommand()
if cmd.GetKind() != pb.MxCommandKind_MX_COMMAND_KIND_WRITE {
t.Fatalf("command kind = %s, want WRITE", cmd.GetKind())
}
w := cmd.GetWrite()
if w.GetServerHandle() != 1 {
t.Fatalf("server handle = %d, want 1", w.GetServerHandle())
}
if w.GetItemHandle() != 2 {
t.Fatalf("item handle = %d, want 2", w.GetItemHandle())
}
if w.GetUserId() == 0 {
t.Fatal("user id = 0, want non-zero")
}
if w.GetUserId() != 42 {
t.Fatalf("user id = %d, want 42", w.GetUserId())
}
val := w.GetValue()
sa, ok := val.Kind.(*pb.MxValue_SparseArrayValue)
if !ok {
t.Fatalf("value kind is %T, want *pb.MxValue_SparseArrayValue", val.Kind)
}
if sa.SparseArrayValue.GetTotalLength() != 10 {
t.Errorf("TotalLength = %d, want 10", sa.SparseArrayValue.GetTotalLength())
}
if sa.SparseArrayValue.GetElementDataType() != DataTypeFloat {
t.Errorf("ElementDataType = %v, want DataTypeFloat", sa.SparseArrayValue.GetElementDataType())
}
if len(sa.SparseArrayValue.GetElements()) != 1 {
t.Fatalf("len(Elements) = %d, want 1", len(sa.SparseArrayValue.GetElements()))
}
elem := sa.SparseArrayValue.GetElements()[0]
if elem.GetIndex() != 3 {
t.Errorf("element index = %d, want 3", elem.GetIndex())
}
if elem.GetValue().GetFloatValue() != 1.5 {
t.Errorf("element float value = %v, want 1.5", elem.GetValue().GetFloatValue())
}
}
+52
View File
@@ -7,6 +7,7 @@ import (
"errors"
"fmt"
"io"
"sort"
"sync"
"time"
@@ -580,6 +581,57 @@ func (s *Session) WriteRaw(ctx context.Context, serverHandle, itemHandle int32,
})
}
// WriteArrayElements writes a sparse, default-filled array: only the given
// elements (index → scalar value) are set; every unmentioned index up to
// totalLength is written as the element type's default (false / 0 / "" / Unix
// epoch for time). The gateway expands the sparse representation into a full
// array write before forwarding to MXAccess — this is a RESET of unmentioned
// indices, not a preserve. Neither RESET semantics nor the original array
// content are retained.
//
// elementDataType must be a scalar MXAccess type (Boolean, Integer, Float,
// Double, String, or Time). totalLength must be at least as large as the
// highest index in elements plus one.
func (s *Session) WriteArrayElements(
ctx context.Context,
serverHandle, itemHandle int32,
elementDataType MxDataType,
totalLength uint32,
elements map[uint32]*MxValue,
userID int32,
) error {
return s.Write(ctx, serverHandle, itemHandle, buildSparseArrayValue(elementDataType, totalLength, elements), userID)
}
// buildSparseArrayValue constructs the MxValue carrying an MxSparseArray oneof
// arm from a map of index → scalar MxValue. Keys are visited in ascending
// order so the produced slice is deterministic (important for test assertions).
func buildSparseArrayValue(elementDataType MxDataType, totalLength uint32, elements map[uint32]*MxValue) *MxValue {
indices := make([]uint32, 0, len(elements))
for idx := range elements {
indices = append(indices, idx)
}
sort.Slice(indices, func(i, j int) bool { return indices[i] < indices[j] })
sparseElements := make([]*MxSparseElement, 0, len(elements))
for _, idx := range indices {
sparseElements = append(sparseElements, &MxSparseElement{
Index: idx,
Value: elements[idx],
})
}
return &MxValue{
Kind: &pb.MxValue_SparseArrayValue{
SparseArrayValue: &MxSparseArray{
ElementDataType: elementDataType,
TotalLength: totalLength,
Elements: sparseElements,
},
},
}
}
// PingRaw sends a diagnostic PING command and returns the raw reply.
// The message is echoed back by the gateway in the reply's DiagnosticMessage field.
func (s *Session) PingRaw(ctx context.Context, message string) (*MxCommandReply, error) {
+7
View File
@@ -36,6 +36,13 @@ type (
Value = pb.MxValue
// MxArray is the protobuf representation of an MXAccess array value.
MxArray = pb.MxArray
// MxSparseArray is the write-only protobuf type for default-fill partial
// array writes. The gateway expands it to a full array before forwarding
// to MXAccess: unmentioned indices receive the element type's default value
// (boolean false, integer 0, float 0.0, string "", time = Unix epoch).
MxSparseArray = pb.MxSparseArray
// MxSparseElement is one index/value pair inside an MxSparseArray.
MxSparseElement = pb.MxSparseElement
// MxStatusProxy mirrors the MXAccess MXSTATUS_PROXY structure.
MxStatusProxy = pb.MxStatusProxy
// ProtocolStatus is the gateway-level status carried on every reply.
+23
View File
@@ -124,6 +124,29 @@ the unchanged elements included. For example, to change 2 elements of a
the 2 new ones). Sending only the 2 changed values overwrites the attribute
with a 2-element array.
When only a few indices need changing and the rest should be reset to the
element type's default, use `writeArrayElements` instead of building the full
array manually:
```java
session.writeArrayElements(
serverHandle, itemHandle,
MxDataType.MX_DATA_TYPE_INTEGER,
20, // totalLength
Map.of(
2, MxValues.int32Value(42),
7, MxValues.int32Value(99)),
userId);
```
The gateway expands the sparse descriptor into a full `totalLength`-element
array before forwarding to the worker. Indices not listed in the map are
written as the element type's default — this is a **reset**, not a preserve;
current values at those positions are discarded. `totalLength` is required and
must match the declared length of the array attribute. Bare-name array items
(`Area001.Pump001.Speed`) are auto-normalized to the `[]` form at `AddItem` so
the array attribute accepts the write.
## Galaxy Repository Browse
The Galaxy Repository service is a separate metadata-only gRPC service exposed
File diff suppressed because it is too large Load Diff
@@ -4,7 +4,9 @@ import java.security.SecureRandom;
import java.time.Duration;
import java.util.HexFormat;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.TreeMap;
import mxaccess_gateway.v1.MxaccessGateway.AddItem2Command;
import mxaccess_gateway.v1.MxaccessGateway.AddItemBulkCommand;
import mxaccess_gateway.v1.MxaccessGateway.AddItemCommand;
@@ -18,6 +20,9 @@ import mxaccess_gateway.v1.MxaccessGateway.MxCommand;
import mxaccess_gateway.v1.MxaccessGateway.MxCommandKind;
import mxaccess_gateway.v1.MxaccessGateway.MxCommandReply;
import mxaccess_gateway.v1.MxaccessGateway.MxCommandRequest;
import mxaccess_gateway.v1.MxaccessGateway.MxDataType;
import mxaccess_gateway.v1.MxaccessGateway.MxSparseArray;
import mxaccess_gateway.v1.MxaccessGateway.MxSparseElement;
import mxaccess_gateway.v1.MxaccessGateway.MxValue;
import mxaccess_gateway.v1.MxaccessGateway.OpenSessionReply;
import mxaccess_gateway.v1.MxaccessGateway.ReadBulkCommand;
@@ -603,6 +608,54 @@ public final class MxGatewaySession implements AutoCloseable {
.build());
}
/**
* Writes a subset of an array's elements using MXAccess {@code Write}, building a
* write-only {@link MxSparseArray} value that the gateway expands into a full,
* default-filled array before forwarding to the worker.
*
* <p><strong>Default-fill semantics:</strong> only the indices supplied in
* {@code elements} are written; every unmentioned index is <em>reset</em> to the
* element type's default (for example {@code 0}, {@code false}, or an empty string),
* <em>not</em> preserved from the array's current contents. Use a full
* {@link MxValue} array write when you need to keep existing element values.
*
* <p>{@code totalLength} is required and defines the length of the expanded array;
* supplied indices must be within {@code [0, totalLength)}. Elements are iterated in
* ascending index order so the produced command is deterministic.
*
* @param serverHandle the {@code ServerHandle} owning the item
* @param itemHandle the {@code ItemHandle} to write
* @param elementDataType the {@link MxDataType} of the array's elements
* @param totalLength the total length of the expanded array
* @param elements the indices to write mapped to their scalar values; unmentioned
* indices are reset to the element type default
* @param userId the MXAccess user id used for security checks
* @throws MxGatewayException on transport or protocol failure
*/
public void writeArrayElements(
int serverHandle,
int itemHandle,
MxDataType elementDataType,
int totalLength,
Map<Integer, MxValue> elements,
int userId) {
Objects.requireNonNull(elementDataType, "elementDataType");
Objects.requireNonNull(elements, "elements");
MxSparseArray.Builder sparse = MxSparseArray.newBuilder()
.setElementDataType(elementDataType)
.setTotalLength(totalLength);
// Iterate in ascending index order so the built command is deterministic.
for (Map.Entry<Integer, MxValue> entry : new TreeMap<>(elements).entrySet()) {
sparse.addElements(MxSparseElement.newBuilder()
.setIndex(entry.getKey())
.setValue(Objects.requireNonNull(entry.getValue(), "elements value")));
}
MxValue value = MxValue.newBuilder()
.setSparseArrayValue(sparse)
.build();
writeRaw(serverHandle, itemHandle, value, userId);
}
/**
* Invokes MXAccess {@code Write2}, which carries an explicit timestamp.
*
@@ -153,6 +153,9 @@ public final class MxValues {
case TIMESTAMP_VALUE -> instant(value.getTimestampValue());
case ARRAY_VALUE -> nativeArray(value.getArrayValue());
case RAW_VALUE -> value.getRawValue().toByteArray();
// Write-only sparse descriptor: never produced by a read/decoded
// value, so it has no native representation.
case SPARSE_ARRAY_VALUE -> null;
case KIND_NOT_SET -> null;
};
}
@@ -19,6 +19,7 @@ import io.grpc.stub.ServerCallStreamObserver;
import io.grpc.stub.StreamObserver;
import java.time.Duration;
import java.util.List;
import java.util.Map;
import java.util.UUID;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;
@@ -36,7 +37,10 @@ import mxaccess_gateway.v1.MxaccessGateway.CloseSessionRequest;
import mxaccess_gateway.v1.MxaccessGateway.MxCommandKind;
import mxaccess_gateway.v1.MxaccessGateway.MxCommandReply;
import mxaccess_gateway.v1.MxaccessGateway.MxCommandRequest;
import mxaccess_gateway.v1.MxaccessGateway.MxDataType;
import mxaccess_gateway.v1.MxaccessGateway.MxEvent;
import mxaccess_gateway.v1.MxaccessGateway.MxSparseElement;
import mxaccess_gateway.v1.MxaccessGateway.MxValue;
import mxaccess_gateway.v1.MxaccessGateway.OpenSessionReply;
import mxaccess_gateway.v1.MxaccessGateway.OpenSessionRequest;
import mxaccess_gateway.v1.MxaccessGateway.ProtocolStatus;
@@ -396,6 +400,57 @@ final class MxGatewayClientSessionTests {
}
}
@Test
void writeArrayElementsBuildsSparseArrayWriteCommand() throws Exception {
AtomicReference<MxCommandRequest> commandRequest = new AtomicReference<>();
TestGatewayService service = new TestGatewayService() {
@Override
public void invoke(MxCommandRequest request, StreamObserver<MxCommandReply> responseObserver) {
commandRequest.set(request);
responseObserver.onNext(MxCommandReply.newBuilder()
.setSessionId(request.getSessionId())
.setKind(request.getCommand().getKind())
.setProtocolStatus(ok())
.build());
responseObserver.onCompleted();
}
};
try (InProcessGateway gateway = InProcessGateway.start(service, new AtomicReference<>());
MxGatewayClient client = gateway.client("", Duration.ofSeconds(5))) {
MxGatewaySession session = MxGatewaySession.forSessionId(client, "sparse-session");
// Supply indices out of order to prove deterministic ascending iteration.
Map<Integer, MxValue> elements = Map.of(
3, MxValues.int32Value(99),
1, MxValues.int32Value(7));
session.writeArrayElements(12, 34, MxDataType.MX_DATA_TYPE_INTEGER, 5, elements, 56);
MxCommandRequest request = commandRequest.get();
assertNotNull(request);
assertEquals(MxCommandKind.MX_COMMAND_KIND_WRITE, request.getCommand().getKind());
assertEquals(12, request.getCommand().getWrite().getServerHandle());
assertEquals(34, request.getCommand().getWrite().getItemHandle());
assertEquals(56, request.getCommand().getWrite().getUserId());
MxValue written = request.getCommand().getWrite().getValue();
assertEquals(MxValue.KindCase.SPARSE_ARRAY_VALUE, written.getKindCase());
assertEquals(5, written.getSparseArrayValue().getTotalLength());
assertEquals(
MxDataType.MX_DATA_TYPE_INTEGER,
written.getSparseArrayValue().getElementDataType());
List<MxSparseElement> sparse = written.getSparseArrayValue().getElementsList();
assertEquals(2, sparse.size());
// Ascending index order is guaranteed by the helper.
assertEquals(1, sparse.get(0).getIndex());
assertEquals(7, sparse.get(0).getValue().getInt32Value());
assertEquals(3, sparse.get(1).getIndex());
assertEquals(99, sparse.get(1).getValue().getInt32Value());
}
}
private static ProtocolStatus ok() {
return ProtocolStatus.newBuilder()
.setCode(ProtocolStatusCode.PROTOCOL_STATUS_CODE_OK)
+27
View File
@@ -148,6 +148,33 @@ the unchanged elements included. For example, to change 2 elements of a
the 2 new ones). Sending only the 2 changed values overwrites the attribute
with a 2-element array.
### Default-fill partial array writes
`Session.write_array_elements` lets you write only the indices you care about.
The gateway fills every unmentioned position with the type default for the
declared `element_data_type` (0, `False`, `""`, Unix epoch for timestamps).
The previous value at those positions is **not** preserved — the gateway expands
the sparse map to a full array before forwarding the write to MXAccess, so this
is still a full replacement:
```python
# Write indices 0 and 5 of a 10-element integer array.
# Positions 1-4 and 6-9 become 0, not their previous values.
await session.write_array_elements(
server_handle=server_handle,
item_handle=item_handle,
element_data_type=pb.MX_DATA_TYPE_INTEGER,
total_length=10,
elements={0: 100, 5: 500},
)
```
Bare-name array items (e.g. `Object.ArrayAttr` without an index suffix) added
via `add_item` auto-normalize to `[]` — they refer to the whole array, not a
single element. Writes through such handles must cover the full array or use
`write_array_elements` to supply `total_length` and let the gateway fill
defaults for the rest.
## Galaxy Repository Browse
The `GalaxyRepositoryClient` wraps the read-only `GalaxyRepository` gRPC
File diff suppressed because one or more lines are too long
@@ -489,6 +489,60 @@ class Session:
correlation_id=correlation_id,
)
async def write_array_elements(
self,
server_handle: int,
item_handle: int,
element_data_type: "pb.MxDataType.ValueType",
total_length: int,
elements: dict[int, MxValueInput],
*,
user_id: int = 0,
correlation_id: str = "",
) -> None:
"""Write a partial array by specifying only the indices you want to set.
The gateway expands the sparse representation into a full ``total_length``
array before forwarding the write to MXAccess. Indices not listed in
*elements* are filled with the type default for *element_data_type* (0,
False, empty string, Unix epoch for timestamps, etc.). The previous
value at those positions is **not** preserved — this is a full array
replacement, not a patch.
Args:
server_handle: Handle returned by :meth:`register`.
item_handle: Handle returned by :meth:`add_item`.
element_data_type: ``pb.MX_DATA_TYPE_*`` enum value for the scalar
element type of the target array attribute.
total_length: Total number of elements in the written array. Must
be > 0 and large enough to contain every index in *elements*.
Both *total_length* and all keys in *elements* must be
non-negative; the gateway rejects negative or out-of-range
values with ``InvalidArgument`` (the proto fields are
``uint32``).
elements: Mapping of zero-based element index to scalar value.
Values are converted with :func:`~zb_mom_ww_mxgateway.values.to_mx_value`.
user_id: Galaxy user id to stamp on the write (requires a prior
supervisory advise to take effect — see README).
correlation_id: Optional client-supplied correlation token echoed
in the command reply.
"""
sparse = pb.MxSparseArray(
element_data_type=element_data_type,
total_length=total_length,
elements=[
pb.MxSparseElement(index=idx, value=to_mx_value(val))
for idx, val in elements.items()
],
)
await self.write(
server_handle,
item_handle,
pb.MxValue(sparse_array_value=sparse),
user_id=user_id,
correlation_id=correlation_id,
)
async def write2(
self,
server_handle: int,
@@ -0,0 +1,209 @@
"""Tests for Session.write_array_elements default-fill sparse-array helper."""
from __future__ import annotations
from typing import Any
import pytest
from zb_mom_ww_mxgateway import ClientOptions, GatewayClient
from zb_mom_ww_mxgateway.generated import mxaccess_gateway_pb2 as pb
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _make_sparse_mx_value(
element_data_type: "pb.MxDataType.ValueType",
total_length: int,
elements: dict[int, Any],
) -> pb.MxValue:
"""Build an MxValue wrapping an MxSparseArray from Python primitives.
Mirrors the logic inside Session.write_array_elements so tests can assert
the exact wire shape the helper produces without going through the full
gRPC stack.
"""
from zb_mom_ww_mxgateway.values import to_mx_value
return pb.MxValue(
sparse_array_value=pb.MxSparseArray(
element_data_type=element_data_type,
total_length=total_length,
elements=[
pb.MxSparseElement(index=idx, value=to_mx_value(val))
for idx, val in elements.items()
],
)
)
# ---------------------------------------------------------------------------
# Fake stub (minimal — only needs Invoke / OpenSession)
# ---------------------------------------------------------------------------
class _FakeUnary:
def __init__(self, replies: list[Any]) -> None:
self.replies = list(replies)
self.requests: list[Any] = []
self.metadata: tuple[tuple[str, str], ...] | None = None
async def __call__(
self,
request: Any,
*,
metadata: tuple[tuple[str, str], ...],
) -> Any:
self.requests.append(request)
self.metadata = metadata
return self.replies.pop(0)
class _FakeStub:
"""Minimal stub that satisfies GatewayClient for a single invoke round-trip."""
def __init__(self) -> None:
ok = pb.ProtocolStatus(code=pb.PROTOCOL_STATUS_CODE_OK)
self.open_session = _FakeUnary([pb.OpenSessionReply(session_id="s1", protocol_status=ok)])
self.invoke = _FakeUnary(
[
pb.MxCommandReply(
session_id="s1",
kind=pb.MX_COMMAND_KIND_WRITE,
protocol_status=ok,
),
]
)
self.OpenSession = self.open_session
self.Invoke = self.invoke
# ---------------------------------------------------------------------------
# Unit tests
# ---------------------------------------------------------------------------
def test_sparse_mx_value_builder_sets_correct_oneof() -> None:
"""Builder helper must produce an MxValue with kind == 'sparse_array_value'."""
mv = _make_sparse_mx_value(pb.MX_DATA_TYPE_INTEGER, 5, {0: 10, 3: 30})
assert mv.WhichOneof("kind") == "sparse_array_value"
def test_sparse_mx_value_builder_total_length() -> None:
"""total_length must equal the value passed to the builder."""
mv = _make_sparse_mx_value(pb.MX_DATA_TYPE_INTEGER, 20, {1: 7})
assert mv.sparse_array_value.total_length == 20
def test_sparse_mx_value_builder_element_count_and_values() -> None:
"""Elements list length and scalar values must match the input dict."""
mv = _make_sparse_mx_value(pb.MX_DATA_TYPE_INTEGER, 10, {0: 11, 4: 55, 9: 99})
sa = mv.sparse_array_value
assert len(sa.elements) == 3
by_index = {e.index: e.value for e in sa.elements}
assert by_index[0].int32_value == 11
assert by_index[4].int32_value == 55
assert by_index[9].int32_value == 99
def test_sparse_mx_value_builder_element_data_type() -> None:
"""element_data_type must be forwarded verbatim."""
mv = _make_sparse_mx_value(pb.MX_DATA_TYPE_FLOAT, 3, {})
assert mv.sparse_array_value.element_data_type == pb.MX_DATA_TYPE_FLOAT
def test_sparse_mx_value_builder_empty_elements() -> None:
"""An empty elements dict must still produce a valid MxSparseArray."""
mv = _make_sparse_mx_value(pb.MX_DATA_TYPE_BOOLEAN, 8, {})
sa = mv.sparse_array_value
assert len(sa.elements) == 0
assert sa.total_length == 8
# ---------------------------------------------------------------------------
# Integration-level: write_array_elements routes through Session.write
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_write_array_elements_sends_sparse_array_write_command() -> None:
"""write_array_elements must send a WRITE command whose value is sparse_array_value."""
stub = _FakeStub()
client = await GatewayClient.connect(
ClientOptions(endpoint="fake", api_key="mxgw_test_secret", plaintext=True),
stub=stub,
)
session = await client.open_session()
await session.write_array_elements(
server_handle=1,
item_handle=2,
element_data_type=pb.MX_DATA_TYPE_INTEGER,
total_length=10,
elements={0: 100, 5: 500},
)
assert len(stub.invoke.requests) == 1
cmd_req: pb.MxCommandRequest = stub.invoke.requests[0]
cmd = cmd_req.command
assert cmd.kind == pb.MX_COMMAND_KIND_WRITE
mv = cmd.write.value
assert mv.WhichOneof("kind") == "sparse_array_value"
sa = mv.sparse_array_value
assert sa.element_data_type == pb.MX_DATA_TYPE_INTEGER
assert sa.total_length == 10
assert len(sa.elements) == 2
by_index = {e.index: e.value for e in sa.elements}
assert by_index[0].int32_value == 100
assert by_index[5].int32_value == 500
@pytest.mark.asyncio
async def test_write_array_elements_forwards_user_id() -> None:
"""user_id must reach the WriteCommand."""
stub = _FakeStub()
client = await GatewayClient.connect(
ClientOptions(endpoint="fake", api_key="mxgw_test_secret", plaintext=True),
stub=stub,
)
session = await client.open_session()
await session.write_array_elements(
server_handle=1,
item_handle=2,
element_data_type=pb.MX_DATA_TYPE_BOOLEAN,
total_length=4,
elements={},
user_id=42,
)
cmd = stub.invoke.requests[0].command
assert cmd.write.user_id == 42
@pytest.mark.asyncio
async def test_write_array_elements_string_elements() -> None:
"""String element values must be encoded as string_value scalars."""
stub = _FakeStub()
client = await GatewayClient.connect(
ClientOptions(endpoint="fake", api_key="mxgw_test_secret", plaintext=True),
stub=stub,
)
session = await client.open_session()
await session.write_array_elements(
server_handle=1,
item_handle=2,
element_data_type=pb.MX_DATA_TYPE_STRING,
total_length=3,
elements={1: "hello", 2: "world"},
)
sa = stub.invoke.requests[0].command.write.value.sparse_array_value
by_index = {e.index: e.value for e in sa.elements}
assert by_index[1].string_value == "hello"
assert by_index[2].string_value == "world"
+33
View File
@@ -168,6 +168,39 @@ the unchanged elements included. For example, to change 2 elements of a
the 2 new ones). Sending only the 2 changed values overwrites the attribute
with a 2-element array.
#### Default-fill partial array writes
When you only need to set a handful of indices and want every other position to
take the element type's default (zero / `false` / empty string / Unix epoch for
timestamps), use `Session::write_array_elements` instead:
```rust
// Write a 10-element integer array; index 0 = 42, index 7 = 99,
// all other indices default to 0 (not preserved from the previous value).
session
.write_array_elements(
server_handle,
item_handle,
MxDataType::Integer,
10,
[(0, MxValue::int32(42)), (7, MxValue::int32(99))],
user_id,
)
.await?;
```
The gateway expands the sparse representation into a full `MxArray` before
forwarding to the worker — the worker and MXAccess COM never see the sparse
form. Unmentioned indices are reset to the type default, **not** preserved from
the existing attribute value.
#### Bare-name array AddItem normalisation
`AddItem` for a bare array attribute name (e.g. `Tank01.Temperature`) is
automatically normalised to `Tank01.Temperature[]` by the gateway so the
worker can resolve the full array. You do not need to append `[]` in client
code; the gateway handles it.
## Galaxy Repository browse
The Galaxy Repository service exposes a read-only browse over the AVEVA System
+60 -6
View File
@@ -17,12 +17,12 @@ use crate::generated::mxaccess_gateway::v1::mx_command_reply;
use crate::generated::mxaccess_gateway::v1::{
AddItem2Command, AddItemBulkCommand, AddItemCommand, AdviseCommand, AdviseItemBulkCommand,
BulkReadResult, BulkWriteResult, CloseSessionRequest, MxCommand, MxCommandKind, MxCommandReply,
MxCommandRequest, MxValue as ProtoMxValue, OpenSessionRequest, ReadBulkCommand,
RegisterCommand, RemoveItemBulkCommand, RemoveItemCommand, StreamEventsRequest,
SubscribeBulkCommand, SubscribeResult, UnAdviseCommand, UnAdviseItemBulkCommand,
UnsubscribeBulkCommand, Write2BulkCommand, Write2BulkEntry, Write2Command, WriteBulkCommand,
WriteBulkEntry, WriteCommand, WriteSecured2BulkCommand, WriteSecured2BulkEntry,
WriteSecuredBulkCommand, WriteSecuredBulkEntry,
MxCommandRequest, MxDataType, MxSparseArray, MxSparseElement, MxValue as ProtoMxValue,
OpenSessionRequest, ReadBulkCommand, RegisterCommand, RemoveItemBulkCommand, RemoveItemCommand,
StreamEventsRequest, SubscribeBulkCommand, SubscribeResult, UnAdviseCommand,
UnAdviseItemBulkCommand, UnsubscribeBulkCommand, Write2BulkCommand, Write2BulkEntry,
Write2Command, WriteBulkCommand, WriteBulkEntry, WriteCommand, WriteSecured2BulkCommand,
WriteSecured2BulkEntry, WriteSecuredBulkCommand, WriteSecuredBulkEntry,
};
use crate::value::MxValue;
@@ -547,6 +547,60 @@ impl Session {
Ok(())
}
/// Write a sparse, default-filled array: only the given elements
/// (index → scalar value) are set; every unmentioned index up to
/// `total_length` is written as the element type's default (a reset,
/// **not** a preserve). The gateway expands the sparse representation into
/// a whole-array write before forwarding to the worker.
///
/// This is a convenience wrapper around [`Session::write`] that builds the
/// `MxSparseArray` wire value for you. Call [`Session::write`] directly
/// if you need to pass a pre-built [`MxValue`] carrying a full
/// `MxArray`.
///
/// # Errors
///
/// Returns [`Error::InvalidArgument`] (propagated from the gateway) if
/// `total_length` is zero, exceeds the gateway's maximum array length, or
/// any element index is out of range. Returns [`Error::Command`] for
/// non-OK worker statuses, plus the usual transport/status errors.
pub async fn write_array_elements(
&self,
server_handle: i32,
item_handle: i32,
element_data_type: MxDataType,
total_length: u32,
elements: impl IntoIterator<Item = (u32, MxValue)>,
user_id: i32,
) -> Result<(), Error> {
use crate::generated::mxaccess_gateway::v1::mx_value::Kind;
let sparse_elements: Vec<MxSparseElement> = elements
.into_iter()
.map(|(index, value)| MxSparseElement {
index,
value: Some(value.into_proto()),
})
.collect();
let sparse_value = ProtoMxValue {
kind: Some(Kind::SparseArrayValue(MxSparseArray {
element_data_type: element_data_type as i32,
total_length,
elements: sparse_elements,
})),
..ProtoMxValue::default()
};
self.write(
server_handle,
item_handle,
MxValue::from_proto(sparse_value),
user_id,
)
.await
}
/// Run MXAccess `Write2` (single-value with caller-supplied timestamp).
///
/// # Errors
+5 -1
View File
@@ -173,7 +173,11 @@ impl MxValueProjection {
Some(Kind::TimestampValue(value)) => Self::Timestamp(*value),
Some(Kind::ArrayValue(value)) => Self::Array(MxArrayValue::from_proto(value.clone())),
Some(Kind::RawValue(value)) => Self::Raw(value.clone()),
None => Self::Unset,
// SparseArrayValue is write-only: the gateway expands it before forwarding
// to the worker and never emits it in events or read replies. Map it to
// Unset so any read-side code that encounters a stale or mis-routed
// sparse value degrades gracefully rather than panicking.
Some(Kind::SparseArrayValue(_)) | None => Self::Unset,
}
}
}
+189 -4
View File
@@ -17,6 +17,7 @@ use tonic::{Request, Response, Status};
use zb_mom_ww_mxgateway_client::generated::mxaccess_gateway::v1::mx_access_gateway_server::{
MxAccessGateway, MxAccessGatewayServer,
};
use zb_mom_ww_mxgateway_client::generated::mxaccess_gateway::v1::mx_command;
use zb_mom_ww_mxgateway_client::generated::mxaccess_gateway::v1::mx_command_reply;
use zb_mom_ww_mxgateway_client::generated::mxaccess_gateway::v1::mx_value::Kind;
use zb_mom_ww_mxgateway_client::generated::mxaccess_gateway::v1::{
@@ -24,11 +25,11 @@ use zb_mom_ww_mxgateway_client::generated::mxaccess_gateway::v1::{
AddItem2Reply, AddItemReply, AlarmConditionState, AlarmFeedMessage, AlarmTransitionKind,
BulkReadReply, BulkReadResult, BulkSubscribeReply, BulkWriteReply, BulkWriteResult,
CloseSessionReply, CloseSessionRequest, MxCommandKind, MxCommandReply, MxDataType, MxEvent,
MxEventFamily, MxStatusCategory, MxStatusProxy, MxStatusSource, MxValue,
OnAlarmTransitionEvent, OpenSessionReply, OpenSessionRequest, ProtocolStatus,
MxEventFamily, MxSparseArray, MxSparseElement, MxStatusCategory, MxStatusProxy, MxStatusSource,
MxValue, OnAlarmTransitionEvent, OpenSessionReply, OpenSessionRequest, ProtocolStatus,
ProtocolStatusCode, QueryActiveAlarmsRequest, RegisterReply, SessionState, StreamAlarmsRequest,
StreamEventsRequest, SubscribeResult, Write2BulkEntry, WriteBulkEntry, WriteSecured2BulkEntry,
WriteSecuredBulkEntry,
StreamEventsRequest, SubscribeResult, Write2BulkEntry, WriteBulkEntry, WriteCommand,
WriteSecured2BulkEntry, WriteSecuredBulkEntry,
};
use zb_mom_ww_mxgateway_client::{
next_correlation_id, ApiKey, ClientOptions, CommandError, Error, GatewayClient, MxStatus,
@@ -659,6 +660,9 @@ struct FakeState {
authorization: Mutex<Option<String>>,
last_command_kind: Mutex<Option<i32>>,
last_correlation_id: Mutex<Option<String>>,
/// Captures the last `WriteCommand` payload received, populated when the
/// `WriteOk` override is active. Used by `write_array_elements` e2e test.
last_write_command: Mutex<Option<WriteCommand>>,
stream_dropped: Arc<AtomicBool>,
/// Optional per-test override that pins the fake's `Invoke` handler to
/// a specific reply shape (or `Err(Status)`). The default of `None`
@@ -683,6 +687,10 @@ enum InvokeOverride {
/// Fail the unary call with `Status::unavailable(...)` so the client's
/// `Code::Unavailable` -> `Error::Unavailable` mapping is exercised.
Unavailable(String),
/// Accept a `Write` command (return `protocol_status = Ok`, no payload)
/// and capture the decoded `WriteCommand` in
/// `FakeState::last_write_command` for inspection.
WriteOk,
}
#[derive(Clone)]
@@ -764,6 +772,23 @@ impl MxAccessGateway for FakeGateway {
..MxCommandReply::default()
})),
InvokeOverride::Unavailable(message) => Err(Status::unavailable(message)),
InvokeOverride::WriteOk => {
// Extract and capture the WriteCommand payload so the test
// can assert on server_handle, item_handle, user_id, and value.
if let Some(mx_command::Payload::Write(write_cmd)) =
request.command.and_then(|c| c.payload)
{
*self.state.last_write_command.lock().await = Some(write_cmd);
}
Ok(Response::new(MxCommandReply {
session_id: request.session_id,
correlation_id: "fake-correlation".to_owned(),
kind,
protocol_status: Some(ok_status("write ok")),
payload: None,
..MxCommandReply::default()
}))
}
};
}
@@ -1091,3 +1116,163 @@ fn case_by_id<'a>(cases: &'a [Value], id: &str) -> &'a Value {
.find(|case| case["id"].as_str() == Some(id))
.unwrap_or_else(|| panic!("missing fixture case {id}"))
}
// ---------------------------------------------------------------------------
// write_array_elements — end-to-end fake-server test
// ---------------------------------------------------------------------------
#[tokio::test]
async fn write_array_elements_routes_sparse_array_write_through_fake_gateway() {
// Arrange: stand up the fake gateway with WriteOk so the Write command
// succeeds and the sent WriteCommand is captured for inspection.
let state = Arc::new(FakeState::default());
*state.invoke_override.lock().await = Some(InvokeOverride::WriteOk);
let endpoint = spawn_fake_gateway(state.clone()).await;
let client = GatewayClient::connect(ClientOptions::new(endpoint))
.await
.unwrap();
let session = client.session("session-fixture");
// Act: call the public write_array_elements helper.
session
.write_array_elements(
12,
34,
MxDataType::Integer,
10,
[(2u32, ClientMxValue::int32(42))],
7,
)
.await
.unwrap();
// Assert: the fake captured a Write command with the expected handles and
// a SparseArrayValue whose total_length and element index/value are correct.
let captured = state
.last_write_command
.lock()
.await
.take()
.expect("fake should have captured a WriteCommand");
assert_eq!(captured.server_handle, 12, "server_handle must round-trip");
assert_eq!(captured.item_handle, 34, "item_handle must round-trip");
assert_eq!(captured.user_id, 7, "user_id must round-trip");
let value = captured.value.expect("WriteCommand must carry a value");
assert_eq!(
value.data_type, 0,
"outer MxValue.data_type must be Unspecified (0), not the element type"
);
let Kind::SparseArrayValue(ref sparse) = value.kind.as_ref().unwrap() else {
panic!(
"expected SparseArrayValue kind on the outer MxValue, got {:?}",
value.kind
);
};
assert_eq!(
sparse.element_data_type,
MxDataType::Integer as i32,
"element_data_type must carry the element type"
);
assert_eq!(sparse.total_length, 10, "total_length must round-trip");
assert_eq!(sparse.elements.len(), 1, "one element supplied");
let elem = &sparse.elements[0];
assert_eq!(elem.index, 2, "element index must round-trip");
assert_eq!(
elem.value.as_ref().unwrap().kind,
Some(Kind::Int32Value(42)),
"element value must round-trip"
);
}
// ---------------------------------------------------------------------------
// write_array_elements — proto shape unit tests
// ---------------------------------------------------------------------------
/// Build the proto `MxValue` that `write_array_elements` would send and assert
/// the sparse oneof variant has the correct `total_length` and elements.
fn sparse_int32_value(
total_length: u32,
elements: impl IntoIterator<Item = (u32, i32)>,
) -> MxValue {
let sparse_elements: Vec<MxSparseElement> = elements
.into_iter()
.map(|(index, v)| MxSparseElement {
index,
value: Some(MxValue {
data_type: MxDataType::Integer as i32,
variant_type: "VT_I4".to_owned(),
kind: Some(Kind::Int32Value(v)),
..MxValue::default()
}),
})
.collect();
MxValue {
data_type: MxDataType::Integer as i32,
variant_type: String::new(),
kind: Some(Kind::SparseArrayValue(MxSparseArray {
element_data_type: MxDataType::Integer as i32,
total_length,
elements: sparse_elements,
})),
..MxValue::default()
}
}
#[test]
fn write_array_elements_proto_shape_has_sparse_oneof_kind() {
let proto = sparse_int32_value(5, [(0, 10), (3, 30)]);
let Kind::SparseArrayValue(ref sparse) = proto.kind.as_ref().unwrap() else {
panic!("expected SparseArrayValue kind, got {:?}", proto.kind);
};
assert_eq!(sparse.total_length, 5, "total_length must round-trip");
assert_eq!(sparse.elements.len(), 2, "two elements supplied");
assert_eq!(sparse.element_data_type, MxDataType::Integer as i32);
let elem0 = &sparse.elements[0];
assert_eq!(elem0.index, 0);
assert_eq!(
elem0.value.as_ref().unwrap().kind,
Some(Kind::Int32Value(10))
);
let elem3 = &sparse.elements[1];
assert_eq!(elem3.index, 3);
assert_eq!(
elem3.value.as_ref().unwrap().kind,
Some(Kind::Int32Value(30))
);
}
#[test]
fn write_array_elements_empty_elements_is_valid_all_defaults() {
let proto = sparse_int32_value(8, []);
let Kind::SparseArrayValue(ref sparse) = proto.kind.as_ref().unwrap() else {
panic!("expected SparseArrayValue kind");
};
assert_eq!(sparse.total_length, 8);
assert!(
sparse.elements.is_empty(),
"no elements means every index defaults"
);
}
#[test]
fn sparse_array_value_round_trips_through_client_mx_value_projection_as_unset() {
// SparseArrayValue is write-only. If it ever arrives on the read path
// (e.g. a future version bug), the projection should degrade to Unset
// rather than panic, because the enum variant is not readable.
let proto = sparse_int32_value(4, [(1, 99)]);
let client_value = ClientMxValue::from_proto(proto);
assert_eq!(
client_value.projection(),
&MxValueProjection::Unset,
"write-only SparseArrayValue must project to Unset, not panic"
);
}
+14
View File
@@ -75,6 +75,20 @@ private static MxValue CreateNullValue(
}
```
### Sparse array expansion (write path, gateway only)
`MxSparseArray` — the `sparse_array_value` arm on `MxValue` — is a write-only
shorthand. The worker never produces or receives it; the gateway expands it into
a full `MxArray` before the command reaches the named pipe. Expansion allocates
a complete array of `total_length` slots, initializes every slot to the element
type's default (bool → `false`; numeric → `0`; string → `""`; time/timestamp →
Unix epoch), then writes each `MxSparseElement` at its declared index. The
resulting `array_value` is an ordinary `MxArray` that passes through the
conversion layer unchanged. The worker therefore still performs a single
whole-array COM write, preserving MXAccess parity. Unmentioned indices are
**reset** to their type default, not preserved from prior state — there is no
read-modify-write merge.
### Array projection
`ConvertArray` records the rank and per-dimension lengths so multi-dimensional `SAFEARRAY` shapes survive the round trip. The element type is resolved from the caller-supplied hint or the CLR element type via `ResolveArrayElementDataType`, then dispatched to the matching typed builder (`ConvertBoolArray`, `ConvertInt64Array`, `ConvertTimestampArray`, and so on).
@@ -0,0 +1,193 @@
# Array Write Ergonomics & Default-Fill Partial Writes — Design
Date: 2026-06-18
## Problem
Writing array-typed MXAccess attributes through the gateway has two ergonomic
shortfalls:
1. **Asymmetric addressing.** An array attribute reads fine by its bare name
(`Obj.Arr`), but writes require the `[]` body suffix (`Obj.Arr[]`). The
handle registered from the bare name is read-capable but not cleanly
write-capable.
2. **Whole-array writes only.** Every write replaces the entire array; to change
a few elements the client must marshal and send the full array. This is a
native MXAccess COM constraint (there is no element-wise write API), but it
pushes avoidable cost onto clients for large arrays.
This design removes both frictions without breaking MXAccess parity. The worker
is not modified — it continues to perform an honest whole-array COM write. All
new behavior lives in the gateway and the contract.
## Why MXAccess forces whole-array writes
The native MXAccess COM `Write` takes a complete VARIANT (`SAFEARRAY` for
arrays). There is no `WriteArrayElement(index, value)`. Confirmed in the worker:
`VariantConverter.ConvertToComArray` marshals the entire CLR array in one shot,
and `MxAccessSession.Write` forwards it verbatim to the COM proxy. Any "partial
write" feature must therefore reconstruct a full array before the COM call.
We deliberately do **not** reconstruct it from current state (no
read-modify-write merge): that would add latency, cache-staleness, and a race
window against other writers, and would paper over MXAccess semantics. Instead
partial writes are **stateless default-fill** (see below).
## Goals
- Writing an array attribute by its bare name works like reading does — the
gateway appends `[]` automatically when it knows the attribute is an array.
- A client can send only the indices it wants plus a total length, instead of
the full array.
## Non-goals
- **No preserve-unchanged merge.** Unmentioned indices are written as the
element type's default, **not** kept at their current value.
- No element-wise COM write — MXAccess has no such API; every write is
whole-array and we keep it that way.
- No change to `ReadBulk` string addressing.
- The gateway does **not** infer total length; the client supplies it.
## Decisions (resolved during brainstorming)
| Question | Decision |
|---|---|
| Scope | Both: suffix ergonomics **and** partial writes |
| Partial-write semantics | Stateless **default-fill**: unmentioned indices = type default (reset, not preserved) |
| Total length | **Client specifies** `total_length` explicitly |
| Time/timestamp default | **Unix epoch** |
| Suffix fix location/actor | **Gateway**, using in-memory Galaxy `is_array` metadata, at **AddItem** time |
| Suffix fallback when metadata unavailable | **Pass through unchanged** (no regression) |
| Partial-write contract shape | New `MxSparseArray` as a `oneof` arm on `MxValue` |
| Per-client helpers | **Included** in this change |
## Contract changes (`mxaccess_gateway.proto`)
A write-only sparse representation, added as a `oneof kind` arm on `MxValue` so
every write command (`Write`, `Write2`, `WriteSecured`, `WriteSecured2`,
`WriteBulkEntry`) accepts it without new RPCs:
```proto
message MxSparseArray {
MxDataType element_data_type = 1;
uint32 total_length = 2;
repeated MxSparseElement elements = 3;
}
message MxSparseElement {
uint32 index = 1;
MxValue value = 2; // scalar
}
// added to MxValue oneof kind:
// MxSparseArray sparse_array_value = 19;
```
`sparse_array_value` is **write-only**: the worker never produces it, and the
gateway rejects it on any read/event path. Regenerate `Generated/` and commit
the generated `.cs` (the net48 worker build needs the checked-in types — see the
proto-codegen-regen rule).
## Suffix normalization — at `AddItem`, in the gateway
The item handle binds to the literal address string at `AddItem` and is reused
for both reads and writes; at write time only the integer handle is available,
which is too late to change the address. So normalization happens at
registration.
In the gateway's `AddItemCommand` / `AddItem2Command` handling
(`GatewaySession`), before forwarding to the worker:
1. If `item_definition` already ends with `[]` → leave unchanged.
2. Else look up `item_definition + "[]"` in the in-memory Galaxy hierarchy cache
(`IGalaxyHierarchyCache` → `GalaxyTagLookup.Attribute.IsArray`). The index is
keyed by `FullTagReference`, which already carries the `[]` suffix for
arrays, so the lookup key must include `[]`. If found and `is_array` →
rewrite `item_definition` to the `[]` form.
3. **Fallback:** metadata unavailable or address not found as an array →
forward verbatim (current behavior).
Store the **normalized** address in `SessionItemRegistration.TagAddress` so
write-time constraint checks (`ConstraintEnforcer`) and readback resolve
consistently against the `[]`-keyed index.
This is safe for reads: both the bare and `[]` forms return the array on read,
so promoting a registration to the `[]` form does not change read behavior — it
only makes the handle write-capable.
`AddItem2Command` (with `item_context`) normalizes `item_definition` the same
way. `ReadBulk` is unaffected — it uses raw address strings with its own
ephemeral registration, so bare-name reads continue to work unchanged.
## Partial-write expansion — at the gateway, worker untouched
In the gateway write path, before forwarding any write command to the worker, if
`MxValue.KindCase == SparseArrayValue`:
1. Allocate a full array of `total_length`, element type `element_data_type`.
2. Initialize every slot to the type default:
- `bool` → `false`
- `int32` / `int64` → `0`
- `float` / `double` → `0`
- `string` → `""`
- `time` / `timestamp` → Unix epoch
3. For each `MxSparseElement`, set `array[index]` from the scalar `value`.
4. Replace the `MxValue` with a normal `array_value` (full `MxArray`).
The worker then receives an ordinary whole-array `MxValue`;
`VariantConverter.ConvertToComArray` and the COM `Write` are unchanged. Parity
preserved — it really is a whole-array write.
Expansion is applied uniformly to every write variant by normalizing the
`MxValue` of each command (`Write`, `Write2`, `WriteSecured`, `WriteSecured2`,
and each `WriteBulkEntry`) before it leaves the gateway.
## Validation & errors (gateway, `InvalidArgument`)
- `total_length == 0`, or any `index >= total_length` → reject.
- Duplicate indices → reject (no silent last-wins).
- `element_data_type` must be a supported scalar element type (not `Raw` /
`Unspecified`); each element `value` must match it.
- Empty `elements` with `total_length = N` → valid: writes an all-defaults array
of length N (explicit reset).
- A sparse value arriving on a read/event path → reject (guard; the worker never
produces one).
## Clients (all five) & docs — same change
Per the repo rule that docs change with the source:
- Regenerate proto types for dotnet, go, python, rust, java. Watch the Java
generated-file churn — revert spurious protobuf-version diffs when no `.proto`
semantics changed beyond the new messages; commit the net48-relevant regen.
- Add a thin per-client helper to build a sparse write, e.g.
`WriteArrayElements(handle, totalLength, {index → value})`.
- Update the **"Array writes replace the whole array"** section in all five
client READMEs: document default-fill semantics (unmentioned = reset to
default, not preserved), the `total_length` requirement, and that bare-name
array writes now auto-normalize to the `[]` form.
- Update `gateway.md` (command/value surface) and the value-conversion doc.
## Testing
- **Gateway (FakeWorkerHarness):**
- Sparse → full expansion per element type; default-fill sizing; correct
placement of specified indices.
- `total_length == 0`, index-out-of-range, and duplicate-index rejection.
- Empty-elements all-defaults case.
- Suffix normalization: bare array → `[]`; bare scalar → unchanged;
already-`[]` → unchanged; metadata-cold → pass-through.
- **Clients:** helper + round-trip serialization per language.
- **Live MXAccess (opt-in, windev):** one default-fill write and one bare-name
array write against real COM.
## Affected components
- Contracts: `mxaccess_gateway.proto` + regenerated `Generated/`.
- Gateway: `GatewaySession` (AddItem normalization, write expansion),
`SessionItemRegistration` (store normalized address), interaction with
`IGalaxyHierarchyCache` / `GalaxyTagLookup`.
- Worker: **unchanged.**
- Clients: dotnet, go, python, rust, java (regenerated types + helper + README).
- Docs: `gateway.md`, value-conversion doc, five client READMEs.
@@ -0,0 +1,329 @@
# Array Write Ergonomics & Default-Fill Partial Writes — Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans to implement this plan task-by-task.
**Goal:** Let clients write array attributes by their bare name (gateway auto-appends `[]` at AddItem), and write a sparse, default-filled array (only the indices they care about + a total length) instead of marshalling the whole array.
**Architecture:** All new behavior lives in the **contract** and the **gateway**; the worker is untouched and keeps doing an honest whole-array COM write. The gateway intercepts outbound commands at the single choke point `GatewaySession.InvokeAsync(WorkerCommand)`: it (a) normalizes `AddItem`/`AddItem2` `item_definition` to the `[]` form when Galaxy metadata says the attribute is an array, and (b) expands an `MxSparseArray` write value into a full default-filled `MxArray` before it leaves the gateway. Partial writes are **stateless default-fill** — unmentioned indices are the type default (reset), never preserved.
**Tech Stack:** .NET 10 (gateway) / .NET Framework 4.8 x86 (worker, unchanged), protobuf + Grpc.Tools, xUnit + FakeWorkerHarness; clients in C#, Go, Python, Rust, Java.
**Design doc:** `docs/plans/2026-06-18-array-write-ergonomics-design.md`
**Key references for the implementer:**
- Choke point for all outbound commands: `src/ZB.MOM.WW.MxGateway.Server/Sessions/GatewaySession.cs:947-955` (`InvokeAsync(WorkerCommand command, ...)`). `command.Command` is the `MxCommand`.
- Handle→address tracking: `GatewaySession.TrackCommandReply` (lines 975-1014, AddItem at 989, AddItem2 at 992) → `TrackItem` (1826-1837) → `SessionItemRegistration` record (`Sessions/SessionItemRegistration.cs`). Tracking reads the **same** `MxCommand` instance that passed through `InvokeAsync`, so mutating `item_definition` there flows through automatically.
- Galaxy metadata lookup: `IGalaxyHierarchyCache.Current.Index.TagsByAddress.TryGetValue(addr, out GalaxyTagLookup)`, then `lookup.Attribute?.IsArray`. The index is keyed by `FullTagReference`, which **already contains** `[]` for arrays — look up `addr + "[]"`. See `Security/Authorization/ConstraintEnforcer.cs:15-17,197-204` for the injection + lookup pattern.
- Proto: `src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_gateway.proto` — `MxValue` (1026-1044), `MxArray` (1046-1063), `WriteCommand` (244-249), `Write2Command` (251-257), `WriteSecuredCommand`/`WriteSecured2Command`, `WriteBulk*`, `AddItemCommand` (192-195), `AddItem2Command` (197-201). Generated into `Contracts/Generated/MxaccessGateway.cs` (Compile-Removed + regenerated by Grpc.Tools).
- Gateway proto regen + commit rule (memory `project_proto_codegen_regen`): after a `.proto` edit, delete `Generated/*.cs`, rebuild contracts to regenerate, and **commit** `Generated/` or the net48 worker build fails CS0246.
- Java client gotcha (memory `project_java_generated_churn`): gradle regenerates a tracked 64k-line file with spurious protobuf-version churn — revert that churn; build/test Java on **windev** (memory `project_java_build_host`), Mac has no JRE.
---
## Task 0: Contract — add `MxSparseArray` and regenerate
**Classification:** high-risk
**Estimated implement time:** ~4 min
**Parallelizable with:** none (blocks all other tasks)
**Files:**
- Modify: `src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_gateway.proto` (MxValue oneof ~line 1043; new messages after MxArray ~line 1063)
- Regenerate + commit: `src/ZB.MOM.WW.MxGateway.Contracts/Generated/MxaccessGateway.cs`
**Step 1: Add the messages to the proto.** After the `MxArray` message (line 1063), add:
```proto
// Write-only sparse array value. The gateway expands this into a full,
// default-filled MxArray before forwarding to the worker; the worker never
// receives or produces it. Unmentioned indices take the element type's
// default (reset, NOT preserved).
message MxSparseArray {
MxDataType element_data_type = 1;
uint32 total_length = 2;
repeated MxSparseElement elements = 3;
}
message MxSparseElement {
uint32 index = 1;
MxValue value = 2; // scalar
}
```
**Step 2: Add the oneof arm to `MxValue`.** Inside the `oneof kind { ... }` block, after `bytes raw_value = 18;`:
```proto
MxSparseArray sparse_array_value = 19;
```
**Step 3: Regenerate generated code.**
Run (PowerShell on windev, or locally on Mac — .NET builds fine):
```
del src/ZB.MOM.WW.MxGateway.Contracts/Generated/*.cs
dotnet build src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csproj
```
Expected: build succeeds; `Generated/MxaccessGateway.cs` now contains `MxSparseArray`, `MxSparseElement`, and `MxValue.SparseArrayValue`.
**Step 4: Verify net10 + net48 both compile** (the worker consumes these types via net48):
```
dotnet build src/ZB.MOM.WW.MxGateway.slnx
```
Expected: PASS (no CS0246 on the new types).
**Step 5: Commit** (include regenerated `Generated/`):
```bash
git add src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_gateway.proto \
src/ZB.MOM.WW.MxGateway.Contracts/Generated/MxaccessGateway.cs
git commit -m "feat(contracts): add MxSparseArray write-only value for default-fill partial writes"
```
---
## Task 1: Gateway — `SparseArrayExpander` (pure expansion + validation)
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** Task 2, Tasks 4-9
**Files:**
- Create: `src/ZB.MOM.WW.MxGateway.Server/Sessions/SparseArrayExpander.cs`
- Test: `src/ZB.MOM.WW.MxGateway.Tests/Gateway/Sessions/SparseArrayExpanderTests.cs`
**Step 1: Write failing tests.** Cover: default-fill sizing + placement (one per element type is enough for two types here, rest in step 4); `total_length == 0` → `InvalidArgument`; index `>= total_length` → `InvalidArgument`; duplicate index → `InvalidArgument`; `Raw`/`Unspecified` element type → `InvalidArgument`; empty `elements` → all-defaults array of length N; timestamp default == Unix epoch.
```csharp
using Grpc.Core;
using Mxaccess.Gateway.V1; // adjust to the generated namespace
using ZB.MOM.WW.MxGateway.Server.Sessions;
using Xunit;
public sealed class SparseArrayExpanderTests
{
private static MxValue Sparse(MxDataType type, uint length, params (uint Index, MxValue Value)[] els)
{
MxSparseArray sparse = new() { ElementDataType = type, TotalLength = length };
foreach ((uint index, MxValue value) in els)
sparse.Elements.Add(new MxSparseElement { Index = index, Value = value });
return new MxValue { SparseArrayValue = sparse };
}
[Fact]
public void Expand_Int32_FillsDefaultsAndPlacesValues()
{
MxValue v = Sparse(MxDataType.Integer, 4, (1, new MxValue { Int32Value = 7 }));
SparseArrayExpander.Expand(v);
Assert.Equal(MxValue.KindOneofCase.ArrayValue, v.KindCase);
Assert.Equal(new[] { 0, 7, 0, 0 }, v.ArrayValue.Int32Values.Values);
Assert.Equal((uint)4, v.ArrayValue.Dimensions[0]);
}
[Fact]
public void Expand_EmptyElements_ProducesAllDefaults()
{
MxValue v = Sparse(MxDataType.Boolean, 3);
SparseArrayExpander.Expand(v);
Assert.Equal(new[] { false, false, false }, v.ArrayValue.BoolValues.Values);
}
[Theory]
[InlineData(0u, 0u)] // total_length == 0
[InlineData(2u, 5u)] // index >= total_length
public void Expand_InvalidShape_Throws(uint length, uint badIndex)
{
MxValue v = Sparse(MxDataType.Integer, length, (badIndex, new MxValue { Int32Value = 1 }));
RpcException ex = Assert.Throws<RpcException>(() => SparseArrayExpander.Expand(v));
Assert.Equal(StatusCode.InvalidArgument, ex.StatusCode);
}
[Fact]
public void Expand_DuplicateIndex_Throws()
{
MxValue v = Sparse(MxDataType.Integer, 4,
(1, new MxValue { Int32Value = 1 }), (1, new MxValue { Int32Value = 2 }));
Assert.Throws<RpcException>(() => SparseArrayExpander.Expand(v));
}
}
```
**Step 2: Run, confirm fail.** `dotnet test src/ZB.MOM.WW.MxGateway.Tests/... --filter FullyQualifiedName~SparseArrayExpanderTests` → FAIL (type not defined).
**Step 3: Implement `SparseArrayExpander`.** Mutates the passed `MxValue` in place, replacing `SparseArrayValue` with `ArrayValue`. Throw `RpcException(new Status(StatusCode.InvalidArgument, msg))` on any validation failure. Element-type switch must cover the supported scalar element types (`Boolean`, `Integer` → int32 or int64, `Float`, `Double`, `String`, `Time`); default/timestamp default = Unix epoch (`new Timestamp { Seconds = 0, Nanos = 0 }`); reject `Raw`/`Unknown`/`Unspecified`. Set `MxArray.Dimensions = { total_length }` and `ElementDataType`. Validate: `total_length > 0`, every index `< total_length`, no duplicate indices, each element `value` scalar kind matches `element_data_type`.
(Mirror the typed sub-array shapes from `VariantConverter.ConvertToComArray` in the worker so the worker's existing read path is satisfied: `Int32Values`/`Int64Values`/`BoolValues`/`FloatValues`/`DoubleValues`/`StringValues`/`TimestampValues` with their `Values` repeated fields.)
**Step 4: Add remaining element-type tests** (int64, float, double, string, time/epoch, type-mismatch element → throws). Run filter → PASS.
**Step 5: Commit.**
```bash
git add src/ZB.MOM.WW.MxGateway.Server/Sessions/SparseArrayExpander.cs \
src/ZB.MOM.WW.MxGateway.Tests/Gateway/Sessions/SparseArrayExpanderTests.cs
git commit -m "feat(gateway): add SparseArrayExpander for default-fill partial array writes"
```
---
## Task 2: Gateway — `ArrayAddressNormalizer` (suffix normalization)
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** Task 1, Tasks 4-9
**Files:**
- Create: `src/ZB.MOM.WW.MxGateway.Server/Sessions/ArrayAddressNormalizer.cs`
- Test: `src/ZB.MOM.WW.MxGateway.Tests/Gateway/Sessions/ArrayAddressNormalizerTests.cs`
**Step 1: Write failing tests** using a fake/in-memory `IGalaxyHierarchyCache` whose `Current.Index.TagsByAddress` contains `"Obj.Arr[]"` (array) and `"Obj.Scalar"` (non-array). Cases:
- `"Obj.Arr"` (bare, is array) → `"Obj.Arr[]"`.
- `"Obj.Arr[]"` (already suffixed) → unchanged.
- `"Obj.Scalar"` (non-array) → unchanged.
- `"Obj.Unknown"` (not in cache / metadata cold) → unchanged (pass-through fallback).
```csharp
[Fact]
public void Normalize_BareArrayName_AppendsSuffix()
{
ArrayAddressNormalizer normalizer = new(FakeCacheWith("Obj.Arr[]", isArray: true));
Assert.Equal("Obj.Arr[]", normalizer.Normalize("Obj.Arr"));
}
[Theory]
[InlineData("Obj.Arr[]")] // already suffixed
[InlineData("Obj.Scalar")] // non-array
[InlineData("Obj.Unknown")] // not in cache → fallback pass-through
public void Normalize_LeavesUnchanged(string address) =>
Assert.Equal(address, new ArrayAddressNormalizer(FakeCacheWith("Obj.Arr[]", true)).Normalize(address));
```
**Step 2: Run, confirm fail.**
**Step 3: Implement.** Constructor injects `IGalaxyHierarchyCache cache`. `Normalize(string)`:
1. If `string.IsNullOrWhiteSpace(address)` or `address.EndsWith("[]", StringComparison.Ordinal)` → return unchanged.
2. Look up `address + "[]"` in `cache.Current.Index.TagsByAddress`. If found and `lookup.Attribute?.IsArray == true` → return `address + "[]"`.
3. Otherwise return `address` unchanged. Never throw (best-effort convenience).
**Step 4: Run filter → PASS.**
**Step 5: Commit.**
```bash
git add src/ZB.MOM.WW.MxGateway.Server/Sessions/ArrayAddressNormalizer.cs \
src/ZB.MOM.WW.MxGateway.Tests/Gateway/Sessions/ArrayAddressNormalizerTests.cs
git commit -m "feat(gateway): add ArrayAddressNormalizer for bare-name array AddItem"
```
---
## Task 3: Gateway — wire normalization + expansion into the outbound path
**Classification:** high-risk
**Estimated implement time:** ~5 min
**Parallelizable with:** none (depends on Tasks 1, 2)
**Files:**
- Modify: `src/ZB.MOM.WW.MxGateway.Server/Sessions/GatewaySession.cs` (constructor — inject `ArrayAddressNormalizer`; `InvokeAsync` at 947-955)
- Modify: DI registration wherever `ArrayAddressNormalizer`/`GatewaySession` deps are registered (search `Security/Authorization/ConstraintEnforcer` registration for the pattern; register `ArrayAddressNormalizer` as scoped/singleton consistent with `IGalaxyHierarchyCache`)
- Test: `src/ZB.MOM.WW.MxGateway.Tests/Gateway/Sessions/GatewayArrayWriteWiringTests.cs`
**Step 1: Write a failing integration test** with `FakeWorkerHarness` (pattern: `src/ZB.MOM.WW.MxGateway.Tests/Gateway/Workers/FakeWorkerHarnessTests.cs:51-69` — `CreateConnectedPairAsync`, `ReadCommandAsync`, `ReplyToCommandAsync`). Two assertions:
1. Client sends `AddItemCommand{ item_definition = "Obj.Arr" }` (array per the test's Galaxy cache) → the `WorkerEnvelope` the fake worker reads has `item_definition == "Obj.Arr[]"`.
2. Client sends `WriteCommand{ value = MxSparseArray(Integer, 4, {1:7}) }` → the worker receives `value.ArrayValue.Int32Values.Values == [0,7,0,0]` (no `SparseArrayValue` reaches the worker).
**Step 2: Run, confirm fail.**
**Step 3: Implement.** At the top of `InvokeAsync(WorkerCommand command, ...)`, before forwarding, transform `command.Command` (the `MxCommand`) by `PayloadCase`:
- `AddItem` → `command.Command.AddItem.ItemDefinition = _addressNormalizer.Normalize(command.Command.AddItem.ItemDefinition);`
- `AddItem2` → same on `AddItem2.ItemDefinition`.
- `Write`/`WriteSecured` → if `cmd.Value?.KindCase == SparseArrayValue` call `SparseArrayExpander.Expand(cmd.Value)`.
- `Write2`/`WriteSecured2` → expand `Value` only (not `TimestampValue`).
- `WriteBulk`/`Write2Bulk`/`WriteSecuredBulk`/`WriteSecured2Bulk` → expand each `entry.Value`.
Keep it a single private helper `NormalizeOutbound(MxCommand)` called once at the choke point. Because `TrackCommandReply` later reads the **same** `MxCommand` instance, the normalized `item_definition` flows into `SessionItemRegistration` with no extra change — add an assertion in the test that `TryGetItemRegistration(...).TagAddress == "Obj.Arr[]"` to lock that in.
**Step 4: Run the wiring test + Tasks 1/2 filters → PASS.** Then run the AddItem/Write fake-worker regression group once:
```
dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter "FullyQualifiedName~ArrayWrite|FullyQualifiedName~SparseArray|FullyQualifiedName~ArrayAddressNormalizer"
```
**Step 5: Commit.**
```bash
git add src/ZB.MOM.WW.MxGateway.Server/Sessions/GatewaySession.cs \
src/ZB.MOM.WW.MxGateway.Tests/Gateway/Sessions/GatewayArrayWriteWiringTests.cs
git commit -m "feat(gateway): normalize array AddItem suffix and expand sparse writes at the worker boundary"
```
---
## Tasks 4-8: Client helpers + READMEs (one task per client, parallelizable)
Each client task does the same four things; only paths/idioms differ. **Depends on Task 0** (needs regenerated proto types). All five are parallelizable with each other and with Tasks 1-3, 9.
Common helper contract: `WriteArrayElements(serverHandle, itemHandle, elementDataType, totalLength, elements /* index→scalar MxValue */, userId)` builds an `MxValue { SparseArrayValue = MxSparseArray{...} }` and calls the existing raw write. Add a unit test that the built command carries `sparse_array_value` with the right `total_length`/indices (serialization round-trip; no live gateway). Update the **"Array writes replace the whole array"** README section to document: default-fill semantics (unmentioned = reset to default, not preserved), the required `total_length`, and that bare-name array writes now auto-normalize.
### Task 4: .NET client
**Classification:** standard · **~4 min** · **Parallelizable with:** Tasks 5-9, 1-3
- Regenerate types: `dotnet build clients/dotnet/ZB.MOM.WW.MxGateway.Client.slnx`.
- Add helper next to `WriteAsync` in `clients/dotnet/ZB.MOM.WW.MxGateway.Client/MxGatewaySession.cs:678-688`.
- Test alongside existing client tests; README `clients/dotnet/README.md:162-170`.
- Verify: build slnx + `dotnet test` the client test project.
### Task 5: Go client
**Classification:** standard · **~4 min** · **Parallelizable with:** Tasks 4,6-9, 1-3
- Regenerate per `clients/go` README; helper next to `Write`/`WriteRaw` in `clients/go/mxgateway/session.go:559-581`.
- README `clients/go/README.md:139-147`.
- Verify: `gofmt`, `go build ./...`, `go test ./...` from `clients/go`.
### Task 6: Python client
**Classification:** standard · **~4 min** · **Parallelizable with:** Tasks 4-5,7-9, 1-3
- Regenerate per `clients/python` README; helper next to `write` in `clients/python/src/zb_mom_ww_mxgateway/session.py:469-490`.
- README `clients/python/README.md:142-150`.
- Verify: `python -m pytest` from `clients/python`.
### Task 7: Rust client
**Classification:** standard · **~4 min** · **Parallelizable with:** Tasks 4-6,8-9, 1-3
- Helper next to `write` in `clients/rust/src/session.rs:530-548`; conversion helpers in `clients/rust/src/value.rs`.
- README `clients/rust/README.md:162-170`.
- Verify: `cargo fmt`, `cargo check --workspace`, `cargo test --workspace`, `cargo clippy --all-targets -- -D warnings` from `clients/rust`.
### Task 8: Java client
**Classification:** standard · **~5 min** · **Parallelizable with:** Tasks 4-7,9, 1-3
- Helper next to `write`/`writeRaw` in `clients/java/.../client/MxGatewaySession.java:581-604`.
- README `clients/java/README.md:118-126`.
- **Build/test on windev (JDK 21) via an isolated `origin/<branch>` worktree — Mac has no JRE** (memory `project_java_build_host`). After gradle regen, **revert the spurious protobuf-version churn** in `clients/java/src/main/generated/.../MxaccessGateway.java` if no proto semantics beyond the new messages changed (memory `project_java_generated_churn`); keep only the real `MxSparseArray` additions.
- Verify: `gradle test` on windev.
(Each client task ends with its own commit, e.g. `feat(client-<lang>): add WriteArrayElements default-fill helper and document semantics`.)
---
## Task 9: Docs — gateway.md + value conversion
**Classification:** small
**Estimated implement time:** ~3 min
**Parallelizable with:** Tasks 1-8 (depends on Task 0 only)
**Files:**
- Modify: `gateway.md` (command/value surface — document `MxSparseArray` as a write-only value and bare-name AddItem normalization)
- Modify: the value-conversion doc under `docs/` (search for where `MxArray`/value conversion is described) — add the default-fill + epoch-default note and the parity statement (worker still does a whole-array COM write)
**Step 1:** Add a subsection describing: `sparse_array_value` is write-only and gateway-expanded; default-fill semantics (epoch for time); `total_length` required; bare-name array writes auto-normalize to `[]` at AddItem with metadata pass-through fallback; non-goal: no preserve-unchanged merge, no element-wise COM write.
**Step 2: Commit.**
```bash
git add gateway.md docs/
git commit -m "docs: document MxSparseArray default-fill writes and bare-name array AddItem"
```
---
## Dependency summary
- **Task 0** blocks everything.
- **Tasks 1, 2** depend on 0; parallel with each other.
- **Task 3** depends on 1 and 2.
- **Tasks 4-8** depend on 0; parallel with each other and with 1-3, 9.
- **Task 9** depends on 0; parallel with all.
## Verification gates (per CLAUDE.md targeted-tests rule)
- Run only the `--filter` for the task you touched; run the array-write fake-worker group once after Task 3.
- Java verified on windev only. .NET/Go/Rust/Python verified locally.
- Live MXAccess (opt-in, windev): after merge, one default-fill write and one bare-name array write against real COM (`MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1`).
@@ -0,0 +1,16 @@
{
"planPath": "docs/plans/2026-06-18-array-write-ergonomics.md",
"tasks": [
{"id": 0, "subject": "Task 0: Contract — add MxSparseArray + regenerate", "status": "pending"},
{"id": 1, "subject": "Task 1: Gateway SparseArrayExpander", "status": "pending", "blockedBy": [0]},
{"id": 2, "subject": "Task 2: Gateway ArrayAddressNormalizer", "status": "pending", "blockedBy": [0]},
{"id": 3, "subject": "Task 3: Wire normalization + expansion into GatewaySession", "status": "pending", "blockedBy": [1, 2]},
{"id": 4, "subject": "Task 4: .NET client WriteArrayElements + README", "status": "pending", "blockedBy": [0]},
{"id": 5, "subject": "Task 5: Go client WriteArrayElements + README", "status": "pending", "blockedBy": [0]},
{"id": 6, "subject": "Task 6: Python client write_array_elements + README", "status": "pending", "blockedBy": [0]},
{"id": 7, "subject": "Task 7: Rust client write_array_elements + README", "status": "pending", "blockedBy": [0]},
{"id": 8, "subject": "Task 8: Java client writeArrayElements + README (windev)", "status": "pending", "blockedBy": [0]},
{"id": 9, "subject": "Task 9: Docs — gateway.md + value conversion", "status": "pending", "blockedBy": [0]}
],
"lastUpdated": "2026-06-18"
}
+75 -1
View File
@@ -481,6 +481,68 @@ metadata rather than dropped. If a value cannot be losslessly converted, the
worker should return both the best typed projection and enough diagnostic
metadata to reproduce the case.
### MxSparseArray — default-fill partial array writes (write-only)
`MxSparseArray` is a write-only `oneof kind` arm on `MxValue` that lets clients
send only the indices they want to change plus a total length, rather than
marshalling the entire array every write. The worker never produces or receives
it; expansion happens entirely in the gateway before the command reaches the pipe.
```protobuf
message MxSparseArray {
MxDataType element_data_type = 1;
uint32 total_length = 2;
repeated MxSparseElement elements = 3;
}
message MxSparseElement {
uint32 index = 1;
MxValue value = 2; // scalar
}
```
**Expansion.** Before forwarding any write command to the worker the gateway
allocates a full array of `total_length` slots, initializes every slot to the
element type's default, places each `MxSparseElement` at its index, then
replaces the `MxValue` with a normal `array_value` (`MxArray`). The worker
receives an ordinary whole-array write — parity is preserved.
Default values by element type:
| Element type | Default |
|---|---|
| `Boolean` | `false` |
| `Integer` | `0` (int32, or int64 when an element value is 64-bit) |
| `Float` / `Double` | `0` |
| `String` | `""` |
| `Time` | Unix epoch (1970-01-01T00:00:00Z) |
Unmentioned indices take the element type's default — this is a **reset**, not a
preserve. There is no read-modify-write merge: adding that would introduce cache
staleness, a race window against other writers, and the latency of a round-trip
read, all of which contradict MXAccess semantics.
**Validation.** The gateway rejects the following with `InvalidArgument`:
- `total_length == 0`
- any `index >= total_length`
- duplicate indices
- `element_data_type` that is `Raw` or `Unspecified`
- an element `value` whose kind does not match `element_data_type`
- `total_length` exceeds the gateway-configured maximum array length
An empty `elements` list with a non-zero `total_length` is valid — it writes an
all-defaults array of length `total_length` (explicit reset). A `sparse_array_value`
arriving on any read or event path is rejected as a guard; the worker never
produces one.
**Non-goals.** There is no preserve-unchanged read-modify-write merge, no
element-wise COM write (MXAccess has no such API), and no change to `ReadBulk`
string addressing.
`sparse_array_value` is accepted by every write variant: `Write`, `Write2`,
`WriteSecured`, `WriteSecured2`, and each `*BulkEntry` entry.
## Status Model
Represent `MXSTATUS_PROXY` explicitly:
@@ -1057,7 +1119,19 @@ Known important parity areas from existing captures:
- Writing an array attribute replaces the whole array — it is not an
element-wise patch. To change a subset of elements the caller must send the
full array (unchanged elements included); sending only the changed elements
resizes the attribute.
resizes the attribute. `MxSparseArray` provides a default-fill shorthand for
this: the gateway reconstructs the full array from the supplied sparse
representation (unmentioned indices → type default) before sending the
whole-array write to the worker.
- Array attribute addresses require the `[]` body suffix to be write-capable.
The gateway normalizes bare-name addresses at `AddItem` time: when Galaxy
metadata confirms `is_array`, the gateway appends `[]` before registering the
handle with the worker. When metadata is unavailable or the address is not
recognized as an array, the address is forwarded unchanged so existing
behavior is not regressed. The normalized address is stored in
`SessionItemRegistration.TagAddress` and applies consistently to all
subsequent writes on that handle. `ReadBulk` is unaffected — it uses raw
address strings with its own ephemeral registration.
The gateway should not "fix" these behaviors unless the client explicitly opts
into a non-parity mode.
File diff suppressed because it is too large Load Diff
@@ -1040,6 +1040,7 @@ message MxValue {
google.protobuf.Timestamp timestamp_value = 16;
MxArray array_value = 17;
bytes raw_value = 18;
MxSparseArray sparse_array_value = 19;
}
}
@@ -1062,6 +1063,21 @@ message MxArray {
}
}
// Write-only sparse array value. The gateway expands this into a full,
// default-filled MxArray before forwarding to the worker; the worker never
// receives or produces it. Unmentioned indices take the element type's
// default (reset, NOT preserved).
message MxSparseArray {
MxDataType element_data_type = 1;
uint32 total_length = 2;
repeated MxSparseElement elements = 3;
}
message MxSparseElement {
uint32 index = 1;
MxValue value = 2; // scalar
}
message BoolArray {
repeated bool values = 1;
}
@@ -196,11 +196,30 @@ public sealed class ConstraintEnforcer(
private GalaxyTagLookup? ResolveTarget(string tagAddress)
{
GalaxyHierarchyCacheEntry entry = cache.Current;
return !string.IsNullOrWhiteSpace(tagAddress)
&& entry.Index.TagsByAddress.TryGetValue(tagAddress, out GalaxyTagLookup? lookup)
? lookup
: null;
if (string.IsNullOrWhiteSpace(tagAddress))
{
return null;
}
IReadOnlyDictionary<string, GalaxyTagLookup> tagsByAddress = cache.Current.Index.TagsByAddress;
if (tagsByAddress.TryGetValue(tagAddress, out GalaxyTagLookup? lookup))
{
return lookup;
}
// Galaxy SQL keys array attributes by their suffixed FullTagReference (e.g. "Obj.Arr[]"),
// but callers pass the bare address ("Obj.Arr") before the worker-boundary normalization
// runs. Probe the suffixed form so a bare array name resolves to its array attribute,
// consistent with ArrayAddressNormalizer. Only build the suffixed string on a direct miss
// when the address is not already suffixed, and only accept it when it is truly an array.
if (!tagAddress.EndsWith("[]", StringComparison.Ordinal)
&& tagsByAddress.TryGetValue(tagAddress + "[]", out GalaxyTagLookup? arrayLookup)
&& arrayLookup.Attribute?.IsArray == true)
{
return arrayLookup;
}
return null;
}
private static bool MatchesPathOrTag(
@@ -0,0 +1,43 @@
using ZB.MOM.WW.MxGateway.Server.Galaxy;
namespace ZB.MOM.WW.MxGateway.Server.Sessions;
/// <summary>
/// Rewrites a bare MXAccess attribute address to its writable array form by appending the
/// trailing <c>[]</c> suffix when Galaxy Repository metadata reports the attribute as an array.
/// MXAccess requires the <c>[]</c> suffix on the AddItem address for an array attribute to be
/// writable; the bare name registers a read-only-ish handle. This is best-effort: when metadata
/// is cold, the address is unknown, or the attribute is not an array, the address is returned
/// unchanged and no exception is thrown.
/// </summary>
public sealed class ArrayAddressNormalizer(IGalaxyHierarchyCache cache)
{
private const string ArraySuffix = "[]";
/// <summary>
/// Returns <paramref name="address"/> with a trailing <c>[]</c> appended when Galaxy metadata
/// reports it as an array attribute; otherwise returns it unchanged. Never throws.
/// </summary>
/// <param name="address">The MXAccess attribute address to normalize.</param>
/// <returns>The normalized address, or the original address when no rewrite applies.</returns>
public string Normalize(string address)
{
if (string.IsNullOrWhiteSpace(address))
{
return address;
}
if (address.EndsWith(ArraySuffix, StringComparison.Ordinal))
{
return address;
}
// Galaxy SQL keys array attributes by their suffixed FullTagReference (e.g. "Obj.Arr[]"),
// so probe for the suffixed form to decide whether the bare name is an array.
string suffixed = address + ArraySuffix;
return cache.Current.Index.TagsByAddress.TryGetValue(suffixed, out GalaxyTagLookup? lookup)
&& lookup.Attribute?.IsArray == true
? suffixed
: address;
}
}
@@ -38,6 +38,7 @@ public sealed class GatewaySession
private Task? _dashboardMirrorTask;
private CancellationTokenSource? _dashboardMirrorCts;
private readonly Dictionary<(int ServerHandle, int ItemHandle), SessionItemRegistration> _items = [];
private readonly ArrayAddressNormalizer? _addressNormalizer;
/// <summary>
/// Initializes a gateway session with session metadata and timeout configuration.
@@ -133,6 +134,12 @@ public sealed class GatewaySession
/// fast immediately. <see cref="TimeSpan.Zero"/> (the default) disables the wait and
/// preserves the original fail-fast behavior byte-for-byte.
/// </param>
/// <param name="addressNormalizer">
/// Rewrites bare array <c>AddItem</c>/<c>AddItem2</c> addresses to their writable <c>[]</c>
/// form using Galaxy metadata at the outbound choke point (and on registration tracking).
/// When <see langword="null"/> (legacy unit-construction paths that do not exercise Galaxy
/// metadata), addresses pass through unchanged.
/// </param>
public GatewaySession(
string sessionId,
string backendName,
@@ -149,7 +156,8 @@ public sealed class GatewaySession
DateTimeOffset openedAt,
SessionEventStreaming? eventStreaming = null,
TimeSpan detachGrace = default,
TimeSpan workerReadyWaitTimeout = default)
TimeSpan workerReadyWaitTimeout = default,
ArrayAddressNormalizer? addressNormalizer = null)
{
if (string.IsNullOrWhiteSpace(sessionId))
{
@@ -189,6 +197,7 @@ public sealed class GatewaySession
_eventStreaming = eventStreaming ?? SessionEventStreaming.Default;
_detachGrace = detachGrace > TimeSpan.Zero ? detachGrace : TimeSpan.Zero;
_workerReadyWaitTimeout = workerReadyWaitTimeout > TimeSpan.Zero ? workerReadyWaitTimeout : TimeSpan.Zero;
_addressNormalizer = addressNormalizer;
}
/// <summary>
@@ -948,12 +957,95 @@ public sealed class GatewaySession
WorkerCommand command,
CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(command);
if (command.Command is not null)
{
NormalizeOutboundCommand(command.Command);
}
IWorkerClient workerClient = await GetReadyWorkerClientAsync(cancellationToken).ConfigureAwait(false);
TouchClientActivity(_eventStreaming.TimeProvider.GetUtcNow());
return await workerClient.InvokeAsync(command, CommandTimeout, cancellationToken).ConfigureAwait(false);
}
// Single outbound choke point for the two array-write ergonomics shims (Task 3):
// 1. AddItem/AddItem2 array addresses gain the writable "[]" suffix when Galaxy metadata
// reports them as arrays, so the worker registers a write-capable handle. The mutation
// lands on the same MxCommand instance forwarded to the worker.
// 2. Sparse array write values are expanded to whole-array values, because MXAccess has no
// partial-array write primitive — the worker only ever sees a full MxArray.
// SparseArrayExpander.Expand throws RpcException(InvalidArgument) for an invalid sparse payload;
// that propagates out of InvokeAsync as the desired client-facing error and is deliberately not
// caught here.
private void NormalizeOutboundCommand(MxCommand command)
{
switch (command.PayloadCase)
{
case MxCommand.PayloadOneofCase.AddItem:
command.AddItem.ItemDefinition = NormalizeAddress(command.AddItem.ItemDefinition);
break;
case MxCommand.PayloadOneofCase.AddItem2:
command.AddItem2.ItemDefinition = NormalizeAddress(command.AddItem2.ItemDefinition);
break;
case MxCommand.PayloadOneofCase.Write:
ExpandValue(command.Write.Value);
break;
case MxCommand.PayloadOneofCase.WriteSecured:
ExpandValue(command.WriteSecured.Value);
break;
case MxCommand.PayloadOneofCase.Write2:
ExpandValue(command.Write2.Value);
break;
case MxCommand.PayloadOneofCase.WriteSecured2:
ExpandValue(command.WriteSecured2.Value);
break;
case MxCommand.PayloadOneofCase.WriteBulk:
foreach (WriteBulkEntry entry in command.WriteBulk.Entries)
{
ExpandValue(entry.Value);
}
break;
case MxCommand.PayloadOneofCase.Write2Bulk:
foreach (Write2BulkEntry entry in command.Write2Bulk.Entries)
{
ExpandValue(entry.Value);
}
break;
case MxCommand.PayloadOneofCase.WriteSecuredBulk:
foreach (WriteSecuredBulkEntry entry in command.WriteSecuredBulk.Entries)
{
ExpandValue(entry.Value);
}
break;
case MxCommand.PayloadOneofCase.WriteSecured2Bulk:
foreach (WriteSecured2BulkEntry entry in command.WriteSecured2Bulk.Entries)
{
ExpandValue(entry.Value);
}
break;
}
}
// Best-effort array-suffix rewrite; the normalizer is null in legacy unit-construction paths
// that do not exercise Galaxy metadata, in which case the address passes through unchanged.
private string NormalizeAddress(string address) =>
_addressNormalizer?.Normalize(address) ?? address;
// MXAccess writes replace the whole array; expand a sparse value in place so the worker only
// ever receives a whole-array MxValue. No-op for null or non-sparse values.
private static void ExpandValue(MxValue? value)
{
if (value is not null)
{
SparseArrayExpander.Expand(value);
}
}
/// <summary>Gets the item registration for a server and item handle pair.</summary>
/// <param name="serverHandle">The MXAccess server handle.</param>
/// <param name="itemHandle">The MXAccess item handle.</param>
@@ -985,11 +1077,16 @@ public sealed class GatewaySession
{
switch (command.Kind)
{
// The public reply is tracked from the pre-mapping MxCommand instance, which is a
// separate copy from the one mutated at the InvokeAsync choke point (the gRPC mapper
// deep-clones before forwarding). Re-apply the array-suffix normalization here so the
// registration's TagAddress matches the address the worker actually registered.
// Normalize is idempotent for an already-suffixed address.
case MxCommandKind.AddItem when reply.AddItem is not null:
TrackItem(command.AddItem.ServerHandle, reply.AddItem.ItemHandle, command.AddItem.ItemDefinition);
TrackItem(command.AddItem.ServerHandle, reply.AddItem.ItemHandle, NormalizeAddress(command.AddItem.ItemDefinition));
break;
case MxCommandKind.AddItem2 when reply.AddItem2 is not null:
TrackItem(command.AddItem2.ServerHandle, reply.AddItem2.ItemHandle, command.AddItem2.ItemDefinition);
TrackItem(command.AddItem2.ServerHandle, reply.AddItem2.ItemHandle, NormalizeAddress(command.AddItem2.ItemDefinition));
break;
case MxCommandKind.AddBufferedItem when reply.AddBufferedItem is not null:
TrackItem(command.AddBufferedItem.ServerHandle, reply.AddBufferedItem.ItemHandle, command.AddBufferedItem.ItemDefinition);
@@ -29,6 +29,7 @@ public sealed class SessionManager : ISessionManager
private readonly Grpc.MxAccessGrpcMapper _eventMapper;
private readonly ILogger<SessionEventDistributor> _distributorLogger;
private readonly Dashboard.Hubs.IDashboardEventBroadcaster? _dashboardEventBroadcaster;
private readonly ArrayAddressNormalizer? _addressNormalizer;
/// <summary>
/// Initializes a new instance of <see cref="SessionManager"/>.
@@ -47,6 +48,11 @@ public sealed class SessionManager : ISessionManager
/// dashboard receives events regardless of whether a gRPC client is streaming. Null in
/// unit tests that do not exercise the dashboard mirror.
/// </param>
/// <param name="addressNormalizer">
/// Rewrites bare array AddItem addresses to their writable <c>[]</c> form using Galaxy
/// metadata; handed to each session so the normalization runs at the outbound choke point.
/// Null in unit tests that do not exercise array-write ergonomics.
/// </param>
public SessionManager(
ISessionRegistry registry,
ISessionWorkerClientFactory workerClientFactory,
@@ -56,7 +62,8 @@ public sealed class SessionManager : ISessionManager
ILogger<SessionManager>? logger = null,
Grpc.MxAccessGrpcMapper? eventMapper = null,
ILogger<SessionEventDistributor>? distributorLogger = null,
Dashboard.Hubs.IDashboardEventBroadcaster? dashboardEventBroadcaster = null)
Dashboard.Hubs.IDashboardEventBroadcaster? dashboardEventBroadcaster = null,
ArrayAddressNormalizer? addressNormalizer = null)
{
_registry = registry ?? throw new ArgumentNullException(nameof(registry));
_workerClientFactory = workerClientFactory ?? throw new ArgumentNullException(nameof(workerClientFactory));
@@ -67,6 +74,7 @@ public sealed class SessionManager : ISessionManager
_eventMapper = eventMapper ?? new Grpc.MxAccessGrpcMapper();
_distributorLogger = distributorLogger ?? NullLogger<SessionEventDistributor>.Instance;
_dashboardEventBroadcaster = dashboardEventBroadcaster;
_addressNormalizer = addressNormalizer;
_options = options.Value;
_sessionSlots = new SemaphoreSlim(_options.Sessions.MaxSessions, _options.Sessions.MaxSessions);
}
@@ -506,7 +514,8 @@ public sealed class SessionManager : ISessionManager
openedAt,
eventStreaming,
TimeSpan.FromSeconds(Math.Max(0, _options.Sessions.DetachGraceSeconds)),
TimeSpan.FromMilliseconds(Math.Max(0, _options.Sessions.WorkerReadyWaitTimeoutMs)));
TimeSpan.FromMilliseconds(Math.Max(0, _options.Sessions.WorkerReadyWaitTimeoutMs)),
_addressNormalizer);
}
private static string CreateClientCorrelationId(
@@ -8,6 +8,9 @@ public static class SessionServiceCollectionExtensions
/// <returns>The service collection for chaining.</returns>
public static IServiceCollection AddGatewaySessions(this IServiceCollection services)
{
// Lifetime consistent with IGalaxyHierarchyCache (singleton); the normalizer reads the
// cache's current snapshot per call, so it holds no per-session or per-request state.
services.AddSingleton<ArrayAddressNormalizer>();
services.AddSingleton<ISessionRegistry, SessionRegistry>();
services.AddSingleton<ISessionWorkerClientFactory, SessionWorkerClientFactory>();
services.AddSingleton<ISessionManager, SessionManager>();
@@ -0,0 +1,285 @@
using Google.Protobuf.WellKnownTypes;
using Grpc.Core;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Server.Sessions;
/// <summary>
/// Expands a client-supplied sparse array write (<see cref="MxSparseArray"/>) into a
/// full, default-filled <see cref="MxArray"/> in place.
/// </summary>
/// <remarks>
/// MXAccess has no partial-array write primitive: a write replaces the whole array.
/// Clients that only care about a few indices send an <see cref="MxSparseArray"/> with
/// the total length plus the indices they want to set; this expander materializes the
/// full array so the worker can do an ordinary whole-array COM write. Indices the client
/// did not mention are reset to the element type's default (they are NOT preserved from
/// the live value); this is intentional, because the gateway cannot read-modify-write
/// without racing the provider.
///
/// For the MXAccess <c>Integer</c> element type the worker's COM-array converter (see
/// <c>VariantConverter.ConvertArray</c>) chooses between a 32-bit and 64-bit sub-array
/// based on the CLR element type. The sparse value carries no CLR array, so this expander
/// mirrors that choice by inspecting the supplied element value kinds: if any element is an
/// <see cref="MxValue.KindOneofCase.Int64Value"/> the whole array is emitted as
/// <see cref="MxArray.Int64Values"/>; otherwise it is emitted as
/// <see cref="MxArray.Int32Values"/> (matching a default Integer array).
/// </remarks>
internal static class SparseArrayExpander
{
/// <summary>
/// Replaces <paramref name="value"/>'s <see cref="MxValue.SparseArrayValue"/> with an
/// equivalent full <see cref="MxValue.ArrayValue"/>. If <paramref name="value"/> is not
/// a sparse array this is a no-op, so callers may invoke it unconditionally.
/// </summary>
/// <param name="value">The value to expand in place.</param>
/// <exception cref="RpcException">
/// <see cref="StatusCode.InvalidArgument"/> when the sparse payload is invalid: zero
/// total length, an index at or beyond the total length, a duplicate index, an
/// unsupported element type, or an element value whose kind does not match the declared
/// element type.
/// </exception>
public static void Expand(MxValue value)
{
ArgumentNullException.ThrowIfNull(value);
if (value.KindCase != MxValue.KindOneofCase.SparseArrayValue)
{
return;
}
MxSparseArray sparse = value.SparseArrayValue;
MxDataType elementType = sparse.ElementDataType;
uint totalLength = sparse.TotalLength;
if (totalLength == 0)
{
throw Invalid("Sparse array total_length must be greater than zero.");
}
if (!IsSupportedElementType(elementType))
{
throw Invalid($"Sparse array element_data_type '{elementType}' is not a supported scalar element type.");
}
if (totalLength > (uint)Array.MaxLength)
{
throw Invalid(
$"Sparse array total_length {totalLength} exceeds the maximum supported array length {Array.MaxLength}.");
}
int length = (int)totalLength;
HashSet<uint> seenIndices = new();
foreach (MxSparseElement element in sparse.Elements)
{
if (element.Index >= totalLength)
{
throw Invalid(
$"Sparse array index {element.Index} is out of range for total_length {totalLength}.");
}
if (!seenIndices.Add(element.Index))
{
throw Invalid($"Sparse array has a duplicate index {element.Index}.");
}
ValidateElementKind(elementType, element);
}
MxArray array = BuildArray(elementType, length, sparse.Elements);
array.ElementDataType = elementType;
array.Dimensions.Add(totalLength);
// Assigning ArrayValue switches the oneof and clears SparseArrayValue.
value.ArrayValue = array;
}
private static MxArray BuildArray(
MxDataType elementType,
int length,
IReadOnlyList<MxSparseElement> elements)
{
MxArray array = new();
switch (elementType)
{
case MxDataType.Boolean:
{
BoolArray values = new();
for (int i = 0; i < length; i++)
{
values.Values.Add(false);
}
foreach (MxSparseElement element in elements)
{
values.Values[(int)element.Index] = element.Value.BoolValue;
}
array.BoolValues = values;
break;
}
case MxDataType.Integer when UsesInt64(elements):
{
Int64Array values = new();
for (int i = 0; i < length; i++)
{
values.Values.Add(0L);
}
foreach (MxSparseElement element in elements)
{
values.Values[(int)element.Index] = ReadInt64(element.Value);
}
array.Int64Values = values;
break;
}
case MxDataType.Integer:
{
Int32Array values = new();
for (int i = 0; i < length; i++)
{
values.Values.Add(0);
}
foreach (MxSparseElement element in elements)
{
values.Values[(int)element.Index] = element.Value.Int32Value;
}
array.Int32Values = values;
break;
}
case MxDataType.Float:
{
FloatArray values = new();
for (int i = 0; i < length; i++)
{
values.Values.Add(0f);
}
foreach (MxSparseElement element in elements)
{
values.Values[(int)element.Index] = element.Value.FloatValue;
}
array.FloatValues = values;
break;
}
case MxDataType.Double:
{
DoubleArray values = new();
for (int i = 0; i < length; i++)
{
values.Values.Add(0d);
}
foreach (MxSparseElement element in elements)
{
values.Values[(int)element.Index] = element.Value.DoubleValue;
}
array.DoubleValues = values;
break;
}
case MxDataType.String:
{
StringArray values = new();
for (int i = 0; i < length; i++)
{
values.Values.Add(string.Empty);
}
foreach (MxSparseElement element in elements)
{
values.Values[(int)element.Index] = element.Value.StringValue;
}
array.StringValues = values;
break;
}
case MxDataType.Time:
{
TimestampArray values = new();
for (int i = 0; i < length; i++)
{
values.Values.Add(new Timestamp { Seconds = 0, Nanos = 0 });
}
foreach (MxSparseElement element in elements)
{
values.Values[(int)element.Index] = element.Value.TimestampValue;
}
array.TimestampValues = values;
break;
}
default:
// Unreachable: IsSupportedElementType gates the element type before BuildArray.
throw Invalid($"Sparse array element_data_type '{elementType}' is not supported.");
}
return array;
}
private static bool IsSupportedElementType(MxDataType elementType) => elementType switch
{
MxDataType.Boolean => true,
MxDataType.Integer => true,
MxDataType.Float => true,
MxDataType.Double => true,
MxDataType.String => true,
MxDataType.Time => true,
_ => false,
};
private static void ValidateElementKind(MxDataType elementType, MxSparseElement element)
{
MxValue.KindOneofCase kind = element.Value?.KindCase ?? MxValue.KindOneofCase.None;
bool matches = elementType switch
{
MxDataType.Boolean => kind == MxValue.KindOneofCase.BoolValue,
MxDataType.Integer => kind is MxValue.KindOneofCase.Int32Value or MxValue.KindOneofCase.Int64Value,
MxDataType.Float => kind == MxValue.KindOneofCase.FloatValue,
MxDataType.Double => kind == MxValue.KindOneofCase.DoubleValue,
MxDataType.String => kind == MxValue.KindOneofCase.StringValue,
MxDataType.Time => kind == MxValue.KindOneofCase.TimestampValue,
_ => false,
};
if (!matches)
{
throw Invalid(
$"Sparse array element at index {element.Index} has value kind '{kind}' which does not match element_data_type '{elementType}'.");
}
}
private static bool UsesInt64(IReadOnlyList<MxSparseElement> elements)
{
foreach (MxSparseElement element in elements)
{
if (element.Value.KindCase == MxValue.KindOneofCase.Int64Value)
{
return true;
}
}
return false;
}
private static long ReadInt64(MxValue value) =>
value.KindCase == MxValue.KindOneofCase.Int64Value ? value.Int64Value : value.Int32Value;
private static RpcException Invalid(string message) =>
new(new Status(StatusCode.InvalidArgument, message));
}
@@ -0,0 +1,105 @@
using ZB.MOM.WW.MxGateway.Contracts.Proto.Galaxy;
using ZB.MOM.WW.MxGateway.Server.Galaxy;
using ZB.MOM.WW.MxGateway.Server.Sessions;
namespace ZB.MOM.WW.MxGateway.Tests.Gateway.Sessions;
public sealed class ArrayAddressNormalizerTests
{
/// <summary>Verifies a bare array attribute name gains the trailing array suffix.</summary>
[Fact]
public void Normalize_BareArrayName_AppendsArraySuffix()
{
ArrayAddressNormalizer normalizer = CreateNormalizer();
Assert.Equal("Obj.Arr[]", normalizer.Normalize("Obj.Arr"));
}
/// <summary>Verifies an already-suffixed address is returned unchanged.</summary>
[Fact]
public void Normalize_AlreadySuffixed_ReturnsUnchanged()
{
ArrayAddressNormalizer normalizer = CreateNormalizer();
Assert.Equal("Obj.Arr[]", normalizer.Normalize("Obj.Arr[]"));
}
/// <summary>Verifies a scalar attribute is returned unchanged.</summary>
[Fact]
public void Normalize_ScalarAttribute_ReturnsUnchanged()
{
ArrayAddressNormalizer normalizer = CreateNormalizer();
Assert.Equal("Obj.Scalar", normalizer.Normalize("Obj.Scalar"));
}
/// <summary>Verifies an address absent from the cache is returned unchanged.</summary>
[Fact]
public void Normalize_UnknownAddress_ReturnsUnchanged()
{
ArrayAddressNormalizer normalizer = CreateNormalizer();
Assert.Equal("Obj.Unknown", normalizer.Normalize("Obj.Unknown"));
}
/// <summary>Verifies null, empty, and whitespace addresses are returned unchanged.</summary>
[Theory]
[InlineData("")]
[InlineData(" ")]
public void Normalize_BlankAddress_ReturnsUnchanged(string address)
{
ArrayAddressNormalizer normalizer = CreateNormalizer();
Assert.Equal(address, normalizer.Normalize(address));
}
private static ArrayAddressNormalizer CreateNormalizer()
{
IReadOnlyList<GalaxyObject> objects =
[
new GalaxyObject
{
GobjectId = 1,
TagName = "Obj",
ContainedName = "Obj",
Attributes =
{
new GalaxyAttribute
{
AttributeName = "Arr",
// Galaxy SQL already appends "[]" to array attribute references.
FullTagReference = "Obj.Arr[]",
IsArray = true,
},
new GalaxyAttribute
{
AttributeName = "Scalar",
FullTagReference = "Obj.Scalar",
IsArray = false,
},
},
},
];
GalaxyHierarchyCacheEntry entry = GalaxyHierarchyCacheEntry.Empty with
{
Status = GalaxyCacheStatus.Healthy,
Objects = objects,
Index = GalaxyHierarchyIndex.Build(objects),
};
return new ArrayAddressNormalizer(new StubGalaxyHierarchyCache(entry));
}
private sealed class StubGalaxyHierarchyCache(GalaxyHierarchyCacheEntry current) : IGalaxyHierarchyCache
{
/// <summary>Gets the current cache entry.</summary>
public GalaxyHierarchyCacheEntry Current { get; } = current;
/// <inheritdoc />
public Task RefreshAsync(CancellationToken cancellationToken) => Task.CompletedTask;
/// <inheritdoc />
public Task WaitForFirstLoadAsync(CancellationToken cancellationToken) => Task.CompletedTask;
}
}
@@ -0,0 +1,373 @@
using System.Runtime.CompilerServices;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
using ZB.MOM.WW.MxGateway.Contracts.Proto.Galaxy;
using ZB.MOM.WW.MxGateway.Server.Galaxy;
using ZB.MOM.WW.MxGateway.Server.Sessions;
using ZB.MOM.WW.MxGateway.Server.Workers;
namespace ZB.MOM.WW.MxGateway.Tests.Gateway.Sessions;
/// <summary>
/// Integration coverage for the single outbound choke point
/// (<see cref="GatewaySession.InvokeAsync(WorkerCommand, System.Threading.CancellationToken)"/>):
/// array <c>AddItem</c> addresses gain the writable <c>[]</c> suffix and sparse array writes are
/// expanded to whole-array values before any command reaches the worker.
/// </summary>
public sealed class GatewayArrayWriteWiringTests
{
/// <summary>
/// A bare array <c>AddItem</c> address is normalized to its writable array form on the wire,
/// and the normalized address lands in the tracked <see cref="SessionItemRegistration"/>.
/// </summary>
[Fact]
public async Task AddItem_BareArrayAddress_NormalizedOnWireAndInRegistration()
{
CapturingWorkerClient worker = new();
GatewaySession session = CreateReadySession(worker);
WorkerCommand command = new()
{
Command = new MxCommand
{
Kind = MxCommandKind.AddItem,
AddItem = new AddItemCommand
{
ServerHandle = 1,
ItemDefinition = "Obj.Arr",
},
},
};
worker.NextReply = new WorkerCommandReply
{
Reply = new MxCommandReply
{
Kind = MxCommandKind.AddItem,
ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok },
AddItem = new AddItemReply { ItemHandle = 42 },
},
};
await session.InvokeAsync(command, CancellationToken.None);
Assert.NotNull(worker.LastCommand);
Assert.Equal("Obj.Arr[]", worker.LastCommand!.Command.AddItem.ItemDefinition);
// Track the reply through the same path the gRPC service uses; the registration must carry
// the normalized address even though the public reply is tracked from a separate command copy.
MxCommand trackingCopy = new()
{
Kind = MxCommandKind.AddItem,
AddItem = new AddItemCommand
{
ServerHandle = 1,
ItemDefinition = "Obj.Arr",
},
};
session.TrackCommandReply(trackingCopy, worker.NextReply.Reply);
Assert.True(session.TryGetItemRegistration(1, 42, out SessionItemRegistration registration));
Assert.Equal("Obj.Arr[]", registration.TagAddress);
}
/// <summary>A bare scalar <c>AddItem</c> address is forwarded unchanged.</summary>
[Fact]
public async Task AddItem_ScalarAddress_ForwardedUnchanged()
{
CapturingWorkerClient worker = new();
GatewaySession session = CreateReadySession(worker);
WorkerCommand command = new()
{
Command = new MxCommand
{
Kind = MxCommandKind.AddItem,
AddItem = new AddItemCommand
{
ServerHandle = 1,
ItemDefinition = "Obj.Scalar",
},
},
};
await session.InvokeAsync(command, CancellationToken.None);
Assert.Equal("Obj.Scalar", worker.LastCommand!.Command.AddItem.ItemDefinition);
}
/// <summary>
/// A sparse-array <see cref="WriteCommand"/> value is expanded to a full, default-filled
/// <see cref="MxArray"/> before reaching the worker; no sparse value is ever forwarded.
/// </summary>
[Fact]
public async Task Write_SparseArrayValue_ExpandedBeforeReachingWorker()
{
CapturingWorkerClient worker = new();
GatewaySession session = CreateReadySession(worker);
WorkerCommand command = new()
{
Command = new MxCommand
{
Kind = MxCommandKind.Write,
Write = new WriteCommand
{
ServerHandle = 1,
ItemHandle = 42,
Value = new MxValue
{
SparseArrayValue = new MxSparseArray
{
ElementDataType = MxDataType.Integer,
TotalLength = 4,
Elements =
{
new MxSparseElement
{
Index = 1,
Value = new MxValue { Int32Value = 7 },
},
},
},
},
},
},
};
await session.InvokeAsync(command, CancellationToken.None);
MxValue forwarded = worker.LastCommand!.Command.Write.Value;
Assert.Equal(MxValue.KindOneofCase.ArrayValue, forwarded.KindCase);
Assert.Equal(new[] { 0, 7, 0, 0 }, forwarded.ArrayValue.Int32Values.Values);
}
/// <summary>
/// A bare array <c>AddItem2</c> address is normalized to its writable array form on the wire,
/// and the normalized address lands in the tracked <see cref="SessionItemRegistration"/>.
/// </summary>
[Fact]
public async Task AddItem2_BareArrayAddress_NormalizedOnWireAndInRegistration()
{
CapturingWorkerClient worker = new();
GatewaySession session = CreateReadySession(worker);
WorkerCommand command = new()
{
Command = new MxCommand
{
Kind = MxCommandKind.AddItem2,
AddItem2 = new AddItem2Command
{
ServerHandle = 1,
ItemDefinition = "Obj.Arr",
},
},
};
worker.NextReply = new WorkerCommandReply
{
Reply = new MxCommandReply
{
Kind = MxCommandKind.AddItem2,
ProtocolStatus = new ProtocolStatus { Code = ProtocolStatusCode.Ok },
AddItem2 = new AddItem2Reply { ItemHandle = 43 },
},
};
await session.InvokeAsync(command, CancellationToken.None);
Assert.NotNull(worker.LastCommand);
Assert.Equal("Obj.Arr[]", worker.LastCommand!.Command.AddItem2.ItemDefinition);
MxCommand trackingCopy = new()
{
Kind = MxCommandKind.AddItem2,
AddItem2 = new AddItem2Command
{
ServerHandle = 1,
ItemDefinition = "Obj.Arr",
},
};
session.TrackCommandReply(trackingCopy, worker.NextReply.Reply);
Assert.True(session.TryGetItemRegistration(1, 43, out SessionItemRegistration registration));
Assert.Equal("Obj.Arr[]", registration.TagAddress);
}
/// <summary>
/// A sparse-array entry in a <see cref="WriteBulkCommand"/> is expanded to a full,
/// default-filled <see cref="MxArray"/> before reaching the worker; no sparse value is ever
/// forwarded inside a bulk write.
/// </summary>
[Fact]
public async Task WriteBulk_SparseArrayEntryValue_ExpandedBeforeReachingWorker()
{
CapturingWorkerClient worker = new();
GatewaySession session = CreateReadySession(worker);
WorkerCommand command = new()
{
Command = new MxCommand
{
Kind = MxCommandKind.WriteBulk,
WriteBulk = new WriteBulkCommand
{
ServerHandle = 1,
Entries =
{
new WriteBulkEntry
{
ItemHandle = 42,
Value = new MxValue
{
SparseArrayValue = new MxSparseArray
{
ElementDataType = MxDataType.Integer,
TotalLength = 4,
Elements =
{
new MxSparseElement
{
Index = 1,
Value = new MxValue { Int32Value = 7 },
},
},
},
},
},
},
},
},
};
await session.InvokeAsync(command, CancellationToken.None);
MxValue forwarded = worker.LastCommand!.Command.WriteBulk.Entries[0].Value;
Assert.Equal(MxValue.KindOneofCase.ArrayValue, forwarded.KindCase);
Assert.Equal(new[] { 0, 7, 0, 0 }, forwarded.ArrayValue.Int32Values.Values);
}
private static GatewaySession CreateReadySession(IWorkerClient workerClient)
{
GatewaySession session = new(
sessionId: "session-array-write-wiring",
backendName: "mxaccess",
pipeName: "mxaccess-gateway-1-session-array-write-wiring",
nonce: "nonce",
clientIdentity: "client-1",
ownerKeyId: null,
clientSessionName: "test-session",
clientCorrelationId: "client-correlation-1",
commandTimeout: TimeSpan.FromSeconds(5),
startupTimeout: TimeSpan.FromSeconds(5),
shutdownTimeout: TimeSpan.FromSeconds(5),
leaseDuration: TimeSpan.FromMinutes(30),
openedAt: DateTimeOffset.UtcNow,
addressNormalizer: CreateNormalizer());
session.AttachWorkerClient(workerClient);
session.MarkReady();
return session;
}
private static ArrayAddressNormalizer CreateNormalizer()
{
IReadOnlyList<GalaxyObject> objects =
[
new GalaxyObject
{
GobjectId = 1,
TagName = "Obj",
ContainedName = "Obj",
Attributes =
{
new GalaxyAttribute
{
AttributeName = "Arr",
FullTagReference = "Obj.Arr[]",
IsArray = true,
},
new GalaxyAttribute
{
AttributeName = "Scalar",
FullTagReference = "Obj.Scalar",
IsArray = false,
},
},
},
];
GalaxyHierarchyCacheEntry entry = GalaxyHierarchyCacheEntry.Empty with
{
Status = GalaxyCacheStatus.Healthy,
Objects = objects,
Index = GalaxyHierarchyIndex.Build(objects),
};
return new ArrayAddressNormalizer(new StubGalaxyHierarchyCache(entry));
}
private sealed class StubGalaxyHierarchyCache(GalaxyHierarchyCacheEntry current) : IGalaxyHierarchyCache
{
/// <summary>Gets the current cache entry.</summary>
public GalaxyHierarchyCacheEntry Current { get; } = current;
/// <inheritdoc />
public Task RefreshAsync(CancellationToken cancellationToken) => Task.CompletedTask;
/// <inheritdoc />
public Task WaitForFirstLoadAsync(CancellationToken cancellationToken) => Task.CompletedTask;
}
private sealed class CapturingWorkerClient : IWorkerClient
{
/// <summary>Gets the most recent command forwarded to the worker.</summary>
public WorkerCommand? LastCommand { get; private set; }
/// <summary>Gets or sets the reply returned by the next invocation.</summary>
public WorkerCommandReply NextReply { get; set; } = new();
/// <summary>Gets the session identifier.</summary>
public string SessionId { get; } = "session-array-write-wiring";
/// <summary>Gets the worker process identifier.</summary>
public int? ProcessId { get; } = 1234;
/// <summary>Gets the worker client state.</summary>
public WorkerClientState State { get; } = WorkerClientState.Ready;
/// <summary>Gets the last recorded heartbeat timestamp.</summary>
public DateTimeOffset LastHeartbeatAt { get; } = DateTimeOffset.UtcNow;
/// <inheritdoc />
public Task StartAsync(CancellationToken cancellationToken) => Task.CompletedTask;
/// <inheritdoc />
public Task<WorkerCommandReply> InvokeAsync(
WorkerCommand command,
TimeSpan timeout,
CancellationToken cancellationToken)
{
LastCommand = command;
return Task.FromResult(NextReply);
}
/// <inheritdoc />
public async IAsyncEnumerable<WorkerEvent> ReadEventsAsync(
[EnumeratorCancellation] CancellationToken cancellationToken)
{
await Task.CompletedTask.ConfigureAwait(false);
yield break;
}
/// <inheritdoc />
public Task ShutdownAsync(TimeSpan timeout, CancellationToken cancellationToken) => Task.CompletedTask;
/// <inheritdoc />
public void Kill(string reason)
{
}
/// <inheritdoc />
public ValueTask DisposeAsync() => ValueTask.CompletedTask;
}
}
@@ -0,0 +1,210 @@
using Google.Protobuf.WellKnownTypes;
using Grpc.Core;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
using ZB.MOM.WW.MxGateway.Server.Sessions;
namespace ZB.MOM.WW.MxGateway.Tests.Gateway.Sessions;
public sealed class SparseArrayExpanderTests
{
private static MxValue SparseValue(
MxDataType elementType,
uint totalLength,
params (uint Index, MxValue Value)[] elements)
{
MxSparseArray sparse = new()
{
ElementDataType = elementType,
TotalLength = totalLength,
};
foreach ((uint index, MxValue value) in elements)
{
sparse.Elements.Add(new MxSparseElement { Index = index, Value = value });
}
return new MxValue { SparseArrayValue = sparse };
}
[Fact]
public void Expand_Int32_FillsDefaultsAndSetsElement()
{
MxValue value = SparseValue(
MxDataType.Integer,
4,
(1, new MxValue { Int32Value = 7 }));
SparseArrayExpander.Expand(value);
Assert.Equal(MxValue.KindOneofCase.ArrayValue, value.KindCase);
Assert.Equal(MxDataType.Integer, value.ArrayValue.ElementDataType);
Assert.Equal(new uint[] { 4 }, value.ArrayValue.Dimensions);
Assert.Equal(MxArray.ValuesOneofCase.Int32Values, value.ArrayValue.ValuesCase);
Assert.Equal(new[] { 0, 7, 0, 0 }, value.ArrayValue.Int32Values.Values);
}
[Fact]
public void Expand_Boolean_EmptyElements_AllDefaultFalse()
{
MxValue value = SparseValue(MxDataType.Boolean, 3);
SparseArrayExpander.Expand(value);
Assert.Equal(MxArray.ValuesOneofCase.BoolValues, value.ArrayValue.ValuesCase);
Assert.Equal(new[] { false, false, false }, value.ArrayValue.BoolValues.Values);
}
[Fact]
public void Expand_ZeroTotalLength_Throws()
{
MxValue value = SparseValue(MxDataType.Integer, 0);
RpcException ex = Assert.Throws<RpcException>(() => SparseArrayExpander.Expand(value));
Assert.Equal(StatusCode.InvalidArgument, ex.StatusCode);
}
[Fact]
public void Expand_IndexOutOfRange_Throws()
{
MxValue value = SparseValue(
MxDataType.Integer,
2,
(5, new MxValue { Int32Value = 1 }));
RpcException ex = Assert.Throws<RpcException>(() => SparseArrayExpander.Expand(value));
Assert.Equal(StatusCode.InvalidArgument, ex.StatusCode);
}
[Fact]
public void Expand_DuplicateIndex_Throws()
{
MxValue value = SparseValue(
MxDataType.Integer,
4,
(1, new MxValue { Int32Value = 1 }),
(1, new MxValue { Int32Value = 2 }));
RpcException ex = Assert.Throws<RpcException>(() => SparseArrayExpander.Expand(value));
Assert.Equal(StatusCode.InvalidArgument, ex.StatusCode);
}
[Fact]
public void Expand_UnsupportedElementType_Throws()
{
MxValue value = SparseValue(MxDataType.Unspecified, 2);
RpcException ex = Assert.Throws<RpcException>(() => SparseArrayExpander.Expand(value));
Assert.Equal(StatusCode.InvalidArgument, ex.StatusCode);
}
[Fact]
public void Expand_ElementValueKindMismatch_Throws()
{
MxValue value = SparseValue(
MxDataType.Integer,
2,
(0, new MxValue { StringValue = "nope" }));
RpcException ex = Assert.Throws<RpcException>(() => SparseArrayExpander.Expand(value));
Assert.Equal(StatusCode.InvalidArgument, ex.StatusCode);
}
[Fact]
public void Expand_String_FillsEmptyStringDefault()
{
MxValue value = SparseValue(
MxDataType.String,
2,
(0, new MxValue { StringValue = "a" }));
SparseArrayExpander.Expand(value);
Assert.Equal(MxArray.ValuesOneofCase.StringValues, value.ArrayValue.ValuesCase);
Assert.Equal(new[] { "a", string.Empty }, value.ArrayValue.StringValues.Values);
}
[Fact]
public void Expand_Time_FillsEpochDefault()
{
MxValue value = SparseValue(
MxDataType.Time,
2,
(1, new MxValue { TimestampValue = new Timestamp { Seconds = 5 } }));
SparseArrayExpander.Expand(value);
Assert.Equal(MxArray.ValuesOneofCase.TimestampValues, value.ArrayValue.ValuesCase);
Assert.Equal(2, value.ArrayValue.TimestampValues.Values.Count);
Assert.Equal(0, value.ArrayValue.TimestampValues.Values[0].Seconds);
Assert.Equal(0, value.ArrayValue.TimestampValues.Values[0].Nanos);
Assert.Equal(5, value.ArrayValue.TimestampValues.Values[1].Seconds);
}
[Fact]
public void Expand_Double_HappyPath()
{
MxValue value = SparseValue(
MxDataType.Double,
3,
(2, new MxValue { DoubleValue = 1.5 }));
SparseArrayExpander.Expand(value);
Assert.Equal(MxArray.ValuesOneofCase.DoubleValues, value.ArrayValue.ValuesCase);
Assert.Equal(new[] { 0d, 0d, 1.5 }, value.ArrayValue.DoubleValues.Values);
}
[Fact]
public void Expand_Float_HappyPath()
{
MxValue value = SparseValue(
MxDataType.Float,
2,
(0, new MxValue { FloatValue = 2.5f }));
SparseArrayExpander.Expand(value);
Assert.Equal(MxArray.ValuesOneofCase.FloatValues, value.ArrayValue.ValuesCase);
Assert.Equal(new[] { 2.5f, 0f }, value.ArrayValue.FloatValues.Values);
}
[Fact]
public void Expand_Int64_WhenElementIsInt64()
{
MxValue value = SparseValue(
MxDataType.Integer,
3,
(2, new MxValue { Int64Value = 9_000_000_000L }));
SparseArrayExpander.Expand(value);
Assert.Equal(MxArray.ValuesOneofCase.Int64Values, value.ArrayValue.ValuesCase);
Assert.Equal(new[] { 0L, 0L, 9_000_000_000L }, value.ArrayValue.Int64Values.Values);
}
[Fact]
public void Expand_NonSparseValue_NoOps()
{
MxValue value = new() { Int32Value = 42 };
SparseArrayExpander.Expand(value);
Assert.Equal(MxValue.KindOneofCase.Int32Value, value.KindCase);
Assert.Equal(42, value.Int32Value);
}
[Fact]
public void Expand_NullValue_ThrowsArgumentNull()
{
Assert.Throws<ArgumentNullException>(() => SparseArrayExpander.Expand(null!));
}
[Fact]
public void Expand_TotalLengthExceedsMaxArrayLength_Throws()
{
MxValue value = SparseValue(MxDataType.Integer, 2_147_483_648u);
RpcException ex = Assert.Throws<RpcException>(() => SparseArrayExpander.Expand(value));
Assert.Equal(StatusCode.InvalidArgument, ex.StatusCode);
}
}
@@ -207,6 +207,62 @@ public sealed class ConstraintEnforcerTests
Assert.Equal("read_historized_only", failure.ConstraintName);
}
/// <summary>
/// A bare array attribute address (no trailing <c>[]</c>) resolves through the Galaxy index
/// even though arrays are keyed by their suffixed FullTagReference (e.g. "Pump_001.Levels[]").
/// Without the <c>[]</c> fallback in ResolveTarget the bare name misses the index and a
/// read-constrained key gets a spurious tag_metadata denial for an AddItem it should allow.
/// </summary>
[Fact]
public async Task CheckReadTagAsync_WithBareArrayName_ResolvesViaArraySuffixFallback()
{
ConstraintEnforcer enforcer = CreateEnforcer(out _);
ApiKeyIdentity identity = CreateIdentity(ApiKeyConstraints.Empty with
{
// A read constraint that covers the Pump_001 subtree; the array attribute is inside it.
ReadTagGlobs = ["Pump_001.*"],
});
ConstraintFailure? failure = await enforcer.CheckReadTagAsync(
identity,
"Pump_001.Levels",
CancellationToken.None);
// Before the fix: bare "Pump_001.Levels" misses the index (keyed "Pump_001.Levels[]") and
// returns a tag_metadata failure. After the fix: it resolves and is within scope -> null.
Assert.Null(failure);
}
/// <summary>
/// A bare non-array name that is genuinely absent from the index still resolves to null:
/// the <c>[]</c> probe must not manufacture a false positive for a scalar/missing tag.
/// </summary>
[Fact]
public async Task CheckReadTagAsync_WithMissingNonArrayName_StillFailsToResolve()
{
ConstraintEnforcer enforcer = CreateEnforcer(out _);
ApiKeyIdentity identity = CreateIdentity(ApiKeyConstraints.Empty with
{
ReadTagGlobs = ["Pump_001.*"],
});
// "Pump_001.Scalar" is not in the index, and "Pump_001.Scalar[]" is not either, so the
// suffix probe must not resolve it. A genuinely-unknown name behaves the same.
ConstraintFailure? missingScalar = await enforcer.CheckReadTagAsync(
identity,
"Pump_001.Scalar",
CancellationToken.None);
Assert.NotNull(missingScalar);
Assert.Equal("tag_metadata", missingScalar.ConstraintName);
ConstraintFailure? unknown = await enforcer.CheckReadTagAsync(
identity,
"DoesNotExist_999.Whatever",
CancellationToken.None);
Assert.NotNull(unknown);
Assert.Equal("tag_metadata", unknown.ConstraintName);
}
private static ConstraintEnforcer CreateEnforcer(out FakeAuditWriter auditWriter)
{
auditWriter = new FakeAuditWriter();
@@ -276,6 +332,14 @@ public sealed class ConstraintEnforcerTests
AttributeName = "NonHistorized",
FullTagReference = "Pump_001.NonHistorized",
},
new GalaxyAttribute
{
// Galaxy SQL keys array attributes by their suffixed FullTagReference,
// so the index entry is "Pump_001.Levels[]", not the bare name.
AttributeName = "Levels",
FullTagReference = "Pump_001.Levels[]",
IsArray = true,
},
},
},
new GalaxyObject