Add bulk read/write command family across worker, gateway, and clients
Adds five new MXAccess command kinds (WriteBulk, Write2Bulk,
WriteSecuredBulk, WriteSecured2Bulk, ReadBulk) that ride the existing
"one round-trip, per-entry results" bulk shape used by AddItemBulk and
SubscribeBulk today. MXAccess COM has no native bulk API; the worker
runs each bulk operation as a sequential loop on its STA, returning
one BulkWriteResult / BulkReadResult per requested entry so per-item
MXAccess failures surface as was_successful=false rather than throwing.
ReadBulk has no MXAccess analogue. The worker satisfies it by:
- Returning the last cached OnDataChange payload (was_cached=true)
when the requested tag is already in the session''s item registry
AND advised — the existing subscription is NOT touched, since the
caller did not create it.
- Otherwise taking the AddItem + Advise + wait-for-OnDataChange +
UnAdvise + RemoveItem snapshot lifecycle itself (was_cached=false)
and leaving the session exactly as it was. The wait pumps Windows
messages on the STA so the inbound MXAccess event can dispatch
while the executor still holds the thread.
The new MxAccessValueCache lives on each MxAccessSession, shared with
MxAccessBaseEventSink which populates it on every OnDataChange after
the event clears the outbound queue. Eviction on RemoveItem keeps
reused MXAccess handles from serving stale values from a previous
lifetime.
Gateway-side authorization wires WriteBulk/Write2Bulk to invoke:write,
WriteSecuredBulk/WriteSecured2Bulk to invoke:secure, ReadBulk to
invoke:read. The constraint-filter pipeline is refactored from a single
BulkConstraintPlan record into an abstract base plus three concretes
(SubscribeBulk, WriteBulk, ReadBulk), each owning its own denied-entry
merge so the dispatch site never branches on reply shape. A new
FilterWriteBulkAsync<TEntry> generic over the four write-entry shapes
runs CheckWriteHandleAsync per entry; denied entries surface as the
BulkWriteResult shape, preserving original-index order.
All five language clients (.NET, Go, Rust, Python, Java) gained the
five new methods following their existing bulk pattern, with regenerated
protobufs.
Tests added:
- MxAccessValueCacheTests (6 cases) — Set/TryGet, Remove resets the
version, TryWaitForUpdate signals on Set, pump step fires each poll.
- MxAccessBaseEventSinkTests — OnDataChange populates the cache,
ValueCache property exposes the bound instance.
- MxAccessCommandExecutorTests — four bulk-write variants (per-entry
success/failure, value+timestamp forwarding, secured user ids),
ReadBulk snapshot lifecycle on uncached tag (timeout surfaces as
was_successful=false), invalid-payload reply.
- GatewayGrpcScopeResolverTests — five new MxCommandKind cases.
- SessionManagerTests — WriteBulk and ReadBulk forwarding through
FakeWorkerHarness; ReadBulk forwards timeout_ms.
- Per-client (.NET, Go, Rust, Python, Java) — WriteBulk builds the
right command and returns per-entry results, ReadBulk forwards the
timeout and unpacks the was_cached flag.
Cross-language e2e CLI subcommands for the new bulks are deliberately
scoped out of this change (each of the five client CLIs would need
five new subcommands plus matching phases in
scripts/run-client-e2e-tests.ps1); coverage equivalent to the existing
bulk-subscribe coverage is provided by worker + gateway + per-client
unit tests.
Docs updated in the same commit: gateway.md (Public MXAccess Command
Surface), docs/DesignDecisions.md (new "Bulk Command Family" section
with the ReadBulk cache-then-snapshot rationale), and every client
README.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
+10
-1
@@ -76,7 +76,16 @@ client, err := mxgateway.Dial(ctx, mxgateway.Options{
|
||||
```
|
||||
|
||||
`Client.OpenSession` returns a `Session` with helpers for `Register`,
|
||||
`AddItem`, `AddItem2`, `Advise`, `Write`, `Events`, and `Close`. Prefer
|
||||
`AddItem`, `AddItem2`, `Advise`, `Write`, the full bulk family
|
||||
(`AddItemBulk`, `AdviseItemBulk`, `RemoveItemBulk`, `UnAdviseItemBulk`,
|
||||
`SubscribeBulk`, `UnsubscribeBulk`, `WriteBulk`, `Write2Bulk`,
|
||||
`WriteSecuredBulk`, `WriteSecured2Bulk`, `ReadBulk`), `Events`, and
|
||||
`Close`. Bulk variants carry a list of entries in one round-trip and
|
||||
return one result per entry; per-entry MXAccess failures appear as
|
||||
`was_successful = false` and never return as Go errors. `ReadBulk` accepts
|
||||
a `time.Duration` per-tag timeout and returns cached `OnDataChange`
|
||||
values when the tag is already advised (`WasCached = true`) without
|
||||
touching the existing subscription. Prefer
|
||||
`SubscribeEvents` or `SubscribeEventsAfter` for long-running streams because the
|
||||
returned subscription owns cancellation and exposes `Close` for deterministic
|
||||
goroutine cleanup. `Events` and `EventsAfter` are a compatibility shim with a
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -243,6 +243,87 @@ func TestSubscribeBulkBuildsOneBulkCommandAndReturnsResults(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestWriteBulkBuildsOneBulkCommandAndReturnsPerEntryResults(t *testing.T) {
|
||||
fake := &fakeGatewayServer{
|
||||
invokeReply: &pb.MxCommandReply{
|
||||
SessionId: "session-1",
|
||||
Kind: pb.MxCommandKind_MX_COMMAND_KIND_WRITE_BULK,
|
||||
ProtocolStatus: &pb.ProtocolStatus{
|
||||
Code: pb.ProtocolStatusCode_PROTOCOL_STATUS_CODE_OK,
|
||||
},
|
||||
Payload: &pb.MxCommandReply_WriteBulk{
|
||||
WriteBulk: &pb.BulkWriteReply{
|
||||
Results: []*pb.BulkWriteResult{
|
||||
{ServerHandle: 12, ItemHandle: 901, WasSuccessful: true},
|
||||
{ServerHandle: 12, ItemHandle: 902, WasSuccessful: false, ErrorMessage: "invalid handle"},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
client, cleanup := newBufconnClient(t, fake)
|
||||
defer cleanup()
|
||||
session := NewSessionForID(client, "session-1")
|
||||
|
||||
results, err := session.WriteBulk(context.Background(), 12, []*pb.WriteBulkEntry{
|
||||
{ItemHandle: 901, UserId: 5, Value: &pb.MxValue{DataType: pb.MxDataType_MX_DATA_TYPE_INTEGER, Kind: &pb.MxValue_Int32Value{Int32Value: 11}}},
|
||||
{ItemHandle: 902, UserId: 5, Value: &pb.MxValue{DataType: pb.MxDataType_MX_DATA_TYPE_INTEGER, Kind: &pb.MxValue_Int32Value{Int32Value: 22}}},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("WriteBulk() error = %v", err)
|
||||
}
|
||||
if len(results) != 2 || !results[0].GetWasSuccessful() || results[1].GetWasSuccessful() {
|
||||
t.Fatalf("results = %#v, want [success, failure]", results)
|
||||
}
|
||||
req := fake.invokeRequest
|
||||
if req.GetCommand().GetKind() != pb.MxCommandKind_MX_COMMAND_KIND_WRITE_BULK {
|
||||
t.Fatalf("command kind = %s", req.GetCommand().GetKind())
|
||||
}
|
||||
if got := req.GetCommand().GetWriteBulk().GetEntries(); len(got) != 2 {
|
||||
t.Fatalf("entries = %#v, want 2", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestReadBulkForwardsTimeoutAndUnpacksCachedFlag(t *testing.T) {
|
||||
fake := &fakeGatewayServer{
|
||||
invokeReply: &pb.MxCommandReply{
|
||||
SessionId: "session-1",
|
||||
Kind: pb.MxCommandKind_MX_COMMAND_KIND_READ_BULK,
|
||||
ProtocolStatus: &pb.ProtocolStatus{
|
||||
Code: pb.ProtocolStatusCode_PROTOCOL_STATUS_CODE_OK,
|
||||
},
|
||||
Payload: &pb.MxCommandReply_ReadBulk{
|
||||
ReadBulk: &pb.BulkReadReply{
|
||||
Results: []*pb.BulkReadResult{
|
||||
{
|
||||
ServerHandle: 12,
|
||||
TagAddress: "Area001.Pump001.Speed",
|
||||
ItemHandle: 34,
|
||||
WasSuccessful: true,
|
||||
WasCached: true,
|
||||
Value: &pb.MxValue{DataType: pb.MxDataType_MX_DATA_TYPE_INTEGER, Kind: &pb.MxValue_Int32Value{Int32Value: 99}},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
client, cleanup := newBufconnClient(t, fake)
|
||||
defer cleanup()
|
||||
session := NewSessionForID(client, "session-1")
|
||||
|
||||
results, err := session.ReadBulk(context.Background(), 12, []string{"Area001.Pump001.Speed"}, 750*time.Millisecond)
|
||||
if err != nil {
|
||||
t.Fatalf("ReadBulk() error = %v", err)
|
||||
}
|
||||
if len(results) != 1 || !results[0].GetWasCached() || results[0].GetValue().GetInt32Value() != 99 {
|
||||
t.Fatalf("results = %#v", results)
|
||||
}
|
||||
if got := fake.invokeRequest.GetCommand().GetReadBulk().GetTimeoutMs(); got != 750 {
|
||||
t.Fatalf("timeout_ms = %d, want 750", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestInvokeReturnsTypedMxAccessErrorWithRawReply(t *testing.T) {
|
||||
hresult := int32(-2147467259)
|
||||
fake := &fakeGatewayServer{
|
||||
|
||||
@@ -389,6 +389,142 @@ func (s *Session) UnsubscribeBulk(ctx context.Context, serverHandle int32, itemH
|
||||
return reply.GetUnsubscribeBulk().GetResults(), nil
|
||||
}
|
||||
|
||||
// WriteBulk invokes MXAccess Write sequentially for each entry inside one gateway command.
|
||||
// Per-entry failures appear as BulkWriteResult entries with WasSuccessful=false; the call
|
||||
// never returns an error for per-entry MXAccess failures (it returns an error only for
|
||||
// protocol-level failures or transport errors).
|
||||
func (s *Session) WriteBulk(ctx context.Context, serverHandle int32, entries []*WriteBulkEntry) ([]*BulkWriteResult, error) {
|
||||
if entries == nil {
|
||||
return nil, errors.New("mxgateway: write bulk entries are required")
|
||||
}
|
||||
if err := ensureBulkSize("write bulk entries", len(entries)); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
reply, err := s.invokeCommand(ctx, &pb.MxCommand{
|
||||
Kind: pb.MxCommandKind_MX_COMMAND_KIND_WRITE_BULK,
|
||||
Payload: &pb.MxCommand_WriteBulk{
|
||||
WriteBulk: &pb.WriteBulkCommand{
|
||||
ServerHandle: serverHandle,
|
||||
Entries: entries,
|
||||
},
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return reply.GetWriteBulk().GetResults(), nil
|
||||
}
|
||||
|
||||
// Write2Bulk invokes MXAccess Write2 (timestamped) for each entry inside one gateway command.
|
||||
func (s *Session) Write2Bulk(ctx context.Context, serverHandle int32, entries []*Write2BulkEntry) ([]*BulkWriteResult, error) {
|
||||
if entries == nil {
|
||||
return nil, errors.New("mxgateway: write2 bulk entries are required")
|
||||
}
|
||||
if err := ensureBulkSize("write2 bulk entries", len(entries)); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
reply, err := s.invokeCommand(ctx, &pb.MxCommand{
|
||||
Kind: pb.MxCommandKind_MX_COMMAND_KIND_WRITE2_BULK,
|
||||
Payload: &pb.MxCommand_Write2Bulk{
|
||||
Write2Bulk: &pb.Write2BulkCommand{
|
||||
ServerHandle: serverHandle,
|
||||
Entries: entries,
|
||||
},
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return reply.GetWrite2Bulk().GetResults(), nil
|
||||
}
|
||||
|
||||
// WriteSecuredBulk invokes MXAccess WriteSecured for each entry. Credential-sensitive
|
||||
// values must not be logged by callers; mirrors the single-item WriteSecured contract.
|
||||
func (s *Session) WriteSecuredBulk(ctx context.Context, serverHandle int32, entries []*WriteSecuredBulkEntry) ([]*BulkWriteResult, error) {
|
||||
if entries == nil {
|
||||
return nil, errors.New("mxgateway: write-secured bulk entries are required")
|
||||
}
|
||||
if err := ensureBulkSize("write-secured bulk entries", len(entries)); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
reply, err := s.invokeCommand(ctx, &pb.MxCommand{
|
||||
Kind: pb.MxCommandKind_MX_COMMAND_KIND_WRITE_SECURED_BULK,
|
||||
Payload: &pb.MxCommand_WriteSecuredBulk{
|
||||
WriteSecuredBulk: &pb.WriteSecuredBulkCommand{
|
||||
ServerHandle: serverHandle,
|
||||
Entries: entries,
|
||||
},
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return reply.GetWriteSecuredBulk().GetResults(), nil
|
||||
}
|
||||
|
||||
// WriteSecured2Bulk invokes MXAccess WriteSecured2 (timestamped) for each entry.
|
||||
func (s *Session) WriteSecured2Bulk(ctx context.Context, serverHandle int32, entries []*WriteSecured2BulkEntry) ([]*BulkWriteResult, error) {
|
||||
if entries == nil {
|
||||
return nil, errors.New("mxgateway: write-secured2 bulk entries are required")
|
||||
}
|
||||
if err := ensureBulkSize("write-secured2 bulk entries", len(entries)); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
reply, err := s.invokeCommand(ctx, &pb.MxCommand{
|
||||
Kind: pb.MxCommandKind_MX_COMMAND_KIND_WRITE_SECURED2_BULK,
|
||||
Payload: &pb.MxCommand_WriteSecured2Bulk{
|
||||
WriteSecured2Bulk: &pb.WriteSecured2BulkCommand{
|
||||
ServerHandle: serverHandle,
|
||||
Entries: entries,
|
||||
},
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return reply.GetWriteSecured2Bulk().GetResults(), nil
|
||||
}
|
||||
|
||||
// ReadBulk snapshots the current value of each requested tag.
|
||||
//
|
||||
// MXAccess COM has no synchronous Read; the worker satisfies this by returning the
|
||||
// most recent cached OnDataChange value when the tag is already advised (WasCached=true),
|
||||
// or by taking a full AddItem + Advise + wait + UnAdvise + RemoveItem snapshot lifecycle
|
||||
// otherwise. timeout bounds the wait per tag in the snapshot case; pass zero to use the
|
||||
// worker default. Per-tag failures (timeout, invalid tag) appear as BulkReadResult entries
|
||||
// with WasSuccessful=false; the call never returns an error for per-tag MXAccess failures.
|
||||
func (s *Session) ReadBulk(ctx context.Context, serverHandle int32, tagAddresses []string, timeout time.Duration) ([]*BulkReadResult, error) {
|
||||
if tagAddresses == nil {
|
||||
return nil, errors.New("mxgateway: tag addresses are required")
|
||||
}
|
||||
if err := ensureBulkSize("tag addresses", len(tagAddresses)); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var timeoutMs uint32
|
||||
if timeout > 0 {
|
||||
ms := timeout.Milliseconds()
|
||||
if ms > int64(^uint32(0)) {
|
||||
timeoutMs = ^uint32(0)
|
||||
} else {
|
||||
timeoutMs = uint32(ms)
|
||||
}
|
||||
}
|
||||
reply, err := s.invokeCommand(ctx, &pb.MxCommand{
|
||||
Kind: pb.MxCommandKind_MX_COMMAND_KIND_READ_BULK,
|
||||
Payload: &pb.MxCommand_ReadBulk{
|
||||
ReadBulk: &pb.ReadBulkCommand{
|
||||
ServerHandle: serverHandle,
|
||||
TagAddresses: tagAddresses,
|
||||
TimeoutMs: timeoutMs,
|
||||
},
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return reply.GetReadBulk().GetResults(), nil
|
||||
}
|
||||
|
||||
// Write invokes MXAccess Write.
|
||||
func (s *Session) Write(ctx context.Context, serverHandle, itemHandle int32, value *MxValue, userID int32) error {
|
||||
_, err := s.WriteRaw(ctx, serverHandle, itemHandle, value, userID)
|
||||
|
||||
@@ -70,6 +70,32 @@ type (
|
||||
WriteCommand = pb.WriteCommand
|
||||
// Write2Command is the payload of an MXAccess Write2 command.
|
||||
Write2Command = pb.Write2Command
|
||||
// WriteBulkCommand carries one bulk-Write request.
|
||||
WriteBulkCommand = pb.WriteBulkCommand
|
||||
// WriteBulkEntry is one (item_handle, value, user_id) tuple in a WriteBulk request.
|
||||
WriteBulkEntry = pb.WriteBulkEntry
|
||||
// Write2BulkCommand carries one bulk-Write2 (timestamped) request.
|
||||
Write2BulkCommand = pb.Write2BulkCommand
|
||||
// Write2BulkEntry is one (item_handle, value, timestamp_value, user_id) tuple in a Write2Bulk request.
|
||||
Write2BulkEntry = pb.Write2BulkEntry
|
||||
// WriteSecuredBulkCommand carries one bulk-WriteSecured request. Values are credential-sensitive.
|
||||
WriteSecuredBulkCommand = pb.WriteSecuredBulkCommand
|
||||
// WriteSecuredBulkEntry is one entry in a WriteSecuredBulk request.
|
||||
WriteSecuredBulkEntry = pb.WriteSecuredBulkEntry
|
||||
// WriteSecured2BulkCommand carries one bulk-WriteSecured2 (timestamped) request.
|
||||
WriteSecured2BulkCommand = pb.WriteSecured2BulkCommand
|
||||
// WriteSecured2BulkEntry is one entry in a WriteSecured2Bulk request.
|
||||
WriteSecured2BulkEntry = pb.WriteSecured2BulkEntry
|
||||
// ReadBulkCommand carries one bulk-Read request.
|
||||
ReadBulkCommand = pb.ReadBulkCommand
|
||||
// BulkWriteResult is one per-entry result in a bulk-write reply.
|
||||
BulkWriteResult = pb.BulkWriteResult
|
||||
// BulkWriteReply aggregates BulkWriteResult entries for a bulk-write command.
|
||||
BulkWriteReply = pb.BulkWriteReply
|
||||
// BulkReadResult is one per-tag result in a bulk-read reply (carries the snapshot value plus a was_cached flag).
|
||||
BulkReadResult = pb.BulkReadResult
|
||||
// BulkReadReply aggregates BulkReadResult entries for a ReadBulk command.
|
||||
BulkReadReply = pb.BulkReadReply
|
||||
// RegisterReply carries the ServerHandle returned by Register.
|
||||
RegisterReply = pb.RegisterReply
|
||||
// AddItemReply carries the ItemHandle returned by AddItem.
|
||||
@@ -155,6 +181,16 @@ const (
|
||||
CommandKindWrite = pb.MxCommandKind_MX_COMMAND_KIND_WRITE
|
||||
// CommandKindWrite2 selects the MXAccess Write2 command.
|
||||
CommandKindWrite2 = pb.MxCommandKind_MX_COMMAND_KIND_WRITE2
|
||||
// CommandKindWriteBulk selects the bulk Write command.
|
||||
CommandKindWriteBulk = pb.MxCommandKind_MX_COMMAND_KIND_WRITE_BULK
|
||||
// CommandKindWrite2Bulk selects the bulk Write2 (timestamped) command.
|
||||
CommandKindWrite2Bulk = pb.MxCommandKind_MX_COMMAND_KIND_WRITE2_BULK
|
||||
// CommandKindWriteSecuredBulk selects the bulk WriteSecured command.
|
||||
CommandKindWriteSecuredBulk = pb.MxCommandKind_MX_COMMAND_KIND_WRITE_SECURED_BULK
|
||||
// CommandKindWriteSecured2Bulk selects the bulk WriteSecured2 (timestamped) command.
|
||||
CommandKindWriteSecured2Bulk = pb.MxCommandKind_MX_COMMAND_KIND_WRITE_SECURED2_BULK
|
||||
// CommandKindReadBulk selects the bulk Read command (cached-or-snapshot per tag).
|
||||
CommandKindReadBulk = pb.MxCommandKind_MX_COMMAND_KIND_READ_BULK
|
||||
|
||||
// DataTypeUnknown denotes an unrecognized MXAccess data type.
|
||||
DataTypeUnknown = pb.MxDataType_MX_DATA_TYPE_UNKNOWN
|
||||
|
||||
Reference in New Issue
Block a user