mxaccessgw/docs/ClientLibrariesDesign.md

# Client Libraries Detailed Design

## Purpose

This document defines the shared design for official MXAccess Gateway gRPC
clients. Each supported language should provide:

- a reusable client library,
- a test CLI built on that library,
- unit tests that run without a live gateway,
- optional integration tests against a live gateway.

Target client languages:

- .NET 10 C#
- Go
- Rust
- Python
- Java

Language-specific plans:

- `clients/dotnet/DotnetClientDesign.md`
- `clients/go/GoClientDesign.md`
- `clients/rust/RustClientDesign.md`
- `clients/python/PythonClientDesign.md`
- `clients/java/JavaClientDesign.md`

Shared generation inputs:

- `docs/ClientProtoGeneration.md`
- `docs/ClientBehaviorFixtures.md`
- `docs/ClientPackaging.md`
- `clients/proto/proto-inputs.json`

Language style guides:

| Client | Style guide |
|--------|-------------|
| .NET C# | [C# Style Guide](./style-guides/CSharpStyleGuide.md) |
| Go | [Go Style Guide](./style-guides/GoStyleGuide.md) |
| Rust | [Rust Style Guide](./style-guides/RustStyleGuide.md) |
| Python | [Python Style Guide](./style-guides/PythonStyleGuide.md) |
| Java | [Java Style Guide](./style-guides/JavaStyleGuide.md) |
| Generated protobuf/gRPC code | [Protobuf Style Guide](./style-guides/ProtobufStyleGuide.md) |

## Goals

Client libraries should make the gateway pleasant to consume without hiding
MXAccess behavior.

Goals:

- expose sessions as first-class objects,
- support unary `OpenSession`, `CloseSession`, and `Invoke`,
- support server-streaming `StreamEvents`,
- attach API key auth metadata to every call,
- preserve gateway, worker, COM, HRESULT, and MXAccess status detail,
- provide method-specific command helpers,
- provide raw command escape hatches for parity work,
- provide deterministic test CLIs for smoke and integration testing,
- keep generated protobuf/gRPC code separate from handwritten wrappers.

Non-goals for v1:

- automatic client-side session reconnection (the library never transparently
  re-opens a dropped stream on the consumer's behalf — reconnect timing and
  policy stay an application concern),
- client-side event replay buffering (the gateway owns the replay ring; the
  client does not retain its own event history),
- client-side command batching,
- synthetic MXAccess events,
- hiding MXAccess handles behind opaque client-only handles.

The gateway's reconnect-replay *protocol* is, however, consumable from every
client: each exposes the `after_worker_sequence` resume cursor on
`StreamEvents` and surfaces the gateway's `ReplayGap` sentinel as a distinct,
typed, non-terminal signal (CLI-15) so a consumer can detect an evicted-history
gap and re-snapshot. Per the no-synthesized-events invariant the client never
fabricates or swallows the sentinel — it only makes the gateway's own signal
observable. The reaction contract is identical across languages: on a gap,
discard cached state, re-snapshot, and resume with
`after_worker_sequence = oldest_available_sequence - 1`. The per-language
surface follows each idiom (.NET `MxEventStreamItem.IsReplayGap` via
`StreamEventItemsAsync`; Go `EventResult.ReplayGap`/`IsReplayGap()`; Rust
`EventItem::ReplayGap`; Python yields a typed `ReplayGap` from `stream_events`;
Java `MxEventStreamItem.isReplayGap()` via `MxEventStream.nextItem()`). Shipped
in all five clients.

## Public Client Concepts

All languages should expose the same core concepts, using idiomatic naming:

- gateway client,
- session,
- command request,
- command reply,
- event stream,
- MXAccess event,
- MX value,
- MX status proxy,
- gateway error,
- client options.

The gateway session id and MXAccess handles must remain visible. The library may
offer helper methods, but it must not invent alternate handle semantics.

## Typed Command Parity

Every command kind in the wire contract has a typed single-item session helper,
not just a raw-`Invoke` escape hatch (CLI-04). Beyond the register/add/advise/
remove/write family, the parity-critical single-item helpers are:
`AdviseSupervisory`, `WriteSecured` / `WriteSecured2`, `AuthenticateUser`,
`ArchestrAUserToId`, `AddBufferedItem`, `SetBufferedUpdateInterval`, `Suspend`,
`Activate`, and `Unregister`. Each is a thin wrapper over the same raw-command
machinery the bulk helpers use — it adds no wire surface — and runs the same
MXAccess-level reply validation (HRESULT `< 0` + per-item `MxStatusProxy`) as the
rest of the client. The per-item rule is **`MxStatusProxy` failure iff
`category != MX_STATUS_CATEGORY_OK`**; `success` is the raw COM member carried
for diagnostics only and never decides the verdict, so an absent entry is
success while a present entry with an unspecified category is a failure. The
shared fixtures in `clients/proto/fixtures/behavior/command-replies/` pin both
rules across all five clients (see
[Client Behavior Fixtures](./ClientBehaviorFixtures.md)).
**MXAccess parity is preserved exactly**: e.g. `WriteSecured`
failing before a prior `AuthenticateUser` + `AdviseSupervisory` surfaces the
native failure unchanged — the helper does not pre-validate or reorder it.

**Malformed-reply extraction:** the id/handle-returning helpers
(`AuthenticateUser`, `ArchestrAUserToId`, `AddBufferedItem`, and the handle
extractors) follow one contract across all five clients — prefer the typed
payload; fall back to `return_value` only when it is present with the expected
int32 variant; when neither is present, raise a typed malformed-reply error.
They never surface a proto3 default `0` and never throw a null-reference (CLI-41).

**Credential handling:** `AuthenticateUser` credentials and `WriteSecured`
secured payloads route through each client's secret-redaction seam so they never
reach logs, exception text, or `ToString`/`Debug`/`Display` — the value is carried
only on the wire. In addition to that by-construction guarantee (exceptions carry
reply-derived text, not the request), every client scrubs the **exact** secret
values it was called with from any surfaced error as defense-in-depth, so a
gateway or MXAccess diagnostic that echoes a credential back cannot leak it
(CLI-40). The scrub covers **both** the rendered message and the structured reply
the error still exposes (`protocolStatus.message`, `statuses[].diagnosticText`,
`diagnosticMessage`): the redacted error carries a scrubbed clone of the reply so
a logger dumping the exception's structured fields cannot reintroduce the leak.
This holds regardless of whether the reply is coded `OK` (with a failing HRESULT)
or `MXACCESS_FAILURE` — every client routes both to its MXAccess error type. Each
client's test suite asserts the distinctive credential is absent from both the
surfaced message and the exposed reply, and that the redaction marker is present.

Shipped in all five clients (.NET / Go / Rust / Python / Java).

## Shared API Shape

Each language should support this conceptual API:

```text
client = GatewayClient.connect(endpoint, apiKey, options)
session = client.openSession(options)

serverHandle = session.register(clientName)
itemHandle = session.addItem(serverHandle, itemReference)
session.advise(serverHandle, itemHandle)

events = session.streamEvents()
session.write(serverHandle, itemHandle, value, userId)

session.close()
client.close()
```

Each language should expose the gateway bulk subscription commands with
idiomatic names:

```text
session.addItemBulk(serverHandle, tagAddresses)
session.adviseItemBulk(serverHandle, itemHandles)
session.removeItemBulk(serverHandle, itemHandles)
session.unAdviseItemBulk(serverHandle, itemHandles)
session.subscribeBulk(serverHandle, tagAddresses)
session.unsubscribeBulk(serverHandle, itemHandles)
```

These methods send one `Invoke` request using the matching bulk command kind.
They return the gateway `SubscribeResult` list without inventing client-only
handles. `SubscribeBulk` performs `AddItem` then `Advise` per tag inside the
worker session. `UnsubscribeBulk` performs `UnAdvise` then `RemoveItem` per item
handle. Per-item failures are returned in `SubscribeResult`; transport,
gateway, and cancellation failures still use each language's normal error
surface.

Each library should also expose lower-level calls:

```text
client.openSession(rawRequest)
client.closeSession(rawRequest)
client.invoke(rawCommandRequest)
client.streamEvents(rawStreamRequest)
```

## Authentication

The gateway uses API key auth for v1. Clients should support:

```text
authorization: Bearer mxgw_<key-id>_<secret>
```

Rules:

- Do not log API keys.
- Redact keys in CLI error output.
- Allow API key from command line, environment variable, or config object.
- Recommended environment variable: `MXGATEWAY_API_KEY`.
- Attach auth metadata to every unary and streaming call.
- Treat `Unauthenticated` and `PermissionDenied` distinctly.

## TLS

Clients should support:

- plaintext for local development,
- TLS with system roots,
- TLS with custom CA file,
- optional server name override for test environments.

Default should be secure for packaged production examples, but the test CLI may
default to plaintext when endpoint is `localhost` or `127.0.0.1`.

## Timeouts And Cancellation

Each client library should support:

- connect timeout,
- unary call timeout,
- command timeout passed to gateway when the public API supports it,
- stream cancellation,
- graceful session close timeout.

Language wrappers should map cancellation to the native ecosystem:

- .NET: `CancellationToken`
- Go: `context.Context`
- Rust: `tokio` cancellation / dropped future plus explicit timeout
- Python: `asyncio` task cancellation and deadlines
- Java: `Deadline`, `CompletableFuture`, and stream cancellation

Canceling a client call does not imply the worker COM call was aborted. Client
docs and errors must make that clear.

## Error Model

Each client should distinguish:

- transport errors,
- authentication/authorization errors,
- gateway session errors,
- worker process/protocol errors,
- MXAccess command failures,
- COM HRESULT/status failures.

Generated gRPC errors should not be the only error surface. The wrapper should
return rich command replies when the gateway reached MXAccess and MXAccess
returned HRESULT/status information.

Recommended high-level error categories:

```text
TransportError
AuthenticationError
AuthorizationError
SessionError
WorkerError
ProtocolError
CommandError
MxAccessError
TimeoutError
CancelledError
```

## Values

Each language should provide ergonomic conversion helpers for `MxValue`:

- bool,
- int32,
- int64,
- float,
- double,
- string,
- timestamp,
- typed arrays,
- raw variant fallback.

The raw protobuf value should always remain accessible.

Do not lose raw variant metadata when conversion is incomplete. For CLI output,
render both typed projection and raw metadata when present.

## Events

Each client should expose event streaming as the idiomatic streaming primitive:

- .NET: `IAsyncEnumerable<MxEvent>`
- Go: receive loop over generated stream
- Rust: `Stream<Item = Result<MxEvent, Error>>`
- Python: async iterator
- Java: blocking iterator and async observer variants

Events must preserve gateway order. Libraries should not reorder, coalesce, or
drop events by default.

Long-lived event streams do not inherit unary call deadlines. Clients apply the
default call timeout to unary operations only, and streams run until the caller
cancels them or an explicit stream timeout is configured.

The event surface must include:

- `OnDataChange`
- `OnWriteComplete`
- `OperationComplete`
- `OnBufferedDataChange`
- terminal session fault when represented as a message

`OperationComplete` is forwarded only when native MXAccess raises it.
`OnBufferedDataChange` payload conversion may include raw metadata until live
multi-sample buffered payloads are fully validated.

## Test CLI Contract

Each language should include a test CLI that exercises the library. The CLI is
not the production gateway server.

Required commands:

```text
version
ping
open-session
close-session
register
add-item
advise
stream-events
write
write2
smoke
```

Optional commands:

```text
add-item2
add-buffered-item
set-buffered-update-interval
authenticate-user
write-secured
write-secured2
get-worker-info
metadata-query
```

Common CLI flags:

```text
--endpoint <host:port or URL>
--api-key <key>
--api-key-env <name>
--plaintext
--tls
--ca-file <path>
--session-id <id>
--client-name <name>
--server-handle <int>
--item-handle <int>
--item <reference>
--context <context>
--value <value>
--type <mx type>
--timeout <duration>
--json
--verbose
```

The `smoke` command should:

1. open a session,
2. register a client name,
3. add one item,
4. advise it,
5. optionally write a value,
6. stream events for a bounded duration,
7. close the session.

CLI output should support JSON for automated tests.

## Unit Tests

Unit tests must run without a live gateway. Use fake gRPC services, mock
transports, or generated test servers depending on language.

Shared behavior fixtures live in `clients/proto/fixtures/behavior`. Every
client should include tests that load the fixture manifest and verify wrapper
behavior against the common command reply, event stream, value conversion,
status conversion, auth error, and timeout/cancel cases.

Required unit test areas:

- options parsing,
- auth metadata injection,
- TLS/plaintext channel setup,
- method-specific request construction,
- value conversion,
- status conversion,
- command reply error mapping,
- stream event iteration,
- stream cancellation,
- timeout behavior,
- CLI argument parsing,
- CLI JSON output redaction of secrets.

## Integration Tests

Integration tests are optional and should be opt-in. They may require a live
gateway and installed MXAccess on the gateway host.

Recommended environment variables:

```text
MXGATEWAY_ENDPOINT
MXGATEWAY_API_KEY
MXGATEWAY_TEST_ITEM
MXGATEWAY_TEST_CONTEXT
MXGATEWAY_TEST_WRITE_VALUE
MXGATEWAY_INTEGRATION=1
```

Integration tests should skip unless `MXGATEWAY_INTEGRATION=1`.

## Repository Layout

Recommended top-level layout:

```text
clients/
  dotnet/
  go/
  rust/
  python/
  java/
```

Each client should contain:

```text
src or package source
generated protobuf/grpc source
test CLI
unit tests
README.md
examples/
```

Generated code should be reproducible from `src/ZB.MOM.WW.MxGateway.Contracts/Protos/`.
Do not hand-edit generated code.

The stable client proto manifest defines the generated-code directories:

```text
clients/dotnet/generated
clients/go/internal/generated
clients/rust/src/generated
clients/python/src/zb_mom_ww_mxgateway/generated
clients/java/src/main/generated
```

## Versioning

All clients should expose:

- client library version,
- supported gateway protocol version,
- generated protobuf version if available.

Version compatibility should be tested against protocol-version mismatch cases.

## Documentation

Each client README should include:

- install instructions,
- minimal open/register/add/advise example,
- API key configuration,
- TLS configuration,
- CLI examples,
- integration test instructions,
- warning that canceling a client call does not abort an in-flight MXAccess COM
  call.

## Related Documentation

- [.NET 10 C# Client Detailed Design](../clients/dotnet/DotnetClientDesign.md)
- [Go Client Detailed Design](../clients/go/GoClientDesign.md)
- [Rust Client Detailed Design](../clients/rust/RustClientDesign.md)
- [Python Client Detailed Design](../clients/python/PythonClientDesign.md)
- [Java Client Detailed Design](../clients/java/JavaClientDesign.md)
- [Client Proto Generation](./ClientProtoGeneration.md)
- [Client Behavior Fixtures](./ClientBehaviorFixtures.md)
- [Client Packaging](./ClientPackaging.md)
- [Cross-Language Smoke Matrix](./CrossLanguageSmokeMatrix.md)