mxaccessgw/clients/python/README.md

# Python Client

The Python client package contains generated MXAccess Gateway protobuf
bindings, the async `zb_mom_ww_mxgateway` package, and the `mxgw-py` test CLI. The
package uses the shared proto inputs documented in
`../../docs/ClientProtoGeneration.md` so gateway and client contracts stay in
sync.

## Layout

```text
clients/python/
  pyproject.toml
  generate-proto.ps1
  src/zb_mom_ww_mxgateway/
  src/zb_mom_ww_mxgateway/generated/
  src/zb_mom_ww_mxgateway_cli/
  tests/
```

`src/zb_mom_ww_mxgateway/generated` contains code produced by `grpc_tools.protoc`. Do not
edit generated files by hand.

## Regenerating Protobuf Bindings

Run generation after the shared `.proto` files or the Python output path
changes:

```powershell
./generate-proto.ps1
```

The script uses the Python tool path recorded in
`../../docs/ToolchainLinks.md`.

## Build And Test

Run the Python checks from `clients/python`:

```powershell
python -m pip install -e ".[dev]"
python -m pytest
python -m pip wheel . --no-deps --wheel-dir "$env:TEMP\mxgateway-python-wheel"
```

The tests import the generated gateway and worker stubs, run fake async gateway
stubs, verify API key metadata, exercise stream cancellation, load shared value
and command fixtures, and check deterministic CLI output.

## Packaging

Install the package in editable mode for local development:

```powershell
python -m pip install -e ".[dev]"
```

Build a wheel from `clients/python`:

```powershell
python -m pip wheel . --no-deps --wheel-dir "$env:TEMP\mxgateway-python-wheel"
```

Install the generated wheel into a target environment:

```powershell
python -m pip install <wheel-path>
```

The wheel exposes the `mxgw-py` console script.

## Library Usage

The library is async-first:

```python
from zb_mom_ww_mxgateway import GatewayClient

async with await GatewayClient.connect(
    endpoint="localhost:5000",
    api_key="<gateway-api-key>",
    plaintext=True,
) as client:
    session = await client.open_session(client_session_name="python-client")
    try:
        server_handle = await session.register("python-client")
        item_handle = await session.add_item(server_handle, "Object.Attribute")
        await session.advise(server_handle, item_handle)
    finally:
        await session.close()
```

`GatewayClient.open_session_raw`, `GatewayClient.invoke_raw`, and
`GatewayClient.stream_events_raw` keep the generated protobuf replies and
events available for parity tests. `Session` helpers call the method-specific
MXAccess commands and preserve raw replies on typed command exceptions.

For alarms, the client exposes `GatewayClient.query_active_alarms` (one-shot
snapshot), `GatewayClient.stream_alarms` (async generator yielding alarm-feed
messages from the gateway's central monitor), and
`GatewayClient.acknowledge_alarm` (ack by alarm reference, optional comment
and ack target). Cancel the surrounding task or `aclose()` the iterator to
terminate the stream.

Canceling a Python task cancels the client-side gRPC call or stream wait. It
does not abort an in-flight MXAccess COM call inside the worker process.

### Event streaming and reconnect gaps

`Session.stream_events()` yields an async iterator whose items are either a
normal `MxEvent` or a `ReplayGap`. Track the `worker_sequence` of the last event
you processed and pass it back as `after_worker_sequence` to resume after a
disconnect:

```python
from zb_mom_ww_mxgateway import ReplayGap

cursor = 0
async for item in session.stream_events(after_worker_sequence=cursor):
    if isinstance(item, ReplayGap):
        # The gateway dropped events between item.requested_after_sequence and
        # item.oldest_available_sequence — they are gone from the replay ring.
        # Discard local tag/alarm state and re-snapshot (e.g. read_bulk /
        # query_active_alarms), then resume without another gap:
        cursor = item.resume_after_worker_sequence  # oldest_available_sequence - 1
        continue
    # Normal MXAccess event.
    cursor = item.worker_sequence
    handle(item)
```

`ReplayGap` is the gateway's reconnect-replay gap sentinel made typed and
observable. It is delivered only at the head of a stream resumed with a non-zero
`after_worker_sequence` when the requested cursor predates the oldest retained
event. It is a **non-terminal** signal — the stream continues with normal events
after it — and it is never yielded as an `MxEvent`, so it can never be mistaken
for a real MXAccess event. The client does not synthesize or swallow it; the
gateway only sets it on `StreamEvents` results (never on a fresh stream or a
`DrainEvents` reply). `GatewayClient.stream_events_raw` remains the raw protobuf
stream (the sentinel arrives there as an `MxEvent` with `replay_gap` set).

## Write Semantics And Common Pitfalls

These are MXAccess parity behaviors that surprise new callers. The gateway
forwards them unchanged — it does not paper over them.

### Attributing a write to a user without `authenticate_user`

MXAccess only stamps a plain `write`/`write2` with a Galaxy user id when the
item carries an active *supervisory* advise. If you are **not** using the
verified/secured path (`authenticate_user` → `write_secured`/`write_secured2`)
but still need the write attributed to a user id, you must first advise the
item supervisory and then pass that user id on the write. Without the
supervisory advise the `user_id` on a plain write is ignored.

The session exposes a typed `advise_supervisory` helper alongside
`advise`/`unadvise`:

```python
await session.advise_supervisory(server_handle, item_handle)

await session.write(server_handle, item_handle, value, user_id=user_id)
```

The CLI exposes the same command as `advise-supervisory`, and `write` /
`write2` take `--user-id`.

For the verified/secured path, `authenticate_user`, `write_secured`,
`write_secured2`, and `archestra_user_to_id` are typed session helpers too. The
credential passed to `authenticate_user` and the values written by
`write_secured`/`write_secured2` are treated as secrets: they are never logged
and are scrubbed from any surfaced error message. MXAccess parity is preserved —
a `write_secured` that fails because no prior `authenticate_user` +
`advise_supervisory` established a supervisory context surfaces the native
failure as `MxAccessError` rather than being silently "fixed":

```python
user_id = await session.authenticate_user(server_handle, "operator", password)
await session.advise_supervisory(server_handle, item_handle)
await session.write_secured(
    server_handle,
    item_handle,
    value,
    current_user_id=user_id,
    verifier_user_id=user_id,
)
```

The CLI mirrors these as `authenticate-user` (credential via `--password` or,
preferably, `--password-env`) and `write-secured`.

### Array writes replace the whole array

A write to an array attribute **replaces the entire array**; it is not an
element-wise patch. To change a subset of elements, send the full array with
the unchanged elements included. For example, to change 2 elements of a
20-element array, build the `MxValue` from all 20 values (the 18 unchanged plus
the 2 new ones). Sending only the 2 changed values overwrites the attribute
with a 2-element array.

### Default-fill partial array writes

`Session.write_array_elements` lets you write only the indices you care about.
The gateway fills every unmentioned position with the type default for the
declared `element_data_type` (0, `False`, `""`, Unix epoch for timestamps).
The previous value at those positions is **not** preserved — the gateway expands
the sparse map to a full array before forwarding the write to MXAccess, so this
is still a full replacement:

```python
# Write indices 0 and 5 of a 10-element integer array.
# Positions 1-4 and 6-9 become 0, not their previous values.
await session.write_array_elements(
    server_handle=server_handle,
    item_handle=item_handle,
    element_data_type=pb.MX_DATA_TYPE_INTEGER,
    total_length=10,
    elements={0: 100, 5: 500},
)
```

Bare-name array items (e.g. `Object.ArrayAttr` without an index suffix) added
via `add_item`, `add_item2`, `add_item_bulk`, or `add_buffered_item`
auto-normalize to `[]` — they refer to the whole array, not a single element.
Writes through such handles must cover the full array or use
`write_array_elements` to supply `total_length` and let the gateway fill
defaults for the rest.

## Galaxy Repository Browse

The `GalaxyRepositoryClient` wraps the read-only `GalaxyRepository` gRPC
service. It lets callers test connectivity to the AVEVA System Platform
Galaxy Repository (ZB SQL database), read the last deploy timestamp, and
enumerate the deployed object hierarchy plus each object's dynamic
attributes:

```python
from zb_mom_ww_mxgateway import GalaxyRepositoryClient

async with await GalaxyRepositoryClient.connect(
    endpoint="localhost:5000",
    api_key="<gateway-api-key>",
    plaintext=True,
) as galaxy:
    if not await galaxy.test_connection():
        raise RuntimeError("gateway cannot reach the Galaxy Repository DB")

    last_deploy = await galaxy.get_last_deploy_time()
    print(f"last deploy: {last_deploy}")

    for obj in await galaxy.discover_hierarchy():
        print(obj.tag_name, obj.contained_name)
        for attr in obj.attributes:
            print("  ", attr.attribute_name, "->", attr.full_tag_reference)
```

The methods return native Python types (`bool`, `datetime | None`, and a
`list[GalaxyObject]` of generated proto messages) so callers can index
into the hierarchy without learning the underlying stub class. The
service requires the `metadata:read` scope on the API key.

### Browsing lazily

For UI trees or OPC UA bridges, use `browse_children_raw` to walk one level at a
time instead of loading the full hierarchy with `discover_hierarchy`. Pass an
empty request for root objects; subsequent calls set `parent_gobject_id`,
`parent_tag_name`, or `parent_contained_path`. Filter fields match
`DiscoverHierarchy`. Each response pairs `children` with `child_has_children` so
you know which nodes to expand. Most callers should prefer the higher-level
`browse()` / `LazyBrowseNode` walker below; `browse_children_raw` is the
low-level escape hatch for direct page-token control. See
[Galaxy Repository](../../docs/GalaxyRepository.md#browsechildren) for full
request and filter semantics.

```python
from zb_mom_ww_mxgateway.generated import galaxy_repository_pb2 as galaxy_pb2

reply = await galaxy.browse_children_raw(galaxy_pb2.BrowseChildrenRequest())
for child, has_children in zip(reply.children, reply.child_has_children):
    print(child.tag_name, "expand=" + str(has_children))
```

#### High-level walker

For UI trees, the client provides a `LazyBrowseNode` walker that handles
sibling pagination and the `child_has_children` hint for you:

```python
async with await GalaxyRepositoryClient.connect(
    endpoint="localhost:5000",
    api_key="<gateway-api-key>",
    plaintext=True,
) as galaxy:
    roots = await galaxy.browse()
    for root in roots:
        if root.has_children_hint:
            await root.expand()
        for child in root.children:
            kind = "has children" if child.has_children_hint else "leaf"
            print(f"{child.object.tag_name} ({kind})")
```

`expand` is idempotent — calling it twice fires only one RPC,
and is safe under concurrent callers. To refresh after a Galaxy redeploy, call
`browse` again from the root.

### Watching deploy events

`GalaxyRepositoryClient.watch_deploy_events` opens a server-streaming
subscription that emits the current cached deploy state immediately and
then one `DeployEvent` per new Galaxy deploy. `sequence` is monotonic per
gateway start; gaps mean events were dropped from the per-subscriber
buffer. Pass `last_seen_deploy_time` to suppress the bootstrap event when
the caller already has the current state cached:

```python
from datetime import datetime, timezone
from zb_mom_ww_mxgateway import DeployEvent, GalaxyRepositoryClient

async with await GalaxyRepositoryClient.connect(
    endpoint="localhost:5000",
    api_key="<gateway-api-key>",
    plaintext=True,
) as galaxy:
    last_seen: datetime | None = None
    async for event in galaxy.watch_deploy_events(last_seen_deploy_time=last_seen):
        assert isinstance(event, DeployEvent)
        print(
            f"#{event.sequence} deploy={event.time_of_last_deploy.ToDatetime(tzinfo=timezone.utc)} "
            f"objects={event.object_count} attributes={event.attribute_count}"
        )
        if event.time_of_last_deploy_present:
            last_seen = event.time_of_last_deploy.ToDatetime(tzinfo=timezone.utc)
```

The method returns an async iterator yielding the generated `DeployEvent`
proto. Breaking out of the loop, calling `aclose()` on the iterator, or
cancelling the surrounding task closes the underlying gRPC stream
cleanly. The streaming RPC requires the same `metadata:read` scope as
the other Galaxy methods.

The CLI exposes the Galaxy Repository RPCs through five subcommands that
mirror the other clients:

```bash
mxgw-py galaxy-test-connection --plaintext --json
mxgw-py galaxy-last-deploy --plaintext --json
mxgw-py galaxy-discover --plaintext --json
mxgw-py galaxy-browse --plaintext --json
mxgw-py galaxy-watch --plaintext --json
```

`galaxy-watch` is bounded by `--max-events` (default `1`) and `--timeout`
(seconds) so it always terminates; pass `--last-seen-deploy-time` (an
ISO-8601 timestamp) to suppress the bootstrap event when it matches the
current cached deploy time.

`galaxy-browse` wraps the lazy `LazyBrowseNode` walker. Without `--depth`
it lists only the root objects; `--depth N` eagerly expands `N` further
levels before printing. Text output is a node count followed by an indented
tree (`+`/`-` marks the server's has-children hint); `--json` emits nested
`{..., "hasChildrenHint": bool, "children": [...]}` nodes that match the
`galaxy-discover` object shape. The `BrowseChildrenRequest` filters are
exposed as `--category-id` (repeatable), `--template-chain-contains`
(repeatable), `--tag-name-glob`, `--include-attributes`,
`--alarm-bearing-only`, and `--historized-only`, all AND-combined.

## Authentication And TLS

`ClientOptions.api_key` adds this metadata to unary calls and streams:

```text
authorization: Bearer <api-key>
```

The client supports plaintext channels for local development, TLS with system
roots, TLS with a custom `ca_file`, and an optional test server name override.
API keys are redacted from option repr output and CLI error output.

The gateway can auto-generate its own self-signed certificate (it has no PKI).
grpc-python has no per-channel skip-verify, so the lenient TLS default is
**trust-on-first-use**: with no `ca_file` and `require_certificate_validation`
left `False`, the client fetches the gateway's presented certificate once
(unverified) and pins it for the channel, defaulting the SNI/target-name override
to `localhost` (the generated certificate always carries a `localhost` SAN) when
none was supplied. To verify instead, pass `ca_file` to verify against a specific
CA, or set `require_certificate_validation=True` to verify against the system
trust roots. The strict posture is reachable through every documented entry
point: the `require_certificate_validation=True` keyword on
`GatewayClient.connect(...)` / `GalaxyRepositoryClient.connect(...)`, the
`ClientOptions(require_certificate_validation=True)` struct, and the
`--require-certificate-validation` CLI flag. See
[Gateway Configuration](../../docs/GatewayConfiguration.md#automatic-self-signed-certificate).

## CLI

The CLI emits deterministic JSON for automation:

```powershell
mxgw-py version --json
mxgw-py open-session --endpoint localhost:5000 --plaintext --json
mxgw-py register --session-id <id> --client-name python-client --json
mxgw-py add-item --session-id <id> --server-handle 1 --item Object.Attribute --json
mxgw-py advise --session-id <id> --server-handle 1 --item-handle 2 --json
mxgw-py stream-events --session-id <id> --max-events 1 --json
mxgw-py stream-alarms --max-messages 1 --json
mxgw-py acknowledge-alarm --reference "\\Galaxy\Area001.Pump001.PumpFault" --json
mxgw-py write --session-id <id> --server-handle 1 --item-handle 2 --type int32 --value 123 --json
```

Use `--api-key` or `--api-key-env MXGATEWAY_API_KEY` to attach API key
metadata. `smoke` opens a session, registers, adds an item, advises, streams a
bounded event count, and closes the session in a `finally` block.

Use TLS options for a secured gateway:

```powershell
mxgw-py smoke --endpoint mxgateway.example.local:5001 --tls --ca-file C:\certs\mxgateway-ca.pem --server-name-override mxgateway.example.local --api-key-env MXGATEWAY_API_KEY --item Object.Attribute --json
```

To force certificate validation against the system trust store instead of the
lenient trust-on-first-use default, add `--require-certificate-validation`:

```powershell
mxgw-py smoke --endpoint mxgateway.example.local:5001 --tls --require-certificate-validation --api-key-env MXGATEWAY_API_KEY --item Object.Attribute --json
```

## Integration Checks

Run live checks only when a gateway and MXAccess-backed worker are available:

```powershell
$env:MXGATEWAY_INTEGRATION = '1'
$env:MXGATEWAY_ENDPOINT = 'localhost:5000'
$env:MXGATEWAY_API_KEY = '<gateway-api-key>'
$env:MXGATEWAY_TEST_ITEM = 'Object.Attribute'
mxgw-py smoke --endpoint $env:MXGATEWAY_ENDPOINT --plaintext --api-key-env MXGATEWAY_API_KEY --item $env:MXGATEWAY_TEST_ITEM --json
```

## Installing from the Gitea PyPI Feed

The client publishes to the internal Gitea PyPI feed:

````bash
pip install \
    --index-url https://gitea.dohertylan.com/api/packages/dohertj2/pypi/simple/ \
    zb-mom-ww-mxaccess-gateway-client
````

If you need authentication (private feed), use `--extra-index-url` and either
a `~/.netrc` entry or `PIP_INDEX_URL=https://<user>:<token>@gitea.dohertylan.com/...`.

## Related Documentation

- [Client Packaging](../../docs/ClientPackaging.md)
- [Client Proto Generation](../../docs/ClientProtoGeneration.md)
- [Python Client Detailed Design](./PythonClientDesign.md)