dohertj2/mxaccessgw
1
0
Fork 0
Code Issues 3 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

Merge branch 'fix/archreview-p2' into main (P2 tier: completeness & polish)

Browse Source 
# Conflicts:
#	archreview/remediation/00-tracking.md
#	clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli/MxGatewayCliSecretRedactor.cs
#	clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli/MxGatewayClientCli.cs
#	src/ZB.MOM.WW.MxGateway.Worker.Tests/Ipc/WorkerFrameProtocolTests.cs
This commit is contained in:
Joseph Doherty
2026-07-12 22:12:41 -04:00
parent b86c6bb47f dca09e7e06
commit 4f5371fdec
91 changed files with 7680 additions and 681 deletions
Download Patch File Download Diff File Expand all files Collapse all files
clients/python/README.md
+61 -11
View File
@@ -105,6 +105,40 @@ terminate the stream.
Canceling a Python task cancels the client-side gRPC call or stream wait. It
does not abort an in-flight MXAccess COM call inside the worker process.
### Event streaming and reconnect gaps
`Session.stream_events()` yields an async iterator whose items are either a
normal `MxEvent` or a `ReplayGap`. Track the `worker_sequence` of the last event
you processed and pass it back as `after_worker_sequence` to resume after a
disconnect:
```python
from zb_mom_ww_mxgateway import ReplayGap
cursor = 0
async for item in session.stream_events(after_worker_sequence=cursor):
if isinstance(item, ReplayGap):
# The gateway dropped events between item.requested_after_sequence and
# item.oldest_available_sequence — they are gone from the replay ring.
# Discard local tag/alarm state and re-snapshot (e.g. read_bulk /
# query_active_alarms), then resume without another gap:
cursor = item.resume_after_worker_sequence # oldest_available_sequence - 1
continue
# Normal MXAccess event.
cursor = item.worker_sequence
handle(item)
```
`ReplayGap` is the gateway's reconnect-replay gap sentinel made typed and
observable. It is delivered only at the head of a stream resumed with a non-zero
`after_worker_sequence` when the requested cursor predates the oldest retained
event. It is a **non-terminal** signal — the stream continues with normal events
after it — and it is never yielded as an `MxEvent`, so it can never be mistaken
for a real MXAccess event. The client does not synthesize or swallow it; the
gateway only sets it on `StreamEvents` results (never on a fresh stream or a
`DrainEvents` reply). `GatewayClient.stream_events_raw` remains the raw protobuf
stream (the sentinel arrives there as an `MxEvent` with `replay_gap` set).
## Write Semantics And Common Pitfalls
These are MXAccess parity behaviors that surprise new callers. The gateway
@@ -119,19 +153,11 @@ but still need the write attributed to a user id, you must first advise the
item supervisory and then pass that user id on the write. Without the
supervisory advise the `user_id` on a plain write is ignored.
The session exposes `advise`/`unadvise` but not supervisory advise, so send it
through the generic command channel:
The session exposes a typed `advise_supervisory` helper alongside
`advise`/`unadvise`:
```python
await session.invoke(
pb.MxCommand(
kind=pb.MX_COMMAND_KIND_ADVISE_SUPERVISORY,
advise_supervisory=pb.AdviseSupervisoryCommand(
server_handle=server_handle,
item_handle=item_handle,
),
)
)
await session.advise_supervisory(server_handle, item_handle)
await session.write(server_handle, item_handle, value, user_id=user_id)
```
@@ -139,6 +165,30 @@ await session.write(server_handle, item_handle, value, user_id=user_id)
The CLI exposes the same command as `advise-supervisory`, and `write` /
`write2` take `--user-id`.
For the verified/secured path, `authenticate_user`, `write_secured`,
`write_secured2`, and `archestra_user_to_id` are typed session helpers too. The
credential passed to `authenticate_user` and the values written by
`write_secured`/`write_secured2` are treated as secrets: they are never logged
and are scrubbed from any surfaced error message. MXAccess parity is preserved —
a `write_secured` that fails because no prior `authenticate_user` +
`advise_supervisory` established a supervisory context surfaces the native
failure as `MxAccessError` rather than being silently "fixed":
```python
user_id = await session.authenticate_user(server_handle, "operator", password)
await session.advise_supervisory(server_handle, item_handle)
await session.write_secured(
server_handle,
item_handle,
value,
current_user_id=user_id,
verifier_user_id=user_id,
)
```
The CLI mirrors these as `authenticate-user` (credential via `--password` or,
preferably, `--password-env`) and `write-secured`.
### Array writes replace the whole array
A write to an array attribute **replaces the entire array**; it is not an
clients/python/src/zb_mom_ww_mxgateway/__init__.py
+2
View File
@@ -9,6 +9,7 @@ from .generated.galaxy_repository_pb2 import (
GalaxyObject,
WatchDeployEventsRequest,
)
from .events import ReplayGap
from .errors import (
MxAccessError,
MxGatewayAuthenticationError,
@@ -43,6 +44,7 @@ __all__ = [
"MxGatewayTransportError",
"MxGatewayWorkerError",
"MxValueView",
"ReplayGap",
"Session",
"WatchDeployEventsRequest",
"__version__",
clients/python/src/zb_mom_ww_mxgateway/events.py
+63
View File
@@ -0,0 +1,63 @@
"""Typed event-stream signals for the MXAccess Gateway Python client."""
from __future__ import annotations
from dataclasses import dataclass
from .generated import mxaccess_gateway_pb2 as pb
@dataclass(frozen=True, slots=True)
class ReplayGap:
"""Reconnect-replay gap signal surfaced on a resumed event stream.
The gateway emits this at the head of a stream resumed with
:meth:`Session.stream_events`'s ``after_worker_sequence`` cursor when the
requested sequence predates the oldest event still retained in the gateway's
replay ring. That means the events between ``requested_after_sequence`` and
``oldest_available_sequence`` were dropped from the ring and can no longer be
replayed — the client has an unrecoverable hole in its event history.
``ReplayGap`` is a *non-terminal, observable* signal: the stream keeps
delivering normal :class:`~zb_mom_ww_mxgateway.generated.mxaccess_gateway_pb2.MxEvent`
values after it. :meth:`Session.stream_events` yields it as a distinct type
(never as an ``MxEvent``) so a consumer can branch on
``isinstance(item, ReplayGap)`` and never mistake a gap for a real MXAccess
event. The client neither synthesizes nor swallows the gateway's sentinel —
it only makes that sentinel typed and observable.
On seeing a gap the consumer must discard any locally cached tag/alarm state
and re-snapshot (for example via :meth:`Session.read_bulk` or
:meth:`~zb_mom_ww_mxgateway.GatewayClient.query_active_alarms`). To resume
the stream without provoking another gap, reconnect with
``after_worker_sequence = gap.resume_after_worker_sequence`` (that is,
``oldest_available_sequence - 1``) so the next replayed event is the oldest
the gateway still retains.
The gateway sets this only on ``StreamEvents`` results — never on a normal
(non-resumed) stream and never on a ``DrainEvents`` reply.
"""
requested_after_sequence: int
"""The ``after_worker_sequence`` cursor the resumed stream was opened with."""
oldest_available_sequence: int
"""Oldest worker sequence the gateway can still replay."""
@classmethod
def from_proto(cls, gap: pb.ReplayGap) -> "ReplayGap":
"""Build a :class:`ReplayGap` from the generated ``ReplayGap`` message."""
return cls(
requested_after_sequence=gap.requested_after_sequence,
oldest_available_sequence=gap.oldest_available_sequence,
)
@property
def resume_after_worker_sequence(self) -> int:
"""``after_worker_sequence`` to resume the stream without another gap.
Equal to ``oldest_available_sequence - 1`` so the next event the gateway
replays is ``oldest_available_sequence`` — the oldest it still retains.
Clamped at ``0`` so it is never negative.
"""
return max(self.oldest_available_sequence - 1, 0)
clients/python/src/zb_mom_ww_mxgateway/session.py
+327 -4
View File
@@ -4,7 +4,9 @@ from __future__ import annotations
from collections.abc import AsyncIterator, Sequence
from .errors import ensure_mxaccess_success
from .auth import redact_secret
from .errors import MxGatewayError, ensure_mxaccess_success
from .events import ReplayGap
from .generated import mxaccess_gateway_pb2 as pb
from .values import MxValueInput, to_mx_value
@@ -568,18 +570,304 @@ class Session:
correlation_id=correlation_id,
)
async def _invoke_redacted(
self,
command: pb.MxCommand,
*,
correlation_id: str,
secrets: Sequence[str | None],
) -> pb.MxCommandReply:
"""Invoke a command whose request carries credential-sensitive data.
Runs the same gateway + MXAccess validation as :meth:`invoke`, but scrubs
the supplied secret substrings from any surfaced error message before it
propagates. MXAccess parity is preserved — the native failure is still
raised as :class:`~zb_mom_ww_mxgateway.errors.MxAccessError`; only the
credential text is removed from the message so it can never reach logs.
"""
try:
return await self.invoke(command, correlation_id=correlation_id)
except MxGatewayError as error:
_redact_error(error, secrets)
raise
async def advise_supervisory(
self,
server_handle: int,
item_handle: int,
*,
correlation_id: str = "",
) -> None:
"""Invoke MXAccess `AdviseSupervisory` for an `ItemHandle`.
Supervisory advise is the prerequisite for user-attributed and secured
writes: it must be established (typically after :meth:`authenticate_user`)
before a ``user_id``-bearing :meth:`write` or a :meth:`write_secured`
takes effect.
"""
await self.invoke(
pb.MxCommand(
kind=pb.MX_COMMAND_KIND_ADVISE_SUPERVISORY,
advise_supervisory=pb.AdviseSupervisoryCommand(
server_handle=server_handle,
item_handle=item_handle,
),
),
correlation_id=correlation_id,
)
async def write_secured(
self,
server_handle: int,
item_handle: int,
value: MxValueInput,
*,
current_user_id: int = 0,
verifier_user_id: int = 0,
correlation_id: str = "",
) -> None:
"""Invoke MXAccess `WriteSecured` — a signed/verified write.
The written *value* is credential-sensitive and is scrubbed from any
surfaced error message (never logged). MXAccess parity is the contract:
``WriteSecured`` failing before a prior :meth:`authenticate_user` +
:meth:`advise_supervisory`, or before a value-bearing body, is the native
behaviour and is surfaced as-is — it is not "fixed".
"""
await self._invoke_redacted(
pb.MxCommand(
kind=pb.MX_COMMAND_KIND_WRITE_SECURED,
write_secured=pb.WriteSecuredCommand(
server_handle=server_handle,
item_handle=item_handle,
current_user_id=current_user_id,
verifier_user_id=verifier_user_id,
value=to_mx_value(value),
),
),
correlation_id=correlation_id,
secrets=_value_secrets(value),
)
async def write_secured2(
self,
server_handle: int,
item_handle: int,
value: MxValueInput,
timestamp_value: MxValueInput,
*,
current_user_id: int = 0,
verifier_user_id: int = 0,
correlation_id: str = "",
) -> None:
"""Invoke MXAccess `WriteSecured2` — a signed/verified, timestamped write.
Like :meth:`write_secured` but also stamps a client-supplied timestamp.
The written *value* is credential-sensitive and is scrubbed from any
surfaced error message. Native pre-condition failures are surfaced as-is.
"""
await self._invoke_redacted(
pb.MxCommand(
kind=pb.MX_COMMAND_KIND_WRITE_SECURED2,
write_secured2=pb.WriteSecured2Command(
server_handle=server_handle,
item_handle=item_handle,
current_user_id=current_user_id,
verifier_user_id=verifier_user_id,
value=to_mx_value(value),
timestamp_value=to_mx_value(timestamp_value),
),
),
correlation_id=correlation_id,
secrets=_value_secrets(value),
)
async def authenticate_user(
self,
server_handle: int,
verify_user: str,
verify_user_password: str,
*,
correlation_id: str = "",
) -> int:
"""Invoke MXAccess `AuthenticateUser` and return the resolved Galaxy user id.
*verify_user_password* is a raw MXAccess credential: it is never logged
and is scrubbed from any surfaced error message. A native authentication
failure is surfaced as :class:`~zb_mom_ww_mxgateway.errors.MxAccessError`
with the credential removed.
"""
reply = await self._invoke_redacted(
pb.MxCommand(
kind=pb.MX_COMMAND_KIND_AUTHENTICATE_USER,
authenticate_user=pb.AuthenticateUserCommand(
server_handle=server_handle,
verify_user=verify_user,
verify_user_password=verify_user_password,
),
),
correlation_id=correlation_id,
secrets=[verify_user_password],
)
return reply.authenticate_user.user_id
async def archestra_user_to_id(
self,
server_handle: int,
user_id_guid: str,
*,
correlation_id: str = "",
) -> int:
"""Invoke MXAccess `ArchestrAUserToId` and return the resolved user id."""
reply = await self.invoke(
pb.MxCommand(
kind=pb.MX_COMMAND_KIND_ARCHESTRA_USER_TO_ID,
archestra_user_to_id=pb.ArchestrAUserToIdCommand(
server_handle=server_handle,
user_id_guid=user_id_guid,
),
),
correlation_id=correlation_id,
)
return reply.archestra_user_to_id.user_id
async def add_buffered_item(
self,
server_handle: int,
item_definition: str,
item_context: str = "",
*,
correlation_id: str = "",
) -> int:
"""Invoke MXAccess `AddBufferedItem` and return the new `ItemHandle`."""
reply = await self.invoke(
pb.MxCommand(
kind=pb.MX_COMMAND_KIND_ADD_BUFFERED_ITEM,
add_buffered_item=pb.AddBufferedItemCommand(
server_handle=server_handle,
item_definition=item_definition,
item_context=item_context,
),
),
correlation_id=correlation_id,
)
return reply.add_buffered_item.item_handle
async def set_buffered_update_interval(
self,
server_handle: int,
update_interval_milliseconds: int,
*,
correlation_id: str = "",
) -> None:
"""Invoke MXAccess `SetBufferedUpdateInterval` for the server handle."""
await self.invoke(
pb.MxCommand(
kind=pb.MX_COMMAND_KIND_SET_BUFFERED_UPDATE_INTERVAL,
set_buffered_update_interval=pb.SetBufferedUpdateIntervalCommand(
server_handle=server_handle,
update_interval_milliseconds=update_interval_milliseconds,
),
),
correlation_id=correlation_id,
)
async def suspend(
self,
server_handle: int,
item_handle: int,
*,
correlation_id: str = "",
) -> pb.MxStatusProxy:
"""Invoke MXAccess `Suspend` for an `ItemHandle` and return its status."""
reply = await self.invoke(
pb.MxCommand(
kind=pb.MX_COMMAND_KIND_SUSPEND,
suspend=pb.SuspendCommand(
server_handle=server_handle,
item_handle=item_handle,
),
),
correlation_id=correlation_id,
)
return reply.suspend.status
async def activate(
self,
server_handle: int,
item_handle: int,
*,
correlation_id: str = "",
) -> pb.MxStatusProxy:
"""Invoke MXAccess `Activate` for a suspended `ItemHandle` and return its status."""
reply = await self.invoke(
pb.MxCommand(
kind=pb.MX_COMMAND_KIND_ACTIVATE,
activate=pb.ActivateCommand(
server_handle=server_handle,
item_handle=item_handle,
),
),
correlation_id=correlation_id,
)
return reply.activate.status
def stream_events(
self,
*,
after_worker_sequence: int = 0,
) -> AsyncIterator[pb.MxEvent]:
"""Return an async iterator of `MxEvent` messages for this session."""
return self.client.stream_events_raw(
) -> AsyncIterator[pb.MxEvent | ReplayGap]:
"""Return an async iterator over this session's `MxEvent` stream.
Each yielded item is either a normal :class:`~...MxEvent` or a
:class:`ReplayGap`. Branch on ``isinstance(item, ReplayGap)`` — a gap is
never delivered as an ``MxEvent`` so it cannot be mistaken for a real
MXAccess event.
Pass a non-zero *after_worker_sequence* to resume a previously observed
stream. If that cursor predates the oldest event the gateway still
retains in its replay ring, the stream opens with a single
:class:`ReplayGap` sentinel (events in the gap were dropped and cannot be
replayed), then continues with normal events. On a gap, discard locally
cached state, re-snapshot, and — to resume without another gap —
reconnect with ``after_worker_sequence = gap.resume_after_worker_sequence``.
See :class:`ReplayGap` for the full semantics. The underlying protobuf
stream is available raw via ``GatewayClient.stream_events_raw``.
"""
raw = self.client.stream_events_raw(
pb.StreamEventsRequest(
session_id=self.session_id,
after_worker_sequence=after_worker_sequence,
),
)
return _surface_replay_gaps(raw)
async def _surface_replay_gaps(
raw: AsyncIterator[pb.MxEvent],
) -> AsyncIterator[pb.MxEvent | ReplayGap]:
"""Map the gateway's ``replay_gap`` sentinel event to a typed :class:`ReplayGap`.
Normal events pass through unchanged. The sentinel (``replay_gap`` set,
``family`` unspecified, body unset) is converted to a distinct
:class:`ReplayGap` so a consumer can branch on it without inspecting proto
presence, and is never yielded as an ``MxEvent``. The sentinel is forwarded
faithfully — it is neither dropped nor turned into a normal event.
Closing this generator (``aclose``) propagates to *raw* so the underlying
gRPC call is cancelled, preserving the raw stream's cancel-on-stop contract.
"""
try:
async for event in raw:
if event.HasField("replay_gap"):
yield ReplayGap.from_proto(event.replay_gap)
else:
yield event
finally:
aclose = getattr(raw, "aclose", None)
if aclose is not None:
await aclose()
def _ensure_bulk_size(name: str, count: int) -> None:
@@ -587,4 +875,39 @@ def _ensure_bulk_size(name: str, count: int) -> None:
raise ValueError(f"{name} bulk commands are limited to {MAX_BULK_ITEMS} item(s)")
def _value_secrets(value: MxValueInput) -> list[str]:
"""Return the redaction candidate strings for a credential-sensitive write value.
Secured-write payloads may carry a password or other secret. Only textual
values can appear verbatim in a surfaced error, so a string value (or a
UTF-8-decodable ``bytes`` value) is returned for scrubbing; other value
kinds have no verbatim text form to leak.
"""
if isinstance(value, str):
return [value] if value else []
if isinstance(value, bytes):
try:
decoded = value.decode("utf-8")
except UnicodeDecodeError:
return []
return [decoded] if decoded else []
return []
def _redact_error(error: MxGatewayError, secrets: Sequence[str | None]) -> None:
"""Scrub secret substrings from a raised error's message in place.
Rewrites ``error.args[0]`` (the message returned by ``str(error)``) through
the shared :func:`~zb_mom_ww_mxgateway.auth.redact_secret` seam so credential
text can never reach logs or be re-raised to a caller. The
``protocol_status`` / ``raw_reply`` context is left untouched — those hold the
gateway's own fields, which never echo the client-supplied secret.
"""
scrubbed = [secret for secret in secrets if secret]
if not scrubbed:
return
if error.args and isinstance(error.args[0], str):
error.args = (redact_secret(error.args[0], scrubbed), *error.args[1:])
from .client import GatewayClient # noqa: E402
clients/python/src/zb_mom_ww_mxgateway/version.py
+1 -1
View File
@@ -1,3 +1,3 @@
"""Package version information."""
__version__ = "0.1.0"
__version__ = "0.1.2"
clients/python/src/zb_mom_ww_mxgateway_cli/commands.py
+97 -8
View File
@@ -294,6 +294,56 @@ def advise_supervisory(**kwargs: Any) -> None:
)
@main.command("write-secured")
@gateway_options
@click.option("--session-id", required=True, help="Gateway session id.")
@click.option("--server-handle", required=True, type=int, help="MXAccess server handle.")
@click.option("--item-handle", required=True, type=int, help="MXAccess item handle.")
@click.option("--type", "value_type", default="string", show_default=True)
@click.option("--value", required=True, help="Value to write (credential-sensitive; never logged).")
@click.option("--current-user-id", default=0, type=int, show_default=True)
@click.option("--verifier-user-id", default=0, type=int, show_default=True)
@click.option("--correlation-id", default="", help="Client correlation id.")
@click.option("--json", "output_json", is_flag=True, help="Emit JSON output.")
def write_secured(**kwargs: Any) -> None:
"""Invoke MXAccess WriteSecured — a signed/verified write (credential-sensitive)."""
_run(
_write_secured(**kwargs),
output_json=kwargs["output_json"],
secrets=_secrets(kwargs) + [kwargs.get("value")],
)
@main.command("authenticate-user")
@gateway_options
@click.option("--session-id", required=True, help="Gateway session id.")
@click.option("--server-handle", required=True, type=int, help="MXAccess server handle.")
@click.option("--verify-user", required=True, help="MXAccess user name to authenticate.")
@click.option(
"--password",
default=None,
help="User password. Prefer --password-env so the secret is not visible on the command line.",
)
@click.option(
"--password-env",
default=None,
help="Environment variable holding the user password.",
)
@click.option("--correlation-id", default="", help="Client correlation id.")
@click.option("--json", "output_json", is_flag=True, help="Emit JSON output.")
def authenticate_user(**kwargs: Any) -> None:
"""Invoke MXAccess AuthenticateUser — resolve a Galaxy user id (credential-sensitive)."""
password = _resolve_password(kwargs)
kwargs["password"] = password
_run(
_authenticate_user(**kwargs),
output_json=kwargs["output_json"],
secrets=_secrets(kwargs) + [password],
)
@main.command("subscribe-bulk")
@gateway_options
@click.option("--session-id", required=True, help="Gateway session id.")
@@ -745,19 +795,58 @@ async def _advise(**kwargs: Any) -> dict[str, Any]:
async def _advise_supervisory(**kwargs: Any) -> dict[str, Any]:
async with await _connect(kwargs) as client:
session = _session(client, kwargs["session_id"])
await session.invoke(
pb.MxCommand(
kind=pb.MX_COMMAND_KIND_ADVISE_SUPERVISORY,
advise_supervisory=pb.AdviseSupervisoryCommand(
server_handle=kwargs["server_handle"],
item_handle=kwargs["item_handle"],
),
),
await session.advise_supervisory(
kwargs["server_handle"],
kwargs["item_handle"],
correlation_id=kwargs["correlation_id"],
)
return {"ok": True}
async def _write_secured(**kwargs: Any) -> dict[str, Any]:
value = _parse_value(kwargs["value"], kwargs["value_type"])
async with await _connect(kwargs) as client:
session = _session(client, kwargs["session_id"])
await session.write_secured(
kwargs["server_handle"],
kwargs["item_handle"],
value,
current_user_id=kwargs["current_user_id"],
verifier_user_id=kwargs["verifier_user_id"],
correlation_id=kwargs["correlation_id"],
)
return {"ok": True}
async def _authenticate_user(**kwargs: Any) -> dict[str, Any]:
async with await _connect(kwargs) as client:
session = _session(client, kwargs["session_id"])
user_id = await session.authenticate_user(
kwargs["server_handle"],
kwargs["verify_user"],
kwargs["password"],
correlation_id=kwargs["correlation_id"],
)
return {"userId": user_id}
def _resolve_password(kwargs: dict[str, Any]) -> str:
"""Resolve the authenticate-user password from --password or --password-env.
Prefers the explicit flag, then falls back to the named environment
variable. The resolved secret is never echoed; callers pass it into the
``secrets`` redaction list so it cannot leak through a surfaced error.
"""
password = kwargs.get("password")
if not password:
env_name = kwargs.get("password_env")
password = os.environ.get(env_name) if env_name else None
if not password:
raise click.UsageError("a password is required via --password or --password-env")
return password
async def _subscribe_bulk(**kwargs: Any) -> dict[str, Any]:
async with await _connect(kwargs) as client:
session = _session(client, kwargs["session_id"])
clients/python/tests/test_cli.py
+172
View File
@@ -645,3 +645,175 @@ def test_galaxy_browse_help_shows_parent_gobject_id() -> None:
assert result.exit_code == 0
assert "--parent-gobject-id" in result.output
class _FakeInvokeClient:
"""Async-context-manager fake whose invoke_raw returns a scripted reply.
Satisfies the session-backed CLI command bodies (register / write-secured /
authenticate-user) which build a Session over this client and call through to
``invoke_raw``. Records the last command so tests can assert credentials are
carried on the wire but never echoed to stdout.
"""
def __init__(self, reply) -> None:
self._reply = reply
self.last_request = None
async def __aenter__(self) -> "_FakeInvokeClient":
return self
async def __aexit__(self, *_exc: object) -> None:
return None
async def invoke_raw(self, request):
self.last_request = request
return self._reply
def test_authenticate_user_command_returns_user_id_without_echoing_password(
monkeypatch: pytest.MonkeyPatch,
) -> None:
from zb_mom_ww_mxgateway.generated import mxaccess_gateway_pb2 as pb
reply = pb.MxCommandReply(
session_id="s1",
kind=pb.MX_COMMAND_KIND_AUTHENTICATE_USER,
protocol_status=pb.ProtocolStatus(code=pb.PROTOCOL_STATUS_CODE_OK),
authenticate_user=pb.AuthenticateUserReply(user_id=42),
)
fake = _FakeInvokeClient(reply)
async def fake_connect(options, **_kwargs):
return fake
monkeypatch.setattr(commands_module.GatewayClient, "connect", fake_connect)
result = CliRunner().invoke(
main,
[
"authenticate-user",
"--plaintext",
"--session-id",
"s1",
"--server-handle",
"3",
"--verify-user",
"operator",
"--password",
"cli-secret-pw",
"--json",
],
)
assert result.exit_code == 0, result.output
assert json.loads(result.output)["userId"] == 42
assert "cli-secret-pw" not in result.output
# Credential is carried on the wire, not echoed.
assert fake.last_request.command.authenticate_user.verify_user_password == "cli-secret-pw"
def test_authenticate_user_reads_password_from_env(monkeypatch: pytest.MonkeyPatch) -> None:
from zb_mom_ww_mxgateway.generated import mxaccess_gateway_pb2 as pb
reply = pb.MxCommandReply(
session_id="s1",
kind=pb.MX_COMMAND_KIND_AUTHENTICATE_USER,
protocol_status=pb.ProtocolStatus(code=pb.PROTOCOL_STATUS_CODE_OK),
authenticate_user=pb.AuthenticateUserReply(user_id=7),
)
fake = _FakeInvokeClient(reply)
async def fake_connect(options, **_kwargs):
return fake
monkeypatch.setattr(commands_module.GatewayClient, "connect", fake_connect)
monkeypatch.setenv("MXGW_TEST_PW", "env-secret-pw")
result = CliRunner().invoke(
main,
[
"authenticate-user",
"--plaintext",
"--session-id",
"s1",
"--server-handle",
"3",
"--verify-user",
"operator",
"--password-env",
"MXGW_TEST_PW",
"--json",
],
)
assert result.exit_code == 0, result.output
assert "env-secret-pw" not in result.output
assert fake.last_request.command.authenticate_user.verify_user_password == "env-secret-pw"
def test_authenticate_user_requires_a_password() -> None:
result = CliRunner().invoke(
main,
[
"authenticate-user",
"--plaintext",
"--session-id",
"s1",
"--server-handle",
"3",
"--verify-user",
"operator",
"--json",
],
)
assert result.exit_code != 0
assert "password is required" in result.output
def test_write_secured_command_does_not_echo_value_on_failure(
monkeypatch: pytest.MonkeyPatch,
) -> None:
from zb_mom_ww_mxgateway.generated import mxaccess_gateway_pb2 as pb
reply = pb.MxCommandReply(
session_id="s1",
kind=pb.MX_COMMAND_KIND_WRITE_SECURED,
protocol_status=pb.ProtocolStatus(
code=pb.PROTOCOL_STATUS_CODE_MXACCESS_FAILURE,
message="WriteSecured rejected value cli-secret-value",
),
hresult=-1,
)
fake = _FakeInvokeClient(reply)
async def fake_connect(options, **_kwargs):
return fake
monkeypatch.setattr(commands_module.GatewayClient, "connect", fake_connect)
result = CliRunner().invoke(
main,
[
"write-secured",
"--plaintext",
"--session-id",
"s1",
"--server-handle",
"3",
"--item-handle",
"4",
"--value",
"cli-secret-value",
"--json",
],
)
assert result.exit_code != 0
assert "cli-secret-value" not in result.output
def test_write_secured_and_authenticate_user_commands_are_registered() -> None:
names = set(main.commands)
assert {"write-secured", "authenticate-user"} <= names
clients/python/tests/test_replay_gap.py
+106
View File
@@ -0,0 +1,106 @@
"""Tests for the typed ReplayGap signal on Session.stream_events (CLI-15)."""
from __future__ import annotations
from collections.abc import AsyncIterator
import pytest
from zb_mom_ww_mxgateway import ReplayGap, Session
from zb_mom_ww_mxgateway.generated import mxaccess_gateway_pb2 as pb
class _FakeClient:
"""Minimal client stub exposing only what Session.stream_events needs."""
def __init__(self, events: list[pb.MxEvent]) -> None:
self._events = events
self.last_request: pb.StreamEventsRequest | None = None
def stream_events_raw(
self,
request: pb.StreamEventsRequest,
) -> AsyncIterator[pb.MxEvent]:
self.last_request = request
async def _gen() -> AsyncIterator[pb.MxEvent]:
for event in self._events:
yield event
return _gen()
def _gap_sentinel(*, requested: int, oldest: int) -> pb.MxEvent:
return pb.MxEvent(
session_id="session-1",
family=pb.MX_EVENT_FAMILY_UNSPECIFIED,
replay_gap=pb.ReplayGap(
requested_after_sequence=requested,
oldest_available_sequence=oldest,
),
)
def _normal_event(worker_sequence: int) -> pb.MxEvent:
return pb.MxEvent(
session_id="session-1",
worker_sequence=worker_sequence,
family=pb.MX_EVENT_FAMILY_ON_DATA_CHANGE,
)
def _session(events: list[pb.MxEvent]) -> tuple[Session, _FakeClient]:
client = _FakeClient(events)
session = Session(client=client, session_id="session-1") # type: ignore[arg-type]
return session, client
@pytest.mark.asyncio
async def test_replay_gap_sentinel_surfaces_as_typed_signal() -> None:
session, _client = _session(
[_gap_sentinel(requested=5, oldest=10), _normal_event(10)],
)
items = [item async for item in session.stream_events(after_worker_sequence=5)]
assert isinstance(items[0], ReplayGap)
assert items[0].requested_after_sequence == 5
assert items[0].oldest_available_sequence == 10
# Resume cursor is oldest_available_sequence - 1 so the next replayed event
# is the oldest the gateway still retains.
assert items[0].resume_after_worker_sequence == 9
# Normal events after the sentinel pass through unchanged as MxEvent.
assert isinstance(items[1], pb.MxEvent)
assert not items[1].HasField("replay_gap")
assert items[1].worker_sequence == 10
@pytest.mark.asyncio
async def test_normal_events_are_unaffected() -> None:
session, _client = _session([_normal_event(1), _normal_event(2)])
items = [item async for item in session.stream_events()]
assert all(isinstance(item, pb.MxEvent) for item in items)
assert [item.worker_sequence for item in items] == [1, 2]
assert not any(isinstance(item, ReplayGap) for item in items)
@pytest.mark.asyncio
async def test_stream_events_forwards_resume_cursor() -> None:
session, client = _session([])
async for _ in session.stream_events(after_worker_sequence=42):
pass
assert client.last_request is not None
assert client.last_request.after_worker_sequence == 42
assert client.last_request.session_id == "session-1"
def test_replay_gap_resume_cursor_never_negative() -> None:
gap = ReplayGap.from_proto(
pb.ReplayGap(requested_after_sequence=0, oldest_available_sequence=0),
)
assert gap.resume_after_worker_sequence == 0
clients/python/tests/test_typed_command_helpers.py
+227
View File
@@ -0,0 +1,227 @@
"""Tests for the typed single-item command helpers (CLI-04).
Covers the parity-critical MXAccess commands promoted from raw ``Invoke`` to
typed async session helpers: ``advise_supervisory``, ``write_secured`` /
``write_secured2``, ``authenticate_user``, ``archestra_user_to_id``, and the
buffered/suspend/activate family. The credential-redaction contract for the
secured/auth helpers is asserted explicitly.
"""
from __future__ import annotations
from typing import Any
import pytest
from zb_mom_ww_mxgateway import ClientOptions, GatewayClient, MxAccessError
from zb_mom_ww_mxgateway.generated import mxaccess_gateway_pb2 as pb
class FakeUnary:
"""Records requests and pops scripted replies, matching the client's call shape."""
def __init__(self, replies: list[Any]) -> None:
self.replies = replies
self.requests: list[Any] = []
self.metadata: tuple[tuple[str, str], ...] | None = None
async def __call__(
self,
request: Any,
*,
metadata: tuple[tuple[str, str], ...],
) -> Any:
self.requests.append(request)
self.metadata = metadata
return self.replies.pop(0)
class FakeGatewayStub:
"""Minimal stub: a fixed open-session reply plus a scriptable invoke queue."""
def __init__(self, invoke_replies: list[Any]) -> None:
self.open_session = FakeUnary(
[
pb.OpenSessionReply(
session_id="session-1",
protocol_status=pb.ProtocolStatus(code=pb.PROTOCOL_STATUS_CODE_OK),
),
],
)
self.invoke = FakeUnary(invoke_replies)
self.OpenSession = self.open_session
self.Invoke = self.invoke
async def _session_with(invoke_replies: list[Any]):
stub = FakeGatewayStub(invoke_replies)
client = await GatewayClient.connect(
ClientOptions(endpoint="fake", api_key="mxgw_test_secret", plaintext=True),
stub=stub,
)
session = await client.open_session()
return session, stub
def _ok(kind: "pb.MxCommandKind.ValueType", **payload: Any) -> pb.MxCommandReply:
return pb.MxCommandReply(
session_id="session-1",
kind=kind,
protocol_status=pb.ProtocolStatus(code=pb.PROTOCOL_STATUS_CODE_OK),
**payload,
)
@pytest.mark.asyncio
async def test_advise_supervisory_sends_typed_command() -> None:
session, stub = await _session_with([_ok(pb.MX_COMMAND_KIND_ADVISE_SUPERVISORY)])
await session.advise_supervisory(12, 34)
command = stub.invoke.requests[0].command
assert command.kind == pb.MX_COMMAND_KIND_ADVISE_SUPERVISORY
assert command.advise_supervisory.server_handle == 12
assert command.advise_supervisory.item_handle == 34
@pytest.mark.asyncio
async def test_authenticate_user_returns_user_id_and_sends_credentials() -> None:
session, stub = await _session_with(
[_ok(pb.MX_COMMAND_KIND_AUTHENTICATE_USER, authenticate_user=pb.AuthenticateUserReply(user_id=77))],
)
user_id = await session.authenticate_user(12, "operator", "s3cr3t-pw")
assert user_id == 77
command = stub.invoke.requests[0].command
assert command.kind == pb.MX_COMMAND_KIND_AUTHENTICATE_USER
assert command.authenticate_user.verify_user == "operator"
# The credential is carried on the wire (redaction is about logs/errors, not the RPC).
assert command.authenticate_user.verify_user_password == "s3cr3t-pw"
@pytest.mark.asyncio
async def test_authenticate_user_scrubs_credential_from_surfaced_error() -> None:
password = "super-secret-pw"
failure = pb.MxCommandReply(
session_id="session-1",
kind=pb.MX_COMMAND_KIND_AUTHENTICATE_USER,
protocol_status=pb.ProtocolStatus(
code=pb.PROTOCOL_STATUS_CODE_MXACCESS_FAILURE,
# Simulate a gateway that unwisely echoed the credential back in the message.
message=f"authentication failed for password {password}",
),
hresult=-1,
)
session, _ = await _session_with([failure])
with pytest.raises(MxAccessError) as captured:
await session.authenticate_user(12, "operator", password)
assert password not in str(captured.value)
assert "[redacted]" in str(captured.value)
@pytest.mark.asyncio
async def test_write_secured_surfaces_native_failure_without_prior_authenticate() -> None:
"""Parity: WriteSecured failing before authenticate/advise-supervisory is surfaced as-is."""
secret_value = "priv-payload"
failure = pb.MxCommandReply(
session_id="session-1",
kind=pb.MX_COMMAND_KIND_WRITE_SECURED,
protocol_status=pb.ProtocolStatus(
code=pb.PROTOCOL_STATUS_CODE_MXACCESS_FAILURE,
message=f"WriteSecured rejected value {secret_value}",
),
hresult=-2147217407,
)
session, stub = await _session_with([failure])
with pytest.raises(MxAccessError) as captured:
await session.write_secured(12, 34, secret_value, current_user_id=5, verifier_user_id=6)
# Native failure is surfaced (not "fixed") and the raw reply is preserved...
assert captured.value.raw_reply is failure
# ...but the credential-sensitive value is scrubbed from the surfaced message.
assert secret_value not in str(captured.value)
command = stub.invoke.requests[0].command
assert command.kind == pb.MX_COMMAND_KIND_WRITE_SECURED
assert command.write_secured.current_user_id == 5
assert command.write_secured.verifier_user_id == 6
@pytest.mark.asyncio
async def test_write_secured2_sends_value_and_timestamp() -> None:
from datetime import datetime, timezone
session, stub = await _session_with([_ok(pb.MX_COMMAND_KIND_WRITE_SECURED2)])
stamp = datetime(2026, 1, 2, 3, 4, 5, tzinfo=timezone.utc)
await session.write_secured2(12, 34, 42, stamp, current_user_id=5, verifier_user_id=6)
command = stub.invoke.requests[0].command
assert command.kind == pb.MX_COMMAND_KIND_WRITE_SECURED2
assert command.write_secured2.value.int32_value == 42
assert command.write_secured2.HasField("timestamp_value")
@pytest.mark.asyncio
async def test_archestra_user_to_id_returns_user_id() -> None:
session, stub = await _session_with(
[_ok(pb.MX_COMMAND_KIND_ARCHESTRA_USER_TO_ID, archestra_user_to_id=pb.ArchestrAUserToIdReply(user_id=9))],
)
user_id = await session.archestra_user_to_id(12, "guid-123")
assert user_id == 9
assert stub.invoke.requests[0].command.archestra_user_to_id.user_id_guid == "guid-123"
@pytest.mark.asyncio
async def test_add_buffered_item_returns_item_handle() -> None:
session, stub = await _session_with(
[_ok(pb.MX_COMMAND_KIND_ADD_BUFFERED_ITEM, add_buffered_item=pb.AddBufferedItemReply(item_handle=55))],
)
item_handle = await session.add_buffered_item(12, "Object.Attribute", "ctx")
assert item_handle == 55
command = stub.invoke.requests[0].command
assert command.add_buffered_item.item_definition == "Object.Attribute"
assert command.add_buffered_item.item_context == "ctx"
@pytest.mark.asyncio
async def test_set_buffered_update_interval_sends_command() -> None:
session, stub = await _session_with([_ok(pb.MX_COMMAND_KIND_SET_BUFFERED_UPDATE_INTERVAL)])
await session.set_buffered_update_interval(12, 250)
command = stub.invoke.requests[0].command
assert command.kind == pb.MX_COMMAND_KIND_SET_BUFFERED_UPDATE_INTERVAL
assert command.set_buffered_update_interval.update_interval_milliseconds == 250
@pytest.mark.asyncio
async def test_suspend_and_activate_return_status() -> None:
session, _ = await _session_with(
[
_ok(
pb.MX_COMMAND_KIND_SUSPEND,
suspend=pb.SuspendReply(status=pb.MxStatusProxy(category=pb.MX_STATUS_CATEGORY_OK)),
),
],
)
status = await session.suspend(12, 34)
assert status.category == pb.MX_STATUS_CATEGORY_OK
session, _ = await _session_with(
[
_ok(
pb.MX_COMMAND_KIND_ACTIVATE,
activate=pb.ActivateReply(status=pb.MxStatusProxy(category=pb.MX_STATUS_CATEGORY_OK)),
),
],
)
status = await session.activate(12, 34)
assert status.category == pb.MX_STATUS_CATEGORY_OK