dohertj2/wwtools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

mxaccesscli: add read-batch / write-batch / subscribe-batch (JSONL input)

Browse Source 
Three new subcommands that take a JSONL file (or '-' for stdin) and reuse a
single MxSession across all entries. The big win is in write-batch: a
two-phase pipeline (Advise all -> drain DataChange to resolve types; Write
all -> drain WriteComplete) reduces wall time from N x (resolve + write_ack)
to ~max(resolve) + ~max(write_ack). Measured 38.2s -> 10.3s (~3.7x) for
four writes against the ZB dev galaxy; the saving grows with N.

Per-item continue-on-error: parse errors are collected line-by-line and
abort with exit 2 before any LMX session opens; runtime failures (resolve
timeout, bad references, coerce errors, write timeouts) get their own
results[] row with a typed `error` string and exit 1. Auth flags mirror
`mxa write` and are resolved once before Phase A.

Shared infra:
  - Mx/JsonlInputReader.cs: lazy line reader (skips blank / '#' lines),
    bare-string or {"tag":"..."} for read/sub, {"tag","value","type"?} for
    write, with array-suffix consistency check at parse time.
  - Mx/ValueCoercion.cs: new CoerceJToken(...) wrapper preserves the
    single source of truth for type vocabulary.

Docs:
  - README run examples extended for each new command.
  - docs/usage.md: new "Batch input format" subsection (shared contract),
    one section per command with envelope examples and a full
    failure-mode table for write-batch, plus a "Batch commands -
    verified live" section capturing the 2026-05-10 ZB-galaxy run and
    pipelining-timing numbers.
  - test-fixtures/ holds the exact JSONL files used in the verified-live
    run so the doc numbers are reproducible.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Joseph Doherty
2026-05-11 08:34:46 -04:00
parent 6cde4d9fe4
commit 2e937228a0
10 changed files with 1260 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
mxaccesscli/docs/usage.md
+218
View File
@@ -11,6 +11,42 @@ Read, write, and subscribe to AVEVA System Platform tags via MxAccess. The CLI r
- **Subsequent events are fast.** Once a tag is bound, value-change updates propagate within ~100 ms.
- **Exit codes:** `0` on success, `1` if any operation timed out or returned a non-Ok / non-Pending `MxStatusCategory`, `2` on argument-validation errors.
## Batch input format
The three `-batch` commands (`read-batch`, `write-batch`, `subscribe-batch`) all
take a single positional `<input>` argument which is either a path to a JSONL
file or `-` to read JSONL from stdin. The line conventions are shared:
- **Blank lines and lines starting with `#`** are skipped. Line numbers still
count them, so error messages stay accurate against the source file.
- **Per-line auth override is not supported in v1.** All entries in a
`write-batch` use the same `--username` / `--password` / `--user-id` /
`--secured` / `--verifier-*` flags supplied on the command line.
- **Duplicate tags are allowed.** Each input line is an independent entry
with its own item handle and its own row in `results[]`.
Read / subscribe lines accept **either** a bare JSON string or an object form:
```jsonl
"TestMachine_001.Speed"
{"tag": "Reactor1.Level"}
```
Write lines are object form only:
```jsonl
{"tag": "TM.Setpoint", "value": 42.5, "type": "double"}
{"tag": "TM.RunFlag", "value": true}
{"tag": "TM.Parts[]", "value": ["", "11111", ""], "type": "string"}
```
`value` may be a JSON `bool`, `number`, `string`, or `array`. A JSON array
requires the tag to end in `[]` (whole-array reference, matches the `mxa write`
rule). `type` is optional; when omitted the JSON token's native type drives the
default. When set, `type` accepts the same vocabulary as `mxa write --type`
(`bool`, `byte`, `short`, `int`, `long`, `float`, `double`, `string`,
`datetime`).
## `mxa info`
Print the loaded `ArchestrA.MxAccess` assembly identity, supported `--type` values, and the full `MxStatusCategory` enum. No tag access.
@@ -193,6 +229,188 @@ LLM-JSON output (one event per line, no surrounding `[ ... ]`):
JSON Lines lets a downstream consumer parse events incrementally rather than buffering the whole stream — the right shape for indefinite or long-running subscriptions.
## `mxa read-batch <input>`
Reads many tags from a JSONL file (or `-` for stdin) in **one** MxSession,
amortizing the 3–8 s LMX bind cost across the whole batch. Line shape is
defined in [Batch input format](#batch-input-format). Result rows arrive in
input-line order regardless of which `OnDataChange` fires first.
| Option | Default | Notes |
| --- | --- | --- |
| `-t`, `--timeout <seconds>` | `5` | Wall budget for the whole batch (not per-tag). Tags that haven't delivered a `DataChange` by the deadline are reported with `error: "timeout"`. |
| `--client <name>` | `mxa` | Passed to `Register()`. |
| `--llm-json` | off | Emit the JSON envelope. |
Example:
```powershell
mxa read-batch tags.jsonl --llm-json
echo '"TM.Speed"' | mxa read-batch - --llm-json
```
LLM-JSON envelope (one results row per non-blank input line, carrying the
1-based `line` for traceability):
```json
{
"query": { "command": "read-batch", "input": "tags.jsonl", "timeout_s": 5.0, "client": "mxa" },
"ok": true,
"results": [
{ "tag": "TM.Speed", "line": 1, "ok": true, "value": 1234.5, "quality": 192, "timestamp": "...", "statuses": [...] },
{ "tag": "Reactor1.Level","line": 2, "ok": true, "value": 78.1, "quality": 192, "timestamp": "...", "statuses": [...] }
]
}
```
Exit code: `0` if every entry resolved Ok; `1` if any timed out or returned a
non-Ok status; `2` for parse errors, missing input, or empty input.
## `mxa write-batch <input>`
Writes many tag/value pairs from a JSONL file (or `-`) in **one** MxSession,
**pipelining** per-item resolve and write so wall time is roughly
`max(resolve_latency) + max(write_ack)` instead of `N × (resolve_latency +
write_ack)`. Auth is resolved once before any item is touched.
Line shape: see [Batch input format](#batch-input-format). Auth flags follow
the same semantics as `mxa write` — see [Authentication](#authentication) for
how `--username` / `--domain` / `--secured` / `--verifier-*` interact. Per-line
auth override is **out of scope for v1** (planned follow-up).
| Option | Default | Notes |
| --- | --- | --- |
| `-t`, `--timeout <seconds>` | `5` | Per-phase wall budget. Phase A (resolve types) and Phase B (await `OnWriteComplete`) each get this many seconds. |
| `--user-id <int>` | `0` | Pre-resolved authenticated user id. See `mxa write` for caveats. |
| `-u`, `--username <name>` | (none) | Galaxy / OS username. Resolved to a userId via `AuthenticateUser` once before Phase A. |
| `--domain <name>` | (none) | Combined with `--username` as `<domain>\<username>`. |
| `-p`, `--password <pwd>` | (none) | Password for `--username`. Redacted in the LLM-JSON `query` echo. |
| `--secured` | off | Route writes through `WriteSecured(currentUserId, verifierUserId, value)`. Required for Secured Write / Verified Write classifications. |
| `--verifier-username` / `--verifier-domain` / `--verifier-password` | (none) | Two-person verified write; implies `--secured`. |
| `--client <name>` | `mxa` | Passed to `Register()`. |
| `--llm-json` | off | Emit the JSON envelope. |
Example:
```powershell
# Set up Hi/HiHi/Lo/LoLo limits and enable alarms for a tag in one shot:
mxa write-batch limits.jsonl --llm-json
```
```jsonl
{"tag":"BCD_T1.HighLimit", "value": 80, "type": "float"}
{"tag":"BCD_T1.HighHighLimit", "value": 95, "type": "float"}
{"tag":"BCD_T1.LowLimit", "value": 20, "type": "float"}
{"tag":"BCD_T1.LowLowLimit", "value": 5, "type": "float"}
{"tag":"BCD_T1.HighAlarm.Alarmed", "value": true}
{"tag":"BCD_T1.HighHighAlarm.Alarmed", "value": true}
{"tag":"BCD_T1.LowAlarm.Alarmed", "value": true}
{"tag":"BCD_T1.LowLowAlarm.Alarmed", "value": true}
```
LLM-JSON envelope (per-entry row with auth attribution and `line` echo):
```json
{
"query": { "command": "write-batch", "input": "limits.jsonl", "entries": 8,
"timeout_s": 5.0, "user_id": 0, "verify_user": null, "secured": false, "client": "mxa" },
"ok": true,
"results": [
{ "tag": "BCD_T1.HighLimit", "line": 1, "ok": true, "error": null,
"authenticated": false, "auth_user_id": null, "secured": false,
"verifier_user_id": null, "statuses": [...] }
]
}
```
### Failure modes — `write-batch`
| `error` string | Cause | Exit |
| --- | --- | --- |
| (top-level) `parse-error` | One or more lines failed JSONL / schema validation; `results[]` lists each. | 2 |
| (top-level) `empty-input` | File / stdin contained no non-blank entries. | 2 |
| (top-level) `authentication-failed` | `AuthenticateUser` returned 0 for the operator credentials. **No items attempted.** | 1 |
| (top-level) `verifier-authentication-failed` | Same for the verifier in two-person Verified Write. | 1 |
| per-entry `add-item-failed: …` | `LMXProxyServer.AddItem` threw (typically a malformed reference). | 1 |
| per-entry `timeout-resolving-type` | No `OnDataChange` arrived for this item before `--timeout` elapsed. | 1 |
| per-entry `type-resolution-failed` | First `OnDataChange` carried a non-Ok status — `statuses` filled. | 1 |
| per-entry `value-coerce-failed: …` | JSON value couldn't be converted to the configured `--type` (or inferred type). The Write was **not** queued. | 1 |
| per-entry `write-call-failed: …` | The `Write` / `WriteSecured` COM call itself threw before `OnWriteComplete`. | 1 |
| per-entry `timeout` | No `OnWriteComplete` arrived before `--timeout` elapsed. | 1 |
| per-entry `write-failed` | `OnWriteComplete` arrived with non-Ok statuses (e.g. read-only attr, security denied). | 1 |
The process exits `1` if any per-entry row failed, `0` if every row was Ok.
The envelope's top-level `ok` matches.
## `mxa subscribe-batch <input>`
Subscribes to many tags from a JSONL file (or `-`) and streams `OnDataChange`
events for a duration. The streaming output is identical to `mxa subscribe` —
no envelope wrap; one event per line when `--llm-json` is set.
Line shape: see [Batch input format](#batch-input-format).
| Option | Default | Notes |
| --- | --- | --- |
| `-s`, `--seconds <seconds>` | `10` | Wall-clock duration of the subscription. |
| `--max <int>` | `1000` | Hard cap on emitted events. |
| `--client <name>` | `mxa` | Passed to `Register()`. |
| `--llm-json` | off | JSON Lines mode — one event per line, no outer envelope. |
First-event latency is still 3–8 s per tag, but `Advise()` is issued for every
tag up front so the binds run in parallel — wall time matches a single-tag
subscribe. For short `--seconds` windows (< 5 s) you may miss the initial
values; budget accordingly.
Example:
```powershell
mxa subscribe-batch tags.jsonl -s 30 --llm-json
```
Output stream (each event as one JSON line):
```jsonl
{"tag":"TM.Speed","ok":true,"value":1234.5,"quality":192,"timestamp":"...","statuses":[...]}
{"tag":"Reactor1.Level","ok":true,"value":78.1,"quality":192,"timestamp":"...","statuses":[...]}
```
Exit code: `0` on a clean run, `1` if every tag failed to subscribe (no item
ever bound), `2` for parse errors / missing input / empty input.
## Batch commands — verified live
Captured on 2026-05-10 against the dev `ZB` galaxy (System Platform 2017
Express, MxAccess `3.2.0.0`). The galaxy is configured permissively
(`eNone` / Free Access) so writes were issued anonymously.
| Scenario | Result |
| --- | --- |
| `read-batch` 3 entries (bare-string + object + duplicate, with blank/`#` lines) | All three returned in input-line order; `line` field correctly reflected `1, 2, 5`. |
| `read-batch` continue-on-error (`NoSuchObject.NoSuchAttribute` mixed with `DevPlatform.CPULoad`) | Good entry returned value, bad entry returned `Category=MxCategoryConfigurationError, Detail=6`. Exit 1. |
| `write-batch` 5 entries (1 attribute with `security_classification=5` failed `SecurityError, Detail=1007`; rest wrote anonymously) | 4/5 wrote; per-item statuses preserved; exit 1. Round-trip read confirmed `TuneValue=7.25`, `ProtectedValue=true`, `ProtectedValue1=false`, `MoveInPartNumbers[1]="PN-42"`. |
| `write-batch` failure-mode coverage (`value-coerce-failed` + `type-resolution-failed` + ok, same `TuneValue` tag duplicated) | Each failure surfaced its distinct `error` string; duplicate tag entries are independent handles. |
| `subscribe-batch` two tags, 18-second window | Both tags bound in parallel; CPULoad streamed ~26 `OnDataChange` events; `SystemStartupTime` delivered its initial value then stayed quiet (constant). Exit 0. |
| Parse error on stdin (`{ malformed json }`) | `results[]` row with `error: "parse-error"`, line + reason, no LMX session opened. Exit 2. |
| Missing input file | Top-level `error` envelope, exit 2. |
### Pipelining timing
Same four writes (`OtOpcUaParityTest_001.TuneValue`,
`TestMachine_001.ProtectedValue`, `TestMachine_001.ProtectedValue1`,
`MESReceiver_001.MoveInPartNumbers[1]`) executed two ways, wall-clock:
| Method | Wall time | Notes |
| --- | --- | --- |
| 4 × `mxa write` (sequential subprocess invocations) | **38.2 s** | Each invocation pays the 3–8 s LMX bind once. |
| 1 × `mxa write-batch` (4 writes in one MxSession, two-phase pipeline) | **10.3 s** | Includes a 5th entry that failed `SecurityError` — still resolved in Phase A. |
≈ **3.7× faster** at N=4. The cost model is roughly `~max(resolve_latency)
+ ~max(write_ack)` for the batched form versus `N × (resolve_latency +
write_ack)` for the sequential form, so the speedup grows with N: at ~10
writes the savings are typically 60–80 seconds, which is the scale of the
test-plan setup phases this command was added for.
## Type support matrix
Verified end-to-end against the live `ZB` galaxy (System Platform 2017 Express, MxAccess `3.2.0.0`). Each row records what the wire shape looks like in the JSON envelope.