f7ca2c5826
The soft deadline T was armed on the host at submit, so the pipe round trip
and the time a starved worker waits to be scheduled counted against the
script: under load a trivial script reported 'script timed out', in Shadow
mode that was a script mismatch, and at T + G the host killed the worker and
struck the key.
- Protocol (additive, negotiated): Hello/Ready.worker_soft_clock and
EvalResults.soft_expired_eval_ids. protocol_version stays 1; a worker that
does not echo the flag keeps the pre-#827 host clock.
- Worker: the watchdog runs each run's soft clock from the moment an
evaluation thread starts it and reports the crossing (woken early for
budgets shorter than its 50 ms poll). It still never cancels by itself;
its fail-fast is anchored at the soft notice + G + margin.
- Host (PendingEval): the soft notice answers the caller and arms the kill G
later. The host's own timer becomes a transport deadline, T + G from submit
(restarted at a deferred run's start); passing it with no result and no
notice yields TransportTimedOut -> Unavailable 'script worker unresponsive'
(BadResourceUnavailable, recovery = the run's completion), no strike, no
kill, no Cancel. A re-trigger refused by that still-unanswered run is
Unavailable too, not 'previous run still running'. Counted as
otopcua.script.ipc.timeouts{kind}.
- Frame writer: coalescing now carries soft_expired_eval_ids (it silently
dropped them); the host processes soft notices before results in a frame.
- Shadow: unresponsive is shadow_unavailable (Unavailable), never a mismatch.
Tests: WorkerSoftClockTests suspends the real worker (SIGSTOP via libc kill /
NtSuspendProcess) - a stall past T still returns 7; a stall past T + G is a
runner fault with nothing struck; a late-starting runaway is still timed out
at T from its start and killed G after; Shadow stays match /
shadow_unavailable; both clocks enforce T and G; PendingEval state machine.
403 lines
35 KiB
Markdown
403 lines
35 KiB
Markdown
# CLAUDE.md
|
||
|
||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||
|
||
## Project Goal
|
||
|
||
Build an OPC UA server (.NET 10) that exposes industrial data sources —
|
||
including AVEVA System Platform (Wonderware) Galaxy — under a unified
|
||
Equipment-based address space. Galaxy access flows through the in-process
|
||
`GalaxyDriver` (`src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Galaxy/`) talking gRPC
|
||
to a separately installed **mxaccessgw** gateway process. The gateway owns the
|
||
MXAccess COM bitness constraint (its worker is x86 net48); everything in this
|
||
repo is .NET 10. PR 7.2 retired the legacy in-process `Galaxy.Host` /
|
||
`Galaxy.Proxy` / `Galaxy.Shared` projects + the `OtOpcUaGalaxyHost` Windows
|
||
service.
|
||
|
||
See `docs/v2/Galaxy.Performance.md` for the runtime perf surface
|
||
(tracing, metrics, soak harness).
|
||
|
||
## Related Projects (scadaproj umbrella)
|
||
|
||
This repo is one of a family of related SCADA / OT / Wonderware / OPC UA
|
||
**sister projects** that live as sibling directories under `~/Desktop/`. They are
|
||
**separate repos and separate processes**, coupled at runtime over wire protocols
|
||
(gRPC + OPC UA) — not by project/compile references — and share the
|
||
`ZB.MOM.WW.*` product namespace. The common subject is AVEVA System Platform
|
||
(Wonderware) "Galaxy" data, and `mxaccessgw` is the linchpin the others connect
|
||
through.
|
||
|
||
- **`~/Desktop/scadaproj`** ([`../scadaproj/CLAUDE.md`](../scadaproj/CLAUDE.md)) —
|
||
the umbrella/index workspace that aggregates the whole family (a high-level scan
|
||
of each sister project: purpose, location, stack, primary commands). It also hosts
|
||
the shared `ZB.MOM.WW.*` libraries — `ZB.MOM.WW.Auth`, `ZB.MOM.WW.Theme`,
|
||
`ZB.MOM.WW.Health`, `ZB.MOM.WW.Telemetry`, `ZB.MOM.WW.Configuration`, and
|
||
`ZB.MOM.WW.GalaxyRepository`. **This repo is indexed there** — see the OtOpcUa
|
||
entry in `../scadaproj/CLAUDE.md`.
|
||
- **`~/Desktop/MxAccessGateway`** (`mxaccessgw`,
|
||
[`../MxAccessGateway/CLAUDE.md`](../MxAccessGateway/CLAUDE.md)) — the gRPC gateway
|
||
that owns the MXAccess 32-bit COM bitness + STA pump (gateway x64 .NET 10 + per-session
|
||
x86 net48 worker). **OtOpcUa's `GalaxyDriver` depends on this** for live Galaxy
|
||
read/write/subscribe and Galaxy Repository browse.
|
||
- **`~/Desktop/HistorianGateway`**
|
||
([`../HistorianGateway/CLAUDE.md`](../HistorianGateway/CLAUDE.md)) — single-process
|
||
gRPC sidecar exposing the AVEVA Historian read/write API + read-only Galaxy browse
|
||
(via the shared `ZB.MOM.WW.GalaxyRepository` lib). Patterned on `mxaccessgw` but with
|
||
no COM / no x86 worker. **OtOpcUa's default historian backend** (`Driver.Historian.Gateway`)
|
||
consumes it; `ServerHistorian:Backend=SqlServer` selects a plain SQL Server historian instead (see Historian section).
|
||
- **`~/Desktop/ScadaBridge`**
|
||
([`../ScadaBridge/CLAUDE.md`](../ScadaBridge/CLAUDE.md)) — the distributed SCADA
|
||
platform (Akka.NET hub-and-spoke) whose Data Connection Layer consumes this server's
|
||
OPC UA address space.
|
||
|
||
**Propagate cross-repo changes to the umbrella index.** When a fact the index records
|
||
about OtOpcUa changes here — remote/push status, the driver set, the Galaxy data flow,
|
||
shared-lib consumption, or per-project commands — update the **OtOpcUa entry in
|
||
`../scadaproj/CLAUDE.md`** in the same change so the index never drifts from this repo.
|
||
**Since 2026-09-03 that entry is a short summary row.** The detailed OtOpcUa narrative —
|
||
change history, live-gate records, redundancy/auth/ACL behaviour, and the per-issue
|
||
consumer-impact record for ScadaBridge — lives in
|
||
`../scadaproj/.claude/skills/sister-project-history/otopcua.md` (the `sister-project-history`
|
||
skill). Append detail **there**; keep the index row to a few lines and only touch it when
|
||
the summary itself (purpose, wire contract, driver set, auth posture, commands) changes.
|
||
|
||
**The index edit belongs on `scadaproj`'s `main`, and must be pushed.** `scadaproj` is a
|
||
separate repo with its own checkout, so committing there inherits whatever branch it
|
||
happens to be on — which is usually *not* `main` and is often an unrelated feature branch
|
||
with no upstream. That silently drifts the index for as long as that branch stays
|
||
unmerged, which is the exact failure the rule above exists to prevent. Do this explicitly:
|
||
|
||
```bash
|
||
cd ~/Desktop/scadaproj
|
||
git rev-parse --abbrev-ref HEAD # note it, to restore afterwards
|
||
git stash list # never commit onto a dirty unrelated branch
|
||
git checkout main && git pull
|
||
# …edit the OtOpcUa row in CLAUDE.md and/or .claude/skills/sister-project-history/otopcua.md…
|
||
git commit -am "docs(index): <what changed about OtOpcUa>"
|
||
git push origin main
|
||
git checkout - # restore the original branch
|
||
```
|
||
|
||
Observed 2026-07-27: the `Sql` poll driver's index entry had sat unpushed on an unrelated
|
||
feature branch since 2026-07-24 because it was committed onto the current checkout, so the
|
||
index disagreed with reality for three days despite this rule. If you find index commits
|
||
stranded on a feature branch, cherry-pick them onto `main` rather than merging that branch.
|
||
|
||
> ⛔ **Wonder box remote access (2026-09-11): no `psbridge`, no `winrm`.** Both skills are retired/disabled —
|
||
> the WinRM remote-exec channel set off the security software on the wonder boxes (vd03/vq03/vq04). Do not
|
||
> run remote commands (service control, publish/swap, PowerShell) on those boxes and do not stand up a
|
||
> replacement channel until the user clears one. Still allowed: UNC/SMB file copies and logging directly
|
||
> into the SQL Servers. Deploy steps that need a command on the box are blocked; see the umbrella
|
||
> [`../scadaproj/CLAUDE.md`](../scadaproj/CLAUDE.md) for the current hold.
|
||
|
||
## Backlog / open work
|
||
|
||
Open work lives in **Gitea issues** on `gitea.dohertylan.com/dohertj2/lmxopcua` (`tea issues list --repo dohertj2/lmxopcua`) — they are the single system of record for status.
|
||
The markdown backlogs — `deferment.md`, `docs/v3/twincat-backlog.md`, `archreview/plans/STATUS.md`, `archreview/plans/INTEGRATION-SWEEP-STATUS.md`, the `docs/reviews/2026-10-21/` reports + `docs/superpowers/plans/2026-10-22-review-remediation.md`, and the known-gaps/follow-ups sections of `docs/drivers/*.md`, `docs/ScriptEditor.md`, `docs/Uns.md`, `docs/security.md`, `docs/VirtualTags.md` — are historical/design records carrying `→ #N` pointers (2026-11-06 sweep, label `migrated`, #749–#759). Do not track status in them.
|
||
File every new follow-up, deferral or review finding as an issue with the family labels `type:{bug,feature,followup,review-finding,deploy-gate,debt,docs,test,infra}`, `priority:p0..p3`, plus `blocked` / `cross-repo` where they apply.
|
||
Reference issues from plans and docs as `#N` (cross-repo: `mxaccessgw#N`).
|
||
|
||
## Subsystem guides (lazy-loaded — read before touching the area)
|
||
|
||
Detailed gotchas live in path-scoped files under `.claude/rules/`, which load automatically when you work on
|
||
matching files. **If you are changing behaviour in one of these areas and the file has not loaded, read it first.**
|
||
|
||
- `.claude/rules/drivers.md` — Galaxy change detection / `IRediscoverable`, the comms-loss watchdog and quality
|
||
contract, AB native transport, and per-driver notes (S7, MQTT, FOCAS, TwinCAT, AbLegacy, OpcUaClient, MTConnect, removal).
|
||
- `.claude/rules/cluster-topology.md` — Redundancy, `EffectiveRoles`, per-cluster meshes, `MeshTransport`,
|
||
`ConfigSource`/`ConfigServe`, telemetry transport, LocalDb pair store, DB-less driver nodes (Phase 4).
|
||
- `.claude/rules/data-plane-auth.md` — API keys on the OPC UA data plane, `Security:Acl` enforcement, anonymous
|
||
principal, equipment-node write checks, ACL scope picker; LDAP is AdminUI-only.
|
||
- `.claude/rules/alarms.md` — scripted alarm ack/shelve, Part 9 condition methods, condition severity / children.
|
||
- `.claude/rules/historian.md` — HistoryRead/HistoryUpdate across the Gateway/SqlServer/QuestDb backends, alarm
|
||
history, continuous historization (#491/#537/#538/#672), provisioning, gateway prerequisites, live gate recipe.
|
||
|
||
**Never commit secrets.** API keys, PSKs and historian keys come from env vars or `${secret:}` (e.g.
|
||
`ServerHistorian__ApiKey`); the docker-dev rig's literal dev keys are the only committed exception.
|
||
|
||
## Architecture Overview
|
||
|
||
### Data Flow
|
||
|
||
1. **Galaxy Repository DB (ZB)** — SQL Server database holding the
|
||
deployed object hierarchy and attribute definitions. The
|
||
mxaccessgw's `GalaxyRepositoryClient` queries it via gRPC; the
|
||
driver consumes the materialised hierarchy through
|
||
`IGalaxyHierarchySource`.
|
||
2. **MXAccess (via mxaccessgw)** — Live read/write/subscribe over a
|
||
gRPC session. The gateway owns the COM apartment + STA pump
|
||
server-side; the driver speaks `MxCommand` / `MxEvent` protos
|
||
exclusively.
|
||
3. **OPC UA Server** — Exposes the deployed tree under **two OPC UA namespaces**
|
||
(see below). Galaxy tags are bound by `TagConfig.FullName`
|
||
(`tag_name.AttributeName`); reads/writes/subscriptions are translated to that
|
||
reference for MXAccess.
|
||
|
||
### v3 OPC UA Address Space (Batch 4): dual namespace
|
||
|
||
The server exposes **two OPC UA namespaces** (replacing the single
|
||
`https://zb.com/otopcua/ns`), built by `AddressSpaceComposer` /
|
||
`AddressSpaceApplier` and served by `OtOpcUaNodeManager` (which registers both
|
||
URIs; `V3NodeIds` + `AddressSpaceRealm` are the identity authority):
|
||
|
||
| Namespace URI | Realm | Subtree | NodeId `s=` scheme |
|
||
|---|---|---|---|
|
||
| `https://zb.com/otopcua/raw` | `AddressSpaceRealm.Raw` | the `/raw` device tree (Folder → Driver → Device → TagGroup → Tag) | the node's **RawPath** (e.g. `Plant/Modbus/dev1/Speed`) |
|
||
| `https://zb.com/otopcua/uns` | `AddressSpaceRealm.Uns` | the UNS tree (Area → Line → Equipment → signal) | slash-joined **`Area/Line/Equipment/EffectiveName`** |
|
||
|
||
- **Single source, fanned to both.** Every device value has exactly one source —
|
||
the raw tag's node. A `UnsTagReference` projects that raw tag into an equipment
|
||
as a UNS-namespace variable that carries an **`Organizes` reference to its raw
|
||
node** and mirrors it. One driver publish for a RawPath fans (in `DriverHostActor`)
|
||
to the raw NodeId AND every referencing UNS NodeId with identical
|
||
value/quality/timestamp — the mux key stays single (RawPath).
|
||
- **Writes via either NodeId** route to the same driver ref under the same
|
||
`WriteOperate` gating (the node-manager write gate is realm-qualified and fails
|
||
closed); a failed device write reverts both NodeIds via the shared fan-out
|
||
(write-outcome self-correction).
|
||
- **Native alarms** materialize once at the raw tag (`ConditionId = RawPath`) and
|
||
fan via SDK notifiers to the raw device folder AND every referencing equipment
|
||
folder — one `ReportEvent`, one Server-object copy; ack/confirm/shelve route on
|
||
`ConditionId = RawPath`.
|
||
- **Historian dual-registration.** Both NodeIds register the **same** historian
|
||
tagname; the mux `HistorizedTagRef` stays single (RawPath). HistoryRead works
|
||
through either NodeId.
|
||
|
||
The retired `EquipmentNodeIds` (`{equipmentId}/{folderPath}/{name}`) scheme and
|
||
the single-namespace `EquipmentTags` materialization path are gone. See
|
||
`docs/plans/2026-07-15-raw-uns-two-subtree-v3-design.md` +
|
||
`docs/plans/2026-07-15-v3-batch4-address-space-plan.md`, and `docs/Uns.md`.
|
||
|
||
**How a deploy reaches that address space (surgical-deploys, 2026-08-15).** The old
|
||
blanket rule — ANY non-empty `ChangedRawTags` / `ChangedUnsReferenceVariables` routes to
|
||
`Rebuild` — is gone, because it made a routine edit a fleet-wide outage:
|
||
`RawTagPlan.ReferencingEquipmentPaths` is part of the record's equality, so adding or
|
||
dropping **one** `UnsTagReference` flipped its backing raw tag to Changed and tore down
|
||
every node on the server, killing every client subscription. `AddressSpaceChangeClassifier`
|
||
is still default-closed; what changed is which deltas it whitelists.
|
||
|
||
- **Still a full rebuild:** an **alarm-bearing** raw-tag delta (`Alarm != null` on either
|
||
side — the WP4 guard: only a rebuild clears and re-wires native-alarm notifiers), an
|
||
**identity** change on a *paired* raw-tag delta (`TagId`/`DriverInstanceId`/`ParentNodeId`/`Realm`
|
||
— pairing is by RawPath, so this guard catches a tag deleted and re-created under the same path;
|
||
a plain **rename never pairs** and is served surgically as remove-one-node + add-one-node with any
|
||
referencing UNS mirror re-pointed, live-gated 2026-08-15),
|
||
a raw **container** (Folder/Driver/Device/TagGroup) rename or removal, `ChangedEquipment` /
|
||
`ChangedAlarms`, a non-surgical equipment-tag delta, and anything else the
|
||
`with`-equality whitelist does not name (including any FUTURE field).
|
||
- **Now surgical:** a **non-alarm** raw-tag delta over the in-place-updatable attributes
|
||
(`Writable` / `IsHistorized` / `HistorianTagname` / `DataType` / `IsArray` / `ArrayLength`)
|
||
and/or `ReferencingEquipmentPaths` — applied via `ISurgicalAddressSpaceSink.UpdateTagAttributes`
|
||
in the **Raw** realm — and **every** `UnsReferenceDelta` (add / remove / re-point /
|
||
backing-edit).
|
||
- ⚠️ **UNS mirrors repair through the RECORD, not the classifier.** A raw-side historian or
|
||
shape edit would otherwise leave every referencing UNS mirror on the old tagname/shape
|
||
(`EnsureVariable` early-returns on an existing node). `UnsReferenceVariable` therefore
|
||
carries the backing tag's `Backing{IsHistorized,HistorianTagname,IsArray,ArrayLength}`, so the
|
||
same edit makes the reference record unequal and the planner emits a **coincident**
|
||
`UnsReferenceDelta`. Adding a newly-inherited field needs no classifier change — **omitting
|
||
one silently reopens the staleness.** Repair is **in place** (`UpdateTagAttributes`, Uns
|
||
realm) when the mirror's identity is unchanged, so its MonitoredItems survive; only an
|
||
identity change (re-point, effective-name edit) is remove+recreate — and only for that
|
||
one node.
|
||
- **A non-rebuild apply materialises the DELTA, not the composition.** `MaterialiseHierarchy`
|
||
still runs full (parents must exist); the other passes take the plan's added/changed subsets.
|
||
Adding one tag to a 50k-node server was ~50k sink calls on the pinned publish thread.
|
||
- ⚠️ **A degraded non-rebuild apply does NOT advance `_lastApplied`.** The full passes used to
|
||
re-ensure every node on every apply, accidentally self-healing any swallowed sink fault; the
|
||
delta path removed that. Holding the composition back is the replacement — the next deploy
|
||
re-diffs the same composition and re-ensures what failed, at the cost of re-emitting that
|
||
apply's removes (a re-remove returns false and ratchets to a rebuild, which is the correct,
|
||
bounded resync). A **rebuild** still advances even with failures.
|
||
- **…and the deploy ack now SAYS so (#601).** The Applied ack waits for `OpcUaPublishActor`'s
|
||
`RebuildCompleted` (timeout = shared `DeployTimings.ApplyDeadline` − 15 s = 105 s ⇒ degraded "outcome not reported in time", never clean) and carries an additive `Degraded`/`DegradedReason`
|
||
on both ack transports; central seals but records `NodeDeploymentState.FailureReason = "Applied degraded: …"`
|
||
(`/fleet` "Applied (degraded)"), and a re-dispatch of a degraded revision re-applies instead of short-circuiting.
|
||
See `docs/Redundancy.md` §"What an `Applied` ack means".
|
||
- **Any surgical `false`/throw still ratchets to `SafeRebuild()`** — that one-way fallback is
|
||
what makes every whitelist above safe.
|
||
- **Announcements:** `AnnounceAddedNodes` announces surgically-added **Raw** containers/tags
|
||
(`ComputeRawAddAnnouncements`, #599 — the former "raw adds are undiscoverable until a re-browse" gap is
|
||
closed) as well as added UNS reference variables.
|
||
- **Array writability rides the same whitelist:** `ArrayWritable` joins the surgical raw-tag attributes and
|
||
`UnsReferenceVariable` carries `BackingArrayWritable` (see "Equipment-node writes" under Authentication). Removing
|
||
an equipment that has references removes the references before the equipment subtree, and
|
||
`RemoveEquipmentSubtree` severs the inverse `Organizes` edges.
|
||
|
||
See `docs/plans/2026-08-15-surgical-deploys.md`, `docs/Uns.md` and `docs/Raw.md`.
|
||
|
||
### Key Concept: Tag Name and FullName
|
||
|
||
Galaxy objects have a **tag_name** — a globally unique system name used for
|
||
MXAccess read/write — and attributes are referenced as `tag_name.AttributeName`
|
||
(e.g. `DelmiaReceiver_001.DownloadPath`). This dot-separated reference is what
|
||
`TagConfig.FullName` stores for a Galaxy equipment tag. The Galaxy address
|
||
picker browses the live hierarchy using contained names for navigation, then
|
||
resolves the selection to the `tag_name.AttributeName` form.
|
||
|
||
**Galaxy is a standard Equipment-kind driver.** Galaxy points are ordinary
|
||
equipment `Tag`s bound to `GalaxyMxGateway` via `TagConfig.FullName`
|
||
(`tag_name.AttributeName`), authored through the standard Tag modal + Galaxy
|
||
address picker on the equipment page's Tags tab — the same flow as Modbus or
|
||
S7. There is no alias machinery, no `SystemPlatform` namespace kind, and no
|
||
relay→alias converter. See
|
||
`docs/plans/2026-06-12-galaxy-standard-driver-design.md` for the full design.
|
||
|
||
## mxaccessgw
|
||
|
||
The gateway lives in a sibling repo at `~/Desktop/MxAccessGateway/` (`mxaccessgw` — see [Related Projects](#related-projects-scadaproj-umbrella)). See `docs/v2/Galaxy.ParityRig.md` for the gw setup recipe (build, API key provisioning via `apikey create-key`, env-var overrides for HTTP/2 cleartext + worker path). The gw's MXAccess Toolkit reference (its `gateway.md`) is the canonical MxAccess API doc; the standalone `mxaccess_documentation.md` previously kept in this repo retired in PR 7.3.
|
||
|
||
## Build Commands
|
||
|
||
`Category=SlowUnit` (#606) tags the 16 wall-clock-bound Driver.Mqtt unit tests (blackhole connect-timeout /
|
||
reconnect-backoff windows, ~1 min of the project's 73 s). CI runs them (its filter excludes only E2E +
|
||
LiveIntegration); for a fast local loop add `--filter "Category!=SlowUnit"`.
|
||
|
||
CVE-2025-6965 / GHSA-2m69-gcr7-jv3q (transitive native `SQLitePCLRaw.lib.e_sqlite3`)
|
||
is **resolved** — the temporary `NuGetAuditSuppress` was removed once the patched bundle
|
||
shipped. `Core.AlarmHistorian` now carries a surgical direct pin of
|
||
`SQLitePCLRaw.bundle_e_sqlite3` to `2.1.12` (the first fixed native bundle, embedding the
|
||
SQLite 3.50.2+ patch), overriding the `2.1.11` that `Microsoft.Data.Sqlite` pulls
|
||
transitively. `dotnet restore`/`build` is clean with the audit active.
|
||
|
||
Test projects live under `tests/<module>/` (Core, Server, Drivers,
|
||
Drivers/Cli, Client, Tooling) — there is no single unit-test project.
|
||
Unit suites are named `*.Tests`; integration suites are `*.IntegrationTests`
|
||
and need their Docker fixture up (see Docker Workflow). DB-backed tests in
|
||
`*.Configuration.Tests`, `*.Admin.Tests`, and `*.Server.Tests` require the
|
||
central SQL Server.
|
||
|
||
## Docker Workflow (driver fixtures + central SQL Server)
|
||
|
||
> **Infra index:** [`infra/README.md`](infra/README.md) is the canonical index of all testing +
|
||
> deployment infrastructure — the Docker host layout, driver fixtures, the env-gated **live-gate**
|
||
> recipes (S7 blackhole, HistorianGateway LiveIntegration), the integration-test
|
||
> harness, the docker-dev rig, and the deploy setup. Start there; the sections below + `docs/v2/dev-environment.md` carry the detail.
|
||
|
||
> **Migrated 2026-04-28**: Docker config + host moved off this dev VM (DESKTOP-6JL3KKO) onto the shared Linux Docker host (`DOCKER`, 10.100.0.35) so the dev VM could shed WSL2/Hyper-V and have its GPU re-attached via ESXi passthrough. Docker Desktop is no longer installed here. All checked-in `appsettings.json` defaults, fixture-class default endpoints, and `e2e-config.sample.json` were rewritten to target `10.100.0.35`. See `docs/v2/dev-environment.md` for the full rewrite (header dated 2026-04-28).
|
||
|
||
> ⚠️ **The `project: lmxopcua` label is aspirational, not universal.** This note used to claim every fixture compose file carries it. As of 2026-07-24 only the **MQTT** fixture actually does (joined 2026-11-10 by the OpcUaClient port-NAT fixture, `Docker/nat/`, #813) — `grep -rn "labels:" tests --include=*.yml` matches `Driver.Mqtt.IntegrationTests/Docker/docker-compose.yml` and nothing else. So `docker ps --filter label=project=lmxopcua` returns the MQTT fixture alone, not the fleet. Add the label when you touch an older fixture; until then enumerate by container name.
|
||
|
||
Docker workloads run on a shared Linux host at **`10.100.0.35`** — not on this VM. Stacks live at `/opt/otopcua-<driver>/` on the host and carry the `project=lmxopcua` label so they're discoverable via `docker ps --filter label=project=lmxopcua`.
|
||
|
||
**`docker -H ssh://...` does NOT work from the Windows VM.** Windows OpenSSH ↔ docker.exe stdio bridging hangs (`docker system dial-stdio` runs server-side but no API data flows). Use the helper below — it SSHes into the docker host and runs `docker compose` server-side.
|
||
|
||
> **On macOS there is no `lmxopcua-fix` — drive the host over SSH directly.** The helper is a Windows-VM
|
||
> script; `~/bin` is empty on the Mac and the commands below will not resolve. Use passwordless SSH
|
||
> (and `rsync` in place of `sync`), which is what `infra/README.md` §1 documents as the Mac path:
|
||
>
|
||
> ```bash
|
||
> ssh dohertj2@10.100.0.35 'docker ps --format "{{.Names}}\t{{.Status}}"'
|
||
> ssh dohertj2@10.100.0.35 'cd /opt/otopcua-mqtt && docker compose up -d'
|
||
> rsync -av tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.IntegrationTests/Docker/ \
|
||
> dohertj2@10.100.0.35:/opt/otopcua-mqtt/ # the `sync` step, by hand
|
||
> ```
|
||
>
|
||
> A local container runtime *does* exist on the Mac (OrbStack), so the `docker-dev/` rig itself runs
|
||
> locally there — it is only the shared **fixtures** that live on `10.100.0.35`.
|
||
|
||
**Use `lmxopcua-fix.ps1` (in `~/bin`) to control fixtures from the Windows VM:**
|
||
|
||
```powershell
|
||
lmxopcua-fix ls # list all lmxopcua-tagged containers on the host
|
||
lmxopcua-fix up modbus standard # bring a profile up
|
||
lmxopcua-fix up abcip controllogix
|
||
lmxopcua-fix up s7 s7_1500
|
||
lmxopcua-fix up opcuaclient # single-service stack, no profile arg
|
||
lmxopcua-fix down modbus # tear stack down
|
||
lmxopcua-fix logs modbus
|
||
lmxopcua-fix sync modbus # rsync this repo's tests/.../Docker/ → /opt/otopcua-modbus/
|
||
```
|
||
|
||
**`sync` is the deployment step.** When you edit a fixture's compose file or Dockerfile under `tests/.../Docker/`, run `lmxopcua-fix sync <driver>` to push the changes to the docker host before bringing the stack up. The repo files are the source of truth; `/opt/otopcua-<driver>/` is a mirrored deployment.
|
||
|
||
**Endpoints (defaults already point at the docker host):**
|
||
- SQL Server (always-on): `10.100.0.35,14330` — used by `appsettings.json` for `ConfigDb`.
|
||
- Modbus: `10.100.0.35:5020` (`MODBUS_SIM_ENDPOINT`)
|
||
- AB CIP: `10.100.0.35:44818` (`AB_SERVER_ENDPOINT`)
|
||
- S7: `10.100.0.35:1102` (`S7_SIM_ENDPOINT`)
|
||
- OPC UA reference (opc-plc): `opc.tcp://10.100.0.35:50000` (`OPCUA_SIM_ENDPOINT`)
|
||
- MQTT (Mosquitto, **auth on both listeners — no anonymous fallback**): `10.100.0.35:8883` TLS+auth
|
||
(`MQTT_FIXTURE_ENDPOINT`) · `10.100.0.35:1883` plaintext-but-authenticated (`MQTT_FIXTURE_PLAIN_ENDPOINT`).
|
||
Also needs `MQTT_FIXTURE_USERNAME` / `MQTT_FIXTURE_PASSWORD`, and `MQTT_FIXTURE_CA_CERT` to pin the
|
||
fixture CA (`scp dohertj2@10.100.0.35:/opt/otopcua-mqtt/secrets/ca.crt`). A co-running publisher loops
|
||
JSON under `otopcua/fixture/…`.
|
||
- MQTT **Sparkplug B**: the same broker, plus the `sparkplug` compose profile's `otopcua-sparkplug-sim`
|
||
(project-owned C# edge-node simulator) — group **`OtOpcUaSim`**, edge nodes **`EdgeA`**/**`EdgeB`**,
|
||
`EdgeA` device **`Filler1`**, ~2 s cadence. It subscribes to `spBv1.0/OtOpcUaSim/NCMD/{node}` and
|
||
re-births on request. **Births are never retained**, so `docker restart otopcua-sparkplug-sim` is how
|
||
you force a fresh one while a browse window is open.
|
||
|
||
Override any endpoint via the env var to point at a real PLC. The local OtOpcUa server runs on this VM at `opc.tcp://localhost:4840` — **that's not on the docker host**.
|
||
|
||
**Local docker-dev rig — login is DISABLED, so do live `/run` verification yourself (don't wait for the user to sign in).** The local `docker-dev/docker-compose.yml` stack (AdminUI at `http://localhost:9200` via Traefik; OPC UA `opc.tcp://localhost:4840` central-1 / `:4841` central-2) runs the AdminUI with `Security__Auth__DisableLogin: "true"` — **no sign-in form; it's auto-authenticated as a full-access admin.** So AdminUI / Razor `/run` verification (deploy a config, drive a page, confirm behavior — e.g. via the Chrome browser-automation tools against `http://localhost:9200`) does **not** require the user to log in. Run it yourself; do not defer it as "user-driven sign-in required." (Caveat: OPC UA *data-plane* auth is still real LDAP against the shared GLAuth on `10.100.0.35:3893` — that only gates Client.CLI read/write **role** operations, e.g. binding a `multi-role` / `opc-writeop` user, and is independent of the AdminUI login. Things genuinely outside the local rig — real PLCs, or the AVEVA Historian reached via the `ZB.MOM.WW.HistorianGateway` sidecar — still need the user.)
|
||
|
||
See `docs/v2/dev-environment.md` for the full inventory and rationale.
|
||
|
||
## Build & Runtime Constraints
|
||
|
||
- Language: C#, .NET 10, AnyCPU. The MXAccess COM bitness constraint
|
||
is owned by the mxaccessgw worker (x86 net48), not by anything in
|
||
this repo.
|
||
- The gateway's MXAccess worker requires a deployed ArchestrA Platform
|
||
on the machine running the gateway. The OtOpcUa server itself does
|
||
not.
|
||
|
||
## Transport Security
|
||
|
||
The server supports configurable OPC UA transport security via the `OpcUa:EnabledSecurityProfiles` list in `appsettings.json`. Phase 1 profiles (the `OpcUaSecurityProfile` enum members): `None` (default), `Basic256Sha256Sign`, `Basic256Sha256SignAndEncrypt`. Security policies are built from the enabled profiles by `BuildSecurityPolicies` at startup. The server certificate is always created even for `None`-only deployments because `UserName` token encryption depends on it. See `docs/security.md` for the full guide.
|
||
|
||
## Library Preferences
|
||
|
||
- **Service hosting (Server, Admin)**: .NET generic host with `AddWindowsService` (decision #30 — replaced TopShelf in v2; see `src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/OtOpcUaServerHostedService.cs`)
|
||
|
||
## OPC UA .NET Standard Documentation
|
||
|
||
Use the DeepWiki MCP (`mcp__deepwiki`) to query documentation for the OPC UA .NET Standard stack: `https://deepwiki.com/OPCFoundation/UA-.NETStandard`. Tools: `read_wiki_structure`, `read_wiki_contents`, and `ask_question` with repo `OPCFoundation/UA-.NETStandard`.
|
||
|
||
## Testing
|
||
|
||
Use the Client CLI at `src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI/` for manual testing against the running OPC UA server. Supports connect, read, write, browse, subscribe, historyread, alarms, and redundancy commands. See `docs/Client.CLI.md` for full documentation.
|
||
|
||
**AdminUI interactive pages must wrap their body in `<PageErrorBoundary>`** — `MainLayout` is static, so it cannot catch an interactive page's render failure.
|
||
|
||
Address pickers in AdminUI support live browse for OpcUaClient and Galaxy drivers — see `docs/plans/2026-05-28-driver-browsers-design.md`.
|
||
|
||
The AdminUI's global **UNS** page (`/uns`) is the single surface for managing the unified namespace fleet-wide (Area → Line → Equipment → Tag/VirtualTag), replacing the old per-cluster UNS/Equipment/Tags tabs. See `docs/Uns.md`.
|
||
|
||
**v3 (Batch 2): device I/O is authored in the `/raw` project tree** (`GlobalRaw.razor` → `RawTree.razor`, `IRawTreeService`) — a cluster-rooted `Folder → Driver → Device → TagGroup → Tag` hierarchy with per-node context menus (the reusable `ContextMenu`), lazy loading, driver/device config modals (endpoint moved to `Device.DeviceConfig`; Test-connect probes the `DriverDeviceConfigMerger`-merged config), manual-entry + CSV import/export (RFC-4180, staged review grid, all-or-nothing), and browse-commit of discovered leaves into raw tags. The routed `/clusters/{id}/drivers` flow (`DriverTypePicker`, `DriverEditRouter`, the 8 `*DriverPage` shells, the Drivers list) is **retired** — its form bodies live on inside the `/raw` modals. The `DriverType` dispatch maps are keyed off the `DriverTypeNames` constants (fixing the `TwinCat`/`Focas` drift). A `Calculation` pseudo-driver (`DriverTypeNames.Calculation`, `IDependencyConsumer`, deploy-time `scriptId`-existence + Tarjan cycle gates) computes signal-level tags from other tags' RawPaths. See `docs/Raw.md`.
|
||
|
||
**v3 (Batch 3): UNS Equipment is reference-only.** The equipment **Tags** tab no longer authors or binds tags — it holds `UnsTagReference` rows pointing at raw tags authored in `/raw`. "+ Add reference" opens the `RawTree` in picker mode, **cluster-scoped** (structurally + server-enforced `tag.cluster == equipment.cluster`); rows show effective name / RawPath / inherited DataType+AccessLevel / display-name override. **Effective-name uniqueness** (references + VirtualTags + ScriptedAlarms share the `{EquipmentId}/{EffectiveName}` NodeId space) is enforced at authoring (`IEffectiveNameGuard`) and at the deploy gate (`DraftValidator.UnsEffectiveNameCollision` — catches rename-induced collisions). Scripts use **`ctx.GetTag("{{equip}}/<RefName>")`** — resolved per-equipment through `UnsTagReference` effective names to the backing RawPath at both compose seams (`AddressSpaceComposer` + `DeploymentArtifact`, via the shared `EquipmentReferenceMap`, byte-parity); an unresolved `<RefName>` is a deploy error (`EquipReferenceUnresolved`) AND a live Monaco diagnostic (`OTSCRIPT_EQUIPREF`) — editor accepts ⇔ publish accepts. `EquipmentScriptPaths.DeriveEquipmentBase` is deleted (the dot-joint `{{equip}}.X` became the slash-joint `{{equip}}/<RefName>`). Raw rename warns when a beneath-it tag is historized-without-override / UNS-referenced / named by a script literal (`RawTreeService` substring scan). `ImportEquipmentModal` dropped the `DriverInstanceId` column. See `docs/Uns.md` + `docs/ScriptEditor.md`.
|
||
|
||
**v3 (Batch 4 = v3.0): dual-namespace address space.** The OPC UA server exposes **two namespaces** — `https://zb.com/otopcua/raw` (Raw device tree, `s=<RawPath>`) + `https://zb.com/otopcua/uns` (UNS tree, `s=<Area>/<Line>/<Equipment>/<EffectiveName>`) — replacing the single `.../ns` and the retired `EquipmentNodeIds` scheme. Every value has ONE source (the raw node's publish); a driver publish for a RawPath fans in `DriverHostActor` to the raw NodeId AND every referencing UNS NodeId with identical value/quality/timestamp, and each UNS variable `Organizes`-references its raw node. Writes route via **either** NodeId to the same driver ref under the same realm-qualified `WriteOperate` gate (failed-write revert works through both); native alarms materialize ONCE at the raw tag (`ConditionId = RawPath`) and fan via SDK `AddNotifier` to the raw device folder + every referencing equipment folder (one `ReportEvent`, one Server-object copy); both NodeIds register the SAME historian tagname (mux `HistorizedTagRef` stays single, keyed by RawPath). Identity authority: `V3NodeIds` + `AddressSpaceRealm` (carried explicitly at every sink seam — never parsed out of the id string). See the "v3 OPC UA Address Space (Batch 4)" section above, `docs/Uns.md`, and `docs/plans/2026-07-15-v3-batch4-address-space-plan.md`.
|
||
|
||
The `/uns` **TagModal** uses **driver-typed tag-config editors**: it dispatches by the bound driver's `DriverType` to a per-driver editor (Modbus/S7/AbCip/AbLegacy/TwinCAT/Focas/OpcUaClient) via `TagConfigEditorMap`, with client-side validation via `TagConfigValidator`; unmapped drivers (only Galaxy) fall back to the generic raw-`TagConfig`-JSON textarea. Each editor is a thin razor shell over a pure `<Driver>TagConfigModel` (`FromJson`/`ToJson`/`Validate`, preserves unknown keys). To add a driver's editor, copy the Modbus template under `Components/Shared/Uns/TagEditors/` + `Uns/TagEditors/`, reusing the driver's enums + camelCase JSON property names, and register it in `TagConfigEditorMap` + `TagConfigValidator`. See `docs/plans/2026-06-09-driver-typed-tag-editors-design.md`.
|
||
|
||
## Scripting / Script Editor
|
||
|
||
C# virtual-tag scripts are authored in a **Roslyn-backed Monaco editor** on the
|
||
ScriptEdit page (`/scripts/{id}`) and inline inside the virtual-tag modal on the
|
||
`/uns` page. The editor provides completions, live diagnostics, hover, signature
|
||
help, document formatting, and tag-path completions inside `ctx.GetTag("…")` /
|
||
`ctx.SetVirtualTag("…")` literals — all backed by the same compiler context the
|
||
runtime publish gate uses, so what the editor accepts/rejects matches publish
|
||
exactly. The backend lives in
|
||
`src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/ScriptAnalysis/` (six minimal-API
|
||
endpoints under `/api/script-analysis/*`, gated by the `ConfigEditor` policy
|
||
(Administrator or Designer) via `RequireAuthorization` in
|
||
`ScriptAnalysisEndpoints.MapScriptAnalysisEndpoints`).
|
||
`Core.VirtualTags` is now just `DependencyGraph`, `ITagUpstreamSource` and `IHistoryWriter` — the dormant `VirtualTagEngine`/Source/Definition and `TimerTriggerScheduler` were deleted.
|
||
See `docs/ScriptEditor.md` for the full guide. Scripts may use the `{{equip}}` token for equipment-relative tag paths that resolve per-equipment at deploy time — see the "Equipment-relative tag paths" section in `docs/ScriptEditor.md`.
|
||
|
||
**Out-of-process script runner (lmxopcua#780, opt-in).** `Scripting:Runner` = `InProcess` (default, unchanged) | `OutOfProcess` | `Shadow` (`ShadowScriptRunner`: publishes the in-process result and mirrors each evaluation to the worker pool off the publish path for comparison only — key-busy runs are dropped, never awaited).
|
||
`OutOfProcess` runs the VirtualTag + Calculation seams' compiled scripts (and the scripted-alarm predicates when `Scripting:Runner:Seams`
|
||
lists `ScriptedAlarm` — never by default) in `ZB.MOM.WW.OtOpcUa.ScriptWorker` processes
|
||
(`src/Core/ZB.MOM.WW.OtOpcUa.Core.Scripting.OutOfProcess`, protocol in `Core.Scripting.Protocol`), shipped in `script-worker/`
|
||
beside the host (self-contained + started via its apphost on a `-r win-x64 --self-contained` publish, through the muxer otherwise); a runner-side failure publishes `BadResourceUnavailable` (ScadaBridge-visible, ScadaBridge#112) and the seam re-evaluates
|
||
when the worker is back. `IScriptRunner.RunBatch` sends ready evaluations as one frame per worker (Calc pump, VT coalescer,
|
||
alarm dirty set); a run queued behind busy worker threads has its deadlines restarted when it starts, so it is never blamed for a sibling. The soft deadline is the **worker's** clock from the run's start (#827, negotiated via `Hello`/`Ready.worker_soft_clock`); the host only keeps a T + G transport deadline, and passing it is a runner fault (`script worker unresponsive` → `BadResourceUnavailable`, Shadow `shadow_unavailable`), never a script timeout. Optional `Scripting:Worker:PeCacheDir` = HMAC'd compiled-PE cache (restart without Roslyn); optional (Windows) `Scripting:Worker:Identity` = `NT SERVICE\OtOpcUaScriptWorker` runs workers under a restricted token, set up by `scripts/install/Install-ScriptWorkerIsolation.ps1` (ACLs + outbound firewall block; `docs/ServiceHosting.md`); the `script-runner`
|
||
readiness check is at worst Degraded (never fails ready). The worker tests
|
||
are `tests/Server/ZB.MOM.WW.OtOpcUa.ScriptWorker.Tests` (`Category=ScriptWorker`, real processes; `Category=Perf` harness opt-in via
|
||
`OTOPCUA_SCRIPT_PERF=1`). docker-dev overlay: `docker-dev/docker-compose.script-runner.yml`. See `docs/VirtualTags.md`
|
||
§"Out-of-process runner".
|