Joseph Doherty c74ccc55b5 feat(mqtt): flatten Sparkplug Template/DataSet metrics to sibling tags (#515)
`SparkplugCodec` handled scalars; `DataSet`, `Template` and `PropertySet` were skipped
with a warning. They now flatten to addressable tags (owner decision, recorded on the
issue):

  Template  → sibling metrics, `/`-joined, recursively: Motor → Motor/Speed, Motor/Stator/Temp
  DataSet   → one metric per cell, {column}/{rowIndex}: Recipe → Recipe/Step/0, Recipe/Target/1
  PropertySet → node metadata, never tags — it describes a metric rather than being one

Flattening was chosen over OPC UA Structures because flattened scalars are consumable by
any client with no type dictionary, and the result mirrors how the `/raw` tree works.

**The row index is always present, including on row 0.** Emitting a bare column name for
a single-row table and an indexed one otherwise would make a tag's name depend on how many
rows the plant published — a table growing from one row to two would rename every tag
already authored against it.

**A DATA metric carries the parent's alias and no name**, so member names cannot be
composed until the alias table has resolved the parent. Expansion therefore happens in
`SparkplugIngestor.PublishStructuredMembers` — after resolution, at the single funnel both
the DATA and birth-value paths already pass through — not at decode time. The alias table
installs a binding per member so browse enumerates them and a DDATA member that omits its
datatype falls back to the birth's (guessing from the CLR type would read a signed Int32 of
-42 out of its unsigned field as 4294967254).

**Skip-and-warn survives, and refusal is whole** — the explicit requirement on the issue.
A template definition, an unnamed member, an extension value, a column/type mismatch, a
short row, or a payload past MaxDepth(5)/MaxMembers(512) refuses the ENTIRE metric. A
silently half-flattened UDT looks exactly like a complete one in the picker.

The browse picker offers the members and deliberately NOT the structured parent, which has
no value and no servable datatype. The rediscovery signature likewise tracks members rather
than the parent, or a UDT gaining a member would leave the signature unmoved.

Member paths contain `/`, which #509 already solved: the slashed name travels intact as
`metricName` while the derived tag Name is coerced to a legal RawPath segment.

Live-gated 3/3 against the Mosquitto fixture with the project-owned simulator extended to
publish a nested Template and a 2x2 DataSet. Falsified at three points: dropping the member
bindings, not expanding the parent, and refusing to flatten templates each turn exactly the
expected tests red.

Also fixes a regression from the #511 commit in the same branch: `AddOtOpcUaDriverFactories`
resolved `IDriverPrimaryRoleSource` with `GetRequiredService` but only `AddOtOpcUaRuntime`
registered it, so a graph wiring driver factories without the full runtime failed at
resolution. Two Host integration suites do exactly that and caught it; both extensions now
TryAdd the same concrete `RedundancyRoleView`, so the registration is order-independent and
the fail-loud resolve is kept.

Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
2026-07-31 08:14:04 -04:00
Pin libplctag ab_server to v2.6.16 — real release tag + SHA256 hashes for all three Windows arches. Closes the "pick a current version + pin" deferral left by the #180 PR docs stub. Verified the release lands ab_server.exe inside libplctag_2.6.16_windows_<arch>_tools.zip alongside plctag.dll + list_tags_* helpers by downloading each tools zip + unzip -l'ing to confirm ab_server.exe is present at 331264 bytes. New ci/ab-server.lock.json is the single source of truth — one file the CI YAML reads via ConvertFrom-Json instead of duplicating the hash across the workflow + the docs. Structure: repo (libplctag/libplctag) + tag (v2.6.16) + published date (2026-03-29) + assets keyed by platform (windows-x64 / windows-x86 / windows-arm64) each carrying filename + sha256. docs/v2/test-data-sources.md §2.CI updated — replaces the prior placeholder (ver = '<pinned libplctag release tag>', expected = '<pinned sha256>') with the real v2.6.16 + 9b78a3de... hashes pinned table, and replaces the hardcoded URL with a lockfile-driven pwsh step that picks windows-x64 by default but swaps to x86/arm64 by changing one line for non-x64 CI runners. Hash-mismatch path throws with both the expected + actual values so on the first drift the CI log tells the maintainer exactly what to update in the lockfile. Two verification notes from the release fetch: (1) libplctag v2.6.16 tools zips ship ab_server.exe + plctag.dll together — tests don't need a separate libplctag NuGet download for the integration path, the extracted tools dir covers both the simulator + the driver's native dependency; (2) the three Windows arches all carry ab_server.exe, so ARM64 Windows GitHub runners (when they arrive) can run the integration suite without changes beyond swapping the asset key. No code changes in this PR — purely docs + the new lockfile. Admin tests + Core tests unchanged + passing per the prior commit.
2026-04-20 00:04:35 -04:00

OtOpcUa

OPC UA server (.NET 10 AnyCPU) that exposes a fleet of industrial drivers as a single OPC UA address space. Drivers ship in-process for AVEVA System Platform Galaxy (via the sibling mxaccessgw repo), Modbus TCP, Siemens S7, Allen-Bradley CIP (ControlLogix / CompactLogix), Allen-Bradley Legacy (SLC 500 / MicroLogix), Beckhoff TwinCAT (ADS), FANUC FOCAS, OPC UA Client (gateway), and MQTT (broker subscribe; Sparkplug B in progress).

A cross-platform client stack (.NET 10) — shared library, CLI, and Avalonia desktop app — connects to any OPC UA server.

Architecture

                      OPC UA Clients (CLI, Desktop UI, 3rd-party)
                                       |
                                       v
                    +-------------------------------------+
                    |  OtOpcUa.Server (.NET 10 AnyCPU)    |
                    |   address space + capability fan-out|
                    +-------------------------------------+
                    |    |    |    |    |    |    |    |
          Galaxy Modbus S7 AbCip AbLeg TwinCAT FOCAS OpcUaClient MQTT
                |
                v
       mxaccessgw (sibling repo, gRPC)
                |
                v
     MXAccess COM (x86 worker, on AVEVA box)

Galaxy is the only driver with an external runtime: it speaks gRPC to a separately installed mxaccessgw server (sibling repo at c:\Users\dohertj2\Desktop\mxaccessgw\) which owns the MXAccess COM apartment and the x86/STA bitness constraint server-side. Everything in this repo is platform-agnostic .NET 10.

Prerequisites

  • .NET 10 SDK (server, drivers, clients all target .NET 10)
  • SQL Server reachable for the central config DB
  • For Galaxy specifically: a running mxaccessgw deployment — see docs/v2/Galaxy.ParityRig.md
  • For historian read-back / alarm history / continuous historization: a running ZB.MOM.WW.HistorianGateway deployment (the sole historian backend; consumed as the ZB.MOM.WW.HistorianGateway.Client gRPC package). It must run RuntimeDb:Enabled=true + RuntimeDb:EventReadsEnabled=true, and the API key must carry historian:read + historian:write + historian:tags:write scopes.

Quick Start

dotnet restore ZB.MOM.WW.OtOpcUa.slnx
dotnet build ZB.MOM.WW.OtOpcUa.slnx
dotnet test ZB.MOM.WW.OtOpcUa.slnx

# Run the server in dev (foreground)
dotnet run --project src/Server/ZB.MOM.WW.OtOpcUa.Host

The server starts on opc.tcp://localhost:4840 with the None security profile. Configure Security.Profiles in src/Server/ZB.MOM.WW.OtOpcUa.Host/appsettings.json to enable Basic256Sha256-Sign or Basic256Sha256-SignAndEncrypt. See docs/security.md.

Install as Windows Services

Production deployment is driven by scripts/install/Install-Services.ps1, which registers the OtOpcUa server service under a chosen service account. Historian support requires a separately deployed ZB.MOM.WW.HistorianGateway and Galaxy support a separately installed mxaccessgw — neither this repo nor the install script provisions them.

.\scripts\install\Install-Services.ps1 `
    -InstallRoot 'C:\Program Files\OtOpcUa' `
    -ServiceAccount 'DOMAIN\svc-otopcua'

The historian backend is the external ZB.MOM.WW.HistorianGateway (not installed by this script). See the script header and docs/ServiceHosting.md for full options.

Client CLI

dotnet run --project src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI -- connect -u opc.tcp://localhost:4840
dotnet run --project src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI -- browse  -u opc.tcp://localhost:4840 -r -d 3
dotnet run --project src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI -- read    -u opc.tcp://localhost:4840 -n "ns=2;s=SomeNode"
dotnet run --project src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI -- write   -u opc.tcp://localhost:4840 -n "ns=2;s=SomeNode" -v 42
dotnet run --project src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI -- subscribe -u opc.tcp://localhost:4840 -n "ns=2;s=SomeNode" -i 500

See docs/Client.CLI.md and docs/Client.UI.md.

Documentation

Architecture deep-dives

Topic Doc
OPC UA server composition, namespace fan-out, Polly invoker docs/OpcUaServer.md
Address space layout docs/AddressSpace.md
Read / Write dispatch (driver vs virtual vs scripted-alarm) docs/ReadWriteOperations.md
Incremental sync (driver-backend rediscovery + config publishes) docs/IncrementalSync.md
Service hosting (Server + Admin; external HistorianGateway backend) docs/ServiceHosting.md
Security (transport, LDAP, certificates) docs/security.md
Redundancy docs/Redundancy.md
Status dashboard docs/StatusDashboard.md

Drivers

Topic Doc
Driver specs (per-driver capability surface, config, addressing) docs/v2/driver-specs.md
Galaxy driver docs/drivers/Galaxy.md
Modbus / S7 / AbCip / AbLegacy / TwinCAT / FOCAS / OpcUaClient / MQTT docs/drivers/
Galaxy parity rig (mxaccessgw setup) docs/v2/Galaxy.ParityRig.md
Galaxy performance + tracing docs/v2/Galaxy.Performance.md

Clients

Topic Doc
Client CLI docs/Client.CLI.md
Client UI (Avalonia desktop) docs/Client.UI.md

v1 archive

The original v1 in-process MXAccess docs (Galaxy.Host topology, Configuration env vars, AlarmTracking, DataTypeMapping, HistoricalDataAccess, Subscriptions, etc.) are preserved under docs/v1/ — historical reference only. PR 7.2 retired the v1 architecture on 2026-04-30; current state is documented in the sections above.

License

Internal use only.

S
Description
No description provided
Readme 65 MiB
Languages
C# 65.6%
JavaScript 30.4%
HTML 2.6%
TSQL 0.5%
PowerShell 0.4%
Other 0.4%