Joseph Doherty 1a7a8f6f22 fix(historian): QuestDB modification trail gets an applied marker and a dedup key (lmxopcua#746)
QuestDB has no multi-table transaction, so a HistoryUpdate writes its trail
row before the sample change. A failed or crashed change left an orphan
trail row that modified reads served as a change that happened, and a trail
INSERT re-sent after a client timeout duplicated (the table had no key).

- sample_modified gains `pending` and DEDUP UPSERT KEYS(ts, tag,
  modified_id). The trail row is written pending; after the change lands
  the same row is re-upserted pending = false (the applied marker).
- A transient trail INSERT failure is re-sent once with the same ids; a
  first attempt that did apply converges on the key.
- ReadModifiedAsync serves a pending row only when it is the latest trail
  row at its time and the sample there was written at or after the
  modification (a crash between change and marker). The orphan of a change
  that never landed, typically followed by the client's retry, is dropped.
  ASOF JOIN over the window's samples of one tag; the resume filter sits
  outside the window function so it never cuts a time's partition.
- The bootstrapper migrates an older trail (ADD COLUMN pending, DEDUP
  ENABLE) and waits for it to apply (bounded 10 s); old rows read
  pending = false and stay visible. Until applied, or with
  AutoCreateSchema=false, the old format is used.

Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 15:40:46 -05:00
Pin libplctag ab_server to v2.6.16 — real release tag + SHA256 hashes for all three Windows arches. Closes the "pick a current version + pin" deferral left by the #180 PR docs stub. Verified the release lands ab_server.exe inside libplctag_2.6.16_windows_<arch>_tools.zip alongside plctag.dll + list_tags_* helpers by downloading each tools zip + unzip -l'ing to confirm ab_server.exe is present at 331264 bytes. New ci/ab-server.lock.json is the single source of truth — one file the CI YAML reads via ConvertFrom-Json instead of duplicating the hash across the workflow + the docs. Structure: repo (libplctag/libplctag) + tag (v2.6.16) + published date (2026-03-29) + assets keyed by platform (windows-x64 / windows-x86 / windows-arm64) each carrying filename + sha256. docs/v2/test-data-sources.md §2.CI updated — replaces the prior placeholder (ver = '<pinned libplctag release tag>', expected = '<pinned sha256>') with the real v2.6.16 + 9b78a3de... hashes pinned table, and replaces the hardcoded URL with a lockfile-driven pwsh step that picks windows-x64 by default but swaps to x86/arm64 by changing one line for non-x64 CI runners. Hash-mismatch path throws with both the expected + actual values so on the first drift the CI log tells the maintainer exactly what to update in the lockfile. Two verification notes from the release fetch: (1) libplctag v2.6.16 tools zips ship ab_server.exe + plctag.dll together — tests don't need a separate libplctag NuGet download for the integration path, the extracted tools dir covers both the simulator + the driver's native dependency; (2) the three Windows arches all carry ab_server.exe, so ARM64 Windows GitHub runners (when they arrive) can run the integration suite without changes beyond swapping the asset key. No code changes in this PR — purely docs + the new lockfile. Admin tests + Core tests unchanged + passing per the prior commit.
2026-04-20 00:04:35 -04:00

OtOpcUa

OPC UA server (.NET 10 AnyCPU) that exposes a fleet of industrial drivers as a single OPC UA address space. Drivers ship in-process for AVEVA System Platform Galaxy (via the sibling mxaccessgw repo), Modbus TCP, Siemens S7, Allen-Bradley CIP (ControlLogix / CompactLogix), Allen-Bradley Legacy (SLC 500 / MicroLogix), Beckhoff TwinCAT (ADS), FANUC FOCAS, OPC UA Client (gateway), and MQTT (broker subscribe; Sparkplug B in progress).

A cross-platform client stack (.NET 10) — shared library, CLI, and Avalonia desktop app — connects to any OPC UA server.

Architecture

                      OPC UA Clients (CLI, Desktop UI, 3rd-party)
                                       |
                                       v
                    +-------------------------------------+
                    |  OtOpcUa.Server (.NET 10 AnyCPU)    |
                    |   address space + capability fan-out|
                    +-------------------------------------+
                    |    |    |    |    |    |    |    |
          Galaxy Modbus S7 AbCip AbLeg TwinCAT FOCAS OpcUaClient MQTT
                |
                v
       mxaccessgw (sibling repo, gRPC)
                |
                v
     MXAccess COM (x86 worker, on AVEVA box)

Galaxy is the only driver with an external runtime: it speaks gRPC to a separately installed mxaccessgw server (sibling repo at c:\Users\dohertj2\Desktop\mxaccessgw\) which owns the MXAccess COM apartment and the x86/STA bitness constraint server-side. Everything in this repo is platform-agnostic .NET 10.

Prerequisites

  • .NET 10 SDK (server, drivers, clients all target .NET 10)
  • SQL Server reachable for the central config DB
  • For Galaxy specifically: a running mxaccessgw deployment — see docs/v2/Galaxy.ParityRig.md
  • For historian read-back / alarm history / continuous historization: a running ZB.MOM.WW.HistorianGateway deployment (the sole historian backend; consumed as the ZB.MOM.WW.HistorianGateway.Client gRPC package). It must run RuntimeDb:Enabled=true + RuntimeDb:EventReadsEnabled=true, and the API key must carry historian:read + historian:write + historian:tags:write scopes.

Quick Start

dotnet restore ZB.MOM.WW.OtOpcUa.slnx
dotnet build ZB.MOM.WW.OtOpcUa.slnx
dotnet test ZB.MOM.WW.OtOpcUa.slnx

# Run the server in dev (foreground)
dotnet run --project src/Server/ZB.MOM.WW.OtOpcUa.Host

The server starts on opc.tcp://localhost:4840 with the None security profile. Configure Security.Profiles in src/Server/ZB.MOM.WW.OtOpcUa.Host/appsettings.json to enable Basic256Sha256-Sign or Basic256Sha256-SignAndEncrypt. See docs/security.md.

Install as Windows Services

Production deployment is driven by scripts/install/Install-Services.ps1, which registers the OtOpcUa server service under a chosen service account. Historian support requires a separately deployed ZB.MOM.WW.HistorianGateway and Galaxy support a separately installed mxaccessgw — neither this repo nor the install script provisions them.

.\scripts\install\Install-Services.ps1 `
    -InstallRoot 'C:\Program Files\OtOpcUa' `
    -ServiceAccount 'DOMAIN\svc-otopcua'

The historian backend is the external ZB.MOM.WW.HistorianGateway (not installed by this script). See the script header and docs/ServiceHosting.md for full options.

Client CLI

dotnet run --project src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI -- connect -u opc.tcp://localhost:4840
dotnet run --project src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI -- browse  -u opc.tcp://localhost:4840 -r -d 3
dotnet run --project src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI -- read    -u opc.tcp://localhost:4840 -n "ns=2;s=SomeNode"
dotnet run --project src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI -- write   -u opc.tcp://localhost:4840 -n "ns=2;s=SomeNode" -v 42
dotnet run --project src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI -- subscribe -u opc.tcp://localhost:4840 -n "ns=2;s=SomeNode" -i 500

See docs/Client.CLI.md and docs/Client.UI.md.

Documentation

Architecture deep-dives

Topic Doc
OPC UA server composition, namespace fan-out, Polly invoker docs/OpcUaServer.md
Address space layout docs/AddressSpace.md
Read / Write dispatch (driver vs virtual vs scripted-alarm) docs/ReadWriteOperations.md
Incremental sync (driver-backend rediscovery + config publishes) docs/IncrementalSync.md
Service hosting (Server + Admin; external HistorianGateway backend) docs/ServiceHosting.md
Security (transport, LDAP, certificates) docs/security.md
Redundancy docs/Redundancy.md
Status dashboard docs/StatusDashboard.md

Drivers

Topic Doc
Driver specs (per-driver capability surface, config, addressing) docs/v2/driver-specs.md
Galaxy driver docs/drivers/Galaxy.md
Modbus / S7 / AbCip / AbLegacy / TwinCAT / FOCAS / OpcUaClient / MQTT docs/drivers/
Galaxy parity rig (mxaccessgw setup) docs/v2/Galaxy.ParityRig.md
Galaxy performance + tracing docs/v2/Galaxy.Performance.md

Clients

Topic Doc
Client CLI docs/Client.CLI.md
Client UI (Avalonia desktop) docs/Client.UI.md

v1 archive

The original v1 in-process MXAccess docs (Galaxy.Host topology, Configuration env vars, AlarmTracking, DataTypeMapping, HistoricalDataAccess, Subscriptions, etc.) are preserved under docs/v1/ — historical reference only. PR 7.2 retired the v1 architecture on 2026-04-30; current state is documented in the sections above.

License

Internal use only.

S
Description
No description provided
Readme 84 MiB
Languages
C# 69.8%
JavaScript 25.8%
HTML 2.4%
Python 0.8%
PowerShell 0.5%
Other 0.6%