scadaproj/otopcua-uns-loader

otopcua-uns-loader

A reloadable populate-and-verify tool for the OtOpcUa galaxy Unified Namespace. Recreates a UNS load grounded in the real AVEVA Galaxy DEV hierarchy (the 40 TestMachine instances) and verifies it streams live values on OPC UA — so you can rebuild the OtOpcUa docker-dev instance and get the namespace back with one populate + one deploy click.

What it loads

One SystemPlatform Tag per (machine, signal) bound to the existing GalaxyMxGateway driver (MAIN-galaxy-mxgw). Each tag's FolderPath is the Galaxy object and its Name the attribute, so the materialised OPC UA variable OtOpcUa/<machine>/<signal> has a NodeId equal to the MXAccess reference the driver subscribes to — which is what makes the value go live.

Signals mirrored per machine (only those the instance actually has): the $TestMachine process UDAs — TestChangingInt, TestHistoryValue, TestDouble/Float/Duration/DateTime, ProtectedValue(1), TestAlarm001..003, InAlarm → 396 tags across 40 machines.

Every row carries the nw-mirror- TagId prefix, so clean removes exactly what the tool created (adopting any pre-existing seed row for the same ref).

Why a deploy click is in the middle

OtOpcUa applies config only from sealed Deployment snapshots, and the only way to seal one is the AdminUI "Deploy current configuration" button (http://localhost:9200/deployments) — there is no SQL/REST/CLI trigger (it's an in-cluster Akka operation). So the flow is:

populate ──SQL──▶ live config tables
                    │  (you click Deploy at :9200, sign in multi-role/password)
                    ▼
   driver applies ▶ materialises variables ▶ SubscribeBulk ▶ live values
                    │
verify ──OPC UA──▶ browse + read Good values on :4840

populate and clean print the reminder; verify --wait polls until the deploy lands.

Live values depend on the driver SubscribeBulk pass (OtOpcUa master ≥ commit c1ce583). On older builds variables materialise but stay BadWaitingForInitialData.

Setup

cd otopcua-uns-loader
python3 -m venv .venv
./.venv/bin/pip install -r requirements.txt

Use

./.venv/bin/python otopcua_uns.py generate   # build load-plan.json from galaxy-hierarchy.json
./.venv/bin/python otopcua_uns.py populate    # upsert the 396 mirror Tag rows (idempotent)
#   → open http://localhost:9200/deployments, sign in, click "Deploy current configuration"
./.venv/bin/python otopcua_uns.py verify --wait   # poll until live values are Good on :4840
./.venv/bin/python otopcua_uns.py status      # config-DB + address-space snapshot
./.venv/bin/python otopcua_uns.py clean       # remove all nw-mirror-* tags (then Deploy again)

Rebuild recovery (the point of the tool)

After the docker-dev instance is rebuilt (DB wiped):

1. Ensure the schema + clusters + the MAIN-galaxy-mxgw driver exist (dotnet ef database update + the docker-dev cluster-seed; see the OtOpcUa docker-dev/README.md).
2. populate → Deploy at the AdminUI → verify --wait.

Troubleshooting

verify stays INCOMPLETE / deployment "Sealed" but drivers never applied it. If you recreate the admin/coordinator node (admin-a) around the same time you click Deploy, the dispatch broadcast can be lost — the deployment seals but NodeDeploymentState shows no row for the driver nodes, and the address space keeps the old content. Recover by: restart the driver nodes (docker restart otopcua-dev-driver-a-1 otopcua-dev-driver-b-1) so they cleanly re-subscribe, then Deploy again. If a no-op "NoChanges" blocks the re-deploy, delete the orphan sealed Deployment row that no node applied (it has no NodeDeploymentState children) so Deploy sees drift again.

A rebuilt/restarted node serves an empty address space until the next Deploy. On bootstrap a node recovers to its last-applied revision and does not re-materialise until a new deployment is dispatched — so after any node restart, click Deploy once (a config change bumps the revision) to repopulate + re-subscribe.

Configuration

Defaults target docker-dev; override via flags or env:

Flag	Env	Default
--sql-host/-port/-user/-password/-db	OTOPCUA_SQL_*	localhost:14330 sa / OtOpcUa!Dev123 / OtOpcUa
--opcua-endpoint	OTOPCUA_OPCUA_ENDPOINT	opc.tcp://localhost:4840
--driver	OTOPCUA_GALAXY_DRIVER	MAIN-galaxy-mxgw
--galaxy-json	OTOPCUA_GALAXY_JSON	../galaxy-hierarchy.json
--deploy-url	—	http://localhost:9200/deployments

Files

• otopcua_uns.py — the CLI (generate / populate / verify / status / clean)
• load-plan.json — generated load plan (machine → signal → MXAccess ref)
• ../galaxy-hierarchy.json — the source of truth, pulled live from the gateway
• requirements.txt, .venv/

Company-shape overlay (populate-equipment)

Besides the galaxy-native mirror, the tool can load the Northwind company shape (filling / line-1 / rinser-01 / speed-rpm) as a second, Equipment-kind namespace (nw-uns, in cluster MAIN) from ../company-uns.json. This needs OtOpcUa master ≥ the Equipment-namespace structure milestone (febe462…9a67ebc), which materialises Equipment Tag/VirtualTag rows on deploy and added a headless deploy endpoint.

./.venv/bin/python otopcua_uns.py populate-equipment        # 3 areas / 8 lines / 40 equipment / 1036 signals
curl -s -X POST http://localhost:9200/api/deployments -H 'X-Api-Key: docker-dev-deploy-key'   # headless deploy
./.venv/bin/python otopcua_uns.py verify-equipment --expect 1036   # browse the company tree (nw-area-* scope)

UNS folders carry the friendly DisplayName (filling); the BrowseName/NodeId stay the stable logical Id (nw-area-filling) — standard OPC UA. Structure-only: the company leaves materialise as BadWaitingForInitialData — live values in the company shape are the next OtOpcUa milestone (driver/VirtualTag source), tracked in OtOpcUa/docs/plans/2026-06-06-equipment-namespace-materialization-scope.md (WS-3). The galaxy-native mirror (populate) still carries live values.

clean removes both the mirror tags and the company overlay.