scadaproj/otopcua-uns-loader/README.md

# otopcua-uns-loader

A **reloadable** populate-and-verify tool for the OtOpcUa galaxy Unified Namespace.
Recreates a UNS load grounded in the real AVEVA Galaxy **DEV** hierarchy (the 40
`TestMachine` instances) and verifies it streams **live values** on OPC UA — so
you can rebuild the OtOpcUa docker-dev instance and get the namespace back with
one populate + one deploy click.

## What it loads

One **SystemPlatform** `Tag` per `(machine, signal)` bound to the existing
`GalaxyMxGateway` driver (`MAIN-galaxy-mxgw`). Each tag's `FolderPath` is the
Galaxy object and its `Name` the attribute, so the materialised OPC UA variable
`OtOpcUa/<machine>/<signal>` has a NodeId equal to the MXAccess reference the
driver subscribes to — which is what makes the value go live.

Signals mirrored per machine (only those the instance actually has): the
`$TestMachine` process UDAs — `TestChangingInt`, `TestHistoryValue`,
`TestDouble/Float/Duration/DateTime`, `ProtectedValue(1)`, `TestAlarm001..003`,
`InAlarm` → **396 tags across 40 machines**.

Every row carries the `nw-mirror-` `TagId` prefix, so `clean` removes exactly
what the tool created (adopting any pre-existing seed row for the same ref).

## Why a deploy click is in the middle

OtOpcUa applies config only from **sealed Deployment snapshots**, and the only
way to seal one is the AdminUI **"Deploy current configuration"** button
(`http://localhost:9200/deployments`) — there is no SQL/REST/CLI trigger (it's
an in-cluster Akka operation). So the flow is:

```
populate ──SQL──▶ live config tables
                    │  (you click Deploy at :9200, sign in multi-role/password)
                    ▼
   driver applies ▶ materialises variables ▶ SubscribeBulk ▶ live values
                    │
verify ──OPC UA──▶ browse + read Good values on :4840
```

`populate` and `clean` print the reminder; `verify --wait` polls until the
deploy lands.

> Live values depend on the driver **SubscribeBulk** pass
> (OtOpcUa `master` ≥ commit `c1ce583`). On older builds variables materialise
> but stay `BadWaitingForInitialData`.

## Setup

```bash
cd otopcua-uns-loader
python3 -m venv .venv
./.venv/bin/pip install -r requirements.txt
```

## Use

```bash
./.venv/bin/python otopcua_uns.py generate   # build load-plan.json from galaxy-hierarchy.json
./.venv/bin/python otopcua_uns.py populate    # upsert the 396 mirror Tag rows (idempotent)
#   → open http://localhost:9200/deployments, sign in, click "Deploy current configuration"
./.venv/bin/python otopcua_uns.py verify --wait   # poll until live values are Good on :4840
./.venv/bin/python otopcua_uns.py status      # config-DB + address-space snapshot
./.venv/bin/python otopcua_uns.py clean       # remove all nw-mirror-* tags (then Deploy again)
```

### Rebuild recovery (the point of the tool)

After the docker-dev instance is rebuilt (DB wiped):

1. Ensure the schema + clusters + the `MAIN-galaxy-mxgw` driver exist
   (`dotnet ef database update` + the docker-dev `cluster-seed`; see the OtOpcUa
   `docker-dev/README.md`).
2. `populate` → **Deploy** at the AdminUI → `verify --wait`.

## Troubleshooting

**`verify` stays INCOMPLETE / deployment "Sealed" but drivers never applied it.**
If you recreate the **admin/coordinator** node (`admin-a`) around the same time
you click Deploy, the dispatch broadcast can be lost — the deployment seals but
`NodeDeploymentState` shows no row for the driver nodes, and the address space
keeps the old content. Recover by: restart the driver nodes
(`docker restart otopcua-dev-driver-a-1 otopcua-dev-driver-b-1`) so they cleanly
re-subscribe, then Deploy again. If a no-op "NoChanges" blocks the re-deploy,
delete the orphan sealed `Deployment` row that no node applied (it has no
`NodeDeploymentState` children) so Deploy sees drift again.

**A rebuilt/restarted node serves an empty address space until the next Deploy.**
On bootstrap a node recovers to its last-*applied* revision and does **not**
re-materialise until a new deployment is dispatched — so after any node restart,
click Deploy once (a config change bumps the revision) to repopulate + re-subscribe.

## Configuration

Defaults target docker-dev; override via flags or env:

| Flag | Env | Default |
|---|---|---|
| `--sql-host/-port/-user/-password/-db` | `OTOPCUA_SQL_*` | `localhost:14330` sa / `OtOpcUa!Dev123` / `OtOpcUa` |
| `--opcua-endpoint` | `OTOPCUA_OPCUA_ENDPOINT` | `opc.tcp://localhost:4840` |
| `--driver` | `OTOPCUA_GALAXY_DRIVER` | `MAIN-galaxy-mxgw` |
| `--galaxy-json` | `OTOPCUA_GALAXY_JSON` | `../galaxy-hierarchy.json` |
| `--deploy-url` | — | `http://localhost:9200/deployments` |

## Files

- `otopcua_uns.py` — the CLI (generate / populate / verify / status / clean)
- `load-plan.json` — generated load plan (machine → signal → MXAccess ref)
- `../galaxy-hierarchy.json` — the source of truth, pulled live from the gateway
- `requirements.txt`, `.venv/`

## Scope note — company-UNS shape

This tool loads the galaxy in its **native hierarchy**
(`OtOpcUa/TestMachine_NNN/<signal>`), which is the only shape that can carry live
Galaxy values: OtOpcUa forbids the `GalaxyMxGateway` driver in an `Equipment`
namespace, so a custom `Enterprise/Site/Area/Line/Equipment` UNS (e.g. the
Northwind model in `../company-uns.json`) must be a separate **Equipment**
namespace fed by an `OpcUaClient` driver + `UnsMappingTable` that remaps this
mirror. That overlay is the designed next layer; `../company-uns.json` already
carries the area/line/equipment → galaxy-ref mapping it needs.