lmxopcua/docs/plans/2026-07-15-raw-uns-two-subtree-v3-design.md

# v3 — Kepware-style Raw tree + UNS projection (two-subtree address space)

**Date:** 2026-07-15
**Status:** Design approved (brainstorming complete) — ready for implementation planning
**Version:** Next major version (v3). The project has no git tags; "v2" is the conceptual
current major (the Akka-fused rebuild, `docs/v2/`), so this is v3.

## Summary

Restructure how drivers and tags are managed so the OPC UA server exposes **two subtrees**:

1. **Raw** — a Kepware-style, driver-centric tree that is the literal I/O map and the
   **single source of truth** for every value node.
2. **UNS** — a semantic re-organization (Area → Line → Equipment) that **references** raw
   tags rather than authoring them.

This is a compatibility-breaking architectural change. It is **greenfield** — no production
config data is preserved; dev rigs are re-seeded against a fresh schema.

## Decisions (locked)

- **Raw is the source of truth; UNS references it.** A tag is authored once, under a driver
  in the Raw tree. UNS holds references that re-point existing raw tags into equipment. One
  tag, two views — rename/retype in Raw propagates automatically.
- **Raw hierarchy:** `Cluster → Folder(s, nestable) → Driver → Device → TagGroup(s, nestable) → Tag`.
  Full Kepware parity.
- **Raw is global, cluster-rooted**, mirroring the existing global `/uns` page (which is
  Enterprise → Cluster → Area → …). Each *running* server still exposes only **its own**
  cluster's two subtrees in its OPC UA address space.
- **UNS authoring is reference-only (clean break).** The Equipment editor no longer binds a
  driver or defines `TagConfig`; it references existing raw tags (with an optional UNS
  display-name override) and continues to host VirtualTags + scripted alarms.
- **OPC UA exposure:** two namespaces — `ns=Raw` (stable device-path NodeIds, **always on**)
  and `ns=UNS` (equipment-path NodeIds). UNS nodes `Organizes →` their backing raw nodes.
- **UNS ↔ Raw linkage = delegate/redirect (not value-copy).** A UNS node is a thin variable
  whose reads/writes/subscriptions are delegated to the backing raw node's single value
  source. No double-buffering, no drift.
- **Tag creation is capability-based:** manual entry + CSV import/export for **all** drivers;
  live **Browse** only for drivers that advertise a browser (OpcUaClient, Galaxy today).
- **Calculated tags — two-tier:**
  - *Signal-level* calcs = raw tags bound to a **`Calculation` pseudo-driver** whose value
    source is the existing Roslyn scripting engine. They reuse the entire raw pipeline
    (folders, CSV, historization, native alarms, UNS references) with no special-casing.
  - *Equipment-context* calcs stay as **UNS VirtualTags** (unchanged) — `{{equip}}`
    templating and per-equipment instantiation are inherently UNS concepts.
  - Author rule: *inputs/meaning all device/signal-level → Raw; needs equipment context or
    spans equipment → UNS VirtualTag.*
- **Migration:** greenfield / reset. Fresh schema; re-seed dev rigs. No data-migration code.
- **Build approach:** phased (Approach B) — clean schema, but **reuse** the existing typed
  driver forms, `TagConfig` editors, and `IDriverProbe`s inside the new tree/popup shell.

## Current state (baseline being replaced)

- A **`ServerCluster`** (PK `ClusterId`) owns a **flat list** of `DriverInstance` rows — there
  is **no folder/grouping** construct for drivers today.
- Drivers have `Device` rows (multi-device drivers only) and `PollGroup` rows.
- **`Tag`** binds to a driver via `DriverInstanceId` + schemaless `TagConfig` JSON and lives
  either under an `Equipment` (`EquipmentId`) or at a namespace-root `FolderPath`.
- The **UNS tree** (Enterprise → Cluster → Area → Line → Equipment → Tag/VirtualTag) is the
  **only** authored tree and is what the address space exposes.
- Driver authoring is a page-per-type flow: `/clusters/{id}/drivers` → `DriverTypePicker` →
  8 typed Razor pages (`ModbusDriverPage`, …) dispatched by `DriverEditRouter._componentMap`.
- `DriverType` dispatch is duplicated in ~3 hard-coded places with pre-existing string
  mismatches: `TwinCAT`/`TwinCat`, `FOCAS`/`Focas`, `GalaxyMxGateway`/`galaxy`.
- Runtime: Akka actors — `DriverHostActor` (per node/role) → `DriverInstanceActor` (per
  driver) — bind driver values through `EquipmentTagRefResolver`/`TagConfig.FullName`.
- Address-space pipeline: `AddressSpace*` (Composer → Composition → Planner → Plan → Applier).

## Data model (greenfield schema)

### Raw side
- **`RawFolder`** *(new)* — nestable driver-organizing folder. `RawFolderId` PK, `ClusterId`
  FK, nullable `ParentRawFolderId` (null = root under cluster), `Name`, `SortOrder`.
- **`DriverInstance`** *(reshaped)* — gains nullable `RawFolderId` (null = cluster root).
  Keeps `ClusterId`, `DriverType`, `DriverConfig`, `ResilienceConfig`. **Drops** per-driver
  `NamespaceId` (namespaces become implicit — see Address space).
- **`Device`** *(kept, now universal)* — `DeviceId` PK, `DriverInstanceId` FK, `Name`,
  `DeviceConfig` JSON. **Every driver has ≥1 device**; single-endpoint drivers (S7, TwinCAT,
  FOCAS, OpcUaClient, Galaxy, Calculation) get an auto-created *default* device.
- **`TagGroup`** *(new)* — nestable tag folder under a device. `TagGroupId` PK, `DeviceId` FK,
  nullable `ParentTagGroupId`, `Name`, `SortOrder`.
- **`Tag`** *(reshaped → raw-only)* — `TagId` PK, `DeviceId` FK (required), nullable
  `TagGroupId`, `Name`, `DataType`, `AccessLevel`, nullable `PollGroupId`, `TagConfig` JSON.
  **Drops `EquipmentId` and `FolderPath`.**
- **`PollGroup`** *(unchanged)* — driver-scoped polling groups.

### UNS side
- **`UnsArea → UnsLine → Equipment`** *(kept)* — but `Equipment` **loses** its
  `DriverInstanceId`/`DeviceId` binding.
- **`UnsTagReference`** *(new)* — the projection. `EquipmentId` FK, `TagId` FK (→ raw tag),
  nullable `DisplayNameOverride`, `SortOrder`.
- **`VirtualTag`, `ScriptedAlarm`** *(unchanged)* — per-equipment, UNS-native.

### Integrity rules
- Deleting a raw `Tag` is **blocked while any `UnsTagReference` points at it**; the UI names
  the referencing equipment.
- `RawFolder`/`TagGroup` deletes are **blocked when non-empty** (matches today's
  `OnDelete(Restrict)` convention).
- The default `Device` of a single-endpoint driver cannot be deleted while it holds tags.

## AdminUI — the Raw project tree (`/raw`)

New global page **`/raw`** (peer to `/uns`), a lazy, cluster-rooted project tree mirroring
`GlobalUns` + `UnsTree`. Every node type has a **right-click context menu**; actions open
**modals** instead of navigating to a separate page:

- **Folder** — New folder / New driver / Rename / Delete.
- **Driver** — Configure / Test connect / New device / Enable-Disable / Rename / Delete.
- **Device** — Configure / New tag-group / Add tags ▸ / Delete.
- **TagGroup** — New group / Add tags ▸ / Rename / Delete.
- **Tag** — Edit / Delete.

**Reuse (not rebuild):**
- *Configure driver* hosts the existing typed driver forms (`ModbusDriverPage` et al.)
  refactored into embeddable form bodies inside a `DriverConfigModal`. Only the page
  shell/routing changes.
- *Test connect* uses the existing `DriverTestConnectButton` + `IDriverProbe` path (transient
  config, no persistence).
- *Edit tag* uses the existing per-driver `TagConfig` editors via `TagConfigEditorMap`.

**Add tags ▸ (capability-based)** on a Device or TagGroup:
- **Manual entry** — grid: name, datatype, access + the driver-typed `TagConfig` editor.
- **Import/Export CSV** — columns `Name, TagGroupPath, DataType, AccessLevel,
  <driver-specific TagConfig columns>`; staged-then-commit like today's
  `EquipmentImportBatch`. Export round-trips the same shape.
- **Browse device…** — enabled only when the driver's factory advertises a browser
  (`DriverBrowseTree` + `IBrowserSessionService`); grayed out with a tooltip otherwise.

**Cleanup folded in:** canonicalize the `DriverType` strings to a single source of truth
(the tree dispatches on driver type in one place instead of three).

## UNS Equipment — reference-only

The `/uns` tree is unchanged; the **Equipment Tags tab** becomes a reference list:
- Remove the driver dropdown and inline `TagConfig` editor from the equipment Tag flow.
- **"+ Add reference"** opens a picker into the Raw tree, **scoped to the same cluster** as
  the equipment (cross-cluster references are structurally impossible), with **multi-select**
  to pull many raw tags at once.
- Each row is a `UnsTagReference`: shows the raw path, inherited datatype/access (read-only),
  and an optional display-name override.
- **VirtualTags and scripted alarms** stay exactly as they are.
- The old `ImportEquipmentModal` drops its `DriverInstanceId` column (equipment no longer
  carries a driver).
- *Deferred to a follow-up:* pattern/CSV-based auto-linking of raw → UNS.

## OPC UA address space + runtime binding

Runs through the existing `AddressSpace*` pipeline. **Two namespaces per server** (each
running cluster exposes only its own):

- **`ns=Raw`** — NodeIds are stable device paths:
  `s=<Folder/…>/<Driver>/<Device>/<Group/…>/<Tag>`. Folders/drivers/devices/tag-groups are
  `Object`/`Folder` nodes; tags are `Variable` nodes.
- **`ns=UNS`** — NodeIds are equipment paths: `s=<Area>/<Line>/<Equipment>/<TagName>`.
  Area/Line/Equipment are folders; each `UnsTagReference` is a UNS variable node.

**UNS ↔ Raw = delegate/redirect:** the UNS node is a thin variable whose reads/writes/
subscriptions delegate to the backing raw node's single value source; an `Organizes`
reference UNS → Raw makes the linkage browsable.

**Runtime binding flows through Raw only.** Driver-host actors bind driver values to **raw**
tag nodes (via `EquipmentTagRefResolver`/`FullName`, now resolving a raw device path). UNS
nodes never bind a driver directly. Writes to a UNS node route to the raw node's driver (same
`WriteOperate` gating). HistoryRead, native alarms, and continuous historization all attach
at the **raw** node and are visible through the UNS reference.

**Cross-repo impact:** ScadaBridge's Data Connection Layer keeps its UNS/equipment bindings
(same equipment paths) and *gains* the option to bind raw device paths directly. Record this
v3 contract change in the scadaproj umbrella index (`../scadaproj/CLAUDE.md`).

## Calculated tags (two-tier)

- **`Calculation` pseudo-driver** *(new DriverType)* — signal-level calc tags are raw tags
  bound to this driver. Registered in `DriverTypeRegistry` like any driver; a single default
  `Engine` device; the tag's `TagConfig` holds a **script reference** instead of a register
  address, and its value source is the existing Roslyn scripting engine (`ctx.GetTag`,
  publish gate, Monaco editor). Wire it into the driver-host value fan-out. It inherits
  folders, CSV, historization, native alarms, and UNS references for free.
- **UNS VirtualTags** *(unchanged)* — retained for equipment-context computations that need
  `{{equip}}` templating / per-equipment instantiation.
- **Author rule:** signal-level → Raw (`Calculation` driver); equipment-context or
  cross-equipment → UNS VirtualTag.

## Build plan (Approach B — phased)

| Batch | Content | Verification |
|---|---|---|
| **1 — Schema** | Greenfield EF model + migration + dev-rig re-seed | build/test; migration applies; seed loads |
| **2 — Raw UI** | `/raw` project tree, right-click popups, driver config/test, tag manual/CSV/browse, `Calculation` driver | Live `/run` on docker-dev `:9200`: author a driver + tags in-tree, Test-connect green, author a calc tag |
| **3 — UNS rework** | Equipment reference-only, raw-tag multi-select picker | Live `/run`: reference raw tags into an equipment; display-name override shows |
| **4 — Address space** | Dual namespace, UNS→Raw delegation, raw-only binding | Live `/run` + Client.CLI: browse both namespaces, read/write through both, HistoryRead/alarm at raw visible via UNS |

Batches are sequential; batch 4 landing = **v3.0**.

## Testing strategy

- **Unit** (xUnit + Shouldly): schema integrity rules (block delete-while-referenced,
  non-empty folder delete), CSV round-trip, `DriverType` canonicalization, UNS-reference
  resolution, `Calculation` driver value production.
- **Integration** (`*.IntegrationTests`): address-space builder emits both namespaces with
  correct NodeIds; UNS delegation resolves to the raw value source.
- **Live verification per batch — non-negotiable.** This project has a repeated history of
  prod-inertness bugs that unit tests + review missed (e.g. the `DeferredAddressSpaceSink`
  forwarding trap). Every batch gets a docker-dev `/run` gate before it is "done."

## Error handling / edge cases

- **Dangling reference guard** — raw tag delete blocked while referenced; UI names the
  equipment.
- **Datatype change on a referenced raw tag** — propagates to UNS (raw is truth); if it
  breaks a UNS consumer, that surfaces at browse/subscribe time (documented, not silently
  patched).
- **Single-endpoint driver default device** — auto-created, uniform in the tree, undeletable
  while holding tags.
- **Capability-gated browse** — non-browsable drivers show a disabled action + tooltip, never
  a broken picker.
- **Cross-cluster reference** — structurally impossible (the UNS picker is cluster-scoped).

## Out of scope for v3.0 (follow-ups)

- Pattern/CSV-based auto-linking of raw → UNS references.
- Live browse for the address-entry protocols (Modbus/S7/AB-CIP/AB-Legacy/TwinCAT/FOCAS) via
  per-protocol template catalogs.
- Optional Raw-namespace hiding toggle (security-sensitive deployments exposing only UNS).

## Implementation tasks

- #7 — Raw/UNS v3 schema
- #8 — Global Raw project-tree AdminUI (`/raw`)
- #9 — UNS Equipment reference-only rework
- #10 — Dual-namespace address space + raw-only runtime binding
- #11 — `Calculation` pseudo-driver for signal-level calc tags