lmxopcua/docs/plans/2026-07-15-universal-discovery-browser-design.md

# Universal Discover-backed browser (`DiscoveryDriverBrowser`) — design

**Status:** draft, 2026-07-15. Foundational Wave-0 item of the next-driver roadmap
(`docs/research/drivers/README.md`).

## 1. Motivation

Today the AdminUI address picker is backed by one **bespoke** `IDriverBrowser` per driver
type — only **OpcUaClient** and **Galaxy** exist, each a full hand-written
`IBrowseSession` (connect + lazy level-at-a-time browse). Every other driver's picker is a
static manual-entry stub.

But the existing-driver browse audit
(`docs/research/drivers/00-existing-driver-browse-audit.md`) found that AbCip, TwinCAT, and
FOCAS **already contain complete, unit-tested symbol enumeration** — it lives in their
runtime `ITagDiscovery.DiscoverAsync`, streamed into an `IAddressSpaceBuilder`, and is
simply never surfaced to the picker. The same is true of every future discovery-capable
driver (MTConnect `/probe`, BACnet Who-Is/object-list).

The insight: **`DiscoverAsync` already produces exactly the tree the picker wants.** Instead
of writing N bespoke browsers, write **one** generic browser that captures whatever a driver
streams into the builder and re-presents it. This is the highest-leverage single piece of
browse work on the roadmap.

## 2. The seam that makes it work

`ITagDiscovery.DiscoverAsync(IAddressSpaceBuilder builder, ct)` does not return a tree — it
**streams structured nodes into a builder** (`Core.Abstractions/IAddressSpaceBuilder.cs`):

```csharp
IAddressSpaceBuilder Folder(string browseName, string displayName);       // → child builder
IVariableHandle      Variable(string browseName, string displayName, DriverAttributeInfo a);
void                 AddProperty(string browseName, DriverDataType t, object? v);
```

Two facts close the loop:
- `Folder(...)` returns a **child builder scoped inside the folder** → the driver's hierarchy
  is expressed as the builder-nesting graph. Capture the nesting and you inherit the driver's
  tree (folders, UDT structure, component nesting) — this is **not** a flat dump.
- `DriverAttributeInfo.FullName` **is the driver-side full reference** for read/write
  addressing — i.e. exactly the value the picker commits as `TagConfig.FullName`
  (`DriverAttributeInfo.cs:13`). No per-driver commit logic needed.

So a **capturing `IAddressSpaceBuilder`** run against any driver's `DiscoverAsync` yields a
`BrowseNode` tree whose leaves already carry the commit value and the datatype metadata.

## 3. Architecture — two-tier browse

```
BrowserSessionService.OpenAsync(driverType, configJson)
        │
        ├─ bespoke IDriverBrowser registered for driverType?  ── yes ──►  use it   (Tier 2)
        │        (OpcUaClient, Galaxy, later: large ControlLogix/BACnet)
        │
        └─ no bespoke browser ───►  DiscoveryDriverBrowser  (Tier 1, universal fallback)
                                     gated on online-discovery capability
```

- **Tier 1 — universal `DiscoveryDriverBrowser`** (this doc): the default. Covers any driver
  whose `DiscoverAsync` enumerates from the *device*.
- **Tier 2 — bespoke `IBrowseSession`** (existing pattern, `2026-05-28-driver-browsers-design.md`):
  reserved for large / structured / lazy address spaces where eager one-shot discovery is too
  heavy. A bespoke browser **overrides** the universal one simply by being registered for that
  `DriverType`.

### 3.1 Resolution change in `BrowserSessionService` (required)

`BrowserSessionService` currently indexes browsers with
`browsers.ToDictionary(b => b.DriverType)` — **one browser per type, and a duplicate
`DriverType` key throws at construction.** Therefore the universal browser **cannot** be one
of the injected `IEnumerable<IDriverBrowser>` (it has no single `DriverType`, and registering
it under every type would collide). Instead:

- Introduce `IUniversalDriverBrowser` with
  `Task<IBrowseSession> OpenAsync(string driverType, string configJson, CancellationToken ct)`
  and `bool CanBrowse(string driverType, string configJson)`.
- Inject it into `BrowserSessionService` as a **separate optional dependency** (not part of the
  `IDriverBrowser` set). `OpenAsync` resolves bespoke-first, falls back to the universal browser
  when no bespoke browser matches **and** `CanBrowse` is true; otherwise returns the existing
  "no browser registered" result (→ picker shows manual entry).
- `RootAsync`/`ExpandAsync`/`AttributesAsync`/`CloseAsync` are unchanged — the universal
  browser returns an ordinary `IBrowseSession` that lands in the same `BrowseSessionRegistry`
  (TTL reaper, 20 s per-call timeout) as today.

## 4. Components

### 4.1 `CapturingAddressSpaceBuilder` (the core mechanism)

An in-memory `IAddressSpaceBuilder` that records the streamed tree instead of materializing
OPC UA nodes. New home: `src/Core/ZB.MOM.WW.OtOpcUa.Commons/Browsing/` (Commons already owns
`BrowseNode`/`IBrowseSession` and is referenced by AdminUI; `IAddressSpaceBuilder`/
`DriverAttributeInfo` live in `Core.Abstractions`, which Commons can reference).

| Builder call | Captured as | Serves |
|---|---|---|
| `Folder(bn, dn)` | node `{ id = parentPath + "/" + bn, display = dn, Folder, children[] }`; returns a child capturing builder scoped to it | `BrowseNode(id, dn, Folder, HasChildrenHint=true)` |
| `Variable(bn, dn, attr)` | leaf `{ id = attr.FullName, display = dn, Leaf, attr }`; returns a stub `IVariableHandle{ FullReference = attr.FullName }` | `BrowseNode(attr.FullName, dn, Leaf, false)` + `AttributeInfo` |
| `AddProperty(...)` | attached to the current node's property bag (optional; feeds the side-panel) | `AttributesAsync` |
| `IVariableHandle.MarkAsAlarmCondition(info)` | records `IsAlarm=true` on the leaf; returns a **no-op `IAlarmConditionSink`** (browse never delivers live transitions) | `AttributeInfo.IsAlarm=true` (picker pre-fills a default `alarm` block, per existing Galaxy behaviour) |

Leaf → `AttributeInfo` mapping (drives the side-panel + commit):
`AttributeInfo(Name: bn, DriverDataType: attr.DriverDataType.ToString(), IsArray: attr.IsArray,
SecurityClass: attr.SecurityClass.ToString(), IsAlarm: attr.IsAlarm)`. Commit `NodeId` =
`attr.FullName` → `TagConfig.FullName`; `attr.ArrayDim`/`IsHistorized`/`WriteIdempotent` are
available to pre-fill the typed tag editor.

The builder enforces a **node-count cap** (config, default e.g. 50 000): once exceeded it stops
recording and marks the tree `truncated=true` so the session can surface "results truncated —
narrow the driver config or use manual entry" rather than OOM on a 100 k-node backend.

### 4.2 `DiscoveryDriverBrowser : IUniversalDriverBrowser`

`OpenAsync(driverType, configJson, ct)`:
1. `driver = _driverFactory.TryCreate(driverType, ephemeralId, PatchForBrowse(driverType, configJson))`
   — `null` ⇒ throw a clean "driver type not available in this host" (see §6 wiring).
2. `if (driver is not ITagDiscovery disc || !disc.SupportsOnlineDiscovery)` ⇒ throw
   "driver has no online discovery — author tags manually" (should not happen — `CanBrowse`
   already gated it, but defend).
3. `await driver.InitializeAsync(patchedConfigJson, openCts.Token)` — **real device connect**,
   under an **open-timeout** (config, default 60 s; separate from the 20 s per-call browse
   timeout, which only covers Root/Expand/Attributes serving from memory).
4. `var capture = new CapturingAddressSpaceBuilder(nodeCap); await disc.DiscoverAsync(capture, openCts.Token);`
   — eager, whole-tree, one-shot. **Except when `disc.RediscoverPolicy == UntilStable`**
   (FOCAS): its `FixedTreeCache` is filled by a background loop a couple of seconds *after*
   connect, so an immediate single pass captures an empty device folder (the comment above
   `FocasDriver.DiscoverAsync` states this explicitly). For `UntilStable` drivers, re-run the
   capture on a short interval until the node set is non-empty and stable across two passes,
   bounded by the open-timeout — the same contract `DriverInstanceActor` honours at deploy.
5. `await driver.ShutdownAsync(...)` in a `finally` — the tree is fully captured; hold nothing
   live. (No lazy expand ⇒ no need to keep the connection.)
6. return `new CapturedTreeBrowseSession(capture.Root)`.

`CanBrowse(driverType, configJson)` = `TryCreate` (cheap, no connect) succeeds **and** the
instance is `ITagDiscovery { SupportsOnlineDiscovery: true }`. Used by the AdminUI to decide
whether to render the **Browse** button vs. manual entry, before any connect.

### 4.3 `CapturedTreeBrowseSession : IBrowseSession`

Serves entirely from the in-memory captured tree (the shape the MTConnect research report
proposed for its cached-`/probe` session — now subsumed by this generic component): `RootAsync` = top-level nodes; `ExpandAsync(nodeId)` = the captured children of that
node; `AttributesAsync(nodeId)` = the leaf's `AttributeInfo` (+ any captured properties).
`Token`/`LastUsedUtc` per the interface; `DisposeAsync` drops the tree. No I/O after open.

## 5. The online-discovery capability gate

The distinction the gate encodes: does `DiscoverAsync` enumerate from the **device** (browsable)
or merely replay **pre-declared** tags (flat-address drivers — Modbus/S7/MELSEC/AbLegacy/
Omron-FINS, where it would show only what you already authored)?

**Add a default-interface property** on `ITagDiscovery` (mirrors the existing
`RediscoverPolicy => UntilStable` default member on the same interface — zero churn for drivers
that don't opt in):

```csharp
/// <summary>True when DiscoverAsync enumerates the tag set from the live backend
/// (browsable), rather than replaying pre-declared/authored tags. Default false.</summary>
bool SupportsOnlineDiscovery => false;
```

- **Return true (device-enumerated):** OpcUaClient, BACnet, MTConnect, and — **config-gated**
  — AbCip/TwinCAT/FOCAS (their device enumeration sits behind a config flag; the browse patch
  guarantees it at open time, see §5.1).
- **Leave false (authored replay):** Modbus, Modbus-RTU, S7, MELSEC, AbLegacy,
  Omron (CIP + FINS) — their pickers stay manual-entry (Omron gets an offline
  tag-export importer later); and **MQTT/Sparkplug + SQL poll**, whose runtime
  `DiscoverAsync` is also deliberately authored-only — they get **bespoke** browsers instead
  (observation-window / `INFORMATION_SCHEMA` walk — see the program doc §4), because a
  universal replay of authored tags would discover nothing new.

(Galaxy/OpcUaClient keep their bespoke browsers regardless — the flag only governs the
*universal fallback*.)

### 5.1 Per-driver browse-config patch (`PatchForBrowse`) — the one wrinkle

For AbCip/TwinCAT/FOCAS, device enumeration is **config-gated**: AbCip/TwinCAT's
`DiscoverAsync` only walks the controller when `EnableControllerBrowse = true`, and FOCAS only
emits its FixedTree when `FixedTree.Enabled = true` (default `false`,
`FocasFixedTreeOptions.Enabled`) — but the picker holds the *authoring* config (often
`false`). The universal browser therefore merges a tiny, **declarative** per-type JSON
patch before constructing the driver:

```csharp
static readonly IReadOnlyDictionary<string, string> BrowsePatches = new Dictionary<string,string>(OrdinalIgnoreCase) {
    ["AbCip"]   = """{ "EnableControllerBrowse": true }""",
    ["TwinCAT"] = """{ "EnableControllerBrowse": true }""",
    ["FOCAS"]   = """{ "FixedTree": { "Enabled": true } }""",
    // OpcUaClient/BACnet/MTConnect: no patch — discovery is unconditional
};
```

This is a dictionary entry, not code — it keeps the "zero per-driver *browser*" property while
being honest that browse-mode activation is one declarative fact per config-gated driver.
`SupportsOnlineDiscovery` for AbCip/TwinCAT/FOCAS returns true because the patch guarantees
browse mode at open time. (FOCAS additionally needs the §4.2 `UntilStable` settle — the patch
turns the FixedTree on; the settle waits for its cache to fill.)

## 6. Wiring & deployment

- Register in AdminUI DI (`EndpointRouteBuilderExtensions.AddAdminUI`):
  `services.AddSingleton<IUniversalDriverBrowser, DiscoveryDriverBrowser>();` — **one line**,
  no per-driver registration (contrast the two bespoke lines at
  `EndpointRouteBuilderExtensions.cs:49-50`).
- Depends on `IDriverFactory` (Core.Abstractions). In the **fused Host** this is the real
  `DriverFactoryRegistryAdapter` (all `Driver.*.Register()` ran) → every discovery-capable type
  is constructible. On a **standalone AdminUI** with no driver assemblies it's
  `NullDriverFactory` → `TryCreate` returns null → `CanBrowse` false → universal browse simply
  unavailable (graceful; picker falls back to manual entry). No new failure mode.
- **Network/credentials posture is unchanged from the existing browsers**: OpcUaClient/Galaxy
  browsers already connect to live backends *from the AdminUI process* using credentials in
  `configJson`. The universal browser just extends that to more driver types — same role gating
  (picker is behind the existing `ConfigEditor`/browse authorization), same
  credentials-in-JSON handling, same `BrowseSessionRegistry` TTL reaping.

## 7. Where universal applies vs. bespoke

| Driver | Universal covers it? | Bespoke later? |
|---|---|---|
| **AbCip** | Yes (patch → `EnableControllerBrowse`) | Only if controller symbol set too large for eager one-shot (UDT drill-down) |
| **TwinCAT** | Yes (patch) | Only for very large symbol sets / Flat-mode caveats |
| **FOCAS** | Yes (patch → `FixedTree.Enabled` + §4.2 UntilStable settle) | Unlikely — surface is small/curated |
| **MTConnect** | Yes (`/probe`) | Optional — `/probe` is already whole-model; eager is fine |
| **BACnet** | Yes for small sites | **Yes** for large multi-device sites (lazy per-device object-list) |
| **MQTT/Sparkplug** | **No** (`SupportsOnlineDiscovery=false` — runtime discovery is authored-only by design) | **Yes** — bespoke observation-window `MqttBrowseSession` (its design doc §4) |
| **SQL poll** | **No** (authored replay) | **Yes** — bespoke `INFORMATION_SCHEMA` schema browser (its design doc) |
| **OpcUaClient** | (bespoke already) | Keep bespoke (100k-node lazy browse) |
| **Galaxy** | (bespoke already) | Keep bespoke (two-stage attribute pick) |
| Modbus/RTU, S7, MELSEC, AbLegacy, Omron | **No** (`SupportsOnlineDiscovery=false`) | N/A — not browsable (Omron: offline importer P-later) |

## 8. Limits (why bespoke browsers still exist)

1. **Eager, not lazy.** `DiscoverAsync` runs the whole enumeration at open. For huge trees
   (ControlLogix 10 k+ tags, OPC UA 100 k nodes) the node-cap truncates and the UX is worse than
   a bespoke per-click `ExpandAsync`. Graduate those to Tier 2.
2. **Runs the real driver in-process** (construct + connect + one-shot discover + shutdown) —
   heavier than the lightweight ad-hoc bespoke session, and requires the driver assemblies in the
   host (fused Host: yes).
3. **No incrementally-fetched attribute panel** beyond what `DriverAttributeInfo` +
   `AddProperty` carry (fine for the picker's needs; Galaxy's richer two-stage pick stays bespoke).

## 9. Testing

- **Unit — `CapturingAddressSpaceBuilder`:** drive it with a fake `ITagDiscovery` that streams a
  known Folder/Variable/AddProperty/alarm graph; assert the `BrowseNode` tree, leaf `NodeId ==
  FullName`, `AttributeInfo` mapping, node-cap truncation, and the no-op alarm sink.
- **Unit — `DiscoveryDriverBrowser`:** fake `IDriverFactory` returning a fake driver; assert
  Initialize→Discover→Shutdown ordering, open-timeout, `PatchForBrowse` merge, `CanBrowse` gate
  (true for `SupportsOnlineDiscovery`, false otherwise / `TryCreate`→null), and the
  `UntilStable` settle (a fake `UntilStable` driver whose tree only fills on the second
  discovery pass must yield the full tree; a never-stable one must stop at the open-timeout).
- **Unit — `BrowserSessionService` resolution:** bespoke-first, universal-fallback, and
  manual-entry-when-neither; confirm no `ToDictionary` collision.
- **Integration (fixture-gated):** point the universal browser at the existing driver fixtures on
  `10.100.0.35` — AbCip (ControlLogix sim) and the OPC UA reference server — and assert a real
  captured tree; reuse the driver fixtures already in `tests/`.
- **Live-verify:** on docker-dev, open the `/uns` TagModal picker for an AbCip driver and confirm
  the controller tag tree renders and a picked leaf commits `TagConfig.FullName` (the usual
  live-`/run` discipline — Razor binding bugs pass unit tests).

## 10. Phasing & effort

- **P1 — universal browser end-to-end:** `CapturingAddressSpaceBuilder` +
  `CapturedTreeBrowseSession` + `DiscoveryDriverBrowser` + `IUniversalDriverBrowser` +
  `BrowserSessionService` fallback + `ITagDiscovery.SupportsOnlineDiscovery` default member +
  the AbCip/TwinCAT/FOCAS patches + the `UntilStable` settle loop (§4.2) + DI line + AdminUI
  `CanBrowse`→Browse-button gate. Set `SupportsOnlineDiscovery=true` on
  OpcUaClient(n/a—bespoke), AbCip, TwinCAT, FOCAS. **~3–5 days.**
- **P2 — light up new drivers for free:** as MTConnect/BACnet land, each sets
  `SupportsOnlineDiscovery=true` (+ patch if config-gated) and gets a working picker with no
  browser code. (MQTT/Sparkplug and SQL poll ship bespoke browsers instead — program doc §4.)
- **P3 — graduate to bespoke** only where §8.1 bites (large BACnet sites, ControlLogix UDT
  drill-down): add a bespoke `IBrowseSession` that overrides the universal fallback for that type.

Net: one ~3–5 day piece replaces the previously-planned per-driver browser builds for AbCip +
TwinCAT (and pre-covers FOCAS + every future discovery driver), with bespoke lazy browsers
reserved for the genuinely-large trees.

## 11. v3 forward note (Raw/UNS two-subtree — lands after this)

This browser ships **before** v3 (`docs/plans/2026-07-15-raw-uns-two-subtree-v3-design.md`)
against the current `/uns` TagModal picker. When v3's Raw tree lands, three things about the
*consumer* change while the session/tree layer here (`CapturingAddressSpaceBuilder`,
`CapturedTreeBrowseSession`, `IUniversalDriverBrowser`, `BrowserSessionService` resolution)
carries over unchanged:

1. **Commit contract:** §2's "commits `TagConfig.FullName`" describes the pre-v3 picker. In
   v3, `TagConfig` no longer carries identity (the tag's identity is its RawPath); the picked
   leaf's `attr.FullName` is written into the driver-typed `TagConfig` as an ordinary
   **address field**, and the commit creates **raw `Tag` rows** (name from browse name,
   datatype from `DriverAttributeInfo`) under a Device/TagGroup — optionally mirroring the
   captured folder nesting as `TagGroup`s.
2. **Config input:** v3 moves endpoint/connection settings from `DriverConfig` into
   `DeviceConfig`. `OpenAsync(driverType, configJson, ct)`'s `configJson` becomes the
   **merged Driver + Device config** composed by the caller (the `/raw` browse modal);
   `PatchForBrowse` merges on top as today.
3. **Browse-button gate:** unchanged mechanism (`CanBrowse` / bespoke registration), hosted
   in the `/raw` tree's "Add tags ▸ Browse device…" action instead of the equipment TagModal.