lmxopcua/docs/plans/2026-07-15-universal-discovery-browser-design.md

Universal Discover-backed browser (DiscoveryDriverBrowser) — design

Status: draft, 2026-07-15. Foundational Wave-0 item of the next-driver roadmap (docs/research/drivers/README.md).

1. Motivation

Today the AdminUI address picker is backed by one bespoke IDriverBrowser per driver type — only OpcUaClient and Galaxy exist, each a full hand-written IBrowseSession (connect + lazy level-at-a-time browse). Every other driver's picker is a static manual-entry stub.

But the existing-driver browse audit (docs/research/drivers/00-existing-driver-browse-audit.md) found that AbCip, TwinCAT, and FOCAS already contain complete, unit-tested symbol enumeration — it lives in their runtime ITagDiscovery.DiscoverAsync, streamed into an IAddressSpaceBuilder, and is simply never surfaced to the picker. The same is true of every future discovery-capable driver (MTConnect /probe, BACnet Who-Is/object-list).

The insight: DiscoverAsync already produces exactly the tree the picker wants. Instead of writing N bespoke browsers, write one generic browser that captures whatever a driver streams into the builder and re-presents it. This is the highest-leverage single piece of browse work on the roadmap.

2. The seam that makes it work

ITagDiscovery.DiscoverAsync(IAddressSpaceBuilder builder, ct) does not return a tree — it streams structured nodes into a builder (Core.Abstractions/IAddressSpaceBuilder.cs):

IAddressSpaceBuilder Folder(string browseName, string displayName);       // → child builder
IVariableHandle      Variable(string browseName, string displayName, DriverAttributeInfo a);
void                 AddProperty(string browseName, DriverDataType t, object? v);

Two facts close the loop:

• Folder(...) returns a child builder scoped inside the folder → the driver's hierarchy is expressed as the builder-nesting graph. Capture the nesting and you inherit the driver's tree (folders, UDT structure, component nesting) — this is not a flat dump.
• DriverAttributeInfo.FullName is the driver-side full reference for read/write addressing — i.e. exactly the value the picker commits as TagConfig.FullName (DriverAttributeInfo.cs:13). No per-driver commit logic needed.

So a capturing IAddressSpaceBuilder run against any driver's DiscoverAsync yields a BrowseNode tree whose leaves already carry the commit value and the datatype metadata.

3. Architecture — two-tier browse

BrowserSessionService.OpenAsync(driverType, configJson)
        │
        ├─ bespoke IDriverBrowser registered for driverType?  ── yes ──►  use it   (Tier 2)
        │        (OpcUaClient, Galaxy, later: large ControlLogix/BACnet)
        │
        └─ no bespoke browser ───►  DiscoveryDriverBrowser  (Tier 1, universal fallback)
                                     gated on online-discovery capability

• Tier 1 — universal DiscoveryDriverBrowser (this doc): the default. Covers any driver whose DiscoverAsync enumerates from the device.
• Tier 2 — bespoke IBrowseSession (existing pattern, 2026-05-28-driver-browsers-design.md): reserved for large / structured / lazy address spaces where eager one-shot discovery is too heavy. A bespoke browser overrides the universal one simply by being registered for that DriverType.

3.1 Resolution change in BrowserSessionService (required)

BrowserSessionService currently indexes browsers with browsers.ToDictionary(b => b.DriverType) — one browser per type, and a duplicate DriverType key throws at construction. Therefore the universal browser cannot be one of the injected IEnumerable<IDriverBrowser> (it has no single DriverType, and registering it under every type would collide). Instead:

• Introduce IUniversalDriverBrowser with Task<IBrowseSession> OpenAsync(string driverType, string configJson, CancellationToken ct) and bool CanBrowse(string driverType, string configJson).
• Inject it into BrowserSessionService as a separate optional dependency (not part of the IDriverBrowser set). OpenAsync resolves bespoke-first, falls back to the universal browser when no bespoke browser matches and CanBrowse is true; otherwise returns the existing "no browser registered" result (→ picker shows manual entry).
• RootAsync/ExpandAsync/AttributesAsync/CloseAsync are unchanged — the universal browser returns an ordinary IBrowseSession that lands in the same BrowseSessionRegistry (TTL reaper, 20 s per-call timeout) as today.

4. Components

4.1 CapturingAddressSpaceBuilder (the core mechanism)

An in-memory IAddressSpaceBuilder that records the streamed tree instead of materializing OPC UA nodes. New home: src/Core/ZB.MOM.WW.OtOpcUa.Commons/Browsing/ (Commons already owns BrowseNode/IBrowseSession and is referenced by AdminUI; IAddressSpaceBuilder/ DriverAttributeInfo live in Core.Abstractions, which Commons can reference).

Builder call	Captured as	Serves
Folder(bn, dn)	node { id = parentPath + "/" + bn, display = dn, Folder, children[] }; returns a child capturing builder scoped to it	BrowseNode(id, dn, Folder, HasChildrenHint=true)
Variable(bn, dn, attr)	leaf { id = attr.FullName, display = dn, Leaf, attr }; returns a stub IVariableHandle{ FullReference = attr.FullName }	BrowseNode(attr.FullName, dn, Leaf, false) + AttributeInfo
AddProperty(...)	attached to the current node's property bag (optional; feeds the side-panel)	AttributesAsync
IVariableHandle.MarkAsAlarmCondition(info)	records IsAlarm=true on the leaf; returns a no-op IAlarmConditionSink (browse never delivers live transitions)	AttributeInfo.IsAlarm=true (picker pre-fills a default alarm block, per existing Galaxy behaviour)

Leaf → AttributeInfo mapping (drives the side-panel + commit): AttributeInfo(Name: bn, DriverDataType: attr.DriverDataType.ToString(), IsArray: attr.IsArray, SecurityClass: attr.SecurityClass.ToString(), IsAlarm: attr.IsAlarm). Commit NodeId = attr.FullName → TagConfig.FullName; attr.ArrayDim/IsHistorized/WriteIdempotent are available to pre-fill the typed tag editor.

The builder enforces a node-count cap (config, default e.g. 50 000): once exceeded it stops recording and marks the tree truncated=true so the session can surface "results truncated — narrow the driver config or use manual entry" rather than OOM on a 100 k-node backend.

4.2 DiscoveryDriverBrowser : IUniversalDriverBrowser

OpenAsync(driverType, configJson, ct):

1. driver = _driverFactory.TryCreate(driverType, ephemeralId, PatchForBrowse(driverType, configJson)) — null ⇒ throw a clean "driver type not available in this host" (see §6 wiring).
2. if (driver is not ITagDiscovery disc || !disc.SupportsOnlineDiscovery) ⇒ throw "driver has no online discovery — author tags manually" (should not happen — CanBrowse already gated it, but defend).
3. await driver.InitializeAsync(patchedConfigJson, openCts.Token) — real device connect, under an open-timeout (config, default 60 s; separate from the 20 s per-call browse timeout, which only covers Root/Expand/Attributes serving from memory).
4. var capture = new CapturingAddressSpaceBuilder(nodeCap); await disc.DiscoverAsync(capture, openCts.Token); — eager, whole-tree, one-shot. Except when disc.RediscoverPolicy == UntilStable (FOCAS): its FixedTreeCache is filled by a background loop a couple of seconds after connect, so an immediate single pass captures an empty device folder (the comment above FocasDriver.DiscoverAsync states this explicitly). For UntilStable drivers, re-run the capture on a short interval until the node set is non-empty and stable across two passes, bounded by the open-timeout — the same contract DriverInstanceActor honours at deploy.
5. await driver.ShutdownAsync(...) in a finally — the tree is fully captured; hold nothing live. (No lazy expand ⇒ no need to keep the connection.)
6. return new CapturedTreeBrowseSession(capture.Root).

CanBrowse(driverType, configJson) = TryCreate (cheap, no connect) succeeds and the instance is ITagDiscovery { SupportsOnlineDiscovery: true }. Used by the AdminUI to decide whether to render the Browse button vs. manual entry, before any connect.

4.3 CapturedTreeBrowseSession : IBrowseSession

Serves entirely from the in-memory captured tree (the shape the MTConnect research report proposed for its cached-/probe session — now subsumed by this generic component): RootAsync = top-level nodes; ExpandAsync(nodeId) = the captured children of that node; AttributesAsync(nodeId) = the leaf's AttributeInfo (+ any captured properties). Token/LastUsedUtc per the interface; DisposeAsync drops the tree. No I/O after open.

5. The online-discovery capability gate

The distinction the gate encodes: does DiscoverAsync enumerate from the device (browsable) or merely replay pre-declared tags (flat-address drivers — Modbus/S7/MELSEC/AbLegacy/ Omron-FINS, where it would show only what you already authored)?

Add a default-interface property on ITagDiscovery (mirrors the existing RediscoverPolicy => UntilStable default member on the same interface — zero churn for drivers that don't opt in):

/// <summary>True when DiscoverAsync enumerates the tag set from the live backend
/// (browsable), rather than replaying pre-declared/authored tags. Default false.</summary>
bool SupportsOnlineDiscovery => false;

• Return true (device-enumerated): OpcUaClient, BACnet, MTConnect, and — config-gated — AbCip/TwinCAT/FOCAS (their device enumeration sits behind a config flag; the browse patch guarantees it at open time, see §5.1).
• Leave false (authored replay): Modbus, Modbus-RTU, S7, MELSEC, AbLegacy, Omron (CIP + FINS) — their pickers stay manual-entry (Omron gets an offline tag-export importer later); and MQTT/Sparkplug + SQL poll, whose runtime DiscoverAsync is also deliberately authored-only — they get bespoke browsers instead (observation-window / INFORMATION_SCHEMA walk — see the program doc §4), because a universal replay of authored tags would discover nothing new.

(Galaxy/OpcUaClient keep their bespoke browsers regardless — the flag only governs the universal fallback.)

5.1 Per-driver browse-config patch (PatchForBrowse) — the one wrinkle

For AbCip/TwinCAT/FOCAS, device enumeration is config-gated: AbCip/TwinCAT's DiscoverAsync only walks the controller when EnableControllerBrowse = true, and FOCAS only emits its FixedTree when FixedTree.Enabled = true (default false, FocasFixedTreeOptions.Enabled) — but the picker holds the authoring config (often false). The universal browser therefore merges a tiny, declarative per-type JSON patch before constructing the driver:

static readonly IReadOnlyDictionary<string, string> BrowsePatches = new Dictionary<string,string>(OrdinalIgnoreCase) {
    ["AbCip"]   = """{ "EnableControllerBrowse": true }""",
    ["TwinCAT"] = """{ "EnableControllerBrowse": true }""",
    ["FOCAS"]   = """{ "FixedTree": { "Enabled": true } }""",
    // OpcUaClient/BACnet/MTConnect: no patch — discovery is unconditional
};

This is a dictionary entry, not code — it keeps the "zero per-driver browser" property while being honest that browse-mode activation is one declarative fact per config-gated driver. SupportsOnlineDiscovery for AbCip/TwinCAT/FOCAS returns true because the patch guarantees browse mode at open time. (FOCAS additionally needs the §4.2 UntilStable settle — the patch turns the FixedTree on; the settle waits for its cache to fill.)

6. Wiring & deployment

• Register in AdminUI DI (EndpointRouteBuilderExtensions.AddAdminUI): services.AddSingleton<IUniversalDriverBrowser, DiscoveryDriverBrowser>(); — one line, no per-driver registration (contrast the two bespoke lines at EndpointRouteBuilderExtensions.cs:49-50).
• Depends on IDriverFactory (Core.Abstractions). In the fused Host this is the real DriverFactoryRegistryAdapter (all Driver.*.Register() ran) → every discovery-capable type is constructible. On a standalone AdminUI with no driver assemblies it's NullDriverFactory → TryCreate returns null → CanBrowse false → universal browse simply unavailable (graceful; picker falls back to manual entry). No new failure mode.
• Network/credentials posture is unchanged from the existing browsers: OpcUaClient/Galaxy browsers already connect to live backends from the AdminUI process using credentials in configJson. The universal browser just extends that to more driver types — same role gating (picker is behind the existing ConfigEditor/browse authorization), same credentials-in-JSON handling, same BrowseSessionRegistry TTL reaping.

7. Where universal applies vs. bespoke

Driver	Universal covers it?	Bespoke later?
AbCip	Yes (patch → EnableControllerBrowse)	Only if controller symbol set too large for eager one-shot (UDT drill-down)
TwinCAT	Yes (patch)	Only for very large symbol sets / Flat-mode caveats
FOCAS	Yes (patch → FixedTree.Enabled + §4.2 UntilStable settle)	Unlikely — surface is small/curated
MTConnect	Yes (/probe)	Optional — /probe is already whole-model; eager is fine
BACnet	Yes for small sites	Yes for large multi-device sites (lazy per-device object-list)
MQTT/Sparkplug	No (SupportsOnlineDiscovery=false — runtime discovery is authored-only by design)	Yes — bespoke observation-window MqttBrowseSession (its design doc §4)
SQL poll	No (authored replay)	Yes — bespoke INFORMATION_SCHEMA schema browser (its design doc)
OpcUaClient	(bespoke already)	Keep bespoke (100k-node lazy browse)
Galaxy	(bespoke already)	Keep bespoke (two-stage attribute pick)
Modbus/RTU, S7, MELSEC, AbLegacy, Omron	No (SupportsOnlineDiscovery=false)	N/A — not browsable (Omron: offline importer P-later)

8. Limits (why bespoke browsers still exist)

1. Eager, not lazy. DiscoverAsync runs the whole enumeration at open. For huge trees (ControlLogix 10 k+ tags, OPC UA 100 k nodes) the node-cap truncates and the UX is worse than a bespoke per-click ExpandAsync. Graduate those to Tier 2.
2. Runs the real driver in-process (construct + connect + one-shot discover + shutdown) — heavier than the lightweight ad-hoc bespoke session, and requires the driver assemblies in the host (fused Host: yes).
3. No incrementally-fetched attribute panel beyond what DriverAttributeInfo + AddProperty carry (fine for the picker's needs; Galaxy's richer two-stage pick stays bespoke).

9. Testing

• Unit — CapturingAddressSpaceBuilder: drive it with a fake ITagDiscovery that streams a known Folder/Variable/AddProperty/alarm graph; assert the BrowseNode tree, leaf NodeId == FullName, AttributeInfo mapping, node-cap truncation, and the no-op alarm sink.
• Unit — DiscoveryDriverBrowser: fake IDriverFactory returning a fake driver; assert Initialize→Discover→Shutdown ordering, open-timeout, PatchForBrowse merge, CanBrowse gate (true for SupportsOnlineDiscovery, false otherwise / TryCreate→null), and the UntilStable settle (a fake UntilStable driver whose tree only fills on the second discovery pass must yield the full tree; a never-stable one must stop at the open-timeout).
• Unit — BrowserSessionService resolution: bespoke-first, universal-fallback, and manual-entry-when-neither; confirm no ToDictionary collision.
• Integration (fixture-gated): point the universal browser at the existing driver fixtures on 10.100.0.35 — AbCip (ControlLogix sim) and the OPC UA reference server — and assert a real captured tree; reuse the driver fixtures already in tests/.
• Live-verify: on docker-dev, open the /uns TagModal picker for an AbCip driver and confirm the controller tag tree renders and a picked leaf commits TagConfig.FullName (the usual live-/run discipline — Razor binding bugs pass unit tests).

10. Phasing & effort

• P1 — universal browser end-to-end: CapturingAddressSpaceBuilder + CapturedTreeBrowseSession + DiscoveryDriverBrowser + IUniversalDriverBrowser + BrowserSessionService fallback + ITagDiscovery.SupportsOnlineDiscovery default member + the AbCip/TwinCAT/FOCAS patches + the UntilStable settle loop (§4.2) + DI line + AdminUI CanBrowse→Browse-button gate. Set SupportsOnlineDiscovery=true on OpcUaClient(n/a—bespoke), AbCip, TwinCAT, FOCAS. ~3–5 days.
• P2 — light up new drivers for free: as MTConnect/BACnet land, each sets SupportsOnlineDiscovery=true (+ patch if config-gated) and gets a working picker with no browser code. (MQTT/Sparkplug and SQL poll ship bespoke browsers instead — program doc §4.)
• P3 — graduate to bespoke only where §8.1 bites (large BACnet sites, ControlLogix UDT drill-down): add a bespoke IBrowseSession that overrides the universal fallback for that type.

Net: one ~3–5 day piece replaces the previously-planned per-driver browser builds for AbCip + TwinCAT (and pre-covers FOCAS + every future discovery driver), with bespoke lazy browsers reserved for the genuinely-large trees.

11. v3 forward note (Raw/UNS two-subtree — lands after this)

This browser ships before v3 (docs/plans/2026-07-15-raw-uns-two-subtree-v3-design.md) against the current /uns TagModal picker. When v3's Raw tree lands, three things about the consumer change while the session/tree layer here (CapturingAddressSpaceBuilder, CapturedTreeBrowseSession, IUniversalDriverBrowser, BrowserSessionService resolution) carries over unchanged:

1. Commit contract: §2's "commits TagConfig.FullName" describes the pre-v3 picker. In v3, TagConfig no longer carries identity (the tag's identity is its RawPath); the picked leaf's attr.FullName is written into the driver-typed TagConfig as an ordinary address field, and the commit creates raw Tag rows (name from browse name, datatype from DriverAttributeInfo) under a Device/TagGroup — optionally mirroring the captured folder nesting as TagGroups.
2. Config input: v3 moves endpoint/connection settings from DriverConfig into DeviceConfig. OpenAsync(driverType, configJson, ct)'s configJson becomes the merged Driver + Device config composed by the caller (the /raw browse modal); PatchForBrowse merges on top as today.
3. Browse-button gate: unchanged mechanism (CanBrowse / bespoke registration), hosted in the /raw tree's "Add tags ▸ Browse device…" action instead of the equipment TagModal.