lmxopcua/docs/plans/2026-07-15-universal-discovery-browser-design.md

Universal Discover-backed browser (DiscoveryDriverBrowser) — design

Status: P1 implemented 2026-07-15 (code-complete + unit-tested, live /run gate pending); P2/P3 pending. Foundational Wave-0 item of the next-driver roadmap (docs/research/drivers/README.md).

1. Motivation

Today the AdminUI address picker is backed by one bespoke IDriverBrowser per driver type — only OpcUaClient and Galaxy exist, each a full hand-written IBrowseSession (connect + lazy level-at-a-time browse). Every other driver's picker is a static manual-entry stub.

But the existing-driver browse audit (docs/research/drivers/00-existing-driver-browse-audit.md) found that AbCip, TwinCAT, and FOCAS already contain complete, unit-tested symbol enumeration — it lives in their runtime ITagDiscovery.DiscoverAsync, streamed into an IAddressSpaceBuilder, and is simply never surfaced to the picker. The same is true of every future discovery-capable driver (MTConnect /probe, BACnet Who-Is/object-list).

The insight: DiscoverAsync already produces exactly the tree the picker wants. Instead of writing N bespoke browsers, write one generic browser that captures whatever a driver streams into the builder and re-presents it. This is the highest-leverage single piece of browse work on the roadmap.

2. The seam that makes it work

ITagDiscovery.DiscoverAsync(IAddressSpaceBuilder builder, ct) does not return a tree — it streams structured nodes into a builder (Core.Abstractions/IAddressSpaceBuilder.cs):

IAddressSpaceBuilder Folder(string browseName, string displayName);       // → child builder
IVariableHandle      Variable(string browseName, string displayName, DriverAttributeInfo a);
void                 AddProperty(string browseName, DriverDataType t, object? v);

Two facts close the loop:

• Folder(...) returns a child builder scoped inside the folder → the driver's hierarchy is expressed as the builder-nesting graph. Capture the nesting and you inherit the driver's tree (folders, UDT structure, component nesting) — this is not a flat dump.
• DriverAttributeInfo.FullName is the driver-side full reference for read/write addressing — i.e. exactly the value the picker commits as TagConfig.FullName (DriverAttributeInfo.cs:13). No per-driver commit logic needed.

So a capturing IAddressSpaceBuilder run against any driver's DiscoverAsync yields a BrowseNode tree whose leaves already carry the commit value and the datatype metadata.

3. Architecture — two-tier browse

BrowserSessionService.OpenAsync(driverType, configJson)
        │
        ├─ bespoke IDriverBrowser registered for driverType?  ── yes ──►  use it   (Tier 2)
        │        (OpcUaClient, Galaxy, later: large ControlLogix/BACnet)
        │
        └─ no bespoke browser ───►  DiscoveryDriverBrowser  (Tier 1, universal fallback)
                                     gated on online-discovery capability

• Tier 1 — universal DiscoveryDriverBrowser (this doc): the default. Covers any driver whose DiscoverAsync enumerates from the device.
• Tier 2 — bespoke IBrowseSession (existing pattern, 2026-05-28-driver-browsers-design.md): reserved for large / structured / lazy address spaces where eager one-shot discovery is too heavy. A bespoke browser overrides the universal one simply by being registered for that DriverType.

3.1 Resolution change in BrowserSessionService (required)

BrowserSessionService currently indexes browsers with browsers.ToDictionary(b => b.DriverType) — one browser per type, and a duplicate DriverType key throws at construction. Therefore the universal browser cannot be one of the injected IEnumerable<IDriverBrowser> (it has no single DriverType, and registering it under every type would collide). Instead:

• Introduce IUniversalDriverBrowser with Task<IBrowseSession> OpenAsync(string driverType, string configJson, CancellationToken ct) and bool CanBrowse(string driverType, string configJson).
• Inject it into BrowserSessionService as a separate optional dependency (not part of the IDriverBrowser set). OpenAsync resolves bespoke-first, falls back to the universal browser when no bespoke browser matches and CanBrowse is true; otherwise returns the existing "no browser registered" result (→ picker shows manual entry).
• RootAsync/ExpandAsync/AttributesAsync/CloseAsync are unchanged — the universal browser returns an ordinary IBrowseSession that lands in the same BrowseSessionRegistry (TTL reaper, 20 s per-call timeout) as today.

4. Components

4.1 CapturingAddressSpaceBuilder (the core mechanism)

An in-memory IAddressSpaceBuilder that records the streamed tree instead of materializing OPC UA nodes. New home: src/Core/ZB.MOM.WW.OtOpcUa.Commons/Browsing/ (Commons already owns BrowseNode/IBrowseSession and is referenced by AdminUI; IAddressSpaceBuilder/ DriverAttributeInfo live in Core.Abstractions, which Commons can reference).

Builder call	Captured as	Serves
Folder(bn, dn)	node { id = parentPath + "/" + bn, display = dn, Folder, children[] }; returns a child capturing builder scoped to it	BrowseNode(id, dn, Folder, HasChildrenHint=true)
Variable(bn, dn, attr)	leaf { id = attr.FullName, display = dn, Leaf, attr }; returns a stub IVariableHandle{ FullReference = attr.FullName }	BrowseNode(attr.FullName, dn, Leaf, false) + AttributeInfo
AddProperty(...)	attached to the current node's property bag (optional; feeds the side-panel)	AttributesAsync
IVariableHandle.MarkAsAlarmCondition(info)	records IsAlarm=true on the leaf; returns a no-op IAlarmConditionSink (browse never delivers live transitions)	AttributeInfo.IsAlarm=true (picker pre-fills a default alarm block, per existing Galaxy behaviour)

Leaf → AttributeInfo mapping (drives the side-panel + commit): AttributeInfo(Name: bn, DriverDataType: attr.DriverDataType.ToString(), IsArray: attr.IsArray, SecurityClass: attr.SecurityClass.ToString(), IsAlarm: attr.IsAlarm). Commit NodeId = attr.FullName → TagConfig.FullName; attr.ArrayDim/IsHistorized/WriteIdempotent are available to pre-fill the typed tag editor.

The builder enforces a node-count cap (config, default e.g. 50 000): once exceeded it stops recording and marks the tree truncated=true so the session can surface "results truncated — narrow the driver config or use manual entry" rather than OOM on a 100 k-node backend.

4.2 DiscoveryDriverBrowser : IUniversalDriverBrowser

OpenAsync(driverType, configJson, ct):

1. driver = _driverFactory.TryCreate(driverType, ephemeralId, PatchForBrowse(driverType, configJson)) — null ⇒ throw a clean "driver type not available in this host" (see §6 wiring).
2. if (driver is not ITagDiscovery disc || !disc.SupportsOnlineDiscovery) ⇒ throw "driver has no online discovery — author tags manually" (should not happen — CanBrowse already gated it, but defend).
3. await driver.InitializeAsync(patchedConfigJson, openCts.Token) — real device connect, under an open-timeout (config, default 60 s; separate from the 20 s per-call browse timeout, which only covers Root/Expand/Attributes serving from memory).
4. var capture = new CapturingAddressSpaceBuilder(nodeCap); await disc.DiscoverAsync(capture, openCts.Token); — eager, whole-tree, one-shot. Except when disc.RediscoverPolicy == UntilStable (FOCAS): its FixedTreeCache is filled by a background loop a couple of seconds after connect, so an immediate single pass captures an empty device folder (the comment above FocasDriver.DiscoverAsync states this explicitly). For UntilStable drivers, re-run the capture on a short interval until the node set is non-empty and stable across two passes, bounded by the open-timeout — the same contract DriverInstanceActor honours at deploy.
5. await driver.ShutdownAsync(...) in a finally — the tree is fully captured; hold nothing live. (No lazy expand ⇒ no need to keep the connection.) The shutdown itself is bounded (the R2-01 per-op-deadline rule, program doc §3.1 — no unbounded waits anywhere): it runs under its own 10 s linked CTS, because a driver wedged during discovery may be equally wedged in shutdown, and cleanup must not extend the open beyond open-timeout + that bound. On shutdown timeout: log a warning, abandon the instance (drop the reference — never block on it or rethrow), and still return / fail the open on the discovery outcome — a hung shutdown neither fails a successful capture nor masks the real discovery error.
6. return new CapturedTreeBrowseSession(capture.Root).

Capture concurrency — coalescing + global cap

Every OpenAsync is heavy: it constructs, connects, and fully enumerates a real driver against the live device. Repeated Browse clicks (or two operators picking against the same controller) must not stack parallel symbol walks on one PLC. Two controls, both inside DiscoveryDriverBrowser:

• In-flight coalescing, keyed on (driverType, hash of configJson) (e.g. SHA-256 of the caller-supplied JSON; PatchForBrowse is a pure function of driverType, so hashing the pre-patch config is equivalent per key). A second open for the same key awaits the same capture task rather than starting another walk; each awaiting caller then receives its own CapturedTreeBrowseSession (own token, own TTL in the BrowseSessionRegistry) over the shared, immutable captured tree. A failed capture fails every coalesced awaiter with the same error. The coalescing map holds in-flight captures only — no result caching: the entry is removed when the capture completes (success or failure), so the next open for that key (including a Refresh, §4.3) triggers a fresh capture.
• A global cap on simultaneous captures — MaxConcurrentCaptures = 4 (a const, like BrowserSessionService.PerCallTimeout). Excess opens queue (semaphore, FIFO) and still honour the caller's cancellation token while queued. Coalesced awaiters don't consume a slot — only distinct in-flight captures count against the cap.

CanBrowse(driverType, configJson) = TryCreate (cheap, no connect) yields a non-null instance and that instance is ITagDiscovery { SupportsOnlineDiscovery: true }. Used by the AdminUI to decide whether to render the Browse button vs. manual entry, before any connect. Two hardening rules:

• TryCreate can throw, not just return null. Factories parse configJson inside TryCreate, so malformed/half-typed JSON (exactly what a picker holds mid-authoring) can surface as an exception. CanBrowse must catch any exception → false — it is a UI affordance gate and must never throw into the page.
• Defensive teardown of the throwaway instance. Whatever the outcome (not discovery-capable, flag false, or an exception after construction), CanBrowse best-effort ShutdownAsyncs/disposes any instance it did create — bounded, swallow-all — so probing never leaks driver instances.

Invariant the throwaway-instance pattern depends on: driver constructors are connection-free today — every driver in the tree connects in InitializeAsync, never in its ctor. CanBrowse (construct → inspect → discard, potentially on every picker render) and the capture path's construct-then-patch flow both rely on this. A future driver that opens a device connection in its constructor would silently turn CanBrowse into a connect storm — treat "no I/O in driver ctors" as a driver-authoring rule this browser now depends on (worth a line in the program doc's cross-cutting rules when P1 lands).

4.3 CapturedTreeBrowseSession : IBrowseSession

Serves entirely from the in-memory captured tree (the shape the MTConnect research report proposed for its cached-/probe session — now subsumed by this generic component): RootAsync = top-level nodes; ExpandAsync(nodeId) = the captured children of that node; AttributesAsync(nodeId) = the leaf's AttributeInfo (+ any captured properties). Token/LastUsedUtc per the interface; DisposeAsync drops the session's reference to the tree (the tree is immutable and may be shared by coalesced sessions, §4.2 — disposing one session never affects another). No I/O after open.

Point-in-time snapshot + Refresh. The captured tree is a snapshot taken at open time and never updates for the session's whole TTL — a tag added on the controller after open does not appear. The picker therefore exposes a Refresh action = close the session + re-open it (CloseAsync then OpenAsync with the same driverType/configJson): a fresh capture that flows through the same coalescing/cap path as any open. Because the coalescing map holds in-flight captures only (no result caching), Refresh always yields a new capture — or joins one already in flight, which is equally fresh. Cheap to build (the picker already has both calls); the UI should label browse results as "snapshot taken at open/refresh time".

5. The online-discovery capability gate

The distinction the gate encodes: does DiscoverAsync enumerate from the device (browsable) or merely replay pre-declared tags (flat-address drivers — Modbus/S7/MELSEC/AbLegacy/ Omron-FINS, where it would show only what you already authored)?

Add a default-interface property on ITagDiscovery (mirrors the existing RediscoverPolicy => UntilStable default member on the same interface — zero churn for drivers that don't opt in):

/// <summary>True when DiscoverAsync enumerates the tag set from the live backend
/// (browsable), rather than replaying pre-declared/authored tags. Default false.</summary>
bool SupportsOnlineDiscovery => false;

• Return true (device-enumerated): OpcUaClient, BACnet, MTConnect, and — config-gated — AbCip/TwinCAT/FOCAS (their device enumeration sits behind a config flag; the browse patch guarantees it at open time, see §5.1).
• Leave false (authored replay): Modbus, Modbus-RTU, S7, MELSEC, AbLegacy, Omron (CIP + FINS) — their pickers stay manual-entry (Omron gets an offline tag-export importer later); and MQTT/Sparkplug + SQL poll, whose runtime DiscoverAsync is also deliberately authored-only — they get bespoke browsers instead (observation-window / INFORMATION_SCHEMA walk — see the program doc §4), because a universal replay of authored tags would discover nothing new.

(Galaxy/OpcUaClient keep their bespoke browsers regardless — the flag only governs the universal fallback.)

5.1 Per-driver browse-config patch (PatchForBrowse) — the one wrinkle

For AbCip/TwinCAT/FOCAS, device enumeration is config-gated: AbCip/TwinCAT's DiscoverAsync only walks the controller when EnableControllerBrowse = true, and FOCAS only emits its FixedTree when FixedTree.Enabled = true (default false, FocasFixedTreeOptions.Enabled) — but the picker holds the authoring config (often false). The universal browser therefore merges a tiny, declarative per-type JSON patch before constructing the driver:

static readonly IReadOnlyDictionary<string, string> BrowsePatches = new Dictionary<string,string>(OrdinalIgnoreCase) {
    ["AbCip"]   = """{ "EnableControllerBrowse": true }""",
    ["TwinCAT"] = """{ "EnableControllerBrowse": true }""",
    ["FOCAS"]   = """{ "FixedTree": { "Enabled": true } }""",
    // OpcUaClient/BACnet/MTConnect: no patch — discovery is unconditional
};

This is a dictionary entry, not code — it keeps the "zero per-driver browser" property while being honest that browse-mode activation is one declarative fact per config-gated driver. SupportsOnlineDiscovery for AbCip/TwinCAT/FOCAS returns true because the patch guarantees browse mode at open time. (FOCAS additionally needs the §4.2 UntilStable settle — the patch turns the FixedTree on; the settle waits for its cache to fill.)

6. Wiring & deployment

• Register in AdminUI DI (EndpointRouteBuilderExtensions.AddAdminUI): services.AddSingleton<IUniversalDriverBrowser, DiscoveryDriverBrowser>(); — one line, no per-driver registration (contrast the two bespoke lines at EndpointRouteBuilderExtensions.cs:49-50).
• Depends on IDriverFactory (Core.Abstractions). In the fused Host this is the real DriverFactoryRegistryAdapter (all Driver.*.Register() ran) → every discovery-capable type is constructible. On a standalone AdminUI with no driver assemblies it's NullDriverFactory → TryCreate returns null → CanBrowse false → universal browse simply unavailable (graceful; picker falls back to manual entry). No new failure mode.
• Network/credentials posture is unchanged from the existing browsers: OpcUaClient/Galaxy browsers already connect to live backends from the AdminUI process using credentials in configJson. The universal browser just extends that to more driver types — same role gating (picker is behind the existing ConfigEditor/browse authorization), same credentials-in-JSON handling, same BrowseSessionRegistry TTL reaping.

7. Where universal applies vs. bespoke

Driver	Universal covers it?	Bespoke later?
AbCip	Yes (patch → EnableControllerBrowse)	Only if controller symbol set too large for eager one-shot (UDT drill-down)
TwinCAT	Yes (patch)	Only for very large symbol sets / Flat-mode caveats
FOCAS	Yes (patch → FixedTree.Enabled + §4.2 UntilStable settle)	Unlikely — surface is small/curated
MTConnect	Yes (/probe)	Optional — /probe is already whole-model; eager is fine
BACnet	Yes for small sites	Yes for large multi-device sites (lazy per-device object-list)
MQTT/Sparkplug	No (SupportsOnlineDiscovery=false — runtime discovery is authored-only by design)	Yes — bespoke observation-window MqttBrowseSession (its design doc §4)
SQL poll	No (authored replay)	Yes — bespoke INFORMATION_SCHEMA schema browser (its design doc)
OpcUaClient	(bespoke already)	Keep bespoke (100k-node lazy browse)
Galaxy	(bespoke already)	Keep bespoke (two-stage attribute pick)
Modbus/RTU, S7, MELSEC, AbLegacy, Omron	No (SupportsOnlineDiscovery=false)	N/A — not browsable (Omron: offline importer P-later)

8. Limits (why bespoke browsers still exist)

1. Eager, not lazy. DiscoverAsync runs the whole enumeration at open. For huge trees (ControlLogix 10 k+ tags, OPC UA 100 k nodes) the node-cap truncates and the UX is worse than a bespoke per-click ExpandAsync. Graduate those to Tier 2.
2. Runs the real driver in-process (construct + connect + one-shot discover + bounded shutdown) — heavier than the lightweight ad-hoc bespoke session, and requires the driver assemblies in the host (fused Host: yes). The §4.2 coalescing + MaxConcurrentCaptures cap bound the aggregate load, but each individual capture is still a full device walk.
3. No incrementally-fetched attribute panel beyond what DriverAttributeInfo + AddProperty carry (fine for the picker's needs; Galaxy's richer two-stage pick stays bespoke).

9. Testing

• Unit — CapturingAddressSpaceBuilder: drive it with a fake ITagDiscovery that streams a known Folder/Variable/AddProperty/alarm graph; assert the BrowseNode tree, leaf NodeId == FullName, AttributeInfo mapping, node-cap truncation, and the no-op alarm sink.
• Unit — DiscoveryDriverBrowser: fake IDriverFactory returning a fake driver; assert Initialize→Discover→Shutdown ordering (shutdown always runs, even on discovery failure), open-timeout, PatchForBrowse merge, CanBrowse gate (true for SupportsOnlineDiscovery, false otherwise / TryCreate→null), and the UntilStable settle (a fake UntilStable driver whose tree only fills on the second discovery pass must yield the full tree; a never-stable one must stop at the open-timeout).
• Unit — capture coalescing: two concurrent OpenAsync calls with the same (driverType, configJson) run one capture (fake driver counts Initialize/Discover invocations) and both callers receive distinct session tokens over the shared tree; disposing one session leaves the other browsable; a failed capture fails both awaiters; a subsequent open after completion (Refresh path) runs a new capture (in-flight-only map, no result caching).
• Unit — capture cap queueing: with MaxConcurrentCaptures = 4 and >4 concurrent opens on distinct keys (fake drivers gated on a test signal), at most 4 captures are in flight at once; queued opens complete after a slot frees; a queued open observes caller cancellation.
• Unit — CanBrowse hardening: a fake factory whose TryCreate throws on malformed configJson yields CanBrowse == false (no exception escapes); when TryCreate returns a non-discovery (or flag-false) instance, CanBrowse is false and the created instance was shut down/disposed.
• Unit — bounded shutdown: a fake driver whose ShutdownAsync never completes — the open still returns within open-timeout + the 10 s shutdown bound, on the discovery outcome, for both legs: successful capture ⇒ a usable session is returned; failed discovery ⇒ the discovery error (not a shutdown timeout) surfaces. The wedged instance is abandoned, not awaited.
• Unit — BrowserSessionService resolution: bespoke-first, universal-fallback, and manual-entry-when-neither; confirm no ToDictionary collision.
• Integration (fixture-gated): point the universal browser at the existing driver fixtures on 10.100.0.35 — AbCip (ControlLogix sim) and the OPC UA reference server — and assert a real captured tree; reuse the driver fixtures already in tests/.
• Live-verify: on docker-dev, open the /uns TagModal picker for an AbCip driver and confirm the controller tag tree renders and a picked leaf commits TagConfig.FullName (the usual live-/run discipline — Razor binding bugs pass unit tests).

10. Phasing & effort

• P1 — universal browser end-to-end: CapturingAddressSpaceBuilder + CapturedTreeBrowseSession + DiscoveryDriverBrowser + IUniversalDriverBrowser + BrowserSessionService fallback + ITagDiscovery.SupportsOnlineDiscovery default member + the AbCip/TwinCAT/FOCAS patches + the UntilStable settle loop (§4.2) + capture coalescing and the MaxConcurrentCaptures cap + the bounded (10 s) shutdown + hardened CanBrowse (catch-throwing-TryCreate, defensive teardown) + DI line + AdminUI CanBrowse→Browse-button gate + the picker Refresh action (§4.3). Set SupportsOnlineDiscovery=true on OpcUaClient(n/a—bespoke), AbCip, TwinCAT, FOCAS. ~3–5 days.
• P2 — light up new drivers for free: as MTConnect/BACnet land, each sets SupportsOnlineDiscovery=true (+ patch if config-gated) and gets a working picker with no browser code. (MQTT/Sparkplug and SQL poll ship bespoke browsers instead — program doc §4.)
• P3 — graduate to bespoke only where §8.1 bites (large BACnet sites, ControlLogix UDT drill-down): add a bespoke IBrowseSession that overrides the universal fallback for that type.

Net: one ~3–5 day piece replaces the previously-planned per-driver browser builds for AbCip + TwinCAT (and pre-covers FOCAS + every future discovery driver), with bespoke lazy browsers reserved for the genuinely-large trees.

11. v3 forward note (Raw/UNS two-subtree — lands after this)

This browser ships before v3 (docs/plans/2026-07-15-raw-uns-two-subtree-v3-design.md) against the current /uns TagModal picker. When v3's Raw tree lands, three things about the consumer change while the session/tree layer here (CapturingAddressSpaceBuilder, CapturedTreeBrowseSession, IUniversalDriverBrowser, BrowserSessionService resolution) carries over unchanged:

1. Commit contract: §2's "commits TagConfig.FullName" describes the pre-v3 picker. In v3, TagConfig no longer carries identity (the tag's identity is its RawPath); the picked leaf's attr.FullName is written into the driver-typed TagConfig as an ordinary address field, and the commit creates raw Tag rows (name from browse name, datatype from DriverAttributeInfo) under a Device/TagGroup — optionally mirroring the captured folder nesting as TagGroups.
2. Config input: v3 moves endpoint/connection settings from DriverConfig into DeviceConfig. OpenAsync(driverType, configJson, ct)'s configJson becomes the merged Driver + Device config composed by the caller (the /raw browse modal); PatchForBrowse merges on top as today.
3. Browse-button gate: unchanged mechanism (CanBrowse / bespoke registration), hosted in the /raw tree's "Add tags ▸ Browse device…" action instead of the equipment TagModal.