dohertj2/lmxopcua
1
0
Fork 0
Code Issues 17 Pull Requests Actions Packages Projects Releases Wiki Activity

docs(drivers): write the two missing driver guides (Sql, Calculation)

Browse Source 
Closes the §7.2 priority-3 item. Sql was a shipped, merged driver with NO
user-facing documentation at all -- only design and plan files, which describe
what was intended rather than what an operator needs to author one.

Sql.md covers: SQL Server as the only constructed provider (the other four
SqlProvider names are reserved, and authoring one fails rather than falling
back), the KeyValue / WideRow tag models with worked TagConfig, Query being
rejected end-to-end, the config table with real defaults read from
SqlDriverOptions, connectionStringRef never carrying credentials into an
artifact, per-source query grouping and its case-sensitive group keys, and the
value-bound-vs-identifier-validated split that makes it injection-safe.

Calculation.md covers: what a pseudo-driver is and why it occupies a driver slot,
a Calculation-vs-VirtualTags decision table (the two are easy to confuse), the
literal-only ctx.GetTag dependency extraction and the runtime-built-path trap,
the two deploy gates, the all-deps-arrived publish gate, and the Good->Bad-once
error semantics.

Two stale claims found by checking against source rather than inheriting them:

  - The driver README's Calculation row still carried G-1 ("no typed config form,
    RunTimeout unauthorable"), fixed on 2026-07-27 by CalculationDriverForm.
  - docs/Raw.md's "one auto-created default Engine device" NEVER SHIPPED. I had
    copied it into the new doc before checking; "Engine" has zero occurrences in
    src/ and nothing special-cases Calculation at device creation. Corrected in
    both files. The config key was also wrong in my first draft: runTimeoutMs, an
    int in milliseconds, not a TimeSpan string -- and it is ignored on reinit
    because the evaluator's timeout is constructor-fixed.

Full solution: 49 suites pass; the 2 failures are the known pre-existing
environment ones (AbLegacy.IntegrationTests docker fixture, Host.IntegrationTests).
OpcUaClient.IntegrationTests, which was silently failing, is now green.

Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
This commit is contained in:
Joseph Doherty
2026-07-28 14:40:51 -04:00
parent 4624141a5d
commit f3cd685da5
5 changed files with 312 additions and 944 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/drivers/Calculation.md
+133
View File
@@ -0,0 +1,133 @@
# Calculation Driver
Getting-started guide for the `Calculation` **pseudo-driver** — signal-level calculated tags computed
by C# scripts over other tags' live values. For the design rationale read
[`docs/plans/2026-07-15-calculation-driver-mini-design.md`](../plans/2026-07-15-calculation-driver-mini-design.md);
[`docs/Raw.md`](../Raw.md) covers where it sits in the `/raw` tree.
## What it talks to
**Nothing.** There is no backend, no endpoint, no connection and no discovery. Its inputs are other
tags' live values, fed to it by the driver host; its outputs are ordinary raw tags. That is why it is
called a pseudo-driver — it occupies a driver slot so its tags get the same authoring, deployment,
fan-out, historization and alarming as any device tag, without any of the I/O.
Because it is an ordinary driver from the host's point of view, a calc tag's value flows out through
`OnDataChange` exactly like a Modbus register's — so the dual-namespace fan-out, historian
registration and **calc-of-calc** (a calc tag reading another calc tag, via mux re-entry) all work
with no special-casing.
## Calculation vs. Virtual Tags — which one to use
They are close cousins and easy to confuse:
| | **Calculation** driver | **Virtual tags** |
|---|---|---|
| Lives in | the `/raw` tree, as a raw tag | the UNS tree, on an equipment |
| Identity | a **RawPath** | `{Equipment}/{EffectiveName}` |
| Reads by | `ctx.GetTag("<RawPath>")` — literal raw paths | `ctx.GetTag("{{equip}}/<RefName>")` — equipment-relative |
| Use when | the computation is a property of the *signal* (scaling, a derived rate, combining two registers) | the computation is a property of the *equipment* (an OEE roll-up, a per-machine state) |
If the same script should apply to many machines, you want a virtual tag with `{{equip}}`. If a single
derived signal belongs next to the device tags it is derived from, you want a calc tag.
## Authoring a calc tag
Calc tags are authored in `/raw` like any other tag: create a `Calculation` driver, add a device under
it, and add tags under that.
> ⚠️ **`docs/Raw.md` says a default `Engine` device is auto-created. It is not** — verified 2026-07-28,
> the string `"Engine"` has zero occurrences in `src/`, and nothing in the AdminUI or the config layer
> special-cases `Calculation` at device-creation time. Create the device yourself; the name is
> arbitrary but becomes a segment of every calc tag's RawPath, so pick it deliberately.
Per-tag `TagConfig`:
```json
{
"scriptId": "scr-oven-delta",
"changeTriggered": true,
"timerIntervalMs": 5000
}
```
| Field | Meaning |
|---|---|
| `scriptId` | FK to the shared `Script` entity. **Required** — a tag with no identifiable script is not a calc tag and is rejected at parse. |
| `changeTriggered` | Re-evaluate whenever any declared dependency changes. |
| `timerIntervalMs` | Also re-evaluate on this cadence. Absent ⇒ change-trigger only. |
`scriptSource` is **resolved at deploy time** from `scriptId` and travels in the artifact — the driver
never reads the config DB. A `scriptId` that resolves to empty source still maps: the driver logs it
and the tag simply never computes, rather than the whole driver failing.
## Dependencies are discovered from the script
You do not declare dependencies. The deploy pipeline extracts them from the script's **literal**
`ctx.GetTag("…")` reads and registers exactly those RawPaths with the dependency mux.
⚠️ **Literal means literal.** A path built at runtime — `ctx.GetTag("Plant/" + line + "/Temp")` — is
invisible to the extractor, so that dependency is never fed and the tag never sees its value. Write
the path as a constant string.
Two deploy-time gates run over the result:
- **`scriptId` existence** — a calc tag naming a script that does not exist fails the deploy.
- **Cycle detection (Tarjan)** — calc-of-calc is supported, but a cycle fails the deploy rather than
livelocking at runtime.
## Trigger and publish semantics
- **Nothing publishes until every declared dependency has arrived at least once.** Before that a read
returns `BadWaitingForInitialData`. This is the same gate `VirtualTagActor` uses, and it exists so a
calculation cannot publish a confident-looking value derived from defaults.
- **Equal results are deduped** — an evaluation producing the same value as last time does not
re-publish.
- **Timers are grouped by interval**, one timer per distinct interval, not one per tag.
## Error semantics
An evaluator failure publishes **Bad quality carrying the last-known value** — the value is not
zeroed, because a stale-but-flagged reading is more useful to an operator than a fabricated one. It
publishes:
- **once per Good→Bad transition**, not on every failed evaluation (a permanently broken script does
not flood), and only after all dependencies have arrived;
- with a `ScriptLogEntry` on the `script-logs` topic, attributed to the tag's `scriptId`, so the
AdminUI `/script-log` page names the failing script;
- recovery to Good is **force-published**, bypassing the dedup, so a recovered tag does not sit Bad
waiting for its value to change.
A historized Bad records `BadInternalError`.
## Config
```json
{ "runTimeoutMs": 2000 }
```
| Key | Default | Notes |
|---|---|---|
| `runTimeoutMs` | `2000` (2 s) | Per-script wall-clock evaluation budget, in **milliseconds** — not a `TimeSpan` string, unlike most driver timeouts in this repo. Parity with the virtual-tag evaluator. |
⚠️ **`runTimeoutMs` is ignored on a config *reinit*** — the evaluator's timeout is fixed at
construction. A changed value takes effect because a changed `DriverConfig` respawns the driver
(#516), not because the driver re-reads it in place.
There is nothing else — no endpoint, no credentials, no pooling. The driver is **single-connection**
in the AdminUI's sense (nothing lives on the device).
## Capabilities
`IDriver` (lifecycle + always-Connected health), `IDependencyConsumer` (the host feeds it upstream
values), `ISubscribable` (computed values flow out), `IReadable` (returns the last computed
snapshot).
No `IWritable` — a computed value has no meaningful write target. No `ITagDiscovery` — there is
nothing to browse.
## See also
- [`docs/Raw.md`](../Raw.md) — the `/raw` authoring tree
- [`docs/VirtualTags.md`](../VirtualTags.md) — the equipment-scoped alternative
- [`docs/ScriptEditor.md`](../ScriptEditor.md) — the Roslyn-backed script editor
docs/drivers/README.md
+2 -2
View File
@@ -35,8 +35,8 @@ Driver **factories** are registered at startup in `DriverFactoryRegistry` (`src/
| [OPC UA Client](OpcUaClient.md) | `Driver.OpcUaClient` | B | OPCFoundation `Opc.Ua.Client` | IDriver, ITagDiscovery, IReadable, IWritable, ISubscribable, IAlarmSource, IHistoryProvider, IHostConnectivityProbe | Gateway/aggregation driver — the only driver implementing driver-side `IHistoryProvider` (forwards HistoryRead to the upstream server). Opens a single `Session` against a remote OPC UA server and re-exposes its address space. Owns its own `ApplicationConfiguration` (distinct from `Client.Shared`) because it's always-on with keep-alive + `TransferSubscriptions` across SDK reconnect, not an interactive CLI |
| [MQTT](Mqtt.md) | `Driver.Mqtt` (+ `.Browser`, `.Contracts`) | A | [MQTTnet 5](https://github.com/dotnet/MQTTnet) | IDriver, ITagDiscovery, IReadable, ISubscribable, IHostConnectivityProbe, IRediscoverable | **Push, not poll, and read-only** — the broker delivers PUBLISH frames the driver forwards straight to `ISubscribable.OnDataChange`; `IReadable` serves the last retained/received value from cache, and there is **no `IWritable`** (nothing publishes back). Subscriptions are indexed **by topic filter, not by topic**, so wildcard tags survive a reconnect. A rejected CONNACK throws `MqttConnectRejectedException`: unrecoverable codes → `Faulted` + supervisor stop, transient → retry. **Two modes:** `Plain` binds a tag to a concrete topic + JSON path; `SparkplugB` decodes vendored Eclipse-Tahu protobuf under one `spBv1.0/{groupId}/#` subscription and binds by the `group/edgeNode/device?/metric` tuple (birth/alias/seq-gap state machine, death→STALE, scoped rebirth NCMD). `IRediscoverable` fires on a DBIRTH but is **inert platform-side** — see [#507](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/507) |
| [MTConnect](MTConnect.md) | `Driver.MTConnect` (+ `.Contracts`) | — | Hand-rolled `System.Xml.Linq` HTTP/XML client against a vendor-neutral MTConnect Agent (`/probe`, `/current`, `/sample`) — no TrakHound dependency (dropped; see the driver doc) | IDriver, ITagDiscovery, IReadable, ISubscribable, IHostConnectivityProbe, IRediscoverable | Read-only by design (no `IWritable`). Production data plane is entirely `ISubscribable`/`OnDataChange` — `IReadable` is CLI/test-only. Browse is free via the Wave-0 universal discovery browser, no bespoke browser project. `IRediscoverable`/`IHostConnectivityProbe` have no server-side consumer yet — a pre-existing fleet-wide gap, not MTConnect-specific |
| **Sql** (no dedicated page — see [design](../plans/2026-07-15-sql-poll-driver-design.md)) | `Driver.Sql` (+ `.Browser`, `.Contracts`) | A | `Microsoft.Data.SqlClient` behind an `ISqlDialect` seam (`SqlServerDialect` shipped; Postgres/ODBC deferred) | IDriver, ITagDiscovery, IReadable, ISubscribable, IHostConnectivityProbe | Read-only DB-poll driver with a bespoke schema browser. **Credentials are never persisted** — the deploy gate rejects a stored `connectionString` (#498) and catalog identifiers are validated against the live catalog's own spelling (#496). `SupportsOnlineDiscovery => false` by design (the bespoke browser wins). `SqlTagModel.Query` is deferred to P3 and rejected end-to-end by parser, validator, planner and editor. Merged `4ad54037` |
| **Calculation** (see [Raw.md](../Raw.md#the-calculation-driver)) | `Driver.Calculation` | A | none — computes from other tags' RawPaths | IDriver, IDependencyConsumer, ISubscribable, IReadable | **Pseudo-driver**, not a wire protocol: signal-level tags computed from other tags via a script, with deploy-time `scriptId`-existence + Tarjan cycle gates. Its probe is parse-only (no backend to reach). ⚠️ **No typed config form** — `DriverConfigModal` has no `Calculation` case, so `RunTimeout` is currently unauthorable through the AdminUI |
| [Sql](Sql.md) | `Driver.Sql` (+ `.Browser`, `.Contracts`) | A | `Microsoft.Data.SqlClient` behind an `ISqlDialect` seam (`SqlServerDialect` shipped; Postgres/ODBC deferred) | IDriver, ITagDiscovery, IReadable, ISubscribable, IHostConnectivityProbe | Read-only DB-poll driver with a bespoke schema browser. **Credentials are never persisted** — the deploy gate rejects a stored `connectionString` (#498) and catalog identifiers are validated against the live catalog's own spelling (#496). `SupportsOnlineDiscovery => false` by design (the bespoke browser wins). `SqlTagModel.Query` is deferred to P3 and rejected end-to-end by parser, validator, planner and editor. Merged `4ad54037` |
| [Calculation](Calculation.md) | `Driver.Calculation` | A | none — computes from other tags' RawPaths | IDriver, IDependencyConsumer, ISubscribable, IReadable | **Pseudo-driver**, not a wire protocol: signal-level tags computed from other tags via a script, with deploy-time `scriptId`-existence + Tarjan cycle gates. Its probe is parse-only (no backend to reach). `runTimeoutMs` is authorable through `CalculationDriverForm` (the "no typed config form" gap — G-1 — was closed 2026-07-27) |
| [Historian.Gateway](../Historian.md) | `Driver.Historian.Gateway` | — | `ZB.MOM.WW.HistorianGateway.Client` gRPC (`historian_gateway.v1`) | IHistorianDataSource (server-side read backend) + alarm `SendEvent` writer + `WriteLiveValues` recorder + `IHistorianProvisioning` | Not a tag driver — the sole historian backend. Registers `GatewayHistorianDataSource : IHistorianDataSource` for HistoryRead and serves alarm-write + continuous historization through the gateway. No `IDriver`/`ITagDiscovery` surface. (The retired Wonderware sidecar backend it replaced is documented at [Historian.Wonderware.md](Historian.Wonderware.md).) |
## Per-driver documentation
docs/drivers/Sql.md
+174
View File
@@ -0,0 +1,174 @@
# SQL Poll Driver
Getting-started guide for the `Sql` driver — a **read-only** Equipment-kind driver that polls rows
out of a relational database and publishes them as OPC UA variables. For the design rationale read
[`docs/plans/2026-07-15-sql-poll-driver-design.md`](../plans/2026-07-15-sql-poll-driver-design.md);
for the build-vs-plan record read
[`docs/plans/2026-07-24-sql-poll-driver.md`](../plans/2026-07-24-sql-poll-driver.md).
## What it talks to
An ordinary SQL database — typically a MES / LIMS / historian staging table that some other system
already writes to. There is no protocol here and nothing to "connect" to a machine: the driver runs
`SELECT`s on a timer and turns cells into tag values.
**v1 constructs SQL Server only** (`Microsoft.Data.SqlClient`). `SqlProvider` also declares
`Postgres`, `MySql`, `Odbc` and `Oracle`, but those are **reserved names, not working backends** —
authoring one fails the driver rather than silently falling back.
**Read-only, structurally.** `SqlDriver` does not implement `IWritable`, so no configuration can
produce a write. The `allowWrites` config field exists and is **inert**; the factory *warns* when it
is authored `true` rather than failing, on the grounds that the flag cannot do harm but an operator
who believes writes are enabled can.
## Tag models
A tag says how to find its value. Two models ship; a third is deferred.
### `KeyValue` — the EAV shape
One row per tag. The driver matches `keyColumn = keyValue` and reads `valueColumn`.
```json
{
"model": "KeyValue",
"table": "dbo.TagValues",
"keyColumn": "TagName",
"keyValue": "Line1.Oven.Temp",
"valueColumn": "Value",
"timestampColumn": "UpdatedUtc"
}
```
### `WideRow` — one row, many signals
The tag names its `columnName`; the row is picked either by a `whereColumn`/`whereValue` pair or by
newest-timestamp.
```json
{
"model": "WideRow",
"table": "dbo.LineStatus",
"columnName": "OvenTemp",
"whereColumn": "LineId",
"whereValue": "L1"
}
```
```json
{
"model": "WideRow",
"table": "dbo.Readings",
"columnName": "OvenTemp",
"topByTimestamp": "CapturedUtc"
}
```
### `Query` — deferred to P3
Named arbitrary-`SELECT`. **The v1 parser rejects it**, and that refusal is enforced end-to-end —
parser, validator, planner and the AdminUI editor all decline it rather than accepting the value and
ignoring it later.
## Optional per-tag fields
| Field | Effect |
|---|---|
| `timestampColumn` | Source timestamp for the value. Absent ⇒ the driver stamps the poll time. |
| `type` | Explicit OPC UA type override. Absent ⇒ inferred from the result set's column metadata. |
| `pollIntervalMs` | Per-tag override of the instance's `defaultPollInterval`. |
## Driver config
```json
{
"provider": "SqlServer",
"connectionStringRef": "MesStaging",
"defaultPollInterval": "00:00:05",
"operationTimeout": "00:00:15",
"commandTimeout": "00:00:10",
"maxConcurrentGroups": 4,
"nullIsBad": false
}
```
| Key | Default | Notes |
|---|---|---|
| `provider` | `SqlServer` | The only provider v1 constructs. |
| `connectionStringRef` | — | **A name, never a connection string.** See below. |
| `defaultPollInterval` | `00:00:05` | Applied to any group without its own interval. |
| `operationTimeout` | `00:00:15` | Client-side wall-clock deadline; a breach surfaces `BadTimeout`. |
| `commandTimeout` | `00:00:10` | ADO.NET server-side backstop. **Must be less than `operationTimeout`** — it is the backstop, not the deadline. |
| `maxConcurrentGroups` | `4` | Connection-pool guard: cap on group queries in flight. |
| `nullIsBad` | `false` | `false` ⇒ a NULL cell publishes **Uncertain**; `true` ⇒ **Bad**. |
| `allowWrites` | — | **Inert.** v1 has no write path; authoring `true` logs a warning. |
### Credentials never enter the deployment artifact
`connectionStringRef` is a **name** resolved at Initialize from the environment / secret store — e.g.
`Sql__ConnectionStrings__MesStaging`. There is no field that holds a connection string, so a sealed
deployment artifact cannot carry database credentials even by accident.
A useful side effect of the in-place reinit added for [#516](#): a **rotated credential is picked up
on the next config apply without a process restart**, because the driver re-resolves the reference
when it re-parses.
## How polling works
Tags are **grouped by source** and one query is issued per group, not per tag — `SqlGroupPlanner`
emits a `SqlQueryPlan` per distinct group key. Grouping is deterministic (groups appear in the input
order of their first member, members keep their input order, parameters keep first-appearance order),
so an unchanged configuration produces byte-identical SQL.
⚠️ **Group keys are case-sensitive on identifiers.** Two tags authored `dbo.T` and `DBO.T` plan as
**two** groups — one extra round-trip, not a correctness problem, but worth knowing if a group count
looks higher than expected.
## SQL injection: how the driver is safe
The two classes of authored string are handled differently, and deliberately so:
- **Value-bearing** fields (`keyValue`, `whereValue`) are bound as `DbParameter`s. They are never
concatenated into command text.
- **Identifier-bearing** fields (`table`, `keyColumn`, `valueColumn`, `timestampColumn`,
`columnName`, `whereColumn`, `topByTimestamp`) cannot be parameterised in SQL. They are validated
against **`INFORMATION_SCHEMA`** and dialect-quoted before they reach a command.
The parser deliberately does **not** sanitise either kind — sanitising a value would corrupt
legitimate data, and sanitising an identifier would mask an authoring error the catalog check reports
properly.
⚠️ **An unreadable catalog FAULTS the driver rather than rejecting every tag** (Gitea #496). This is
the safer failure: "I cannot verify these identifiers" and "these identifiers are invalid" are
different conditions, and reporting the second when the first is true would send an operator hunting a
configuration bug that does not exist. The catalog gate also substitutes the catalog's own spelling of
an identifier, so a case difference between the authored blob and the database resolves rather than
failing.
## Browse
`SqlDriverBrowser` browses the live catalog — schema → table → column — so the `/raw` browse picker
can commit columns as tags. A committed leaf carries **schema, table and column**: a column name alone
cannot address a SQL tag, and an earlier version that emitted only the column produced tags whose
typed editor opened empty.
## Authoring in the AdminUI
- **Driver config** — `/raw` → driver node → *Configure*. Sql is a **single-connection** driver: the
connection lives on the driver, not on a device, and the modal says so.
- **Tags** — the typed `SqlTagConfigModel` editor, or CSV import (Sql has typed CSV columns), or
browse-commit from the picker.
## Capabilities
`IDriver`, `ITagDiscovery`, `IReadable`, `ISubscribable`, `IHostConnectivityProbe`.
No `IWritable` (structural, above). No `IRediscoverable` — a SQL schema change is not signalled, and
`DiscoverAsync` echoes authored config rather than enumerating the database, so this driver is
deliberately excluded from any discovery-diff machinery.
## See also
- [`docs/Raw.md`](../Raw.md) — the `/raw` authoring tree
- [`docs/drivers/README.md`](README.md) — the driver index
- Gitea **#496** / **#497** / **#498** — the shipped follow-ups