lmxopcua/docs/drivers/Sql.md

# SQL Poll Driver

Getting-started guide for the `Sql` driver — a **read-only** Equipment-kind driver that polls rows
out of a relational database and publishes them as OPC UA variables. For the design rationale read
[`docs/plans/2026-07-15-sql-poll-driver-design.md`](../plans/2026-07-15-sql-poll-driver-design.md);
for the build-vs-plan record read
[`docs/plans/2026-07-24-sql-poll-driver.md`](../plans/2026-07-24-sql-poll-driver.md).

## What it talks to

An ordinary SQL database — typically a MES / LIMS / historian staging table that some other system
already writes to. There is no protocol here and nothing to "connect" to a machine: the driver runs
`SELECT`s on a timer and turns cells into tag values.

**Three providers construct**: `SqlServer` (`Microsoft.Data.SqlClient`), `Postgres`
(`Npgsql`) and `Oracle` (ODP.NET managed — **Oracle 12.2 or later**, the floor the
`FETCH FIRST` paging needs). `SqlProvider` also declares `MySql` and `Odbc`, but those remain
**reserved names, not working backends** — authoring one fails the driver rather than silently
falling back.

Two provider-specific things are worth knowing before you author against them:

- **Identifier case folds, and the catalog gate absorbs it.** Postgres lower-cases and Oracle
  upper-cases unquoted identifiers. Every authored identifier is resolved to the catalog's own
  spelling, so an authored `widgets` binds to Oracle's `WIDGETS` rather than failing.
- **Oracle `NUMBER` infers as `Float64`.** Column metadata does not carry precision/scale to the
  type map, so an integer-shaped `NUMBER` still infers double. Use the per-tag `type` override
  (see [Optional per-tag fields](#optional-per-tag-fields)) when the OPC UA type matters.

**Read-only, structurally.** `SqlDriver` does not implement `IWritable`, so no configuration can
produce a write. The `allowWrites` config field exists and is **inert**; the factory *warns* when it
is authored `true` rather than failing, on the grounds that the flag cannot do harm but an operator
who believes writes are enabled can.

## Tag models

A tag says how to find its value. Two models ship; a third is deferred.

### `KeyValue` — the EAV shape

One row per tag. The driver matches `keyColumn = keyValue` and reads `valueColumn`.

```json
{
  "model": "KeyValue",
  "table": "dbo.TagValues",
  "keyColumn": "TagName",
  "keyValue": "Line1.Oven.Temp",
  "valueColumn": "Value",
  "timestampColumn": "UpdatedUtc"
}
```

### `WideRow` — one row, many signals

The tag names its `columnName`; the row is picked either by a `whereColumn`/`whereValue` pair or by
newest-timestamp.

⚠️ **The selector keys live inside a nested `rowSelector` object**, not at the top level. A blob that
spells `whereColumn` / `whereValue` / `topByTimestamp` beside `columnName` carries *no* row selector
as far as `SqlEquipmentTagParser` is concerned: the tag is refused, the driver skips it, and it reads
`BadNodeIdUnknown` forever. The typed editor, the browse-commit mapper and the deploy gate all use
the nested form.

```json
{
  "model": "WideRow",
  "table": "dbo.LineStatus",
  "columnName": "OvenTemp",
  "rowSelector": { "whereColumn": "LineId", "whereValue": "L1" }
}
```

```json
{
  "model": "WideRow",
  "table": "dbo.Readings",
  "columnName": "OvenTemp",
  "rowSelector": { "topByTimestamp": "CapturedUtc" }
}
```

⚠️ **`topByTimestamp` needs an ordering column that is unique per row.** It is executed as an
`ORDER BY <column> DESC` with the dialect's single-row limit, so when the column ties the server
returns an arbitrary row among the tied ones — and the driver cannot warn about it, because the other
rows were discarded before the result set reached it (the `whereColumn`/`whereValue` form *does*
warn, on `Rows.Count > 1`). A `date`-typed column ties for every row written on the same day, and
browse-commit will nominate one. Prefer a `datetime2`/sequence column that is unique per row, or
point the tag at a latest-row view.

Both value-bearing fields — `whereValue` here and `keyValue` on the `KeyValue` model — may be
authored as a JSON string, number or boolean, and are captured as text. They are **bound as
parameters** either way, and matched back as text (the key cell is stringified), so a key column
that holds an INTEGER station id can be authored either `"7"` or `7`. A JSON object, array or `null`
is not a value and is refused. (They were read asymmetrically until 2026-08-10, so a numerically
authored `keyValue` refused the tag while the identically-bound `whereValue` was served — and the
deploy gate reported the key as *missing* when it was right there in the blob.)

### `Query` — deferred to P3

Named arbitrary-`SELECT`. **The v1 parser rejects it**, and that refusal is enforced end-to-end —
parser, validator, planner and the AdminUI editor all decline it rather than accepting the value and
ignoring it later.

## Optional per-tag fields

| Field | Effect |
|---|---|
| `timestampColumn` | Source timestamp for the value. Absent ⇒ the driver stamps the poll time. |
| `type` | Explicit OPC UA type override. Absent ⇒ inferred from the result set's column metadata. |

There is **no per-tag poll-interval field**. The interval is the subscriber's, floored by
`PollGroupEngine.DefaultMinInterval`, falling back to the instance's `defaultPollInterval` when the
subscriber names none. (A `pollIntervalMs` per-tag override was documented here for a while and never
existed in the parser — nothing reads it.)

## Driver config

```json
{
  "provider": "SqlServer",
  "connectionStringRef": "MesStaging",
  "defaultPollInterval": "00:00:05",
  "operationTimeout": "00:00:15",
  "commandTimeout": "00:00:10",
  "maxConcurrentGroups": 4,
  "nullIsBad": false
}
```

| Key | Default | Notes |
|---|---|---|
| `provider` | `SqlServer` | `SqlServer`, `Postgres` or `Oracle`. `MySql` / `Odbc` are reserved and rejected. |
| `connectionStringRef` | — | **A name, never a connection string.** See below. |
| `defaultPollInterval` | `00:00:05` | Applied to any group without its own interval. |
| `operationTimeout` | `00:00:15` | Client-side wall-clock deadline; a breach surfaces `BadTimeout`. |
| `commandTimeout` | `00:00:10` | ADO.NET server-side backstop. **Must be less than `operationTimeout`** — it is the backstop, not the deadline. |
| `maxConcurrentGroups` | `4` | Connection-pool guard: cap on group queries in flight. |
| `nullIsBad` | `false` | `false` ⇒ a NULL cell publishes **Uncertain**; `true` ⇒ **Bad**. |
| `allowWrites` | — | **Inert.** v1 has no write path; authoring `true` logs a warning. |

### Credentials never enter the deployment artifact

`connectionStringRef` is a **name** resolved at Initialize from the environment / secret store — e.g.
`Sql__ConnectionStrings__MesStaging`. There is no field that holds a connection string, so a sealed
deployment artifact cannot carry database credentials even by accident.

A useful side effect of the in-place reinit added for [#516](#): a **rotated credential is picked up
on the next config apply without a process restart**, because the driver re-resolves the reference
when it re-parses.

## How polling works

Tags are **grouped by source** and one query is issued per group, not per tag — `SqlGroupPlanner`
emits a `SqlQueryPlan` per distinct group key. Grouping is deterministic (groups appear in the input
order of their first member, members keep their input order, parameters keep first-appearance order),
so an unchanged configuration produces byte-identical SQL.

⚠️ **Group keys are case-sensitive on identifiers.** Two tags authored `dbo.T` and `DBO.T` plan as
**two** groups — one extra round-trip, not a correctness problem, but worth knowing if a group count
looks higher than expected.

## SQL injection: how the driver is safe

The two classes of authored string are handled differently, and deliberately so:

- **Value-bearing** fields (`keyValue`, `whereValue`) are bound as `DbParameter`s. They are never
  concatenated into command text.
- **Identifier-bearing** fields (`table`, `keyColumn`, `valueColumn`, `timestampColumn`,
  `columnName`, `whereColumn`, `topByTimestamp`) cannot be parameterised in SQL. They are validated
  against the live catalog — **`INFORMATION_SCHEMA`** on SQL Server and Postgres, the **`ALL_*`**
  views (`ALL_USERS` / `ALL_TABLES` ∪ `ALL_VIEWS` / `ALL_TAB_COLUMNS`) on Oracle — and
  dialect-quoted before they reach a command.

The parser deliberately does **not** sanitise either kind — sanitising a value would corrupt
legitimate data, and sanitising an identifier would mask an authoring error the catalog check reports
properly.

⚠️ **An unreadable catalog FAULTS the driver rather than rejecting every tag** (Gitea #496). This is
the safer failure: "I cannot verify these identifiers" and "these identifiers are invalid" are
different conditions, and reporting the second when the first is true would send an operator hunting a
configuration bug that does not exist. The catalog gate also substitutes the catalog's own spelling of
an identifier, so a case difference between the authored blob and the database resolves rather than
failing.

## Browse

`SqlDriverBrowser` browses the live catalog — schema → table → column — so the `/raw` browse picker
can commit columns as tags. A committed leaf carries **schema, table and column**: a column name alone
cannot address a SQL tag, and an earlier version that emitted only the column produced tags whose
typed editor opened empty.

Browse works against all three constructed providers; the catalog query is the dialect's own
(`INFORMATION_SCHEMA` on SQL Server and Postgres — with `pg_catalog` / `information_schema`
themselves excluded — and the `ALL_*` views on Oracle).

### The committed row selector (Gitea #552)

A browse-committed column becomes a **WideRow** tag, and a wide-row tag must say *which row* it reads
— the table and column alone are not a binding. The browse session therefore states a fourth address
field alongside schema/table/column: the table's preferred **date/time-typed** column, which the
commit writes as `rowSelector.topByTimestamp` ("read the newest row"). Before this the commit emitted
a selector-less blob that `SqlEquipmentTagParser` and the authoring gate both refuse, so no
browse-committed Sql tag was ever readable.

Three properties of the choice are deliberate:

- **Candidacy is by declared TYPE, never by column name.** The session asks
  `ISqlDialect.MapColumnType` for `DriverDataType.DateTime` — the same metadata the browse tree already
  renders per column. A name heuristic (`*_ts`, `*time*`) would happily order a wide-row read by a
  *string* column called `timestamp`, binding the tag to whichever row sorts last lexically.
- **Candidates are RANKED: a time-bearing type beats a date-only one.** `MapColumnType` reports
  `DriverDataType.DateTime` for SQL Server's and PostgreSQL's `date` too, and ordering a single-row read
  by a date-only column ties for *every row written on the same day* — "the newest row" quietly becomes
  storage order among the ties, and the reader cannot warn about it because the server has already
  discarded the others. So a `datetime2`/`timestamptz`/… column wins over a `date` one wherever both
  exist, regardless of declaration order. A date-only column is still stated when it is **all** the table
  has, so such a table stays bindable by browse rather than becoming un-committable.
  **Oracle's `DATE` is exempt** — it carries a time-of-day component (which is why `OracleDialect` maps
  it faithfully rather than as a widening), so it is not demoted there.
- **The tie-break within a rank is first in catalog order** (every dialect's column query orders by
  ordinal position), so the pick is stable across calls, connections and servers. On a
  `CreatedUtc`/`ModifiedUtc` pair it takes the earlier-declared one. It is a **visible, editable
  default**: the typed editor shows the selector and the operator can change it — to the other
  timestamp, or to a `whereColumn`/`whereValue` pair entirely.

**A table with no date/time column cannot be bound by browse, and the commit says so.** The remaining
selector is a `whereColumn`/`whereValue` pair whose value is *data*, not schema, so nothing in a
catalog can supply it. `RawBrowseCommitMapper.DescribeUncommittableLeaf` refuses that leaf by name,
names the table, and points at the tag editor; no selector is invented, and the mapper's blob stays
selector-less rather than dropping to a generic `{"address": …}` the typed editor cannot read.

## Authoring in the AdminUI

- **Driver config** — `/raw` → driver node → *Configure*. Sql is a **single-connection** driver: the
  connection lives on the driver, not on a device, and the modal says so.
- **Tags** — the typed `SqlTagConfigModel` editor, or CSV import (Sql has typed CSV columns), or
  browse-commit from the picker.

## Capabilities

`IDriver`, `ITagDiscovery`, `IReadable`, `ISubscribable`, `IHostConnectivityProbe`.

No `IWritable` (structural, above). No `IRediscoverable` — a SQL schema change is not signalled, and
`DiscoverAsync` echoes authored config rather than enumerating the database, so this driver is
deliberately excluded from any discovery-diff machinery.

## See also

- [`docs/Raw.md`](../Raw.md) — the `/raw` authoring tree
- [`docs/drivers/README.md`](README.md) — the driver index
- Gitea **#496** / **#497** / **#498** — the shipped follow-ups