f5c30ed0ef
v2-ci / unit-tests (push) Has been cancelled
Item 2: a tag's optional `write` block picks the statement the driver builds from its own catalog-validated identifiers (no authored SQL; values always bound): - Update (default, unchanged back-compat UPDATE ... WHERE key; 0 rows = BadNoData) - Insert (key+value[+stamp]; append for a WideRow topByTimestamp tag) - Upsert: SQL Server MERGE WITH (HOLDLOCK), PostgreSQL INSERT ... ON CONFLICT, MySQL ON DUPLICATE KEY UPDATE (1-or-2 rows), Oracle MERGE from DUAL - Procedure: named stored procedure resolved against the routine catalog by the catalog gate (INFORMATION_SCHEMA.ROUTINES / ALL_PROCEDURES), value/key/timestamp params by name (MySQL positional) - stampTimestamp sets the tag's timestamp column to the write time (UTC) Odbc serves Update/Insert only (shared SqlWriteSupport table). Constraint failures map to BadEntryExists (unique) / BadOutOfRange (other integrity, data exception); everything else, permission denied included, stays BadCommunicationError. WriteIdempotent stays false. The parser refuses malformed blocks (strict keys/enums/param charset); DraftValidator emits blocking SqlWriteBlockInvalid / SqlWriteModeUnsupported from the driver's own parse; the AdminUI editor surfaces the same message. Items 1 + 4 (docs/drivers/Sql.md): ODBC stays the ANSI SQL:2008 profile (native providers for SQL Server/MySQL, DB2 unsupported) with an ops note on driver manager + vendor driver on every Odbc and AdminUI host; read-only drivers require a SELECT-only login, writable drivers a login limited to their write targets. Tests: SqlWriteModeTests (parse, every dialect's SQL, gate, status map, SQLite end-to-end), DraftValidatorSqlWriteBlockTests, AdminUI round-trip; live SqlWriteModesLiveTests on SQL Server, PostgreSQL, MySQL, Oracle (seeds gain write_targets + set_target) and an Odbc write leg. Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
493 lines
31 KiB
Markdown
493 lines
31 KiB
Markdown
# SQL Poll Driver
|
||
|
||
> **Status lives in Gitea.** Open follow-ups and known gaps noted in this doc are tracked as issues on
|
||
> gitea.dohertylan.com/dohertj2/lmxopcua (label `migrated`, 2026-11-06). This doc describes the system — do not track status here; file new work as an issue.
|
||
|
||
Getting-started guide for the `Sql` driver — an Equipment-kind driver that polls rows out of a
|
||
relational database and publishes them as OPC UA variables. It is **read-only unless writes are
|
||
explicitly enabled** at both the driver and the tag (see [Writes](#writes-opt-in-753-781)). For the design rationale read
|
||
[`docs/plans/2026-07-15-sql-poll-driver-design.md`](../plans/2026-07-15-sql-poll-driver-design.md);
|
||
for the build-vs-plan record read
|
||
[`docs/plans/2026-07-24-sql-poll-driver.md`](../plans/2026-07-24-sql-poll-driver.md).
|
||
|
||
## What it talks to
|
||
|
||
An ordinary SQL database — typically a MES / LIMS / historian staging table that some other system
|
||
already writes to. There is no protocol here and nothing to "connect" to a machine: the driver runs
|
||
`SELECT`s on a timer and turns cells into tag values.
|
||
|
||
**All five providers construct**: `SqlServer` (`Microsoft.Data.SqlClient`), `Postgres`
|
||
(`Npgsql`), `Oracle` (ODP.NET managed — **Oracle 12.2 or later**, the floor the
|
||
`FETCH FIRST` paging needs), `MySql` (`MySqlConnector`, MIT, fully managed — MySQL 8 /
|
||
MariaDB) and `Odbc` (`System.Data.Odbc`, see below). An undeclared provider value fails the driver
|
||
rather than silently falling back. The provider → dialect
|
||
map is one shared table (`SqlDialects`) that the factory, the Test Connect probe and the browse
|
||
picker all read, so the three can no longer disagree about what a build supports.
|
||
|
||
MySQL specifics: a MySQL **database is the schema** (`schema.table` = `database.table`; the default
|
||
schema is the connection's `Database`), identifiers are backtick-quoted (64-character ceiling, no
|
||
trailing space), and table names are case-sensitive on Linux — the catalog gate's spelling
|
||
substitution is what lets `PLANT.TAG_VALUES` bind to `plant.tag_values`. `TINYINT(1)` reads as a
|
||
boolean under MySqlConnector's default `TreatTinyAsBoolean`; `BIT(n)`, `TIME`, `JSON` and binary
|
||
types infer as `String`. Live gate: `Docker/mysql/docker-compose.yml` + `MySqlReadTests`
|
||
(`SQL_MYSQL_ENDPOINT` / `SQL_MYSQL_PASSWORD`).
|
||
|
||
ODBC specifics: `OdbcDialect` is the **ANSI SQL:2008 profile, and only that** — double-quoted
|
||
identifiers, positional `?` parameters, the standard `INFORMATION_SCHEMA` views and `FETCH FIRST n
|
||
ROWS ONLY`. The scope is deliberately narrow (owner decision 2026-11-09, lmxopcua#781): there are no
|
||
per-backend ODBC dialect variants and no `SQLTables`/`SQLColumns` catalog path. ODBC abstracts the
|
||
wire, not the SQL, so the backend behind the connection string must accept all four: PostgreSQL
|
||
(psqlODBC — the live gate), Snowflake and similar do. **SQL Server** (needs `OFFSET … FETCH` after an
|
||
`ORDER BY`) and **MySQL / MariaDB** (no `FETCH FIRST`) do not — use the native `SqlServer` / `MySql`
|
||
provider. **DB2** (no `INFORMATION_SCHEMA`) is not supported through Odbc at all. Writes over Odbc
|
||
are **Update and Insert only** — see [Writes](#writes-opt-in-753-781).
|
||
|
||
Parameters bind in order, so the planner, the catalog loader and the browse session all add them in
|
||
marker order, and a named query's `params` bind in authored order (the factory refuses a statement
|
||
whose `?` count differs). Live gate: `Docker/odbc/run-odbc-gate.sh` runs `OdbcReadTests` (reads,
|
||
named queries, browse, and Update/Insert writes) in a throwaway Linux container with unixODBC +
|
||
psqlODBC against a throwaway PostgreSQL; on macOS the suite is unit-tested only (`OdbcDialectTests`)
|
||
because no vendor ODBC driver is installed.
|
||
|
||
> **Ops note — Odbc is the one provider that is not fully managed.** `System.Data.Odbc` is only the
|
||
> ADO.NET shim; the work is done by native code that must be installed on **every host that runs an
|
||
> Odbc driver instance** (every node of its cluster, since either can become Primary) **and every
|
||
> AdminUI host** (the browse picker and Test Connect open the connection from the AdminUI process):
|
||
>
|
||
> - **Linux:** the driver manager (`unixodbc`) plus the vendor's driver package (e.g.
|
||
> `odbc-postgresql`), registered in `odbcinst.ini`. Check with `odbcinst -q -d` — the name it prints
|
||
> is the `Driver={…}` value the connection string needs.
|
||
> - **Windows:** the driver manager ships with the OS; install the vendor's **64-bit** ODBC driver
|
||
> (the server is x64) and confirm it in *ODBC Data Sources (64-bit)* → *Drivers*.
|
||
> - Prefer a **DSN-less** connection string (`Driver={PostgreSQL Unicode};Server=…;Database=…`) in
|
||
> the secret store: a DSN is per-host configuration that every node and AdminUI host would also
|
||
> need, identically.
|
||
>
|
||
> A host missing either piece fails the driver's Initialize (and the AdminUI browse) with the driver
|
||
> manager's "data source name not found / no default driver" error — a host-provisioning fault, not a
|
||
> tag fault.
|
||
|
||
Two provider-specific things are worth knowing before you author against them:
|
||
|
||
- **Identifier case folds, and the catalog gate absorbs it.** Postgres lower-cases and Oracle
|
||
upper-cases unquoted identifiers. Every authored identifier is resolved to the catalog's own
|
||
spelling, so an authored `widgets` binds to Oracle's `WIDGETS` rather than failing.
|
||
- **Oracle `NUMBER` infers as `Float64`.** Column metadata does not carry precision/scale to the
|
||
type map, so an integer-shaped `NUMBER` still infers double. Use the per-tag `type` override
|
||
(see [Optional per-tag fields](#optional-per-tag-fields)) when the OPC UA type matters.
|
||
|
||
**Read-only by default.** `allowWrites` is off unless authored `true`, and even then only a tag whose
|
||
own TagConfig says `"writable": true` can be written — see [Writes](#writes-opt-in-753-781).
|
||
|
||
## Tag models
|
||
|
||
A tag says how to find its value. Three models ship: `KeyValue`, `WideRow` and the named-query `Query`.
|
||
|
||
### `KeyValue` — the EAV shape
|
||
|
||
One row per tag. The driver matches `keyColumn = keyValue` and reads `valueColumn`.
|
||
|
||
```json
|
||
{
|
||
"model": "KeyValue",
|
||
"table": "dbo.TagValues",
|
||
"keyColumn": "TagName",
|
||
"keyValue": "Line1.Oven.Temp",
|
||
"valueColumn": "Value",
|
||
"timestampColumn": "UpdatedUtc"
|
||
}
|
||
```
|
||
|
||
### `WideRow` — one row, many signals
|
||
|
||
The tag names its `columnName`; the row is picked either by a `whereColumn`/`whereValue` pair or by
|
||
newest-timestamp.
|
||
|
||
⚠️ **The selector keys live inside a nested `rowSelector` object**, not at the top level. A blob that
|
||
spells `whereColumn` / `whereValue` / `topByTimestamp` beside `columnName` carries *no* row selector
|
||
as far as `SqlEquipmentTagParser` is concerned: the tag is refused, the driver skips it, and it reads
|
||
`BadNodeIdUnknown` forever. The typed editor, the browse-commit mapper and the deploy gate all use
|
||
the nested form.
|
||
|
||
```json
|
||
{
|
||
"model": "WideRow",
|
||
"table": "dbo.LineStatus",
|
||
"columnName": "OvenTemp",
|
||
"rowSelector": { "whereColumn": "LineId", "whereValue": "L1" }
|
||
}
|
||
```
|
||
|
||
```json
|
||
{
|
||
"model": "WideRow",
|
||
"table": "dbo.Readings",
|
||
"columnName": "OvenTemp",
|
||
"rowSelector": { "topByTimestamp": "CapturedUtc" }
|
||
}
|
||
```
|
||
|
||
⚠️ **`topByTimestamp` needs an ordering column that is unique per row.** It is executed as an
|
||
`ORDER BY <column> DESC` with the dialect's single-row limit, so when the column ties the server
|
||
returns an arbitrary row among the tied ones — and the driver cannot warn about it, because the other
|
||
rows were discarded before the result set reached it (the `whereColumn`/`whereValue` form *does*
|
||
warn, on `Rows.Count > 1`). A `date`-typed column ties for every row written on the same day, and
|
||
browse-commit will nominate one. Prefer a `datetime2`/sequence column that is unique per row, or
|
||
point the tag at a latest-row view.
|
||
|
||
Both value-bearing fields — `whereValue` here and `keyValue` on the `KeyValue` model — may be
|
||
authored as a JSON string, number or boolean, and are captured as text. They are **bound as
|
||
parameters** either way, and matched back as text (the key cell is stringified), so a key column
|
||
that holds an INTEGER station id can be authored either `"7"` or `7`. A JSON object, array or `null`
|
||
is not a value and is refused. On **PostgreSQL** a string bind is sent untyped (`NpgsqlDbType.Unknown`,
|
||
`PostgresDialect.ConfigureParameter`, STAB-24) so the server infers its type from the compared column;
|
||
bound as `text`, a `"7"` against an `integer` key fails with `operator does not exist: integer = text`.
|
||
|
||
⚠️ **Known limitation — key normalisation.** Row slice-back matches the returned key cell by its *text* form
|
||
(`Convert.ToString`, ordinal), while the server now matches by the column's inferred type. An authored key the
|
||
server treats as equal but whose text differs therefore selects a row that the driver then cannot find, and the
|
||
tag reads **no-row**: `"007"` against an `integer` column (returns `7`), `"1.50"` against `numeric` (returns
|
||
`1.5`), and an upper-case GUID against a `uuid` column (returned lower-case). Author keys in the column's
|
||
canonical text form. SQL Server behaves the same way.
|
||
(They were read asymmetrically until 2026-08-10, so a numerically
|
||
authored `keyValue` refused the tag while the identically-bound `whereValue` was served — and the
|
||
deploy gate reported the key as *missing* when it was right there in the blob.)
|
||
|
||
### `Query` — a named read-only query (#753)
|
||
|
||
For a join, a view you cannot create, or a computed projection. The **driver config** declares the
|
||
statement under `queries`; the **tag** names the query, the result column it reads, and optionally the
|
||
row:
|
||
|
||
```json
|
||
{ "queries": {
|
||
"activeBatch": {
|
||
"sql": "SELECT line, speed, temp, ts FROM dbo.LineStatus WHERE active = @active",
|
||
"params": { "active": true },
|
||
"maxRows": 100
|
||
} } }
|
||
```
|
||
|
||
```json
|
||
{ "model": "Query", "query": "activeBatch", "column": "speed",
|
||
"rowKey": { "column": "line", "value": "Line1" }, "timestampColumn": "ts", "type": "Float64" }
|
||
```
|
||
|
||
- **One execution per query per poll**, however many tags read it.
|
||
- **`rowKey` picks the row** (matched as text against that result column, like `keyValue`). Without a
|
||
`rowKey` the query must return exactly one row. No matching row ⇒ `BadNoData`; two matching rows (or
|
||
two rows with no `rowKey`) ⇒ `BadConfigurationError` — never "last row wins".
|
||
- **Result column names** (`column`, `rowKey.column`, `timestampColumn`) match exactly first, then by a
|
||
unique case-insensitive match, so Postgres' lower-cased and Oracle's upper-cased aliases still bind. A
|
||
column the query does not return ⇒ `BadConfigurationError` plus a rate-limited warning listing what it
|
||
did return.
|
||
- **`maxRows`** (default 1000, at most 100 000) is enforced client-side: the reader stops one row past
|
||
it and the whole query publishes `BadConfigurationError`. A statement cannot be wrapped in a provider
|
||
row limit portably, so narrow the `WHERE` rather than relying on the cap.
|
||
- **Parameters are always bound.** Keys are bare names (`active`; a leading `@`/`:` is tolerated);
|
||
values are JSON scalars or `null`. Write the provider's own marker in the SQL: `@active` on SqlServer,
|
||
Postgres and MySql, `:p_active` on Oracle, a positional `?` on Odbc (bound in authored order). The
|
||
factory refuses a declared parameter whose marker the statement never uses.
|
||
- **Read-only, twice over.** `SqlReadOnlyStatement` refuses at construction (and in the AdminUI form)
|
||
anything but a single `SELECT`/`WITH` statement, and any of `INSERT UPDATE DELETE MERGE UPSERT DROP
|
||
ALTER CREATE TRUNCATE EXEC EXECUTE CALL GRANT REVOKE DENY INTO COPY LOCK RENAME` outside a literal or
|
||
comment — so `SELECT … INTO`, `FOR UPDATE` and a data-modifying CTE are refused too. Behind that guard
|
||
the statement runs **inside a transaction the driver always rolls back**, so a side effect the guard
|
||
cannot see (a function that writes) is never committed. A non-transactional side effect (an e-mail
|
||
procedure, a sequence, a call out to another system) is outside both — which is why a read-only
|
||
driver **must** connect with a `SELECT`-only login (see [Database login](#database-login-required)).
|
||
- A tag naming a query the driver config does not declare is **skipped** at Initialize (it reads
|
||
`BadNodeIdUnknown`, and `/hosts` names the missing query). One unusable `queries` entry **fails the
|
||
driver** instead, because every tag that names it would otherwise go dark unexplained.
|
||
- Query tags read no table, so they need **no catalog gate** — a driver serving only named queries skips
|
||
the catalog load entirely. Nothing from a Query tag is ever quoted into SQL.
|
||
|
||
## Optional per-tag fields
|
||
|
||
| Field | Effect |
|
||
|---|---|
|
||
| `timestampColumn` | Source timestamp for the value. Absent ⇒ the driver stamps the poll time. |
|
||
| `type` | Explicit OPC UA type override. Absent ⇒ inferred from the result set's column metadata. A write is coerced to it. |
|
||
| `writable` | The tag's write opt-in — a JSON **boolean**; absent ⇒ read-only. Anything else (`"true"`, `1`) refuses the tag. Effective only with the driver's `allowWrites`. |
|
||
| `write` | Optional: how a write is executed — `Update` (default), `Insert`, `Upsert` or `Procedure`, plus the timestamp stamp. See [Writes](#writes-opt-in-753-781). A malformed block refuses the whole tag and blocks deploy. |
|
||
|
||
There is **no per-tag poll-interval field**. The interval is the subscriber's, floored by
|
||
`PollGroupEngine.DefaultMinInterval`, falling back to the instance's `defaultPollInterval` when the
|
||
subscriber names none. (A `pollIntervalMs` per-tag override was documented here for a while and never
|
||
existed in the parser — nothing reads it.)
|
||
|
||
## Driver config
|
||
|
||
```json
|
||
{
|
||
"provider": "SqlServer",
|
||
"connectionStringRef": "MesStaging",
|
||
"defaultPollInterval": "00:00:05",
|
||
"operationTimeout": "00:00:15",
|
||
"commandTimeout": "00:00:10",
|
||
"maxConcurrentGroups": 4,
|
||
"nullIsBad": false
|
||
}
|
||
```
|
||
|
||
| Key | Default | Notes |
|
||
|---|---|---|
|
||
| `provider` | `SqlServer` | `SqlServer`, `Postgres`, `Oracle`, `MySql` or `Odbc`. |
|
||
| `connectionStringRef` | — | **A name, never a connection string.** See below. |
|
||
| `defaultPollInterval` | `00:00:05` | Applied to any group without its own interval. |
|
||
| `operationTimeout` | `00:00:15` | Client-side wall-clock deadline; a breach surfaces `BadTimeout`. |
|
||
| `commandTimeout` | `00:00:10` | ADO.NET server-side backstop. **Must be less than `operationTimeout`** — it is the backstop, not the deadline. |
|
||
| `maxConcurrentGroups` | `4` | Connection-pool guard: cap on group queries in flight. |
|
||
| `nullIsBad` | `false` | `false` ⇒ a NULL cell publishes **Uncertain**; `true` ⇒ **Bad**. |
|
||
| `allowWrites` | `false` | Driver-level write switch. Off ⇒ every write is `BadNotWritable`. See [Writes](#writes-opt-in-753-781). |
|
||
| `queries` | — | Named read-only queries for `Query` tags — see [`Query`](#query--a-named-read-only-query-753). Edited as JSON in the driver form, which applies the factory's own rules. |
|
||
|
||
### Credentials never enter the deployment artifact
|
||
|
||
`connectionStringRef` is a **name** resolved at Initialize from the environment / secret store — e.g.
|
||
`Sql__ConnectionStrings__MesStaging`. There is no field that holds a connection string, so a sealed
|
||
deployment artifact cannot carry database credentials even by accident.
|
||
|
||
A useful side effect of the in-place reinit added for [#516](#): a **rotated credential is picked up
|
||
on the next config apply without a process restart**, because the driver re-resolves the reference
|
||
when it re-parses.
|
||
|
||
## How polling works
|
||
|
||
Tags are **grouped by source** and one query is issued per group, not per tag — `SqlGroupPlanner`
|
||
emits a `SqlQueryPlan` per distinct group key. Grouping is deterministic (groups appear in the input
|
||
order of their first member, members keep their input order, parameters keep first-appearance order),
|
||
so an unchanged configuration produces byte-identical SQL.
|
||
|
||
**Every key-value and where-pair query is row-capped (R06:S-1).** A key-value plan reads at most twice as
|
||
many rows as it has distinct keys and asks for one more (`TOP (2n+1)` / `LIMIT 2n+1` / `FETCH FIRST 2n+1 ROWS
|
||
ONLY`); a where-pair wide-row plan asks for two. The reader stops there even if a provider ignores the limit.
|
||
|
||
- **A duplicated key is refused per key.** When a key comes back more than once, only the tags on THAT key
|
||
publish `BadConfigurationError`; every other key in the group reads normally, whatever else is present or
|
||
absent. (Before this, a duplicate was last-row-wins Good.)
|
||
- **A gross excess fails the group.** Receiving the extra row past the cap means a history/events table or a
|
||
non-unique where column, which cannot be attributed because the rest is unread, so every member of the group
|
||
publishes `BadConfigurationError`. For a where-pair any second row is this case.
|
||
- Each case logs a per-plan rate-limited warning naming the table and the selector (or an example key).
|
||
`topByTimestamp` keeps its single-row limit.
|
||
- **Upgrade note:** a deployment whose key-value source carries stray duplicate rows, or whose where column is
|
||
not unique, will see those tags go Bad where they used to read an arbitrary row at Good.
|
||
|
||
Every such refusal, like the other refusals (absent row, NULL cell, uncoercible cell), is the database
|
||
answering: it advances `LastSuccessfulRead` and restores `Healthy` after a comms fault.
|
||
|
||
⚠️ **Group keys are case-sensitive on identifiers.** Two tags authored `dbo.T` and `DBO.T` plan as
|
||
**two** groups — one extra round-trip, not a correctness problem, but worth knowing if a group count
|
||
looks higher than expected.
|
||
|
||
## SQL injection: how the driver is safe
|
||
|
||
The two classes of authored string are handled differently, and deliberately so:
|
||
|
||
- **Value-bearing** fields (`keyValue`, `whereValue`) are bound as `DbParameter`s. They are never
|
||
concatenated into command text.
|
||
- **Identifier-bearing** fields (`table`, `keyColumn`, `valueColumn`, `timestampColumn`,
|
||
`columnName`, `whereColumn`, `topByTimestamp`) cannot be parameterised in SQL. They are validated
|
||
against the live catalog — **`INFORMATION_SCHEMA`** on SQL Server and Postgres, the **`ALL_*`**
|
||
views (`ALL_USERS` / `ALL_TABLES` ∪ `ALL_VIEWS` / `ALL_TAB_COLUMNS`) on Oracle (MySQL also uses
|
||
`information_schema`) — and
|
||
dialect-quoted before they reach a command.
|
||
|
||
The parser deliberately does **not** sanitise either kind — sanitising a value would corrupt
|
||
legitimate data, and sanitising an identifier would mask an authoring error the catalog check reports
|
||
properly.
|
||
|
||
⚠️ **An unreadable catalog FAULTS the driver rather than rejecting every tag** (Gitea #496). This is
|
||
the safer failure: "I cannot verify these identifiers" and "these identifiers are invalid" are
|
||
different conditions, and reporting the second when the first is true would send an operator hunting a
|
||
configuration bug that does not exist. The catalog gate also substitutes the catalog's own spelling of
|
||
an identifier, so a case difference between the authored blob and the database resolves rather than
|
||
failing.
|
||
|
||
## Browse
|
||
|
||
`SqlDriverBrowser` browses the live catalog — schema → table → column — so the `/raw` browse picker
|
||
can commit columns as tags. A committed leaf carries **schema, table and column**: a column name alone
|
||
cannot address a SQL tag, and an earlier version that emitted only the column produced tags whose
|
||
typed editor opened empty.
|
||
|
||
Browse works against every constructed provider; the catalog query is the dialect's own
|
||
(`INFORMATION_SCHEMA` on SQL Server, Postgres and MySQL — with `pg_catalog` / `information_schema`
|
||
and MySQL's `mysql` / `performance_schema` / `sys` excluded — and the `ALL_*` views on Oracle).
|
||
|
||
### The committed row selector (Gitea #552)
|
||
|
||
A browse-committed column becomes a **WideRow** tag, and a wide-row tag must say *which row* it reads
|
||
— the table and column alone are not a binding. The browse session therefore states a fourth address
|
||
field alongside schema/table/column: the table's preferred **date/time-typed** column, which the
|
||
commit writes as `rowSelector.topByTimestamp` ("read the newest row"). Before this the commit emitted
|
||
a selector-less blob that `SqlEquipmentTagParser` and the authoring gate both refuse, so no
|
||
browse-committed Sql tag was ever readable.
|
||
|
||
Three properties of the choice are deliberate:
|
||
|
||
- **Candidacy is by declared TYPE, never by column name.** The session asks
|
||
`ISqlDialect.MapColumnType` for `DriverDataType.DateTime` — the same metadata the browse tree already
|
||
renders per column. A name heuristic (`*_ts`, `*time*`) would happily order a wide-row read by a
|
||
*string* column called `timestamp`, binding the tag to whichever row sorts last lexically.
|
||
- **Candidates are RANKED: a time-bearing type beats a date-only one.** `MapColumnType` reports
|
||
`DriverDataType.DateTime` for SQL Server's and PostgreSQL's `date` too, and ordering a single-row read
|
||
by a date-only column ties for *every row written on the same day* — "the newest row" quietly becomes
|
||
storage order among the ties, and the reader cannot warn about it because the server has already
|
||
discarded the others. So a `datetime2`/`timestamptz`/… column wins over a `date` one wherever both
|
||
exist, regardless of declaration order. A date-only column is still stated when it is **all** the table
|
||
has, so such a table stays bindable by browse rather than becoming un-committable.
|
||
**Oracle's `DATE` is exempt** — it carries a time-of-day component (which is why `OracleDialect` maps
|
||
it faithfully rather than as a widening), so it is not demoted there.
|
||
- **The tie-break within a rank is first in catalog order** (every dialect's column query orders by
|
||
ordinal position), so the pick is stable across calls, connections and servers. On a
|
||
`CreatedUtc`/`ModifiedUtc` pair it takes the earlier-declared one. It is a **visible, editable
|
||
default**: the typed editor shows the selector and the operator can change it — to the other
|
||
timestamp, or to a `whereColumn`/`whereValue` pair entirely.
|
||
|
||
**A table with no date/time column cannot be bound by browse, and the commit says so.** The remaining
|
||
selector is a `whereColumn`/`whereValue` pair whose value is *data*, not schema, so nothing in a
|
||
catalog can supply it. `RawBrowseCommitMapper.DescribeUncommittableLeaf` refuses that leaf by name,
|
||
names the table, and points at the tag editor; no selector is invented, and the mapper's blob stays
|
||
selector-less rather than dropping to a generic `{"address": …}` the typed editor cannot read.
|
||
|
||
## Writes (opt-in, #753, #781)
|
||
|
||
A write reaches the database only when **every** gate holds, outermost first:
|
||
|
||
1. the OPC UA node is Writable (the Tag's own flag) and the caller holds `WriteOperate` — the server;
|
||
2. the driver config says `"allowWrites": true` — otherwise `BadNotWritable`;
|
||
3. the tag's TagConfig says `"writable": true` — otherwise `BadNotWritable`;
|
||
4. the tag's write mode has a target on its model (table below) — otherwise `BadNotWritable`;
|
||
5. the provider serves the mode — otherwise deploy refuses it (`SqlWriteModeUnsupported`) and, at
|
||
runtime, `BadNotSupported`.
|
||
|
||
There is **no authored write SQL.** A tag's optional `write` block picks one of four statement
|
||
shapes the driver builds itself from the tag's own catalog-validated identifiers:
|
||
|
||
```json
|
||
{ "model": "KeyValue", "table": "plant.setpoints", "keyColumn": "tag_name", "keyValue": "Line1.Speed",
|
||
"valueColumn": "sp_value", "timestampColumn": "updated_at", "type": "Float64", "writable": true,
|
||
"write": { "mode": "Upsert", "stampTimestamp": true } }
|
||
```
|
||
|
||
| `mode` | Statement | Models | Rows that count as success |
|
||
|---|---|---|---|
|
||
| `Update` *(default — also what a writable tag with no `write` block does)* | `UPDATE t SET value [, stamp] WHERE key = @k` | KeyValue; WideRow with `whereColumn`/`whereValue` | exactly 1 |
|
||
| `Insert` | `INSERT INTO t (key, value [, stamp]) VALUES (…)` — no key column for a WideRow `topByTimestamp` tag (an append) | KeyValue; WideRow (either selector) | exactly 1 |
|
||
| `Upsert` | SQL Server `MERGE … WITH (HOLDLOCK)`; PostgreSQL `INSERT … ON CONFLICT (key) DO UPDATE`; MySQL `INSERT … ON DUPLICATE KEY UPDATE`; Oracle `MERGE … USING (SELECT 1 FROM dual)` | KeyValue; WideRow with a where-pair | 1 (MySQL: 1 or 2) |
|
||
| `Procedure` | SQL Server `EXEC p @value = @v, …`; PostgreSQL `CALL p(value => @v, …)`; MySQL `CALL p(@v, …)` (**positional**); Oracle `BEGIN p(value => :p_v, …); END;` | any, including `Query` | the call did not throw |
|
||
|
||
`write` keys — anything else refuses the tag:
|
||
|
||
| Key | Modes | Meaning |
|
||
|---|---|---|
|
||
| `mode` | all | `Update` \| `Insert` \| `Upsert` \| `Procedure`; absent ⇒ `Update`. |
|
||
| `stampTimestamp` | Update / Insert / Upsert | `true` ⇒ also set the tag's `timestampColumn` (a WideRow `topByTimestamp` tag: that column) to the write time, **UTC, from the driver host's clock**. Requires such a column. Default `false` — the timestamp column is then untouched and the next read keeps the row's old timestamp unless the database maintains it (a trigger, `ON UPDATE CURRENT_TIMESTAMP`). |
|
||
| `procedure` | Procedure (required) | `procedure` or `schema.procedure` — a **stored procedure**, resolved against the routine catalog like a table (case folds; an unknown, non-executable or package-member name rejects the tag). |
|
||
| `valueParam` | Procedure | The procedure parameter the value binds to. Default `value`. |
|
||
| `keyParam` | Procedure | Optional: the parameter the tag's key binds to (`keyValue`, `whereValue` or a Query tag's `rowKey.value`). |
|
||
| `timestampParam` | Procedure | Optional: the parameter the write time (UTC) binds to. |
|
||
|
||
Parameter names are plain identifiers (letter/underscore, then letters/digits/underscores; a leading
|
||
`@`/`:` is stripped) — they reach the statement unquoted, in named notation. **MySQL has no named
|
||
notation**: its arguments are passed positionally in the fixed order *value, key, timestamp* (those
|
||
configured), so declare the procedure's parameters in that order. **PostgreSQL** resolves procedure
|
||
arguments by implicit cast, so declare a numeric value parameter `double precision` (a `Float64`
|
||
binds as that) and a timestamp parameter `timestamptz`.
|
||
|
||
**Provider support** (one table, `SqlWriteSupport`, read by the driver and the deploy gate):
|
||
|
||
| | Update | Insert | Upsert | Procedure |
|
||
|---|---|---|---|---|
|
||
| SqlServer, Postgres, MySql, Oracle | ✓ | ✓ | ✓ | ✓ |
|
||
| Odbc | ✓ | ✓ | — | — |
|
||
|
||
Odbc has no upsert every ODBC backend accepts (and positional `?` markers cannot repeat, which every
|
||
single-statement upsert needs) and no portable routine catalog or calling convention; use the
|
||
backend's native provider for those modes.
|
||
|
||
**Upsert needs a unique key on the key column.** PostgreSQL refuses `ON CONFLICT (key)` without a
|
||
unique constraint or index on exactly that column; MySQL's `ON DUPLICATE KEY` fires on *any* unique
|
||
key of the table, so the key column should be the primary key and no other unique key should be able
|
||
to collide; SQL Server and Oracle `MERGE` work without one, but then a key matching two rows updates
|
||
both and is rolled back (`BadConfigurationError`).
|
||
|
||
### Outcomes
|
||
|
||
| Outcome | Status |
|
||
|---|---|
|
||
| Committed | `Good` |
|
||
| `Update` (or a non-MySQL `Upsert`) touched no row — the key is absent; nothing is inserted | `BadNoData` |
|
||
| More rows than the mode allows — the key is not unique; rolled back | `BadConfigurationError` |
|
||
| The provider could not report a row count; rolled back | `BadNotSupported` |
|
||
| Unique / primary-key violation — e.g. an `Insert` of a key that exists (SQLSTATE 23505, SQL Server 2627/2601, MySQL 1062, ORA-00001) | `BadEntryExists` |
|
||
| Another integrity rule — NOT NULL, foreign key, CHECK (SQLSTATE class 23, SQL Server 515/547, ORA-01400/02290/02291) — or a data exception: overflow, truncation, an unconvertible value (SQLSTATE class 22, SQL Server 8115/8152/245, ORA-01438/12899/01722) | `BadOutOfRange` |
|
||
| Any other provider error after the connection opened — **a missing grant included**, a missing object, a broken link | `BadCommunicationError` |
|
||
| The value will not coerce to the tag's `type` | `BadTypeMismatch` |
|
||
| `operationTimeout` elapsed | `BadTimeout` |
|
||
|
||
Nothing is committed on any non-`Good` outcome. Writes are **never retried**
|
||
(`WriteIdempotent = false` — an `Insert` or a procedure is not idempotent, and an `Update` is only
|
||
idempotent until something else writes the row), and a failed write reverts the node to its prior
|
||
value. Writes never change driver health. Discovery advertises `Operate` only for a tag where gates
|
||
2–5 hold, `ViewOnly` otherwise.
|
||
|
||
### Safety
|
||
|
||
- **Identifiers are the catalog gate's.** A write resolves against the gate-accepted definition, so
|
||
every table, column and procedure name in the statement was read back out of the catalog
|
||
(`INFORMATION_SCHEMA.ROUTINES` / Oracle `ALL_PROCEDURES` for procedures) and dialect-quoted. A
|
||
gate-rejected tag answers `BadNodeIdUnknown`.
|
||
- **Values are always bound** — the written value, the key and the stamp. Nothing an OPC UA client
|
||
sends reaches command text.
|
||
- **Deploy checks the block with the driver's own parser** (`SqlTagDefinitionFactory.DescribeWriteBlock`):
|
||
a malformed block is the blocking error `SqlWriteBlockInvalid`, a mode the instance's provider does
|
||
not serve is `SqlWriteModeUnsupported`. The AdminUI tag editor shows the same parser message.
|
||
- Live gate: `SqlWriteModesLiveTests` runs the same suite (Update+stamp, absent key, Insert and its
|
||
duplicate, NOT NULL, Upsert insert-then-update, Procedure, unknown procedure) against SQL Server,
|
||
PostgreSQL, MySQL and Oracle; `OdbcReadTests` covers Odbc's Update/Insert.
|
||
|
||
## Database login (required)
|
||
|
||
The driver's own guards are not a substitute for the database's grants:
|
||
|
||
- **A read-only driver (`allowWrites` off) must connect with a `SELECT`-only login** — `SELECT` on
|
||
the tables and views its tags and named queries read, and nothing else. The named-query guard is a
|
||
keyword check plus an always-rolled-back transaction; neither can stop a **non-transactional** side
|
||
effect — a sequence `nextval`, a function that sends mail or calls out, autonomous transactions,
|
||
DDL that auto-commits on some backends. Only the grant can.
|
||
- **A writable driver must connect with a login limited to its write targets** — `SELECT` as above,
|
||
plus exactly what its modes need on exactly the tables / procedures its writable tags name:
|
||
`UPDATE` (Update), `INSERT` (Insert), `INSERT` + `UPDATE` (Upsert; SQL Server's `MERGE` also needs
|
||
`SELECT` on the target), `EXECUTE` on the named procedures (Procedure). Never `DELETE`, DDL or
|
||
ownership. The live gates provision exactly this (`Docker/*/seed-*.sql`) and prove a write to a
|
||
table outside the grant is refused by the server (`BadCommunicationError`) even with every driver
|
||
gate open.
|
||
- A procedure runs with *its* rights (SQL Server `EXECUTE AS`, PostgreSQL `SECURITY DEFINER`, MySQL
|
||
`SQL SECURITY DEFINER`, Oracle definer's rights by default) — review what it can touch, since the
|
||
login's narrow grant does not bound it.
|
||
|
||
## Authoring in the AdminUI
|
||
|
||
- **Driver config** — `/raw` → driver node → *Configure*. Sql is a **single-connection** driver: the
|
||
connection lives on the driver, not on a device, and the modal says so.
|
||
- **Tags** — the typed `SqlTagConfigModel` editor, or CSV import (Sql has typed CSV columns), or
|
||
browse-commit from the picker.
|
||
|
||
## Capabilities
|
||
|
||
`IDriver`, `ITagDiscovery`, `IReadable`, `IWritable` (opt-in, above), `ISubscribable`,
|
||
`IHostConnectivityProbe`.
|
||
|
||
No `IRediscoverable` — a SQL schema change is not signalled, and
|
||
`DiscoverAsync` echoes authored config rather than enumerating the database, so this driver is
|
||
deliberately excluded from any discovery-diff machinery.
|
||
|
||
## See also
|
||
|
||
- [`docs/Raw.md`](../Raw.md) — the `/raw` authoring tree
|
||
- [`docs/drivers/README.md`](README.md) — the driver index
|
||
- Gitea **#496** / **#497** / **#498** — the shipped follow-ups
|