lmxopcua/docs/plans/2026-08-05-sql-postgres-oracle-design.md

# Sql driver: Postgres + Oracle providers (managed drivers only) — design

**Status:** DONE 2026-08-05 — all eight tasks delivered and reviewed. Suites: Driver.Sql.Tests
355/355, Browser.Tests 75/75, IntegrationTests 33 skip-clean offline; **both new integration suites
also ran LIVE during development** against real local containers (postgres:17-alpine 5/5,
gvenzl/oracle-free:23-slim 6/6), which surfaced and fixed a provider-fatal defect no offline test
could see: **ORA-01745** — Oracle rejects a bind variable named after a reserved word (`:table`),
so every Oracle catalog load failed until OracleDialect namespaced all binds with a `p_` prefix
(`ec90b7c4`, pinned by a never-bare-base-name invariant theory). Merged to master `d54f3c83`.

**Shared-host live run PASSED (2026-08-05, Task 8).** Both fixtures deployed to the shared docker
host (`/opt/otopcua-sql-pg` port 15432, `/opt/otopcua-sql-oracle` port 11521, both `project:
lmxopcua`-labelled, seeds exited 0) and left running; the env-gated suites ran against them from
this repo: **11/11 passed, 0 skipped** (5 Postgres + 6 Oracle — initialize/catalog, KeyValue,
WideRow `topByTimestamp`, catalog case-folding, browse walk). Env recipe used:
`SQL_PG_ENDPOINT=10.100.0.35:15432 SQL_PG_USER=otopcua SQL_PG_PASSWORD=<compose-up value>` +
`SQL_ORACLE_ENDPOINT=10.100.0.35:11521 SQL_ORACLE_USER=otopcua SQL_ORACLE_PASSWORD=<compose-up
value>` (service defaults `FREEPDB1`).

**§4.3's chrome/AdminUI gate PASSED (2026-08-05) — and it found a real defect.** Run against a
docker-dev rig rebuilt from master (both centrals on one image, provider assemblies confirmed
present in `/app`) with `Sql__ConnectionStrings__FixturePg` / `__FixtureOracle` added to central-1
**and** central-2 (`:9200` round-robins, so a browse can land on either node).

⚠️ **The gate's first finding: Test Connect was broken for BOTH new providers.**
`SqlDriverProbe.DefaultDialectFor` was never widened — it still constructed `SqlServerDialect`
only and threw `InvalidOperationException` for Postgres/Oracle, so a config that **deployed and
browsed fine failed Test Connect**. There are *three* dialect-selection sites and Task 4's `Files:`
block named two (`SqlDriverFactoryExtensions`, `SqlDriverBrowser`) plus the enum. No test caught
it because **every case in `SqlDriverProbeTests` builds its probe through `ForTest(...)`**, which
injects `_ => dialect` and never reaches the production switch — the parameterless constructor had
zero coverage. Fixed in `75a057f2`: switch widened arm-for-arm with the factory, plus an internal
seam covering the gate directly, a reserved-provider (MySql/Odbc) falsification control, and an
end-to-end production-constructor case. Red-first confirmed (3 failures) before the widening;
Driver.Sql.Tests then 361/361.

Legs, after the fix:

1. **Provider dropdown authoring ✅** — dropdown offers SqlServer/Postgres/Oracle; both saved and
   persisted as `{"provider":"Postgres",…}` / `{"provider":"Oracle",…}`, i.e. serialized by **name**,
   not ordinal (the driver-enum-serialization defect class does not recur here).
2. **Test-connect ✅** — Postgres `OK · 174 ms`, Oracle `OK · 386 ms`. **Negative control:** the
   Oracle driver cross-wired to the Postgres connection string returns
   `FAILED · … (ARGUMENTEXCEPTION)` — the provider itself rejecting the string, and *not*
   `InvalidOperationException`, which proves the dialect was constructed and the green came from a
   real round trip rather than an unconditional OK.
3. **`/raw` browse + browse-commit ✅** — Postgres browse discovered schema `sqlpoll` → `tag_values`
   / `latest_status` → per-column LEAFs; **the schema is owned by `sqlpoll_owner` while the poller
   connects as `otopcua`**, so this is the live confirmation that `PostgresDialect.ListSchemasSql`
   sources from `information_schema.tables` rather than `.schemata`. Oracle browse listed the PDB's
   schemas → `OTOPCUA` → `TAG_VALUES`/`LATEST_STATUS` → LEAFs. One leaf committed per provider,
   both persisting schema-qualified with the right type:
   `{"model":"WideRow","table":"sqlpoll.tag_values","columnName":"num_value"}` [Double] and
   `{"model":"WideRow","table":"OTOPCUA.TAG_VALUES","columnName":"NUM_VALUE"}` [Double].

## 1. Problem

`docs/drivers/Sql.md:15-18`: v1 constructs SQL Server only; `SqlProvider` declares `Postgres`,
`MySql`, `Odbc`, `Oracle` as reserved names that fail the driver. This branch makes **Postgres and
Oracle** working backends via fully managed ADO.NET providers — `Npgsql` and
`Oracle.ManagedDataAccess.Core` — no native/thin-client installs. `MySql`/`Odbc` stay reserved.

The seam already exists and is proven: `ISqlDialect` exposes an abstract `DbProviderFactory`
(`ISqlDialect.cs:38`), all runtime I/O goes through `DbConnection`/`DbCommand`/`DbParameter`
(`SqlPollReader.cs:379-389`), and the test-only `SqliteDialect` is the standing falsifiability
control that the planner emits dialect SQL rather than T-SQL. The concrete gaps:

1. **Parameter markers are hardcoded `@`** in two places: `SqlGroupPlanner.cs:21-25` (`@k`/`@w`)
   and the catalog path (`SqlServerDialect.cs:56/:61` SQL text + `SqlCatalogLoader.cs:97-113` bind
   names). Npgsql happens to accept `@`; Oracle requires `:`.
2. **Two provider gates** reject everything but SqlServer: `CreateDialect`
   (`SqlDriverFactoryExtensions.cs:208-215`) and the browser's `DefaultDialectSelector`
   (`SqlDriverBrowser.cs:126-127`).
3. **Oracle has no `INFORMATION_SCHEMA`** — catalog SQL must come from `ALL_USERS` /
   `ALL_TABLES` + `ALL_VIEWS` / `ALL_TAB_COLUMNS`; liveness is `SELECT 1 FROM DUAL`; row-limit is
   `FETCH FIRST 1 ROWS ONLY`.
4. **AdminUI** provider dropdown is single-option (`SqlDriverForm.razor:18-25`).
5. **Packages** — `Npgsql` and `Oracle.ManagedDataAccess.Core` are absent from
   `Directory.Packages.props`.

## 2. Decisions

- **Both dialects live in the existing `ZB.MOM.WW.OtOpcUa.Driver.Sql.Dialect` project**, which
  exists precisely because provider packages are unavoidable there (`.Dialect.csproj:20-26`). Its
  csproj gains the two `PackageReference`s; consumers get them transitively exactly as
  `Microsoft.Data.SqlClient` flows today. No new projects, no `.slnx` churn.
- **Dialect-owned parameter naming.** `ISqlDialect` gains three members with SqlServer-compatible
  semantics implemented on every dialect:
  - `string ParameterMarker(string baseName)` — the token in SQL text (`"@k0"` / `":k0"`).
  - `string ParameterBindName(string baseName)` — the `DbParameter.ParameterName` value (`"@k0"`
    for SqlClient/Npgsql; bare `"k0"` for ODP.NET, its documented convention).
  - `void ConfigureCommand(DbCommand command)` — default no-op; **OracleDialect sets
    `OracleCommand.BindByName = true`** (ODP.NET defaults to bind-by-position, which would silently
    mis-bind the planner's multi-parameter `IN` lists — this is the sharpest correctness edge in
    the branch).
  `SqlGroupPlanner` drops its `@k`/`@w` consts and takes markers from the dialect (bind names ride
  `SqlQueryPlan` alongside, keeping `SqlPollReader.cs:398-405`'s copy-don't-derive discipline).
  `SqlCatalogLoader` binds `schema`/`table` through the dialect instead of the literal `"@schema"`.
  Every command creation site (`SqlPollReader`, `SqlCatalogLoader`, `SqlDriverProbe`,
  `SqlBrowseSession`) calls `ConfigureCommand` after `CreateCommand`.
- **`PostgresDialect`** (`NpgsqlFactory.Instance`): `@` markers; `"`-quoting with `""` escape, max
  identifier 63; `LivenessSql = "SELECT 1"`; limit prefix `""` / suffix `" LIMIT 1"`;
  `INFORMATION_SCHEMA` catalog SQL with `pg_catalog`/`information_schema` schemas excluded and
  `DefaultSchemaSql = "SELECT current_schema()"`; type map over Postgres names (`integer`,
  `bigint`, `smallint`, `real`, `double precision`, `numeric`, `boolean`, `text`,
  `character varying`, `timestamp*`, `uuid`, `bytea`, …).
- **`OracleDialect`** (`OracleClientFactory.Instance`): `:` markers, bare bind names, BindByName;
  `"`-quoting with `""` escape, max identifier 128 (declares Oracle 12.2+ as the supported floor —
  `FETCH FIRST` needs 12c anyway); `LivenessSql = "SELECT 1 FROM DUAL"`; limit suffix
  `" FETCH FIRST 1 ROWS ONLY"`; catalog SQL from `ALL_USERS` (schemas),
  `ALL_TABLES` ∪ `ALL_VIEWS` (tables — matching `INFORMATION_SCHEMA.TABLES`' view inclusion),
  `ALL_TAB_COLUMNS` (columns), `DefaultSchemaSql = "SELECT sys_context('userenv','current_schema') FROM dual"`;
  type map over Oracle names — **`NUMBER` maps to `Float64`** (precision/scale are not visible to
  `MapColumnType(string)`; the per-tag `DeclaredType` override is the documented escape hatch),
  plus `BINARY_DOUBLE`/`BINARY_FLOAT`, `VARCHAR2`/`NVARCHAR2`/`CHAR`/`NCHAR`, `DATE`/`TIMESTAMP*`,
  `CLOB`, `RAW`.
- **Case-folding rides the existing catalog gate.** Postgres lower-cases and Oracle upper-cases
  unquoted identifiers; `SqlCatalogGate` already resolves every authored identifier to the
  catalog's own spelling, so authored `widgets` binds to Oracle `WIDGETS`. The implementer verifies
  the gate's comparison is case-insensitive and pins it per-dialect with tests.
- **Gates widen to exactly these two.** `CreateDialect` and `DefaultDialectSelector` add
  `Postgres`/`Oracle`; `MySql`/`Odbc` keep the existing loud rejection (tests updated: the
  "Postgres is rejected" assertions at `SqlDriverFactoryTests.cs:133-135` and
  `SqlDriverBrowserTests.cs:188-196` flip to `MySql` as the still-reserved control).
- **AdminUI**: `SqlDriverForm.razor` dropdown gains `Postgres` and `Oracle` options; helper text
  updated. No other UI change — Sql stays a single-connection driver with no device form
  (`DriverFormMaps.cs:81-89`), `connectionStringRef` + `Sql__ConnectionStrings__<ref>` env
  resolution is provider-agnostic already, and the `SqlConnectionStringPersisted` deploy gate needs
  no change (it keys on the literal `connectionString` key, not the provider).
- **Packages**: `Directory.Packages.props` gains `Npgsql` (between `MQTTnet` and `Novell.*`) and
  `Oracle.ManagedDataAccess.Core` (between the `OPCFoundation.*` block and `OpenTelemetry`), latest
  stable versions on the feed. `CentralPackageTransitivePinningEnabled` is OFF by design, so if
  restore surfaces an NU1903 in either package's transitive tree, the fix is the established
  surgical direct-pin pattern (`Directory.Packages.props:118-145`), never a suppress.
- **Credential discipline holds automatically**: `SqlDriver` already renders endpoints via the
  provider-neutral `DbConnectionStringBuilder` reading `host`/`data source` keys
  (`SqlDriver.cs:47-51, :952-968`) and logs exception types only.

## 3. Out of scope

- `MySql`/`Odbc` dialects, write support (`IWritable` stays structurally absent, `allowWrites`
  stays inert), the deferred `Query` tag model, connection pooling knobs, and Kerberos/wallet
  auth schemes (connection-string concerns, owned by the operator's `Sql__ConnectionStrings__*`
  env value).

## 4. Testing

1. **Unit.**
   - `PostgresDialectTests` / `OracleDialectTests`: quoting (escape doubling, length caps,
     control-char rejection — mirror `SqlServerDialectTests`), liveness/limit/catalog SQL text,
     type maps, markers/bind names, and `ConfigureCommand` (Oracle: a real `OracleCommand` has
     `BindByName == true` after the call; others: no-op).
   - Planner/loader marker plumbing: a colon-marker fake dialect (or `SqliteDialect` extended)
     proves the planner emits dialect markers and the reader binds the dialect's bind names —
     the falsifiability control for gap 1.
   - Gate tests updated as in §2 (MySql becomes the rejected control).
   - The SQLite-backed read/poll suites keep running unchanged — they prove provider-agnosticism.
2. **Integration (env-gated, skip-clean, docker).** Two new fixtures under
   `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Sql.IntegrationTests/Docker/`, deployed to the shared
   docker host per the established pattern:
   - `postgres:17-alpine` (project `otopcua-sql-pg`, host port 15432) — env
     `SQL_PG_ENDPOINT`/`SQL_PG_USER`/`SQL_PG_PASSWORD`.
   - `gvenzl/oracle-free:23-slim` (project `otopcua-sql-oracle`, host port 11521) — env
     `SQL_ORACLE_ENDPOINT`/`SQL_ORACLE_USER`/`SQL_ORACLE_PASSWORD`.
   Each suite: initialize (liveness + catalog load), KeyValue + WideRow reads against a seeded
   schema, catalog-gate case-folding leg (authored lower-case vs Oracle upper-case catalog),
   browse-session schema/table/column walk. The integration csproj keeps its no-direct-provider-
   reference discipline (`.IntegrationTests.csproj:29-33`) — packages arrive transitively.
3. **Live `/run` chrome gate — DEFERRED, run separately by the operator.** AdminUI legs (provider
   dropdown authoring, Test-connect, `/raw` browse against the Postgres and Oracle fixtures,
   browse-commit of a column). The executing session must NOT run this; it hands the operator the
   command at the end.

## 5. Bookkeeping

`docs/drivers/Sql.md` (§providers table, config table, browse section),
`SqlProvider.cs` doc comments ("reserved for P2" → supported), the tracking doc's Sql row, and
`docs/research/drivers/sql-poll.md` stays as-is (research is a record, not living doc).