lmxopcua/docs/plans/2026-08-05-sql-postgres-oracle-design.md

Sql driver: Postgres + Oracle providers (managed drivers only) — design

Status: DONE 2026-08-05 — all eight tasks delivered and reviewed. Suites: Driver.Sql.Tests 355/355, Browser.Tests 75/75, IntegrationTests 33 skip-clean offline; both new integration suites also ran LIVE during development against real local containers (postgres:17-alpine 5/5, gvenzl/oracle-free:23-slim 6/6), which surfaced and fixed a provider-fatal defect no offline test could see: ORA-01745 — Oracle rejects a bind variable named after a reserved word (:table), so every Oracle catalog load failed until OracleDialect namespaced all binds with a p_ prefix (ec90b7c4, pinned by a never-bare-base-name invariant theory). Merged to master d54f3c83.

Shared-host live run PASSED (2026-08-05, Task 8). Both fixtures deployed to the shared docker host (/opt/otopcua-sql-pg port 15432, /opt/otopcua-sql-oracle port 11521, both project: lmxopcua-labelled, seeds exited 0) and left running; the env-gated suites ran against them from this repo: 11/11 passed, 0 skipped (5 Postgres + 6 Oracle — initialize/catalog, KeyValue, WideRow topByTimestamp, catalog case-folding, browse walk). Env recipe used: SQL_PG_ENDPOINT=10.100.0.35:15432 SQL_PG_USER=otopcua SQL_PG_PASSWORD=<compose-up value> + SQL_ORACLE_ENDPOINT=10.100.0.35:11521 SQL_ORACLE_USER=otopcua SQL_ORACLE_PASSWORD=<compose-up value> (service defaults FREEPDB1).

§4.3's chrome/AdminUI gate PASSED (2026-08-05) — and it found a real defect. Run against a docker-dev rig rebuilt from master (both centrals on one image, provider assemblies confirmed present in /app) with Sql__ConnectionStrings__FixturePg / __FixtureOracle added to central-1 and central-2 (:9200 round-robins, so a browse can land on either node).

⚠️ The gate's first finding: Test Connect was broken for BOTH new providers. SqlDriverProbe.DefaultDialectFor was never widened — it still constructed SqlServerDialect only and threw InvalidOperationException for Postgres/Oracle, so a config that deployed and browsed fine failed Test Connect. There are three dialect-selection sites and Task 4's Files: block named two (SqlDriverFactoryExtensions, SqlDriverBrowser) plus the enum. No test caught it because every case in SqlDriverProbeTests builds its probe through ForTest(...), which injects _ => dialect and never reaches the production switch — the parameterless constructor had zero coverage. Fixed in 75a057f2: switch widened arm-for-arm with the factory, plus an internal seam covering the gate directly, a reserved-provider (MySql/Odbc) falsification control, and an end-to-end production-constructor case. Red-first confirmed (3 failures) before the widening; Driver.Sql.Tests then 361/361.

Legs, after the fix:

1. Provider dropdown authoring ✅ — dropdown offers SqlServer/Postgres/Oracle; both saved and persisted as {"provider":"Postgres",…} / {"provider":"Oracle",…}, i.e. serialized by name, not ordinal (the driver-enum-serialization defect class does not recur here).
2. Test-connect ✅ — Postgres OK · 174 ms, Oracle OK · 386 ms. Negative control: the Oracle driver cross-wired to the Postgres connection string returns FAILED · … (ARGUMENTEXCEPTION) — the provider itself rejecting the string, and not InvalidOperationException, which proves the dialect was constructed and the green came from a real round trip rather than an unconditional OK.
3. /raw browse + browse-commit ✅ — Postgres browse discovered schema sqlpoll → tag_values / latest_status → per-column LEAFs; the schema is owned by sqlpoll_owner while the poller connects as otopcua, so this is the live confirmation that PostgresDialect.ListSchemasSql sources from information_schema.tables rather than .schemata. Oracle browse listed the PDB's schemas → OTOPCUA → TAG_VALUES/LATEST_STATUS → LEAFs. One leaf committed per provider, both persisting schema-qualified with the right type: {"model":"WideRow","table":"sqlpoll.tag_values","columnName":"num_value"} [Double] and {"model":"WideRow","table":"OTOPCUA.TAG_VALUES","columnName":"NUM_VALUE"} [Double].

1. Problem

docs/drivers/Sql.md:15-18: v1 constructs SQL Server only; SqlProvider declares Postgres, MySql, Odbc, Oracle as reserved names that fail the driver. This branch makes Postgres and Oracle working backends via fully managed ADO.NET providers — Npgsql and Oracle.ManagedDataAccess.Core — no native/thin-client installs. MySql/Odbc stay reserved.

The seam already exists and is proven: ISqlDialect exposes an abstract DbProviderFactory (ISqlDialect.cs:38), all runtime I/O goes through DbConnection/DbCommand/DbParameter (SqlPollReader.cs:379-389), and the test-only SqliteDialect is the standing falsifiability control that the planner emits dialect SQL rather than T-SQL. The concrete gaps:

1. Parameter markers are hardcoded @ in two places: SqlGroupPlanner.cs:21-25 (@k/@w) and the catalog path (SqlServerDialect.cs:56/:61 SQL text + SqlCatalogLoader.cs:97-113 bind names). Npgsql happens to accept @; Oracle requires :.
2. Two provider gates reject everything but SqlServer: CreateDialect (SqlDriverFactoryExtensions.cs:208-215) and the browser's DefaultDialectSelector (SqlDriverBrowser.cs:126-127).
3. Oracle has no INFORMATION_SCHEMA — catalog SQL must come from ALL_USERS / ALL_TABLES + ALL_VIEWS / ALL_TAB_COLUMNS; liveness is SELECT 1 FROM DUAL; row-limit is FETCH FIRST 1 ROWS ONLY.
4. AdminUI provider dropdown is single-option (SqlDriverForm.razor:18-25).
5. Packages — Npgsql and Oracle.ManagedDataAccess.Core are absent from Directory.Packages.props.

2. Decisions

• Both dialects live in the existing ZB.MOM.WW.OtOpcUa.Driver.Sql.Dialect project, which exists precisely because provider packages are unavoidable there (.Dialect.csproj:20-26). Its csproj gains the two PackageReferences; consumers get them transitively exactly as Microsoft.Data.SqlClient flows today. No new projects, no .slnx churn.
• Dialect-owned parameter naming. ISqlDialect gains three members with SqlServer-compatible semantics implemented on every dialect:
  • string ParameterMarker(string baseName) — the token in SQL text ("@k0" / ":k0").
  • string ParameterBindName(string baseName) — the DbParameter.ParameterName value ("@k0" for SqlClient/Npgsql; bare "k0" for ODP.NET, its documented convention).
  • void ConfigureCommand(DbCommand command) — default no-op; OracleDialect sets OracleCommand.BindByName = true (ODP.NET defaults to bind-by-position, which would silently mis-bind the planner's multi-parameter IN lists — this is the sharpest correctness edge in the branch). SqlGroupPlanner drops its @k/@w consts and takes markers from the dialect (bind names ride SqlQueryPlan alongside, keeping SqlPollReader.cs:398-405's copy-don't-derive discipline). SqlCatalogLoader binds schema/table through the dialect instead of the literal "@schema". Every command creation site (SqlPollReader, SqlCatalogLoader, SqlDriverProbe, SqlBrowseSession) calls ConfigureCommand after CreateCommand.
• PostgresDialect (NpgsqlFactory.Instance): @ markers; "-quoting with "" escape, max identifier 63; LivenessSql = "SELECT 1"; limit prefix "" / suffix " LIMIT 1"; INFORMATION_SCHEMA catalog SQL with pg_catalog/information_schema schemas excluded and DefaultSchemaSql = "SELECT current_schema()"; type map over Postgres names (integer, bigint, smallint, real, double precision, numeric, boolean, text, character varying, timestamp*, uuid, bytea, …).
• OracleDialect (OracleClientFactory.Instance): : markers, bare bind names, BindByName; "-quoting with "" escape, max identifier 128 (declares Oracle 12.2+ as the supported floor — FETCH FIRST needs 12c anyway); LivenessSql = "SELECT 1 FROM DUAL"; limit suffix " FETCH FIRST 1 ROWS ONLY"; catalog SQL from ALL_USERS (schemas), ALL_TABLES ∪ ALL_VIEWS (tables — matching INFORMATION_SCHEMA.TABLES' view inclusion), ALL_TAB_COLUMNS (columns), DefaultSchemaSql = "SELECT sys_context('userenv','current_schema') FROM dual"; type map over Oracle names — NUMBER maps to Float64 (precision/scale are not visible to MapColumnType(string); the per-tag DeclaredType override is the documented escape hatch), plus BINARY_DOUBLE/BINARY_FLOAT, VARCHAR2/NVARCHAR2/CHAR/NCHAR, DATE/TIMESTAMP*, CLOB, RAW.
• Case-folding rides the existing catalog gate. Postgres lower-cases and Oracle upper-cases unquoted identifiers; SqlCatalogGate already resolves every authored identifier to the catalog's own spelling, so authored widgets binds to Oracle WIDGETS. The implementer verifies the gate's comparison is case-insensitive and pins it per-dialect with tests.
• Gates widen to exactly these two. CreateDialect and DefaultDialectSelector add Postgres/Oracle; MySql/Odbc keep the existing loud rejection (tests updated: the "Postgres is rejected" assertions at SqlDriverFactoryTests.cs:133-135 and SqlDriverBrowserTests.cs:188-196 flip to MySql as the still-reserved control).
• AdminUI: SqlDriverForm.razor dropdown gains Postgres and Oracle options; helper text updated. No other UI change — Sql stays a single-connection driver with no device form (DriverFormMaps.cs:81-89), connectionStringRef + Sql__ConnectionStrings__<ref> env resolution is provider-agnostic already, and the SqlConnectionStringPersisted deploy gate needs no change (it keys on the literal connectionString key, not the provider).
• Packages: Directory.Packages.props gains Npgsql (between MQTTnet and Novell.*) and Oracle.ManagedDataAccess.Core (between the OPCFoundation.* block and OpenTelemetry), latest stable versions on the feed. CentralPackageTransitivePinningEnabled is OFF by design, so if restore surfaces an NU1903 in either package's transitive tree, the fix is the established surgical direct-pin pattern (Directory.Packages.props:118-145), never a suppress.
• Credential discipline holds automatically: SqlDriver already renders endpoints via the provider-neutral DbConnectionStringBuilder reading host/data source keys (SqlDriver.cs:47-51, :952-968) and logs exception types only.

3. Out of scope

• MySql/Odbc dialects, write support (IWritable stays structurally absent, allowWrites stays inert), the deferred Query tag model, connection pooling knobs, and Kerberos/wallet auth schemes (connection-string concerns, owned by the operator's Sql__ConnectionStrings__* env value).

4. Testing

1. Unit.
   • PostgresDialectTests / OracleDialectTests: quoting (escape doubling, length caps, control-char rejection — mirror SqlServerDialectTests), liveness/limit/catalog SQL text, type maps, markers/bind names, and ConfigureCommand (Oracle: a real OracleCommand has BindByName == true after the call; others: no-op).
   • Planner/loader marker plumbing: a colon-marker fake dialect (or SqliteDialect extended) proves the planner emits dialect markers and the reader binds the dialect's bind names — the falsifiability control for gap 1.
   • Gate tests updated as in §2 (MySql becomes the rejected control).
   • The SQLite-backed read/poll suites keep running unchanged — they prove provider-agnosticism.
2. Integration (env-gated, skip-clean, docker). Two new fixtures under tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Sql.IntegrationTests/Docker/, deployed to the shared docker host per the established pattern:
   • postgres:17-alpine (project otopcua-sql-pg, host port 15432) — env SQL_PG_ENDPOINT/SQL_PG_USER/SQL_PG_PASSWORD.
   • gvenzl/oracle-free:23-slim (project otopcua-sql-oracle, host port 11521) — env SQL_ORACLE_ENDPOINT/SQL_ORACLE_USER/SQL_ORACLE_PASSWORD. Each suite: initialize (liveness + catalog load), KeyValue + WideRow reads against a seeded schema, catalog-gate case-folding leg (authored lower-case vs Oracle upper-case catalog), browse-session schema/table/column walk. The integration csproj keeps its no-direct-provider- reference discipline (.IntegrationTests.csproj:29-33) — packages arrive transitively.
3. Live /run chrome gate — DEFERRED, run separately by the operator. AdminUI legs (provider dropdown authoring, Test-connect, /raw browse against the Postgres and Oracle fixtures, browse-commit of a column). The executing session must NOT run this; it hands the operator the command at the end.

5. Bookkeeping

docs/drivers/Sql.md (§providers table, config table, browse section), SqlProvider.cs doc comments ("reserved for P2" → supported), the tracking doc's Sql row, and docs/research/drivers/sql-poll.md stays as-is (research is a record, not living doc).