feat(sql): Odbc provider — ANSI SQL:2008 profile over System.Data.Odbc (lmxopcua#753)

SqlProvider.Odbc is now constructed; every enum member is a working backend.

- OdbcDialect: System.Data.Odbc 10.0.14 (no vulnerable packages), ANSI double-
  quote quoting, positional `?` markers, INFORMATION_SCHEMA catalog (system
  schemas of the common backends excluded by name — the live gate caught
  psqlODBC listing pg_catalog), FETCH FIRST n ROWS ONLY limits, a conservative
  1000-parameter chunk. Positional binding is safe because every caller adds
  parameters in marker order (planner IN list, catalog schema-then-table).
- Named queries on a positional provider are checked by `?` count against the
  declared params (literals/comments ignored), since there is no name to match.
- Supported backends are those whose SQL accepts that profile (PostgreSQL via
  psqlODBC proven, Snowflake-like); SQL Server/MySQL/DB2 should use their native
  provider. The host needs the ODBC driver manager + vendor driver.
- Reserved-provider controls in factory/probe/browser tests now use an
  undeclared ordinal (99); the AdminUI form offers Odbc and a test pins the
  dropdown theory to SqlDialects.Supported.
- Tests: OdbcDialectTests (unit, any host). OdbcReadTests + Docker/odbc/
  run-odbc-gate.sh run the built suite in a throwaway Linux SDK container with
  unixODBC + psqlODBC against a throwaway seeded PostgreSQL: KV IN-list
  positional binding, where-pair, topByTimestamp via FETCH FIRST, a two-
  parameter named query, and the browse walk. macOS has no vendor ODBC driver,
  so there the suite skips.

(lmxopcua#753)

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
2026-11-09 17:13:47 -05:00
parent 63372e757a
commit 3c9ab4b668
16 changed files with 606 additions and 39 deletions
+5
View File
@@ -118,6 +118,11 @@
(MySqlConnectorFactory.Instance, lmxopcua#753). MIT-licensed and truly async, unlike Oracle's
GPL MySql.Data, which is why the design names it. No native client install. -->
<PackageVersion Include="MySqlConnector" Version="2.6.2" />
<!-- Generic ODBC for the Sql poll driver's OdbcDialect (OdbcFactory.Instance, lmxopcua#753). The managed
package P/Invokes the platform's driver manager (odbc32 on Windows, unixODBC/iODBC elsewhere), so a
host running an Odbc driver instance also needs that and the vendor's ODBC driver installed — the one
provider that breaks the "managed only" rule, which is why it is opt-in per instance. -->
<PackageVersion Include="System.Data.Odbc" Version="10.0.14" />
<PackageVersion Include="Novell.Directory.Ldap.NETStandard" Version="3.6.0" />
<PackageVersion Include="OPCFoundation.NetStandard.Opc.Ua.Client" Version="1.5.378.106" />
<!-- Core carries Opc.Ua.StatusCodes, the oracle StatusCodeParityTests checks the drivers'
+19 -5
View File
@@ -15,11 +15,11 @@ An ordinary SQL database — typically a MES / LIMS / historian staging table th
already writes to. There is no protocol here and nothing to "connect" to a machine: the driver runs
`SELECT`s on a timer and turns cells into tag values.
**Four providers construct**: `SqlServer` (`Microsoft.Data.SqlClient`), `Postgres`
**All five providers construct**: `SqlServer` (`Microsoft.Data.SqlClient`), `Postgres`
(`Npgsql`), `Oracle` (ODP.NET managed — **Oracle 12.2 or later**, the floor the
`FETCH FIRST` paging needs) and `MySql` (`MySqlConnector`, MIT, fully managed — MySQL 8 /
MariaDB). `SqlProvider` also declares `Odbc`, which remains a **reserved name, not a working
backend** — authoring it fails the driver rather than silently falling back. The provider → dialect
`FETCH FIRST` paging needs), `MySql` (`MySqlConnector`, MIT, fully managed — MySQL 8 /
MariaDB) and `Odbc` (`System.Data.Odbc`, see below). An undeclared provider value fails the driver
rather than silently falling back. The provider → dialect
map is one shared table (`SqlDialects`) that the factory, the Test Connect probe and the browse
picker all read, so the three can no longer disagree about what a build supports.
@@ -31,6 +31,20 @@ boolean under MySqlConnector's default `TreatTinyAsBoolean`; `BIT(n)`, `TIME`, `
types infer as `String`. Live gate: `Docker/mysql/docker-compose.yml` + `MySqlReadTests`
(`SQL_MYSQL_ENDPOINT` / `SQL_MYSQL_PASSWORD`).
ODBC specifics: `OdbcDialect` is the **ANSI SQL:2008 profile** — double-quoted identifiers,
positional `?` parameters, the standard `INFORMATION_SCHEMA` views and `FETCH FIRST n ROWS ONLY`.
ODBC abstracts the wire, not the SQL, so the backend behind the DSN must accept all four:
PostgreSQL (psqlODBC — the live gate), Snowflake and similar do; **SQL Server** (needs `OFFSET …
FETCH` after an `ORDER BY`), **MySQL** (no `FETCH FIRST`) and **DB2** (no `INFORMATION_SCHEMA`) do
not and should use their native provider. It is also the one provider that is **not fully
managed**: every host running an Odbc instance needs the platform driver manager (unixODBC / iODBC
/ Windows' own) and the vendor's ODBC driver — on the driver nodes and, for browse, the AdminUI
nodes. Parameters bind in order, so the planner, the catalog loader and the browse session all add
them in marker order, and a named query's `params` bind in authored order (the factory refuses a
statement whose `?` count differs). Live gate: `Docker/odbc/run-odbc-gate.sh` runs `OdbcReadTests`
in a throwaway Linux container with unixODBC + psqlODBC against a throwaway PostgreSQL; on macOS
the suite is unit-tested only (`OdbcDialectTests`) because no vendor ODBC driver is installed.
Two provider-specific things are worth knowing before you author against them:
- **Identifier case folds, and the catalog gate absorbs it.** Postgres lower-cases and Oracle
@@ -195,7 +209,7 @@ existed in the parser — nothing reads it.)
| Key | Default | Notes |
|---|---|---|
| `provider` | `SqlServer` | `SqlServer`, `Postgres`, `Oracle` or `MySql`. `Odbc` is reserved and rejected. |
| `provider` | `SqlServer` | `SqlServer`, `Postgres`, `Oracle`, `MySql` or `Odbc`. |
| `connectionStringRef` | — | **A name, never a connection string.** See below. |
| `defaultPollInterval` | `00:00:05` | Applied to any group without its own interval. |
| `operationTimeout` | `00:00:15` | Client-side wall-clock deadline; a breach surfaces `BadTimeout`. |
@@ -223,6 +223,18 @@ public static class SqlReadOnlyStatement
return null;
}
/// <summary>The marker a positional provider (ODBC) spells every parameter with.</summary>
public const string PositionalMarker = "?";
/// <summary>Counts the positional <c>?</c> markers outside literals and comments.</summary>
/// <param name="sql">The statement.</param>
/// <returns>The marker count.</returns>
public static int CountPositionalMarkers(string sql)
{
ArgumentNullException.ThrowIfNull(sql);
return StripLiteralsAndComments(sql).Count(ch => ch == '?');
}
/// <summary>
/// True when <paramref name="marker"/> appears in <paramref name="sql"/> as a parameter token — outside
/// literals and comments, and not as the prefix of a longer name (<c>@line</c> inside <c>@lineId</c>).
@@ -1,9 +1,9 @@
namespace ZB.MOM.WW.OtOpcUa.Driver.Sql.Contracts;
/// <summary>
/// Which ADO.NET provider/dialect backs a Sql driver instance. <b>Constructed:</b> SqlServer, Postgres,
/// Oracle — each over a fully managed provider package, no native client install. <b>Reserved:</b> MySql
/// and Odbc are declared names only; authoring one fails at driver construction and at browse.
/// Which ADO.NET provider/dialect backs a Sql driver instance. Every member is constructed: SqlServer,
/// Postgres, Oracle and MySql over fully managed provider packages, and Odbc over <c>System.Data.Odbc</c>
/// (which also needs the host's ODBC driver manager and the vendor driver).
/// </summary>
public enum SqlProvider
{
@@ -13,10 +13,13 @@ public enum SqlProvider
/// <summary>PostgreSQL via the managed <c>Npgsql</c> provider.</summary>
Postgres,
/// <summary>MySQL / MariaDB — reserved; no dialect ships, so this is not constructed.</summary>
/// <summary>MySQL / MariaDB via the fully managed <c>MySqlConnector</c> provider.</summary>
MySql,
/// <summary>Generic ODBC — reserved; no dialect ships, so this is not constructed.</summary>
/// <summary>
/// Generic ODBC via <c>System.Data.Odbc</c>, speaking the ANSI SQL:2008 profile (<c>INFORMATION_SCHEMA</c>,
/// <c>FETCH FIRST</c>, positional <c>?</c>) — see <c>OdbcDialect</c> for which backends that fits.
/// </summary>
Odbc,
/// <summary>Oracle Database via the fully managed <c>Oracle.ManagedDataAccess.Core</c> provider.</summary>
@@ -0,0 +1,173 @@
using System.Data.Common;
using System.Data.Odbc;
using System.Globalization;
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
using ZB.MOM.WW.OtOpcUa.Driver.Sql.Contracts;
namespace ZB.MOM.WW.OtOpcUa.Driver.Sql;
/// <summary>
/// Generic ODBC dialect (design §2, P3; lmxopcua#753) over <c>System.Data.Odbc</c> — the <b>ANSI SQL:2008
/// profile</b>: double-quote identifier quoting, positional <c>?</c> parameter markers, the standard
/// <c>INFORMATION_SCHEMA</c> catalog views, and the standard <c>FETCH FIRST n ROWS ONLY</c> row limit.
/// <para><b>What that means for a backend.</b> ODBC abstracts the wire, not the SQL: this dialect works
/// against a data source whose SQL accepts all four of those — PostgreSQL (psqlODBC), Snowflake, Oracle 12c+
/// with an <c>INFORMATION_SCHEMA</c> compatibility view set, and similar. A backend that lacks one (SQL Server
/// needs <c>OFFSET … FETCH</c> after an <c>ORDER BY</c>; MySQL has no <c>FETCH FIRST</c>; DB2 has no
/// <c>INFORMATION_SCHEMA</c>) should use its native provider. Unlike every other dialect, ODBC also needs a
/// native driver manager (unixODBC / iODBC / the Windows one) and the vendor's ODBC driver installed on the
/// host.</para>
/// <para><b>Parameters are positional.</b> ODBC ignores <see cref="DbParameter.ParameterName"/> and binds
/// <c>?</c> markers in the order parameters are added, so every caller must add them in the order their
/// markers appear: the planner's key list is already in marker order, and the catalog SQL below binds
/// <c>schema</c> before <c>table</c>, the order <c>SqlCatalogLoader</c> and the browse session add them.</para>
/// <para>Stateless and immutable, so one instance is safely shared.</para>
/// </summary>
public sealed class OdbcDialect : ISqlDialect
{
/// <summary>
/// The identifier ceiling — SQL:2008's 128 characters, which is also SQL Server's and Oracle's. A
/// backend with a lower limit rejects an over-long name itself; the catalog gate only ever quotes
/// names the catalog returned.
/// </summary>
internal const int MaxIdentifierLength = 128;
/// <summary>The positional marker every ODBC parameter is spelled with.</summary>
internal const string PositionalMarker = "?";
/// <inheritdoc/>
public SqlProvider Provider => SqlProvider.Odbc;
/// <inheritdoc/>
public DbProviderFactory Factory => OdbcFactory.Instance;
/// <inheritdoc />
public string ParameterMarker(string baseName) => PositionalMarker;
// The name is ignored by the ODBC driver manager (binding is positional) but kept as the bare base name so
// the parameter collection stays readable in a log.
/// <inheritdoc />
public string ParameterBindName(string baseName) => baseName;
/// <inheritdoc />
public void ConfigureCommand(DbCommand command)
{
// Intentionally empty: OdbcCommand needs no per-command setup.
}
// 1000 — conservative, because the backend behind the DSN is unknown: it is Oracle's IN-list ceiling (the
// strictest common one) and well inside SQL Server's 2100. A larger key-value group is chunked into several
// statements, which costs round-trips and nothing else.
/// <inheritdoc />
public int MaxBindParameters => 1000;
/// <inheritdoc/>
public string LivenessSql => "SELECT 1";
/// <inheritdoc />
public string SingleRowLimitPrefix => string.Empty;
/// <inheritdoc />
public string SingleRowLimitSuffix => " FETCH FIRST 1 ROWS ONLY";
/// <inheritdoc />
public string RowLimitPrefix(int rows)
{
ArgumentOutOfRangeException.ThrowIfLessThan(rows, 1);
return string.Empty;
}
/// <inheritdoc />
public string RowLimitSuffix(int rows)
{
ArgumentOutOfRangeException.ThrowIfLessThan(rows, 1);
return " FETCH FIRST " + rows.ToString(CultureInfo.InvariantCulture) + " ROWS ONLY";
}
// The schemas holding relations this connection can see — derived from TABLES (not SCHEMATA, which some
// backends restrict to owned schemas; the PostgresDialect note). The backend is unknown, so the system
// schemas of the common INFORMATION_SCHEMA backends are excluded by name, case-insensitively (UPPER is
// standard): the standard's own, PostgreSQL's pg_catalog / pg_toast* / pg_temp* (psqlODBC lists pg_catalog,
// proven by the live gate), and SQL Server's / MySQL's sys, mysql, performance_schema. Aliases are
// double-quoted so a backend that folds unquoted output names still returns the spelling the consumers read
// by name.
/// <inheritdoc />
public string ListSchemasSql =>
"SELECT DISTINCT TABLE_SCHEMA AS \"TABLE_SCHEMA\" FROM INFORMATION_SCHEMA.TABLES " +
"WHERE UPPER(TABLE_SCHEMA) NOT IN ('INFORMATION_SCHEMA', 'PG_CATALOG', 'SYS', 'MYSQL', 'PERFORMANCE_SCHEMA') " +
"AND UPPER(TABLE_SCHEMA) NOT LIKE 'PG\\_TOAST%' ESCAPE '\\' " +
"AND UPPER(TABLE_SCHEMA) NOT LIKE 'PG\\_TEMP%' ESCAPE '\\' " +
"ORDER BY 1";
// SQL:2008's CURRENT_SCHEMA is the portable spelling where it exists; a backend without it fails the query
// and the loader falls back to the sole visible schema (or authoring must qualify every table).
/// <inheritdoc />
public string DefaultSchemaSql => "SELECT CURRENT_SCHEMA";
/// <inheritdoc />
public string ListTablesSql =>
"SELECT TABLE_NAME AS \"TABLE_NAME\", TABLE_TYPE AS \"TABLE_TYPE\" FROM INFORMATION_SCHEMA.TABLES " +
"WHERE TABLE_SCHEMA = ? ORDER BY 1";
// schema BEFORE table — positional binding makes this order part of the contract.
/// <inheritdoc />
public string ListColumnsSql =>
"SELECT COLUMN_NAME AS \"COLUMN_NAME\", DATA_TYPE AS \"DATA_TYPE\", IS_NULLABLE AS \"IS_NULLABLE\" " +
"FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA = ? AND TABLE_NAME = ? ORDER BY ORDINAL_POSITION";
// ANSI double-quote quoting, doubling an embedded quote. Same rejection discipline as the sibling dialects
// (blank, over-long, control characters incl. NUL, Unicode format characters), never echoing the value.
/// <inheritdoc />
public string QuoteIdentifier(string ident)
{
if (string.IsNullOrWhiteSpace(ident))
throw new ArgumentException("A SQL identifier must be a non-empty, non-whitespace name.", nameof(ident));
if (ident.Length > MaxIdentifierLength)
throw new ArgumentException(
$"An ODBC identifier may not exceed {MaxIdentifierLength} characters (got {ident.Length}).",
nameof(ident));
foreach (var ch in ident)
{
if (char.IsControl(ch))
throw new ArgumentException(
"A SQL identifier may not contain control characters (including NUL).", nameof(ident));
if (CharUnicodeInfo.GetUnicodeCategory(ch) == UnicodeCategory.Format)
throw new ArgumentException(
"A SQL identifier may not contain Unicode format characters "
+ "(bidi overrides, zero-width spaces, soft hyphen, BOM).", nameof(ident));
}
return string.Concat("\"", ident.Replace("\"", "\"\"", StringComparison.Ordinal), "\"");
}
// The union of the standard type names and the common backends' spellings (INFORMATION_SCHEMA.DATA_TYPE and
// the ODBC driver's GetDataTypeName both reach here). Never throws; unknown ⇒ String.
/// <inheritdoc />
public DriverDataType MapColumnType(string sqlDataType)
{
if (string.IsNullOrWhiteSpace(sqlDataType)) return DriverDataType.String;
var name = sqlDataType.Trim().ToLowerInvariant();
var paren = name.IndexOf('(', StringComparison.Ordinal);
if (paren >= 0) name = name[..paren].Trim();
return name switch
{
"boolean" or "bool" => DriverDataType.Boolean,
"smallint" or "int2" or "tinyint" => DriverDataType.Int16,
"integer" or "int" or "int4" => DriverDataType.Int32,
"bigint" or "int8" => DriverDataType.Int64,
"real" or "float4" => DriverDataType.Float32,
"double precision" or "double" or "float" or "float8" or "decimal" or "numeric" or "number"
=> DriverDataType.Float64,
"character varying" or "varchar" or "character" or "char" or "nvarchar" or "nchar" or "text"
or "varchar2" or "nvarchar2" or "clob" or "uuid" => DriverDataType.String,
"date" or "timestamp" or "timestamp without time zone" or "timestamp with time zone"
or "timestamptz" or "datetime" or "datetime2" => DriverDataType.DateTime,
_ => DriverDataType.String,
};
}
}
@@ -26,6 +26,7 @@ public static class SqlDialects
SqlProvider.Postgres => new PostgresDialect(),
SqlProvider.Oracle => new OracleDialect(),
SqlProvider.MySql => new MySqlDialect(),
SqlProvider.Odbc => new OdbcDialect(),
_ => null,
};
@@ -33,6 +33,9 @@
<PackageReference Include="Oracle.ManagedDataAccess.Core" />
<!-- MySqlDialect (lmxopcua#753): fully managed, so it keeps the "no native client" rule. -->
<PackageReference Include="MySqlConnector" />
<!-- OdbcDialect (lmxopcua#753). Managed wrapper only; the native driver manager is a host prerequisite
that is touched only when an Odbc instance opens a connection. -->
<PackageReference Include="System.Data.Odbc" />
</ItemGroup>
<ItemGroup>
<InternalsVisibleTo Include="ZB.MOM.WW.OtOpcUa.Driver.Sql.Tests"/>
@@ -284,7 +284,8 @@ public static class SqlDriverFactoryExtensions
/// <para>The shape rules are <see cref="SqlNamedQueryRules.Describe"/> — the same call the AdminUI driver
/// form makes — so the form refuses what this refuses. The one rule added here needs the dialect: every
/// declared parameter's marker (<see cref="ISqlDialect.ParameterMarker"/>) must appear in the statement.
/// A provider that binds positionally (<c>?</c>) is exempt, since there is no name to look for.</para>
/// A provider that binds positionally (<c>?</c>) is checked by count instead, since there is no name to
/// look for.</para>
/// </summary>
private static IReadOnlyDictionary<string, SqlNamedQuery> BuildQueries(
SqlDriverConfigDto dto, string driverInstanceId, ISqlDialect? dialect)
@@ -306,7 +307,7 @@ public static class SqlDriverFactoryExtensions
if (dialect is not null)
{
var marker = dialect.ParameterMarker(bare);
if (marker != "?" && !SqlReadOnlyStatement.UsesMarker(sql, marker))
if (marker != SqlReadOnlyStatement.PositionalMarker && !SqlReadOnlyStatement.UsesMarker(sql, marker))
{
throw new InvalidOperationException(
$"Sql driver config for '{driverInstanceId}': named query '{name}' declares parameter " +
@@ -318,6 +319,18 @@ public static class SqlDriverFactoryExtensions
values.Add(ToParameterValue(value));
}
// A positional provider binds in authored order, so the only check available is the count — and it
// matters: one '?' too few binds every later value to the wrong marker.
if (dialect is not null
&& dialect.ParameterMarker("p") == SqlReadOnlyStatement.PositionalMarker
&& SqlReadOnlyStatement.CountPositionalMarkers(sql) != names.Count)
{
throw new InvalidOperationException(
$"Sql driver config for '{driverInstanceId}': named query '{name}' declares {names.Count} " +
$"parameter(s), but its statement has {SqlReadOnlyStatement.CountPositionalMarkers(sql)} " +
$"positional '?' marker(s); {dialect.Provider} binds them in authored order, so the two must match.");
}
queries[name] = new SqlNamedQuery(
name, sql, names, values, query.MaxRows ?? SqlNamedQueryRules.DefaultMaxRows);
}
@@ -18,14 +18,17 @@
<div class="col-md-4">
<label class="form-label">Provider</label>
<InputSelect @bind-Value="_form.Provider" @bind-Value:after="EmitAsync" class="form-select form-select-sm">
@* Every provider SqlDialects constructs; Odbc stays a reserved SqlProvider member and is
deliberately not offered (authoring one fails the driver). *@
@* Every provider SqlDialects constructs. *@
<option value="@SqlProvider.SqlServer">@SqlProvider.SqlServer</option>
<option value="@SqlProvider.Postgres">@SqlProvider.Postgres</option>
<option value="@SqlProvider.Oracle">@SqlProvider.Oracle</option>
<option value="@SqlProvider.MySql">@SqlProvider.MySql</option>
<option value="@SqlProvider.Odbc">@SqlProvider.Odbc</option>
</InputSelect>
<div class="form-text">SQL Server, PostgreSQL, Oracle and MySQL/MariaDB supported; Odbc reserved.</div>
<div class="form-text">
SQL Server, PostgreSQL, Oracle and MySQL/MariaDB use managed providers. Odbc speaks ANSI SQL
(INFORMATION_SCHEMA, FETCH FIRST, positional ?) and needs the ODBC driver manager + vendor driver on every host.
</div>
</div>
<div class="col-md-8">
<label class="form-label">Connection string ref <span class="text-danger">*</span></label>
@@ -58,10 +58,12 @@ public sealed class SqlDriverBrowserTests
SqlDriverBrowser.DefaultDialectSelector(SqlProvider.Postgres).ShouldBeOfType<PostgresDialect>();
SqlDriverBrowser.DefaultDialectSelector(SqlProvider.Oracle).ShouldBeOfType<OracleDialect>();
SqlDriverBrowser.DefaultDialectSelector(SqlProvider.MySql).ShouldBeOfType<MySqlDialect>();
SqlDriverBrowser.DefaultDialectSelector(SqlProvider.Odbc).ShouldBeOfType<OdbcDialect>();
SqlProvider[] carried = [SqlProvider.SqlServer, SqlProvider.Postgres, SqlProvider.Oracle, SqlProvider.MySql];
foreach (var provider in Enum.GetValues<SqlProvider>().Where(p => !carried.Contains(p)))
SqlDriverBrowser.DefaultDialectSelector(provider).ShouldBeNull();
// Every declared member is carried now; an undeclared ordinal is the control that must stay null.
foreach (var provider in Enum.GetValues<SqlProvider>())
SqlDriverBrowser.DefaultDialectSelector(provider).ShouldNotBeNull();
SqlDriverBrowser.DefaultDialectSelector((SqlProvider)99).ShouldBeNull();
}
// ---- connection-string reference resolution ----
@@ -189,8 +191,8 @@ public sealed class SqlDriverBrowserTests
// ---- provider availability ----
/// <summary>
/// The falsifiability control for the widened browse gate: <see cref="SqlProvider.Odbc"/> is still
/// reserved and must be reported as unavailable, naming itself.
/// The falsifiability control for the widened browse gate: an undeclared provider ordinal must be
/// reported as unavailable, naming itself.
/// </summary>
[Fact]
public async Task Open_providerThisBuildDoesNotCarry_saysNotAvailableInThisBuild()
@@ -199,9 +201,9 @@ public sealed class SqlDriverBrowserTests
var ex = await Should.ThrowAsync<InvalidOperationException>(() =>
browser.OpenAsync(
ConfigJson("Odbc", connectionStringRef: "MesStaging"), CancellationToken.None));
ConfigJson("99", connectionStringRef: "MesStaging"), CancellationToken.None));
ex.Message.ShouldContain("Odbc");
ex.Message.ShouldContain("99");
ex.Message.ShouldContain("not available in this build");
}
@@ -217,7 +219,7 @@ public sealed class SqlDriverBrowserTests
var ex = await Should.ThrowAsync<InvalidOperationException>(() =>
browser.OpenAsync(
ConfigJson("Odbc", connectionStringRef: "MesStaging"), CancellationToken.None));
ConfigJson("99", connectionStringRef: "MesStaging"), CancellationToken.None));
ex.Message.ShouldContain("not available in this build");
reads.ShouldBeEmpty();
@@ -0,0 +1,43 @@
#!/usr/bin/env bash
# The Sql driver's ODBC live gate (OdbcReadTests, lmxopcua#753), fully throwaway.
#
# A macOS dev box has no ODBC vendor driver, so the gate runs the ALREADY-BUILT test assembly inside a Linux
# .NET SDK container that apt-installs unixODBC + psqlODBC, against a disposable PostgreSQL seeded with the same
# seed-pg.sql the Postgres suite uses. Everything it starts is removed on exit (trap), success or failure.
#
# dotnet build tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Sql.IntegrationTests -c Release
# tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Sql.IntegrationTests/Docker/odbc/run-odbc-gate.sh
#
# Needs network egress from the SDK container for apt. SDK_IMAGE can be overridden.
set -euo pipefail
HERE="$(cd "$(dirname "$0")" && pwd)"
PROJECT="$(cd "$HERE/../.." && pwd)"
BIN="$PROJECT/bin/Release/net10.0"
SDK_IMAGE="${SDK_IMAGE:-mcr.microsoft.com/dotnet/sdk:10.0.103-noble}"
NET=otopcua-sql-odbc-net
PG=otopcua-sql-odbc-pg
PW="Sqlodbc_dev_pw1"
[ -f "$BIN/ZB.MOM.WW.OtOpcUa.Driver.Sql.IntegrationTests.dll" ] || { echo "build the suite in Release first: $BIN" >&2; exit 2; }
cleanup() { docker rm -f "$PG" >/dev/null 2>&1 || true; docker network rm "$NET" >/dev/null 2>&1 || true; }
trap cleanup EXIT
cleanup
docker network create "$NET" >/dev/null
docker run -d --name "$PG" --network "$NET" --label project=lmxopcua \
-e POSTGRES_PASSWORD="$PW" -e POSTGRES_DB=sqlpollfixture postgres:17-alpine >/dev/null
for _ in $(seq 1 60); do docker exec "$PG" pg_isready -U postgres -d sqlpollfixture >/dev/null 2>&1 && break; sleep 1; done
docker exec -i "$PG" psql -q -v ON_ERROR_STOP=1 -v app_password="$PW" -U postgres -d sqlpollfixture \
< "$PROJECT/Docker/postgres/seed-pg.sql"
docker run --rm --network "$NET" -v "$BIN":/suite -w /suite \
-e SQL_ODBC_CONNECTION_STRING="Driver={PostgreSQL Unicode};Server=$PG;Port=5432;Database=sqlpollfixture;Uid=otopcua;Pwd=$PW" \
"$SDK_IMAGE" bash -c '
set -e
apt-get update -qq >/dev/null && apt-get install -y -qq unixodbc odbc-postgresql >/dev/null
odbcinst -q -d
dotnet ZB.MOM.WW.OtOpcUa.Driver.Sql.IntegrationTests.dll -class ZB.MOM.WW.OtOpcUa.Driver.Sql.IntegrationTests.OdbcReadTests
'
@@ -0,0 +1,161 @@
using System.Globalization;
using System.Text.Json.Nodes;
using Shouldly;
using Xunit;
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
using ZB.MOM.WW.OtOpcUa.Driver.Sql.Browser;
using ZB.MOM.WW.OtOpcUa.Driver.Sql.Contracts;
namespace ZB.MOM.WW.OtOpcUa.Driver.Sql.IntegrationTests;
/// <summary>
/// The <c>Sql</c> driver over <b>ODBC</b> (lmxopcua#753) — <c>OdbcDialect</c>'s ANSI profile through
/// <c>System.Data.Odbc</c>, the platform driver manager and a vendor ODBC driver, against the PostgreSQL
/// fixture's seed (<c>Docker/postgres</c>). What only a live ODBC stack can prove: positional <c>?</c>
/// binding in planner/catalog order, <c>FETCH FIRST</c> paging, the <c>INFORMATION_SCHEMA</c> catalog through
/// a driver, and the named-query transaction over ODBC's manual-commit mode.
/// <para><b>Env-gated on <see cref="ConnectionStringEnvVar"/></b> — a full ODBC connection string, e.g.
/// <c>Driver={PostgreSQL Unicode};Server=…;Port=…;Database=sqlpollfixture;Uid=otopcua;Pwd=…</c>. A macOS dev
/// box normally has no ODBC driver installed, so these skip there; they run inside a Linux container with
/// <c>unixodbc</c> + <c>odbc-postgresql</c> (see <c>Docker/odbc/run-odbc-gate.sh</c>).</para>
/// </summary>
public sealed class OdbcReadTests
{
/// <summary>The full ODBC connection string. Absent ⇒ skip.</summary>
public const string ConnectionStringEnvVar = "SQL_ODBC_CONNECTION_STRING";
private static readonly string? ConnectionString =
Environment.GetEnvironmentVariable(ConnectionStringEnvVar) is { Length: > 0 } cs ? cs : null;
[Fact]
public async Task InitializeAndRead_overOdbc_keyValueWideRowAndTopByTimestamp()
{
SkipUnlessLive();
await using var driver = await StartAsync(null,
Tag("Kv/Speed", new JsonObject
{
["model"] = "KeyValue", ["table"] = "sqlpoll.tag_values", ["keyColumn"] = "tag_name",
["keyValue"] = "Line1.Speed", ["valueColumn"] = "num_value", ["timestampColumn"] = "sample_ts",
}),
Tag("Kv/Pressure", new JsonObject
{
["model"] = "KeyValue", ["table"] = "sqlpoll.tag_values", ["keyColumn"] = "tag_name",
["keyValue"] = "Line1.Pressure", ["valueColumn"] = "num_value",
}),
Tag("Wide/Newest", new JsonObject
{
["model"] = "WideRow", ["table"] = "sqlpoll.latest_status", ["columnName"] = "oven_temp",
["rowSelector"] = new JsonObject { ["topByTimestamp"] = "sample_ts" },
}),
Tag("Wide/Station7", new JsonObject
{
["model"] = "WideRow", ["table"] = "sqlpoll.latest_status", ["columnName"] = "pressure",
["rowSelector"] = new JsonObject { ["whereColumn"] = "station_id", ["whereValue"] = 7 },
}));
driver.GetHealth().State.ShouldBe(DriverState.Healthy);
var snapshots = await driver.ReadAsync(
["Kv/Speed", "Kv/Pressure", "Wide/Newest", "Wide/Station7"], TestContext.Current.CancellationToken);
snapshots.Select(s => s.StatusCode).ShouldAllBe(code => code == SqlStatusCodes.Good);
// Two keys in one IN (?, ?) list: each value landing on its own tag is the positional-binding proof.
Number(snapshots[0]).ShouldBe(42.0);
Number(snapshots[1]).ShouldBe(3.5);
snapshots[0].SourceTimestampUtc.ShouldBe(new DateTime(2026, 7, 24, 10, 0, 0, DateTimeKind.Utc));
Number(snapshots[2]).ShouldBe(210.25); // ORDER BY … DESC FETCH FIRST 1 ROWS ONLY
Number(snapshots[3]).ShouldBe(1.2);
}
[Fact]
public async Task NamedQuery_overOdbc_bindsPositionalParametersInAuthoredOrder()
{
SkipUnlessLive();
await using var driver = await StartAsync(
"""{"band":{"sql":"SELECT station_id AS station, oven_temp AS temp FROM sqlpoll.latest_status WHERE pressure > ? AND pressure < ?","params":{"lo":1.3,"hi":1.5}}}""",
Tag("Q/Eight", new JsonObject
{
["model"] = "Query", ["query"] = "band", ["column"] = "temp",
["rowKey"] = new JsonObject { ["column"] = "station", ["value"] = 8 },
}),
Tag("Q/Seven", new JsonObject
{
["model"] = "Query", ["query"] = "band", ["column"] = "temp",
["rowKey"] = new JsonObject { ["column"] = "station", ["value"] = 7 },
}));
var snapshots = await driver.ReadAsync(["Q/Eight", "Q/Seven"], TestContext.Current.CancellationToken);
snapshots[0].StatusCode.ShouldBe(SqlStatusCodes.Good); // 1.4 is inside (1.3, 1.5)
Number(snapshots[0]).ShouldBe(210.25);
snapshots[1].StatusCode.ShouldBe(SqlStatusCodes.BadNoData); // 1.2 is outside — the bounds were bound in order
}
[Fact]
public async Task Browse_overOdbc_walksSchemaToColumns()
{
SkipUnlessLive();
var config = new JsonObject
{
["provider"] = nameof(SqlProvider.Odbc),
["connectionString"] = ConnectionString,
}.ToJsonString();
await using var session = await new SqlDriverBrowser().OpenAsync(config, TestContext.Current.CancellationToken);
var schema = (await session.RootAsync(TestContext.Current.CancellationToken))
.ShouldHaveSingleItem("the fixture login can SELECT in exactly one schema");
var tables = await session.ExpandAsync(schema.NodeId, TestContext.Current.CancellationToken);
var columns = await session.ExpandAsync(
tables.First(t => t.DisplayName == "tag_values").NodeId, TestContext.Current.CancellationToken);
columns.Select(c => c.DisplayName).ShouldBe(["tag_name", "num_value", "sample_ts"]);
}
private static double Number(DataValueSnapshot snapshot) => Convert.ToDouble(snapshot.Value, CultureInfo.InvariantCulture);
private static void SkipUnlessLive()
{
if (ConnectionString is null)
Assert.Skip($"Set {ConnectionStringEnvVar} to a full ODBC connection string (and install the driver manager + a " +
"vendor ODBC driver) to run the Sql driver's ODBC gate. See Docker/odbc/run-odbc-gate.sh.");
}
/// <summary>Builds the driver through the production factory with <c>provider: Odbc</c>.</summary>
private static async Task<SqlDriver> StartAsync(string? queriesJson, params RawTagEntry[] tags)
{
var reference = $"OdbcIt{Guid.NewGuid():N}";
Environment.SetEnvironmentVariable(SqlConnectionStringResolver.EnvironmentVariableFor(reference), ConnectionString);
var rawTags = new JsonArray();
foreach (var tag in tags)
rawTags.Add(new JsonObject { ["rawPath"] = tag.RawPath, ["tagConfig"] = tag.TagConfig, ["writeIdempotent"] = false });
var config = new JsonObject
{
["provider"] = nameof(SqlProvider.Odbc),
["connectionStringRef"] = reference,
["rawTags"] = rawTags,
};
if (queriesJson is not null) config["queries"] = JsonNode.Parse(queriesJson);
var json = config.ToJsonString();
var driver = SqlDriverFactoryExtensions.CreateInstance("sql-odbc-integration", json);
try
{
await driver.InitializeAsync(json, TestContext.Current.CancellationToken);
}
catch
{
await driver.DisposeAsync();
throw;
}
return driver;
}
private static RawTagEntry Tag(string rawPath, JsonObject config)
{
config["driver"] = "Sql";
return new RawTagEntry(rawPath, config.ToJsonString(), WriteIdempotent: false);
}
}
@@ -0,0 +1,130 @@
using System.Data.Odbc;
using System.Text.Json;
using System.Text.Json.Nodes;
using Shouldly;
using Xunit;
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
using ZB.MOM.WW.OtOpcUa.Driver.Sql.Contracts;
namespace ZB.MOM.WW.OtOpcUa.Driver.Sql.Tests;
/// <summary>
/// Golden tests for <see cref="OdbcDialect"/> (lmxopcua#753) — the ANSI SQL:2008 profile over
/// <c>System.Data.Odbc</c>. These run on any host because none of them opens a connection (opening one needs
/// the platform's ODBC driver manager plus a vendor driver); the live leg is <c>OdbcReadTests</c>.
/// <para><b>The sharp edge is positional binding.</b> ODBC ignores parameter names and binds <c>?</c> markers in
/// the order parameters are added, so every SQL text this dialect produces must place its markers in the order
/// its callers bind — pinned below for the planner's key list and the catalog's schema-then-table pair.</para>
/// </summary>
public class OdbcDialectTests
{
[Fact]
public void Dialect_identifiesItselfAsOdbc_withPositionalMarkers()
{
var d = new OdbcDialect();
d.Provider.ShouldBe(SqlProvider.Odbc);
d.Factory.ShouldBeSameAs(OdbcFactory.Instance);
d.ParameterMarker("k0").ShouldBe("?");
d.ParameterMarker("schema").ShouldBe("?");
d.ParameterBindName("k0").ShouldBe("k0");
d.LivenessSql.ShouldBe("SELECT 1");
d.MaxBindParameters.ShouldBe(1000);
}
[Fact]
public void QuoteIdentifier_isAnsiDoubleQuoting()
{
var d = new OdbcDialect();
d.QuoteIdentifier("Speed").ShouldBe("\"Speed\"");
d.QuoteIdentifier("x\" ; DROP TABLE users --").ShouldBe("\"x\"\" ; DROP TABLE users --\"");
}
[Theory]
[InlineData("")]
[InlineData(" ")]
[InlineData("a\0b")]
[InlineData("a‮b")]
public void QuoteIdentifier_rejectsWhatCannotBeACatalogName(string ident)
=> Should.Throw<ArgumentException>(() => new OdbcDialect().QuoteIdentifier(ident));
[Fact]
public void QuoteIdentifier_capsAt128Characters()
{
new OdbcDialect().QuoteIdentifier(new string('a', 128)).Length.ShouldBe(130);
Should.Throw<ArgumentException>(() => new OdbcDialect().QuoteIdentifier(new string('a', 129)));
}
[Fact]
public void RowLimits_areTheStandardFetchFirst_inTheSuffixPosition()
{
var d = new OdbcDialect();
d.SingleRowLimitPrefix.ShouldBeEmpty();
d.SingleRowLimitSuffix.ShouldBe(" FETCH FIRST 1 ROWS ONLY");
d.RowLimitPrefix(3).ShouldBeEmpty();
d.RowLimitSuffix(3).ShouldBe(" FETCH FIRST 3 ROWS ONLY");
}
[Fact]
public void CatalogSql_isInformationSchema_withSchemaBoundBeforeTable()
{
var d = new OdbcDialect();
d.ListSchemasSql.ShouldContain("INFORMATION_SCHEMA.TABLES");
d.ListSchemasSql.ShouldContain("AS \"TABLE_SCHEMA\"");
// The backend is unknown, so the common backends' system schemas are excluded by name (the live gate
// caught psqlODBC listing pg_catalog).
foreach (var system in new[] { "'INFORMATION_SCHEMA'", "'PG_CATALOG'", "'SYS'", "PG\\_TOAST%" })
d.ListSchemasSql.ShouldContain(system);
d.ListTablesSql.Count(ch => ch == '?').ShouldBe(1);
// Two positional markers: the first must be the schema, because SqlCatalogLoader and SqlBrowseSession
// bind schema first.
d.ListColumnsSql.Count(ch => ch == '?').ShouldBe(2);
d.ListColumnsSql.IndexOf("TABLE_SCHEMA = ?", StringComparison.Ordinal)
.ShouldBeLessThan(d.ListColumnsSql.IndexOf("TABLE_NAME = ?", StringComparison.Ordinal));
}
[Theory]
[InlineData("integer", DriverDataType.Int32)]
[InlineData("bigint", DriverDataType.Int64)]
[InlineData("smallint", DriverDataType.Int16)]
[InlineData("double precision", DriverDataType.Float64)]
[InlineData("numeric(10,2)", DriverDataType.Float64)]
[InlineData("real", DriverDataType.Float32)]
[InlineData("boolean", DriverDataType.Boolean)]
[InlineData("character varying", DriverDataType.String)]
[InlineData("timestamp without time zone", DriverDataType.DateTime)]
[InlineData("date", DriverDataType.DateTime)]
[InlineData("geometry", DriverDataType.String)]
[InlineData("", DriverDataType.String)]
public void MapColumnType_mapsTheStandardAndCommonSpellings(string sql, DriverDataType expected)
=> new OdbcDialect().MapColumnType(sql).ShouldBe(expected);
[Fact]
public void Planner_keyValuePlan_placesOneMarkerPerKey_inBindOrder()
{
var plan = SqlGroupPlanner.Plan(
[
new SqlTagDefinition("A", SqlTagModel.KeyValue, "s.t", KeyColumn: "k", KeyValue: "a", ValueColumn: "v"),
new SqlTagDefinition("B", SqlTagModel.KeyValue, "s.t", KeyColumn: "k", KeyValue: "b", ValueColumn: "v"),
], new OdbcDialect()).ShouldHaveSingleItem();
plan.SqlText.ShouldBe("SELECT \"k\", \"v\" FROM \"s\".\"t\" WHERE \"k\" IN (?, ?) FETCH FIRST 5 ROWS ONLY");
plan.Parameters.ShouldBe(["a", "b"]); // positional: value i binds marker i
}
[Fact]
public void Factory_aPositionalNamedQuery_mustCarryOneMarkerPerParameter()
{
SqlDriverFactoryExtensions.BuildOptions(
Dto("""{"q":{"sql":"SELECT a FROM t WHERE x = ? AND y = '?' AND z = ?","params":{"x":1,"z":2}}}"""),
"d1", new OdbcDialect())
.Queries["q"].ParameterNames.ShouldBe(["x", "z"]); // the '?' inside a literal is not a marker
Should.Throw<InvalidOperationException>(() => SqlDriverFactoryExtensions.BuildOptions(
Dto("""{"q":{"sql":"SELECT a FROM t WHERE x = ?","params":{"x":1,"z":2}}}"""), "d1", new OdbcDialect()))
.Message.ShouldContain("positional");
}
private static SqlDriverConfigDto Dto(string queriesJson) => JsonSerializer.Deserialize<SqlDriverConfigDto>(
new JsonObject { ["connectionStringRef"] = "x", ["queries"] = JsonNode.Parse(queriesJson) }.ToJsonString(),
SqlDriverFactoryExtensions.JsonOptionsForTest)!;
}
@@ -145,8 +145,9 @@ public sealed class SqlDriverFactoryTests
// ---- provider gate ----
/// <summary>
/// The falsifiability control for the widened gate: <see cref="SqlProvider.Odbc"/> is still a
/// reserved name and must fail loudly at construction, naming itself. Without this leg, a gate that
/// The falsifiability control for the widened gate: every declared member now constructs, so an
/// out-of-range ordinal (a config authored against a future enum) stands in for "a provider this build
/// does not carry" — it must fail loudly at construction, naming itself. Without this leg, a gate that
/// had quietly become "construct anything" would pass every test below.
/// </summary>
[Fact]
@@ -157,9 +158,9 @@ public sealed class SqlDriverFactoryTests
SqlConnectionStringResolver.EnvironmentVariableFor(name), "Host=h;Database=db");
var ex = Should.Throw<InvalidOperationException>(() => SqlDriverFactoryExtensions.CreateInstance(
"d1", $$"""{"provider":"Odbc","connectionStringRef":"{{name}}"}""", null));
"d1", $$"""{"provider":99,"connectionStringRef":"{{name}}"}""", null));
ex.Message.ShouldContain("Odbc");
ex.Message.ShouldContain("99");
ex.Message.ShouldContain("not available in this build");
}
@@ -175,9 +176,10 @@ public sealed class SqlDriverFactoryTests
SqlConnectionStringResolver.EnvironmentVariableFor(name), "Host=h;Database=db");
var ex = Should.Throw<InvalidOperationException>(() => SqlDriverFactoryExtensions.CreateInstance(
"d1", $$"""{"provider":"Odbc","connectionStringRef":"{{name}}"}""", null));
"d1", $$"""{"provider":99,"connectionStringRef":"{{name}}"}""", null));
ex.Message.ShouldContain(nameof(SqlProvider.SqlServer));
ex.Message.ShouldContain(nameof(SqlProvider.Odbc));
ex.Message.ShouldContain(nameof(SqlProvider.Postgres));
ex.Message.ShouldContain(nameof(SqlProvider.Oracle));
ex.Message.ShouldContain(nameof(SqlProvider.MySql));
@@ -194,6 +196,7 @@ public sealed class SqlDriverFactoryTests
[InlineData("Postgres", typeof(PostgresDialect))]
[InlineData("Oracle", typeof(OracleDialect))]
[InlineData("MySql", typeof(MySqlDialect))]
[InlineData("Odbc", typeof(OdbcDialect))]
public void ParseBinding_aSupportedProvider_bindsThatProvidersDialect(string provider, Type expected)
{
var name = NewRef();
@@ -215,6 +218,7 @@ public sealed class SqlDriverFactoryTests
[InlineData("Postgres")]
[InlineData("Oracle")]
[InlineData("MySql")]
[InlineData("Odbc")]
public async Task CreateInstance_aNewlySupportedProvider_constructsPastTheProviderGate(string provider)
{
var name = NewRef();
@@ -172,6 +172,7 @@ public sealed class SqlDriverProbeTests
[InlineData(SqlProvider.Postgres)]
[InlineData(SqlProvider.Oracle)]
[InlineData(SqlProvider.MySql)]
[InlineData(SqlProvider.Odbc)]
public void ProductionDialectGate_constructsEveryShippedProvider(SqlProvider provider)
=> SqlDriverProbe.DefaultDialectForTest(provider).Provider.ShouldBe(provider);
@@ -180,7 +181,7 @@ public sealed class SqlDriverProbeTests
/// rejected. Without this, widening the gate to "return a dialect for anything" would pass.
/// </summary>
[Theory]
[InlineData(SqlProvider.Odbc)]
[InlineData((SqlProvider)99)] // every declared member constructs; an undeclared ordinal must not
public void ProductionDialectGate_stillRejectsReservedProviders(SqlProvider provider)
=> Should.Throw<InvalidOperationException>(() => SqlDriverProbe.DefaultDialectForTest(provider))
.Message.ShouldContain(provider.ToString());
@@ -102,6 +102,7 @@ public sealed class SqlDriverFormContractTests
[InlineData(SqlProvider.Postgres, "Postgres")]
[InlineData(SqlProvider.Oracle, "Oracle")]
[InlineData(SqlProvider.MySql, "MySql")]
[InlineData(SqlProvider.Odbc, "Odbc")]
public void Every_dropdown_provider_round_trips_through_the_form_model(SqlProvider provider, string name)
{
var form = new SqlDriverForm.FormModel { Provider = provider, ConnectionStringRef = "MesStaging" };
@@ -117,18 +118,16 @@ public sealed class SqlDriverFormContractTests
}
/// <summary>
/// Odbc stays reserved-and-rejected in this build, so the dropdown deliberately does not offer it. Nothing in the form model blocks the value — this records the intent so a later "the enum has
/// five members, the dropdown should too" edit has to argue with a test.
/// The dropdown offers every provider the build constructs (lmxopcua#753 made all five enum members real),
/// so the form-model theory above must cover exactly <see cref="SqlDialects.Supported"/> — a provider added
/// to the map without a dropdown option (or the reverse) has to argue with this test.
/// </summary>
[Theory]
[InlineData(SqlProvider.Odbc)]
public void Reserved_providers_are_not_offered_but_still_serialize_by_name(SqlProvider reserved)
{
var json = JsonSerializer.Serialize(
new SqlDriverForm.FormModel { Provider = reserved, ConnectionStringRef = "R" }.ToDto(), FormOptions);
[Fact]
public void Every_constructed_provider_is_covered_by_the_dropdown_theory()
=> SqlDialects.Supported.ShouldBe(
[SqlProvider.SqlServer, SqlProvider.Postgres, SqlProvider.MySql, SqlProvider.Odbc, SqlProvider.Oracle],
ignoreOrder: true);
json.ShouldContain($"\"provider\":\"{reserved}\"");
}
// ---- named queries (lmxopcua#753): the form refuses exactly what the factory refuses ----
/// <summary>