# Inbound API Key Shared SQL Server Store — Implementation Plan > **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans to implement this plan task-by-task. **Goal:** Give the shared `ZB.MOM.WW.Auth.ApiKeys` library a SQL Server provider, point both ScadaBridge central nodes at one `zbauth` schema in the configuration database, import each node's existing SQLite keys once, prove a central failover no longer 401s inbound callers on the docker rig, and retire the rig's file-copy workaround. **Architecture:** Two repos, strictly ordered. First `~/Desktop/scadaproj/ZB.MOM.WW.Auth` (0.2.1 → 0.3.0): an `IApiKeyStoreMigrator` abstraction, SQLite implementing it, and a new package `ZB.MOM.WW.Auth.ApiKeys.SqlServer` whose `AddZbApiKeyAuthSqlServer` registers SQL Server stores *before* `AddZbApiKeyAuth` so the library's `TryAdd` calls leave them in place (the Secrets 0.5.0 mechanism). Then ScadaBridge pins 0.3.0, adds `ApiKeyStore:Provider`, composes the SQL provider on central, runs a one-shot SQLite import at boot, and flips the docker rig. Design: `docs/plans/2026-09-02-inbound-api-key-shared-store-design.md`. **Tech Stack:** .NET 10, `Microsoft.Data.SqlClient` 6.0.2 (family pin), xUnit + `Skip.IfNot` live gating, Gitea NuGet feed (`build/push.sh`), docker rig (`docker/`, MSSQL from `infra/`). **Facts the implementer must not re-derive** (verified 2026-09-02): - Library layout: `src/ZB.MOM.WW.Auth.Abstractions/ApiKeys/ApiKeyContracts.cs` holds every contract (`ApiKeyOptions`, `IApiKeyStore`, `IApiKeyAdminStore`, `IApiKeyAuditStore`, records). SQLite lives in `src/ZB.MOM.WW.Auth.ApiKeys/Sqlite/` (`SqliteAuthSchema` v3, `SqliteAuthStoreMigrator`, three stores, `ScopeSerializer` **public**, `SqliteValueParsing` internal). DI: `DependencyInjection/ApiKeyServiceCollectionExtensions.cs` + `ApiKeyMigrationHostedService` (takes `SqliteAuthStoreMigrator` concretely). `Admin/ApiKeyAdminCommands` ctor takes `SqliteAuthStoreMigrator` concretely — MxAccessGateway, HistorianGateway and ScadaBridge all construct it that way, so that ctor **must survive**. - Versioning is lockstep from `Directory.Build.props` (`0.2.1`); packages are packed to `artifacts/` by `build/pack.sh` and pushed by `build/push.sh` (`GITEA_NUGET_SOURCE`, `GITEA_NUGET_KEY`). `README.md` carries the changelog and still says "current release is 0.1.4" — stale, fix it. - Secrets' SQL Server pattern to copy: `~/Desktop/scadaproj/ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Replicator.SqlServer/` — `SecretsSqlServerConnectionFactory`, `SqlServerSecretsSchema` (`CREATE SCHEMA` via `EXEC`, `IF NOT EXISTS` tables), `SqlServerSecretsStoreMigrator` (retry on 1205/2714/1913/2627, `MaxMigrationRetries = 5`), and the DI extension's "register before `AddZbSecrets`" comment. Live tests: `tests/ZB.MOM.WW.Secrets.Replicator.SqlServer.Tests/Live/LiveSqlServerFixture.cs` gated on `SECRETS_SQLSERVER_CONNSTR`. - HMAC-SHA256 hashes are 32 bytes (`ApiKeySecretHasher.Hash`). Scopes are serialised by `ScopeSerializer.Serialize/Deserialize` — reuse it, do not invent a second format. - ScadaBridge touchpoints: `src/ZB.MOM.WW.ScadaBridge.Host/Program.cs:427-470` (store defaults + `AddZbApiKeyAuth` + `ApiKeyAdminCommands` composition), `Directory.Packages.props:101-104` (four `0.2.1` pins), `tests/ZB.MOM.WW.ScadaBridge.Security.Tests/LibraryInboundApiKeyAdminTests.cs`, `tests/ZB.MOM.WW.ScadaBridge.InboundAPI.Tests/EndpointExtensionsTests.cs:753-758` and `EndpointContentTypeTests.cs:157` (construct `ApiKeyAdminCommands` with the SQLite migrator — keep compiling via the retained ctor). - Rig: both centrals reach `scadabridge-mssql` with `ScadaBridge__Database__ConfigurationDb` (env in `docker/docker-compose.yml`, user `scadabridge_app`, which already created the `zbsecrets` schema in `ZbSecretsHub` on 2026-08-07, so it holds DDL rights). Local MSSQL for library tests: `Server=localhost,1433;User Id=sa;Password=;TrustServerCertificate=true` (`infra/docker-compose.yml`). - Register row 108 already records the defect; row 107 is the unrelated `DelmiaRecv` item. Do not add a new row — close 108. - The executing session's `docker/sync-inbound-api-keys.sh` and the preflight in `docker/verify-delmia-notifier.sh` exist because of this defect; Task 14 removes the script and the preflight's dependency on it. **Model assignment:** Opus for the library core (schema, migrator race handling, stores, importer) and the ScadaBridge Host composition; Sonnet for the abstraction plumbing, tests that mirror existing ones, config/docs; Opus for the rig verification; Fable (this session) for the final cross-repo review and the publish decision. --- ## Part A — `~/Desktop/scadaproj/ZB.MOM.WW.Auth` (library 0.3.0) All Part A commands run from `~/Desktop/scadaproj/ZB.MOM.WW.Auth`. Commit in the **scadaproj** repo (it is a plain directory of that repo, no nested `.git`). ### Task 1: `IApiKeyStoreMigrator` abstraction + SQLite implements it **Classification:** small **Estimated implement time:** ~4 min **Parallelizable with:** none (everything in Part A builds on it) **Model:** sonnet **Files:** - Modify: `src/ZB.MOM.WW.Auth.Abstractions/ApiKeys/ApiKeyContracts.cs` (append) - Modify: `src/ZB.MOM.WW.Auth.ApiKeys/Sqlite/SqliteAuthStoreMigrator.cs:14` - Modify: `src/ZB.MOM.WW.Auth.ApiKeys/DependencyInjection/ApiKeyMigrationHostedService.cs:13-14` - Modify: `src/ZB.MOM.WW.Auth.ApiKeys/DependencyInjection/ApiKeyServiceCollectionExtensions.cs:64-68` - Modify: `src/ZB.MOM.WW.Auth.ApiKeys/Admin/ApiKeyAdminCommands.cs:36-73` - Test: `tests/ZB.MOM.WW.Auth.ApiKeys.Tests/ApiKeyServiceCollectionExtensionsTests.cs` **Step 1: Write the failing test** (append to `ApiKeyServiceCollectionExtensionsTests.cs`, following the file's existing container-building helper) ```csharp [Fact] public void AddZbApiKeyAuth_RegistersSqliteMigratorAsIApiKeyStoreMigrator() { using ServiceProvider sp = BuildProvider(); // the file's existing helper that calls AddZbApiKeyAuth over a temp SqlitePath Assert.IsType(sp.GetRequiredService()); Assert.Same(sp.GetRequiredService(), sp.GetRequiredService()); } [Fact] public void AddZbApiKeyAuth_LeavesPreRegisteredMigratorInPlace() { var services = new ServiceCollection(); var fake = new FakeMigrator(); services.AddSingleton(fake); services.AddZbApiKeyAuth(MinimalConfig(), "ApiKeys"); // the file's existing config helper using ServiceProvider sp = services.BuildServiceProvider(); Assert.Same(fake, sp.GetRequiredService()); } private sealed class FakeMigrator : IApiKeyStoreMigrator { public Task MigrateAsync(CancellationToken ct) => Task.CompletedTask; } ``` **Step 2: Run to verify it fails** Run: `dotnet test tests/ZB.MOM.WW.Auth.ApiKeys.Tests --filter "FullyQualifiedName~ApiKeyServiceCollectionExtensionsTests" 2>&1 | tail -5` Expected: build error — `IApiKeyStoreMigrator` does not exist. **Step 3: Implement** `ApiKeyContracts.cs` (append): ```csharp /// /// Creates or upgrades the API-key store schema to the version this build supports. Each store /// provider (SQLite, SQL Server) supplies one; hosts run it at startup and before admin commands. /// Implementations must be idempotent and safe to run concurrently from several nodes. /// public interface IApiKeyStoreMigrator { /// Migrates the store schema. /// A token to cancel the operation. /// A task that completes when the schema is current. Task MigrateAsync(CancellationToken ct); } ``` `SqliteAuthStoreMigrator.cs`: `public sealed class SqliteAuthStoreMigrator(AuthSqliteConnectionFactory connectionFactory) : IApiKeyStoreMigrator` (add `using ZB.MOM.WW.Auth.Abstractions.ApiKeys;`). `ApiKeyMigrationHostedService.cs`: change the primary-ctor parameter type to `IApiKeyStoreMigrator migrator`. `ApiKeyServiceCollectionExtensions.cs` — replace the migrator block: ```csharp // Migrator: the SQLite implementation is the default; a provider package that registers // IApiKeyStoreMigrator (and the three stores) BEFORE this call displaces it — that is the // whole mechanism by which AddZbApiKeyAuthSqlServer swaps the backend. services.TryAddSingleton(sp => new SqliteAuthStoreMigrator(sp.GetRequiredService())); services.TryAddSingleton(sp => sp.GetRequiredService()); ``` `ApiKeyAdminCommands.cs`: field becomes `private readonly IApiKeyStoreMigrator _migrator;`; add a **new** ctor taking `IApiKeyStoreMigrator migrator` with the same body; the existing ctor taking `SqliteAuthStoreMigrator` stays and calls `this(options, adminStore, auditStore, pepperProvider, (IApiKeyStoreMigrator)migrator, clock)`. XML-doc both. **Step 4: Run all library tests** Run: `dotnet test ZB.MOM.WW.Auth.slnx 2>&1 | tail -5` Expected: `Passed!`, 0 failed (the two new tests included). **Step 5: Commit** (in `~/Desktop/scadaproj`) ```bash git add ZB.MOM.WW.Auth/src ZB.MOM.WW.Auth/tests git commit -m "feat(auth.apikeys): IApiKeyStoreMigrator abstraction — SQLite implements it, DI registers it via TryAdd so a provider registered first wins; ApiKeyAdminCommands accepts the interface (old ctor kept)" ``` --- ### Task 2: New package skeleton `ZB.MOM.WW.Auth.ApiKeys.SqlServer` + options + connection factory **Classification:** small **Estimated implement time:** ~5 min **Parallelizable with:** none **Model:** sonnet **Files:** - Create: `src/ZB.MOM.WW.Auth.ApiKeys.SqlServer/ZB.MOM.WW.Auth.ApiKeys.SqlServer.csproj` - Create: `src/ZB.MOM.WW.Auth.ApiKeys.SqlServer/SqlServerApiKeyOptions.cs` - Create: `src/ZB.MOM.WW.Auth.ApiKeys.SqlServer/AuthSqlServerConnectionFactory.cs` - Create: `tests/ZB.MOM.WW.Auth.ApiKeys.SqlServer.Tests/ZB.MOM.WW.Auth.ApiKeys.SqlServer.Tests.csproj` - Create: `tests/ZB.MOM.WW.Auth.ApiKeys.SqlServer.Tests/SqlServerApiKeyOptionsTests.cs` - Modify: `ZB.MOM.WW.Auth.slnx` (add both projects), `Directory.Packages.props` (add `Microsoft.Data.SqlClient` **6.0.2**, `Xunit.SkippableFact` if the Secrets tests use it — check `~/Desktop/scadaproj/ZB.MOM.WW.Secrets/Directory.Packages.props` and copy the exact package/version) **Step 1: csproj** — copy `src/ZB.MOM.WW.Auth.ApiKeys/ZB.MOM.WW.Auth.ApiKeys.csproj`, then: `PackageId` `ZB.MOM.WW.Auth.ApiKeys.SqlServer`; `Description` "SQL Server API-key store for the ZB.MOM.WW SCADA family — one shared row set for clustered hosts (central pairs)."; `ProjectReference` to `..\ZB.MOM.WW.Auth.ApiKeys\ZB.MOM.WW.Auth.ApiKeys.csproj` (brings Abstractions, `ScopeSerializer`, hasher); `PackageReference Include="Microsoft.Data.SqlClient"`; drop `Microsoft.Data.Sqlite` from this csproj (it is transitive). Same `InternalsVisibleTo` pattern for the new test project. **Step 2: Failing tests** (`SqlServerApiKeyOptionsTests.cs`) ```csharp public sealed class SqlServerApiKeyOptionsTests { [Fact] public void Validate_RejectsMissingConnectionString() => Assert.Throws(() => new SqlServerApiKeyOptions { ConnectionString = " " }.Validate()); [Fact] public void Validate_RejectsUnsafeSchemaName() => Assert.Throws(() => new SqlServerApiKeyOptions { ConnectionString = "Server=x", Schema = "zb;auth" }.Validate()); [Fact] public void Schema_DefaultsToZbauth() => Assert.Equal("zbauth", new SqlServerApiKeyOptions().Schema); } ``` **Step 3: Implement** ```csharp namespace ZB.MOM.WW.Auth.ApiKeys.SqlServer; /// Options for the shared SQL Server API-key store (section supplied by the host, e.g. ScadaBridge:InboundApi:ApiKeyStore:SqlServer). public sealed class SqlServerApiKeyOptions { /// Full SQL Server connection string. Required. Never a ${secret:} reference on hosts whose secret store is itself SQL-backed. public string ConnectionString { get; set; } = string.Empty; /// Schema that owns the three tables. Letters, digits and underscore only (it is interpolated into DDL). public string Schema { get; set; } = "zbauth"; /// Fails fast on a blank connection string or an unsafe schema identifier. /// The options are not usable. public void Validate() { if (string.IsNullOrWhiteSpace(ConnectionString)) throw new InvalidOperationException("SQL Server API-key store requires a ConnectionString."); if (string.IsNullOrEmpty(Schema) || !Schema.All(c => char.IsAsciiLetterOrDigit(c) || c == '_') || Schema.Length > 64) throw new InvalidOperationException($"SQL Server API-key store schema name '{Schema}' is not a safe identifier."); } } ``` `AuthSqlServerConnectionFactory(SqlServerApiKeyOptions options)` mirrors `SecretsSqlServerConnectionFactory`: `Validate()` in the ctor, `OpenConnectionAsync(ct)` returning an opened `SqlConnection`, and a `Schema` property. **Step 4: Run** `dotnet test tests/ZB.MOM.WW.Auth.ApiKeys.SqlServer.Tests` → 3 passed. `dotnet build ZB.MOM.WW.Auth.slnx` → succeeded. **Step 5: Commit** ```bash git add ZB.MOM.WW.Auth git commit -m "feat(auth.apikeys.sqlserver): package skeleton, options (schema default zbauth, validated) and connection factory" ``` --- ### Task 3: Schema + migrator (concurrent-boot safe) **Classification:** high-risk (DDL run concurrently by both central nodes at every boot) **Estimated implement time:** ~5 min **Parallelizable with:** none **Model:** opus **Files:** - Create: `src/ZB.MOM.WW.Auth.ApiKeys.SqlServer/SqlServerAuthSchema.cs` - Create: `src/ZB.MOM.WW.Auth.ApiKeys.SqlServer/SqlServerApiKeyStoreMigrator.cs` - Create: `tests/ZB.MOM.WW.Auth.ApiKeys.SqlServer.Tests/Live/LiveSqlServerFixture.cs` (copy of Secrets' fixture, env var **`AUTH_SQLSERVER_CONNSTR`**, creates a throwaway database `zbauth_test_` per run and drops it on dispose) - Create: `tests/ZB.MOM.WW.Auth.ApiKeys.SqlServer.Tests/Live/SqlServerApiKeyStoreMigratorLiveTests.cs` **Step 1: Failing live tests** (all `[SkippableFact]` + `Skip.IfNot(LiveSqlServer.IsEnabled, …)` exactly as Secrets does) - `Migrate_OnEmptyDatabase_CreatesSchemaTablesAndVersionRow` — asserts `sys.schemas` has `zbauth`, the three tables exist, `schema_version.version == SqlServerAuthSchema.CurrentVersion`. - `Migrate_Twice_IsIdempotent`. - `Migrate_Concurrently_FromEightTasks_AllSucceed` — `Task.WhenAll` of 8 migrators on a virgin database; this is the case that bit Secrets live. - `Migrate_WhenStoredVersionIsNewer_Throws` — write `version = 99`, expect `AuthStoreMigrationException` (reuse the existing exception type from `SqliteAuthStoreMigrator.cs`). **Step 2: Run** with `AUTH_SQLSERVER_CONNSTR="Server=localhost,1433;User Id=sa;Password=;TrustServerCertificate=true"` (start `infra/` first: `docker compose -f ~/Desktop/ScadaBridge/infra/docker-compose.yml up -d scadabridge-mssql`). Expected: compile failure (types missing). Without the variable the tests must report **Skipped**, not failed. **Step 3: Implement** `SqlServerAuthSchema.CurrentVersion = 1`. DDL (`{schema}` interpolated from the validated option only): ```sql IF NOT EXISTS (SELECT 1 FROM sys.schemas WHERE name = N'{schema}') EXEC(N'CREATE SCHEMA [{schema}]'); IF OBJECT_ID(N'[{schema}].[schema_version]', N'U') IS NULL CREATE TABLE [{schema}].[schema_version] ( id INT NOT NULL CONSTRAINT PK_{schema}_schema_version PRIMARY KEY CHECK (id = 1), version INT NOT NULL, applied_utc DATETIMEOFFSET(7) NOT NULL); IF OBJECT_ID(N'[{schema}].[api_keys]', N'U') IS NULL CREATE TABLE [{schema}].[api_keys] ( key_id NVARCHAR(64) NOT NULL CONSTRAINT PK_{schema}_api_keys PRIMARY KEY, key_prefix NVARCHAR(16) NOT NULL, secret_hash VARBINARY(64) NOT NULL, display_name NVARCHAR(256) NOT NULL, scopes NVARCHAR(MAX) NOT NULL, constraints NVARCHAR(MAX) NULL, created_utc DATETIMEOFFSET(7) NOT NULL, last_used_utc DATETIMEOFFSET(7) NULL, revoked_utc DATETIMEOFFSET(7) NULL, expires_utc DATETIMEOFFSET(7) NULL); IF OBJECT_ID(N'[{schema}].[api_key_audit]', N'U') IS NULL CREATE TABLE [{schema}].[api_key_audit] ( audit_id BIGINT IDENTITY(1,1) NOT NULL CONSTRAINT PK_{schema}_api_key_audit PRIMARY KEY, key_id NVARCHAR(64) NULL, event_type NVARCHAR(64) NOT NULL, remote_address NVARCHAR(64) NULL, created_utc DATETIMEOFFSET(7) NOT NULL, details NVARCHAR(MAX) NULL); IF NOT EXISTS (SELECT 1 FROM sys.indexes WHERE name = N'ix_api_keys_revoked_utc' AND object_id = OBJECT_ID(N'[{schema}].[api_keys]')) CREATE INDEX ix_api_keys_revoked_utc ON [{schema}].[api_keys] (revoked_utc); IF NOT EXISTS (SELECT 1 FROM sys.indexes WHERE name = N'ix_api_key_audit_key_id_created_utc' AND object_id = OBJECT_ID(N'[{schema}].[api_key_audit]')) CREATE INDEX ix_api_key_audit_key_id_created_utc ON [{schema}].[api_key_audit] (key_id, created_utc); MERGE [{schema}].[schema_version] AS t USING (SELECT 1 AS id) AS s ON t.id = s.id WHEN NOT MATCHED THEN INSERT (id, version, applied_utc) VALUES (1, @version, SYSDATETIMEOFFSET()); ``` `SqlServerApiKeyStoreMigrator : IApiKeyStoreMigrator` — port `SqlServerSecretsStoreMigrator` **including its retry loop and the four error numbers verbatim**, its "stored version newer than supported → throw" check, and its `MigrateOnceAsync` structure (Serializable transaction). Keep the class comment that explains why the retry exists (cite the 2026-08-07 central-pair first boot). Also ship the same DDL as a standalone file for DBAs: `docs/sql/zbauth-schema.sql` in the library repo (plain `zbauth`, `@version` replaced by the literal), referenced from the README. **Step 4: Run** the live tests with the env var → 4 passed; without → 4 skipped. `dotnet test ZB.MOM.WW.Auth.slnx` still green. **Step 5: Commit** ```bash git commit -am "feat(auth.apikeys.sqlserver): zbauth schema v1 + concurrent-boot-safe migrator (retry on 1205/2714/1913/2627, newer-version fail-closed); live tests gated on AUTH_SQLSERVER_CONNSTR" ``` --- ### Task 4: The three SQL Server stores (contract tests shared with SQLite) **Classification:** high-risk (auth data path) **Estimated implement time:** ~5 min per store — split into 4a store, 4b admin store, 4c audit store if the executor's clock says so **Parallelizable with:** none **Model:** opus **Files:** - Create: `src/ZB.MOM.WW.Auth.ApiKeys.SqlServer/SqlServerApiKeyStore.cs` - Create: `src/ZB.MOM.WW.Auth.ApiKeys.SqlServer/SqlServerApiKeyAdminStore.cs` - Create: `src/ZB.MOM.WW.Auth.ApiKeys.SqlServer/SqlServerApiKeyAuditStore.cs` - Create: `tests/ZB.MOM.WW.Auth.ApiKeys.SqlServer.Tests/Live/SqlServerApiKeyStoreLiveTests.cs`, `SqlServerApiKeyAdminStoreLiveTests.cs`, `SqlServerApiKeyAuditStoreLiveTests.cs` **Step 1: Failing tests** — port every `[Fact]` in `tests/ZB.MOM.WW.Auth.ApiKeys.Tests/SqliteApiKeyStoreTests.cs` and `SqliteApiKeyAdminStoreTests.cs` one-for-one to `[SkippableFact]`s against the live fixture (same names, same assertions, `SampleRecord`/`AssertRecordEqual` helpers copied). Add for the audit store: `Append_ThenListRecent_ReturnsNewestFirstWithinLimit`. Add one SQL-specific case: `FindByKeyId_RoundTripsDateTimeOffsetToTheTick` (SQLite stores text; SQL Server must not truncate to milliseconds). **Step 2: Run** → compile failure. **Step 3: Implement** — parameterised `SqlCommand`s, one connection per operation via the factory, `ConfigureAwait(false)`, `ScopeSerializer` for `scopes`, `DateTimeOffset` bound as `SqlDbType.DateTimeOffset`. Semantics **must match SQLite exactly** — read each SQLite method before porting: - `FindActiveByKeyIdAsync` = `revoked_utc IS NULL` (expiry is the verifier's job, not the store's). - `MarkUsedAsync` = `UPDATE … SET last_used_utc = @when WHERE key_id = @id` (no row → no-op). - `CreateAsync` = plain `INSERT` (a duplicate `key_id` throws, as SQLite does). - `RevokeAsync`/`SetEnabledAsync(false)` set `revoked_utc = @when` only when currently NULL and return whether a row changed; `SetEnabledAsync(true)` clears it. `RotateAsync` updates `secret_hash` only (preserves `expires_utc`). `DeleteAsync` returns rows-affected > 0. `ListAsync` ordered by `created_utc, key_id`. - Audit `ListRecentAsync(limit)` = `SELECT TOP (@limit) … ORDER BY audit_id DESC`. **Step 4: Run** live tests → all pass; SQLite suite unchanged. **Step 5: Commit** — `feat(auth.apikeys.sqlserver): IApiKeyStore/IApiKeyAdminStore/IApiKeyAuditStore over SQL Server, semantics pinned by the SQLite contract tests re-run live` --- ### Task 5: One-shot SQLite → SQL Server importer **Classification:** standard **Estimated implement time:** ~5 min **Parallelizable with:** Task 6 **Model:** opus **Files:** - Create: `src/ZB.MOM.WW.Auth.ApiKeys.SqlServer/SqliteApiKeyStoreImporter.cs` - Create: `tests/ZB.MOM.WW.Auth.ApiKeys.SqlServer.Tests/Live/SqliteApiKeyStoreImporterLiveTests.cs` **Step 1: Failing tests** - `Import_CopiesKeysAndAuditRows_AndReturnsCounts` — seed a temp SQLite via `SqliteAuthStoreMigrator` + `SqliteApiKeyAdminStore` (2 keys, one revoked, one with expiry) + 3 audit rows; import; assert SQL Server `ListAsync` shows both with identical fields and audit count 3. - `Import_Twice_IsIdempotent` — second run returns `Inserted = 0`, `Skipped = 2`. - `Import_UnionsWithExistingRows` — SQL already has key C; SQLite has A and B; result has A, B, C and no row of C was modified (compare `last_used_utc`). - `Import_MissingFile_ReturnsZeroWithoutThrowing`. **Step 2: Run** → compile failure. **Step 3: Implement** ```csharp /// Result of a one-shot import. public sealed record ApiKeyImportResult(int KeysInserted, int KeysSkipped, int AuditInserted, int AuditSkipped, bool SourceFound); /// /// Copies every key and audit row from a per-node SQLite store into the shared SQL Server store, /// insert-where-not-exists (keys by key_id; audit rows by key_id + event_type + created_utc), so it is /// idempotent and every node of a pair may run it: the result is the UNION of the nodes' files. A row /// that already exists in SQL Server is never modified — a revoke recorded on one node but not the /// other therefore survives only if the revoked copy is imported first; operators reconcile by listing. /// public sealed class SqliteApiKeyStoreImporter(AuthSqlServerConnectionFactory sql) { public async Task ImportAsync(string sqlitePath, CancellationToken ct) { … } } ``` Read the SQLite side with a raw `SqliteConnection` (`Mode=ReadOnly`) and the column list from `SqliteApiKeyStore`, parse timestamps exactly like `SqliteValueParsing` (make that class `internal` + `InternalsVisibleTo` the SqlServer project, or duplicate the two-line parser — prefer `InternalsVisibleTo`). Insert with `IF NOT EXISTS (SELECT 1 … WHERE key_id = @id) INSERT …` per row inside one transaction. **Step 4: Run** → 4 passed live. **Step 5: Commit** — `feat(auth.apikeys.sqlserver): SqliteApiKeyStoreImporter — idempotent insert-where-not-exists copy of keys + audit (union across a pair)` --- ### Task 6: DI extension `AddZbApiKeyAuthSqlServer` + README/changelog + version 0.3.0 **Classification:** standard **Estimated implement time:** ~5 min **Parallelizable with:** Task 5 **Model:** sonnet **Files:** - Create: `src/ZB.MOM.WW.Auth.ApiKeys.SqlServer/DependencyInjection/SqlServerApiKeyServiceCollectionExtensions.cs` - Test: `tests/ZB.MOM.WW.Auth.ApiKeys.SqlServer.Tests/SqlServerApiKeyServiceCollectionExtensionsTests.cs` (no live DB needed — resolves types only) - Modify: `Directory.Build.props` (`0.2.1` → `0.3.0`), `README.md` (Packages table row, Consumer Matrix column, "current release" line, new `### 0.3.0` changelog entry above `### 0.1.4`) **Step 1: Failing test** ```csharp [Fact] public void AddZbApiKeyAuthSqlServer_DisplacesSqliteStoresAndMigrator() { var cfg = new ConfigurationBuilder().AddInMemoryCollection(new Dictionary { ["ApiKeys:TokenPrefix"] = "sbk", ["ApiKeys:PepperSecretName"] = "Pepper", ["Pepper"] = "0123456789abcdef", ["ApiKeys:SqlServer:ConnectionString"] = "Server=unused;TrustServerCertificate=true" }).Build(); var services = new ServiceCollection(); services.AddZbApiKeyAuthSqlServer(cfg, "ApiKeys", "ApiKeys:SqlServer"); using var sp = services.BuildServiceProvider(); Assert.IsType(sp.GetRequiredService()); Assert.IsType(sp.GetRequiredService()); Assert.IsType(sp.GetRequiredService()); Assert.IsType(sp.GetRequiredService()); Assert.NotNull(sp.GetRequiredService()); Assert.NotNull(sp.GetRequiredService()); } ``` **Step 3: Implement** ```csharp public static IServiceCollection AddZbApiKeyAuthSqlServer( this IServiceCollection services, IConfiguration config, string apiKeySectionPath, string sqlServerSectionPath) { services.Configure(config.GetSection(sqlServerSectionPath)); services.TryAddSingleton(sp => new AuthSqlServerConnectionFactory(sp.GetRequiredService>().Value)); // Registered BEFORE AddZbApiKeyAuth so its TryAdd calls find these and leave them alone. services.TryAddSingleton(sp => new SqlServerApiKeyStore(sp.GetRequiredService())); services.TryAddSingleton(sp => new SqlServerApiKeyAdminStore(sp.GetRequiredService())); services.TryAddSingleton(sp => new SqlServerApiKeyAuditStore(sp.GetRequiredService())); services.TryAddSingleton(); services.TryAddSingleton(sp => sp.GetRequiredService()); services.TryAddSingleton(); return services.AddZbApiKeyAuth(config, apiKeySectionPath); } ``` Note `AddZbApiKeyAuth` still registers `AuthSqliteConnectionFactory` from `ApiKeyOptions.SqlitePath` — harmless (lazy, never opened) and it keeps the importer's source path in one option. Document that in the extension's XML comment. README changelog entry `### 0.3.0 — SQL Server API-key store (ScadaBridge central pair)`: the abstraction, the new package, `AddZbApiKeyAuthSqlServer`, the importer, "SQLite remains default; `ApiKeyAdminCommands` old ctor retained; consumers on SQLite need no code change to move to 0.3.0", and the `docs/sql/zbauth-schema.sql` pointer. Fix "current release is **0.1.4**" → **0.3.0**. **Step 4: Run** `dotnet test ZB.MOM.WW.Auth.slnx` green; `bash build/pack.sh` produces five `0.3.0` nupkgs in `artifacts/`. **Step 5: Commit** — `feat(auth)!: 0.3.0 — ZB.MOM.WW.Auth.ApiKeys.SqlServer package (AddZbApiKeyAuthSqlServer, importer), README/changelog` (the `!` is for the new package/version, not an API break). --- ### Task 7: Publish 0.3.0 to the Gitea feed + umbrella index **Classification:** trivial (but outward-facing — **confirm with the user before pushing**) **Estimated implement time:** ~3 min **Parallelizable with:** none **Model:** sonnet (the push itself is the user's call) **Files:** - Modify: `~/Desktop/scadaproj/CLAUDE.md` (Auth row in the shared-libs table: feed carries `0.3.0`, new `ApiKeys.SqlServer` package, ScadaBridge is its first consumer) **Steps:** `export GITEA_NUGET_SOURCE=… GITEA_NUGET_KEY=…; bash build/push.sh` (needs the user's token — ask, do not guess). Verify: `dotnet nuget list source` then `curl -s "$GITEA_NUGET_SOURCE" | head -c 200` and `dotnet package search ZB.MOM.WW.Auth.ApiKeys.SqlServer --source dohertj2-gitea` from the ScadaBridge checkout shows `0.3.0`. Commit the index change in scadaproj. --- ## Part B — ScadaBridge (`~/Desktop/ScadaBridge`) ### Task 8: Pin 0.3.0 and add `ApiKeyStore:Provider` options **Classification:** small **Estimated implement time:** ~4 min **Parallelizable with:** none **Model:** sonnet **Files:** - Modify: `Directory.Packages.props:101-104` (four pins → `0.3.0`, add ``) - Modify: `src/ZB.MOM.WW.ScadaBridge.Host/ZB.MOM.WW.ScadaBridge.Host.csproj` (add ``) - Create: `src/ZB.MOM.WW.ScadaBridge.Host/InboundApiKeyStoreOptions.cs` - Test: `tests/ZB.MOM.WW.ScadaBridge.Host.Tests/InboundApiKeyStoreOptionsTests.cs` **Step 1: Failing tests** — `Provider_DefaultsToSqlite`, `Provider_ParsesCaseInsensitively("sqlserver" → SqlServer)`, `Provider_RejectsUnknownValue` (validation error names the two legal values). **Step 3: Implement** ```csharp namespace ZB.MOM.WW.ScadaBridge.Host; /// Which backend holds inbound API keys. Bound from ScadaBridge:InboundApi:ApiKeyStore alongside the library's own ApiKeyOptions. public enum InboundApiKeyStoreProvider { Sqlite, SqlServer } /// Host-side selector for the inbound API key store. Sqlite is per-node (single-node installs); SqlServer shares one row set across a central pair (design 2026-09-02). public sealed class InboundApiKeyStoreOptions { public const string SectionPath = "ScadaBridge:InboundApi:ApiKeyStore"; public InboundApiKeyStoreProvider Provider { get; set; } = InboundApiKeyStoreProvider.Sqlite; /// SQLite file to import once when is SqlServer; defaults to the library's SqlitePath. Renamed *.imported afterwards. public string? ImportFromSqlitePath { get; set; } } ``` Bind with `ValidateOnStart` (enum binding already rejects unknown names; add a `Validate` that produces the friendly message). **Step 4:** `dotnet build ZB.MOM.WW.ScadaBridge.slnx` (restores 0.3.0 — Task 7 must have pushed) + the new tests green. **Step 5: Commit** — `chore(host): pin ZB.MOM.WW.Auth 0.3.0 + ApiKeys.SqlServer; InboundApiKeyStoreOptions (Provider Sqlite|SqlServer)` --- ### Task 9: Compose the provider in `Program.cs` + import hosted service **Classification:** high-risk (auth composition on central) **Estimated implement time:** ~5 min **Parallelizable with:** none **Model:** opus **Files:** - Modify: `src/ZB.MOM.WW.ScadaBridge.Host/Program.cs:427-470` - Create: `src/ZB.MOM.WW.ScadaBridge.Host/InboundApiKeyStoreImportService.cs` - Test: `tests/ZB.MOM.WW.ScadaBridge.Host.Tests/CompositionRootTests.cs` (extend the existing central-composition test: with `Provider=SqlServer` and a dummy connection string, `IApiKeyStore` resolves to `SqlServerApiKeyStore` and `IApiKeyStoreMigrator` to the SQL migrator; with the default, both resolve to the SQLite types) - Test: `tests/ZB.MOM.WW.ScadaBridge.Host.Tests/InboundApiKeyStoreImportServiceTests.cs` (fake importer: file present → `ImportAsync` called once and file renamed `.imported`; file absent → not called; `Provider=Sqlite` → service inert) **Step 3: Implement** (`Program.cs`, replacing lines 447-450 and the `ApiKeyAdminCommands` migrator argument): ```csharp var storeOptions = builder.Configuration.GetSection(InboundApiKeyStoreOptions.SectionPath).Get() ?? new(); if (storeOptions.Provider == InboundApiKeyStoreProvider.SqlServer) { // Shared store for a central pair (design 2026-09-02). The connection string defaults to the // configuration database — the pair already shares it — under its own `zbauth` schema. const string sqlSection = apiKeyStoreSection + ":SqlServer"; if (string.IsNullOrWhiteSpace(builder.Configuration[$"{sqlSection}:ConnectionString"])) builder.Configuration.AddInMemoryCollection(new Dictionary { [$"{sqlSection}:ConnectionString"] = builder.Configuration["ScadaBridge:Database:ConfigurationDb"] }); builder.Services.AddZbApiKeyAuthSqlServer(builder.Configuration, apiKeyStoreSection, sqlSection); builder.Services.AddHostedService(); } else { builder.Services.AddZbApiKeyAuth(builder.Configuration, apiKeyStoreSection); } … sp.GetRequiredService(), // was SqliteAuthStoreMigrator ``` `InboundApiKeyStoreImportService : IHostedService` — registered **after** `AddZbApiKeyAuthSqlServer` so it starts after the library's migration hosted service (hosted services start in registration order); resolves `SqliteApiKeyStoreImporter`, `IOptions`, `IOptions` (for the default path), `ILogger`. `StartAsync`: path = `ImportFromSqlitePath ?? ApiKeyOptions.SqlitePath`; if `File.Exists` → `ImportAsync`, log Information with the counts, `File.Move(path, path + ".imported")` (also move `-wal`/`-shm` if present). Any exception → log **Error** and continue (keys already in SQL keep working; a failed import must not take the node down). Keep the `${secret:}` caveat from the docker compose comment: the connection string must be a literal or an env override, never a secret reference resolved by the SQL-backed secrets store (circularity). **Step 4:** `dotnet test tests/ZB.MOM.WW.ScadaBridge.Host.Tests` green; `dotnet test ZB.MOM.WW.ScadaBridge.slnx` green (the InboundAPI and Security tests still construct `ApiKeyAdminCommands` with the SQLite migrator — the retained ctor keeps them compiling; do not rewrite them). **Step 5: Commit** — `feat(host): inbound API key store provider — SqlServer shares one zbauth row set across the central pair; one-shot SQLite import at boot (file renamed .imported)` --- ### Task 10: `StartupValidator` warning for a clustered central on the per-node store **Classification:** small **Estimated implement time:** ~4 min **Parallelizable with:** Task 11, Task 12 **Model:** sonnet **Files:** - Modify: `src/ZB.MOM.WW.ScadaBridge.Host/StartupValidator.cs` (near the existing inbound-API pepper check at ~line 80) - Test: `tests/ZB.MOM.WW.ScadaBridge.Host.Tests/StartupValidatorTests.cs` Rule: Central role AND `ScadaBridge:Cluster:SeedNodes` (whatever key the validator already reads for the self-first check) lists ≥ 2 nodes AND `Provider == Sqlite` → **Warning** `"Inbound API keys use the per-node SQLite store on a {n}-node central cluster; a failover will reject every inbound client until keys are re-issued. Set ScadaBridge:InboundApi:ApiKeyStore:Provider=SqlServer (design 2026-09-02)."` Not an error (availability-first, like auto-down). Tests: warns for 2 seeds + Sqlite; silent for 1 seed; silent for SqlServer. Commit — `feat(host): StartupValidator warns when a clustered central keeps per-node inbound API keys` --- ### Task 11: Docker rig on the shared store; retire the file-copy workaround **Classification:** standard **Estimated implement time:** ~5 min (+ ~3 min rebuild) **Parallelizable with:** Task 10, Task 12 **Model:** opus **Files:** - Modify: `docker/docker-compose.yml` (both central `environment:` blocks: `ScadaBridge__InboundApi__ApiKeyStore__Provider: "SqlServer"` with a comment pointing at the design; connection string deliberately NOT set — falls back to `ScadaBridge__Database__ConfigurationDb`) - Modify: `docker-env2/docker-compose.yml` (same for its two centrals — that rig is also a pair) - Delete: `docker/sync-inbound-api-keys.sh` - Modify: `docker/verify-delmia-notifier.sh` (header lines 9-13 + the preflight: keep the "both nodes list the key" check — it is now the *proof*, not a gate that needs a script — and reword the comment) - Modify: `docker/README.md` (remove the sync script line if listed; note the provider) **Steps:** 1. Edit compose + scripts. `bash docker/deploy.sh` (rebuilds the image with 0.3.0 and recreates the pair). 2. Watch both central logs for the import: `docker logs scadabridge-central-a 2>&1 | grep -i "api-key store import\|zbauth"` — expect one node reporting `KeysInserted=1` (the `delmia-notifier` key) and the other `KeysInserted=0, KeysSkipped=1` (both files hold the same key after the earlier sync), and `inbound-api-keys.sqlite.imported` on both volumes (`docker exec scadabridge-central-a ls /app/data`). 3. Confirm one row set: `docker exec scadabridge-mssql /opt/mssql-tools18/bin/sqlcmd -C -S localhost -U sa -P 'ScadaBridge_Dev1#' -d ScadaBridgeConfig -Q "SELECT key_id, display_name, revoked_utc FROM zbauth.api_keys"` (adapt the tools path to what the image has). Both `scadabridge security api-key list` via `:9001` and `:9002` show the same keys. 4. **Cluster-wide revoke proof:** `security api-key create --name revoke-probe --methods DelmiaRecipeDownload` via `:9001`; `curl` the method through `:9002` with it → 200; `security api-key update --key-id --enabled false` via `:9001`; the same curl via `:9002` → **401**. Then `delete` it. 5. Run `export DELMIA_API_KEY=…; bash docker/verify-delmia-notifier.sh` **without any file copy** → A, B, E, F `YES`; C, G `NO`. Record the takeover seconds. 6. `docker-env2`: `bash docker-env2/deploy.sh`, confirm both of its centrals boot and one imports. Commit — `chore(docker): both rigs' centrals on the shared SQL Server inbound key store; delete sync-inbound-api-keys.sh (superseded); verify script proves both nodes see the key without copying` --- ### Task 12: Documentation + register + umbrella index **Classification:** trivial (docs only, but wide) **Estimated implement time:** ~5 min **Parallelizable with:** Task 10, Task 11 **Model:** sonnet **Files:** - Modify: `docs/requirements/Component-InboundAPI.md` §Storage (lines 23-24: keys live in the `ZB.MOM.WW.Auth.ApiKeys` store — SQLite per node by default, or the shared `zbauth` schema in the configuration database on a central pair via `Provider=SqlServer`; revoke/rotate/scope are cluster-wide only under SqlServer) and the Dependencies line 309. - Modify: `docs/operations/inbound-api-key-reissue.md` §2 config table (add `Provider`, `SqlServer:ConnectionString` default, `ImportFromSqlitePath`) + a new §"Central pairs" paragraph: stop both together, set `Provider`, start both, check the import log line on each, `security api-key list` from each node, then delete the `.imported` files after a quarantine. - Create: `docs/operations/sql/zbauth-schema.sql` (copy of the library's DDL for DBAs whose app login lacks DDL rights; the migrator then finds the version row and skips). - Modify: `docs/deployment/topology-guide.md` (central-pair section: the key store is shared under SqlServer; SQLite on a pair is a documented misconfiguration the validator warns about). - Modify: `env_variables.md` (rows for `ScadaBridge__InboundApi__ApiKeyStore__Provider`, `…__SqlServer__ConnectionString`, `…__ImportFromSqlitePath`). - Modify: `CLAUDE.md` → *Security & Auth*: one bullet — inbound key store per node by default; central pairs MUST run `Provider=SqlServer` (shared `zbauth` schema in the config DB, import once at boot, revoke cluster-wide); rig does; QA/prod appsettings are user-held (deployment record). - Modify: `docs/plans/2026-07-08-deferred-work-register.md` row 108 → resolved (library 0.3.0 + host provider + rig proof; QA/vd03 rollout pending as a deployment record). - Modify: `docs/api-specs/Delmia-Recipe-Download-API.md` §8 (the note about the sync script → superseded). - Modify: `../scadaproj/CLAUDE.md` ScadaBridge entry (inbound keys on the shared store; Auth 0.3.0 consumer). Check: `grep -rn "sync-inbound-api-keys" docs docker CLAUDE.md` → only historical mentions in the register/plan. Commit — `docs: inbound API key store — shared SQL Server provider on central pairs (spec, runbook, topology, env inventory, CLAUDE.md, register row 108 closed)` --- ### Task 13: Deployment record template for QA (vq03/vq04) and prod (vd03) **Classification:** trivial **Estimated implement time:** ~3 min **Parallelizable with:** none (after Task 12) **Model:** sonnet **Files:** - Create: `docs/deployment/deployment-records/2026-09-XX-zb-servers-inbound-key-store.md` (a **planned** record with the exact steps and blanks for results — the user runs it; appsettings on those boxes are user-held) Content: build/stage per the 2026-08-21 recipe; **QA:** stop both centrals together → add `"ApiKeyStore": { "Provider": "SqlServer" }` under `ScadaBridge:InboundApi` in both `appsettings.Central.json` → if `scadabridge_app` lacks DDL rights run `docs/operations/sql/zbauth-schema.sql` on the QA config DB first → start both → expect one `KeysInserted=N` line and one `KeysSkipped` line → `security api-key list` from `10.220.158.2:8085` and `.3:8085` agree → `sync`-free failover check with a curl against `DelmiaRecipeDownload` from the non-active node. **Prod vd03:** single node, same change (`KeysInserted=5` expected: `mes`, `mes api key`, `delmia-mes-receivers`, `TestKey`, `JOE_TEST_KEY`). Rollback: revert `Provider`, rename `.imported` back — no SQL rows need removing (the SQLite path ignores them). Commit — `docs(deployment): planned record for moving QA/prod central inbound key stores to SQL Server` --- ### Task 14: Final cross-repo review **Classification:** standard **Estimated implement time:** ~5 min **Parallelizable with:** none **Model:** **fable** (this session) Check: (1) scadaproj `git log` shows Tasks 1-7 committed and `artifacts/` holds five `0.3.0` packages that match the feed; (2) ScadaBridge `git diff main` contains no connection string or token (`grep -n "Password=\|sbk_[A-Za-z0-9]" ` → only the dev-only compose values that were already committed); (3) `dotnet test ZB.MOM.WW.ScadaBridge.slnx` green; (4) Task 11 step 4 (revoke visible on the peer) and step 5 (drill without copying) are recorded with numbers; (5) MxAccessGateway and HistorianGateway still build against 0.3.0 without edits (`dotnet build` each after bumping their pins in a scratch branch — do **not** commit there; report only); (6) the rig's `sync-inbound-api-keys.sh` is gone and nothing references it. Then ask the user about pushing both repos. --- ## Out of scope - Site nodes, MxAccessGateway, HistorianGateway (SQLite stays their store). - Hard-failing a clustered central on `Provider=Sqlite` (warning only; revisit after QA is on SqlServer). - Encrypting `secret_hash` at rest; the pepper never enters SQL. - Fixing register row 107 (prod `DelmiaRecv`) — separate change.