Files
scadaproj/docs/plans/2026-07-16-secrets-adoption-design.md
T
Joseph Doherty 6be5f746d5 docs(secrets): G-2..G-6 adoption design + 3 per-repo plans
Shared design (two resolution layers, wiring template, master-key/clustering
fork) + per-repo executable plans with code-verified anchors and co-located
.tasks.json:
  - mxaccessgw (G-4/G-5/G-6, 8 tasks)
  - otopcua    (G-2/G-4/G-5/G-6, 10 tasks, 2 slices)
  - scadabridge (G-3/G-4/G-5/G-6, 10 tasks, 2 slices)
Linked from components/secrets/GAPS.md.
2026-07-16 09:19:33 -04:00

321 lines
20 KiB
Markdown

# Secrets Adoption (G-2 … G-6) — Shared Design
> **For Claude:** This is the shared design that the three per-repo implementation plans
> depend on. It is NOT itself executed. The executable plans are:
> - [`2026-07-16-secrets-adoption-otopcua.md`](2026-07-16-secrets-adoption-otopcua.md) — G-2, G-4, G-5, G-6
> - [`2026-07-16-secrets-adoption-scadabridge.md`](2026-07-16-secrets-adoption-scadabridge.md) — G-3, G-4, G-5, G-6
> - [`2026-07-16-secrets-adoption-mxaccessgw.md`](2026-07-16-secrets-adoption-mxaccessgw.md) — G-4, G-5, G-6
**Goal:** Adopt the already-built, already-published `ZB.MOM.WW.Secrets` library
(envelope-encrypted secrets manager + `${secret:}` config expander + runtime `ISecretResolver`
+ `/admin/secrets` Blazor UI) into the three remaining family apps — OtOpcUa, ScadaBridge,
MxAccessGateway — retiring plaintext credentials at rest.
**Architecture:** No library construction — the lib is built, published to the Gitea feed at
`0.1.2`, and **reference-consumer-proven** in HistorianGateway (live wonder end-to-end,
2026-07-16). Every task below is *per-app wiring* of an existing package, following the
HistorianGateway template verbatim. The gaps map to two distinct resolution layers (below).
**Tech Stack:** .NET 10, C#, `ZB.MOM.WW.Secrets{,.Abstractions,.Ui}` @ `0.1.2` from the Gitea
NuGet feed, AES-256-GCM envelope crypto, SQLite store, ASP.NET Core authorization + Blazor RCL.
**Backlog source:** [`components/secrets/GAPS.md`](../../components/secrets/GAPS.md). Per-app
baselines: [`components/secrets/current-state/`](../../components/secrets/current-state/).
---
## 1. What is being adopted — the library API surface
Verified against `scadaproj/ZB.MOM.WW.Secrets/` (the source of truth). These are the exact
identifiers every plan calls; do not guess variants.
| Concern | Type / member | Notes |
|---|---|---|
| DI registration | `SecretsServiceCollectionExtensions.AddZbSecrets(this IServiceCollection, IConfiguration config, string sectionPath)` | **One overload only.** Both the pre-host throwaway provider and the runtime registration call this identically, section `"Secrets"`. |
| Pre-host expander | `SecretReferenceExpander(ISecretResolver)``Task ExpandConfigurationAsync(IConfigurationRoot config, CancellationToken ct)` | Rewrites `${secret:name}` values **in place** via the `IConfigurationRoot` indexer. Fail-closed: unknown ref throws `SecretNotFoundException`. **Skips** any key whose leaf segment (after the last `:`) starts with `_` (the `_comment` convention). |
| Store migrator | `SqliteSecretsStoreMigrator.MigrateAsync(CancellationToken)` | Idempotent (`CREATE … IF NOT EXISTS`). `AddZbSecrets` also registers `SecretsMigrationHostedService` to run it at startup, but the **pre-host** path must call `MigrateAsync` explicitly before the first resolve. |
| Runtime resolver | `ISecretResolver.GetAsync(SecretName name, CancellationToken ct)``Task<string?>` | **`GetAsync`, not `ResolveAsync`.** Returns decrypted plaintext, or `null` if absent/tombstoned. This is the seam G-2/G-3 driver-secret resolution calls. |
| Secret key type | `readonly record struct SecretName``new SecretName("some/name")` | Normalizes `Trim().ToLowerInvariant()`; validates allow-list `[a-z0-9._/-]`, no `..`, no leading/trailing/`//`. Implicit conversion **to** `string` only. |
| Authorization | `SecretsAuthorization.AddSecretsAuthorization(this AuthorizationOptions)`**in the `.Ui` package** | Adds policies `secrets:manage` (`ManagePolicy`) + `secrets:reveal` (`RevealPolicy`). `AdminRole = nameof(CanonicalRole.Administrator)` == `"Administrator"`, so an existing Administrator satisfies both. Additive — composes with existing `Configure<AuthorizationOptions>` callbacks. |
| UI page | `ZB.MOM.WW.Secrets.Ui.SecretsPage``@page "/admin/secrets"`, `[Authorize(Policy = ManagePolicy)]` | Mount handle: `typeof(ZB.MOM.WW.Secrets.Ui.SecretsPage).Assembly`. RCL depends on `ZB.MOM.WW.Theme` + `ZB.MOM.WW.Auth.AspNetCore` + `ZB.MOM.WW.Audit`. |
| Master key | `IMasterKeyProvider` via `MasterKeyProviderFactory.Create(MasterKeyOptions)` | `MasterKeySource { Environment, File, Dpapi }`; default `Environment`, env var `ZB_SECRETS_MASTER_KEY` (base64 32 bytes). File/DPAPI use `FilePath`; optional explicit `KekId`. |
| Options | `SecretsOptions` bound from the `"Secrets"` section | `SqlitePath` (default `secrets.db`), `MasterKey`, `RunMigrationsOnStartup` (default true), `ResolveCacheTtl` (default 30 s). |
The `Secrets` appsettings block (matches HistorianGateway):
```json
"Secrets": {
"SqlitePath": "<app>-secrets.db",
"MasterKey": { "Source": "Environment", "EnvVarName": "ZB_SECRETS_MASTER_KEY" },
"RunMigrationsOnStartup": true,
"ResolveCacheTtl": "00:00:30"
}
```
---
## 2. The two resolution layers — the load-bearing distinction
The gaps split cleanly across **two different layers** that must not be conflated. This is the
single most important design point, surfaced by the OtOpcUa recon (driver secrets are *not*
`IConfiguration`).
### Layer A — pre-host config expansion (`${secret:}`) — covers G-4 (all apps) + parts of G-2/G-3
Secrets that live in `IConfiguration` (appsettings / env): LDAP passwords, SQL connection
strings, JWT signing keys, deploy/API keys, peppers. These are rewritten **once, before the
host is built**, by a throwaway bootstrap provider — the HistorianGateway template (§3). A
config value becomes the literal token `"${secret:sql/<app>/db-password}"`; the expander
replaces it with the decrypted value before any options validator binds.
### Layer B — runtime resolution (`ISecretResolver.GetAsync`) — covers G-2, G-3
Secrets that live **in a database row as data**, not in config: OtOpcUa's per-driver
`DriverConfig` JSON (Galaxy API key, OpcUaClient `Password`/`UserCertificatePassword`), and
ScadaBridge's MxGateway per-endpoint `ApiKey` inside the `DataConnection` JSON blob. The
pre-host expander never sees these — they are read at driver-instantiation / connection time.
The stored value becomes a `secret:<name>` reference; the **consuming code** calls
`ISecretResolver.GetAsync(new SecretName(name), ct)` at the point of use.
> **Rule of thumb:** if the secret is in appsettings/env → Layer A. If it is a column/JSON
> field in the app's own database → Layer B. G-4 is entirely Layer A. G-2 and G-3 are Layer B.
> G-6 (UI) and G-5 (master key) underpin both.
---
## 3. The proven wiring template (from HistorianGateway `Program.cs`)
Every app reuses these four pieces. Insertion points differ per app (see each plan); the code
shape is identical.
**(a) Pre-host `${secret:}` expansion** — before `builder.Build()` / before any options bind:
```csharp
#pragma warning disable ASP0000 // deliberate throwaway container, disposed here, shares no singletons
await using (var secretsProvider = new ServiceCollection()
.AddZbSecrets(builder.Configuration, "Secrets")
.BuildServiceProvider())
#pragma warning restore ASP0000
{
await secretsProvider.GetRequiredService<SqliteSecretsStoreMigrator>().MigrateAsync(default);
var resolver = secretsProvider.GetRequiredService<ISecretResolver>();
await new SecretReferenceExpander(resolver)
.ExpandConfigurationAsync((IConfigurationRoot)builder.Configuration, default);
}
```
Usings: `ZB.MOM.WW.Secrets.Abstractions`, `.Configuration`, `.DependencyInjection`, `.Sqlite`.
Note the cast to `IConfigurationRoot`. **ScadaBridge differs** — it assembles config in a bare
`ConfigurationBuilder` (not `WebApplication.CreateBuilder`) *before* the host exists, so its
expander runs against that `IConfigurationRoot` local, still via the same throwaway provider.
**(b) Runtime registration** — on the real host container (backs Layer B + the UI):
```csharp
builder.Services.AddZbSecrets(builder.Configuration, "Secrets");
```
**(c) Authorization** — additive:
```csharp
builder.Services.Configure<AuthorizationOptions>(o => o.AddSecretsAuthorization());
```
**(d) UI mount** — register the RCL assembly in **both** places or `/admin/secrets` 404s:
```csharp
// Router.razor / Routes.razor:
// AdditionalAssemblies="@(new[] { typeof(ZB.MOM.WW.Secrets.Ui.SecretsPage).Assembly })"
endpoints.MapRazorComponents<App>()
.AddInteractiveServerRenderMode()
.AddAdditionalAssemblies(typeof(ZB.MOM.WW.Secrets.Ui.SecretsPage).Assembly);
```
**Package plumbing (every app):** add `ZB.MOM.WW.Secrets`, `.Abstractions`, `.Ui` as
`PackageReference`s at `0.1.2`; add nuget.config `packageSourceMapping` patterns
`ZB.MOM.WW.Secrets` and `ZB.MOM.WW.Secrets.*` under the `dohertj2-gitea` source (none of the
three has a `ZB.MOM.WW.*` wildcard — the mapping is explicit, so restore fails without these).
ScadaBridge additionally needs `<PackageVersion>` rows in `Directory.Packages.props` (CPM).
---
## 4. G-4 — pre-host config-secret expansion (all three apps)
Same Layer-A mechanism (§3a); per-app insertion point and target keys:
| App | Insert expander at | Runs before | Config-secret targets |
|---|---|---|---|
| **OtOpcUa** | after `Host/Program.cs:73` (role overlay done, env/cmdline re-appended) | `AddOtOpcUaConfigDb` (`:94`) + first `ValidateOnStart` (`:102`) | `Security:Jwt:SigningKey`, `Security:Ldap:ServiceAccountPassword`, `Security:DeployApiKey`, `ConfigDb` connstr, `ServerHistorian:ApiKey` |
| **ScadaBridge** | between `Host/Program.cs:43` (`.Build()`) and `:46` (`StartupValidator.Validate`) | `StartupValidator`/`ConfigPreflight` (`:46`) | `Database:ConfigurationDb`, `Database:MachineDataDb`, `Security:Ldap:ServiceAccountPassword`, `Security:JwtSigningKey`, `InboundApi:ApiKeyPepper`**plus** the docker-per-node committed plaintext (see its plan) |
| **mxaccessgw** | after `GatewayApplication.cs:64` (`CreateBuilder`), before `:67` (TLS) | `AddGatewayConfiguration` (`:71`), `AddZbGalaxyRepository` (`:103`), `AddZbApiKeyAuth` | `MxGateway:Ldap:ServiceAccountPassword`, `MxGateway:Galaxy:ConnectionString`, `MxGateway:ApiKeyPepper` |
Each plan also **deletes the committed dev plaintext** (and the loose `*_login.txt` files in
ScadaBridge) and, where a hardcoded default exists in an options class (mxaccessgw
`LdapOptions.cs:61`, `serviceaccount123`), removes it so a blanked appsettings can't fall back
to a leaked default.
---
## 5. G-5 — master-key provider & clustering (the real fork)
The KEK never lives in the store — it is supplied out-of-band, base64 32 bytes, and **never
committed**. The clustering posture differs sharply between the single-box gateway and the two
Akka-clustered apps.
### mxaccessgw — single box, no clustering
- **Recommended: `Environment` provider** (`ZB_SECRETS_MASTER_KEY` delivered via the NSSM
service env, exactly how `MxGateway__Ldap__ServiceAccountPassword` is already delivered).
Consistent with HistorianGateway; simplest operationally.
*Best-practice fit:* container/NSSM env delivery is the standard 12-factor secret-bootstrap;
matches the app's existing out-of-band password convention.
- **Alternative: `Dpapi` provider** (machine-bound key file). Stronger at-rest binding (the key
is unusable off the box) but ties the store to one machine and complicates DR restore.
### OtOpcUa & ScadaBridge — Akka-clustered → every node needs the *same* KEK and the *same* rows
Both apps resolve secrets on multiple nodes (OtOpcUa driver-role nodes resolve Layer-B driver
secrets; both central nodes of a ScadaBridge pair resolve Layer-A config secrets at boot). Two
constraints follow: (1) identical KEK on every node, (2) identical store contents on every node.
The library today ships a **SQLite-only store** with a **`NoOpSecretReplicator`** — there is no
built-in shared-SQL store and no cross-node replication. So the near-term (G-5) posture is:
- **Recommended interim: `File` provider with a shared mounted key** (`MasterKey.Source=File`,
`FilePath` = a read-only mounted secret file identical on every node) **+ a single shared
SQLite store on a shared/replicated volume** (`SqlitePath` → the shared mount). Secrets are
written rarely, from one node's UI/CLI; every node reads the same file.
*Best-practice fit:* satisfies the library's documented hard requirement (same KEK per node)
with zero new code; acceptable because secret **writes** are rare and human-driven while
**reads** dominate. *Caveat:* SQLite over a network share has known multi-writer locking
limits — fine for read-mostly, single-writer; not a substitute for real replication.
- **Alternative (more integrated, more work): a ConfigDb-backed `ISecretStore`** mirroring each
app's existing `AddDataProtection().PersistKeysToDbContext<…>()` pattern (OtOpcUa persists its
DP key ring to ConfigDb precisely to share key material across nodes). This is the "right"
long-term shape but requires building a custom `ISecretStore` — it **overlaps G-7** and is
deferred there, not attempted in this cut.
> **Decision to confirm at execution time:** for the two clustered apps, take the interim
> File-KEK + shared-SQLite posture now (ships G-5 with no new library code) and track the
> ConfigDb/`ISecretStore` integration under G-7 — **unless** you want the integrated store
> built first, which enlarges G-5 into library work. Each clustered plan is written for the
> interim posture and flags the G-7 hand-off.
---
## 6. G-6 — mount `/admin/secrets` (all three apps)
Same mechanism (§3d) in each dashboard:
| App | Dashboard project | Register RCL assembly in |
|---|---|---|
| OtOpcUa | `…OtOpcUa.AdminUI` | `EndpointRouteBuilderExtensions.cs:31` (`MapRazorComponents<TApp>()` — add `.AddAdditionalAssemblies`) **and** thread `AdditionalAssemblies` from `App.razor:22` into `<Routes/>` (param already plumbed at `Routes.razor:8,37-38`) |
| ScadaBridge | `…ScadaBridge.CentralUI` | `EndpointExtensions.cs:28` `.AddAdditionalAssemblies(...)` **and** `Host/Components/Routes.razor:3` `AdditionalAssemblies` |
| mxaccessgw | `…MxGateway.Server` (Dashboard) | `DashboardEndpointRouteBuilderExtensions.cs:133-135` (add `.AddAdditionalAssemblies`) **and** `Dashboard/Components/Routes.razor:1` (add `AdditionalAssemblies`) |
All three already have an `Administrator` canonical role, so `AddSecretsAuthorization()` grants
secrets access to admins with no new role mapping — **except** verify the claim-type match
below.
**Claim-type verification (ScadaBridge especially):** the library's policies use
`RequireRole(...)` (reads `ClaimsIdentity.RoleClaimType` / `ZbClaimTypes.Role`). OtOpcUa and
mxaccessgw already gate on role claims, so they match. ScadaBridge gates on
`RequireClaim(JwtTokenService.RoleClaimType, …)` — its plan includes a task to confirm
`JwtTokenService.RoleClaimType` equals the principal's role-claim type the library reads, or an
Administrator won't satisfy `secrets:manage`/`secrets:reveal` even though ScadaBridge's own
`RequireAdmin` works.
---
## 7. G-2 / G-3 — Layer-B database-resident secrets
### G-2 — OtOpcUa driver secrets (highest value; its own plan, full detail)
Three call sites read `DriverConfig`/API-key JSON and must resolve `secret:` refs at runtime:
`OpcUaClientDriverFactoryExtensions.CreateInstance` (`:54`), `OpcUaClientDriverProbe`
(`:38`, AdminUI Test-Connect), and `GalaxySecretRef.ResolveApiKey` (`:43`, currently a static
method with `env:`/`file:`/`dev:`/literal arms — add a `secret:` arm). Structural gotchas: the
`Drivers` project does not reference `…Security`, so `ISecretResolver` must be threaded in as a
dependency (and `GalaxySecretRef` needs a signature change off `static`); the resolver must be
registered **unconditionally** (driver-role nodes have no auth/Data-Protection/AdminUI); the
AdminUI editors must round-trip `secret:` refs rather than resolving-then-resaving cleartext.
### G-3 — ScadaBridge MxGateway `ApiKey` (its own plan)
The `ApiKey` is a field inside the `DataConnection.PrimaryConfiguration`/`BackupConfiguration`
JSON blob (`MxGatewayEndpointConfig.ApiKey`), consumed at `MxGatewayDataConnection.cs:96`.
Because it is a field *within* a JSON column (not its own column), the existing
`EncryptedStringConverter` cannot target it directly. Two options:
- **Recommended: store a `secret:<name>` ref in the `ApiKey` field and resolve via
`ISecretResolver.GetAsync` at consumption** (`MxGatewayDataConnection.cs:96`). Consistent with
G-2's Layer-B pattern; leaves the JSON blob human-readable; no schema change.
*Best-practice fit:* same indirection model as OtOpcUa's driver secrets — one mental model
across the family.
- **Alternative: encrypt the whole `PrimaryConfiguration`/`BackupConfiguration` column** with
`EncryptedStringConverter`. Fewer code touch-points but encrypts a mixed blob (the scrubber
and any display/edit path must decrypt first), and the column is currently read in several
places — higher blast radius.
---
## 8. Cross-cutting gotchas (apply to every plan)
1. **Fail-closed at boot.** A `${secret:missing}` throws `SecretNotFoundException` during the
pre-host expansion — the app won't start. Seed every referenced secret (via the `secret` CLI
or the UI) before switching a config value to a token. Plans stage this: add the token *and*
seed the secret in the same task, and keep a rollback (the env-var override still works if
the token is reverted).
2. **`_`-prefixed comment keys are skipped** — keep documentation example tokens under a
`_secretsComment`-style key (the family already uses `_secrets`/`_nodeName`); they won't be
resolved. Confirmed safe in every app's appsettings.
3. **Register `AddZbSecrets` on the runtime host too** (§3b) even for apps that only need
Layer A today — the UI (G-6) and any future Layer-B use need the runtime resolver, and
`SecretsMigrationHostedService` keeps the store schema current.
4. **Audit is automatic.** `AddZbSecrets` lazily binds `IAuditWriter` — all three apps have
`ZB.MOM.WW.Audit` registered, so reveals/resolves audit through the app's existing writer
with no ordering constraint (falls back to no-op otherwise). The reveal audit event was
proven to carry **no plaintext** in HistorianGateway.
5. **Do not disturb existing Data Protection key rings.** OtOpcUa/ScadaBridge persist DP keys to
their ConfigDb (`SetApplicationName` on OtOpcUa; none on ScadaBridge); mxaccessgw uses the
framework-default provider for SignalR hub tokens only. Secrets envelope crypto is
independent of Data Protection — do **not** reconfigure DP as part of this work, or you risk
invalidating cookies/hub-tokens.
6. **KEK is never committed** and never printed. Same discipline as `$GITEA_TOKEN`.
---
## 9. Recommended rollout order
1. **mxaccessgw first** — simplest (single box, no clustering, no Layer-B). Proves the
template a second time after HistorianGateway on the least-risky app.
2. **OtOpcUa** — highest security value (retires cleartext-in-DB driver secrets), but the
heaviest (Layer A + Layer B + clustered KEK). Do Layer A (G-4) + UI (G-6) first, then the
Layer-B driver work (G-2) as a separate reviewable slice.
3. **ScadaBridge** — heaviest config-secret surface (incl. docker-committed plaintext) + the
claim-type verification + G-3. Benefits from the OtOpcUa Layer-B pattern being settled first.
Each app is an independent branch (`feat/adopt-zb-secrets` per repo), mirroring how auth /
theme / audit were adopted across the family — commit + review, then FF-merge to the repo
default and push to Gitea.
## 10. Testing strategy (per app)
- **Offline:** unit test the driver/connection resolver hook (G-2/G-3) with a fake
`ISecretResolver`; assert `secret:` refs resolve and unknown refs fail closed. Confirm the
app builds 0-warning with the three new package refs.
- **Boot smoke:** start the app with one config value switched to `${secret:…}` and the secret
seeded; assert clean boot. Negative control: unseed the secret → assert
`SecretNotFoundException` at startup (proves fail-closed, exactly as HistorianGateway did).
- **UI:** browser-verify `/admin/secrets` — unauthenticated → login redirect (policy enforced);
Administrator → metadata-only list + reveal shows plaintext + an audit row with no plaintext.
- **Live (VPN):** for OtOpcUa/ScadaBridge, prove a real driver/connection secret (`secret:` ref)
authenticates real traffic, mirroring the HistorianGateway live proof. Requires the shared
KEK present on the box.
## 11. Task tracking
Each per-repo plan co-locates its `*.tasks.json` beside the `.md`. Deferred items (G-7 Akka
replicator / shared-SQL `ISecretStore`, G-8 `RewrapAll` KEK-rotation) stay in
[`components/secrets/GAPS.md`](../../components/secrets/GAPS.md) and are out of scope here.