ZB.MOM.WW.Secrets.Replicator.SqlServer (0.3.0)
Installation
dotnet nuget add source --name dohertj2 --username your_username --password your_token https://gitea.dohertylan.com/api/packages/dohertj2/nuget/index.jsondotnet add package --source dohertj2 --version 0.3.0 ZB.MOM.WW.Secrets.Replicator.SqlServerAbout this package
Shared SQL-Server secret store and hub replication for the ZB.MOM.WW SCADA family — cluster-wide, ciphertext-only secrets.
ZB.MOM.WW.Secrets
A reusable, envelope-encrypted secrets manager for the ZB.MOM.WW.* SCADA family:
store secrets (SQL passwords, API tokens, connection strings) encrypted at rest and get the
plaintext back on demand — from application code, from configuration, or from an operator UI.
Packages
| Package | Purpose |
|---|---|
ZB.MOM.WW.Secrets.Abstractions |
Contracts only (ISecretStore, ISecretResolver, IMasterKeyProvider, ISecretCipher, ISecretReplicator, ISecretCacheInvalidator, ISecretActorAccessor), value types, exceptions. Dependency-light. |
ZB.MOM.WW.Secrets |
Implementation: AES-256-GCM envelope cipher, env/file/DPAPI master-key providers, SQLite store, TTL resolver (audited), ${secret:} config expander, AddZbSecrets. |
ZB.MOM.WW.Secrets.Ui |
Blazor RCL on ZB.MOM.WW.Theme: list / add / rotate / delete + policy-gated, audited reveal. |
ZB.MOM.WW.Secrets.Replicator.SqlServer |
Cluster-wide secrets via SQL Server — as one shared store, or as a hub each node syncs against. |
ZB.MOM.WW.Secrets.Replicator.AkkaDotNet |
Peer-to-peer replication over an Akka.NET cluster — partition-tolerant, no shared database. |
A secret CLI (set / get / list / rm / rotate / rewrap-all) ships in the repo (not packed).
Interactive console
Run secret with no arguments in a real terminal and it opens an interactive console
instead of printing usage — a menu-driven session for an operator sitting at a deployment,
rather than a scripted one-shot verb. The launch condition is exact: no verb and neither
stdin nor stdout is redirected. Any redirection (a pipe, | cat, CI, a script) falls straight
through to the headless usage/exit-2 behavior above, so nothing scripted ever hangs on a
prompt.
The first screen picks a store target — recent targets (persisted to
~/.zb-secrets/recent-targets.json, names + paths only, never key material), an app's
appsettings.json (the console composes base + appsettings.<env>.json + environment
variables and binds Secrets exactly like the app itself, so every operation hits precisely
the store the app reads), or manual entry (a raw SQLite path + a master-key environment
variable name). If the configured KEK cannot be resolved in the operator's shell, the session
still opens — degraded, metadata-only — and any flow that needs the key prompts for one
(masked paste or a key file) to upgrade it in place.
Once a target is open, the menu offers:
- List secrets — metadata table (name, content type, KEK id, revision, updated); no KEK needed.
- Set / rotate a secret — masked value prompt, seals a fresh row under the session KEK.
- Get (reveal) a secret — confirm-gated; prints the plaintext once, never cached or logged.
- Delete a secret — confirm-gated tombstone; no KEK needed.
- Reference audit & seed — scans the target's composed configuration for
${secret:NAME}tokens, classifies each Ok / Missing / Tombstoned / Undecryptable / InvalidName, and walks the gaps through a seed prompt. This is the deployment-setup flow. - KEK doctor (lockout recovery) — per-row diagnosis (Ok / WrongKek / Corrupt, with the foreign KEK id) plus a guided remedy: rewrap every row from an old KEK, or re-set individual rows when the old KEK is unrecoverable.
- Export bundle (ciphertext-only) / Import bundle — move rows between stores. Bundles carry name + metadata + wrapped DEK + ciphertext, never plaintext; importing across KEKs verifies the pasted source key against the bundle's KEK id before re-wrapping, conflicts resolve last-writer-wins (with an optional per-row prompt), and the format is versioned.
There is no login — the console runs locally with direct file/DB access, the same trust model
as any other admin CLI. See the operator runbook for the full walkthrough:
docs/operations/interactive-console.md.
How it protects secrets
Each value is encrypted with a fresh data key (DEK) using AES-256-GCM; the DEK is then
wrapped by a master KEK that never touches the database. The AAD binds the ciphertext
to the secret's name and the wrapped DEK to the KEK id, so rows can't be swapped and a
tampered/mis-keyed row fails closed (SecretDecryptionException) rather than returning a
wrong value. The master KEK is resolved through a pluggable IMasterKeyProvider
(environment variable, key file, or Windows DPAPI).
Wiring it up
builder.Services.AddZbSecrets(builder.Configuration, "Secrets");
// After config load, BEFORE options validation — expand ${secret:...} references:
var expander = new SecretReferenceExpander(app.Services.GetRequiredService<ISecretResolver>());
await expander.ExpandConfigurationAsync((IConfigurationRoot)builder.Configuration, ct);
// appsettings.json
"Secrets": {
"SqlitePath": "secrets.db",
"MasterKey": { "Source": "Environment", "EnvVarName": "ZB_SECRETS_MASTER_KEY" },
"RunMigrationsOnStartup": true,
"ResolveCacheTtl": "00:00:30"
},
// Keep plaintext out of config — reference a stored secret instead:
"ConnectionStrings": {
"Historian": "Server=...;User Id=sa;Password=${secret:sql/historian-password}"
}
The master key is 32 bytes, provided base64 in ZB_SECRETS_MASTER_KEY (Environment source).
A missing/invalid key fails closed at startup (MasterKeyUnavailableException).
Rotating the master KEK
Because each row is a body sealed under a per-secret DEK that is wrapped by the KEK, rotating
the master key only re-wraps DEKs — bodies are never re-encrypted and no value history changes.
The secret rewrap-all CLI verb (backed by KekRotationService) migrates every row from the
old KEK to the new one; it is idempotent and safe to re-run:
# New KEK defaults to the configured Secrets:MasterKey; supply the OLD key by env-var name or path.
ZB_SECRETS_MASTER_KEY=<new-base64> ZB_SECRETS_OLD_KEY=<old-base64> \
secret rewrap-all --old-key-env ZB_SECRETS_OLD_KEY
# → {"action":"rewrap-all","total":N,"rewrapped":N,"alreadyCurrent":0}
Run it with resolve traffic quiesced and once per independent store (once for a shared
SQL-Server store; once per node for per-node SQLite on a shared KEK). See the operator runbook:
docs/operations/kek-rotation.md.
Runtime + human access
- App code: inject
ISecretResolverand callGetAsync(name, ct). Every resolve is audited viaZB.MOM.WW.Audit(name + outcome only — the value is never logged). - Operators: mount the
ZB.MOM.WW.Secrets.Ui/admin/secretspage. The list shows metadata only; revealing a value requires thesecrets:revealauthorization policy and is audited. Add/rotate/delete requiresecrets:manage.
Clustered deployments
Storage is local SQLite by default, which is correct for a single-process app and wrong the moment a secret written on one node has to be readable on another. Three topologies are supported; pick the least complicated one that meets the requirement.
Every topology requires the same master KEK on every node. The database and the wire carry ciphertext only, so a node with a different key cannot decrypt what another wrote — it fails closed on a
kek_idmismatch, which looks like corruption but is a deployment mistake. Mount the same key file, or supply the same key environment variable, everywhere.
1. Shared SQL-Server store — the default choice
Every node points at one database. There is exactly one copy of each row, so there is no replication, no reconciliation, no split-brain, and no clock skew to reason about.
builder.Services.AddZbSecretsSqlServerStore(builder.Configuration);
"Secrets": {
"MasterKey": { "Source": "File", "FilePath": "/run/secrets/zb-kek" },
"SqlServer": {
"ConnectionString": "Server=sql01;Database=ZbSecrets;...",
"SchemaName": "zbsecrets"
}
}
Trade-off: availability is coupled to that database. A node cut off from it falls back only to
the resolver's short in-memory cache. Rotation (rewrap-all) runs once, against the one store.
2. SQL-Server hub replication
Each node keeps its local store and stays converged with a shared hub: writes publish immediately, and a background sweep reconciles both directions on an interval (default 30s).
builder.Services.AddZbSecretsSqlServerReplication(builder.Configuration);
Choose this over (1) when a node must keep resolving secrets while cut off from the hub — reads never leave the node, so a hub outage degrades propagation rather than availability.
3. Akka cluster peer-to-peer
No shared database at all. Nodes broadcast writes over distributed pub/sub and repair divergence with a periodic manifest exchange.
builder.Services.AddZbSecretsAkkaReplication(builder.Configuration);
// Recommended — pins the serializer that carries ciphertext:
Config config = AkkaSecretsReplication.SerializationConfig.WithFallback(myAppConfig);
Requires the application to register its ActorSystem in DI. The local store is SQLite. This is
the right answer for air-gapped or intermittently-connected sites; it is the wrong answer if every
node can always reach a shared database, because you pay real distributed-systems behaviour for it.
What replication costs you (2 and 3)
- Eventual consistency. Rows converge within the sweep interval, not instantly.
- Last-writer-wins. Two nodes writing the same secret concurrently: the later
updated_utcwins and the other write is discarded. There is no merge and no conflict report. - Publishing is best-effort. A failed broadcast is logged, not thrown — the local write is already durable and anti-entropy repairs it. Writes never fail because a peer was unreachable.
- KEK rotation does not replicate. A re-wrap deliberately leaves the revision untouched so it
stays invisible to last-writer-wins. Run
rewrap-allagainst each independent store.
See docs/operations/clustered-secrets.md for operator
setup, docs/operations/kek-rotation.md for rotation, and
docs/operations/interactive-console.md for the
secret interactive console (deployment seeding, lockout recovery, bundles).