scadaproj/ZB.MOM.WW.Secrets

ZB.MOM.WW.Secrets

A reusable, envelope-encrypted secrets manager for the ZB.MOM.WW.* SCADA family: store secrets (SQL passwords, API tokens, connection strings) encrypted at rest and get the plaintext back on demand — from application code, from configuration, or from an operator UI.

Packages

Package	Purpose
ZB.MOM.WW.Secrets.Abstractions	Contracts only (ISecretStore, ISecretResolver, IMasterKeyProvider, ISecretCipher, ISecretReplicator, ISecretCacheInvalidator, ISecretActorAccessor), value types, exceptions. Dependency-light.
ZB.MOM.WW.Secrets	Implementation: AES-256-GCM envelope cipher, env/file/DPAPI master-key providers, SQLite store, TTL resolver (audited), ${secret:} config expander, AddZbSecrets.
ZB.MOM.WW.Secrets.Ui	Blazor RCL on ZB.MOM.WW.Theme: list / add / rotate / delete + policy-gated, audited reveal.
ZB.MOM.WW.Secrets.Replicator.SqlServer	Cluster-wide secrets via SQL Server — as one shared store, or as a hub each node syncs against.
ZB.MOM.WW.Secrets.Replicator.AkkaDotNet	Peer-to-peer replication over an Akka.NET cluster — partition-tolerant, no shared database.
ZB.MOM.WW.Secrets.Replicator.Grpc	Central-hosted secrets_hub.v1 gRPC hub, pull-only (sites sync from central; central never accepts a write over this channel) — for topologies where sites talk only to central, never a shared DB.

A secret CLI (set / get / list / rm / rotate / rewrap-all) ships in the repo (not packed).

Interactive console

Run secret with no arguments in a real terminal and it opens an interactive console instead of printing usage — a menu-driven session for an operator sitting at a deployment, rather than a scripted one-shot verb. The launch condition is exact: no verb and neither stdin nor stdout is redirected. Any redirection (a pipe, | cat, CI, a script) falls straight through to the headless usage/exit-2 behavior above, so nothing scripted ever hangs on a prompt.

The first screen picks a store target — recent targets (persisted to ~/.zb-secrets/recent-targets.json, names + paths only, never key material), an app's appsettings.json (the console composes base + appsettings.<env>.json + environment variables and binds Secrets exactly like the app itself, so every operation hits precisely the store the app reads), or manual entry (a raw SQLite path + a master-key environment variable name). If the configured KEK cannot be resolved in the operator's shell, the session still opens — degraded, metadata-only — and any flow that needs the key prompts for one (masked paste or a key file) to upgrade it in place.

Once a target is open, the menu offers:

• List secrets — metadata table (name, content type, KEK id, revision, updated); no KEK needed.
• Set / rotate a secret — masked value prompt, seals a fresh row under the session KEK.
• Get (reveal) a secret — confirm-gated; prints the plaintext once, never cached or logged.
• Delete a secret — confirm-gated tombstone; no KEK needed.
• Reference audit & seed — scans the target's composed configuration for ${secret:NAME} tokens, classifies each Ok / Missing / Tombstoned / Undecryptable / InvalidName, and walks the gaps through a seed prompt. This is the deployment-setup flow.
• KEK doctor (lockout recovery) — per-row diagnosis (Ok / WrongKek / Corrupt, with the foreign KEK id) plus a guided remedy: rewrap every row from an old KEK, or re-set individual rows when the old KEK is unrecoverable.
• Export bundle (ciphertext-only) / Import bundle — move rows between stores. Bundles carry name + metadata + wrapped DEK + ciphertext, never plaintext; importing across KEKs verifies the pasted source key against the bundle's KEK id before re-wrapping, conflicts resolve last-writer-wins (with an optional per-row prompt), and the format is versioned.

There is no login — the console runs locally with direct file/DB access, the same trust model as any other admin CLI. New to it? Start with the task-first quickstart (seed a new deployment; recover a KEK lockout). For the full flow catalogue, exit-code/Ctrl-C semantics, and security model, see the operator runbook: docs/operations/interactive-console.md.

How it protects secrets

Each value is encrypted with a fresh data key (DEK) using AES-256-GCM; the DEK is then wrapped by a master KEK that never touches the database. The AAD binds the ciphertext to the secret's name and the wrapped DEK to the KEK id, so rows can't be swapped and a tampered/mis-keyed row fails closed (SecretDecryptionException) rather than returning a wrong value. The master KEK is resolved through a pluggable IMasterKeyProvider (environment variable, key file, or Windows DPAPI).

Wiring it up

builder.Services.AddZbSecrets(builder.Configuration, "Secrets");

// After config load, BEFORE options validation — expand ${secret:...} references:
var expander = new SecretReferenceExpander(app.Services.GetRequiredService<ISecretResolver>());
await expander.ExpandConfigurationAsync((IConfigurationRoot)builder.Configuration, ct);

// appsettings.json
"Secrets": {
  "SqlitePath": "secrets.db",
  "MasterKey": { "Source": "Environment", "EnvVarName": "ZB_SECRETS_MASTER_KEY" },
  "RunMigrationsOnStartup": true,
  "ResolveCacheTtl": "00:00:30"
},
// Keep plaintext out of config — reference a stored secret instead:
"ConnectionStrings": {
  "Historian": "Server=...;User Id=sa;Password=${secret:sql/historian-password}"
}

The master key is 32 bytes, provided base64 in ZB_SECRETS_MASTER_KEY (Environment source). A missing/invalid key fails closed at startup (MasterKeyUnavailableException).

Rotating the master KEK

Because each row is a body sealed under a per-secret DEK that is wrapped by the KEK, rotating the master key only re-wraps DEKs — bodies are never re-encrypted and no value history changes. The secret rewrap-all CLI verb (backed by KekRotationService) migrates every row from the old KEK to the new one; it is idempotent and safe to re-run:

# New KEK defaults to the configured Secrets:MasterKey; supply the OLD key by env-var name or path.
ZB_SECRETS_MASTER_KEY=<new-base64>  ZB_SECRETS_OLD_KEY=<old-base64> \
  secret rewrap-all --old-key-env ZB_SECRETS_OLD_KEY
# → {"action":"rewrap-all","total":N,"rewrapped":N,"alreadyCurrent":0}

Run it with resolve traffic quiesced and once per independent store (once for a shared SQL-Server store; once per node for per-node SQLite on a shared KEK). See the operator runbook: docs/operations/kek-rotation.md.

Runtime + human access

• App code: inject ISecretResolver and call GetAsync(name, ct). Every resolve is audited via ZB.MOM.WW.Audit (name + outcome only — the value is never logged).
• Operators: mount the ZB.MOM.WW.Secrets.Ui /admin/secrets page. The list shows metadata only; revealing a value requires the secrets:reveal authorization policy and is audited. Add/rotate/delete require secrets:manage.

Clustered deployments

Storage is local SQLite by default, which is correct for a single-process app and wrong the moment a secret written on one node has to be readable on another. Three topologies are supported; pick the least complicated one that meets the requirement.

Every topology requires the same master KEK on every node. The database and the wire carry ciphertext only, so a node with a different key cannot decrypt what another wrote — it fails closed on a kek_id mismatch, which looks like corruption but is a deployment mistake. Mount the same key file, or supply the same key environment variable, everywhere.

1. Shared SQL-Server store — the default choice

Every node points at one database. There is exactly one copy of each row, so there is no replication, no reconciliation, no split-brain, and no clock skew to reason about.

builder.Services.AddZbSecretsSqlServerStore(builder.Configuration);

"Secrets": {
  "MasterKey": { "Source": "File", "FilePath": "/run/secrets/zb-kek" },
  "SqlServer": {
    "ConnectionString": "Server=sql01;Database=ZbSecrets;...",
    "SchemaName": "zbsecrets"
  }
}

Trade-off: availability is coupled to that database. A node cut off from it falls back only to the resolver's short in-memory cache. Rotation (rewrap-all) runs once, against the one store.

2. SQL-Server hub replication

Each node keeps its local store and stays converged with a shared hub: writes publish immediately, and a background sweep reconciles both directions on an interval (default 30s).

builder.Services.AddZbSecretsSqlServerReplication(builder.Configuration);

Choose this over (1) when a node must keep resolving secrets while cut off from the hub — reads never leave the node, so a hub outage degrades propagation rather than availability.

3. Akka cluster peer-to-peer

No shared database at all. Nodes broadcast writes over distributed pub/sub and repair divergence with a periodic manifest exchange.

builder.Services.AddZbSecretsAkkaReplication(builder.Configuration);
// Recommended — pins the serializer that carries ciphertext:
Config config = AkkaSecretsReplication.SerializationConfig.WithFallback(myAppConfig);

Requires the application to register its ActorSystem in DI. The local store is SQLite. This is the right answer for air-gapped or intermittently-connected sites; it is the wrong answer if every node can always reach a shared database, because you pay real distributed-systems behaviour for it.

What replication costs you (2 and 3)

• Eventual consistency. Rows converge within the sweep interval, not instantly.
• Last-writer-wins. Two nodes writing the same secret concurrently: the later updated_utc wins and the other write is discarded. There is no merge and no conflict report.
• Publishing is best-effort. A failed broadcast is logged, not thrown — the local write is already durable and anti-entropy repairs it. Writes never fail because a peer was unreachable.
• KEK rotation does not replicate. A re-wrap deliberately leaves the revision untouched so it stays invisible to last-writer-wins. Run rewrap-all against each independent store.

See docs/operations/clustered-secrets.md for operator setup, docs/operations/kek-rotation.md for rotation, and docs/operations/interactive-console.md for the secret interactive console (deployment seeding, lockout recovery, bundles).