Files
scadaproj/ZB.MOM.WW.Secrets/CHANGELOG.md
T
Joseph Doherty 0d4978953f feat(secrets-grpc): FallbackEndpoints + sticky failover hub reader (scadaproj#4)
SecretsGrpcHubClientOptions gains a getter-only FallbackEndpoints list:
additional hub addresses tried in order when a call fails with a
transport-level RpcException. Validation fails startup on blank/relative
entries and on any duplicate of the primary or another fallback (a
duplicated endpoint silently halves the configured failover).

New FailoverSecretsHubReader composes one GrpcSecretsHubClient per
endpoint: the answering endpoint is sticky (a sweep is GetManifest + N
GetSecrets batches — without stickiness every call after a primary
outage re-pays a full CallDeadline before reaching the healthy
fallback), rotation wraps so a recovered primary is retried when the
stuck-to endpoint fails, only RpcException triggers failover
(cancellation and defects propagate untouched), and when every endpoint
fails the last RpcException is rethrown so the sweep degrades to
last-known-good exactly as today. Failover warnings name only the two
endpoint URIs and the status code — never any credential material.

DI: zero fallbacks keeps the registration graph byte-identical to 0.4.x
(single keyed channel under ChannelServiceKey, plain
GrpcSecretsHubClient); with fallbacks, one container-owned keyed channel
per endpoint (fallback i under "{ChannelServiceKey}:fallback:{i}") wired
into the failover reader. GrpcSecretsHubClient itself is unchanged.

Safe only when every listed endpoint serves the SAME store — for
ScadaBridge, the central pair over one shared SQL Server store; the
options doc and CHANGELOG carry the warning.

Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1
2026-08-07 09:37:22 -04:00

6.7 KiB

Changelog

All notable changes to the ZB.MOM.WW.Secrets family are recorded here. Format loosely follows Keep a Changelog; versions are shared across every package in the family (see Directory.Build.props) even when a given release only touches one of them.

0.5.0 — Unreleased

Added

  • ZB.MOM.WW.Secrets.Replicator.Grpc — hub endpoint failover for followers (scadaproj#4). SecretsGrpcHubClientOptions gains FallbackEndpoints: additional hub addresses tried, in order, when a call to the primary Endpoint (or an earlier fallback) fails with a transport-level gRPC error. The answering endpoint is sticky — a sweep that failed over does not re-pay the dead endpoint's full CallDeadline on every subsequent call — and rotation wraps, so a recovered primary is retried the next time the stuck-to endpoint fails. Only RpcException triggers failover; cancellation and everything else propagate untouched, and when every endpoint fails the last RpcException is rethrown so the sweep degrades to last-known-good exactly as before. Each failover logs a Warning naming the failed and next endpoint URIs (never any credential material).
    • Same-store requirement: list only endpoints that serve the SAME store — for ScadaBridge, the central pair over one shared SQL Server store. Endpoints backed by independent stores turn a primary outage into a silent convergence stop (failing over to an emptier hub "succeeds"), which is the exact defect scadaproj#4 recorded.
    • Validation fails startup on a blank/relative fallback or one duplicating the primary or another fallback (a duplicated endpoint silently halves the configured failover).

Unchanged

  • With FallbackEndpoints empty (the default), the registration graph is identical to 0.4.x — one keyed channel, one GrpcSecretsHubClient, no failover wrapper — and the wire behavior of every call is byte-identical. Every other package in the family is behaviorally unchanged in this release.

0.4.1 — 2026-08-07

Fixed

  • ZB.MOM.WW.Secrets.Replicator.Grpc — the hub now logs its denials. A ScadaBridge live gate found that SecretsHubAuthInterceptor refused calls correctly but wrote nothing about it: the only trace of a refusal was a framework Grpc.AspNetCore.Server Information line, so a follower whose bearer token had been mis-rotated silently stopped converging and the hub showed it only at INF. Every denial is now counted, and a Warning summarising the window is emitted on the first denial after startup or after 60 quiet seconds — carrying the number of denials since the previous summary plus a breakdown by cause (missing/malformed credential, wrong credential, hub token unconfigured). A single probe is therefore visible immediately, while a sustained attack is bounded to one log line per minute — the endpoint is unauthenticated by construction, so a per-denial warning would be an unbounded log write driven by an unauthenticated caller.

Unchanged

  • Wire behavior is byte-identical. The cause breakdown is server-side diagnostics only; every rejection still returns the same uniform Unauthenticated status with the same detail, and the credential comparison is still constant-time. No token material, presented or configured, appears in any log line at any level.
  • The existing warn-once-on-unconfigured-token line is unchanged and still emitted exactly once; denials in that state also count toward the new summary's unconfigured category.
  • Every other package in the family is behaviorally unchanged in this release.

0.4.0 — 2026-08-07

Added

  • New package ZB.MOM.WW.Secrets.Replicator.Grpc — a central-hosted secrets_hub.v1 gRPC hub for the sites-talk-to-central production topology (scadaproj#3; design doc docs/plans/2026-08-07-secrets-production-topology-and-grpc-hub.md).
    • Pull-only by construction: the proto exposes no write RPC. Updates originate at central only; a site never pushes a secret upstream.
    • Fail-closed bearer auth on the hub server: constant-time comparison, uniform denial response (no signal distinguishing "wrong token" from "missing token"), a warn-once log on an unconfigured/empty token rather than a silent open door.
    • GetSecrets enforces a batch-size cap, rejecting an oversized request with ResourceExhausted instead of serving it partially or unbounded.
    • Site-side GrpcSecretSyncService sweeps the hub on an interval with per-row parse isolation — one malformed/undecryptable row logs and is skipped, it does not fault the whole sweep.
    • Keyed GrpcChannel registration so multiple hub targets can be wired in one process without channel collisions.
    • Offline behavior is last-known-good: if the hub is unreachable, the site keeps serving the most recently synced values rather than failing secret resolution.

Unchanged

  • ZB.MOM.WW.Secrets.Abstractions, ZB.MOM.WW.Secrets, ZB.MOM.WW.Secrets.Ui, ZB.MOM.WW.Secrets.Replicator.SqlServer, and ZB.MOM.WW.Secrets.Replicator.AkkaDotNet are behaviorally unchanged in this release — no breaking changes to any existing package.

0.3.0 — 2026-07-19

  • Menu-driven Spectre.Console interactive operator console added to the secret CLI (list / set / reveal / delete, reference audit & seed, KEK doctor, ciphertext-only bundle export/import). See docs/operations/interactive-console.md.

0.2.3 — 2026-07-19

  • ZB.MOM.WW.Secrets.Ui: fixed the delete-confirmation modal being permanently invisible on every Bootstrap host (ConfirmDeleteModal used bare class="modal" with no shipped CSS; Bootstrap's .modal{display:none} hid it everywhere but mxaccessgw). Fixed via a zb-secrets-modal* class rename plus a component-emitted <style> block.

0.2.2 — 2026-07-18

  • Fixed the Akka replicator's hosted-process DI deadlock (a circular singleton dependency hidden behind factory lambdas that MS.DI's StackGuard turned into a silent cross-thread hang rather than a stack overflow). New DeferredSecretCacheInvalidator defers the one cycle-closing edge to first eviction.

0.2.1 — 2026-07-18

  • Fixed the Akka replicator being silently inert since 0.2.0 (a TryAdd registration-order bug published into a no-op sink and never spawned an actor). SQL-Server replication was never affected.

0.2.0 — 2026-07-18

  • Added clustered replication (G-7): ZB.MOM.WW.Secrets.Replicator.SqlServer (shared store, or local-store-with-hub) and ZB.MOM.WW.Secrets.Replicator.AkkaDotNet (peer-to-peer over distributed pub/sub).