diff --git a/ZB.MOM.WW.Secrets/docs/operations/clustered-secrets.md b/ZB.MOM.WW.Secrets/docs/operations/clustered-secrets.md index 29a2d44..cfcb19e 100644 --- a/ZB.MOM.WW.Secrets/docs/operations/clustered-secrets.md +++ b/ZB.MOM.WW.Secrets/docs/operations/clustered-secrets.md @@ -201,7 +201,10 @@ Which topology each app runs in production is decided — tracked as the DB directly — they talk to central for everything — so the SqlServer hub's site-side connection string violates the topology rule. A `ZB.MOM.WW.Secrets.Replicator.Grpc` package (central-hosted `secrets_hub.v1`, **pull-only: updates originate at central only, by design**) - replaces it; until that ships, ScadaBridge replication stays off. + replaces it; until that ships, ScadaBridge replication stays off. The gRPC mode is **cached on + the site clusters** exactly like §2's hub mode — each site node keeps a full local store, + serves last-known-good secrets while central is unreachable, and converges on the first + successful sweep. - **KEK: one fleet-wide production KEK, delivered per node via appsettings/environment override, never committed.** The committed dev KEK is dev-only. Verify per pair with the §0 digest compare. A single fleet KEK is what makes bundle-based seeding between pairs possible. diff --git a/components/secrets/GAPS.md b/components/secrets/GAPS.md index 3aa03ce..5b457a3 100644 --- a/components/secrets/GAPS.md +++ b/components/secrets/GAPS.md @@ -176,7 +176,10 @@ project 38/38 green. pair-local with one fleet-wide appsettings-delivered KEK; ScadaBridge = a new central gRPC hub replicator (`ZB.MOM.WW.Secrets.Replicator.Grpc`, pull-only — updates originate at central only), because the SqlServer hub's site-side connection string violates ScadaBridge's - sites-talk-to-central-not-the-DB rule. Enablement waits on the hub package (ScadaBridge) and the + sites-talk-to-central-not-the-DB rule. **The gRPC mode keeps the cached-on-site-clusters + posture:** each site node holds a full local store and serves last-known-good secrets when + central is unreachable, converging on the first successful sweep — same offline guarantee as + the SQL hub mode, without the site-side DB connection. Enablement waits on the hub package (ScadaBridge) and the pair-local re-gate on the post-Phase-6 rig (OtOpcUa). - ✅ ScadaBridge's hub topology **is** live-validated and technically safe to enable — but the 2026-08-07 decision rejects it for production on topology grounds (site nodes must not hold a diff --git a/docs/plans/2026-08-07-secrets-grpc-hub-implementation.md b/docs/plans/2026-08-07-secrets-grpc-hub-implementation.md new file mode 100644 index 0000000..54760dd --- /dev/null +++ b/docs/plans/2026-08-07-secrets-grpc-hub-implementation.md @@ -0,0 +1,174 @@ +# Secrets gRPC hub replicator — implementation plan + +> **For Claude:** REQUIRED SUB-SKILL: execute with superpowers-extended-cc:subagent-driven-development +> (owner chose subagent execution, parallel where safe, 2026-08-07 — do not re-ask). +> Task metadata (`Classification`, `Parallelizable with`) drives the review chain and dispatch. + +**Goal:** Ship `ZB.MOM.WW.Secrets.Replicator.Grpc` (central-hosted pull-only secrets hub), wire it +into ScadaBridge, live-gate it, and re-gate OtOpcUa's Akka pair-local topology — completing +[scadaproj#3](https://gitea.dohertylan.com/dohertj2/scadaproj/issues/3). + +**Design authority:** `docs/plans/2026-08-07-secrets-production-topology-and-grpc-hub.md` (D1–D4). +Pull-only is an owner-confirmed invariant: **the proto exposes no write RPC — the narrower +contract is the enforcement.** + +**Architecture:** central hosts a `secrets_hub.v1` gRPC service adapting its `ISecretStore` (read +half only) to the wire; site nodes run a `BackgroundService` sweep that pulls via +`SecretReplicationReconciler.ComputePullSet` → fetch → `ApplyAsync`. Ciphertext-only; offline = +last-known-good (sweep never throws out of `ExecuteAsync`). + +**Verified seams (read before coding):** +- `ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Replicator.SqlServer/ISecretReplicationHub.cs` — + `GetManifestAsync` / `GetManyAsync` / `ApplyReplicatedAsync`. The gRPC hub mirrors ONLY the + first two. +- `ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets/Replication/SecretReplicationReconciler.cs` — + `ComputePullSet` (static), `ApplyAsync`, `ReadLocalAsync`. Pull-only = never call + `ComputePushSet`. +- `SqlServerSecretSyncService.cs` — the sweep shape to mirror (SyncOnStartup, `PeriodicTimer`, + never-throw, log + retry next tick). + +**Standing security invariants (every task):** never echo bearer/KEK material in logs, errors, or +status; error messages name exact config keys; fail closed on every unresolvable auth case; auth +interceptors keep EXACTLY ONE public constructor (Grpc.AspNetCore silently stops invoking +interceptors with more); ciphertext only on the wire. + +--- + +## Task 1: Package scaffold + `secrets_hub.v1` proto + +**Classification:** standard · **Est:** ~5 min · **Parallelizable with:** Task 8 +**Files:** Create `ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Replicator.Grpc/ZB.MOM.WW.Secrets.Replicator.Grpc.csproj`, +`…/Protos/secrets_hub.proto`; Modify `ZB.MOM.WW.Secrets/ZB.MOM.WW.Secrets.slnx` (add project). + +Proto (`package zb.mom.ww.secrets.hub.v1`), service `SecretsHub`, exactly two RPCs: +- `rpc GetManifest(GetManifestRequest) returns (GetManifestResponse)` — repeated entries mirroring + `SecretManifestEntry` (read the Abstractions type and map EVERY field losslessly: name, + revision, timestamp, tombstone flag, and whatever else it carries). +- `rpc GetSecrets(GetSecretsRequest {repeated string names}) returns (GetSecretsResponse)` — + repeated rows mirroring `StoredSecret` verbatim (ciphertext + metadata; read the type first). +NO other RPC. csproj: `Grpc.AspNetCore` + `Grpc.Net.Client` + `Grpc.Tools` (match versions used +elsewhere in the family), `` inherits the family's shared version. Follow the SqlServer +package's csproj as the template. Build must pass; commit. + +## Task 2: Server half — hub service + bearer auth + DI + +**Classification:** high-risk · **Est:** ~5 min · **Parallelizable with:** Task 3 +**Files:** Create `…Replicator.Grpc/SecretsHubService.cs`, `…/SecretsHubAuthInterceptor.cs`, +`…/DependencyInjection/SecretsGrpcHubServerExtensions.cs`, `…/SecretsGrpcHubServerOptions.cs`. + +`SecretsHubService : SecretsHub.SecretsHubBase` adapts the registered `ISecretStore` (manifest + +bulk read — reuse/mirror how `SqlServerSecretStore` produces manifest entries). Auth: interceptor +validating `Authorization: Bearer ` against `SecretsGrpcHubServerOptions.BearerToken` +(bound from config; empty/unset ⇒ the service REFUSES to serve — fail closed, log names the +config key once). Constant-time comparison. Uniform denial (`Unauthenticated`, no detail). +`AddZbSecretsGrpcHub(IServiceCollection, IConfiguration, sectionPath)` + +`MapZbSecretsHub(IEndpointRouteBuilder)`. TDD: red-first unit tests for the interceptor (valid / +invalid / missing / unset-key-refuses) and options validation. + +## Task 3: Client half — pull-only sweep service + DI + +**Classification:** high-risk · **Est:** ~5 min · **Parallelizable with:** Task 2 +**Files:** Create `…Replicator.Grpc/GrpcSecretsHubClient.cs`, `…/GrpcSecretSyncService.cs`, +`…/SecretsGrpcHubClientOptions.cs`, `…/DependencyInjection/SecretsGrpcHubClientExtensions.cs`. + +`GrpcSecretsHubClient` wraps the generated client (bearer header per call; per-call deadline from +options). `GrpcSecretSyncService : BackgroundService` mirrors `SqlServerSecretSyncService` +EXCEPT pull-only: manifest → `SecretReplicationReconciler.ComputePullSet` → `GetSecrets` → +`ApplyAsync` (+ cache invalidation seam, same as SqlServer's ctor wiring). NEVER computes a push +set; NEVER throws out of `ExecuteAsync` (hub outage ⇒ warn + retry next tick — copy the +SqlServer service's log/exception discipline). Options: `Endpoint` (required), `BearerToken` +(required, fail-closed), `SyncInterval` (default 30 s), `SyncOnStartup` (default true), +`CallDeadline` (default 30 s); `Validate()` called at registration like +`SqlServerSecretsOptions`. TDD: red-first tests with a fake hub — pull convergence, tombstone +apply, outage → no throw + local store untouched, options validation. + +## Task 4: Integration tests + no-write-surface pin + +**Classification:** standard · **Est:** ~5 min · **Parallelizable with:** none (needs 2+3) +**Files:** Create `ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Replicator.Grpc.Tests/` (project + +suites); Modify slnx. + +In-process server via `Microsoft.AspNetCore.TestHost` (real Kestrel-less gRPC): end-to-end +convergence (seed hub store → client store converges, byte-identical ciphertext), delete +propagates as tombstone, hub down at client boot ⇒ client serves last-known-good and converges +after the hub returns, auth negatives fail closed (bad/missing/unset both sides). Plus the +contract pin: a reflection test over the generated `SecretsHub` service asserting the RPC set is +EXACTLY `{GetManifest, GetSecrets}` — a third RPC (any write surface) fails the build. Full +`dotnet test` on the Secrets slnx green, 0 warnings. + +## Task 5: Version bump + pack + publish 0.4.0 + +**Classification:** small · **Est:** ~4 min · **Parallelizable with:** none +**Files:** Modify `ZB.MOM.WW.Secrets/Directory.Build.props` (version → 0.4.0), CHANGELOG, pack docs. + +Bump the family version to `0.4.0`, CHANGELOG entry (new package, no breaking changes; existing +packages unchanged behaviorally), `dotnet pack -c Release`, push ALL packages (now 6 + CLI as +applicable) to the `dohertj2-gitea` feed, restore-verify from a scratch consumer (the established +pattern from 0.2.x/0.3.0 publishes). Commit + push scadaproj `main`. + +## Task 6: ScadaBridge wiring — central hub + site sweep client + +**Classification:** high-risk · **Est:** ~5 min · **Parallelizable with:** none (needs Task 5's feed package) +**Repo:** `~/Desktop/ScadaBridge` (branch `feat/secrets-grpc-hub` off `main`). +**Files:** Modify the central gRPC endpoint composition (find where the ClusterClient→gRPC +migration's central services are mapped), the site-role composition root, `Directory.Packages.props` +(Secrets → 0.4.0 + new package ref), appsettings templates; Create DI pin tests. + +Central: `AddZbSecretsGrpcHub` + `MapZbSecretsHub` on the existing central gRPC listener, bearer +token from appsettings/env (`${secret:}` is NOT usable here — chicken-and-egg; follow the mesh +PSK pattern), fail-closed. Site: register the sweep client under +`Secrets:Replication:Enabled=true` + a mode/section selecting Grpc — mirror how the existing +replicator registration is switched; central must NOT register the client, sites must NOT +register the hub. Red-first WebApplicationFactory/DI pin tests over the real Program.cs for both +roles (the ScadaBridge#22 test pattern). Full ScadaBridge offline suite green before commit. + +## Task 7: ScadaBridge live gate (docker cluster) + +**Classification:** high-risk · **Est:** live-rig, serial · **Parallelizable with:** Task 8 +**Files:** Create `ScadaBridge/docs/plans/2026-08-07-secrets-grpc-hub-live-gate.md` (gate record). + +On the local 8-node docker cluster (rebuild via `bash docker/deploy.sh` with the flag enabled on +site-a + central): (1) secret added centrally converges to both site-a nodes ≤ sweep interval, +byte-identical, decrypt-verified; (2) site pair cold-boots with central stopped ⇒ serves +last-known-good, converges after central returns; (3) central delete tombstone propagates, no +resurrection on site restart; (4) auth negatives fail closed, zero key material in logs +(grep the logs to prove it). Record PASS/FAIL per check in the gate doc; fix-forward defects on +the branch. Merge `feat/secrets-grpc-hub` → `main`, push. + +## Task 8: OtOpcUa Akka pair-local re-gate (three-mesh rig) + +**Classification:** high-risk · **Est:** live-rig, config-only (no product code) · **Parallelizable with:** Tasks 1–7 (independent repo + rig) +**Repo:** `~/Desktop/OtOpcUa`. **Files:** Create `OtOpcUa/docs/plans/2026-08-07-secrets-pair-local-live-gate.md`. + +The 2026-07-18 six-node convergence gate predates the mesh Phase 6 partition — re-gate pair-local +semantics on the current three-mesh docker-dev rig (fixtures on `10.100.0.35`; dev KEK): (1) seed +on one node of site-a ⇒ partner converges ≤60 s byte-identical; (2) **no cross-pair propagation** +to site-b/central (this is now CORRECT behavior); (3) delete propagates within the pair; (4) +clean pair restart, no resurrection. Record the gate doc, commit + push OtOpcUa `master`. + +## Task 9: Closeout — issue, runbook, index + +**Classification:** small · **Est:** ~4 min · **Parallelizable with:** none (last) + +Check off the remaining boxes on scadaproj#3 (implementation, ScadaBridge gate, OtOpcUa re-gate, +rotation doc — verify the rotation fan-out paragraph in `clustered-secrets.md` §Production +enablement is still accurate post-implementation and extend if needed). Update +`components/secrets/GAPS.md` + the CLAUDE.md Secrets index row (0.4.0 published, hub shipped, +gates PASS with SHAs). Commit + push scadaproj `main`. Close #3 if every box is checked. + +--- + +## Dispatch notes for the executor + +- **Wave 1:** Task 1 ∥ Task 8 (different repos; Task 8 is the long pole — start it first). +- **Wave 2:** Task 2 ∥ Task 3 (disjoint files, same new project). +- **Wave 3:** Task 4 → Task 5 (serial). +- **Wave 4:** Task 6 → Task 7 (serial, ScadaBridge). +- **Wave 5:** Task 9 after everything. +- Live-gate tasks (7, 8) need docker orchestration + log inspection — run them via a single agent + each (or main loop), never split one gate across agents. They use different docker hosts + (ScadaBridge = local docker; OtOpcUa rig = `10.100.0.35` + local compose) so they may overlap. +- Rig credentials/keys: dev KEK and rig PSKs only; never commit rig overrides; never echo key + material (standing rules). +- Commits end with the session trailer: + `Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1` diff --git a/docs/plans/2026-08-07-secrets-grpc-hub-implementation.md.tasks.json b/docs/plans/2026-08-07-secrets-grpc-hub-implementation.md.tasks.json new file mode 100644 index 0000000..babbd49 --- /dev/null +++ b/docs/plans/2026-08-07-secrets-grpc-hub-implementation.md.tasks.json @@ -0,0 +1,15 @@ +{ + "planPath": "docs/plans/2026-08-07-secrets-grpc-hub-implementation.md", + "tasks": [ + {"id": 1, "subject": "Task 1: Package scaffold + secrets_hub.v1 proto", "status": "pending", "classification": "standard", "parallelizableWith": [8]}, + {"id": 2, "subject": "Task 2: Server half — hub service + bearer auth + DI", "status": "pending", "classification": "high-risk", "blockedBy": [1], "parallelizableWith": [3]}, + {"id": 3, "subject": "Task 3: Client half — pull-only sweep service + DI", "status": "pending", "classification": "high-risk", "blockedBy": [1], "parallelizableWith": [2]}, + {"id": 4, "subject": "Task 4: Integration tests + no-write-surface pin", "status": "pending", "classification": "standard", "blockedBy": [2, 3]}, + {"id": 5, "subject": "Task 5: Version bump + pack + publish 0.4.0", "status": "pending", "classification": "small", "blockedBy": [4]}, + {"id": 6, "subject": "Task 6: ScadaBridge wiring — central hub + site sweep client", "status": "pending", "classification": "high-risk", "blockedBy": [5]}, + {"id": 7, "subject": "Task 7: ScadaBridge live gate (docker cluster)", "status": "pending", "classification": "high-risk", "blockedBy": [6], "parallelizableWith": [8]}, + {"id": 8, "subject": "Task 8: OtOpcUa Akka pair-local re-gate (three-mesh rig)", "status": "pending", "classification": "high-risk", "parallelizableWith": [1, 2, 3, 4, 5, 6, 7]}, + {"id": 9, "subject": "Task 9: Closeout — issue #3, runbook, GAPS, index", "status": "pending", "classification": "small", "blockedBy": [7, 8]} + ], + "lastUpdated": "2026-08-07" +}