Files

T

Joseph Doherty ad7df925ae docs(secrets): gRPC hub implementation plan (9 tasks, subagent-executable) + cached-on-sites posture made explicit

Plan + tasks ledger for scadaproj#3 execution: package (proto/server/client/
tests/publish 0.4.0), ScadaBridge wiring + live gate, OtOpcUa pair-local
re-gate, closeout. GAPS + runbook now state the gRPC mode keeps the
cached-on-site-clusters last-known-good posture (owner-confirmed).

Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1

2026-08-07 05:36:32 -04:00

11 KiB

Raw Blame History

Secrets gRPC hub replicator — implementation plan

For Claude: REQUIRED SUB-SKILL: execute with superpowers-extended-cc:subagent-driven-development (owner chose subagent execution, parallel where safe, 2026-08-07 — do not re-ask). Task metadata (Classification, Parallelizable with) drives the review chain and dispatch.

Goal: Ship ZB.MOM.WW.Secrets.Replicator.Grpc (central-hosted pull-only secrets hub), wire it into ScadaBridge, live-gate it, and re-gate OtOpcUa's Akka pair-local topology — completing scadaproj#3.

Design authority: docs/plans/2026-08-07-secrets-production-topology-and-grpc-hub.md (D1–D4). Pull-only is an owner-confirmed invariant: the proto exposes no write RPC — the narrower contract is the enforcement.

Architecture: central hosts a secrets_hub.v1 gRPC service adapting its ISecretStore (read half only) to the wire; site nodes run a BackgroundService sweep that pulls via SecretReplicationReconciler.ComputePullSet → fetch → ApplyAsync. Ciphertext-only; offline = last-known-good (sweep never throws out of ExecuteAsync).

Verified seams (read before coding):

ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Replicator.SqlServer/ISecretReplicationHub.cs — GetManifestAsync / GetManyAsync / ApplyReplicatedAsync. The gRPC hub mirrors ONLY the first two.
ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets/Replication/SecretReplicationReconciler.cs — ComputePullSet (static), ApplyAsync, ReadLocalAsync. Pull-only = never call ComputePushSet.
SqlServerSecretSyncService.cs — the sweep shape to mirror (SyncOnStartup, PeriodicTimer, never-throw, log + retry next tick).

Standing security invariants (every task): never echo bearer/KEK material in logs, errors, or status; error messages name exact config keys; fail closed on every unresolvable auth case; auth interceptors keep EXACTLY ONE public constructor (Grpc.AspNetCore silently stops invoking interceptors with more); ciphertext only on the wire.

Task 1: Package scaffold + `secrets_hub.v1` proto

Classification: standard · Est: ~5 min · Parallelizable with: Task 8 Files: Create ZB.MOM.WW.Secrets/src/ZB.MOM.WW.Secrets.Replicator.Grpc/ZB.MOM.WW.Secrets.Replicator.Grpc.csproj, …/Protos/secrets_hub.proto; Modify ZB.MOM.WW.Secrets/ZB.MOM.WW.Secrets.slnx (add project).

Proto (package zb.mom.ww.secrets.hub.v1), service SecretsHub, exactly two RPCs:

rpc GetManifest(GetManifestRequest) returns (GetManifestResponse) — repeated entries mirroring SecretManifestEntry (read the Abstractions type and map EVERY field losslessly: name, revision, timestamp, tombstone flag, and whatever else it carries).
rpc GetSecrets(GetSecretsRequest {repeated string names}) returns (GetSecretsResponse) — repeated rows mirroring StoredSecret verbatim (ciphertext + metadata; read the type first). NO other RPC. csproj: Grpc.AspNetCore + Grpc.Net.Client + Grpc.Tools (match versions used elsewhere in the family), <Version> inherits the family's shared version. Follow the SqlServer package's csproj as the template. Build must pass; commit.

Task 2: Server half — hub service + bearer auth + DI

Classification: high-risk · Est: ~5 min · Parallelizable with: Task 3 Files: Create …Replicator.Grpc/SecretsHubService.cs, …/SecretsHubAuthInterceptor.cs, …/DependencyInjection/SecretsGrpcHubServerExtensions.cs, …/SecretsGrpcHubServerOptions.cs.

SecretsHubService : SecretsHub.SecretsHubBase adapts the registered ISecretStore (manifest + bulk read — reuse/mirror how SqlServerSecretStore produces manifest entries). Auth: interceptor validating Authorization: Bearer <token> against SecretsGrpcHubServerOptions.BearerToken (bound from config; empty/unset ⇒ the service REFUSES to serve — fail closed, log names the config key once). Constant-time comparison. Uniform denial (Unauthenticated, no detail). AddZbSecretsGrpcHub(IServiceCollection, IConfiguration, sectionPath) + MapZbSecretsHub(IEndpointRouteBuilder). TDD: red-first unit tests for the interceptor (valid / invalid / missing / unset-key-refuses) and options validation.

Task 3: Client half — pull-only sweep service + DI

Classification: high-risk · Est: ~5 min · Parallelizable with: Task 2 Files: Create …Replicator.Grpc/GrpcSecretsHubClient.cs, …/GrpcSecretSyncService.cs, …/SecretsGrpcHubClientOptions.cs, …/DependencyInjection/SecretsGrpcHubClientExtensions.cs.

GrpcSecretsHubClient wraps the generated client (bearer header per call; per-call deadline from options). GrpcSecretSyncService : BackgroundService mirrors SqlServerSecretSyncService EXCEPT pull-only: manifest → SecretReplicationReconciler.ComputePullSet → GetSecrets → ApplyAsync (+ cache invalidation seam, same as SqlServer's ctor wiring). NEVER computes a push set; NEVER throws out of ExecuteAsync (hub outage ⇒ warn + retry next tick — copy the SqlServer service's log/exception discipline). Options: Endpoint (required), BearerToken (required, fail-closed), SyncInterval (default 30 s), SyncOnStartup (default true), CallDeadline (default 30 s); Validate() called at registration like SqlServerSecretsOptions. TDD: red-first tests with a fake hub — pull convergence, tombstone apply, outage → no throw + local store untouched, options validation.

Task 4: Integration tests + no-write-surface pin

Classification: standard · Est: ~5 min · Parallelizable with: none (needs 2+3) Files: Create ZB.MOM.WW.Secrets/tests/ZB.MOM.WW.Secrets.Replicator.Grpc.Tests/ (project + suites); Modify slnx.

In-process server via Microsoft.AspNetCore.TestHost (real Kestrel-less gRPC): end-to-end convergence (seed hub store → client store converges, byte-identical ciphertext), delete propagates as tombstone, hub down at client boot ⇒ client serves last-known-good and converges after the hub returns, auth negatives fail closed (bad/missing/unset both sides). Plus the contract pin: a reflection test over the generated SecretsHub service asserting the RPC set is EXACTLY {GetManifest, GetSecrets} — a third RPC (any write surface) fails the build. Full dotnet test on the Secrets slnx green, 0 warnings.

Task 5: Version bump + pack + publish 0.4.0

Classification: small · Est: ~4 min · Parallelizable with: none Files: Modify ZB.MOM.WW.Secrets/Directory.Build.props (version → 0.4.0), CHANGELOG, pack docs.

Bump the family version to 0.4.0, CHANGELOG entry (new package, no breaking changes; existing packages unchanged behaviorally), dotnet pack -c Release, push ALL packages (now 6 + CLI as applicable) to the dohertj2-gitea feed, restore-verify from a scratch consumer (the established pattern from 0.2.x/0.3.0 publishes). Commit + push scadaproj main.

Task 6: ScadaBridge wiring — central hub + site sweep client

Classification: high-risk · Est: ~5 min · Parallelizable with: none (needs Task 5's feed package) Repo: ~/Desktop/ScadaBridge (branch feat/secrets-grpc-hub off main). Files: Modify the central gRPC endpoint composition (find where the ClusterClient→gRPC migration's central services are mapped), the site-role composition root, Directory.Packages.props (Secrets → 0.4.0 + new package ref), appsettings templates; Create DI pin tests.

Central: AddZbSecretsGrpcHub + MapZbSecretsHub on the existing central gRPC listener, bearer token from appsettings/env (${secret:} is NOT usable here — chicken-and-egg; follow the mesh PSK pattern), fail-closed. Site: register the sweep client under Secrets:Replication:Enabled=true + a mode/section selecting Grpc — mirror how the existing replicator registration is switched; central must NOT register the client, sites must NOT register the hub. Red-first WebApplicationFactory/DI pin tests over the real Program.cs for both roles (the ScadaBridge#22 test pattern). Full ScadaBridge offline suite green before commit.

Task 7: ScadaBridge live gate (docker cluster)

Classification: high-risk · Est: live-rig, serial · Parallelizable with: Task 8 Files: Create ScadaBridge/docs/plans/2026-08-07-secrets-grpc-hub-live-gate.md (gate record).

On the local 8-node docker cluster (rebuild via bash docker/deploy.sh with the flag enabled on site-a + central): (1) secret added centrally converges to both site-a nodes ≤ sweep interval, byte-identical, decrypt-verified; (2) site pair cold-boots with central stopped ⇒ serves last-known-good, converges after central returns; (3) central delete tombstone propagates, no resurrection on site restart; (4) auth negatives fail closed, zero key material in logs (grep the logs to prove it). Record PASS/FAIL per check in the gate doc; fix-forward defects on the branch. Merge feat/secrets-grpc-hub → main, push.

Task 8: OtOpcUa Akka pair-local re-gate (three-mesh rig)

Classification: high-risk · Est: live-rig, config-only (no product code) · Parallelizable with: Tasks 1–7 (independent repo + rig) Repo: ~/Desktop/OtOpcUa. Files: Create OtOpcUa/docs/plans/2026-08-07-secrets-pair-local-live-gate.md.

The 2026-07-18 six-node convergence gate predates the mesh Phase 6 partition — re-gate pair-local semantics on the current three-mesh docker-dev rig (fixtures on 10.100.0.35; dev KEK): (1) seed on one node of site-a ⇒ partner converges ≤60 s byte-identical; (2) no cross-pair propagation to site-b/central (this is now CORRECT behavior); (3) delete propagates within the pair; (4) clean pair restart, no resurrection. Record the gate doc, commit + push OtOpcUa master.

Task 9: Closeout — issue, runbook, index

Classification: small · Est: ~4 min · Parallelizable with: none (last)

Check off the remaining boxes on scadaproj#3 (implementation, ScadaBridge gate, OtOpcUa re-gate, rotation doc — verify the rotation fan-out paragraph in clustered-secrets.md §Production enablement is still accurate post-implementation and extend if needed). Update components/secrets/GAPS.md + the CLAUDE.md Secrets index row (0.4.0 published, hub shipped, gates PASS with SHAs). Commit + push scadaproj main. Close #3 if every box is checked.

Dispatch notes for the executor

Wave 1: Task 1 ∥ Task 8 (different repos; Task 8 is the long pole — start it first).
Wave 2: Task 2 ∥ Task 3 (disjoint files, same new project).
Wave 3: Task 4 → Task 5 (serial).
Wave 4: Task 6 → Task 7 (serial, ScadaBridge).
Wave 5: Task 9 after everything.
Live-gate tasks (7, 8) need docker orchestration + log inspection — run them via a single agent each (or main loop), never split one gate across agents. They use different docker hosts (ScadaBridge = local docker; OtOpcUa rig = 10.100.0.35 + local compose) so they may overlap.
Rig credentials/keys: dev KEK and rig PSKs only; never commit rig overrides; never echo key material (standing rules).
Commits end with the session trailer: Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1

11 KiB Raw Blame History Unescape Escape