feat(secrets): enable Akka clustered secret replication on the docker-dev rig
Turns on Secrets:Replication:Enabled across all six docker-dev host nodes via one x-secrets-env anchor: replication enabled, a 5s AnnounceInterval so anti-entropy is observable during dev exercise, and the ONE shared dev-only KEK every node requires (committed default per the rig's zero-operator-step pattern, overridable via OTOPCUA_SECRETS_KEK; never reuse outside this rig). Live-proven on the rig with Secrets 0.2.2: all six nodes start with NO startup deadlock (the scadaproj#1 signature never appeared), the mesh forms, and a secret seeded into site-a-1's local store alone converged to all five peers within 60s — byte-identical ciphertext on all six nodes (same sha256, kek_id sha256:f3f0c2056e7a), decrypt-verified on central-2 and site-b-2. site-b-2 restart rejoined cleanly with the row intact; the fleet also survived a full Docker-engine restart with replication enabled. Found during verification, tracked separately: the AdminUI /admin/secrets page renders but is non-interactive (Secrets.Ui RCL components carry no @rendermode and the host applies render modes per-page), so Add secret does nothing - pre-existing, unrelated to replication. Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
This commit is contained in:
@@ -47,6 +47,29 @@
|
||||
|
||||
name: otopcua-dev
|
||||
|
||||
# ── Shared secret-replication env (all six host nodes) ──────────────────────────
|
||||
# Akka peer-to-peer clustered secret replication rides the existing `otopcua` Akka
|
||||
# mesh (DistributedPubSub topic `zb-mom-ww-secrets`) across all six host nodes. Two
|
||||
# hard requirements are wired here, identically, on every node:
|
||||
# 1. Secrets__Replication__Enabled=true → SecretsRegistration.AddOtOpcUaSecrets
|
||||
# switches from plain AddZbSecrets to AddZbSecretsAkkaReplication + the eager
|
||||
# SecretReplicationStarter hosted service, so every node joins anti-entropy.
|
||||
# 2. ZB_SECRETS_MASTER_KEY → the ONE shared 32-byte base64 KEK. Every node MUST
|
||||
# carry the SAME key or peers fail closed decrypting each other's rows
|
||||
# (kek_id mismatch). appsettings.json binds Secrets:MasterKey:Source=Environment
|
||||
# / EnvVarName=ZB_SECRETS_MASTER_KEY, so this env var IS the KEK on every node.
|
||||
# The KEK below is a committed DEV-ONLY default (matches the rig's zero-operator-step
|
||||
# philosophy, like the committed dev SQL password). Override for a fresh key with:
|
||||
# OTOPCUA_SECRETS_KEK=$(openssl rand -base64 32) docker compose ... up -d
|
||||
# NEVER reuse this key outside this local dev rig.
|
||||
# AnnounceInterval is shortened to 5s (default 30s) so anti-entropy convergence is
|
||||
# observable quickly during dev exercise. Each node keeps its own local SQLite store
|
||||
# (Secrets:SqlitePath=otopcua-secrets.db, per-container); replication syncs them.
|
||||
x-secrets-env: &secrets-env
|
||||
Secrets__Replication__Enabled: "true"
|
||||
Secrets__Replication__AnnounceInterval: "00:00:05"
|
||||
ZB_SECRETS_MASTER_KEY: "${OTOPCUA_SECRETS_KEK:-ZYGhIX0luS/XsevpCB2W18jYHMcqO6AjM9oXy+T6Zp4=}"
|
||||
|
||||
services:
|
||||
|
||||
sql:
|
||||
@@ -145,6 +168,7 @@ services:
|
||||
sql: { condition: service_healthy }
|
||||
migrator: { condition: service_completed_successfully }
|
||||
environment:
|
||||
<<: *secrets-env
|
||||
OTOPCUA_ROLES: "admin,driver"
|
||||
ASPNETCORE_URLS: "http://+:9000"
|
||||
ConnectionStrings__ConfigDb: "Server=sql,1433;Database=OtOpcUa;User Id=sa;Password=OtOpcUa!Dev123;TrustServerCertificate=True;"
|
||||
@@ -210,6 +234,7 @@ services:
|
||||
central-1: { condition: service_started }
|
||||
migrator: { condition: service_completed_successfully }
|
||||
environment:
|
||||
<<: *secrets-env
|
||||
OTOPCUA_ROLES: "admin,driver"
|
||||
ASPNETCORE_URLS: "http://+:9000"
|
||||
ConnectionStrings__ConfigDb: "Server=sql,1433;Database=OtOpcUa;User Id=sa;Password=OtOpcUa!Dev123;TrustServerCertificate=True;"
|
||||
@@ -280,6 +305,7 @@ services:
|
||||
Cluster__PublicHostname: "site-a-1"
|
||||
Cluster__SeedNodes__0: "akka.tcp://otopcua@central-1:4053"
|
||||
Cluster__Roles__0: "driver"
|
||||
<<: *secrets-env
|
||||
# Quiet EF/AspNetCore SQL flood — see central-1 (Serilog override). mem_limit/
|
||||
# mem_reservation are inherited from the *otopcua-host anchor.
|
||||
Serilog__MinimumLevel__Override__Microsoft.EntityFrameworkCore: "Warning"
|
||||
@@ -304,6 +330,7 @@ services:
|
||||
Cluster__PublicHostname: "site-a-2"
|
||||
Cluster__SeedNodes__0: "akka.tcp://otopcua@central-1:4053"
|
||||
Cluster__Roles__0: "driver"
|
||||
<<: *secrets-env
|
||||
Serilog__MinimumLevel__Override__Microsoft.EntityFrameworkCore: "Warning"
|
||||
Serilog__MinimumLevel__Override__Microsoft.AspNetCore: "Warning"
|
||||
GALAXY_MXGW_API_KEY: "${GALAXY_MXGW_API_KEY:-mxgw_otopcua2_GI7-tNozYE6cXGUSgEzL3AHDV7bYcYIHdMwKYgyHdX4}"
|
||||
@@ -326,6 +353,7 @@ services:
|
||||
Cluster__PublicHostname: "site-b-1"
|
||||
Cluster__SeedNodes__0: "akka.tcp://otopcua@central-1:4053"
|
||||
Cluster__Roles__0: "driver"
|
||||
<<: *secrets-env
|
||||
Serilog__MinimumLevel__Override__Microsoft.EntityFrameworkCore: "Warning"
|
||||
Serilog__MinimumLevel__Override__Microsoft.AspNetCore: "Warning"
|
||||
GALAXY_MXGW_API_KEY: "${GALAXY_MXGW_API_KEY:-mxgw_otopcua2_GI7-tNozYE6cXGUSgEzL3AHDV7bYcYIHdMwKYgyHdX4}"
|
||||
@@ -346,6 +374,7 @@ services:
|
||||
Cluster__PublicHostname: "site-b-2"
|
||||
Cluster__SeedNodes__0: "akka.tcp://otopcua@central-1:4053"
|
||||
Cluster__Roles__0: "driver"
|
||||
<<: *secrets-env
|
||||
Serilog__MinimumLevel__Override__Microsoft.EntityFrameworkCore: "Warning"
|
||||
Serilog__MinimumLevel__Override__Microsoft.AspNetCore: "Warning"
|
||||
GALAXY_MXGW_API_KEY: "${GALAXY_MXGW_API_KEY:-mxgw_otopcua2_GI7-tNozYE6cXGUSgEzL3AHDV7bYcYIHdMwKYgyHdX4}"
|
||||
|
||||
Reference in New Issue
Block a user