fix(docker): give every node an explicit secret store path, not just the hub four

The 0.6.2 adoption set Secrets__SqlitePath inside x-secrets-hub-env, which only the
central pair and site-a merge — clustered replication is deliberately enabled on four
nodes so the default-OFF posture is proven side by side. site-b and site-c merge no
secrets anchor at all, so they fell back to the shipped default and crash-looped at
boot (exit 134, nine times each) while the other four came up clean.

Splitting the store path into its own anchor that all eight merge fixes it. The path
is not optional config: every node has a store, only four have a hub.

Also corrects the assumption behind deleting the key from appsettings.json. Measured
here: with a Secrets section present but no SqlitePath, the effective value is the
RELATIVE "secrets.db", which fails both validation rules — the per-user default does
not take over. A deployment must set it explicitly.

Verified live: all 8 nodes up, zero validation errors, stores at /data on the same
host directory as before, all four pairs converged to one cluster each through a
simultaneous recreate, central-a active / central-b standby, secrets hub sweeps
recovered after the expected boot race.
This commit is contained in:
Joseph Doherty
2026-08-11 10:26:03 -04:00
parent a4e920902c
commit da1669f87a
+27 -11
View File
@@ -1,3 +1,22 @@
# EVERY node needs a secret store path; only four take the hub anchor below. Keep this
# anchor separate from that one — folding it in is what crash-looped site-b and site-c on
# 2026-08-11: they merge no secrets anchor, fell back to the shipped default, and were
# rejected at boot.
#
# The value must be absolute AND outside the content root (/app) — Secrets 0.6.2 validates
# both at startup, because a store inside the deployment directory is destroyed by an
# ordinary upgrade (this wiped the MxGateway key store on 2026-08-09). /data is the node's
# own bind mount, the SAME host directory as /app/data, so an existing store is found
# unmoved; only the path used to reach it changes. Do NOT point this back at /app/data —
# absolute is not sufficient, and the boot will fail.
#
# Do NOT delete this key expecting the library's per-user default to take over. Measured on
# this rig: with a Secrets section present but no SqlitePath, the effective value is the
# RELATIVE "secrets.db", which fails both rules. Setting it explicitly is the supported
# posture for a deployment in any case.
x-secrets-store-env: &secrets-store-env
Secrets__SqlitePath: "/data/scadabridge-secrets.db"
# ── Clustered secret replication: the pull-only gRPC hub (scadaproj#3) ─────────
#
# Central hosts the hub on its EXISTING h2c control-plane listener (CentralGrpcPort
@@ -21,13 +40,6 @@ x-secrets-hub-env: &secrets-hub-env
# ciphertext replicates, so a node with a different KEK fails closed on resolve with a
# kek_id mismatch that reads like corruption but is a deployment error.
ZB_SECRETS_MASTER_KEY: "zZiBWuoaVMbJmGXToLk9Lakw0iJozXoL/7Gxac3GwJ4="
# Must be absolute AND outside the content root (/app) — Secrets 0.6.2 validates both
# at startup, because a store inside the deployment directory is destroyed by an
# ordinary upgrade (this wiped the MxGateway key store on 2026-08-09). /data is the
# node's own bind mount, the SAME host directory as /app/data, so the existing store
# is found unmoved; only the path used to reach it changes. Do NOT point this back at
# /app/data — absolute is not sufficient, and the boot will fail.
Secrets__SqlitePath: "/data/scadabridge-secrets.db"
Secrets__Replication__Enabled: "true"
Secrets__Replication__Mode: "Grpc"
# DEV-ONLY shared bearer token — NOT a real secret. Presented by every follower and
@@ -77,7 +89,7 @@ services:
environment:
# Hub half of the pull-only gRPC secrets hub + the central-only shared SQL store
# (anchors at the top of this file).
<<: [*secrets-hub-env, *secrets-hub-central-env]
<<: [*secrets-store-env, *secrets-hub-env, *secrets-hub-central-env]
SCADABRIDGE_CONFIG: Central
ASPNETCORE_ENVIRONMENT: Development
ASPNETCORE_URLS: "http://+:5000"
@@ -138,7 +150,7 @@ services:
environment:
# Hub half of the pull-only gRPC secrets hub + the central-only shared SQL store
# (anchors at the top of this file).
<<: [*secrets-hub-env, *secrets-hub-central-env]
<<: [*secrets-store-env, *secrets-hub-env, *secrets-hub-central-env]
SCADABRIDGE_CONFIG: Central
ASPNETCORE_ENVIRONMENT: Development
ASPNETCORE_URLS: "http://+:5000"
@@ -199,7 +211,7 @@ services:
environment:
# Follower half of the pull-only gRPC secrets hub (anchors at the top of this
# file). site-b and site-c deliberately carry neither.
<<: [*secrets-hub-env, *secrets-hub-site-env]
<<: [*secrets-store-env, *secrets-hub-env, *secrets-hub-site-env]
SCADABRIDGE_CONFIG: Site
ports:
- "9021:8082" # Akka remoting (host access for debugging)
@@ -227,7 +239,7 @@ services:
environment:
# Follower half of the pull-only gRPC secrets hub (anchors at the top of this
# file). site-b and site-c deliberately carry neither.
<<: [*secrets-hub-env, *secrets-hub-site-env]
<<: [*secrets-store-env, *secrets-hub-env, *secrets-hub-site-env]
SCADABRIDGE_CONFIG: Site
ports:
- "9022:8082" # Akka remoting
@@ -253,6 +265,7 @@ services:
stop_grace_period: 30s
container_name: scadabridge-site-b-a
environment:
<<: *secrets-store-env
SCADABRIDGE_CONFIG: Site
ports:
- "9031:8082" # Akka remoting
@@ -278,6 +291,7 @@ services:
stop_grace_period: 30s
container_name: scadabridge-site-b-b
environment:
<<: *secrets-store-env
SCADABRIDGE_CONFIG: Site
ports:
- "9032:8082" # Akka remoting
@@ -303,6 +317,7 @@ services:
stop_grace_period: 30s
container_name: scadabridge-site-c-a
environment:
<<: *secrets-store-env
SCADABRIDGE_CONFIG: Site
ports:
- "9041:8082" # Akka remoting
@@ -328,6 +343,7 @@ services:
stop_grace_period: 30s
container_name: scadabridge-site-c-b
environment:
<<: *secrets-store-env
SCADABRIDGE_CONFIG: Site
ports:
- "9042:8082" # Akka remoting