docs(secrets): pair-local replication live gate on the three-mesh rig — PASS 4/4
Re-gates Akka peer-to-peer secret replication on the current three-mesh docker-dev rig and supersedes the 2026-07-18 six-node gate (lmxopcua#482,c878fbbd) as evidence. That gate proved convergence to all five peers on the old single fleet-wide mesh; after mesh Phase 6 (2839deb5) partitioned the rig into three independent 2-node meshes, six-node convergence would be a FAILURE, not a pass. Phase 6's own leg 6 recorded "PASS (by fallback criterion)" — it inferred pair-locality structurally and never seeded a secret. This replaces that inference with direct evidence. All four checks PASS, on master @21009874with one image (OtOpcUa.Host.dll a434b04f byte-identical on all six nodes), Secrets libs 0.2.3: 1. Pair convergence — seeded on site-a-1 alone, on site-a-2 in <=7s (budget 60s), byte-identical ciphertext + kek_id + updated_utc, decrypt-verified on the partner. 2. No cross-pair propagation — absent from site-b and central across 8 samples / 240s / 48 announce intervals, and clean in the reverse direction too. Isolation controls seeded on site-b-1 and central-1 converged within their OWN pairs, so the absence is isolation rather than a dead replicator. 3. Delete propagates within the pair — issued on site-a-2 (the receiving node, so this also covers the reverse intra-pair direction); tombstone on site-a-1 in <1s. 4. Clean pair restart (both together) — mesh re-formed in 7s, no resurrection of the tombstone, live secret survived and decrypted correctly on both nodes. Zero secret-related warnings or errors fleet-wide across the whole run. Satisfies the scadaproj#3 precondition for the OtOpcUa production topology (Akka pair-local). Secrets:Replication:Enabled remains default-false in the product; the rig KEK is dev-only. Rig config: SqlitePath moved onto each node's existing per-node /app/data volume. The appsettings default resolves to /app, inside the image's writable container layer, so the store was destroyed by any `up -d --build` recreate and was unreachable from the host — seeding or inspecting a node meant stopping the container to docker cp the file out. Still one private store per node; replication is still what syncs them. Residuals in the gate doc: the live-push path is not covered (anti-entropy is what a driver-only node can be gated on), and appsettings.json still carries a stale "KNOWN NON-FUNCTIONAL against 0.2.0 / do not enable" comment that this gate disproves — left uncorrected here because this run is config-only. Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1
This commit is contained in:
@@ -115,8 +115,19 @@ name: otopcua-dev
|
||||
# OTOPCUA_SECRETS_KEK=$(openssl rand -base64 32) docker compose ... up -d
|
||||
# NEVER reuse this key outside this local dev rig.
|
||||
# AnnounceInterval is shortened to 5s (default 30s) so anti-entropy convergence is
|
||||
# observable quickly during dev exercise. Each node keeps its own local SQLite store
|
||||
# (Secrets:SqlitePath=otopcua-secrets.db, per-container); replication syncs them.
|
||||
# observable quickly during dev exercise. Each node keeps its OWN local SQLite store;
|
||||
# replication syncs them.
|
||||
#
|
||||
# SqlitePath is overridden onto the node's existing per-node `/app/data` volume (the one
|
||||
# LocalDb already uses) instead of the appsettings default `otopcua-secrets.db`, which
|
||||
# resolves to /app — inside the image's writable container layer. Two things that costs,
|
||||
# both of which bit the 2026-08-07 pair-local gate: the store is DESTROYED by any
|
||||
# `up -d --build` container recreate (so a seeded row cannot outlive a rig rebuild), and
|
||||
# it is unreachable from the host, so seeding or inspecting a node's store meant stopping
|
||||
# the container and `docker cp`-ing the file out. On the volume, each node still has a
|
||||
# private store (one volume per node — this is NOT shared storage, and replication is
|
||||
# still what syncs them), but it survives recreate and a helper container can mount it
|
||||
# read/write while the node keeps running.
|
||||
#
|
||||
# Per-cluster mesh Phase 6: this DPS topic now rides each pair's OWN 2-node mesh, not one
|
||||
# shared six-node mesh — so secret replication is PAIR-LOCAL. central-1/central-2 replicate
|
||||
@@ -127,6 +138,7 @@ name: otopcua-dev
|
||||
# was needed for this — Enabled=true + the same KEK on every node is still correct; it just now
|
||||
# converges within three independent 2-node groups instead of one six-node group.
|
||||
x-secrets-env: &secrets-env
|
||||
Secrets__SqlitePath: "/app/data/otopcua-secrets.db"
|
||||
Secrets__Replication__Enabled: "true"
|
||||
Secrets__Replication__AnnounceInterval: "00:00:05"
|
||||
ZB_SECRETS_MASTER_KEY: "${OTOPCUA_SECRETS_KEK:-ZYGhIX0luS/XsevpCB2W18jYHMcqO6AjM9oXy+T6Zp4=}"
|
||||
|
||||
Reference in New Issue
Block a user