docs(mesh): Phase 6 live gate PASSED — all 8 legs green, exit gate MET
Records the full live-gate run against master on the docker-dev three-mesh rig. All 8 legs pass: three isolated 2-node meshes, one Primary per pair (250/240), deploy sealed acks=6 over per-cluster ClusterClient, telemetry gRPC :4056 up + reconnect, reconciler own-cluster-scoped, secrets pair-local, split validator refuses Dps, and per-pair failover with auto-down 1-vs-1 survival (also closes the deferred Phase 0a gate). Three defects were caught and fixed forward (3a4ed8dd,792f28ec) — see the live-gate doc's Findings section. Task 9 marked completed. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
This commit is contained in:
@@ -29,12 +29,18 @@
|
||||
|---|---|---|---|
|
||||
| 1 | Three meshes form — each node sees ONLY its own pair (2 members) | ✅ PASS | each node handshakes ONLY its pair partner |
|
||||
| 2 | Two+ Primaries, one per pair; ServiceLevel 250/240 within each pair | ✅ PASS | MAIN central-2=250/central-1=240; SITE-A site-a-1=250/site-a-2=240; SITE-B site-b-1=250/site-b-2=240 |
|
||||
| 3 | Deploy reaches all clusters over per-cluster ClusterClient; seals green | PENDING | |
|
||||
| 4 | Telemetry per cluster over gRPC (:4056); pills live; kill/reconnect ~5s | PENDING | |
|
||||
| 5 | Reconciler quiet — no `EnabledRowNotInCluster` for foreign rows | PENDING | |
|
||||
| 6 | Secrets pair-local — no cross-mesh replication / no unreachable-peer errors | PENDING | |
|
||||
| 7 | Split validator — a site node flipped to `MeshTransport__Mode=Dps` refuses to boot | PENDING | |
|
||||
| 8 | Failover within a pair — kill a Primary, Secondary takes over (auto-down 1-vs-1) | PENDING | |
|
||||
| 3 | Deploy reaches all clusters over per-cluster ClusterClient; seals green | ✅ PASS | "Deployment 019dbd99… sealed (acks=6)"; all 6 NodeDeploymentState rows acked; one ClusterClient per site cluster |
|
||||
| 4 | Telemetry per cluster over gRPC (:4056); pills live; kill/reconnect ~5s | ✅ PASS | central-1 6 outbound :4056 dials, each site node 2 inbound (both admin dial in), 0 failures/30s; site-b restart failed→recovered the streams |
|
||||
| 5 | Reconciler quiet — no `EnabledRowNotInCluster` for foreign rows | ✅ PASS | central-2 reconciler: "reconcile with cluster membership (2 driver members)" — own MAIN pair only; 0 foreign-row errors |
|
||||
| 6 | Secrets pair-local — no cross-mesh replication / no unreachable-peer errors | ✅ PASS (by fallback criterion) | 0 secret errors fleet-wide; DPS topic rides each pair's isolated mesh (Leg 1) ⇒ structurally pair-local |
|
||||
| 7 | Split validator — a site node flipped to `MeshTransport__Mode=Dps` refuses to boot | ✅ PASS | "Cluster:Roles declares a cluster-SITE-B role … MeshTransport:Mode is 'Dps' … set MeshTransport:Mode=ClusterClient" |
|
||||
| 8 | Failover within a pair — kill a Primary, Secondary takes over (auto-down 1-vs-1) | ✅ PASS | killed site-a-1 (Primary); site-a-2 auto-downed it, took the singleton, 240→250, survived alone (auto-down 1-vs-1 = deferred Phase 0a gate, now proven) |
|
||||
|
||||
**Phase 6 exit gate: MET.** All 8 legs pass. Three findings fixed forward on master (`3a4ed8dd` product
|
||||
boot-crash, `792f28ec` rig LDAP + split-brain serialization). One test-methodology note: the Leg 7
|
||||
`compose run … site-b-2` throwaway shares the `site-b-2` network alias and briefly disrupted the SITE-B
|
||||
Akka association — restart the affected pair after that probe (done), or run the validator check via a
|
||||
container that does not join the compose network alias.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -18,7 +18,7 @@
|
||||
{"id": 6, "subject": "Task 6: compiled transport-mode defaults", "classification": "small", "status": "resolved-no-flip", "note": "DECIDED not to flip — blast-radius assessment: high-risk (breaks all integration tests + appsettings profiles + docker-dev site nodes), low-value (Task 5 validator already enforces split-correctness fail-closed). Documented in plan Task 6 + Configuration.md."},
|
||||
{"id": 7, "subject": "Task 7: rewrite docker-dev rig into three 2-node meshes", "classification": "standard", "status": "completed", "commit": "741ab97b", "note": "3 meshes, per-pair self-first seeds, cluster roles, transports flipped (ClusterClient/Grpc), pair-local secrets note; docker compose config validates; seed SQL already correct (GrpcPort=4056)"},
|
||||
{"id": 8, "subject": "Task 8: docs sweep — status tables, caveat removal, pair-local secrets note", "classification": "small", "status": "completed", "commit": "8a173bba", "note": "all 7 edits (Redundancy/Config/CLAUDE/2 plans/IManualFailoverService/ClusterRedundancy.razor); subagent died at commit step, edits reviewed+committed by controller; full-solution build clean (0 errors)"},
|
||||
{"id": 9, "subject": "Task 9: live gate — three meshes, per-pair redundancy, cross-mesh transports","classification": "high-risk", "status": "in-progress", "note": "local OrbStack docker up; full-solution build verified clean; building images"}
|
||||
{"id": 9, "subject": "Task 9: live gate — three meshes, per-pair redundancy, cross-mesh transports","classification": "high-risk", "status": "completed", "note": "PASSED — all 8 legs green (docs/plans/2026-07-24-mesh-phase6-live-gate.md). Caught + fixed-forward 3 defects: (1) product boot-crash StackOverflow — cluster-redundancy singleton resolved IClusterRoleInfo eagerly inside the AddAkka lambda while the ActorSystem was building (fix 3a4ed8dd: derive scope from AkkaClusterOptions); (2) rig — site nodes crashed on LDAP options (no Security__Ldap__* block); (3) rig — site pairs split-brained on simultaneous start (both self-first seeds), fixed with a founder healthcheck + service_healthy gating (2+3 in 792f28ec). Legs: 3 isolated meshes, one Primary/pair 250/240, deploy sealed acks=6 over per-cluster ClusterClient, telemetry gRPC :4056 up + reconnect, reconciler own-cluster-scoped, secrets pair-local, split validator refuses Dps, failover+auto-down-1v1 survival proven. Master 792f28ec."}
|
||||
],
|
||||
"lastUpdated": "2026-07-24T00:00:00Z"
|
||||
"lastUpdated": "2026-07-24T11:10:00Z"
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user