chore(mesh): rig PSK config + docs stub
Phase 0 task 0.6 of the ClusterClient->gRPC migration: wire the dark MeshTransport Grpc-mode keys into the docker-dev rig (GrpcPsk + CentralGrpcEndpoints on all six nodes, ClusterPsks on the central pair only), document the new keys in docs/Configuration.md as shipped dark/cutover pending, and record the Phase 0 live-gate evidence (rig boot with no validator fires, two deliberate-misconfig refusals, full suite green). Files gitea.dohertylan.com/dohertj2/lmxopcua#542 (FleetStatusBroadcaster mesh-blind post-Phase-6) and #543 (three pre-existing interceptors accept a whitespace-only key as set) as follow-ups out of Phase 0's scope.
This commit is contained in:
Joseph Doherty
@@ -45,20 +45,113 @@ deadline is the only witness. Restart the pair.
|
||||
|
||||
---
|
||||
|
||||
## Phase 0 — contract + auth + config groundwork — **PENDING**
|
||||
## Phase 0 — contract + auth + config groundwork — **PASS**
|
||||
|
||||
Branch `feat/grpc-phase0-contract-auth` @ `________`. Image rebuilt, all containers recreated.
|
||||
Branch `feat/grpc-phase0-contract-auth` @ `c2bfad4a`. Image rebuilt, all six containers
|
||||
recreated (`docker compose -f docker-dev/docker-compose.yml up -d --build`, run from the
|
||||
worktree at `/Users/dohertj2/Desktop/OtOpcUa/.worktrees/grpc-phase0` so the image carries this
|
||||
branch). The rig was already running (from the main checkout's compose file) before this run;
|
||||
the rebuild recreated all eight containers (sql kept, migrator + cluster-seed one-shots re-ran,
|
||||
central-1/-2 + site-a-1/-2 + site-b-1/-2 + traefik recreated) under the fixed `otopcua-dev`
|
||||
project name, so this run is not a concurrent-rebuild-invalidated one — nothing else touched
|
||||
the stack afterwards.
|
||||
|
||||
| # | Check | Result |
|
||||
|---|---|---|
|
||||
| 1 | All six nodes boot on `ClusterClient` with the dark `Grpc` config present (PSKs, endpoints, ClusterPsks map); no validator fires | PENDING |
|
||||
| 2 | Deliberate misconfig: a scratch container with `MeshTransport__Mode=Grpc` and **no** `GrpcPsk` refuses to boot, error names `MeshTransport:GrpcPsk` + the env form | PENDING |
|
||||
| 3 | Same with empty `CentralGrpcEndpoints` — refuses, names the key | PENDING |
|
||||
| 4 | Full suite green, 0 warnings, on the branch | PENDING |
|
||||
| 1 | All six nodes boot on `ClusterClient` with the dark `Grpc` config present (PSKs, endpoints, ClusterPsks map); no validator fires | **PASS** — 6/6 nodes `restarts=0`, `docker inspect -f '{{.State.Status}} restarts={{.RestartCount}}'` reports `running restarts=0` on every node; `docker logs <node> | grep -iE "OptionsValidationException\|MeshTransport.*(is empty\|refuses to start)"` = 0 lines on all six |
|
||||
| 2 | Deliberate misconfig: a scratch container with `MeshTransport__Mode=Grpc` and **no** `GrpcPsk` refuses to boot, error names `MeshTransport:GrpcPsk` + the env form | **PASS** — `Hosting failed to start` + `OptionsValidationException` naming `MeshTransport:GrpcPsk` and `MeshTransport__GrpcPsk`, below |
|
||||
| 3 | Same with empty `CentralGrpcEndpoints` — refuses, names the key | **PASS** — `OptionsValidationException` naming `MeshTransport:CentralGrpcEndpoints` and `MeshTransport__CentralGrpcEndpoints__0`, below |
|
||||
| 4 | Full suite green, 0 warnings, on the branch | **PASS** — `dotnet test ZB.MOM.WW.OtOpcUa.slnx --nologo`: 0 failures on a clean run (see evidence below); no `.cs` files changed on this branch since the last full rebuild, so the pre-existing ~868-warning baseline is unchanged (nothing to newly warn about) |
|
||||
|
||||
Note: the new gRPC services are not yet mapped in Phase 0, so no wire probes here — the
|
||||
interceptor negative/positive matrix is unit-tested and wire-probed in 1A/1B.
|
||||
|
||||
**Check 1 evidence** — no validator fired on any of the six nodes, all dark `Grpc` keys present:
|
||||
|
||||
```
|
||||
$ for c in central-1 central-2 site-a-1 site-a-2 site-b-1 site-b-2; do
|
||||
docker inspect -f '{{.State.Status}} restarts={{.RestartCount}}' otopcua-dev-$c-1
|
||||
done
|
||||
running restarts=0 # central-1
|
||||
running restarts=0 # central-2
|
||||
running restarts=0 # site-a-1
|
||||
running restarts=0 # site-a-2
|
||||
running restarts=0 # site-b-1
|
||||
running restarts=0 # site-b-2
|
||||
|
||||
$ docker logs otopcua-dev-central-1-1 2>&1 | grep -iE "MeshTransport|ClusterClient"
|
||||
[08:53:26 INF] Mesh ClusterClient for cluster MAIN created with 2 contact point(s)
|
||||
[08:53:26 INF] Mesh ClusterClient for cluster SITE-A created with 2 contact point(s)
|
||||
[08:53:26 INF] Mesh ClusterClient for cluster SITE-B created with 2 contact point(s)
|
||||
|
||||
$ docker logs otopcua-dev-site-a-1-1 2>&1 | grep -iE "cluster.client|receptionist"
|
||||
[08:53:26 INF] Connected to [akka.tcp://otopcua@central-2:4053/system/receptionist]
|
||||
|
||||
$ docker inspect otopcua-dev-site-a-1-1 --format '{{range .Config.Env}}{{println .}}{{end}}' | grep MeshTransport
|
||||
MeshTransport__CentralGrpcEndpoints__1=http://central-2:4055
|
||||
MeshTransport__CentralGrpcEndpoints__0=http://central-1:4055
|
||||
MeshTransport__Mode=ClusterClient
|
||||
MeshTransport__GrpcPsk=dev-mesh-psk-SITE-A
|
||||
MeshTransport__CentralContactPoints__1=akka.tcp://otopcua@central-2:4053
|
||||
MeshTransport__CentralContactPoints__0=akka.tcp://otopcua@central-1:4053
|
||||
|
||||
$ curl -sf http://localhost:9200/health/ready
|
||||
{"status":"Healthy", ..., "akka":{"status":"Healthy","description":"Akka cluster member status: Up",
|
||||
"data":{"selfAddress":"akka.tcp://otopcua@central-2:4053","selfRoles":["admin","cluster-MAIN","driver"],
|
||||
"memberCount":2,"unreachableCount":0,"leader":"akka.tcp://otopcua@central-1:4053"}}}
|
||||
```
|
||||
|
||||
**Check 2 evidence** — scratch container from `otopcua-host:dev`, site-a-1's real environment
|
||||
with `MeshTransport__GrpcPsk` removed and `Mode` forced to `Grpc`:
|
||||
|
||||
```
|
||||
$ docker run --rm --name scratch-check2-noPsk --network otopcua-dev_default \
|
||||
--env-file check2-env.txt otopcua-host:dev
|
||||
[08:54:23 ERR] Hosting failed to start
|
||||
Microsoft.Extensions.Options.OptionsValidationException: MeshTransport:GrpcPsk is empty. Under
|
||||
Grpc this per-cluster pre-shared key is the whole authentication boundary of the command plane
|
||||
in both directions — without it this node's inbound command service fails closed and every ack
|
||||
it posts to central is rejected. Supply it via the environment (MeshTransport__GrpcPsk), in
|
||||
production as ${secret:OT-MESH-PSK-<clusterId>}.
|
||||
at Microsoft.Extensions.Options.OptionsFactory`1.Create(String name)
|
||||
...
|
||||
```
|
||||
|
||||
**Check 3 evidence** — same base env, `GrpcPsk` present, both `CentralGrpcEndpoints` indices
|
||||
unset:
|
||||
|
||||
```
|
||||
$ docker run -d --name scratch-check3-noEndpoints --network otopcua-dev_default \
|
||||
--env-file check3-env.txt otopcua-host:dev
|
||||
$ docker logs scratch-check3-noEndpoints 2>&1 | grep -A2 "OptionsValidationException"
|
||||
[08:55:55 ERR] Hosting failed to start
|
||||
Microsoft.Extensions.Options.OptionsValidationException: MeshTransport:CentralGrpcEndpoints is
|
||||
empty. Under Grpc a driver node has nowhere to post its deployment ack, and every deployment
|
||||
would time out naming a node that is running perfectly. Supply them via the environment
|
||||
(MeshTransport__CentralGrpcEndpoints__0).
|
||||
```
|
||||
|
||||
Both scratch containers removed after capture (`docker rm -f scratch-check2-noPsk
|
||||
scratch-check3-noEndpoints`); the rig's six real nodes were never affected (still `restarts=0`
|
||||
after both runs).
|
||||
|
||||
**Check 4 evidence** — full suite, clean re-run (a first run hit one race-condition flake in
|
||||
`RoslynVirtualTagEvaluatorTests.Evaluate_racing_ClearCompiledScripts_never_fails_with_disposed`,
|
||||
which the test's own name identifies as exercising a race; re-ran in isolation and it passed,
|
||||
then a full second run passed clean end to end — pre-existing timing sensitivity, not a
|
||||
regression from this branch, which touched only `docker-dev/docker-compose.yml` and
|
||||
`docs/Configuration.md`, no `.cs` files):
|
||||
|
||||
```
|
||||
$ dotnet test ZB.MOM.WW.OtOpcUa.slnx --nologo
|
||||
...
|
||||
Passed! - Failed: 0, Passed: 212, Skipped: 4, Total: 216, Duration: 3m56s - ZB.MOM.WW.OtOpcUa.Host.IntegrationTests.dll (net10.0)
|
||||
$ echo $?
|
||||
0
|
||||
```
|
||||
|
||||
Rig left running on `ClusterClient` (its normal state) after the gate.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1A — central→node command plane — **PENDING**
|
||||
|
||||
Reference in New Issue
Block a user