docs(mesh): ConfigSource/ConfigServe config, rig flip switch, Phase 3 as shipped
Phase 3 Task 9. docker-dev rig: central-1/2 carry ConfigServe (:4055 h2c + committed dev key) and stay Direct; the four site nodes carry ConfigSource (both central endpoints + matching key) defaulting to Direct. Flip only the site nodes with OTOPCUA_CONFIG_MODE=FetchAndCache at 'docker compose up' — central keeps SQL and models the target topology. Binding :4055 activates the dedicated-h2c Kestrel takeover on central (re-binds :9000 too). Docs: new docs/Configuration.md ConfigSource/ConfigServe section; docs/Redundancy.md 'config bytes travel out-of-band' note; the program plan Phase 3 section + tracking row flipped to CODE-COMPLETE; a CLAUDE.md 'Config source' section. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
This commit is contained in:
@@ -276,6 +276,45 @@ and `docs/plans/2026-07-22-mesh-phase2-clusterclient-transport.md`.
|
||||
- **There is no such thing as a cluster-scoped deployment.** `Deployment` has no `ClusterId`, `ConfigComposer` always snapshots the whole DB, and `DeploymentArtifact.ResolveClusterScope` is *node-side* self-scoping of a fleet-wide artifact. The phase-1 plan asked for cluster-scope filtering of the ack set; it was dropped as describing a feature that does not exist.
|
||||
- `ClusterNode` gained `AkkaPort` (NOT NULL, 4053) + `GrpcPort` (nullable) as **central's dial targets** for Phases 2/5. They duplicate the node's own `Cluster:Port` / `Cluster:PublicHostname` with no schema-level enforcement, so `ClusterNodeAddressReconcilerActor` (admin singleton) logs an Error on drift. **Phase 2 must revisit it** — once the meshes split, an admin node cannot see site members and every site row would report `EnabledRowNotInCluster` forever.
|
||||
|
||||
## Config source (`ConfigSource` / `ConfigServe`)
|
||||
|
||||
Per-cluster mesh **Phase 3** (config fetch-and-cache) added a second way a **driver** node obtains its
|
||||
deployed configuration, selected by `ConfigSource:Mode`: `Direct` (read `Deployment.ArtifactBlob` from
|
||||
central SQL — **the default**, byte-for-byte today's behaviour) or `FetchAndCache` (fetch the artifact
|
||||
bytes from central over a dedicated-h2c **gRPC stream**, verify `SHA-256 == RevisionHash`, cache in
|
||||
LocalDb, and read **only** the cache — no central-SQL config read). Central (admin role) serves via
|
||||
`DeploymentArtifactService` behind `ConfigServeAuthInterceptor` on `ConfigServe:GrpcListenPort`, gated by
|
||||
a **shared node bearer key** (`ConfigServe:ApiKey == ConfigSource:ApiKey`, `FixedTimeEquals`,
|
||||
fail-closed). Things worth knowing before touching it:
|
||||
|
||||
- **The serve side is wired in both modes** (mapped whenever `ConfigServe:GrpcListenPort > 0` on an
|
||||
admin node), so flipping a node to `FetchAndCache` is a config change, not a redeploy — the same
|
||||
dark-switch discipline as [MeshTransport](#mesh-command-transport-meshtransport).
|
||||
- **`RevisionHash == SHA-256(artifact bytes)` is load-bearing.** `ConfigComposer` computes the revision
|
||||
hash as the lowercase-hex SHA-256 of the exact blob, so the node verifies a fetch against the
|
||||
`RevisionHash` the dispatch already carries — **no hash rides in the proto, and `DispatchDeployment`
|
||||
stays payload-free.** A future change to how the revision hash is derived silently breaks every fetch.
|
||||
- **A null fetch is "no answer," never an empty config (#485).** All endpoints down / `NotFound` / SHA
|
||||
mismatch / zero-length ⇒ the fetcher returns null ⇒ the apply FAILS, the revision does not advance,
|
||||
and the node keeps serving last-known-good. It never tears down to an empty address space. At boot a
|
||||
`FetchAndCache` node restores served state from the LocalDb pointer with **no** central-SQL read; an
|
||||
empty/unreadable cache lands Steady-with-no-revision (the first dispatch fetches), **never Stale**.
|
||||
- **Both pair nodes fetch independently** — no Primary gating of the fetch (same shape as both reading
|
||||
SQL today; `StoreAsync` is idempotent and the cache replicates).
|
||||
- **The dedicated h2c listener coexists with the LocalDb-sync listener.** A fused admin+driver node with
|
||||
both ports set re-binds its existing HTTP surface exactly **once** and adds both dedicated HTTP/2-only
|
||||
ports — getting this wrong either double-binds (crash) or silently unbinds the AdminUI (the Kestrel
|
||||
`Listen*`-discards-`ASPNETCORE_URLS` trap the LocalDb block already warned about).
|
||||
- **Phase boundary: config *reads* only.** A `FetchAndCache` node still writes its `NodeDeploymentState`
|
||||
ack row to central SQL; `DbHealthProbe` and `EfAlarmConditionStateStore` still touch SQL. Cutting those
|
||||
is Phase 4 — resist folding it in, or the "no SQL" scope becomes unprovable-in-parts.
|
||||
|
||||
On the docker-dev rig the central pair carries `ConfigServe` (`:4055`, committed dev key) and stays
|
||||
`Direct`; the four site nodes carry `ConfigSource` (endpoints + matching key) defaulting to `Direct`.
|
||||
Flip only the site nodes with `OTOPCUA_CONFIG_MODE=FetchAndCache` at `docker compose up`. See
|
||||
`docs/Configuration.md` §`ConfigSource` / `ConfigServe`, `docs/Redundancy.md` §"Command transport", and
|
||||
`docs/plans/2026-07-22-mesh-phase3-config-fetch-and-cache.md`.
|
||||
|
||||
## LocalDb pair-local store (Phases 1 + 2)
|
||||
|
||||
Every **driver-role** node keeps a consolidated `ZB.MOM.WW.LocalDb` SQLite database (the retired
|
||||
|
||||
Reference in New Issue
Block a user