dohertj2/lmxopcua
1
0
Fork 0
Code Issues 8 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(config): add ClusterNode.MaintenanceMode — the hatch the live gate proved missing

Browse Source 
Phase 1 gate step 4 failed: setting Enabled = 0 to take a node out of service
returns 422 ClusterEnabledNodeCountMismatch, because
DraftValidator.ValidateClusterTopology requires the enabled-node count to equal
ServerCluster.NodeCount. On a Warm/Hot pair — every cluster on the rig, and
every cluster in the target topology — Enabled can therefore never be the
maintenance hatch. The plan called step 4 "the check that makes step 3
acceptable to ship", so Task 3's behaviour change was not shippable as it stood:
a node down for maintenance would block every deployment to its cluster.

The two rules were each reasonable and contradictory together — the validator
reads Enabled as "part of the declared topology", Phase 1 additionally read it
as "expect an ack". Split the meanings rather than weaken either rule:

  Enabled          part of the declared topology  (validator, untouched)
  MaintenanceMode  expected to participate now    (coordinator + reconciler)

Rejected alternatives: counting configured rather than enabled nodes (drops the
guard against booting a pair into InvalidTopology); downgrading the rule to a
warning (weakens a deploy gate for everyone); shipping with no hatch.

Sabotage: dropping !n.MaintenanceMode from the coordinator query turns the new
test red. It also asserts both nodes remain Enabled, so the fix cannot quietly
regress to disabling the row after all.

Configuration.Tests 95/95, ControlPlane.Tests 101/101, solution builds clean.

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
This commit is contained in:
Joseph Doherty
2026-07-22 09:07:29 -04:00
parent 57e1d01766
commit ee69caa270
12 changed files with 2039 additions and 32 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/v2/config-db-schema.md
+22 -4
View File
@@ -154,6 +154,7 @@ CREATE TABLE dbo.ClusterNode (
ServiceLevelBase tinyint NOT NULL DEFAULT 200,
DriverConfigOverridesJson nvarchar(max) NULL CHECK (DriverConfigOverridesJson IS NULL OR ISJSON(DriverConfigOverridesJson) = 1),
Enabled bit NOT NULL DEFAULT 1,
MaintenanceMode bit NOT NULL DEFAULT 0,
LastSeenAt datetime2(3) NULL,
CreatedAt datetime2(3) NOT NULL DEFAULT SYSUTCDATETIME(),
CreatedBy nvarchar(128) NOT NULL
@@ -190,10 +191,27 @@ rows against observed cluster membership and logs an Error on mismatch. It reads
than having each driver node assert its own row, because Phase 4 removes the driver nodes' ConfigDb
connection entirely.
`Enabled` also gained a second meaning in Phase 1: it is the **expected-ack set** for a deployment.
`ConfigPublishCoordinator` no longer derives that set from cluster membership, so an enabled row whose
node is down now fails the deployment at the apply deadline instead of letting it seal green without
that node. Set `Enabled = 0` for a node taken down for maintenance.
#### `Enabled` vs `MaintenanceMode` — two flags, two questions
Phase 1 made the rows the deploy path's **expected-ack set**: `ConfigPublishCoordinator` no longer
derives it from cluster membership, so a row whose node is down now fails the deployment at the apply
deadline instead of letting it seal green without that node.
The escape hatch is **`MaintenanceMode = 1`**, not `Enabled = 0`:
| Flag | Question it answers | Checked by |
|---|---|---|
| `Enabled` | Is this node part of the cluster's declared topology? | `DraftValidator.ValidateClusterTopology` — the enabled-node count must equal `ServerCluster.NodeCount` |
| `MaintenanceMode` | Should we expect it to participate right now? | `ConfigPublishCoordinator` (expected-ack set) and `ClusterNodeAddressReconciler` (absence warnings) |
`Enabled = 0` on one node of a Warm/Hot pair is **rejected at the deploy gate** with
`ClusterEnabledNodeCountMismatch` — the enabled count drops to 1 while `NodeCount` stays 2. Since
every cluster in the target topology is a 2-node pair, `Enabled` cannot serve as the maintenance
hatch at all. The Phase 1 live gate found this; the two rules were independently reasonable and
contradictory together, so the meanings were split rather than either rule being weakened.
A node in maintenance is still enabled, so the topology gate still passes, and the runtime redundancy
posture (`NodeCount` / `RedundancyMode`) is unchanged.
`DriverConfigOverridesJson` shape: