diff --git a/docker-dev/seed/seed-clusters.sql b/docker-dev/seed/seed-clusters.sql index 62bfa596..97ceca09 100644 --- a/docker-dev/seed/seed-clusters.sql +++ b/docker-dev/seed/seed-clusters.sql @@ -106,6 +106,28 @@ IF NOT EXISTS (SELECT 1 FROM dbo.ClusterNode WHERE NodeId = 'site-b-2:4053') (NodeId, ClusterId, Host, OpcUaPort, DashboardPort, AkkaPort, GrpcPort, ApplicationUri, ServiceLevelBase, Enabled, CreatedBy) VALUES ('site-b-2:4053', 'SITE-B', 'site-b-2', 4840, 8081, 4053, 4056, 'urn:OtOpcUa:site-b-2', 150, 1, 'docker-dev-seed'); +------------------------------------------------------------------------------ +-- Backfill: GrpcPort on rows that predate the column (#493) +-- +-- Every insert above is INSERT-if-not-exists, which is the right shape for a re-runnable seed +-- but means a *new nullable* column never reaches rows created by an earlier version of this +-- file. On a persisted volume that is exactly what happened when Phase 5 added GrpcPort: the +-- six ClusterNode rows already existed, so they kept NULL, TelemetryNodeSource found no dial +-- targets, and TelemetryDial:Mode=Grpc silently connected to nothing (throttled Warning per +-- node — the code degrades gracefully, so nothing failed loudly). +-- +-- Backfills NULL only. It must not overwrite a port an operator set deliberately on a +-- long-lived dev volume; a NULL is the unambiguous "this row predates the column" marker. +-- +-- AkkaPort deliberately has no equivalent here: it is NOT NULL with a default of 4053, so +-- pre-existing rows already carry a usable value and there is nothing to backfill. +------------------------------------------------------------------------------ + +UPDATE dbo.ClusterNode + SET GrpcPort = 4056 -- matches Telemetry__GrpcListenPort on all six nodes in docker-compose.yml + WHERE GrpcPort IS NULL + AND CreatedBy = 'docker-dev-seed'; + ------------------------------------------------------------------------------ -- Galaxy MxAccess gateway — INTENTIONALLY NOT SEEDED (retired 2026-06-15) -- diff --git a/docs/Telemetry.md b/docs/Telemetry.md index a24570f1..9f04a30d 100644 --- a/docs/Telemetry.md +++ b/docs/Telemetry.md @@ -128,6 +128,39 @@ transports. ScadaBridge itself has since closed that gap with this identical interceptor pattern, so Phase 5 ships authenticated from the start rather than deferring auth to a later hardening pass. See the design doc's superseded note for the full history. +## Upgrading an existing deployment: populate `ClusterNode.GrpcPort` first (#493) + +`GrpcPort` is a **nullable column added by Phase 5**, and nothing backfills it onto rows that +already existed. On an upgraded deployment every `ClusterNode` row therefore carries `NULL`, +central finds **no dial targets**, and flipping `TelemetryDial:Mode` to `Grpc` connects to +nothing. Fresh installs are unaffected. `AkkaPort` did not have this problem only because it is +`NOT NULL` with a default of 4053. + +Nothing fails loudly, which is what makes this worth stating up front — the code degrades +gracefully, once per node, throttled: + +``` +ClusterNode has no GrpcPort; it exposes no telemetry stream and is skipped in this dial refresh +(further skips of this node are silent) +``` + +Other symptoms: no `ESTABLISHED` sockets on the telemetry port (a `LISTEN` but nothing else in +`docker exec cat /proc/net/tcp6`), and the `/alerts` / `/script-log` pill never turning +live in `Grpc` mode. + +**Before flipping any node to `Grpc`**, set each driver node's `GrpcPort` to that node's own +`Telemetry:GrpcListenPort` — via the AdminUI node editor, or directly: + +```sql +UPDATE dbo.ClusterNode SET GrpcPort = WHERE NodeId = ''; +``` + +There is deliberately **no EF data migration** backfilling this. The correct value is that node's +own listener port, which is per-deployment configuration the database cannot derive; a migration +could only invent one, and a *wrong* dial target is worse than the `NULL` the dialer already skips +explicitly. The docker-dev seed does backfill (`GrpcPort IS NULL AND CreatedBy = 'docker-dev-seed'` +→ 4056) because there the port is fixed by `docker-compose.yml` and therefore actually knowable. + ## Reconnect and reliability Central's `TelemetryDialSupervisor` (an admin-role actor) runs **one reconnecting dialer per