From 50f7439e33255f4fb071acbfdd7e4d0ed8398ec6 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Thu, 20 Aug 2026 16:57:45 -0400 Subject: [PATCH] =?UTF-8?q?docs(deployment):=20QA=20hotfix=20da0bfe2b=20?= =?UTF-8?q?=E2=80=94=20MxGateway=20graceful-stream-end=20disconnect;=20inc?= =?UTF-8?q?ident=20root=20cause=20(dashboard=20session=20close=2015:29:03)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../2026-08-20-wonder-app-vq03-vq04.md | 20 +++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/docs/deployment/deployment-records/2026-08-20-wonder-app-vq03-vq04.md b/docs/deployment/deployment-records/2026-08-20-wonder-app-vq03-vq04.md index daf1066a..4f5983be 100644 --- a/docs/deployment/deployment-records/2026-08-20-wonder-app-vq03-vq04.md +++ b/docs/deployment/deployment-records/2026-08-20-wonder-app-vq03-vq04.md @@ -59,3 +59,23 @@ start Central pair → wait 20 s → start Site pair. Pairs were stopped and sta ## 3. Outstanding - None from this deploy. ZIPSEN (id 7) remains the known non-deployable draft instance. + +## 4. Hotfix `da0bfe2b` — same day, 16:53 (QA only so far) + +**Incident (reported on vq03 Z28061 MoveIn):** `SetAttribute('RightMESReceiver.MoveInCompleteFlag') +failed: Session session-736b2d47… was not found` while the attribute showed GOOD quality. +**Root cause (from both logs):** at 15:29:03 the gateway dashboard user `admin` (client +`10.181.200.169`) closed ScadaBridge's MxGateway session. The gateway completes a closed +session's event stream *gracefully* (OK status), and `MxGatewayDataConnection.RunEventLoopAsync` +only raised `Disconnected` on a *fault* — so the adapter stayed Connected on a dead session for +an hour (values frozen at last GOOD, every write `NotFound`; the 10 s `NotFound` heartbeat was +the unresolved-tag resolution probe re-subscribing on the dead session). The 16:28 upgrade +restart cleared it incidentally. A graceful gateway shutdown takes the same path. + +**Fix:** `da0bfe2b` — a non-cancelled end of the event stream now signals `Disconnected` +(pinned by `MxGatewayDataConnectionReconnectTests.EventStreamGracefulCompletion_*`). +Deployed to all four QA nodes as a binary-only swap (no migrations, no config): backups +`app.bak.20260820T165357` / `cli.bak.20260820T165357` on both boxes; `/health/ready` 200 +Healthy, 2/2 members, site replication connected. **Not yet on prod vd03.** Live gate still to +run: close the site's session from the gateway dashboard and confirm the site reconnects +(`MxGateway event stream ended without a fault … signalling disconnect` → reconnect → values resume).