Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Deploy the x86 Worker time-conversion fixes #226 (
892bdc1a) and #228 (9ef6fc84) together: DEVwonder-app-vd03first, then QAvq03/vq04. DEV and QA deployed 2026-11-12 (76a71bc8). The exact procedure is indocs/deploy/2026-11-12-pending-worker-time-fixes-226-228.md(76a71bc8).mainSHA at or after9ef6fc84on windev (Worker-p:Platform=x86), and deploy withscripts/deploy/Upgrade-MxGatewayService.ps1perdocs/runbooks/DeployUpgradeNssmService.md.2b13564enow): the only extra change is #227, which is additive and opt-in. Post-deploy checks:LiveGatewayWrite2TestswithMXGATEWAY_LIVE_GATEWAY_ASSERT_WRITE2_TIMESTAMP=1onMESReceiver_025.MoveInBatchIDpasses. Stored timestamp = sent timestamp.QueryActiveAlarmsforGalaxy!CVDAisle_1.TableAlarms_008.RuntimeAlertreads2026-07-28T20:02:46.830Z.4f931e4now): amainbuild also carries the operator-authenticated alarm ack, so #170's prerequisites apply first: lmxopcuadc0a24f2+ plus anAlarmAckfallback account or per-key identities on every QA OtOpcUa Galaxy driver node. Then deploy rolling, vq03 then vq04, and run the same checks.Write2today.Close this issue once both tiers are deployed and checked, and the deploy records are written.
Checklist
mainSHA (76a71bc8); Worker.Tests x86 783 passed / 0 failed / 12 skipped2b13564e->76a71bc8(2026-11-12, about 9 s downtime, env block and pepper prefix unchanged)LiveGatewayWrite2TestswithASSERT_WRITE2_TIMESTAMP=1passed (sent and stored both02:20:34.000Z)RuntimeAlertreads2026-07-28T20:02:46.830Zdocs/deploy/2026-11-12-vd03-76a71bc8-worker-time-fixes.md,31f7f9fc)dc0a24f2+, plus anAlarmAckfallback account or per-key identities)dashboard.pfxover)DEV wonder-app-vd03 deployed, 2026-11-12:
2b13564e->76a71bc8. Both checks pass. Record:docs/deploy/2026-11-12-vd03-76a71bc8-worker-time-fixes.md(31f7f9fc).Build. Built from one
git archiveof76a71bc8.-c Release -p:Platform=x86. Worker.Tests x86: 783 passed, 0 failed, 12 skipped.0.1.2+76a71bc8. The zip was hash-checked at every hop.Deploy. Stop, back up, rename-swap Server and Worker, start once. About 9 s of downtime (stop at 21:19:50 EST, ready at 21:19:59).
MxGateway__ApiKeyPeppersha256 prefix is still0d01abcc1fe5.Server\appsettings.jsonwas restored hash-identical (464ACE1B..., the #171 version without the pepper).install-service.batand the install-root master were not used.Upgrade-MxGatewayService.ps1. Its process guards only matchZB.MOM.WW.MxGateway.*names, but vd03 runsMxGateway.Server.exeandMxGateway.Worker.exe. It also dropsdashboard.pfx. Filed as #230. I used adapted copies of the proven2b13564escripts instead.Verification
/healthz,/health/ready,/health/activeall 200 (2 of 2 sessions after the restart)MESReceiver_025.MoveInBatchID, and closed. A wrong key got Unauthenticated.LiveGatewayWrite2Testswith..._ASSERT_WRITE2_TIMESTAMP=1onMESReceiver_025.MoveInBatchID: 2/2 passed.KeepsSentTimestampsent02:20:34.000Zand stored02:20:34.000Z. The value was restored to0.QueryActiveAlarms:Galaxy!CVDAisle_1.TableAlarms_008.RuntimeAlertreads2026-07-28T20:02:46.830Z(before the deploy:2026-07-29T01:02:46.830Z).CommunicationAlarmmoved from16:44:38.420Zto11:44:38.420Zon 11-10, which is -5 h.Connected, 810/810 tags, good 398 / bad 412. Identical to the pre-deploy report; the Z28061/Z28062 snapshots are also identical./health/readyand/health/active200Test key.
issue229-write2-test, with scopes session open/close, invoke read/write and events:read, and globsMESReceiver_025.*. The secret only lived in a mode-0600 scratch file and was never printed. Afterwards I revoked and deleted it, and a probe with it then got Unauthenticated. The scratch file is deleted, and the key list is back to the original 6.Rollback.
Server.bak.20261112T211945+Worker.bak.20261112T211945.dashboard.pfx, service.reg):E:\psrem-stage\mxgw-229-76a71bc8\backup-20261112T211945\, readable by SYSTEM and Administrators only.rollback229.ps1 -Stamp 20261112T211945in the same stage folder.Keeping this issue open for QA (vq03/vq04 still on
4f931e4). The #170 prerequisites still apply.For QA (vq03/vq04): use the fixed
scripts/deploy/Upgrade-MxGatewayService.ps1frommain4890a3d1(#230, closed), not the pre-#230 copy.The old copy's process guards were blind to
MxGateway.Server.exe/MxGateway.Worker.exe. It also droppedServer\dashboard.pfx, and it would have refused a file-based master key.The fixed script fails closed if it can't see the running gateway, carries
dashboard.pfx, fingerprints the NSSM env block (pepper included) without ever writing it, refuses a runtime downgrade, and can roll back the.bak.<stamp>sets left on vd03.Run it with
-DryRunon each box first and read the plan. In particular, check that the gateway-process line showsMxGateway.Server#<pid>as the service host's child, that the carry plan includesdashboard.pfx, and that the env fingerprint names match what you expect. Then run it for real.It has had a real dry run on windev but no real upgrade yet. Keep runbook section 8 at hand.
QA wonder-app-vq03 + vq04 deployed, 2026-11-12:
4f931e4->76a71bc8, rolling (vq03 first, then vq04). Record:docs/deploy/2026-11-12-qa-vq03-vq04-76a71bc8.md(2cc0d575).Artifacts. I reused the DEV-deployed package, pulled from vd03's stage over SMB.
mxgw-76a71bc8.zipsha256B2771EEC2B6406A7AA84D7A125176F0C0F9A97138F82808AF519A693F211A302.MANIFEST.sha256on the Mac and again on each box.MxGateway.Server.exe=ZB.MOM.WW.MxGateway.Server.exesha2567d7974f0…6ebedc; WorkerMxGateway.Worker.exesha256844ab6ad…38f1871, PE32 x86.0.1.2+76a71bc8. The runtime goes from 10.0.5 to 10.0.14, an upgrade.Prerequisites (#170).
script-runner-canaryCalculation driver, so no OtOpcUa node acks through this gateway.b3021e7cnever callsAcknowledgeAlarm(NativeAlarmActoris read-only). It is on Client 0.2.1, the same pin as DEV ScadaBridge, which runs fine against76a71bc8.Procedure. I used the #230-fixed
Upgrade-MxGatewayService.ps1from4890a3d1. This was its first real run.ZB.MOM.WW.MxGateway.Server#<pid>as the service host's child and carriedappsettings*.json. QA has nodashboard.pfx.88188ab11b50on both.install-service.batwas not used, and no second gateway was started./healthz,/health/ready,/health/active200 (2/2 sessions)otopcua-galaxy-qagalaxy-test-connectionok; open/close session ok (Worker path)Missing or invalid API key.MESReceiver_025.MoveInBatchID(Z28062 Left, the same name as DEV)KeepsSentTimestampFAILED: sent04:26:07Z, stored09:26:07Z(+5 h; negative control). New build: 2/2 passed, sent = stored04:32:02.000Z04:38:27.000Z:5320Temporary keys
qa229-write2-test, scoped toMESReceiver_025.*, were minted before each box's deploy and deleted afterwards. A probe with each deleted key is refused, and each box is back to its two original keys. Only Z28062 was written; each test wrote212212and restored0.Rollback (on the box):
E:\psrem-stage\mxgw-229-qa-76a71bc8\rollback229qa.ps1 -Stamp 20261112-233019on vq03,-Stamp 20261112-233532on vq04. Backups are underE:\ApiInstall\MxAccessGateway\:app.bak-<stamp>-issue229,worker.bak-…, the protected_deploy-backup-…, anddata\*.bak-….Both tiers are deployed and checked, and the records are written, so I'm closing this issue. The empty QA alarm snapshot predates this deploy and was not investigated.