Claude-Session: https://claude.ai/code/session_01GT67AD99zQQqpLCVm3p7xk
10 KiB
Galaxy H6 (#614): OPC UA ack of a real Galaxy native alarm to AVEVA history, 2026-11-11
Gitea #614 item H6: "an OPC UA Acknowledge of a REAL Galaxy native alarm reaches AVEVA event history".
Result: FAIL — the item cannot be ticked. The OPC UA path worked end to end inside OtOpcUa: the
Part 9 Acknowledge returned Good, the client saw the condition go Active, Acknowledged, and the driver
sent the ack to mxaccessgw as the owner-provided Galaxy account with no error. The acknowledgement
never reached Galaxy. After the ack, Galaxy's own .Acked was still False, .AckMsg was still
empty, the gateway feed still showed the alarm as unacknowledged, and AVEVA event history has no ACK
row. So OtOpcUa now shows the condition acknowledged while Galaxy and the Historian do not (a
silent-divergence defect, below).
Exactly one acknowledge was sent (approved write, 2026-11-11T17:37:55Z). It was not retried. No remote command ran on any wonder box. Galaxy access was through the gateway's gRPC API and read-only SQL only.
Target
| Alarm | Galaxy!CVDAisle_1.TableAlarms_008.RuntimeAlert (object TableAlarms_008, contained as Right_004.TableAlarms), priority 400, "Run Time Exceeded" |
| Gateway | wonder-app-vd03.zmr.zimmer.com:5120 (h2c) |
| Galaxy security | OS/domain accounts (ZB.dbo.user_profile). Every 2026 ack in the Historian before this gate is a NAM\… operator from an InTouch node; none came through the gateway. |
| Historian | wonder-sql-vd03 → Runtime.dbo.v_AlarmEventHistory2. Queries need an EventStamp range. There is no A2ALMDB or WWALMDB. |
Setup (docker-dev, temporary, all removed afterwards)
| Item | Value |
|---|---|
| Node | SITE-B site-b-1 (OPC UA opc.tcp://localhost:4844). Its partner site-b-2 was stopped for the run so site-b-1 was the sole (Primary) node. |
| Driver | h6gal, DriverType GalaxyMxGateway, seeded straight into the ConfigDb. Gateway.Endpoint = the VD03 gateway, UseTls=false, ApiKeySecretRef=env:GALAXY_MXGW_API_KEY, MxAccess.ClientName=OtOpcUa-H6-gate, Repository.WatchDeployEvents=false. AlarmAck.FallbackUser/FallbackDomain = the owner-provided account (domain NAM); FallbackPasswordSecretRef=env:OTOPCUA_GALAXY_ACKPW |
| Device / tag | device vd03 ({}), tag RuntimeAlert, Boolean, Read, TagConfig = {"attributeRef":"Galaxy!CVDAisle_1.TableAlarms_008.RuntimeAlert","alarm":{"alarmType":"AlarmCondition","severity":400,"historizeToAveva":false}}. historizeToAveva:false keeps OtOpcUa's own alarm historian from writing anything to AVEVA. |
| Secrets | the gateway key and the ack password were injected into the site-b-1 container as runtime env only, through a scratch compose override that held ${VAR} references, never values. Nothing secret went into the repo, the compose files, the ConfigDb or this file. |
| OPC UA client | otopcua-cli (src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI), UserName apikey + a temporary data-plane key (h6-gate, principal h6gate, 4 h expiry) with a SITE-B Cluster grant: Browse, Read, Subscribe, HistoryRead, AlarmRead and AlarmAcknowledge (399). LDAP was not used. |
| Deploy | POST /api/deployments. The first attempt used DriverType Galaxy, which is wrong: the registered name is GalaxyMxGateway. It was corrected and deployed again. Deployments to SITE-B end TimedOut while site-b-2 is stopped (the same as the Heidenhain gate); site-b-1 applied each one. |
Driver start (site-b-1): GalaxyMxSession connected — clientName=OtOpcUa-H6-gate, DriverInstance h6gal: connected,
native-alarm subscription established for 1 alarm ref(s).
Step 1: OPC UA, condition and Acknowledge (PASS within OtOpcUa)
alarms --refresh before the ack:
[2026-07-29T01:02:46.8300000Z] ALARM h6gal/vd03/RuntimeAlert
Condition: RuntimeAlert
State: Active, Unacknowledged
Severity: 500
Retain: True
ConditionNodeId: ns=2;s=h6gal/vd03/RuntimeAlert
EventId: 8DB71E793FBD7040993404E30808FADC
The single ack, at 17:37:55Z:
otopcua-cli ack -u opc.tcp://localhost:4844 -n "ns=2;s=h6gal/vd03/RuntimeAlert" -e 8DB71E793FBD7040993404E30808FADC -c "OtOpcUa #614 H6 gate" -U apikey -P <temp key>
Acknowledge successful: ns=2;s=h6gal/vd03/RuntimeAlert
The live subscriber running during the ack saw:
[2026-11-11T17:37:55.8648690Z] ALARM h6gal/vd03/RuntimeAlert
State: Active, Acknowledged
Severity: 500
Retain: True
EventId: 0F15D34C9D0B6D43B7A4B08F3A72D047
Driver log (account name redacted):
[17:37:55 INF] Galaxy driver h6gal: acknowledging Galaxy!CVDAisle_1.TableAlarms_008.RuntimeAlert as Galaxy user NAM\<ack account> (Fallback) for OPC UA principal h6gate (key fa1d95d325ab9ab5)
[17:37:55 INF] DriverInstance h6gal: acknowledged native condition h6gal/vd03/RuntimeAlert by h6gate
No warning, refusal or revert followed. So the gateway answered PROTOCOL_STATUS_CODE_OK with
hresult 0 (or absent): a non-OK status or a non-zero hresult throws and is logged. The comment the
driver sent was OtOpcUa #614 H6 gate [OPC UA: h6gate, key fa1d95d325ab9ab5].
Step 2: Galaxy and the gateway alarm feed (FAIL)
QueryActiveAlarms (prefix = the alarm reference) about 30 s after the ack, unchanged from before:
"currentState": "ALARM_CONDITION_STATE_ACTIVE", "severity": 400,
"lastTransitionTimestamp": "2026-07-29T01:02:46.830Z", "operatorComment": "Run Time Exceeded"
Read-only gateway session (OpenSession → Register → ReadBulk → CloseSession), at 17:39:24Z and again at 17:40:18Z:
| Attribute | Value | Quality |
|---|---|---|
TableAlarms_008.RuntimeAlert.Acked |
False | 192 |
TableAlarms_008.RuntimeAlert.AckMsg |
"" (empty) | 192 |
TableAlarms_008.RuntimeAlert.InAlarm |
True (since 2026-07-28T20:02:46.830Z) | 192 |
On the windev rig, mxaccessgw's gateway-level verification (AlarmProbeFindings.md) showed a
successful ack turning .Acked True and leaving the comment in .AckMsg. Neither happened here.
Step 3: AVEVA event history (FAIL)
v_AlarmEventHistory2, EventStamp 2026-11-11 12:00 to 23:59 local, queried at 17:38, 17:39 and 17:46 UTC.
The Historian is live: it stored other events at 17:00 UTC. Nothing for this alarm:
2026-11-11 12:00:30.922 | 17:00:30.922 | UNACK_ALM | DevWinPlatform.CPULoadHiAlarm | NULL | NULL
2026-11-11 12:00:50.918 | 17:00:50.918 | UNACK_RTN | DevWinPlatform.CPULoadHiAlarm | NULL | NULL
There is no ACK_ALM or ACK_RTN row for TableAlarms_008.RuntimeAlert and no row with OperatorNode=MxGateway.
Defect: OtOpcUa reports Acknowledged on an ack Galaxy never applied (reported, not fixed)
mxaccessgw returns OK/hresult 0 both for "write applied" and for "write sent, no OnWriteComplete
within the wait (unconfirmed, with a diagnostic)" (mxaccessgw docs/Grpc.md, AcknowledgeAlarm
outcome table). Its own docs say that "a client that must be certain should watch for the resulting
transition on StreamAlarms".
GatewayGalaxyAlarmAcknowledger.EvaluateReply
(src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Galaxy/Runtime/GatewayGalaxyAlarmAcknowledger.cs) treats OK +
hresult 0 as success. It never reads or logs AcknowledgeAlarmReply.diagnostic_message, and nothing
in the driver or host waits for the upstream Acknowledge transition. So the #760 revert never fires,
and the OPC UA condition stays Acknowledged while Galaxy holds .Acked=False. A ScadaBridge or
operator client would believe the alarm was handled.
Why the gateway-side write did not take effect cannot be determined from here: the gateway's logs are on wonder-app-vd03, and remote commands are barred. Candidates:
- an unconfirmed
AckMsgwrite that was later dropped or rejected; - the account lacking acknowledge permission on the object's security group;
- a VD03 gateway build that predates the
AckMsgpath.
Not attempted again (one-ack rule). Suggested follow-ups (not filed, no Gitea per brief):
- OtOpcUa: log
diagnostic_messageon every ack reply. Treat an unconfirmed ack as pending until the feed shows the Acknowledge transition, and revert on timeout. - mxaccessgw: expose applied vs unconfirmed as a field rather than only in a diagnostic.
- Owner: check the VD03 gateway log around 2026-11-11T17:37:55Z for the ack's diagnostic.
Other observations (not filed)
- The gateway snapshot timestamp is 5 h off. It reports
lastTransitionTimestamp 2026-07-29T01:02:46.830Z. MXAccess's ownInAlarmtimestamp and the Historian'sEventStampUTCboth say 2026-07-28T20:02:46.830Z. Cross-repo, mxaccessgw. - The gateway snapshot and the Historian disagree on state. The gateway says ACTIVE since July,
and
.InAlarmagrees. The Historian logged five later "set off scan"UNACK_RTNrows (latest 2026-10-22) with no newUNACK_ALM. - Severity differs between the three views. The OPC UA condition shows Severity 500. The authored
alarm.severitywas 400, and so is the Galaxy priority. - site-b-2 never acknowledged the cleanup deploys. After cleanup, the standby
site-b-2(restarted and restored from its LocalDb at the same revisione221b179…) did not ack the re-dispatched deployment, so the reseal deploy also endedTimedOut(site-b-1 Applied). - The rig needed a nudge after the OrbStack restart.
otopcua-dev-sql-1andotopcua-dev-traefik-1stayed Exited. I started them and restarted central-1 and central-2 (#868 ordering) before seeding.
Cleanup
- Deleted from the ConfigDb:
Tag h6gal-rtalert,Device h6gal-vd03andDriverInstance h6gal;NodeAcl h6gate-aclandApiKey h6-gate(fa1d95d325ab9ab5);- the two gate
Deploymentrows that carried the driver config, and theirNodeDeploymentStaterows, so the account name is not left in an artifact blob.
- Cleanup deployed (
h6-gate-cleanup,h6-gate-cleanup-reseal). site-b-1 loggedGalaxyDriver h6gal shutting down … disposed. - site-b-1 recreated from the plain compose file: the ack password is absent and the gateway key is back to the committed default. site-b-2 restarted.
- The scratch key file was deleted. The shell env was unset.
- The site-b-1 LocalDb cache may still hold a gate-era artifact until its next replacement.