# Galaxy G6 (#615): end to end against a live Galaxy, 2026-11-12 Gitea #615 gate G6 (`docs/plans/v2-ga-lab-gates-plan.md` §G6): "at least one end-to-end integration run against live Galaxy succeeds". **Result: PASS.** Every G6 pass criterion was met, plus write and native-alarm checks that the plan does not require: | G6 criterion (plan §G6) | Result | |---|---| | Browse returns a non-empty tree that mirrors the Galaxy hierarchy | PASS. Both subtrees: raw `galaxy-windev/dev/TestArea/TestMachine_015/…`, `…/TestArea3/TestMachine_036/…`; UNS `testarea/line-1/testmachine-015/…`, `testarea3/line-1/testmachine-036/…` | | Read returns `Good` with a non-null value | PASS. Every authored value tag, through both NodeIds | | Subscribe receives at least one data change within 5 s | PASS. First notification inside the first second, then one per second (1 s publishing) through both NodeIds | | No `BadNoCommunication` / `BadTimeout` in the server log | PASS, with one exception. The only `BadNoCommunication` line is the planned driver respawn during a redeploy (`driver g6gal disconnected — 10 value node(s) marked BadNoCommunication (value retained)`), and the driver reconnected 10 s later. There are no `BadTimeout` lines, and no `ERR`/`FTL` lines in either run | The extra checks: | Extra step | Result | |---|---| | Write a TestMachine attribute, then read it back | PASS through raw and UNS, cross-checked through an independent gateway session. The anonymous negative control was refused `BadUserAccessDenied` | | Native alarm becomes a Part 9 condition, then Ack, then Clear | PASS, with the ack confirmed in Galaxy (`.Acked=True`, `.AckMsg` = the comment) | | Native alarm authored the way the AdminUI picker authors it | **FAIL on master `00a5be26`** (the condition never went active). Fixed in `e113faf5` (#895) and **PASS live** after the fix | | DeployWatcher on a Galaxy redeploy | **NOT RUN.** See below | Two defects were filed: **#895** (fixed in this branch) and **#896** (open). ## Target and versions | | | |---|---| | Galaxy | windev `DESKTOP-6JL3KKO` test Galaxy `DEV` (GR DB `ZB`). AVEVA System Platform 2020 / Application Server 2020 (20.0.000). **129 deployed instances**, including all 40 `$TestMachine` objects (`TestMachine_001..040`) under areas `TestArea`, `TestArea2` and `TestArea3`. Last Galaxy deploy 2026-10-19T15:58:30Z (`GetLastDeployTime`) | | Gateway | mxaccessgw main `447070a5`, NSSM `MxAccessGw`, gRPC h2c `http://10.100.0.48:5120`. Not restarted or reconfigured. `http://10.100.0.48:5130/health/ready` returned 200 before and after the run | | Gateway key | temporary `g6-otopcua-e2e` (`session:open,session:close,invoke:read,invoke:write,events:read,metadata:read`, 1-day expiry), minted with the gateway admin CLI and revoked afterwards | | OtOpcUa | `origin/master` `00a5be26` for the main run. The #895 re-check ran the same tree plus the fix (committed as `e113faf5`) | | OPC UA client | `otopcua-cli` (`src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI`), security None | **Clock note.** windev's clock is about **9 min 12 s behind** the Mac (04:07:57 against 04:17:09 UTC, checked during the run). Galaxy `SourceTimestamp`s therefore read about 9 min behind `ServerTimestamp` throughout. This is box skew and not a defect: the gap stayed constant while values changed every second. ## Setup (local, temporary, removed afterwards) The docker-dev rig was in use by other agents and its image commit could not be identified, so a single OtOpcUa Host ran on the Mac from this worktree: - Fused `admin,driver` single node, Akka self-seeded on `localhost:4063`. - OPC UA on `opc.tcp://localhost:4850`, AdminUI and REST on `http://127.0.0.1:9310`. - Its own ConfigDb `OtOpcUaG6` on the rig's SQL container (`localhost:14330`), migrated with `dotnet ef database update`. - Cluster `G6`: `NodeCount=1`, `RedundancyMode=None`, one `ClusterNode localhost:4063`. - Committed docker-dev dev secrets (KEK, pepper, JWT, LDAP); `DisableLogin=true`; `Security:Acl:Enabled=false`. Authored config, seeded into the ConfigDb and deployed with `POST /api/deployments` (`X-Api-Key`): | Row | Value | |---|---| | DriverInstance | `g6gal` "galaxy-windev", `GalaxyMxGateway`, `Gateway.Endpoint=http://10.100.0.48:5120`, `UseTls=false`, `ApiKeySecretRef=env:OTOPCUA_GALAXY_APIKEY`, `MxAccess.ClientName=OtOpcUa-G6-gate`, `Repository.WatchDeployEvents=true`, `AlarmAck.FallbackUser=Administrator` with `FallbackPasswordSecretRef=dev:` (empty password, the windev rig's documented identity, mxaccessgw `docs/AlarmProbeFindings.md`) | | Device / groups | device `dev`; TagGroups `TestArea/TestMachine_015` and `TestArea3/TestMachine_036` (mirroring Galaxy area → object) | | Tags | `TestMachine_015`: `TestChangingInt` (Int32 R), `TestDouble` (Double RW), `TestAlarm002` (Boolean RW, value), `TestAlarm002Condition` (`attributeRef "Galaxy!TestArea.TestMachine_015.TestAlarm002"` + `alarm` AlarmCondition 500); `TestMachine_036`: `TestChangingInt`, `TestHistoryValue` (Int32 R). Later added: `TestAlarm003` (`attributeRef "TestMachine_015.TestAlarm003"` + `alarm` OffNormalAlarm 700, the picker's form) | | UNS | areas `testarea` / `testarea3` → `line-1` → equipment `testmachine-015` (`EQ-a6f600000000`) / `testmachine-036` (`EQ-b6f600000000`), with all six tags referenced | | Data-plane key | `g6-gate` (principal `g6gate`, 8 h), `NodeAcl` Cluster grant 927 (Browse/Read/Subscribe/HistoryRead/WriteOperate/AlarmRead/AlarmAck/AlarmConfirm) | Deploys `8ee64aa9…` (driver), `0646dee1…` (key and ACL), `20affbc8…` (TestAlarm003). Each one was `Sealed`, with node `localhost:4063` `Applied`. A first key deploy was correctly refused `422 NodeAclScopeIdUnexpected`: a Cluster grant must carry no ScopeId. Driver start: ``` DriverHost localhost:4063: spawned GalaxyMxGateway driver g6gal (stub=False) GalaxyMxSession connected — clientName=OtOpcUa-G6-gate serverHandle=1 GalaxyDriver g6gal initialized — endpoint=http://10.100.0.48:5120 clientName=OtOpcUa-G6-gate DriverInstance g6gal: subscribed to 5 refs (galaxy-sub-1) DriverInstance g6gal: native-alarm subscription established for 1 alarm ref(s) ``` ## Step 1: browse both subtrees (PASS) `browse -r -d 8` (excerpt; condition children elided): ``` [Object] OtOpcUa (NodeId: ns=2;s=OtOpcUa) [Object] testarea (NodeId: ns=3;s=g6-area-test) [Object] line-1 (NodeId: ns=3;s=g6-line-a) [Object] testmachine-015 (NodeId: ns=3;s=EQ-a6f600000000) [Variable] TestChangingInt (NodeId: ns=3;s=testarea/line-1/testmachine-015/TestChangingInt) [Variable] TestDouble (NodeId: ns=3;s=testarea/line-1/testmachine-015/TestDouble) [Variable] TestAlarm002 (NodeId: ns=3;s=testarea/line-1/testmachine-015/TestAlarm002) [Variable] TestAlarm002Condition (NodeId: ns=3;s=testarea/line-1/testmachine-015/TestAlarm002Condition) <- #896 [Object] TestAlarm002Condition (NodeId: ns=2;s=galaxy-windev/dev/TestArea/TestMachine_015/TestAlarm002Condition) [Object] testarea3 (NodeId: ns=3;s=g6-area-test3) [Object] line-1 (NodeId: ns=3;s=g6-line-b) [Object] testmachine-036 (NodeId: ns=3;s=EQ-b6f600000000) [Variable] TestChangingInt (NodeId: ns=3;s=testarea3/line-1/testmachine-036/TestChangingInt) [Variable] TestHistoryValue (NodeId: ns=3;s=testarea3/line-1/testmachine-036/TestHistoryValue) [Object] galaxy-windev (NodeId: ns=2;s=galaxy-windev) [Object] dev (NodeId: ns=2;s=galaxy-windev/dev) [Object] TestArea (NodeId: ns=2;s=galaxy-windev/dev/TestArea) [Object] TestMachine_015 (NodeId: ns=2;s=galaxy-windev/dev/TestArea/TestMachine_015) [Variable] TestAlarm002 (…/TestMachine_015/TestAlarm002) [Object] TestAlarm002Condition (…/TestMachine_015/TestAlarm002Condition) (+ the full Part 9 child set: ActiveState, AckedState, Acknowledge, Confirm, ShelvingState, …) [Variable] TestChangingInt (…/TestMachine_015/TestChangingInt) [Variable] TestDouble (…/TestMachine_015/TestDouble) [Object] TestArea3 (NodeId: ns=2;s=galaxy-windev/dev/TestArea3) [Object] TestMachine_036 (NodeId: ns=2;s=galaxy-windev/dev/TestArea3/TestMachine_036) [Variable] TestChangingInt (…/TestMachine_036/TestChangingInt) [Variable] TestHistoryValue (…/TestMachine_036/TestHistoryValue) ``` The raw subtree uses `ns=2;s=` and the UNS subtree uses `ns=3;s=///`. The Part 9 condition appears under both the raw TagGroup and the referencing equipment folder (notifier fan-out). ## Step 2: read through both NodeIds (PASS) | NodeId | Value | Status | |---|---|---| | raw `…/TestMachine_015/TestChangingInt` | 841 | Good | | raw `…/TestMachine_015/TestDouble` | 0 | Good | | raw `…/TestMachine_015/TestAlarm002` | False | Good | | raw `…/TestMachine_036/TestChangingInt` | 479 | Good | | raw `…/TestMachine_036/TestHistoryValue` | 0 | Good | | uns `testarea/line-1/testmachine-015/TestChangingInt` | 904 | Good | | uns `testarea/line-1/testmachine-015/TestDouble` | 0 | Good | | uns `testarea3/line-1/testmachine-036/TestChangingInt` | 330 | Good | | `Server.ServiceLevel` (i=2267) | 250 | Good | | uns `…/testmachine-015/TestAlarm002Condition` | (none) | **BadWaitingForInitialData**, permanently. #896 | ## Step 3: subscribe (PASS) Raw `…/TestMachine_015/TestChangingInt`, 1000 ms, 10 s: 9 Good notifications, the first inside the first second (`49, 474, 899, 325, 750, 176, 601, 27, 452`). UNS equipment `EQ-b6f600000000` recursive: 2 variables, 8 notifications in 8 s (`TestChangingInt` every second; `TestHistoryValue` initial value only, since the attribute is static). Summaries: `Ever went BAD during window: 0`, `No update received at all: 0`. ## Step 4: write and read back (PASS) ``` anonymous write raw TestDouble = 42.125 -> Write failed: BadUserAccessDenied (negative control; anonymous has no WriteOperate) apikey g6gate write raw TestDouble = 42.125 -> Write successful read raw …/TestMachine_015/TestDouble -> 42.125 Good read uns testarea/line-1/testmachine-015/TestDouble -> 42.125 Good apikey g6gate write uns TestDouble = 17.5 -> Write successful read raw …/TestMachine_015/TestDouble -> 17.5 Good ``` Independent readback through a separate gateway session (mxgw-dotnet `smoke --item TestMachine_015.TestDouble`): `OnDataChange doubleValue: 17.5, quality 192`. TestDouble was then written back to its original 0. Before the key deploy, a write with the new key was refused (`AclGate: … key … is no longer enabled or has expired in generation 1`). That is expected: data-plane key usability follows the deployed generation, so a key inserted after a deploy needs a redeploy. ## Step 5: native alarm to Part 9, ack and clear (PASS) The alarm was raised through OPC UA: `write raw …/TestMachine_015/TestAlarm002 = True` (key `g6gate`). The live `alarms --refresh` subscriber (key `g6gate`) saw: ``` [2026-11-13T04:11:05.2390000Z] ALARM galaxy-windev/dev/TestArea/TestMachine_015/TestAlarm002Condition State: Active, Unacknowledged Severity: 750 Retain: True EventId: 9503F17B7A8F82498FBCEDD868C4AD98 ``` ``` otopcua-cli ack -n "ns=2;s=…/TestAlarm002Condition" -e 9503F17B… -c "OtOpcUa #615 G6 gate" -U apikey -P Acknowledge successful [2026-11-13T04:20:34.4620500Z] … State: Active, Acknowledged Retain: True ``` Driver log: ``` Galaxy driver g6gal: acknowledging Galaxy!TestArea.TestMachine_015.TestAlarm002 as Galaxy user Administrator (Fallback) for OPC UA principal g6gate (key 8e142bf2a2b70df8) DriverInstance g6gal: acknowledged native condition galaxy-windev/dev/TestArea/TestMachine_015/TestAlarm002Condition by g6gate ``` Galaxy side, read through a separate gateway session after the ack: | Attribute | Value | |---|---| | `TestMachine_015.TestAlarm002.Acked` | **True** (q 192) | | `TestMachine_015.TestAlarm002.AckMsg` | **`OtOpcUa #615 G6 gate [OPC UA: g6gate, key 8e142bf2a2b70df8]`** | | `TestMachine_015.TestAlarm002.InAlarm` | True | | gateway `QueryActiveAlarms` | `ALARM_CONDITION_STATE_ACTIVE_ACKED`, `operatorUser: Administrator`, the same comment | This is the leg H6 could not get on vd03 (`galaxy-h6-ack-2026-11-11.md`): the ack reached Galaxy. The alarm was cleared through the UNS NodeId: `write uns testarea/line-1/testmachine-015/TestAlarm002 = False`. Galaxy `TestAlarm002`/`.InAlarm` went False at once. The gateway feed's return-to-normal arrived between 40 s and 100 s later; the gateway `ReconcileIntervalSeconds` is 30. The condition then went `Inactive, Acknowledged`, `Retain: False`. The clear event's `Time` carried the earlier transition timestamp, not the return-to-normal time. That latency and timestamp are gateway-side behaviour, so they are recorded here but not filed against OtOpcUa. **Severity (#894).** The gateway reported `severity 500` (AVEVA priority 500, High band 251–500), and the condition read **750**. The authored `alarm.severity` was 500 for TestAlarm002 and 700 for TestAlarm003; the live feed value overrode both. This is the #887 band mapping working live (750, as #894 expects), on another alarm and another gateway than #894's vd03 target. ## Step 6: picker-form native alarm (FAIL on master, PASS with #895) `TestAlarm003` was authored as the AdminUI Galaxy picker would author it: `attributeRef "TestMachine_015.TestAlarm003"` plus the seeded `{"alarmType":"OffNormalAlarm","severity":700}`. It was raised by a supervisory write through a separate gateway session. A plain `Write` is refused `SecurityError 1008`, because `TestAlarm003` is `MxSecurityOperate`. - **Master `00a5be26`:** the gateway feed reported `Galaxy!TestArea.TestMachine_015.TestAlarm003` as `ALARM_CONDITION_STATE_ACTIVE`. The OPC UA condition stayed `Inactive, Acknowledged`. Cause: `OnAlarmFeedTransition` mapped only the feed's full reference to the authored attributeRef. The `sourceObjectReference` the feed also sends (`TestMachine_015.TestAlarm003`) was ignored, so the transition got a ConditionId matching no condition. → **#895**. - **With the fix (`e113faf5`):** after a host restart on the same deployment, which the host restored from LocalDb on bootstrap, ConditionRefresh returned `TestAlarm003 State: Active, Unacknowledged Severity: 750 Retain: True`. The OPC UA `Acknowledge` dialled `Galaxy!TestArea.TestMachine_015.TestAlarm003` (log line as above), and the condition went `Active, Acknowledged`. Galaxy then read `.Acked = True` and `.AckMsg = "OtOpcUa #615 G6 gate picker-form (#895) [OPC UA: g6gate, key 8e142bf2a2b70df8]"`. After the supervisory clear the condition went `Inactive, Acknowledged`, `Retain: False`, and the gateway showed no active alarms. ## Not run: DeployWatcher on a Galaxy redeploy This step was not run for two reasons: - **No way to trigger a redeploy.** A Galaxy redeploy needs the ArchestrA IDE or a GRAccess script on windev. No GRAccess assembly was found in the ArchestrA Framework bin directory. - **Risk to another agent.** The test Galaxy was serving another agent's MXAccess captures, and redeploying its objects would have disturbed them. Expected behaviour if it is run later: `docs/drivers/Galaxy.md` says the `IRediscoverable` / `DeployWatcher` path raises a "re-browse" prompt on `/hosts` and does **not** rebuild the address space (#518). The live check would be a `/hosts` chip, not a tree rebuild. `Repository.WatchDeployEvents=true` was set and the driver started cleanly with it. ## Defects - **#895 (fixed here, `e113faf5`).** A native alarm authored by the AdminUI picker (`attributeRef tag.Attr`) never went active, and its ack would dial a reference the gateway refuses. Three unit tests were added to `GalaxyDriverRawPathKeyingTests`. The main one fails with the fix reverted, which was checked by reverting the fix and re-running it. - **#896 (fixed after this run).** The UNS mirror of an alarm-bearing raw tag gets a value Variable that stays `BadWaitingForInitialData` forever, against the documented "condition instead of value variable" contract. It was not fixed here because the composer test fixtures assert that mirror. Fixed afterwards: the composer no longer emits a UNS Variable for a reference to an alarm-bearing tag, so the equipment shows only the condition (the `<- #896` Variable in the Step 1 browse is gone). ## Cleanup - **windev:** gateway key `g6-otopcua-e2e` revoked. No other key was touched. `MxAccessGw` was never stopped, restarted or reconfigured, and `/health/ready` returned 200 throughout. Temporary `.ps1` files in `C:\Users\Public` were deleted. - **Galaxy:** TestDouble restored to 0. TestAlarm002 and TestAlarm003 are both back to False, and the gateway shows no active alarms. Their raise/ack/clear rows stay in the Galaxy's own alarm history. - **Mac:** local Host stopped. Database `OtOpcUaG6` dropped from the rig SQL container. The docker-dev rig containers and their `OtOpcUa` DB were untouched. Scratch key files deleted.