09b373e4db
v2-ci / unit-tests (push) Has been cancelled
A UnsTagReference to a raw tag that carries an "alarm" object produced a UNS
value Variable that read BadWaitingForInitialData forever (G6 live gate,
TestAlarm002Condition on testmachine-015). BuildUnsReferenceVariables now skips
those references; the equipment gets the alarm only as the raw condition wired
as an event notifier of its folder, which is unchanged (the reference still
feeds RawTagPlan.ReferencingEquipmentPaths).
Why condition-only and not "feed the mirror":
- The raw side has no value node for such a tag: AddressSpaceApplier
materialises an alarm tag as a condition OR a value variable, never both.
There is no raw value for the UNS mirror to mirror.
- DriverHostActor skips alarm tags when it builds the value fan-out and write
routing maps (`continue` before the UNS registration), and the driver leaves
them out of its value subscription ("subscribed to 5 refs" with 6 tags in
G6). Feeding the mirror would mean subscribing a value the design says the
tag does not have.
- Docs already state condition-only: ScriptedAlarms.md ("instead of a value
variable"), AlarmTracking.md ("only the condition, with no sibling value
variable"), the v3 design (native alarm = one condition at the raw tag,
surfaced at equipment via notifiers).
- The composer tests that asserted the mirror were incidental: f4f3e17e
(v3 B4-WP1) put the alarm on the fixture's Speed tag to test raw alarm
attachment, and later tests reused Speed for writable / historian / array
checks of the mirror. No commit or doc chose a value mirror for alarm tags.
- ScadaBridge does not read it: its OtOpcUa alarm path is event subscriptions
on the equipment/raw folders (v3 alarm B/C live gate), and a node that never
had a value cannot be a working binding. G6 itself authored the alarm's
Boolean as a separate value tag (TestAlarm002), which is the supported way to
read it.
Tests: the composer fixture moves the alarm onto its own HighTemp tag
(Speed is now a plain historized value tag), with a new test asserting no UNS
Variable for the alarm reference, with positive controls (the raw tag still
carries the equipment path; the value references still emit). New real-SDK
test Alarm_bearing_reference_surfaces_as_the_condition_only_against_real_SDK:
no Variable at the UNS path, the Speed mirror present, one condition, the
equipment folder wired as its notifier. Both new tests fail with the skip
removed. Docs: Uns.md, ScriptedAlarms.md, G6 evidence note.
Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
255 lines
17 KiB
Markdown
255 lines
17 KiB
Markdown
# Galaxy G6 (#615): end to end against a live Galaxy, 2026-11-12
|
||
|
||
Gitea #615 gate G6 (`docs/plans/v2-ga-lab-gates-plan.md` §G6): "at least one end-to-end
|
||
integration run against live Galaxy succeeds".
|
||
|
||
**Result: PASS.** Every G6 pass criterion was met, plus write and native-alarm checks that the plan
|
||
does not require:
|
||
|
||
| G6 criterion (plan §G6) | Result |
|
||
|---|---|
|
||
| Browse returns a non-empty tree that mirrors the Galaxy hierarchy | PASS. Both subtrees: raw `galaxy-windev/dev/TestArea/TestMachine_015/…`, `…/TestArea3/TestMachine_036/…`; UNS `testarea/line-1/testmachine-015/…`, `testarea3/line-1/testmachine-036/…` |
|
||
| Read returns `Good` with a non-null value | PASS. Every authored value tag, through both NodeIds |
|
||
| Subscribe receives at least one data change within 5 s | PASS. First notification inside the first second, then one per second (1 s publishing) through both NodeIds |
|
||
| No `BadNoCommunication` / `BadTimeout` in the server log | PASS, with one exception. The only `BadNoCommunication` line is the planned driver respawn during a redeploy (`driver g6gal disconnected — 10 value node(s) marked BadNoCommunication (value retained)`), and the driver reconnected 10 s later. There are no `BadTimeout` lines, and no `ERR`/`FTL` lines in either run |
|
||
|
||
The extra checks:
|
||
|
||
| Extra step | Result |
|
||
|---|---|
|
||
| Write a TestMachine attribute, then read it back | PASS through raw and UNS, cross-checked through an independent gateway session. The anonymous negative control was refused `BadUserAccessDenied` |
|
||
| Native alarm becomes a Part 9 condition, then Ack, then Clear | PASS, with the ack confirmed in Galaxy (`.Acked=True`, `.AckMsg` = the comment) |
|
||
| Native alarm authored the way the AdminUI picker authors it | **FAIL on master `00a5be26`** (the condition never went active). Fixed in `e113faf5` (#895) and **PASS live** after the fix |
|
||
| DeployWatcher on a Galaxy redeploy | **NOT RUN.** See below |
|
||
|
||
Two defects were filed: **#895** (fixed in this branch) and **#896** (open).
|
||
|
||
## Target and versions
|
||
|
||
| | |
|
||
|---|---|
|
||
| Galaxy | windev `DESKTOP-6JL3KKO` test Galaxy `DEV` (GR DB `ZB`). AVEVA System Platform 2020 / Application Server 2020 (20.0.000). **129 deployed instances**, including all 40 `$TestMachine` objects (`TestMachine_001..040`) under areas `TestArea`, `TestArea2` and `TestArea3`. Last Galaxy deploy 2026-10-19T15:58:30Z (`GetLastDeployTime`) |
|
||
| Gateway | mxaccessgw main `447070a5`, NSSM `MxAccessGw`, gRPC h2c `http://10.100.0.48:5120`. Not restarted or reconfigured. `http://10.100.0.48:5130/health/ready` returned 200 before and after the run |
|
||
| Gateway key | temporary `g6-otopcua-e2e` (`session:open,session:close,invoke:read,invoke:write,events:read,metadata:read`, 1-day expiry), minted with the gateway admin CLI and revoked afterwards |
|
||
| OtOpcUa | `origin/master` `00a5be26` for the main run. The #895 re-check ran the same tree plus the fix (committed as `e113faf5`) |
|
||
| OPC UA client | `otopcua-cli` (`src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI`), security None |
|
||
|
||
**Clock note.** windev's clock is about **9 min 12 s behind** the Mac (04:07:57 against 04:17:09 UTC,
|
||
checked during the run). Galaxy `SourceTimestamp`s therefore read about 9 min behind `ServerTimestamp`
|
||
throughout. This is box skew and not a defect: the gap stayed constant while values changed every second.
|
||
|
||
## Setup (local, temporary, removed afterwards)
|
||
|
||
The docker-dev rig was in use by other agents and its image commit could not be identified, so a single
|
||
OtOpcUa Host ran on the Mac from this worktree:
|
||
|
||
- Fused `admin,driver` single node, Akka self-seeded on `localhost:4063`.
|
||
- OPC UA on `opc.tcp://localhost:4850`, AdminUI and REST on `http://127.0.0.1:9310`.
|
||
- Its own ConfigDb `OtOpcUaG6` on the rig's SQL container (`localhost:14330`), migrated with `dotnet ef database update`.
|
||
- Cluster `G6`: `NodeCount=1`, `RedundancyMode=None`, one `ClusterNode localhost:4063`.
|
||
- Committed docker-dev dev secrets (KEK, pepper, JWT, LDAP); `DisableLogin=true`; `Security:Acl:Enabled=false`.
|
||
|
||
Authored config, seeded into the ConfigDb and deployed with `POST /api/deployments` (`X-Api-Key`):
|
||
|
||
| Row | Value |
|
||
|---|---|
|
||
| DriverInstance | `g6gal` "galaxy-windev", `GalaxyMxGateway`, `Gateway.Endpoint=http://10.100.0.48:5120`, `UseTls=false`, `ApiKeySecretRef=env:OTOPCUA_GALAXY_APIKEY`, `MxAccess.ClientName=OtOpcUa-G6-gate`, `Repository.WatchDeployEvents=true`, `AlarmAck.FallbackUser=Administrator` with `FallbackPasswordSecretRef=dev:` (empty password, the windev rig's documented identity, mxaccessgw `docs/AlarmProbeFindings.md`) |
|
||
| Device / groups | device `dev`; TagGroups `TestArea/TestMachine_015` and `TestArea3/TestMachine_036` (mirroring Galaxy area → object) |
|
||
| Tags | `TestMachine_015`: `TestChangingInt` (Int32 R), `TestDouble` (Double RW), `TestAlarm002` (Boolean RW, value), `TestAlarm002Condition` (`attributeRef "Galaxy!TestArea.TestMachine_015.TestAlarm002"` + `alarm` AlarmCondition 500); `TestMachine_036`: `TestChangingInt`, `TestHistoryValue` (Int32 R). Later added: `TestAlarm003` (`attributeRef "TestMachine_015.TestAlarm003"` + `alarm` OffNormalAlarm 700, the picker's form) |
|
||
| UNS | areas `testarea` / `testarea3` → `line-1` → equipment `testmachine-015` (`EQ-a6f600000000`) / `testmachine-036` (`EQ-b6f600000000`), with all six tags referenced |
|
||
| Data-plane key | `g6-gate` (principal `g6gate`, 8 h), `NodeAcl` Cluster grant 927 (Browse/Read/Subscribe/HistoryRead/WriteOperate/AlarmRead/AlarmAck/AlarmConfirm) |
|
||
|
||
Deploys `8ee64aa9…` (driver), `0646dee1…` (key and ACL), `20affbc8…` (TestAlarm003). Each one was
|
||
`Sealed`, with node `localhost:4063` `Applied`. A first key deploy was correctly refused
|
||
`422 NodeAclScopeIdUnexpected`: a Cluster grant must carry no ScopeId.
|
||
|
||
Driver start:
|
||
|
||
```
|
||
DriverHost localhost:4063: spawned GalaxyMxGateway driver g6gal (stub=False)
|
||
GalaxyMxSession connected — clientName=OtOpcUa-G6-gate serverHandle=1
|
||
GalaxyDriver g6gal initialized — endpoint=http://10.100.0.48:5120 clientName=OtOpcUa-G6-gate
|
||
DriverInstance g6gal: subscribed to 5 refs (galaxy-sub-1)
|
||
DriverInstance g6gal: native-alarm subscription established for 1 alarm ref(s)
|
||
```
|
||
|
||
## Step 1: browse both subtrees (PASS)
|
||
|
||
`browse -r -d 8` (excerpt; condition children elided):
|
||
|
||
```
|
||
[Object] OtOpcUa (NodeId: ns=2;s=OtOpcUa)
|
||
[Object] testarea (NodeId: ns=3;s=g6-area-test)
|
||
[Object] line-1 (NodeId: ns=3;s=g6-line-a)
|
||
[Object] testmachine-015 (NodeId: ns=3;s=EQ-a6f600000000)
|
||
[Variable] TestChangingInt (NodeId: ns=3;s=testarea/line-1/testmachine-015/TestChangingInt)
|
||
[Variable] TestDouble (NodeId: ns=3;s=testarea/line-1/testmachine-015/TestDouble)
|
||
[Variable] TestAlarm002 (NodeId: ns=3;s=testarea/line-1/testmachine-015/TestAlarm002)
|
||
[Variable] TestAlarm002Condition (NodeId: ns=3;s=testarea/line-1/testmachine-015/TestAlarm002Condition) <- #896
|
||
[Object] TestAlarm002Condition (NodeId: ns=2;s=galaxy-windev/dev/TestArea/TestMachine_015/TestAlarm002Condition)
|
||
[Object] testarea3 (NodeId: ns=3;s=g6-area-test3)
|
||
[Object] line-1 (NodeId: ns=3;s=g6-line-b)
|
||
[Object] testmachine-036 (NodeId: ns=3;s=EQ-b6f600000000)
|
||
[Variable] TestChangingInt (NodeId: ns=3;s=testarea3/line-1/testmachine-036/TestChangingInt)
|
||
[Variable] TestHistoryValue (NodeId: ns=3;s=testarea3/line-1/testmachine-036/TestHistoryValue)
|
||
[Object] galaxy-windev (NodeId: ns=2;s=galaxy-windev)
|
||
[Object] dev (NodeId: ns=2;s=galaxy-windev/dev)
|
||
[Object] TestArea (NodeId: ns=2;s=galaxy-windev/dev/TestArea)
|
||
[Object] TestMachine_015 (NodeId: ns=2;s=galaxy-windev/dev/TestArea/TestMachine_015)
|
||
[Variable] TestAlarm002 (…/TestMachine_015/TestAlarm002)
|
||
[Object] TestAlarm002Condition (…/TestMachine_015/TestAlarm002Condition) (+ the full Part 9 child set: ActiveState, AckedState, Acknowledge, Confirm, ShelvingState, …)
|
||
[Variable] TestChangingInt (…/TestMachine_015/TestChangingInt)
|
||
[Variable] TestDouble (…/TestMachine_015/TestDouble)
|
||
[Object] TestArea3 (NodeId: ns=2;s=galaxy-windev/dev/TestArea3)
|
||
[Object] TestMachine_036 (NodeId: ns=2;s=galaxy-windev/dev/TestArea3/TestMachine_036)
|
||
[Variable] TestChangingInt (…/TestMachine_036/TestChangingInt)
|
||
[Variable] TestHistoryValue (…/TestMachine_036/TestHistoryValue)
|
||
```
|
||
|
||
The raw subtree uses `ns=2;s=<RawPath>` and the UNS subtree uses `ns=3;s=<Area>/<Line>/<Equipment>/<Signal>`.
|
||
The Part 9 condition appears under both the raw TagGroup and the referencing equipment folder (notifier fan-out).
|
||
|
||
## Step 2: read through both NodeIds (PASS)
|
||
|
||
| NodeId | Value | Status |
|
||
|---|---|---|
|
||
| raw `…/TestMachine_015/TestChangingInt` | 841 | Good |
|
||
| raw `…/TestMachine_015/TestDouble` | 0 | Good |
|
||
| raw `…/TestMachine_015/TestAlarm002` | False | Good |
|
||
| raw `…/TestMachine_036/TestChangingInt` | 479 | Good |
|
||
| raw `…/TestMachine_036/TestHistoryValue` | 0 | Good |
|
||
| uns `testarea/line-1/testmachine-015/TestChangingInt` | 904 | Good |
|
||
| uns `testarea/line-1/testmachine-015/TestDouble` | 0 | Good |
|
||
| uns `testarea3/line-1/testmachine-036/TestChangingInt` | 330 | Good |
|
||
| `Server.ServiceLevel` (i=2267) | 250 | Good |
|
||
| uns `…/testmachine-015/TestAlarm002Condition` | (none) | **BadWaitingForInitialData**, permanently. #896 |
|
||
|
||
## Step 3: subscribe (PASS)
|
||
|
||
Raw `…/TestMachine_015/TestChangingInt`, 1000 ms, 10 s: 9 Good notifications, the first inside the first
|
||
second (`49, 474, 899, 325, 750, 176, 601, 27, 452`). UNS equipment `EQ-b6f600000000` recursive: 2 variables,
|
||
8 notifications in 8 s (`TestChangingInt` every second; `TestHistoryValue` initial value only, since the
|
||
attribute is static). Summaries: `Ever went BAD during window: 0`, `No update received at all: 0`.
|
||
|
||
## Step 4: write and read back (PASS)
|
||
|
||
```
|
||
anonymous write raw TestDouble = 42.125 -> Write failed: BadUserAccessDenied (negative control; anonymous has no WriteOperate)
|
||
apikey g6gate write raw TestDouble = 42.125 -> Write successful
|
||
read raw …/TestMachine_015/TestDouble -> 42.125 Good
|
||
read uns testarea/line-1/testmachine-015/TestDouble -> 42.125 Good
|
||
apikey g6gate write uns TestDouble = 17.5 -> Write successful
|
||
read raw …/TestMachine_015/TestDouble -> 17.5 Good
|
||
```
|
||
|
||
Independent readback through a separate gateway session (mxgw-dotnet `smoke --item TestMachine_015.TestDouble`):
|
||
`OnDataChange doubleValue: 17.5, quality 192`. TestDouble was then written back to its original 0.
|
||
|
||
Before the key deploy, a write with the new key was refused (`AclGate: … key … is no longer enabled or has
|
||
expired in generation 1`). That is expected: data-plane key usability follows the deployed generation, so a
|
||
key inserted after a deploy needs a redeploy.
|
||
|
||
## Step 5: native alarm to Part 9, ack and clear (PASS)
|
||
|
||
The alarm was raised through OPC UA: `write raw …/TestMachine_015/TestAlarm002 = True` (key `g6gate`). The live
|
||
`alarms --refresh` subscriber (key `g6gate`) saw:
|
||
|
||
```
|
||
[2026-11-13T04:11:05.2390000Z] ALARM galaxy-windev/dev/TestArea/TestMachine_015/TestAlarm002Condition
|
||
State: Active, Unacknowledged
|
||
Severity: 750
|
||
Retain: True
|
||
EventId: 9503F17B7A8F82498FBCEDD868C4AD98
|
||
```
|
||
|
||
```
|
||
otopcua-cli ack -n "ns=2;s=…/TestAlarm002Condition" -e 9503F17B… -c "OtOpcUa #615 G6 gate" -U apikey -P <key>
|
||
Acknowledge successful
|
||
[2026-11-13T04:20:34.4620500Z] … State: Active, Acknowledged Retain: True
|
||
```
|
||
|
||
Driver log:
|
||
|
||
```
|
||
Galaxy driver g6gal: acknowledging Galaxy!TestArea.TestMachine_015.TestAlarm002 as Galaxy user Administrator (Fallback) for OPC UA principal g6gate (key 8e142bf2a2b70df8)
|
||
DriverInstance g6gal: acknowledged native condition galaxy-windev/dev/TestArea/TestMachine_015/TestAlarm002Condition by g6gate
|
||
```
|
||
|
||
Galaxy side, read through a separate gateway session after the ack:
|
||
|
||
| Attribute | Value |
|
||
|---|---|
|
||
| `TestMachine_015.TestAlarm002.Acked` | **True** (q 192) |
|
||
| `TestMachine_015.TestAlarm002.AckMsg` | **`OtOpcUa #615 G6 gate [OPC UA: g6gate, key 8e142bf2a2b70df8]`** |
|
||
| `TestMachine_015.TestAlarm002.InAlarm` | True |
|
||
| gateway `QueryActiveAlarms` | `ALARM_CONDITION_STATE_ACTIVE_ACKED`, `operatorUser: Administrator`, the same comment |
|
||
|
||
This is the leg H6 could not get on vd03 (`galaxy-h6-ack-2026-11-11.md`): the ack reached Galaxy.
|
||
|
||
The alarm was cleared through the UNS NodeId: `write uns testarea/line-1/testmachine-015/TestAlarm002 = False`.
|
||
Galaxy `TestAlarm002`/`.InAlarm` went False at once. The gateway feed's return-to-normal arrived between 40 s
|
||
and 100 s later; the gateway `ReconcileIntervalSeconds` is 30. The condition then went
|
||
`Inactive, Acknowledged`, `Retain: False`. The clear event's `Time` carried the earlier transition timestamp,
|
||
not the return-to-normal time. That latency and timestamp are gateway-side behaviour, so they are recorded
|
||
here but not filed against OtOpcUa.
|
||
|
||
**Severity (#894).** The gateway reported `severity 500` (AVEVA priority 500, High band 251–500), and the
|
||
condition read **750**. The authored `alarm.severity` was 500 for TestAlarm002 and 700 for TestAlarm003; the
|
||
live feed value overrode both. This is the #887 band mapping working live (750, as #894 expects), on another
|
||
alarm and another gateway than #894's vd03 target.
|
||
|
||
## Step 6: picker-form native alarm (FAIL on master, PASS with #895)
|
||
|
||
`TestAlarm003` was authored as the AdminUI Galaxy picker would author it: `attributeRef "TestMachine_015.TestAlarm003"`
|
||
plus the seeded `{"alarmType":"OffNormalAlarm","severity":700}`. It was raised by a supervisory write through a
|
||
separate gateway session. A plain `Write` is refused `SecurityError 1008`, because `TestAlarm003` is `MxSecurityOperate`.
|
||
|
||
- **Master `00a5be26`:** the gateway feed reported `Galaxy!TestArea.TestMachine_015.TestAlarm003` as
|
||
`ALARM_CONDITION_STATE_ACTIVE`. The OPC UA condition stayed `Inactive, Acknowledged`. Cause:
|
||
`OnAlarmFeedTransition` mapped only the feed's full reference to the authored attributeRef. The
|
||
`sourceObjectReference` the feed also sends (`TestMachine_015.TestAlarm003`) was ignored, so the transition got
|
||
a ConditionId matching no condition. → **#895**.
|
||
- **With the fix (`e113faf5`):** after a host restart on the same deployment, which the host restored from
|
||
LocalDb on bootstrap, ConditionRefresh returned
|
||
`TestAlarm003 State: Active, Unacknowledged Severity: 750 Retain: True`. The OPC UA `Acknowledge` dialled
|
||
`Galaxy!TestArea.TestMachine_015.TestAlarm003` (log line as above), and the condition went `Active, Acknowledged`.
|
||
Galaxy then read `.Acked = True` and `.AckMsg = "OtOpcUa #615 G6 gate picker-form (#895) [OPC UA: g6gate, key 8e142bf2a2b70df8]"`.
|
||
After the supervisory clear the condition went `Inactive, Acknowledged`, `Retain: False`, and the gateway
|
||
showed no active alarms.
|
||
|
||
## Not run: DeployWatcher on a Galaxy redeploy
|
||
|
||
This step was not run for two reasons:
|
||
|
||
- **No way to trigger a redeploy.** A Galaxy redeploy needs the ArchestrA IDE or a GRAccess script on windev.
|
||
No GRAccess assembly was found in the ArchestrA Framework bin directory.
|
||
- **Risk to another agent.** The test Galaxy was serving another agent's MXAccess captures, and redeploying
|
||
its objects would have disturbed them.
|
||
|
||
Expected behaviour if it is run later: `docs/drivers/Galaxy.md` says the `IRediscoverable` / `DeployWatcher`
|
||
path raises a "re-browse" prompt on `/hosts` and does **not** rebuild the address space (#518). The live check
|
||
would be a `/hosts` chip, not a tree rebuild. `Repository.WatchDeployEvents=true` was set and the driver started
|
||
cleanly with it.
|
||
|
||
## Defects
|
||
|
||
- **#895 (fixed here, `e113faf5`).** A native alarm authored by the AdminUI picker (`attributeRef tag.Attr`)
|
||
never went active, and its ack would dial a reference the gateway refuses. Three unit tests were added to
|
||
`GalaxyDriverRawPathKeyingTests`. The main one fails with the fix reverted, which was checked by reverting
|
||
the fix and re-running it.
|
||
- **#896 (fixed after this run).** The UNS mirror of an alarm-bearing raw tag gets a value Variable that stays
|
||
`BadWaitingForInitialData` forever, against the documented "condition instead of value variable" contract.
|
||
It was not fixed here because the composer test fixtures assert that mirror.
|
||
Fixed afterwards: the composer no longer emits a UNS Variable for a reference to an alarm-bearing tag, so the
|
||
equipment shows only the condition (the `<- #896` Variable in the Step 1 browse is gone).
|
||
|
||
## Cleanup
|
||
|
||
- **windev:** gateway key `g6-otopcua-e2e` revoked. No other key was touched. `MxAccessGw` was never stopped,
|
||
restarted or reconfigured, and `/health/ready` returned 200 throughout. Temporary `.ps1` files in
|
||
`C:\Users\Public` were deleted.
|
||
- **Galaxy:** TestDouble restored to 0. TestAlarm002 and TestAlarm003 are both back to False, and the gateway
|
||
shows no active alarms. Their raise/ack/clear rows stay in the Galaxy's own alarm history.
|
||
- **Mac:** local Host stopped. Database `OtOpcUaG6` dropped from the rig SQL container. The docker-dev rig
|
||
containers and their `OtOpcUa` DB were untouched. Scratch key files deleted.
|