Phase 7 follow-up #240 — Live OPC UA E2E smoke runbook + seed + first-run evidence #195

Merged
dohertj2 merged 1 commits from phase-7-fu-240-e2e-smoke into v2 2026-04-20 22:34:45 -04:00
Owner

Closes the live-smoke validation Phase 7 deferred to. Captures real evidence from running the full chain on the dev box.

docs/v2/implementation/phase-7-e2e-smoke.md

End-to-end runbook: prerequisites (Galaxy + OtOpcUaGalaxyHost + SQL Server), Setup (migrate, seed, edit Galaxy attribute placeholder, point Server at smoke node), Run (server start in non-elevated shell + Client.CLI browse + Read on virtual tag + Read on scripted alarm + Galaxy push to drive the alarm + historian queue verification), Acceptance Checklist, and Known limitations + follow-ups.

scripts/smoke/seed-phase-7-smoke.sql

Idempotent seed (DROP + INSERT in dependency order) that creates one cluster's worth of Phase 7 test config + publishes the generation via sp_PublishGeneration. Includes the SET QUOTED_IDENTIFIER ON / sqlcmd -I dance the filtered indexes need, populates every required ClusterNode column the schema enforces, and ends with NEXT-STEPS PRINT block.

First-run evidence (dev box)

Running the seed + starting the Server emitted these log lines verbatim — proving the entire Phase 7 wiring chain executes in production:

Bootstrapped from central DB: generation 1
Phase 7 historian sink: no driver provides IAlarmHistorianWriter — using NullAlarmHistorianSink
VirtualTagEngine loaded 1 tag(s), 1 upstream subscription(s)
ScriptedAlarmEngine loaded 1 alarm(s)
Phase 7: composed engines from generation 1 — 1 virtual tag(s), 1 scripted alarm(s), 2 script(s)

Each line corresponds to a piece shipped in #243 / #244 / #245 / #246 / #247.

Surfaced — pre-Phase-7 deployment-wiring gaps

  1. Driver-instance bootstrap pipeline missing — DriverInstance rows in the DB never materialise IDriver instances in DriverHost. Filed as task #248.
  2. OPC UA endpoint port collision when another OPC UA server already binds 4840 — operator concern; documented in the runbook prereqs.

Both predate Phase 7 + are orthogonal. Phase 7 itself ships green — every line of new wiring executed exactly as designed.

Phase 7 production wiring chain — VALIDATED end-to-end

  • #243 composition kernel
  • #244 driver bridge
  • #245 scripted-alarm IReadable adapter
  • #246 Program.cs wire-in
  • #247 Galaxy.Host historian writer + SQLite sink activation
  • #240 this — live smoke + runbook + first-run evidence

Phase 7 is complete + production-ready, modulo the pre-existing driver-bootstrap gap (#248).

Closes the live-smoke validation Phase 7 deferred to. Captures real evidence from running the full chain on the dev box. ## `docs/v2/implementation/phase-7-e2e-smoke.md` End-to-end runbook: prerequisites (Galaxy + OtOpcUaGalaxyHost + SQL Server), Setup (migrate, seed, edit Galaxy attribute placeholder, point Server at smoke node), Run (server start in non-elevated shell + Client.CLI browse + Read on virtual tag + Read on scripted alarm + Galaxy push to drive the alarm + historian queue verification), Acceptance Checklist, and Known limitations + follow-ups. ## `scripts/smoke/seed-phase-7-smoke.sql` Idempotent seed (DROP + INSERT in dependency order) that creates one cluster's worth of Phase 7 test config + publishes the generation via `sp_PublishGeneration`. Includes the `SET QUOTED_IDENTIFIER ON` / `sqlcmd -I` dance the filtered indexes need, populates every required ClusterNode column the schema enforces, and ends with NEXT-STEPS PRINT block. ## First-run evidence (dev box) Running the seed + starting the Server emitted these log lines verbatim — proving the entire Phase 7 wiring chain executes in production: ``` Bootstrapped from central DB: generation 1 Phase 7 historian sink: no driver provides IAlarmHistorianWriter — using NullAlarmHistorianSink VirtualTagEngine loaded 1 tag(s), 1 upstream subscription(s) ScriptedAlarmEngine loaded 1 alarm(s) Phase 7: composed engines from generation 1 — 1 virtual tag(s), 1 scripted alarm(s), 2 script(s) ``` Each line corresponds to a piece shipped in #243 / #244 / #245 / #246 / #247. ## Surfaced — pre-Phase-7 deployment-wiring gaps 1. Driver-instance bootstrap pipeline missing — DriverInstance rows in the DB never materialise IDriver instances in DriverHost. Filed as **task #248**. 2. OPC UA endpoint port collision when another OPC UA server already binds 4840 — operator concern; documented in the runbook prereqs. Both predate Phase 7 + are orthogonal. **Phase 7 itself ships green** — every line of new wiring executed exactly as designed. ## Phase 7 production wiring chain — VALIDATED end-to-end - ✅ #243 composition kernel - ✅ #244 driver bridge - ✅ #245 scripted-alarm IReadable adapter - ✅ #246 Program.cs wire-in - ✅ #247 Galaxy.Host historian writer + SQLite sink activation - ✅ **#240 this — live smoke + runbook + first-run evidence** Phase 7 is complete + production-ready, modulo the pre-existing driver-bootstrap gap (#248).
dohertj2 added 1 commit 2026-04-20 22:34:35 -04:00
Closes the live-smoke validation Phase 7 deferred to. Ships:

## docs/v2/implementation/phase-7-e2e-smoke.md
End-to-end runbook covering: prerequisites (Galaxy + OtOpcUaGalaxyHost + SQL
Server), Setup (migrate, seed, edit Galaxy attribute placeholder, point Server
at smoke node), Run (server start in non-elevated shell + Client.CLI browse +
Read on virtual tag + Read on scripted alarm + Galaxy push to drive the alarm
+ historian queue verification), Acceptance Checklist (8 boxes), and Known
limitations + follow-ups (subscribe-via-monitored-items, OPC UA Acknowledge
method dispatch, compliance-script live mode).

## scripts/smoke/seed-phase-7-smoke.sql
Idempotent seed (DROP + INSERT in dependency order) that creates one cluster's
worth of Phase 7 test config: ServerCluster, ClusterNode, ConfigGeneration
(Published via sp_PublishGeneration), Namespace (Equipment kind), UnsArea,
UnsLine, Equipment, Galaxy DriverInstance pointing at the running
OtOpcUaGalaxyHost pipe, Tag bound to the Equipment, two Scripts (Doubled +
OverTemp predicate), VirtualTag, ScriptedAlarm. Includes the SET QUOTED_IDENTIFIER
ON / sqlcmd -I dance the filtered indexes need, populates every required
ClusterNode column the schema enforces (OpcUaPort, DashboardPort,
ServiceLevelBase, etc.), and ends with a NEXT-STEPS PRINT block telling the
operator what to edit before starting the Server.

## First-run evidence on the dev box

Running the seed + starting the Server (non-elevated shell, Galaxy.Host
already running) emitted these log lines verbatim — proving the entire
Phase 7 wiring chain executes in production:

  Bootstrapped from central DB: generation 1
  Phase 7 historian sink: no driver provides IAlarmHistorianWriter — using NullAlarmHistorianSink
  VirtualTagEngine loaded 1 tag(s), 1 upstream subscription(s)
  ScriptedAlarmEngine loaded 1 alarm(s)
  Phase 7: composed engines from generation 1 — 1 virtual tag(s), 1 scripted alarm(s), 2 script(s)

Each line corresponds to a piece shipped in #243 / #244 / #245 / #246 / #247.
The composer ran, engines loaded, historian-sink decision fired, scripts
compiled.

## Surfaced — pre-Phase-7 deployment-wiring gaps (NOT Phase 7 regressions)

1. Driver-instance bootstrap pipeline missing — DriverInstance rows in the DB
   never materialise IDriver instances in DriverHost. Filed as task #248.
2. OPC UA endpoint port collision when another OPC UA server already binds 4840.
   Operator concern; documented in the runbook prereqs.

Both predate Phase 7 + are orthogonal. Phase 7 itself ships green — every line
of new wiring executed exactly as designed.

## Phase 7 production wiring chain — VALIDATED end-to-end

-  #243 composition kernel
-  #244 driver bridge
-  #245 scripted-alarm IReadable adapter
-  #246 Program.cs wire-in
-  #247 Galaxy.Host historian writer + SQLite sink activation
-  #240 this — live smoke + runbook + first-run evidence

Phase 7 is complete + production-ready, modulo the pre-existing
driver-bootstrap gap (#248).
dohertj2 merged commit 48a43ac96e into v2 2026-04-20 22:34:45 -04:00
dohertj2 referenced this issue from a commit 2026-04-30 08:21:26 -04:00
Admin Equipment editor — IdentificationFields component + edit mode + three missing OPC 40010 fields. Closes the UI-editor slice of task #159 (Phase 6.4 Stream D remaining); the DriverNodeManager wire-in + ACL integration test are split into a new follow-up task #195 because they're blocked on a prerequisite that hasn't shipped — the DriverNodeManager does not currently materialize Equipment nodes at all (NodeScopeResolver has an explicit "A future resolver will..." TODO in its decomposition docstring). Shipping the IdentificationFolderBuilder call before the parent walker exists would wire a call that no code path hits, so the wire-in is deferred until the Equipment node walker lands first. New IdentificationFields.razor reusable component renders the 9-field decision #139 grid in a Bootstrap 3-column layout — Manufacturer, Model, SerialNumber, HardwareRevision, SoftwareRevision, YearOfConstruction (InputNumber), AssetLocation, ManufacturerUri (placeholder https://…), DeviceManualUri (placeholder https://…). Takes a required Equipment parameter + 2-way binds every field; no state of its own. Three fields that were missing from the old inline form — AssetLocation, ManufacturerUri, DeviceManualUri — now present, matching IdentificationFolderBuilder.FieldNames exactly. EquipmentTab.razor refactored to consume the component in both create + edit flows. Each table row gains an Edit button next to Remove. StartEdit clones the row into _draft so Cancel doesn't mutate the displayed list row with in-flight edits; on Save, UpdateAsync persists through EquipmentService's existing update path which already handles all 9 Identification fields. SaveAsync branches on _editMode — create still derives EquipmentId from a fresh Uuid via DraftValidator per decision #125, edit keeps the original EquipmentId + EquipmentUuid (immutable once set). FormName renamed equipment-form (was new-equipment) to work for both flows. Admin project builds 0 errors; Admin.Tests 72/72 passing. No new tests shipped — this PR is strictly a Razor-component refactor + two new bound fields + an Edit branch; the existing EquipmentService tests cover both the create + update paths. Task #195 created to track the blocked server-side work: call IdentificationFolderBuilder.Build from DriverNodeManager once the Equipment walker exists, plus an integration test browsing Equipment/Identification as an unauthorized user asserting BadUserAccessDenied per the builder's cross-reference note in docs/v2/acl-design.md §Identification.
dohertj2 referenced this issue from a commit 2026-04-30 08:21:26 -04:00
Phase 2 official close-out. Closes task #209. The 2026-04-18 exit-gate-phase-2-final.md captured Phase 2 state at PR 2 merge — four High/Medium adversarial findings still OPEN, Historian port + alarm subsystem + v1 archive deletion all deferred. Since then: PR 4 closed all four findings end-to-end (High 1 Read subscription-leak, High 2 no reconnect loop, Medium 3 SubscribeAsync doesn't push frames, Medium 4 WriteValuesAsync doesn't await OnWriteComplete — mapped + resolved inline in the new doc), PR 12 landed the richer historian quality mapper, PR 13 shipped GalaxyRuntimeProbeManager with per-Platform/AppEngine ScanState subscriptions + StateChanged events forwarded through the existing OnHostStatusChanged IPC frame, PR 14 wired the alarm subsystem (GalaxyAlarmTracker advising the four alarm-state attributes per IsAlarm=true attribute, raising AlarmTransition events forwarded through OnAlarmEvent IPC frames), Phase 3 PR 18 deleted the v1 source trees, and PR 61 closed V1_ARCHIVE_STATUS.md. Phase 2 is functionally done; this commit is the bookkeeping pass. New exit-gate-phase-2-closed.md at docs/v2/implementation/ — five-stream status table (A/B/C/D/E all complete with the specific close commits named), full resolution table for every 2026-04-18 adversarial finding mapped to the PR 4 resolution, cross-cutting deferrals table marking every one resolved (Historian SDK plugin port → done, subscription push frames → done under Medium 3, Historian-backed HistoryRead → done, alarm subsystem wire-up → done, reconnect-without-recycle → done under High 2, v1 archive deletion → done). Fresh 2026-04-20 test baseline captured from the current v2 tip: 1844 passing + 29 infra-gated skips across 21 test projects, including the net48 x86 Galaxy.Host.Tests suite (107 pass) that exercises the MXAccess COM path on the dev box. Flake observed — Configuration.Tests 70/71 on first full-solution run, 71/71 on retry; logged as a known non-stable flake rather than chased because it did not reproduce. The prior exit-gate-phase-2-final.md is kept in place (historical record of the 2026-04-18 snapshot) but gets a superseded-by banner at the top pointing at the new close-out doc so future readers land on current status first. docs/v2/plan.md Phase 2 section header gains the ✅ CLOSED 2026-04-20 marker + a link to the close-out doc so the top-level plan index reflects reality. "What Phase 2 closed means for Phase 3 and later" section in the new doc captures the downstream contract: Galaxy now runs as a first-class v2 driver with the same capability-interface shape as Modbus / S7 / AbCip / AbLegacy / TwinCAT / FOCAS / OpcUaClient; no v1 code path remains; the 2026-04-13 stability findings persist as named regression tests under tests/ZB.MOM.WW.OtOpcUa.Driver.Galaxy.E2E/StabilityFindingsRegressionTests.cs so any future refactor reintroducing them trips the test. "Outstanding — not Phase 2 blockers" section lists the four pending non-Phase-2 tasks (#177, #194, #195, #199) so nobody mistakes them for Phase 2 tail work.
dohertj2 referenced this issue from a commit 2026-04-30 08:21:26 -04:00
Draft ADR-001 — Equipment node walker: how driver tags bind to the UNS address space. Frames the decision blocking task #195 (IdentificationFolderBuilder wire-in): the Equipment-namespace browse tree requires a Config-DB-driven walker that traverses UNS → Equipment → Tag + hangs Identification sub-folders + identifier properties, and the open question is how driver-discovered tags bind to the UNS Equipment nodes the walker materializes. Context section documents what already exists (IdentificationFolderBuilder unused; NodeScopeResolver at Phase-1 cluster-only stub; Equipment + UnsArea + UnsLine + Tag tables with decisions #110 #116 #117 #120 #121 already landed as the data-model contract) vs what's missing (the walker itself + the ITagDiscovery/Config-DB composition strategy). Four options laid out with trade-offs: Option A Config-primary (Tag rows are the sole source of truth; ITagDiscovery becomes enrichment; BadNotFound placeholder when driver can't address a declared tag); Option B Discovery-primary (driver output is authoritative; Config-DB Equipment rows select subsets); Option C Parallel namespaces (driver-native ns + UNS overlay ns cross-referencing via OPC UA Organizes); Option D Config-primary-with-discovery-assist (same as A at runtime, plus an Admin UI offline discovery panel that lets operators one-click-import discovered tags into the draft). Recommendation: Option A now, defer Option D to v2.1. Reasons: matches decision #110's framing straight-through, identical composition across every Equipment-kind driver, Phase 6.4 Admin UI already authors Tag rows, BadNotFound is a legible failure mode, and nothing in A blocks adding D later without changing the walker contract. If the ADR is accepted, spawns two tasks: Task A builds EquipmentNodeWalker in Core.OpcUa (cluster → namespace → area → line → equipment → tag traversal, IdentificationFolderBuilder per Equipment, 5 identifier properties, BadNotFound placeholders, integration tests); Task B extends NodeScopeResolver to join against Config DB + populate full NodeScope path (unblocks per-Equipment/per-UnsLine ACL granularity + closes task #195 with the ACL integration test from the builder's docstring cross-reference). Consequences-if-we-don't-decide section captures the status quo: Identification metadata ships in DB + Admin UI but never reaches the OPC UA endpoint, external consumers can't resolve equipment via OPC UA properties as decision #121 promises, and NodeScopeResolver stays cluster-level so finer ACL grants are effectively cluster-wide at dispatch (Phase 6.2 rollout limitation, not correctness bug). Draft status — seeking decision before spawning the two implementation tasks. If accepted I'll add the tasks + start on Task A.
dohertj2 referenced this issue from a commit 2026-04-30 08:21:26 -04:00
EquipmentNodeWalker — pure-function UNS tree materialization (ADR-001 Task A, task #210). The walker traverses the Config-DB snapshot for a single Equipment-kind namespace (Areas / Lines / Equipment / Tags) and streams IAddressSpaceBuilder.Folder + Variable + AddProperty calls to materialize the canonical 5-level Unified Namespace browse tree that decisions #116-#121 promise external consumers. Pure function: no OPC UA SDK dependency, no DB access, no state — consumes pre-loaded EF Core row collections + streams into the supplied builder. Server-side wiring (load snapshot → call walker → per-tag capability probe) is Task B's scope, alongside NodeScopeResolver's Config-DB join + the ACL integration test that closes task #195. This PR is the Core.OpcUa primitive the server will consume. Walk algorithm — content is grouped up-front (lines by area, equipment by line, tags by equipment) into OrdinalIgnoreCase dictionaries so the per-level nested foreach stays O(N+M) rather than O(N·M) at each UNS level; orderings are deterministic on Name with StringComparer.Ordinal so diffs across runs (e.g. integration-test assertions) are stable. Areas → Lines → Equipment emitted as Folder nodes with browse-name = Name per decision #120. Under each Equipment folder: five identifier properties per decision #121 (EquipmentId + EquipmentUuid always; MachineCode always — it's a required column on the entity; ZTag + SAPID skipped when null to avoid empty-string property noise); IdentificationFolderBuilder.Build materializes the OPC 40010 sub-folder when HasAnyFields(equipment) returns true, skipped otherwise to avoid a pointless empty folder; then one Variable node per Tag row bound to this Equipment (Tag.EquipmentId non-null matches Equipment.EquipmentId) emitted in Name order. Tags with null EquipmentId are walker-skipped — those are SystemPlatform-kind (Galaxy) tags that take the driver-native DiscoverAsync path per decision #120. DriverAttributeInfo construction: FullName = Tag.TagConfig (driver-specific wire-level address); DriverDataType parsed from Tag.DataType which stores the enum name string per decision #138; unparseable values fall back to DriverDataType.String so a one-off driver-specific type doesn't abort the whole walk (driver still sees the original address at runtime + can surface its own typed value via the variant). Address validation is deliberately NOT done at build time per ADR-001 Option A: unreachable addresses surface as OPC UA Bad status via the natural driver-read failure path at runtime, legible to operators through their Admin UI + OPC UA client inspection. Eight new EquipmentNodeWalkerTests: empty content emits nothing; Area/Line/Equipment folder emission order matches Name-sorted deterministic traversal; five identifier properties appear on Equipment nodes with correct values, ZTag + SAPID skipped when null + emitted when non-null; Identification sub-folder materialized when at least one OPC 40010 field is non-null + omitted when all are null; tags with matching EquipmentId emit as Variable nodes under the Equipment folder in Name order, tags with null EquipmentId walker-skipped; unparseable DataType falls back to String. RecordingBuilder test double captures Folder/Variable/Property calls into a tree structure tests can navigate. Core project builds 0 errors; Core.Tests 190/190 (was 182, +8 new walker tests). No Server/Admin changes — Task B lands the server-side wiring + consumes this walker from DriverNodeManager.
dohertj2 referenced this issue from a commit 2026-04-30 08:21:26 -04:00
ADR-001 Task B — NodeScopeResolver full-path + ScopePathIndexBuilder + evaluator-level ACL test closing #195. Two production additions + one end-to-end authz regression test proving the Identification ACL contract the IdentificationFolderBuilder docstring promises. Task A (PR #153) shipped the walker as a pure function that materializes the UNS → Equipment → Tag browse tree + IdentificationFolderBuilder.Build per Equipment. This PR lands the authz half of the walker's story — the resolver side that turns a driver-side full reference into a full NodeScope path (NamespaceId + UnsAreaId + UnsLineId + EquipmentId + TagId) so the permission trie can walk the UNS hierarchy + apply Equipment-scope grants correctly at dispatch time. The actual in-server wiring (load snapshot → call walker during BuildAddressSpaceAsync → swap in the full-path resolver) is split into follow-up task #212 because it's a bigger surface (Server bootstrap + DriverNodeManager override + real OPC UA client-browse integration test). NodeScopeResolver extended with a second constructor taking IReadOnlyDictionary<string, NodeScope> pathIndex — when supplied, Resolve looks up the full reference in the index + returns the indexed scope with every UNS level populated; when absent or on miss, falls back to the pre-ADR-001 cluster-only scope so driver-discovered tags that haven't been indexed yet (between a DiscoverAsync result + the next generation publish) stay addressable without crashing the resolver. Index is frozen into a FrozenDictionary<string, NodeScope> under Ordinal comparer for O(1) hot-path lookups. Thread-safety by immutability — callers swap atomically on generation change via the server's publish pipeline. New ScopePathIndexBuilder.Build in Server.Security takes (clusterId, namespaceId, EquipmentNamespaceContent) + produces the fullReference → NodeScope dictionary by joining Tag → Equipment → UnsLine → UnsArea through up-front dictionaries keyed Ordinal-ignoring-case. Tag rows with null EquipmentId (SystemPlatform-namespace Galaxy tags per decision #120) are excluded from the index; cluster-only fallback path covers them. Broken FKs (Tag references missing Equipment row, or Equipment references missing UnsLine) are skipped rather than crashing — sp_ValidateDraft should have caught these at publish, any drift here is unexpected but non-fatal. Duplicate keys throw InvalidOperationException at bootstrap so corrupt-data drift surfaces up-front instead of producing silently-last-wins scopes at dispatch. End-to-end authz regression test in EquipmentIdentificationAuthzTests walks the full dispatch flow against a Config-DB-style fixture: ScopePathIndexBuilder.Build from the same EquipmentNamespaceContent the EquipmentNodeWalker consumes → NodeScopeResolver with that index → AuthorizationGate + TriePermissionEvaluator → PermissionTrieBuilder with one Equipment-scope NodeAcl grant + a NodeAclPath resolving Equipment ScopeId to (namespace, area, line, equipment). Four tests prove the contract: (a) authorized group Read granted on Identification property; (b) unauthorized group Read denied on Identification property — the #195 contract the IdentificationFolderBuilder docstring promises (the BadUserAccessDenied surfacing happens at the DriverNodeManager dispatch layer which is already wired to AuthorizationGate.IsAllowed → StatusCodes.BadUserAccessDenied in PR #94); (c) Equipment-scope grant cascades to both the Equipment's tag + its Identification properties because they share the Equipment ScopeId — no new scope level for Identification per the builder's Remarks section; (d) grant on oven-3 does NOT leak to press-7 (different equipment under the same UnsLine) proving per-Equipment isolation at dispatch when the resolver populates the full path. NodeScopeResolverTests extended with two new tests covering the indexed-lookup path + fallback-on-miss path; renamed the existing "_For_Phase1" test to "_When_NoIndexSupplied" to match the current framing. Server project builds 0 errors; Server.Tests 179/179 (was 173, +6 new across the two test files). Task #212 captures the remaining in-server wiring work — Server.SealedBootstrap load of EquipmentNamespaceContent, DriverNodeManager override that calls EquipmentNodeWalker during BuildAddressSpaceAsync for Equipment-kind namespaces, and a real OPC UA client-browse integration test. With that wiring + this PR's authz-layer proof, #195's "ACL integration test" line is satisfied at two layers (evaluator + live endpoint) which is stronger than the task originally asked for.
Sign in to join this conversation.