Compare commits

...

36 Commits

Author SHA1 Message Date
Joseph Doherty b80abbb14b docs(index): HistorianGateway store-forward now FasterLog-backed; refresh test count (702/681) 2026-06-24 11:19:56 -04:00
Joseph Doherty 6c2d16d4af docs: refresh HistorianGateway + GalaxyRepository status in index
HistorianGateway is now pushed to gitea (gitea.dohertylan.com/dohertj2/historiangw), and
ZB.MOM.WW.GalaxyRepository is published to the Gitea feed and consumed as a PackageReference
(no longer a cross-repo ProjectReference). Updates the sister-project row, the component
table, and the GalaxyRepository narrative; test figure 584 green -> 590 total (584 on macOS).
2026-06-24 07:28:25 -04:00
Joseph Doherty a08ddab9dd chore: retire unused ZB.MOM.WW.SPHistorianClient (stale partial port; superseded by histsdk vendored in HistorianGateway; no consumers, not on feed) 2026-06-24 06:45:10 -04:00
Joseph Doherty 744eb090ac docs(scadaproj): index ZB.MOM.WW.HistorianGateway sidecar + GalaxyRepository shared lib
- Add HistorianGateway to the Runtime/implementation table (single-process
  .NET 10 x64 gRPC sidecar; no COM/x86; 584 tests; local only, not yet
  pushed to gitea)
- Update "What this repository is" count (five → six pieces of source;
  add GalaxyRepository)
- Add HistorianGateway paragraph to Cross-project relationships / Net effect
  (independent sidecar; no runtime coupling to the other three; depends on
  shared GalaxyRepository lib via ProjectReference)
- Add ZB.MOM.WW.GalaxyRepository row to Component normalization table +
  full description paragraph (built 0.1.0; consumed by HistorianGateway;
  mxaccessgw adoption is a follow-on; not yet published to Gitea feed)
- Add HistorianGateway primary commands block (build/test/run/live-integration)
- Extend Shared GLAuth note to cover HistorianGateway
2026-06-24 00:41:29 -04:00
Joseph Doherty 94512acf1f fix(galaxyrepo): drop no-op ValidateOnStart (consumer owns validation) 2026-06-23 20:36:28 -04:00
Joseph Doherty 2c6c764d3c test(galaxyrepo): projector + cache tests; dispose semaphores; pack 0.1.0 2026-06-23 20:34:32 -04:00
Joseph Doherty a30f8551e9 feat(galaxyrepo): reusable gRPC service + AddZbGalaxyRepository DI 2026-06-23 20:26:59 -04:00
Joseph Doherty afd0287f54 feat(galaxyrepo): hierarchy cache + snapshot + refresh service + projector 2026-06-23 20:22:35 -04:00
Joseph Doherty 1041f87b59 feat(galaxyrepo): SQL browse provider (hierarchy + attributes) 2026-06-23 20:12:33 -04:00
Joseph Doherty 5572edda85 feat(galaxyrepo): canonical galaxy_repository.v1 proto (neutral namespace) 2026-06-23 20:05:39 -04:00
Joseph Doherty aff7264df8 feat(galaxyrepo): scaffold ZB.MOM.WW.GalaxyRepository shared lib 2026-06-23 19:48:43 -04:00
Joseph Doherty 510b0010d6 docs(historian-gateway): implementation plan + task ledger (31 tasks) 2026-06-23 19:43:08 -04:00
Joseph Doherty 42ad31aded docs(historian-gateway): brainstormed design for ZB.MOM.WW.HistorianGateway sidecar 2026-06-23 19:31:54 -04:00
Joseph Doherty e3c0503a4f docs(sphistorianclient): mark RemoteGrpc (2023 R2) live-verified 2026-06-19 06:57:06 -04:00
Joseph Doherty a0527f9b5a fix(sphistorianclient): gRPC auth handshake uses StorageService.ValidateClientCredential
The RemoteGrpc orchestrator drove the SSPI/NTLM token loop through
HistoryService.ExchangeKey, which the 2023 R2 contract analysis shows is a
separate key-exchange/cert op — not the credential handshake. The server
rejected the NTLM Type-1 token at round 0. The Negotiate loop belongs on
StorageService.ValidateClientCredential (Handle/InBuff -> Status/OutBuff;
field names match the 2020 native contract). Live-verified end-to-end against
a 2023 R2 Historian (wonder-sql-vd03): SysTimeSec raw read returns correct
timestamped values.
2026-06-19 06:56:44 -04:00
Joseph Doherty 5f7d7e1b58 docs(sphistorianclient): document HISTORIAN_PORT env var; mark plan tasks complete 2026-06-19 06:09:43 -04:00
Joseph Doherty 78418346df build(sphistorianclient): pack 0.1.0 nupkg 2026-06-19 06:02:05 -04:00
Joseph Doherty 4920b89666 docs(sphistorianclient): correct retrieval-mode count (15) + EnsureTag verification scope 2026-06-19 06:01:07 -04:00
Joseph Doherty 989db9317d docs(sphistorianclient): add CLAUDE.md + README.md 2026-06-19 05:58:13 -04:00
Joseph Doherty 81bf7322f0 feat(sphistorianclient): add AddZbSpHistorianClient DI extension 2026-06-19 05:53:56 -04:00
Joseph Doherty 8033a7f12d fix(sphistorianclient): resolve port build/test fallout 2026-06-19 05:49:22 -04:00
Joseph Doherty 63cddfb65b feat(sphistorianclient): port SDK source + tests, rebrand namespace to ZB.MOM.WW.SPHistorianClient 2026-06-19 05:45:06 -04:00
Joseph Doherty 965f5006f2 feat(sphistorianclient): scaffold shared library skeleton (props, csprojs, slnx) 2026-06-19 05:40:10 -04:00
Joseph Doherty 294da8b2db docs(sphistorianclient): implementation plan + task tracking 2026-06-19 05:36:24 -04:00
Joseph Doherty bbb7942788 docs(sphistorianclient): approved design for ZB.MOM.WW.SPHistorianClient port 2026-06-19 05:29:51 -04:00
Joseph Doherty d5b134b117 docs: add MES + Delmia-DNC integration API/MXAccess specs
mes-delmia-integration-api.md: endpoints, request/response DTOs, and the MXAccess flag handshake for MESAPI (in-repo MesNotifier) and DelmiaIntegration (DNC Downloader.asmx -> WWNotifier /notify -> Galaxy $DelmiaReceiver). mesrec.md / nj.md: live Galaxy receiver + reactor attribute references.
2026-06-17 06:52:36 -04:00
Joseph Doherty eb8b44c29d loader: purge legacy driver in overlay namespace on teardown (self-heal nw-uns-modbus placeholder) 2026-06-08 07:07:22 -04:00
Joseph Doherty a6fa36043a loader: equipment is driver-less (drop Modbus placeholder, NULL DriverInstanceId) 2026-06-08 06:42:31 -04:00
Joseph Doherty 05a4a547f4 feat(loader): canonical EQ-+uuid EquipmentIds (passes OtOpcUa full DraftValidator); clean by UnsLine scope 2026-06-07 11:18:39 -04:00
Joseph Doherty 4d57e34ff3 docs(loader): record live-values verification + 396/1036 explanation for company overlay 2026-06-07 06:08:36 -04:00
Joseph Doherty b3d8990a0f fix(loader): keep empty folderPath distinct in vtag ids; dedupe verify args; readme wait-seconds 2026-06-07 05:07:00 -04:00
Joseph Doherty 5655b75fe6 feat(loader): company overlay as VirtualTags mirroring the galaxy mirror + verify --require-good 2026-06-07 04:59:51 -04:00
Joseph Doherty dce6f83488 loader: add populate-equipment (company-shape Equipment overlay) + scope verify-equipment
populate-equipment loads the Northwind Enterprise/Site/Area/Line/Equipment/Signal
shape from company-uns.json as a second Equipment-kind namespace (nw-uns) alongside
the galaxy mirror — 3 areas / 8 lines / 40 equipment / 1036 signals. Friendly
DisplayName, stable logical-Id NodeId. verify-equipment now scopes to the nw-area-*
overlay by default (--all for the whole tree). Verified live on :4840 against OtOpcUa
master's Equipment-namespace materialization (structure-only; leaves are
BadWaitingForInitialData). clean now drops the overlay too.
2026-06-06 16:19:53 -04:00
Joseph Doherty fd34e25cb1 feat(uns-loader): verify-equipment — recursive Equipment UNS tree browse + leaf count
browse_summary assumes the flat 2-level Galaxy hierarchy; the Equipment tree is deep
(Area/Line/Equipment/[FolderPath]/Signal). Add browse_tree (recursive leaf descent) + a
verify-equipment subcommand that reports/asserts the leaf signal count (--expect N), for
verifying OtOpcUa equipment-namespace structure materialisation. Smoke-tested against a live
:4840 (40 folders / 396 leaf signals).
2026-06-06 15:25:17 -04:00
Joseph Doherty eb26bf3248 Add Galaxy UNS artifacts + reloadable OtOpcUa loader tool
galaxy-hierarchy.json: full AVEVA Galaxy DEV hierarchy pulled live via the
MxGateway .NET client (129 objects, 14k attrs). company-uns.json/.tree.txt +
gen_uns.py: a fake-company (Northwind) ISA-95 UNS modeled on OtOpcUa's
Cluster->Namespace->Area->Line->Equipment->Tag schema, grounded in the 40
TestMachine instances. otopcua-uns-loader/: reloadable generate/populate/verify/
clean tool that recreates + verifies the galaxy mirror (396 live tags across 40
machines) in OtOpcUa's config DB after a rebuild.
2026-06-06 14:22:25 -04:00
Joseph Doherty e5a609be83 docs(theme): mark themeissues #6 resolved in 0.3.1
Interactive-render nav fix (CSS display:none-when-closed + nav-state.js
MutationObserver re-wire) shipped in 0.3.1 and verified — ScadaBridge Central UI
NavCollapseTests now pass. All six issues now resolved (5 fixed, 1 tradeoff).
2026-06-05 08:32:03 -04:00
56 changed files with 141228 additions and 16 deletions
+49 -11
View File
@@ -6,12 +6,13 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
`scadaproj` is primarily an umbrella/index workspace that aggregates a family of `scadaproj` is primarily an umbrella/index workspace that aggregates a family of
related SCADA / OT / Wonderware / OPC UA "sister projects" that live as **sibling related SCADA / OT / Wonderware / OPC UA "sister projects" that live as **sibling
directories under `~/Desktop/`**. It now also **hosts five pieces of source itself** directories under `~/Desktop/`**. It now also **hosts six pieces of source itself**
the shared [`ZB.MOM.WW.Auth/`](ZB.MOM.WW.Auth/) library, the shared the shared [`ZB.MOM.WW.Auth/`](ZB.MOM.WW.Auth/) library, the shared
[`ZB.MOM.WW.Theme/`](ZB.MOM.WW.Theme/) UI kit, the shared [`ZB.MOM.WW.Theme/`](ZB.MOM.WW.Theme/) UI kit, the shared
[`ZB.MOM.WW.Health/`](ZB.MOM.WW.Health/) health-check library, the shared [`ZB.MOM.WW.Health/`](ZB.MOM.WW.Health/) health-check library, the shared
[`ZB.MOM.WW.Telemetry/`](ZB.MOM.WW.Telemetry/) observability library, and the shared [`ZB.MOM.WW.Telemetry/`](ZB.MOM.WW.Telemetry/) observability library, the shared
[`ZB.MOM.WW.Configuration/`](ZB.MOM.WW.Configuration/) config-validation library — all the realized output of their [`ZB.MOM.WW.Configuration/`](ZB.MOM.WW.Configuration/) config-validation library, and the new
[`ZB.MOM.WW.GalaxyRepository/`](ZB.MOM.WW.GalaxyRepository/) Galaxy browse library — all the realized output of their
respective component normalizations (see [Component normalization](#component-normalization)). respective component normalizations (see [Component normalization](#component-normalization)).
The point of this file is to give a high-level scan of each sister project — its purpose, The point of this file is to give a high-level scan of each sister project — its purpose,
location, stack, and primary commands — so a fresh Claude Code session can orient across location, stack, and primary commands — so a fresh Claude Code session can orient across
@@ -30,9 +31,10 @@ own `CLAUDE.md` for the full picture. See [Refreshing this index](#refreshing-th
| Project | Location | Stack | Repo | Summary | | Project | Location | Stack | Repo | Summary |
|---|---|---|---|---| |---|---|---|---|---|
| **OtOpcUa** | `~/Desktop/OtOpcUa` | .NET 10, OPC UA, gRPC | `gitea.dohertylan.com/dohertj2/lmxopcua` | OPC UA server that exposes AVEVA System Platform (Wonderware) Galaxy tags as an OPC UA address space. Galaxy access flows through an in-process `GalaxyDriver` → gRPC → the **mxaccessgw** gateway. | | **OtOpcUa** | `~/Desktop/OtOpcUa` | .NET 10, OPC UA, gRPC | `gitea.dohertylan.com/dohertj2/lmxopcua` | OPC UA server that exposes industrial data sources under a **unified Equipment-based address space** — native-protocol drivers (Modbus, S7, AB CIP/Legacy, TwinCAT, FOCAS, OpcUaClient) **and AVEVA System Platform (Wonderware) Galaxy, now a standard Equipment-kind driver** (the old SystemPlatform mirror / alias-tag model was retired ~2026-06-12). Galaxy access flows through the in-process `GalaxyDriver` → gRPC → the **mxaccessgw** gateway. Surfaces live read + authorized write, native OPC UA Part 9 alarms, and server-side HistoryRead. |
| **MxAccessGateway** (`mxaccessgw`) | `~/Desktop/MxAccessGateway` | .NET 10 gateway (x64) + .NET 4.8 worker (**x86**), gRPC | `gitea.dohertylan.com/dohertj2/mxaccessgw` | gRPC gateway giving modern clients full MXAccess parity without loading 32-bit COM. Two-process: gateway (ASP.NET Core gRPC + Blazor dashboard) + per-session x86 worker that owns the MXAccess COM STA. **OtOpcUa depends on this.** | | **MxAccessGateway** (`mxaccessgw`) | `~/Desktop/MxAccessGateway` | .NET 10 gateway (x64) + .NET 4.8 worker (**x86**), gRPC | `gitea.dohertylan.com/dohertj2/mxaccessgw` | gRPC gateway giving modern clients full MXAccess parity without loading 32-bit COM. Two-process: gateway (ASP.NET Core gRPC + Blazor dashboard) + per-session x86 worker that owns the MXAccess COM STA. **OtOpcUa depends on this.** |
| **ScadaBridge** | `~/Desktop/ScadaBridge` | .NET 10, Akka.NET, Docker | _git_ | Full implementation of the distributed SCADA platform — hub-and-spoke (1 central cluster + N site clusters). Projects prefixed `ZB.MOM.WW.ScadaBridge.*`; solution `ZB.MOM.WW.ScadaBridge.slnx`. Ships `src/`, `tests/`, `docker/` topology, and the design docs that are the spec. | | **ScadaBridge** | `~/Desktop/ScadaBridge` | .NET 10, Akka.NET, Docker | _git_ | Full implementation of the distributed SCADA platform — hub-and-spoke (1 central cluster + N site clusters). Projects prefixed `ZB.MOM.WW.ScadaBridge.*`; solution `ZB.MOM.WW.ScadaBridge.slnx`. Ships `src/`, `tests/`, `docker/` topology, and the design docs that are the spec. |
| **HistorianGateway** | `~/Desktop/HistorianGateway` | .NET 10 x64, gRPC, Blazor | `gitea.dohertylan.com/dohertj2/historiangw` | Single-process gRPC sidecar exposing (1) full read/write API to the AVEVA Historian (5 gRPC services; 15 retrieval modes; historical/backfill writes; tag-config lifecycle; SQL live-value path; store-forward + redundancy resilience; all default-disabled) and (2) read-only Galaxy object-hierarchy browse via the shared `ZB.MOM.WW.GalaxyRepository` lib (consumed as a Gitea-feed package). No COM, no x86 worker. Dashboard on `:5220` (HTTP/1.1); gRPC h2c on `:5221`. Vendors `AVEVA.Historian.Client` from `histsdk`. Store-forward uses a crash-safe FasterLog append-only outbox (`Microsoft.FASTER.Core` 2.6.5; `CommitMode` PerEntry/Periodic), not SQLite. 702 tests total — 681 green on macOS; the env-gated live historian + Galaxy integration suite (21 tests) skips without a live server. |
## Cross-project relationships ## Cross-project relationships
@@ -84,8 +86,10 @@ the gateway uses `MxGateway.*`). The common subject is **AVEVA System Platform (
`GalaxyRepositoryClient` for the static hierarchy, and an MXAccess session `GalaxyRepositoryClient` for the static hierarchy, and an MXAccess session
(`MxCommand`/`MxEvent` protos) for live read/write/subscribe. A `DeployWatcher` polls the (`MxCommand`/`MxEvent` protos) for live read/write/subscribe. A `DeployWatcher` polls the
gateway's deploy-event signal to rebuild the OPC UA address space on Galaxy redeploy. gateway's deploy-event signal to rebuild the OPC UA address space on Galaxy redeploy.
OtOpcUa's job is purely a **protocol bridge**: it republishes Galaxy as an OPC UA address OtOpcUa's job is a **protocol bridge**: it republishes Galaxy — now bound as a *standard
space for *any* OPC UA client. Equipment-kind driver* alongside its native-protocol drivers, not a special SystemPlatform
mirror — as an OPC UA address space (live values, Part 9 alarms, HistoryRead) for *any* OPC
UA client.
- **ScadaBridge → OPC UA** (OPC UA client). ScadaBridge's DCL has an OPC UA adapter that - **ScadaBridge → OPC UA** (OPC UA client). ScadaBridge's DCL has an OPC UA adapter that
collects data and mirrors native OPC UA Alarms & Conditions. OtOpcUa is exactly such a collects data and mirrors native OPC UA Alarms & Conditions. OtOpcUa is exactly such a
server, so ScadaBridge can ingest Wonderware data **indirectly via OtOpcUa**. server, so ScadaBridge can ingest Wonderware data **indirectly via OtOpcUa**.
@@ -101,15 +105,21 @@ the gateway uses `MxGateway.*`). The common subject is **AVEVA System Platform (
- ScadaBridge has **two paths** to the same Wonderware data: (1) OPC UA → OtOpcUa → - ScadaBridge has **two paths** to the same Wonderware data: (1) OPC UA → OtOpcUa →
gateway, or (2) MxGateway adapter → gateway directly. Path 1 gives standards-based OPC UA gateway, or (2) MxGateway adapter → gateway directly. Path 1 gives standards-based OPC UA
decoupling; path 2 gives a more direct/native feed. decoupling; path 2 gives a more direct/native feed.
- **HistorianGateway is a new, independent sidecar** (no runtime coupling to the three above).
It reaches the Historian via its vendored gRPC client and the Galaxy Repository SQL DB directly,
not through `mxaccessgw`. It consumes the shared `ZB.MOM.WW.GalaxyRepository` lib
(cross-repo `ProjectReference`). Any client that needs Historian data or Galaxy browse can
target HistorianGateway independently; it is not a dependency of OtOpcUa or ScadaBridge today.
- Coupling is loose: each repo references the others only as **sibling context** (the - Coupling is loose: each repo references the others only as **sibling context** (the
`## Sister Projects` note in ScadaBridge's own `CLAUDE.md` lists `MxAccessGateway` and `## Sister Projects` note in ScadaBridge's own `CLAUDE.md` lists `MxAccessGateway` and
`OtOpcUa` with their Gitea URLs but states they are *not part of its solution*). `OtOpcUa` with their Gitea URLs but states they are *not part of its solution*).
- **The break surface is the wire contracts, not code.** Because coupling is by network - **The break surface is the wire contracts, not code.** Because coupling is by network
protocol, the things that break across repo boundaries are: the gateway's `.proto` files protocol, the things that break across repo boundaries are: the gateway's `.proto` files
(`mxaccess_gateway.proto`, `mxaccess_worker.proto`, `galaxy_repository.proto`), and the (`mxaccess_gateway.proto`, `mxaccess_worker.proto`, `galaxy_repository.proto`), the
OPC UA address-space shape OtOpcUa publishes (browse paths, node IDs, A&C alarm model). `historian_gateway.v1` proto (HistorianGateway's own contract), and the OPC UA address-space
Changes to any of these must be coordinated across the affected repos — a green build in shape OtOpcUa publishes (browse paths, node IDs, A&C alarm model). Changes to any of these
one repo does not prove the others still interoperate. must be coordinated across the affected repos — a green build in one repo does not prove the
others still interoperate.
## Component normalization ## Component normalization
@@ -126,6 +136,7 @@ each project's **code-verified current state**, and the **gaps** between. See
| Observability (metrics / traces / logs) | Built (lib `0.1.0`) | Shared `ZB.MOM.WW.Telemetry` lib + `.Serilog` | [`components/observability/`](components/observability/) | [`ZB.MOM.WW.Telemetry/`](ZB.MOM.WW.Telemetry/) | | Observability (metrics / traces / logs) | Built (lib `0.1.0`) | Shared `ZB.MOM.WW.Telemetry` lib + `.Serilog` | [`components/observability/`](components/observability/) | [`ZB.MOM.WW.Telemetry/`](ZB.MOM.WW.Telemetry/) |
| Config + validation (options / startup validation) | Adopted (lib `0.1.0`; all 3 apps, local) | Shared `ZB.MOM.WW.Configuration` lib | [`components/configuration/`](components/configuration/) | [`ZB.MOM.WW.Configuration/`](ZB.MOM.WW.Configuration/) | | Config + validation (options / startup validation) | Adopted (lib `0.1.0`; all 3 apps, local) | Shared `ZB.MOM.WW.Configuration` lib | [`components/configuration/`](components/configuration/) | [`ZB.MOM.WW.Configuration/`](ZB.MOM.WW.Configuration/) |
| Audit (event model + writer seam) | Adopted (lib `0.1.0`; all 3 apps, merged to **local default** main/master + **pushed to origin** (gitea)) | Shared `ZB.MOM.WW.Audit` lib | [`components/audit/`](components/audit/) | [`ZB.MOM.WW.Audit/`](ZB.MOM.WW.Audit/) | | Audit (event model + writer seam) | Adopted (lib `0.1.0`; all 3 apps, merged to **local default** main/master + **pushed to origin** (gitea)) | Shared `ZB.MOM.WW.Audit` lib | [`components/audit/`](components/audit/) | [`ZB.MOM.WW.Audit/`](ZB.MOM.WW.Audit/) |
| Galaxy Repository (object-hierarchy SQL browse + gRPC service) | Built (lib `0.1.0`, **published to the Gitea feed**; consumed by HistorianGateway as a feed `PackageReference`) | Shared `ZB.MOM.WW.GalaxyRepository` lib | _(design in histsdk + design doc 2026-06-23)_ | [`ZB.MOM.WW.GalaxyRepository/`](ZB.MOM.WW.GalaxyRepository/) |
The auth component is fully populated: a normalized [`spec`](components/auth/spec/SPEC.md), a The auth component is fully populated: a normalized [`spec`](components/auth/spec/SPEC.md), a
proposed [`shared-contract`](components/auth/shared-contract/ZB.MOM.WW.Auth.md), three proposed [`shared-contract`](components/auth/shared-contract/ZB.MOM.WW.Auth.md), three
@@ -261,6 +272,25 @@ migration, MSSQL-verified). Phase 3 wires `Actor` from the Auth principal at aut
Build/test from `ZB.MOM.WW.Audit/`: `dotnet test`. Consumer matrix: all three apps consume the single Build/test from `ZB.MOM.WW.Audit/`: `dotnet test`. Consumer matrix: all three apps consume the single
`ZB.MOM.WW.Audit` package (OtOpcUa, MxAccessGateway, ScadaBridge — DEEP-adopted as the canonical record). `ZB.MOM.WW.Audit` package (OtOpcUa, MxAccessGateway, ScadaBridge — DEEP-adopted as the canonical record).
The Galaxy Repository component normalizes the **Galaxy object-hierarchy SQL browse + reusable gRPC service**
that was previously embedded in `mxaccessgw`. Shared = canonical `galaxy_repository.v1` proto (wire-compatible
with `mxaccessgw`'s existing contract so OtOpcUa's `GalaxyRepositoryClient` is unaffected), the SQL browse
provider (`HierarchySql` / `AttributesSql` validated reverse-engineered queries), in-memory hierarchy cache +
snapshot + deploy-poll refresh `BackgroundService`, `GalaxyHierarchyProjector`, and `AddZbGalaxyRepository` /
`MapZbGalaxyRepository` DI extension. Left per-consumer = section path, subtree auth filtering, and any
app-specific paging defaults.
The shared library is **built and lives in this repo** at [`ZB.MOM.WW.GalaxyRepository/`](ZB.MOM.WW.GalaxyRepository/)
(.NET 10; single package `ZB.MOM.WW.GalaxyRepository`; `dotnet pack` → 1 nupkg @ 0.1.0, **published to
the Gitea NuGet feed** `gitea.dohertylan.com/api/packages/dohertj2/nuget`). The design doc is at
[`docs/plans/2026-06-23-historian-gateway-design.md`](docs/plans/2026-06-23-historian-gateway-design.md) (§10, component 1).
**Consumed by HistorianGateway** as a `PackageReference` from that Gitea feed, pinned at `0.1.0` (originally a
cross-repo `ProjectReference` to this scadaproj tree; switched to the feed package 2026-06-24).
**mxaccessgw adoption is a tracked follow-on** — once adopted, mxaccessgw's inline Galaxy browse code is replaced
by the shared lib (the `galaxy_repository.v1` wire contract is already identical, so OtOpcUa and ScadaBridge
clients are unaffected). Build/test from `ZB.MOM.WW.GalaxyRepository/`: `dotnet test`.
Consumer matrix: HistorianGateway (initial); mxaccessgw (follow-on adoption).
## Per-project primary commands ## Per-project primary commands
Run these from inside each project directory (not from `scadaproj`). Run these from inside each project directory (not from `scadaproj`).
@@ -282,9 +312,17 @@ dotnet run --project src/MxGateway.Server/MxGateway.Server.csproj
dotnet build ZB.MOM.WW.ScadaBridge.slnx dotnet build ZB.MOM.WW.ScadaBridge.slnx
bash docker/deploy.sh # rebuild + redeploy the 8-node cluster bash docker/deploy.sh # rebuild + redeploy the 8-node cluster
cd infra && docker compose up -d # local test services (SQL, OPC UA, SMTP, REST, Traefik) — LDAP is NOT here cd infra && docker compose up -d # local test services (SQL, OPC UA, SMTP, REST, Traefik) — LDAP is NOT here
# HistorianGateway (~/Desktop/HistorianGateway)
dotnet build ZB.MOM.WW.HistorianGateway.slnx
dotnet test ZB.MOM.WW.HistorianGateway.slnx # unit + golden; live integration tests skip without env vars
dotnet run --project src/ZB.MOM.WW.HistorianGateway.Server/ZB.MOM.WW.HistorianGateway.Server.csproj
# dashboard on :5220, gRPC h2c on :5221
# Live integration (need HISTORIAN_GRPC_HOST + HISTORIAN_GRPC_WRITE_SANDBOX_TAG + GALAXY_SQL_CONNSTR set)
dotnet test ZB.MOM.WW.HistorianGateway.slnx --filter "Category=LiveIntegration"
``` ```
> **Shared GLAuth (all three apps):** LDAP auth for every local dev/test stack is provided by a > **Shared GLAuth (all three apps + HistorianGateway):** LDAP auth for every local dev/test stack is provided by a
> single `zb-shared-glauth` container on the Linux fixture host **`10.100.0.35:3893`** > single `zb-shared-glauth` container on the Linux fixture host **`10.100.0.35:3893`**
> (`baseDN dc=zb,dc=local`, Transport=None). Source of truth and deploy runbook: > (`baseDN dc=zb,dc=local`, Transport=None). Source of truth and deploy runbook:
> [`scadaproj/infra/glauth/`](infra/glauth/) (`config.toml` + `docker-compose.yml` + `README.md`). > [`scadaproj/infra/glauth/`](infra/glauth/) (`config.toml` + `docker-compose.yml` + `README.md`).
@@ -0,0 +1,11 @@
<Project>
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<LangVersion>latest</LangVersion>
<Version>0.1.0</Version>
<ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
</PropertyGroup>
</Project>
@@ -0,0 +1,24 @@
<Project>
<PropertyGroup>
<ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
</PropertyGroup>
<ItemGroup>
<!-- Library -->
<PackageVersion Include="Microsoft.Data.SqlClient" Version="6.0.2" />
<PackageVersion Include="Grpc.AspNetCore" Version="2.76.0" />
<!-- Google.Protobuf and Grpc.Tools must be >= the minimums Grpc.AspNetCore 2.76.0 requires -->
<PackageVersion Include="Google.Protobuf" Version="3.31.1" />
<PackageVersion Include="Microsoft.Extensions.Hosting.Abstractions" Version="10.0.0" />
<PackageVersion Include="Microsoft.Extensions.Options.ConfigurationExtensions" Version="10.0.0" />
<PackageVersion Include="Grpc.Tools" Version="2.76.0" />
<!-- Test -->
<PackageVersion Include="xunit" Version="2.9.3" />
<PackageVersion Include="xunit.runner.visualstudio" Version="3.1.4" />
<PackageVersion Include="Microsoft.NET.Test.Sdk" Version="17.14.1" />
<PackageVersion Include="coverlet.collector" Version="6.0.4" />
</ItemGroup>
</Project>
@@ -0,0 +1,8 @@
<Solution>
<Folder Name="/src/">
<Project Path="src/ZB.MOM.WW.GalaxyRepository/ZB.MOM.WW.GalaxyRepository.csproj" />
</Folder>
<Folder Name="/tests/">
<Project Path="tests/ZB.MOM.WW.GalaxyRepository.Tests/ZB.MOM.WW.GalaxyRepository.Tests.csproj" />
</Folder>
</Solution>
@@ -0,0 +1,71 @@
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Routing;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Options;
using ZB.MOM.WW.GalaxyRepository.Grpc;
namespace ZB.MOM.WW.GalaxyRepository.DependencyInjection;
/// <summary>
/// Dependency-injection and endpoint-routing extensions that register the reusable
/// Galaxy Repository services and map the canonical gRPC service. A consuming gateway
/// calls <see cref="AddZbGalaxyRepository"/> during service registration and
/// <see cref="MapZbGalaxyRepository"/> while building its endpoint pipeline.
/// </summary>
public static class GalaxyRepositoryServiceCollectionExtensions
{
/// <summary>
/// Registers the Galaxy Repository SQL provider, shared hierarchy cache, deploy
/// notifier, on-disk snapshot store, and the background refresh service, binding
/// <see cref="GalaxyRepositoryOptions"/> from the supplied configuration section.
/// </summary>
/// <param name="services">The service collection to add registrations to.</param>
/// <param name="configuration">The application configuration root.</param>
/// <param name="sectionPath">
/// The configuration section path to bind <see cref="GalaxyRepositoryOptions"/> from
/// (for example <c>MxGateway:Galaxy</c> or <c>HistorianGateway:Galaxy</c>).
/// </param>
/// <returns>The service collection for chaining.</returns>
public static IServiceCollection AddZbGalaxyRepository(
this IServiceCollection services,
IConfiguration configuration,
string sectionPath)
{
ArgumentNullException.ThrowIfNull(services);
ArgumentNullException.ThrowIfNull(configuration);
ArgumentException.ThrowIfNullOrWhiteSpace(sectionPath);
// Bind only — this shared lib ships no validator, so a .ValidateOnStart() here
// would be a silent no-op. The consuming application owns option validation
// (e.g. the sidecar's ConfigPreflight / validated-options layer).
services
.AddOptions<GalaxyRepositoryOptions>()
.Bind(configuration.GetSection(sectionPath));
services.AddSingleton(sp =>
new GalaxyRepository(sp.GetRequiredService<IOptions<GalaxyRepositoryOptions>>().Value));
services.AddSingleton<IGalaxyRepository>(sp => sp.GetRequiredService<GalaxyRepository>());
services.AddSingleton<IGalaxyDeployNotifier, GalaxyDeployNotifier>();
services.AddSingleton<IGalaxyHierarchySnapshotStore, GalaxyHierarchySnapshotStore>();
services.AddSingleton<IGalaxyHierarchyCache, GalaxyHierarchyCache>();
services.AddHostedService<GalaxyHierarchyRefreshService>();
return services;
}
/// <summary>
/// Maps the canonical <see cref="GalaxyRepositoryGrpcService"/> onto the consuming
/// application's endpoint pipeline. Call after <see cref="AddZbGalaxyRepository"/> and
/// after gRPC has been added to the application's services.
/// </summary>
/// <param name="endpoints">The endpoint route builder to map the gRPC service onto.</param>
/// <returns>The endpoint route builder for chaining.</returns>
public static IEndpointRouteBuilder MapZbGalaxyRepository(this IEndpointRouteBuilder endpoints)
{
ArgumentNullException.ThrowIfNull(endpoints);
endpoints.MapGrpcService<GalaxyRepositoryGrpcService>();
return endpoints;
}
}
@@ -0,0 +1,41 @@
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>One row from <see cref="GalaxyRepository.GetAttributesAsync"/>.</summary>
public sealed class GalaxyAttributeRow
{
/// <summary>Gets the Galaxy object identifier.</summary>
public int GobjectId { get; init; }
/// <summary>Gets the tag name.</summary>
public string TagName { get; init; } = string.Empty;
/// <summary>Gets the attribute name.</summary>
public string AttributeName { get; init; } = string.Empty;
/// <summary>Gets the full tag reference.</summary>
public string FullTagReference { get; init; } = string.Empty;
/// <summary>Gets the MXAccess data type code.</summary>
public int MxDataType { get; init; }
/// <summary>Gets the data type name.</summary>
public string? DataTypeName { get; init; }
/// <summary>Gets a value indicating whether this is an array.</summary>
public bool IsArray { get; init; }
/// <summary>Gets the array dimension, if applicable.</summary>
public int? ArrayDimension { get; init; }
/// <summary>Gets the MXAccess attribute category code.</summary>
public int MxAttributeCategory { get; init; }
/// <summary>Gets the security classification code.</summary>
public int SecurityClassification { get; init; }
/// <summary>Gets a value indicating whether this is historized.</summary>
public bool IsHistorized { get; init; }
/// <summary>Gets a value indicating whether this is an alarm.</summary>
public bool IsAlarm { get; init; }
}
@@ -0,0 +1,19 @@
using ZB.MOM.WW.GalaxyRepository.Grpc;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// Result of one <see cref="GalaxyBrowseProjector.ProjectChildren"/> call. Holds a
/// materialized page of direct children for the requested parent, along with a
/// parallel-indexed <see cref="ChildHasChildren"/> hint and the total post-filter
/// sibling count for paging.
/// </summary>
/// <param name="Children">The page of direct children, sorted areas-first then by display name.</param>
/// <param name="ChildHasChildren">Parallel array indicating whether each child has at least one matching descendant under the same filter set.</param>
/// <param name="TotalChildCount">Total matching direct children of the parent (post-filter).</param>
/// <param name="FilterSignature">Stable signature of the filter and parent selector, used to bind page tokens.</param>
public sealed record GalaxyBrowseChildrenResult(
IReadOnlyList<GalaxyObject> Children,
IReadOnlyList<bool> ChildHasChildren,
int TotalChildCount,
string FilterSignature);
@@ -0,0 +1,281 @@
using System.Collections.Concurrent;
using System.Runtime.CompilerServices;
using System.Security.Cryptography;
using System.Text;
using Grpc.Core;
using ZB.MOM.WW.GalaxyRepository.Grpc;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// Projects one level of children of a parent object out of an immutable
/// <see cref="GalaxyHierarchyCacheEntry"/>. Pure and side-effect free. Memoizes the
/// filtered child list per cache-entry instance so repeated paging is an O(pageSize)
/// slice rather than an O(siblings) filter scan per page. The memo is keyed on the
/// immutable cache entry, so when the cache publishes a new entry the stale memo
/// becomes unreachable and is reclaimed with it.
/// </summary>
public static class GalaxyBrowseProjector
{
private static readonly ConditionalWeakTable<
GalaxyHierarchyCacheEntry,
ConcurrentDictionary<string, FilteredChildren>> FilteredChildrenCache = new();
/// <summary>Projects one page of direct children of the resolved parent.</summary>
/// <param name="entry">The Galaxy hierarchy cache entry to query.</param>
/// <param name="request">The browse-children request.</param>
/// <param name="browseSubtreeGlobs">Optional API-key browse-subtree constraints.</param>
/// <param name="offset">Zero-based offset into the filtered child list.</param>
/// <param name="pageSize">Maximum number of children to return.</param>
public static GalaxyBrowseChildrenResult ProjectChildren(
GalaxyHierarchyCacheEntry entry,
BrowseChildrenRequest request,
IReadOnlyList<string>? browseSubtreeGlobs,
int offset,
int pageSize)
{
ArgumentNullException.ThrowIfNull(entry);
ArgumentNullException.ThrowIfNull(request);
if (offset < 0)
{
throw new ArgumentOutOfRangeException(nameof(offset), offset, "Offset must be greater than or equal to zero.");
}
if (pageSize <= 0)
{
throw new ArgumentOutOfRangeException(nameof(pageSize), pageSize, "Page size must be greater than zero.");
}
int parentId = ResolveParentId(entry, request);
string filterSignature = ComputeFilterSignature(request, browseSubtreeGlobs, parentId);
FilteredChildren filtered = GetFilteredChildren(entry, request, browseSubtreeGlobs, parentId, filterSignature);
bool includeAttributes = IncludeAttributes(request);
int end = (int)Math.Min((long)offset + pageSize, filtered.Children.Count);
List<GalaxyObject> page = new(Math.Max(0, end - offset));
List<bool> hasChildren = new(Math.Max(0, end - offset));
for (int index = offset; index < end; index++)
{
page.Add(CloneObject(filtered.Children[index].Object, includeAttributes));
hasChildren.Add(filtered.HasMatchingDescendant[index]);
}
return new GalaxyBrowseChildrenResult(page, hasChildren, filtered.Children.Count, filterSignature);
}
/// <summary>
/// Resolves the request's parent oneof to a gobject id, throwing
/// <see cref="RpcException"/> with <see cref="StatusCode.NotFound"/> when the
/// parent does not exist. Public so the gRPC handler can compute the same
/// parent id (needed for the page-token signature) without reimplementing the
/// resolution rules.
/// </summary>
/// <param name="entry">The Galaxy hierarchy cache entry to query.</param>
/// <param name="request">The browse-children request.</param>
public static int ResolveParentId(GalaxyHierarchyCacheEntry entry, BrowseChildrenRequest request)
{
switch (request.ParentCase)
{
case BrowseChildrenRequest.ParentOneofCase.None:
return 0;
case BrowseChildrenRequest.ParentOneofCase.ParentGobjectId:
if (request.ParentGobjectId == 0)
{
return 0;
}
if (!entry.Index.ObjectViewsById.ContainsKey(request.ParentGobjectId))
{
throw new RpcException(new Status(StatusCode.NotFound, "BrowseChildren parent was not found."));
}
return request.ParentGobjectId;
case BrowseChildrenRequest.ParentOneofCase.ParentTagName:
{
if (!entry.Index.ObjectViewsByTagName.TryGetValue(request.ParentTagName, out GalaxyObjectView? match))
{
throw new RpcException(new Status(StatusCode.NotFound, "BrowseChildren parent was not found."));
}
return match.Object.GobjectId;
}
case BrowseChildrenRequest.ParentOneofCase.ParentContainedPath:
{
if (!entry.Index.ObjectViewsByContainedPath.TryGetValue(request.ParentContainedPath, out GalaxyObjectView? match))
{
throw new RpcException(new Status(StatusCode.NotFound, "BrowseChildren parent was not found."));
}
return match.Object.GobjectId;
}
default:
return 0;
}
}
private static FilteredChildren GetFilteredChildren(
GalaxyHierarchyCacheEntry entry,
BrowseChildrenRequest request,
IReadOnlyList<string>? browseSubtreeGlobs,
int parentId,
string filterSignature)
{
ConcurrentDictionary<string, FilteredChildren> memo =
FilteredChildrenCache.GetValue(entry, static _ => new ConcurrentDictionary<string, FilteredChildren>(StringComparer.Ordinal));
return memo.GetOrAdd(
filterSignature,
static (_, state) =>
{
IReadOnlyDictionary<int, IReadOnlyList<GalaxyObjectView>> map = state.Entry.Index.ChildrenByParent;
IReadOnlyList<GalaxyObjectView> directChildren = map.TryGetValue(state.ParentId, out IReadOnlyList<GalaxyObjectView>? list)
? list
: Array.Empty<GalaxyObjectView>();
List<GalaxyObjectView> matched = [];
List<bool> hasMatching = [];
foreach (GalaxyObjectView view in directChildren)
{
if (!MatchesBrowseSubtrees(view, state.BrowseSubtreeGlobs))
{
continue;
}
if (!MatchesFilters(view.Object, state.Request))
{
// Even if the direct child itself fails the filter, a matching
// descendant should still surface its ancestor — but only when
// there is one. Mirror the dashboard browse-tree semantics: if a
// descendant matches, include the parent with has-children true.
if (HasMatchingDescendant(view, state.Entry.Index, state.Request, state.BrowseSubtreeGlobs))
{
matched.Add(view);
hasMatching.Add(true);
}
continue;
}
matched.Add(view);
hasMatching.Add(HasMatchingDescendant(view, state.Entry.Index, state.Request, state.BrowseSubtreeGlobs));
}
return new FilteredChildren(matched, hasMatching);
},
(Entry: entry, ParentId: parentId, Request: request, BrowseSubtreeGlobs: browseSubtreeGlobs));
}
private static bool HasMatchingDescendant(
GalaxyObjectView parent,
GalaxyHierarchyIndex index,
BrowseChildrenRequest request,
IReadOnlyList<string>? browseSubtreeGlobs)
{
if (!index.ChildrenByParent.TryGetValue(parent.Object.GobjectId, out IReadOnlyList<GalaxyObjectView>? children))
{
return false;
}
// Defend against pathological cycles in Galaxy data (e.g. a corrupt A→B→A chain).
// BuildContainedPath uses the same visited-id pattern; mirror it so this walk
// terminates even when ChildrenByParent forms a cycle.
HashSet<int> visited = new() { parent.Object.GobjectId };
Stack<GalaxyObjectView> stack = new();
foreach (GalaxyObjectView child in children)
{
if (visited.Add(child.Object.GobjectId))
{
stack.Push(child);
}
}
while (stack.Count > 0)
{
GalaxyObjectView candidate = stack.Pop();
if (MatchesBrowseSubtrees(candidate, browseSubtreeGlobs)
&& MatchesFilters(candidate.Object, request))
{
return true;
}
if (index.ChildrenByParent.TryGetValue(candidate.Object.GobjectId, out IReadOnlyList<GalaxyObjectView>? grandchildren))
{
foreach (GalaxyObjectView grandchild in grandchildren)
{
if (visited.Add(grandchild.Object.GobjectId))
{
stack.Push(grandchild);
}
}
}
}
return false;
}
private static bool MatchesBrowseSubtrees(GalaxyObjectView view, IReadOnlyList<string>? browseSubtreeGlobs)
{
return browseSubtreeGlobs is null
|| browseSubtreeGlobs.Count == 0
|| browseSubtreeGlobs.Any(glob => GalaxyGlobMatcher.IsMatch(view.ContainedPath, glob));
}
private static bool MatchesFilters(GalaxyObject obj, BrowseChildrenRequest request)
{
if (request.CategoryIds.Count > 0 && !request.CategoryIds.Contains(obj.CategoryId))
{
return false;
}
foreach (string templateFilter in request.TemplateChainContains)
{
if (!obj.TemplateChain.Any(template => template.Contains(templateFilter, StringComparison.OrdinalIgnoreCase)))
{
return false;
}
}
if (!string.IsNullOrWhiteSpace(request.TagNameGlob)
&& !GalaxyGlobMatcher.IsMatch(obj.TagName, request.TagNameGlob))
{
return false;
}
if (request.AlarmBearingOnly && !obj.Attributes.Any(attribute => attribute.IsAlarm))
{
return false;
}
if (request.HistorizedOnly && !obj.Attributes.Any(attribute => attribute.IsHistorized))
{
return false;
}
return true;
}
private static bool IncludeAttributes(BrowseChildrenRequest request)
{
return !request.HasIncludeAttributes || request.IncludeAttributes;
}
private static GalaxyObject CloneObject(GalaxyObject source, bool includeAttributes)
{
GalaxyObject clone = source.Clone();
if (!includeAttributes)
{
clone.Attributes.Clear();
}
return clone;
}
/// <summary>Computes a stable filter signature for memoization purposes.</summary>
/// <param name="request">The browse-children request.</param>
/// <param name="browseSubtreeGlobs">Optional API-key browse-subtree constraints.</param>
/// <param name="parentId">Resolved parent gobject id (0 for roots).</param>
public static string ComputeFilterSignature(
BrowseChildrenRequest request,
IReadOnlyList<string>? browseSubtreeGlobs,
int parentId)
{
StringBuilder builder = new();
builder.Append("parent=").Append(parentId.ToString(System.Globalization.CultureInfo.InvariantCulture));
builder.Append("|cat=").AppendJoin(',', request.CategoryIds.Order());
builder.Append("|tpl=").AppendJoin(',', request.TemplateChainContains.Order(StringComparer.OrdinalIgnoreCase));
builder.Append("|glob=").Append(request.TagNameGlob);
builder.Append("|attrs=").Append(request.HasIncludeAttributes ? request.IncludeAttributes.ToString() : "unset");
builder.Append("|alarm=").Append(request.AlarmBearingOnly);
builder.Append("|hist=").Append(request.HistorizedOnly);
builder.Append("|browse=").AppendJoin(',', (browseSubtreeGlobs ?? Array.Empty<string>()).Order(StringComparer.OrdinalIgnoreCase));
byte[] hash = SHA256.HashData(Encoding.UTF8.GetBytes(builder.ToString()));
return Convert.ToHexString(hash, 0, 12);
}
private sealed record FilteredChildren(
IReadOnlyList<GalaxyObjectView> Children,
IReadOnlyList<bool> HasMatchingDescendant);
}
@@ -0,0 +1,18 @@
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>Freshness state of the shared Galaxy hierarchy cache entry.</summary>
public enum GalaxyCacheStatus
{
/// <summary>Cache has never completed a refresh.</summary>
Unknown = 0,
/// <summary>Cache holds data from a recent successful refresh.</summary>
Healthy = 1,
/// <summary>Cache holds data, but the most recent refresh attempt failed
/// or no successful refresh has happened within the staleness threshold.</summary>
Stale = 2,
/// <summary>Latest refresh failed and no prior data is available.</summary>
Unavailable = 3,
}
@@ -0,0 +1,19 @@
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// A single Galaxy deploy notification. Published by <see cref="GalaxyHierarchyCache"/>
/// whenever a refresh detects that <c>galaxy.time_of_last_deploy</c> has changed (or on
/// the first successful refresh). Consumed by <see cref="IGalaxyDeployNotifier"/>
/// subscribers (the streaming gRPC RPC).
/// </summary>
/// <param name="Sequence">Monotonically increasing per process start; gaps indicate dropped events.</param>
/// <param name="ObservedAt">Server wall-clock when the cache observed the deploy.</param>
/// <param name="TimeOfLastDeploy">The <c>galaxy.time_of_last_deploy</c> value, or <see langword="null"/> when the Galaxy table reports none.</param>
/// <param name="ObjectCount">Number of objects in the hierarchy at the time of the event.</param>
/// <param name="AttributeCount">Number of attributes in the hierarchy at the time of the event.</param>
public sealed record GalaxyDeployEventInfo(
long Sequence,
DateTimeOffset ObservedAt,
DateTimeOffset? TimeOfLastDeploy,
int ObjectCount,
int AttributeCount);
@@ -0,0 +1,79 @@
using System.Collections.Concurrent;
using System.Runtime.CompilerServices;
using System.Threading.Channels;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// Channel-based fan-out of Galaxy deploy events to streaming gRPC subscribers. Each
/// subscriber gets a private bounded channel so a slow client cannot back-pressure
/// other subscribers or the publisher. When a subscriber's channel is full the oldest
/// event is dropped — clients use the sequence field to detect gaps.
/// </summary>
public sealed class GalaxyDeployNotifier : IGalaxyDeployNotifier
{
private const int SubscriberQueueCapacity = 16;
private readonly ConcurrentDictionary<Guid, Channel<GalaxyDeployEventInfo>> _subscribers = new();
private GalaxyDeployEventInfo? _latest;
/// <summary>
/// The most recent deploy event, or null if none has been published.
/// </summary>
public GalaxyDeployEventInfo? Latest => Volatile.Read(ref _latest);
/// <inheritdoc />
public void Publish(GalaxyDeployEventInfo info)
{
ArgumentNullException.ThrowIfNull(info);
Volatile.Write(ref _latest, info);
foreach (Channel<GalaxyDeployEventInfo> channel in _subscribers.Values)
{
// BoundedChannelFullMode.DropOldest -> writes never wait; we only fail if the
// channel was completed by the subscriber side, which we ignore.
channel.Writer.TryWrite(info);
}
}
/// <inheritdoc />
public async IAsyncEnumerable<GalaxyDeployEventInfo> SubscribeAsync(
[EnumeratorCancellation] CancellationToken cancellationToken)
{
Guid subscriberId = Guid.NewGuid();
Channel<GalaxyDeployEventInfo> channel = Channel.CreateBounded<GalaxyDeployEventInfo>(
new BoundedChannelOptions(SubscriberQueueCapacity)
{
FullMode = BoundedChannelFullMode.DropOldest,
SingleReader = true,
SingleWriter = false,
});
_subscribers[subscriberId] = channel;
// Bootstrap: emit the latest known event so subscribers don't need to wait for
// the next deploy to know current state.
GalaxyDeployEventInfo? bootstrap = Volatile.Read(ref _latest);
if (bootstrap is not null)
{
channel.Writer.TryWrite(bootstrap);
}
try
{
while (await channel.Reader.WaitToReadAsync(cancellationToken).ConfigureAwait(false))
{
while (channel.Reader.TryRead(out GalaxyDeployEventInfo? next))
{
yield return next;
}
}
}
finally
{
_subscribers.TryRemove(subscriberId, out _);
channel.Writer.TryComplete();
}
}
}
@@ -0,0 +1,131 @@
using System.Collections.Concurrent;
using System.Text;
using System.Text.RegularExpressions;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// Anchored, case-insensitive glob matcher (<c>*</c> and <c>?</c> wildcards) used by the
/// hierarchy and browse projectors to filter object tag names and browse subtrees.
/// Compiled regexes are cached and the cache is bounded so an unbounded stream of distinct
/// client-supplied globs cannot grow memory without limit.
/// </summary>
public static class GalaxyGlobMatcher
{
/// <summary>
/// Maximum number of compiled-regex entries retained in <see cref="RegexCache"/>.
/// The cache is keyed by glob pattern and patterns flow in from two sources:
/// admin-controlled API-key constraints (naturally bounded) and the
/// client-supplied <c>DiscoverHierarchyRequest.TagNameGlob</c> (unbounded — a
/// client can iterate through generated names and create millions of distinct
/// globs over the process lifetime). Capping the cache bounds memory while
/// keeping the hot working set hit-cached.
/// </summary>
internal const int RegexCacheCapacity = 256;
/// <summary>
/// Bounded compiled-regex cache keyed by glob pattern. <c>IsMatch</c> is called
/// once per object per <c>DiscoverHierarchy</c>/<c>WatchDeployEvents</c>
/// evaluation, so the same handful of glob patterns are translated
/// repeatedly; caching avoids rebuilding and recompiling the regex on every
/// call. Beyond <see cref="RegexCacheCapacity"/> entries the oldest insertion
/// is evicted so a client cannot grow the cache without bound by submitting
/// unique patterns. Eviction is approximate (FIFO over insertion order, not
/// true LRU) because we only need the bound, not exact recency tracking.
/// </summary>
private static readonly ConcurrentDictionary<string, Regex> RegexCache = new(StringComparer.Ordinal);
/// <summary>
/// Insertion-order queue used to evict the oldest cache entry when the cache
/// exceeds <see cref="RegexCacheCapacity"/>. A separate queue keeps the
/// <see cref="RegexCache"/> reads lock-free; the lock below only guards the
/// eviction path.
/// </summary>
private static readonly ConcurrentQueue<string> InsertionOrder = new();
private static readonly object EvictionLock = new();
/// <summary>
/// Current cache size, exposed for tests asserting the cap is honoured.
/// </summary>
internal static int CurrentCacheSize => RegexCache.Count;
/// <summary>Determines whether a value matches a glob pattern (with * and ? wildcards).</summary>
/// <param name="value">The value to test against the glob pattern.</param>
/// <param name="glob">The glob pattern with * and ? wildcards.</param>
public static bool IsMatch(string value, string glob)
{
if (string.IsNullOrWhiteSpace(glob))
{
return true;
}
return GetOrCreateRegex(glob).IsMatch(value ?? string.Empty);
}
private static Regex GetOrCreateRegex(string glob)
{
if (RegexCache.TryGetValue(glob, out Regex? existing))
{
return existing;
}
Regex compiled = new(
BuildRegex(glob),
RegexOptions.CultureInvariant | RegexOptions.IgnoreCase | RegexOptions.Compiled,
TimeSpan.FromMilliseconds(100));
// GetOrAdd atomically returns whichever instance is in the cache after the
// call — either the locally-compiled regex (we won the race) or the regex
// another thread inserted (we lost). It also avoids the TryAdd-then-indexer
// pattern where the key could be evicted between the failed TryAdd and the
// indexer read, producing a KeyNotFoundException under contention near the cap.
Regex result = RegexCache.GetOrAdd(glob, compiled);
if (ReferenceEquals(result, compiled))
{
// We were the inserter — track for FIFO eviction and bound the cache.
InsertionOrder.Enqueue(glob);
EvictIfOverCapacity();
}
return result;
}
private static void EvictIfOverCapacity()
{
if (RegexCache.Count <= RegexCacheCapacity)
{
return;
}
// Serialize eviction so two threads do not race past the cap together.
lock (EvictionLock)
{
while (RegexCache.Count > RegexCacheCapacity && InsertionOrder.TryDequeue(out string? oldest))
{
RegexCache.TryRemove(oldest, out _);
}
}
}
private static string BuildRegex(string glob)
{
StringBuilder builder = new("^", glob.Length + 2);
foreach (char character in glob)
{
switch (character)
{
case '*':
builder.Append(".*");
break;
case '?':
builder.Append('.');
break;
default:
builder.Append(Regex.Escape(character.ToString()));
break;
}
}
builder.Append('$');
return builder.ToString();
}
}
@@ -0,0 +1,365 @@
using Microsoft.Extensions.Logging;
using ZB.MOM.WW.GalaxyRepository.Grpc;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// Server-side cache of Galaxy Repository browse data. All gRPC clients share the same
/// entry — the materialized object list is produced once per refresh and reused across
/// requests. Refreshes are deploy-time gated: every tick queries
/// <c>galaxy.time_of_last_deploy</c> (cheap), and the heavy hierarchy + attributes rowsets
/// are pulled only when that timestamp has advanced.
/// Each successful heavy refresh is persisted to disk through
/// <see cref="IGalaxyHierarchySnapshotStore"/>; the first refresh restores that
/// snapshot (as <see cref="GalaxyCacheStatus.Stale"/>) so clients can browse
/// last-known data when the Galaxy database is unreachable on a cold start.
/// </summary>
public sealed class GalaxyHierarchyCache : IGalaxyHierarchyCache, IDisposable
{
private static readonly TimeSpan StaleThreshold = TimeSpan.FromMinutes(5);
private readonly IGalaxyRepository _repository;
private readonly IGalaxyDeployNotifier _notifier;
private readonly IGalaxyHierarchySnapshotStore? _snapshotStore;
private readonly TimeProvider _timeProvider;
private readonly ILogger<GalaxyHierarchyCache>? _logger;
private readonly TaskCompletionSource _firstLoad = new(TaskCreationOptions.RunContinuationsAsynchronously);
private readonly SemaphoreSlim _refreshGate = new(1, 1);
private GalaxyHierarchyCacheEntry _current = GalaxyHierarchyCacheEntry.Empty;
private bool _restoreAttempted;
/// <summary>Initializes a new instance of the <see cref="GalaxyHierarchyCache"/> class.</summary>
/// <param name="repository">Galaxy Repository client for SQL queries.</param>
/// <param name="notifier">Galaxy deploy event notifier.</param>
/// <param name="timeProvider">Provider for current time; defaults to system time.</param>
/// <param name="logger">Optional logger for diagnostic output.</param>
/// <param name="snapshotStore">
/// Optional on-disk snapshot store. When supplied, the cache persists each
/// successful refresh and restores the last snapshot on first load.
/// </param>
public GalaxyHierarchyCache(
IGalaxyRepository repository,
IGalaxyDeployNotifier notifier,
TimeProvider? timeProvider = null,
ILogger<GalaxyHierarchyCache>? logger = null,
IGalaxyHierarchySnapshotStore? snapshotStore = null)
{
_repository = repository;
_notifier = notifier;
_timeProvider = timeProvider ?? TimeProvider.System;
_logger = logger;
_snapshotStore = snapshotStore;
}
/// <summary>Gets the current Galaxy hierarchy cache entry with projected status.</summary>
public GalaxyHierarchyCacheEntry Current
{
get
{
GalaxyHierarchyCacheEntry snapshot = Volatile.Read(ref _current);
GalaxyCacheStatus projected = ProjectStatus(snapshot);
return projected == snapshot.Status
? snapshot
: snapshot with { Status = projected };
}
}
/// <summary>Refreshes the Galaxy hierarchy cache if the deploy time has advanced.</summary>
/// <param name="cancellationToken">Token to cancel the asynchronous operation.</param>
/// <returns>Asynchronous task representing the refresh operation.</returns>
public async Task RefreshAsync(CancellationToken cancellationToken)
{
await _refreshGate.WaitAsync(cancellationToken).ConfigureAwait(false);
try
{
await RefreshCoreAsync(cancellationToken).ConfigureAwait(false);
}
finally
{
_refreshGate.Release();
}
}
/// <summary>Waits for the Galaxy hierarchy cache to complete its first load.</summary>
/// <param name="cancellationToken">Token to cancel the asynchronous operation.</param>
/// <returns>Asynchronous task representing the wait operation.</returns>
public Task WaitForFirstLoadAsync(CancellationToken cancellationToken)
{
return _firstLoad.Task.WaitAsync(cancellationToken);
}
/// <summary>
/// Disposes the refresh gate. As a DI singleton the cache is disposed once at host
/// shutdown, after the refresh <see cref="GalaxyHierarchyRefreshService"/> has stopped,
/// so no in-flight refresh can be holding the gate.
/// </summary>
public void Dispose()
{
_refreshGate.Dispose();
}
private async Task RefreshCoreAsync(CancellationToken cancellationToken)
{
// First refresh only: seed the cache from the on-disk snapshot before
// querying SQL, so a cold start with an unreachable Galaxy database can
// still serve last-known browse data. Runs under the refresh gate.
if (!_restoreAttempted)
{
_restoreAttempted = true;
await TryRestoreFromDiskAsync(cancellationToken).ConfigureAwait(false);
}
GalaxyHierarchyCacheEntry previous = Volatile.Read(ref _current);
DateTimeOffset queriedAt = _timeProvider.GetUtcNow();
try
{
DateTime? deployRaw = await _repository.GetLastDeployTimeAsync(cancellationToken).ConfigureAwait(false);
DateTimeOffset? deployTime = deployRaw.HasValue
? new DateTimeOffset(DateTime.SpecifyKind(deployRaw.Value, DateTimeKind.Utc))
: null;
bool hasPriorData = previous.HasData;
bool deployChanged = !hasPriorData || deployTime != previous.LastDeployTime;
if (!deployChanged)
{
// No deploy change — skip heavy queries; just bump LastSuccessAt.
GalaxyHierarchyCacheEntry refreshed = previous with
{
Status = GalaxyCacheStatus.Healthy,
LastQueriedAt = queriedAt,
LastSuccessAt = queriedAt,
LastError = null,
};
Volatile.Write(ref _current, refreshed);
_firstLoad.TrySetResult();
return;
}
Task<List<GalaxyHierarchyRow>> hierarchyTask = _repository.GetHierarchyAsync(cancellationToken);
Task<List<GalaxyAttributeRow>> attributesTask = _repository.GetAttributesAsync(cancellationToken);
await Task.WhenAll(hierarchyTask, attributesTask).ConfigureAwait(false);
List<GalaxyHierarchyRow> hierarchy = hierarchyTask.Result;
List<GalaxyAttributeRow> attributes = attributesTask.Result;
long nextSequence = previous.Sequence + 1;
GalaxyHierarchyCacheEntry next = BuildEntry(
status: GalaxyCacheStatus.Healthy,
sequence: nextSequence,
lastQueriedAt: queriedAt,
lastSuccessAt: queriedAt,
lastDeployTime: deployTime,
lastError: null,
hierarchy: hierarchy,
attributes: attributes);
Volatile.Write(ref _current, next);
_firstLoad.TrySetResult();
_notifier.Publish(new GalaxyDeployEventInfo(
Sequence: nextSequence,
ObservedAt: queriedAt,
TimeOfLastDeploy: deployTime,
ObjectCount: hierarchy.Count,
AttributeCount: attributes.Count));
await PersistSnapshotAsync(deployTime, queriedAt, hierarchy, attributes, cancellationToken).ConfigureAwait(false);
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
{
throw;
}
catch (Exception exception)
{
// Catch every non-cancellation failure — not just SqlException /
// InvalidOperationException. A TimeoutException or Win32Exception
// from connection establishment, or another DbException subtype,
// must still degrade gracefully to Stale/Unavailable and complete
// _firstLoad rather than escape and fault the refresh BackgroundService.
_logger?.LogWarning(exception, "Galaxy hierarchy cache refresh failed.");
GalaxyHierarchyCacheEntry failed = previous with
{
Status = previous.HasData ? GalaxyCacheStatus.Stale : GalaxyCacheStatus.Unavailable,
LastQueriedAt = queriedAt,
LastError = exception.Message,
};
Volatile.Write(ref _current, failed);
_firstLoad.TrySetResult();
}
}
/// <summary>
/// Materializes a complete <see cref="GalaxyHierarchyCacheEntry"/> from raw
/// hierarchy and attribute rowsets. Shared by the live refresh path and the
/// on-disk restore path so both produce an identical object list and index.
/// </summary>
private static GalaxyHierarchyCacheEntry BuildEntry(
GalaxyCacheStatus status,
long sequence,
DateTimeOffset? lastQueriedAt,
DateTimeOffset? lastSuccessAt,
DateTimeOffset? lastDeployTime,
string? lastError,
IReadOnlyList<GalaxyHierarchyRow> hierarchy,
IReadOnlyList<GalaxyAttributeRow> attributes)
{
IReadOnlyList<GalaxyObject> objects = BuildObjects(hierarchy, attributes);
GalaxyHierarchyIndex index = GalaxyHierarchyIndex.Build(objects);
int areaCount = hierarchy.Count(row => row.IsArea);
int historized = attributes.Count(row => row.IsHistorized);
int alarms = attributes.Count(row => row.IsAlarm);
return new GalaxyHierarchyCacheEntry(
Status: status,
Sequence: sequence,
LastQueriedAt: lastQueriedAt,
LastSuccessAt: lastSuccessAt,
LastDeployTime: lastDeployTime,
LastError: lastError,
Objects: objects,
Index: index,
ObjectCount: hierarchy.Count,
AreaCount: areaCount,
AttributeCount: attributes.Count,
HistorizedAttributeCount: historized,
AlarmAttributeCount: alarms);
}
/// <summary>
/// Seeds the cache from the on-disk snapshot when no live data has loaded yet.
/// The restored entry is marked <see cref="GalaxyCacheStatus.Stale"/> — it is
/// last-known data, not live. A later refresh that observes the same deploy
/// time promotes it to healthy; one that observes a newer deploy replaces it.
/// </summary>
private async Task TryRestoreFromDiskAsync(CancellationToken cancellationToken)
{
if (_snapshotStore is null)
{
return;
}
if (Volatile.Read(ref _current).HasData)
{
return;
}
GalaxyHierarchySnapshot? snapshot;
try
{
snapshot = await _snapshotStore.TryLoadAsync(cancellationToken).ConfigureAwait(false);
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
{
throw;
}
catch (Exception exception)
{
_logger?.LogWarning(exception, "Failed to restore the Galaxy hierarchy from the on-disk snapshot.");
return;
}
if (snapshot is null)
{
return;
}
long sequence = Volatile.Read(ref _current).Sequence + 1;
GalaxyHierarchyCacheEntry restored = BuildEntry(
status: GalaxyCacheStatus.Stale,
sequence: sequence,
lastQueriedAt: snapshot.SavedAt,
lastSuccessAt: snapshot.SavedAt,
lastDeployTime: snapshot.LastDeployTime,
lastError: null,
hierarchy: snapshot.Hierarchy,
attributes: snapshot.Attributes);
Volatile.Write(ref _current, restored);
// Restored data is a valid completed first load: unblock callers waiting on
// the bootstrap gate immediately, rather than making them wait out the full
// wait budget for a live query that — when the database is unreachable, the
// scenario this restore exists for — may not return for seconds.
_firstLoad.TrySetResult();
_notifier.Publish(new GalaxyDeployEventInfo(
Sequence: sequence,
ObservedAt: _timeProvider.GetUtcNow(),
TimeOfLastDeploy: snapshot.LastDeployTime,
ObjectCount: snapshot.Hierarchy.Count,
AttributeCount: snapshot.Attributes.Count));
_logger?.LogInformation(
"Restored Galaxy hierarchy from on-disk snapshot saved {SavedAt:o}: {ObjectCount} objects, {AttributeCount} attributes (status Stale until the Galaxy database confirms).",
snapshot.SavedAt,
snapshot.Hierarchy.Count,
snapshot.Attributes.Count);
}
/// <summary>
/// Persists a successful refresh to disk. Persistence failures are logged and
/// swallowed — a cache that cannot write its backup is still fully usable.
/// </summary>
private async Task PersistSnapshotAsync(
DateTimeOffset? deployTime,
DateTimeOffset savedAt,
IReadOnlyList<GalaxyHierarchyRow> hierarchy,
IReadOnlyList<GalaxyAttributeRow> attributes,
CancellationToken cancellationToken)
{
if (_snapshotStore is null)
{
return;
}
try
{
await _snapshotStore.SaveAsync(
new GalaxyHierarchySnapshot(deployTime, savedAt, hierarchy, attributes),
cancellationToken).ConfigureAwait(false);
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
{
// The refresh was cancelled (service shutdown) before the write finished.
// That is not a persistence failure — do not log it as a warning.
}
catch (Exception exception)
{
_logger?.LogWarning(exception, "Failed to persist the Galaxy hierarchy snapshot to disk.");
}
}
private static IReadOnlyList<GalaxyObject> BuildObjects(
IReadOnlyList<GalaxyHierarchyRow> hierarchy,
IReadOnlyList<GalaxyAttributeRow> attributes)
{
Dictionary<int, List<GalaxyAttributeRow>> attributesByGobjectId = attributes
.GroupBy(a => a.GobjectId)
.ToDictionary(g => g.Key, g => g.ToList());
List<GalaxyObject> objects = new(hierarchy.Count);
foreach (GalaxyHierarchyRow row in hierarchy)
{
objects.Add(GalaxyProtoMapper.MapObject(row, attributesByGobjectId));
}
return objects;
}
private GalaxyCacheStatus ProjectStatus(GalaxyHierarchyCacheEntry snapshot)
{
if (snapshot.Status is GalaxyCacheStatus.Unknown or GalaxyCacheStatus.Unavailable)
{
return snapshot.Status;
}
if (snapshot.LastSuccessAt is { } success
&& _timeProvider.GetUtcNow() - success > StaleThreshold)
{
return GalaxyCacheStatus.Stale;
}
return snapshot.Status;
}
}
@@ -0,0 +1,56 @@
using ZB.MOM.WW.GalaxyRepository.Grpc;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// Immutable snapshot of the Galaxy Repository browse data held by
/// <see cref="GalaxyHierarchyCache"/>. Multiple gRPC clients share the same
/// materialized object list and precomputed hierarchy index.
/// </summary>
/// <param name="Status">The cache freshness state at the time the entry was produced.</param>
/// <param name="Sequence">Monotonically increasing per process start; bumped on each heavy refresh.</param>
/// <param name="LastQueriedAt">UTC wall-clock of the most recent refresh attempt.</param>
/// <param name="LastSuccessAt">UTC wall-clock of the most recent successful refresh.</param>
/// <param name="LastDeployTime">The <c>galaxy.time_of_last_deploy</c> the data was pulled at.</param>
/// <param name="LastError">The most recent refresh error message, or <see langword="null"/>.</param>
/// <param name="Objects">The materialized Galaxy object list.</param>
/// <param name="Index">Precomputed lookup structures over <paramref name="Objects"/>.</param>
/// <param name="ObjectCount">Number of objects in the hierarchy.</param>
/// <param name="AreaCount">Number of area objects in the hierarchy.</param>
/// <param name="AttributeCount">Number of attributes across all objects.</param>
/// <param name="HistorizedAttributeCount">Number of historized attributes.</param>
/// <param name="AlarmAttributeCount">Number of alarm-bearing attributes.</param>
public sealed record GalaxyHierarchyCacheEntry(
GalaxyCacheStatus Status,
long Sequence,
DateTimeOffset? LastQueriedAt,
DateTimeOffset? LastSuccessAt,
DateTimeOffset? LastDeployTime,
string? LastError,
IReadOnlyList<GalaxyObject> Objects,
GalaxyHierarchyIndex Index,
int ObjectCount,
int AreaCount,
int AttributeCount,
int HistorizedAttributeCount,
int AlarmAttributeCount)
{
/// <summary>Gets an empty Galaxy hierarchy cache entry.</summary>
public static GalaxyHierarchyCacheEntry Empty { get; } = new(
Status: GalaxyCacheStatus.Unknown,
Sequence: 0,
LastQueriedAt: null,
LastSuccessAt: null,
LastDeployTime: null,
LastError: null,
Objects: Array.Empty<GalaxyObject>(),
Index: GalaxyHierarchyIndex.Empty,
ObjectCount: 0,
AreaCount: 0,
AttributeCount: 0,
HistorizedAttributeCount: 0,
AlarmAttributeCount: 0);
/// <summary>Gets a value indicating whether the cache entry contains usable data.</summary>
public bool HasData => Status is GalaxyCacheStatus.Healthy or GalaxyCacheStatus.Stale;
}
@@ -0,0 +1,206 @@
using ZB.MOM.WW.GalaxyRepository.Grpc;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// Precomputed lookup structures over a materialized Galaxy object list. Built once per
/// cache entry so browse/discover handlers can resolve roots/parents by id, tag name, or
/// contained path in O(1), enumerate direct children, and resolve tag addresses to objects
/// or attributes without rescanning the full object list.
/// </summary>
public sealed class GalaxyHierarchyIndex
{
private GalaxyHierarchyIndex(
IReadOnlyList<GalaxyObjectView> objectViews,
IReadOnlyDictionary<int, GalaxyObjectView> objectViewsById,
IReadOnlyDictionary<string, GalaxyTagLookup> tagsByAddress,
IReadOnlyDictionary<int, IReadOnlyList<GalaxyObjectView>> childrenByParent,
IReadOnlyDictionary<string, GalaxyObjectView> objectViewsByTagName,
IReadOnlyDictionary<string, GalaxyObjectView> objectViewsByContainedPath)
{
ObjectViews = objectViews;
ObjectViewsById = objectViewsById;
TagsByAddress = tagsByAddress;
ChildrenByParent = childrenByParent;
ObjectViewsByTagName = objectViewsByTagName;
ObjectViewsByContainedPath = objectViewsByContainedPath;
}
/// <summary>Gets an empty Galaxy hierarchy index.</summary>
public static GalaxyHierarchyIndex Empty { get; } = new(
Array.Empty<GalaxyObjectView>(),
new Dictionary<int, GalaxyObjectView>(),
new Dictionary<string, GalaxyTagLookup>(StringComparer.OrdinalIgnoreCase),
new Dictionary<int, IReadOnlyList<GalaxyObjectView>>(),
new Dictionary<string, GalaxyObjectView>(StringComparer.OrdinalIgnoreCase),
new Dictionary<string, GalaxyObjectView>(StringComparer.OrdinalIgnoreCase));
/// <summary>Gets the object views.</summary>
public IReadOnlyList<GalaxyObjectView> ObjectViews { get; }
/// <summary>Gets the object views indexed by gobject id.</summary>
public IReadOnlyDictionary<int, GalaxyObjectView> ObjectViewsById { get; }
/// <summary>Gets tags indexed by address.</summary>
public IReadOnlyDictionary<string, GalaxyTagLookup> TagsByAddress { get; }
/// <summary>Gets direct children grouped by parent gobject id. Root objects (no parent, or self-parented) live under key 0. Each list is sorted areas-first, then by display name (OrdinalIgnoreCase).</summary>
public IReadOnlyDictionary<int, IReadOnlyList<GalaxyObjectView>> ChildrenByParent { get; }
/// <summary>Gets object views indexed by <see cref="GalaxyObject.TagName"/> (OrdinalIgnoreCase). Lets browse/discover handlers resolve parents/roots by tag name in O(1) instead of scanning <see cref="ObjectViews"/>.</summary>
public IReadOnlyDictionary<string, GalaxyObjectView> ObjectViewsByTagName { get; }
/// <summary>Gets object views indexed by contained path (OrdinalIgnoreCase). Lets browse/discover handlers resolve parents/roots by path in O(1) instead of scanning <see cref="ObjectViews"/>.</summary>
public IReadOnlyDictionary<string, GalaxyObjectView> ObjectViewsByContainedPath { get; }
/// <summary>Builds a Galaxy hierarchy index from the given objects.</summary>
/// <param name="objects">The Galaxy objects to index.</param>
/// <returns>A new Galaxy hierarchy index.</returns>
public static GalaxyHierarchyIndex Build(IReadOnlyList<GalaxyObject> objects)
{
if (objects.Count == 0)
{
return Empty;
}
Dictionary<int, GalaxyObject> objectsById = new();
foreach (GalaxyObject obj in objects)
{
objectsById.TryAdd(obj.GobjectId, obj);
}
List<GalaxyObjectView> views = new(objects.Count);
Dictionary<int, GalaxyObjectView> viewsById = new();
Dictionary<string, GalaxyTagLookup> tagsByAddress = new(StringComparer.OrdinalIgnoreCase);
Dictionary<string, GalaxyObjectView> viewsByTagName = new(StringComparer.OrdinalIgnoreCase);
Dictionary<string, GalaxyObjectView> viewsByContainedPath = new(StringComparer.OrdinalIgnoreCase);
foreach (GalaxyObject obj in objects)
{
string path = BuildContainedPath(obj, objectsById);
int depth = string.IsNullOrWhiteSpace(path) ? 0 : path.Count(character => character == '/');
GalaxyObjectView view = new(obj, path, depth);
views.Add(view);
viewsById.TryAdd(obj.GobjectId, view);
if (!string.IsNullOrWhiteSpace(obj.TagName))
{
tagsByAddress.TryAdd(obj.TagName, new GalaxyTagLookup(obj, Attribute: null, path));
viewsByTagName.TryAdd(obj.TagName, view);
}
if (!string.IsNullOrWhiteSpace(path))
{
viewsByContainedPath.TryAdd(path, view);
}
foreach (GalaxyAttribute attribute in obj.Attributes)
{
if (!string.IsNullOrWhiteSpace(attribute.FullTagReference))
{
tagsByAddress.TryAdd(attribute.FullTagReference, new GalaxyTagLookup(obj, attribute, path));
}
}
}
Dictionary<int, List<GalaxyObjectView>> childrenByParent = new();
foreach (GalaxyObjectView view in views)
{
int parentKey = view.Object.ParentGobjectId;
// Treat self-parented (corrupt) rows as roots.
if (parentKey == view.Object.GobjectId)
{
parentKey = 0;
}
// Re-root orphans whose parent object is absent from the set (e.g. a deleted or
// never-loaded container area). Otherwise they bucket under a phantom parent id
// that is never reached from the root, so they vanish from browse entirely.
else if (parentKey != 0 && !objectsById.ContainsKey(parentKey))
{
parentKey = 0;
}
if (!childrenByParent.TryGetValue(parentKey, out List<GalaxyObjectView>? bucket))
{
bucket = [];
childrenByParent[parentKey] = bucket;
}
bucket.Add(view);
}
foreach (List<GalaxyObjectView> bucket in childrenByParent.Values)
{
bucket.Sort(CompareByAreaThenDisplayName);
}
Dictionary<int, IReadOnlyList<GalaxyObjectView>> readOnlyChildren = new(childrenByParent.Count);
foreach (KeyValuePair<int, List<GalaxyObjectView>> kvp in childrenByParent)
{
readOnlyChildren[kvp.Key] = kvp.Value;
}
return new GalaxyHierarchyIndex(
views,
viewsById,
tagsByAddress,
readOnlyChildren,
viewsByTagName,
viewsByContainedPath);
}
private static string BuildContainedPath(
GalaxyObject obj,
IReadOnlyDictionary<int, GalaxyObject> objectsById)
{
Stack<string> names = new();
HashSet<int> seen = [];
GalaxyObject? current = obj;
while (current is not null && seen.Add(current.GobjectId))
{
names.Push(ResolvePathSegment(current));
current = current.ParentGobjectId != 0
&& objectsById.TryGetValue(current.ParentGobjectId, out GalaxyObject? parent)
? parent
: null;
}
return string.Join('/', names.Where(name => !string.IsNullOrWhiteSpace(name)));
}
private static string ResolvePathSegment(GalaxyObject obj)
{
if (!string.IsNullOrWhiteSpace(obj.ContainedName))
{
return obj.ContainedName;
}
if (!string.IsNullOrWhiteSpace(obj.BrowseName))
{
return obj.BrowseName;
}
return obj.TagName;
}
private static int CompareByAreaThenDisplayName(GalaxyObjectView left, GalaxyObjectView right)
{
if (left.Object.IsArea != right.Object.IsArea)
{
return left.Object.IsArea ? -1 : 1;
}
return string.Compare(DisplayNameOf(left), DisplayNameOf(right), StringComparison.OrdinalIgnoreCase);
}
private static string DisplayNameOf(GalaxyObjectView view)
{
GalaxyObject obj = view.Object;
if (!string.IsNullOrWhiteSpace(obj.BrowseName))
{
return obj.BrowseName;
}
if (!string.IsNullOrWhiteSpace(obj.ContainedName))
{
return obj.ContainedName;
}
return obj.TagName;
}
}
@@ -0,0 +1,317 @@
using System.Collections.Concurrent;
using System.Runtime.CompilerServices;
using System.Security.Cryptography;
using System.Text;
using Grpc.Core;
using ZB.MOM.WW.GalaxyRepository.Grpc;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// Projects a <c>DiscoverHierarchy</c> request against an immutable
/// <see cref="GalaxyHierarchyCacheEntry"/>: applies the root/depth/category/template/glob
/// filters, pages the result, and memoizes the filtered list per cache-entry instance so
/// paging is O(pageSize) rather than O(total) per page. Pure and side-effect free.
/// </summary>
public static class GalaxyHierarchyProjector
{
/// <summary>
/// Per-cache-entry memo of filtered, ordered <see cref="GalaxyObjectView"/> lists
/// keyed by filter signature. Without it, paging through a large hierarchy
/// re-applies every filter and re-scans the full <see cref="GalaxyHierarchyIndex.ObjectViews"/>
/// collection on every page — O(total) per page, O(total²/pageSize) end-to-end.
/// With it, the first page builds the filtered list and each subsequent page is an
/// O(pageSize) slice. The table is keyed on the immutable cache-entry instance, so
/// when the cache publishes a new entry the stale memo becomes unreachable and is
/// reclaimed with it — no explicit invalidation needed.
/// </summary>
private static readonly ConditionalWeakTable<GalaxyHierarchyCacheEntry, ConcurrentDictionary<string, IReadOnlyList<GalaxyObjectView>>> FilteredViewCache = new();
/// <summary>Projects a discovery request against a cache entry and returns all matching objects.</summary>
/// <param name="entry">The Galaxy hierarchy cache entry.</param>
/// <param name="request">The discovery hierarchy request.</param>
/// <param name="browseSubtreeGlobs">Optional glob patterns to filter browse subtrees.</param>
public static GalaxyHierarchyQueryResult Project(
GalaxyHierarchyCacheEntry entry,
DiscoverHierarchyRequest request,
IReadOnlyList<string>? browseSubtreeGlobs = null)
{
return Project(
entry,
request,
browseSubtreeGlobs,
offset: 0,
pageSize: int.MaxValue);
}
/// <summary>Projects a discovery request with paging against a cache entry and returns a page of matching objects.</summary>
/// <param name="entry">The Galaxy hierarchy cache entry.</param>
/// <param name="request">The discovery hierarchy request.</param>
/// <param name="browseSubtreeGlobs">Optional glob patterns to filter browse subtrees.</param>
/// <param name="offset">The zero-based offset into the result set.</param>
/// <param name="pageSize">The maximum number of results to return.</param>
public static GalaxyHierarchyQueryResult Project(
GalaxyHierarchyCacheEntry entry,
DiscoverHierarchyRequest request,
IReadOnlyList<string>? browseSubtreeGlobs,
int offset,
int pageSize)
{
ArgumentNullException.ThrowIfNull(entry);
ArgumentNullException.ThrowIfNull(request);
if (offset < 0)
{
throw new ArgumentOutOfRangeException(nameof(offset), offset, "Offset must be greater than or equal to zero.");
}
if (pageSize <= 0)
{
throw new ArgumentOutOfRangeException(nameof(pageSize), pageSize, "Page size must be greater than zero.");
}
int? maxDepth = request.MaxDepth;
if (maxDepth < 0)
{
throw new RpcException(new Status(
StatusCode.InvalidArgument,
"DiscoverHierarchy max_depth must be greater than or equal to zero when provided."));
}
string filterSignature = ComputeFilterSignature(request, browseSubtreeGlobs);
IReadOnlyList<GalaxyObjectView> matchedViews = GetFilteredViews(
entry,
request,
browseSubtreeGlobs,
maxDepth,
filterSignature);
bool includeAttributes = IncludeAttributes(request);
List<GalaxyObject> page = new(Math.Min(pageSize, Math.Max(0, matchedViews.Count - offset)));
int end = (int)Math.Min((long)offset + pageSize, matchedViews.Count);
for (int index = offset; index < end; index++)
{
page.Add(CloneObject(matchedViews[index].Object, includeAttributes));
}
return new GalaxyHierarchyQueryResult(
page,
matchedViews.Count,
filterSignature);
}
private static IReadOnlyList<GalaxyObjectView> GetFilteredViews(
GalaxyHierarchyCacheEntry entry,
DiscoverHierarchyRequest request,
IReadOnlyList<string>? browseSubtreeGlobs,
int? maxDepth,
string filterSignature)
{
// ResolveRoot can throw RpcException(NotFound); run it before consulting the
// memo so a bad root surfaces consistently regardless of cache state.
IReadOnlyList<GalaxyObjectView> views = entry.Index.ObjectViews;
GalaxyObjectView? root = ResolveRoot(request, entry.Index);
ConcurrentDictionary<string, IReadOnlyList<GalaxyObjectView>> memo =
FilteredViewCache.GetValue(entry, static _ => new ConcurrentDictionary<string, IReadOnlyList<GalaxyObjectView>>(StringComparer.Ordinal));
return memo.GetOrAdd(
filterSignature,
static (_, state) =>
{
List<GalaxyObjectView> matched = [];
foreach (GalaxyObjectView view in state.Views)
{
if (MatchesRoot(view, state.Root, state.MaxDepth)
&& MatchesBrowseSubtrees(view, state.BrowseSubtreeGlobs)
&& MatchesFilters(view.Object, state.Request))
{
matched.Add(view);
}
}
return matched;
},
(Views: views, Root: root, MaxDepth: maxDepth, BrowseSubtreeGlobs: browseSubtreeGlobs, Request: request));
}
/// <summary>Finds an object in the hierarchy by its tag address.</summary>
/// <param name="entry">The Galaxy hierarchy cache entry.</param>
/// <param name="tagAddress">The tag address to search for.</param>
public static GalaxyObject? FindObjectForTag(
GalaxyHierarchyCacheEntry entry,
string tagAddress)
{
if (string.IsNullOrWhiteSpace(tagAddress))
{
return null;
}
return entry.Index.TagsByAddress.TryGetValue(tagAddress, out GalaxyTagLookup? lookup)
? lookup.Object
: null;
}
/// <summary>Finds an attribute in the hierarchy by its tag address.</summary>
/// <param name="entry">The Galaxy hierarchy cache entry.</param>
/// <param name="tagAddress">The tag address to search for.</param>
public static GalaxyAttribute? FindAttributeForTag(
GalaxyHierarchyCacheEntry entry,
string tagAddress)
{
if (string.IsNullOrWhiteSpace(tagAddress))
{
return null;
}
return entry.Index.TagsByAddress.TryGetValue(tagAddress, out GalaxyTagLookup? lookup)
? lookup.Attribute
: null;
}
/// <summary>Gets the contained path for an object by its gobject ID.</summary>
/// <param name="entry">The Galaxy hierarchy cache entry.</param>
/// <param name="gobjectId">The Galaxy object ID.</param>
public static string GetContainedPath(
GalaxyHierarchyCacheEntry entry,
int gobjectId)
{
return entry.Index.ObjectViewsById.TryGetValue(gobjectId, out GalaxyObjectView? view)
? view.ContainedPath
: string.Empty;
}
private static GalaxyObjectView? ResolveRoot(
DiscoverHierarchyRequest request,
GalaxyHierarchyIndex index)
{
GalaxyObjectView? root = request.RootCase switch
{
DiscoverHierarchyRequest.RootOneofCase.None => null,
DiscoverHierarchyRequest.RootOneofCase.RootGobjectId =>
index.ObjectViewsById.TryGetValue(request.RootGobjectId, out GalaxyObjectView? byId) ? byId : null,
DiscoverHierarchyRequest.RootOneofCase.RootTagName =>
index.ObjectViewsByTagName.TryGetValue(request.RootTagName, out GalaxyObjectView? byTag) ? byTag : null,
DiscoverHierarchyRequest.RootOneofCase.RootContainedPath =>
index.ObjectViewsByContainedPath.TryGetValue(request.RootContainedPath, out GalaxyObjectView? byPath) ? byPath : null,
_ => null,
};
if (request.RootCase != DiscoverHierarchyRequest.RootOneofCase.None && root is null)
{
throw new RpcException(new Status(StatusCode.NotFound, "DiscoverHierarchy root was not found."));
}
return root;
}
private static bool MatchesRoot(
GalaxyObjectView view,
GalaxyObjectView? root,
int? maxDepth)
{
if (root is null)
{
return true;
}
bool isRoot = view.Object.GobjectId == root.Object.GobjectId;
bool isDescendant = view.ContainedPath.StartsWith(root.ContainedPath + "/", StringComparison.OrdinalIgnoreCase);
if (!isRoot && !isDescendant)
{
return false;
}
return maxDepth is null || view.Depth - root.Depth <= maxDepth.Value;
}
private static bool MatchesBrowseSubtrees(
GalaxyObjectView view,
IReadOnlyList<string>? browseSubtreeGlobs)
{
return browseSubtreeGlobs is null
|| browseSubtreeGlobs.Count == 0
|| browseSubtreeGlobs.Any(glob => GalaxyGlobMatcher.IsMatch(view.ContainedPath, glob));
}
private static bool MatchesFilters(
GalaxyObject obj,
DiscoverHierarchyRequest request)
{
if (request.CategoryIds.Count > 0 && !request.CategoryIds.Contains(obj.CategoryId))
{
return false;
}
foreach (string templateFilter in request.TemplateChainContains)
{
if (!obj.TemplateChain.Any(template => template.Contains(templateFilter, StringComparison.OrdinalIgnoreCase)))
{
return false;
}
}
if (!string.IsNullOrWhiteSpace(request.TagNameGlob)
&& !GalaxyGlobMatcher.IsMatch(obj.TagName, request.TagNameGlob))
{
return false;
}
if (request.AlarmBearingOnly && !obj.Attributes.Any(attribute => attribute.IsAlarm))
{
return false;
}
if (request.HistorizedOnly && !obj.Attributes.Any(attribute => attribute.IsHistorized))
{
return false;
}
return true;
}
private static bool IncludeAttributes(DiscoverHierarchyRequest request)
{
return !request.HasIncludeAttributes || request.IncludeAttributes;
}
private static GalaxyObject CloneObject(GalaxyObject source, bool includeAttributes)
{
GalaxyObject clone = source.Clone();
if (!includeAttributes)
{
clone.Attributes.Clear();
}
return clone;
}
/// <summary>Computes a stable filter signature for memoization purposes.</summary>
/// <param name="request">The discovery hierarchy request.</param>
/// <param name="browseSubtreeGlobs">Optional glob patterns to filter browse subtrees.</param>
public static string ComputeFilterSignature(
DiscoverHierarchyRequest request,
IReadOnlyList<string>? browseSubtreeGlobs)
{
StringBuilder builder = new();
builder.Append("root=").Append(request.RootCase).Append('|');
builder.Append(request.RootCase switch
{
DiscoverHierarchyRequest.RootOneofCase.RootGobjectId => request.RootGobjectId.ToString(
System.Globalization.CultureInfo.InvariantCulture),
DiscoverHierarchyRequest.RootOneofCase.RootTagName => request.RootTagName,
DiscoverHierarchyRequest.RootOneofCase.RootContainedPath => request.RootContainedPath,
_ => string.Empty,
});
builder.Append("|max=").Append(request.MaxDepth?.ToString(System.Globalization.CultureInfo.InvariantCulture) ?? "");
builder.Append("|cat=").AppendJoin(',', request.CategoryIds.Order());
builder.Append("|tpl=").AppendJoin(',', request.TemplateChainContains.Order(StringComparer.OrdinalIgnoreCase));
builder.Append("|glob=").Append(request.TagNameGlob);
builder.Append("|attrs=").Append(request.HasIncludeAttributes ? request.IncludeAttributes.ToString() : "unset");
builder.Append("|alarm=").Append(request.AlarmBearingOnly);
builder.Append("|hist=").Append(request.HistorizedOnly);
builder.Append("|browse=").AppendJoin(',', (browseSubtreeGlobs ?? Array.Empty<string>()).Order(StringComparer.OrdinalIgnoreCase));
byte[] hash = SHA256.HashData(Encoding.UTF8.GetBytes(builder.ToString()));
return Convert.ToHexString(hash, 0, 12);
}
}
@@ -0,0 +1,16 @@
using ZB.MOM.WW.GalaxyRepository.Grpc;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// Result of one <see cref="GalaxyHierarchyProjector.Project(GalaxyHierarchyCacheEntry, DiscoverHierarchyRequest, System.Collections.Generic.IReadOnlyList{string}, int, int)"/>
/// call: a materialized page of matching objects, the total post-filter object count, and
/// the stable filter signature used to bind page tokens.
/// </summary>
/// <param name="Objects">The page of matching objects.</param>
/// <param name="TotalObjectCount">Total matching objects across the whole hierarchy (post-filter).</param>
/// <param name="FilterSignature">Stable signature of the filter set, used to bind page tokens.</param>
public sealed record GalaxyHierarchyQueryResult(
IReadOnlyList<GalaxyObject> Objects,
int TotalObjectCount,
string FilterSignature);
@@ -0,0 +1,62 @@
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>Background service that periodically refreshes the Galaxy Repository hierarchy cache off the request path.</summary>
public sealed class GalaxyHierarchyRefreshService(
IGalaxyHierarchyCache cache,
IOptions<GalaxyRepositoryOptions> options,
ILogger<GalaxyHierarchyRefreshService> logger,
TimeProvider? timeProvider = null) : BackgroundService
{
private readonly TimeProvider _timeProvider = timeProvider ?? TimeProvider.System;
/// <inheritdoc />
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
TimeSpan interval = TimeSpan.FromSeconds(Math.Max(1, options.Value.DashboardRefreshIntervalSeconds));
try
{
await cache.RefreshAsync(stoppingToken).ConfigureAwait(false);
}
catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
{
return;
}
catch (Exception exception)
{
// A transient first-load failure (e.g. a TimeoutException or
// Win32Exception from connection establishment, or a DbException
// subtype the cache does not catch) must not fault this
// BackgroundService and stop the whole host. The cache records
// its own Unavailable/Stale status; the periodic tick below retries.
logger.LogWarning(exception, "Initial Galaxy hierarchy cache load failed; will retry on the refresh interval.");
}
using PeriodicTimer timer = new(interval, _timeProvider);
try
{
while (await timer.WaitForNextTickAsync(stoppingToken).ConfigureAwait(false))
{
try
{
await cache.RefreshAsync(stoppingToken).ConfigureAwait(false);
}
catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
{
return;
}
catch (Exception exception)
{
logger.LogWarning(exception, "Galaxy hierarchy cache refresh tick failed.");
}
}
}
catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
{
}
}
}
@@ -0,0 +1,35 @@
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// One row from <see cref="GalaxyRepository.GetHierarchyAsync"/>: a deployed Galaxy
/// <c>gobject</c> with its hierarchy parent and template-derivation chain.
/// </summary>
public sealed class GalaxyHierarchyRow
{
/// <summary>Gets the Galaxy object identifier.</summary>
public int GobjectId { get; init; }
/// <summary>Gets the tag name.</summary>
public string TagName { get; init; } = string.Empty;
/// <summary>Gets the contained name.</summary>
public string ContainedName { get; init; } = string.Empty;
/// <summary>Gets the browse name.</summary>
public string BrowseName { get; init; } = string.Empty;
/// <summary>Gets the parent Galaxy object identifier.</summary>
public int ParentGobjectId { get; init; }
/// <summary>Gets a value indicating whether this is an area.</summary>
public bool IsArea { get; init; }
/// <summary>Gets the category identifier.</summary>
public int CategoryId { get; init; }
/// <summary>Gets the Galaxy object identifier of the host.</summary>
public int HostedByGobjectId { get; init; }
/// <summary>Gets the template derivation chain.</summary>
public IReadOnlyList<string> TemplateChain { get; init; } = Array.Empty<string>();
}
@@ -0,0 +1,24 @@
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// A serializable point-in-time copy of the Galaxy Repository browse data.
/// Holds the raw hierarchy and attribute rowsets — not the materialized
/// protobuf objects — so the restore path runs the exact same
/// materialization as a live refresh. Persisted by
/// <see cref="IGalaxyHierarchySnapshotStore"/> after a successful refresh
/// and reloaded at startup when the Galaxy database is unreachable.
/// </summary>
/// <param name="LastDeployTime">
/// The <c>galaxy.time_of_last_deploy</c> the rowsets were pulled at, or
/// <see langword="null"/> when the Galaxy table reported no deploy. A later
/// live refresh that observes this same timestamp can promote the restored
/// entry to healthy without re-running the heavy queries.
/// </param>
/// <param name="SavedAt">UTC wall-clock when the snapshot was written to disk.</param>
/// <param name="Hierarchy">The persisted object-hierarchy rowset.</param>
/// <param name="Attributes">The persisted attribute rowset.</param>
public sealed record GalaxyHierarchySnapshot(
DateTimeOffset? LastDeployTime,
DateTimeOffset SavedAt,
IReadOnlyList<GalaxyHierarchyRow> Hierarchy,
IReadOnlyList<GalaxyAttributeRow> Attributes);
@@ -0,0 +1,152 @@
using System.Text.Json;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// JSON-file implementation of <see cref="IGalaxyHierarchySnapshotStore"/>.
/// Writes the on-disk snapshot atomically (temp file + rename) so a crash
/// mid-write can never leave a torn file, and ignores files whose schema
/// version it does not recognize. When
/// <see cref="GalaxyRepositoryOptions.PersistSnapshot"/> is <see langword="false"/>
/// — or <see cref="GalaxyRepositoryOptions.SnapshotCachePath"/> is empty —
/// both operations are no-ops. The snapshot path is fully consumer-supplied;
/// this store imposes no platform-specific default, so it is cross-platform.
/// </summary>
public sealed class GalaxyHierarchySnapshotStore : IGalaxyHierarchySnapshotStore, IDisposable
{
/// <summary>
/// On-disk format version. Bump this whenever the persisted shape changes
/// in a way an older or newer consumer cannot read; a mismatched file is
/// ignored rather than misparsed.
/// </summary>
private const int CurrentSchemaVersion = 1;
private static readonly JsonSerializerOptions SerializerOptions = new()
{
WriteIndented = false,
};
private readonly string? _path;
private readonly TimeSpan _writeTimeout;
private readonly ILogger<GalaxyHierarchySnapshotStore>? _logger;
private readonly SemaphoreSlim _ioGate = new(1, 1);
/// <summary>Initializes a new instance of the <see cref="GalaxyHierarchySnapshotStore"/> class.</summary>
/// <param name="options">Galaxy repository options carrying the snapshot path and enable flag.</param>
/// <param name="logger">Optional logger for diagnostic output.</param>
public GalaxyHierarchySnapshotStore(
IOptions<GalaxyRepositoryOptions> options,
ILogger<GalaxyHierarchySnapshotStore>? logger = null)
{
GalaxyRepositoryOptions value = options.Value;
_path = value.PersistSnapshot && !string.IsNullOrWhiteSpace(value.SnapshotCachePath)
? value.SnapshotCachePath
: null;
_writeTimeout = TimeSpan.FromSeconds(Math.Max(1, value.CommandTimeoutSeconds));
_logger = logger;
}
/// <inheritdoc />
public async Task SaveAsync(GalaxyHierarchySnapshot snapshot, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(snapshot);
if (_path is null)
{
return;
}
PersistedFile file = new(CurrentSchemaVersion, snapshot);
await _ioGate.WaitAsync(cancellationToken).ConfigureAwait(false);
try
{
// Bound the write so a stuck disk — e.g. a SnapshotCachePath on an
// unresponsive network share — cannot stall the caller. On the cache
// refresh path that would otherwise pin the whole refresh loop.
using CancellationTokenSource writeCts = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
writeCts.CancelAfter(_writeTimeout);
string? directory = Path.GetDirectoryName(_path);
if (!string.IsNullOrEmpty(directory))
{
Directory.CreateDirectory(directory);
}
string tempPath = _path + ".tmp";
await using (FileStream stream = new(tempPath, FileMode.Create, FileAccess.Write, FileShare.None))
{
await JsonSerializer.SerializeAsync(stream, file, SerializerOptions, writeCts.Token).ConfigureAwait(false);
}
File.Move(tempPath, _path, overwrite: true);
_logger?.LogDebug(
"Persisted Galaxy hierarchy snapshot to {Path} ({ObjectCount} objects, {AttributeCount} attributes).",
_path,
snapshot.Hierarchy.Count,
snapshot.Attributes.Count);
}
finally
{
_ioGate.Release();
}
}
/// <inheritdoc />
public async Task<GalaxyHierarchySnapshot?> TryLoadAsync(CancellationToken cancellationToken)
{
if (_path is null || !File.Exists(_path))
{
return null;
}
await _ioGate.WaitAsync(cancellationToken).ConfigureAwait(false);
try
{
PersistedFile? file;
await using (FileStream stream = new(_path, FileMode.Open, FileAccess.Read, FileShare.Read))
{
file = await JsonSerializer.DeserializeAsync<PersistedFile>(
stream, SerializerOptions, cancellationToken).ConfigureAwait(false);
}
if (file is null || file.SchemaVersion != CurrentSchemaVersion || file.Snapshot is null)
{
_logger?.LogWarning(
"Ignoring Galaxy hierarchy snapshot at {Path}: unrecognized or empty schema version.",
_path);
return null;
}
return file.Snapshot;
}
catch (Exception exception) when (exception is JsonException or IOException or UnauthorizedAccessException)
{
// A corrupt, truncated, locked, or access-denied snapshot file is an
// expected failure mode for a disk cache — honor the Try contract and
// return null rather than throwing.
_logger?.LogWarning(
exception,
"Ignoring Galaxy hierarchy snapshot at {Path}: the file is unreadable or not valid JSON.",
_path);
return null;
}
finally
{
_ioGate.Release();
}
}
/// <summary>
/// Disposes the I/O gate. As a DI singleton the store is disposed once at host
/// shutdown, by which point no save/load is in flight.
/// </summary>
public void Dispose()
{
_ioGate.Dispose();
}
/// <summary>On-disk envelope: a schema version plus the snapshot payload.</summary>
private sealed record PersistedFile(int SchemaVersion, GalaxyHierarchySnapshot? Snapshot);
}
@@ -0,0 +1,16 @@
using ZB.MOM.WW.GalaxyRepository.Grpc;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// A <see cref="GalaxyObject"/> paired with its computed contained path and hierarchy
/// depth. Materialized once per cache entry by <see cref="GalaxyHierarchyIndex"/> so
/// browse/discover projection can filter and page without recomputing paths.
/// </summary>
/// <param name="Object">The projected Galaxy object.</param>
/// <param name="ContainedPath">The slash-delimited contained path from the hierarchy root.</param>
/// <param name="Depth">The number of path segments from the root (zero for top-level objects).</param>
public sealed record GalaxyObjectView(
GalaxyObject Object,
string ContainedPath,
int Depth);
@@ -0,0 +1,76 @@
using ZB.MOM.WW.GalaxyRepository.Grpc;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// Maps <see cref="GalaxyHierarchyRow"/> + <see cref="GalaxyAttributeRow"/> rows produced
/// by <see cref="GalaxyRepository"/> into <c>galaxy_repository.v1</c> proto messages.
/// Pure function, separated so it can be unit-tested without a SQL connection.
/// </summary>
public static class GalaxyProtoMapper
{
/// <summary>Maps Galaxy hierarchy and attribute rows to Galaxy object protos.</summary>
/// <param name="hierarchy">Hierarchy rows from Galaxy Repository.</param>
/// <param name="attributes">Attribute rows from Galaxy Repository.</param>
public static IEnumerable<GalaxyObject> MapHierarchy(
IReadOnlyList<GalaxyHierarchyRow> hierarchy,
IReadOnlyList<GalaxyAttributeRow> attributes)
{
Dictionary<int, List<GalaxyAttributeRow>> attributesByGobjectId = attributes
.GroupBy(a => a.GobjectId)
.ToDictionary(g => g.Key, g => g.ToList());
foreach (GalaxyHierarchyRow row in hierarchy)
{
yield return MapObject(row, attributesByGobjectId);
}
}
/// <summary>Maps a Galaxy hierarchy row to a Galaxy object proto.</summary>
/// <param name="row">Hierarchy row from Galaxy Repository.</param>
/// <param name="attributesByGobjectId">Attributes indexed by gobject ID.</param>
public static GalaxyObject MapObject(
GalaxyHierarchyRow row,
IReadOnlyDictionary<int, List<GalaxyAttributeRow>> attributesByGobjectId)
{
GalaxyObject obj = new()
{
GobjectId = row.GobjectId,
TagName = row.TagName,
ContainedName = row.ContainedName,
BrowseName = row.BrowseName,
ParentGobjectId = row.ParentGobjectId,
IsArea = row.IsArea,
CategoryId = row.CategoryId,
HostedByGobjectId = row.HostedByGobjectId,
};
obj.TemplateChain.AddRange(row.TemplateChain);
if (attributesByGobjectId.TryGetValue(row.GobjectId, out List<GalaxyAttributeRow>? attrs))
{
foreach (GalaxyAttributeRow attr in attrs)
{
obj.Attributes.Add(MapAttribute(attr));
}
}
return obj;
}
/// <summary>Maps a Galaxy attribute row to a Galaxy attribute proto.</summary>
/// <param name="row">Attribute row from Galaxy Repository.</param>
public static GalaxyAttribute MapAttribute(GalaxyAttributeRow row) => new()
{
AttributeName = row.AttributeName,
FullTagReference = row.FullTagReference,
MxDataType = row.MxDataType,
DataTypeName = row.DataTypeName ?? string.Empty,
IsArray = row.IsArray,
ArrayDimension = row.ArrayDimension ?? 0,
ArrayDimensionPresent = row.ArrayDimension.HasValue,
MxAttributeCategory = row.MxAttributeCategory,
SecurityClassification = row.SecurityClassification,
IsHistorized = row.IsHistorized,
IsAlarm = row.IsAlarm,
};
}
@@ -0,0 +1,257 @@
using Microsoft.Data.SqlClient;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// SQL access to the AVEVA System Platform Galaxy Repository database.
/// <para>
/// <see cref="HierarchySql" /> is the query originally ported from the OtOpcUa
/// project. <see cref="AttributesSql" /> has diverged: it additionally enumerates the
/// built-in attributes contributed by each object's primitives (from
/// <c>attribute_definition</c> via <c>primitive_instance</c>), so engine/platform objects
/// and extension sub-attributes (e.g. <c>TestAlarm001.Acked</c>) are surfaced. The
/// OtOpcUa query is not kept in sync.
/// </para>
/// </summary>
public sealed class GalaxyRepository(GalaxyRepositoryOptions options) : IGalaxyRepository
{
/// <summary>Tests the connection to the Galaxy Repository database.</summary>
/// <param name="ct">Token to cancel the asynchronous operation.</param>
public async Task<bool> TestConnectionAsync(CancellationToken ct = default)
{
try
{
using SqlConnection conn = new(options.ConnectionString);
await conn.OpenAsync(ct).ConfigureAwait(false);
using SqlCommand cmd = new("SELECT 1", conn) { CommandTimeout = options.CommandTimeoutSeconds };
object? result = await cmd.ExecuteScalarAsync(ct).ConfigureAwait(false);
return result is int i && i == 1;
}
catch (SqlException) { return false; }
catch (InvalidOperationException) { return false; }
}
/// <summary>Retrieves the last deployment time from the Galaxy Repository.</summary>
/// <param name="ct">Token to cancel the asynchronous operation.</param>
public async Task<DateTime?> GetLastDeployTimeAsync(CancellationToken ct = default)
{
using SqlConnection conn = new(options.ConnectionString);
await conn.OpenAsync(ct).ConfigureAwait(false);
using SqlCommand cmd = new("SELECT time_of_last_deploy FROM galaxy", conn)
{ CommandTimeout = options.CommandTimeoutSeconds };
object? result = await cmd.ExecuteScalarAsync(ct).ConfigureAwait(false);
return result is DateTime dt ? dt : null;
}
/// <summary>Retrieves the complete hierarchy of Galaxy objects from the repository.</summary>
/// <param name="ct">Token to cancel the asynchronous operation.</param>
public async Task<List<GalaxyHierarchyRow>> GetHierarchyAsync(CancellationToken ct = default)
{
List<GalaxyHierarchyRow> rows = new();
using SqlConnection conn = new(options.ConnectionString);
await conn.OpenAsync(ct).ConfigureAwait(false);
using SqlCommand cmd = new(HierarchySql, conn) { CommandTimeout = options.CommandTimeoutSeconds };
using SqlDataReader reader = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false);
while (await reader.ReadAsync(ct).ConfigureAwait(false))
{
string templateChainRaw = reader.IsDBNull(8) ? string.Empty : reader.GetString(8);
string[] templateChain = templateChainRaw.Length == 0
? Array.Empty<string>()
: templateChainRaw.Split(['|'], StringSplitOptions.RemoveEmptyEntries)
.Select(s => s.Trim())
.Where(s => s.Length > 0)
.ToArray();
rows.Add(new GalaxyHierarchyRow
{
GobjectId = Convert.ToInt32(reader.GetValue(0)),
TagName = reader.GetString(1),
ContainedName = reader.IsDBNull(2) ? string.Empty : reader.GetString(2),
BrowseName = reader.GetString(3),
ParentGobjectId = Convert.ToInt32(reader.GetValue(4)),
IsArea = Convert.ToInt32(reader.GetValue(5)) == 1,
CategoryId = Convert.ToInt32(reader.GetValue(6)),
HostedByGobjectId = Convert.ToInt32(reader.GetValue(7)),
TemplateChain = templateChain,
});
}
return rows;
}
/// <summary>Retrieves all attributes for Galaxy objects from the repository.</summary>
/// <param name="ct">Token to cancel the asynchronous operation.</param>
public async Task<List<GalaxyAttributeRow>> GetAttributesAsync(CancellationToken ct = default)
{
List<GalaxyAttributeRow> rows = new();
using SqlConnection conn = new(options.ConnectionString);
await conn.OpenAsync(ct).ConfigureAwait(false);
using SqlCommand cmd = new(AttributesSql, conn) { CommandTimeout = options.CommandTimeoutSeconds };
using SqlDataReader reader = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false);
while (await reader.ReadAsync(ct).ConfigureAwait(false))
{
rows.Add(new GalaxyAttributeRow
{
GobjectId = Convert.ToInt32(reader.GetValue(0)),
TagName = reader.GetString(1),
AttributeName = reader.GetString(2),
FullTagReference = reader.GetString(3),
MxDataType = Convert.ToInt32(reader.GetValue(4)),
DataTypeName = reader.IsDBNull(5) ? null : reader.GetString(5),
IsArray = Convert.ToInt32(reader.GetValue(6)) == 1,
ArrayDimension = reader.IsDBNull(7) ? null : Convert.ToInt32(reader.GetValue(7)),
MxAttributeCategory = Convert.ToInt32(reader.GetValue(8)),
SecurityClassification = Convert.ToInt32(reader.GetValue(9)),
IsHistorized = Convert.ToInt32(reader.GetValue(10)) == 1,
IsAlarm = Convert.ToInt32(reader.GetValue(11)) == 1,
});
}
return rows;
}
// Area objects (category 13) are returned even when undeployed (deployed_package_id = 0):
// they are organizational/model nodes that group deployed objects, so excluding them
// orphans every area whose containing area is not itself deployed. All non-area objects
// still require deployment. Orphans left by a missing/deleted parent area are re-rooted
// by GalaxyHierarchyIndex.Build so nothing disappears from browse.
private const string HierarchySql = @"
;WITH template_chain AS (
SELECT g.gobject_id AS instance_gobject_id, t.gobject_id AS template_gobject_id,
t.tag_name AS template_tag_name, t.derived_from_gobject_id, 0 AS depth
FROM gobject g
INNER JOIN gobject t ON t.gobject_id = g.derived_from_gobject_id
WHERE g.is_template = 0 AND g.deployed_package_id <> 0 AND g.derived_from_gobject_id <> 0
UNION ALL
SELECT tc.instance_gobject_id, t.gobject_id, t.tag_name, t.derived_from_gobject_id, tc.depth + 1
FROM template_chain tc
INNER JOIN gobject t ON t.gobject_id = tc.derived_from_gobject_id
WHERE tc.derived_from_gobject_id <> 0 AND tc.depth < 10
)
SELECT DISTINCT
g.gobject_id,
g.tag_name,
g.contained_name,
CASE WHEN g.contained_name IS NULL OR g.contained_name = ''
THEN g.tag_name
ELSE g.contained_name
END AS browse_name,
CASE WHEN g.contained_by_gobject_id = 0
THEN g.area_gobject_id
ELSE g.contained_by_gobject_id
END AS parent_gobject_id,
CASE WHEN td.category_id = 13
THEN 1
ELSE 0
END AS is_area,
td.category_id AS category_id,
g.hosted_by_gobject_id AS hosted_by_gobject_id,
ISNULL(
STUFF((
SELECT '|' + tc.template_tag_name
FROM template_chain tc
WHERE tc.instance_gobject_id = g.gobject_id
ORDER BY tc.depth
FOR XML PATH('')
), 1, 1, ''),
''
) AS template_chain
FROM gobject g
INNER JOIN template_definition td
ON g.template_definition_id = td.template_definition_id
WHERE td.category_id IN (1, 3, 4, 10, 11, 13, 17, 24, 26)
AND g.is_template = 0
AND (g.deployed_package_id <> 0 OR td.category_id = 13)
ORDER BY parent_gobject_id, g.tag_name";
// Unlike HierarchySql, this query has diverged from the OtOpcUa original. It returns two
// kinds of attribute: user-configured dynamic attributes (the original `dynamic_attribute`
// body, src_pri 0) and the built-in attributes every object inherits from its primitives
// (`attribute_definition` joined through `primitive_instance`, src_pri 1). Built-in
// attributes are why engine/platform objects and extension sub-attributes such as
// `TestAlarm001.Acked` show up at all. Built-in rows carry no category filter (the
// `attribute_definition` category numbering differs from `dynamic_attribute`'s — only the
// `_`-prefix and `.Description` name exclusions apply) and are never flagged
// `is_historized`/`is_alarm`: those flags describe a user attribute that anchors an
// extension, not the extension's machinery leaves.
private const string AttributesSql = @"
;WITH deployed_package_chain AS (
SELECT g.gobject_id, p.package_id, p.derived_from_package_id, 0 AS depth
FROM gobject g
INNER JOIN package p ON p.package_id = g.deployed_package_id
WHERE g.is_template = 0 AND g.deployed_package_id <> 0
UNION ALL
SELECT dpc.gobject_id, p.package_id, p.derived_from_package_id, dpc.depth + 1
FROM deployed_package_chain dpc
INNER JOIN package p ON p.package_id = dpc.derived_from_package_id
WHERE dpc.derived_from_package_id <> 0 AND dpc.depth < 10
),
candidate AS (
SELECT
dpc.gobject_id, g.tag_name, da.attribute_name, da.mx_data_type, da.is_array,
CASE WHEN da.is_array = 1
THEN CONVERT(int, CONVERT(varbinary(2),
SUBSTRING(da.mx_value, 15, 2) + SUBSTRING(da.mx_value, 13, 2), 2))
ELSE NULL END AS array_dimension,
da.mx_attribute_category, da.security_classification, dpc.depth, 0 AS src_pri
FROM deployed_package_chain dpc
INNER JOIN dynamic_attribute da ON da.package_id = dpc.package_id
INNER JOIN gobject g ON g.gobject_id = dpc.gobject_id
INNER JOIN template_definition td ON td.template_definition_id = g.template_definition_id
WHERE td.category_id IN (1, 3, 4, 10, 11, 13, 17, 24, 26)
AND da.attribute_name NOT LIKE '[_]%'
AND da.attribute_name NOT LIKE '%.Description'
AND da.mx_attribute_category IN (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 24)
UNION ALL
SELECT
dpc.gobject_id, g.tag_name,
CASE WHEN pi.primitive_name IS NULL OR pi.primitive_name = ''
THEN ad.attribute_name
ELSE pi.primitive_name + '.' + ad.attribute_name END AS attribute_name,
ad.mx_data_type, ad.is_array,
CASE WHEN ad.is_array = 1
THEN CONVERT(int, CONVERT(varbinary(2),
SUBSTRING(ad.mx_value, 15, 2) + SUBSTRING(ad.mx_value, 13, 2), 2))
ELSE NULL END AS array_dimension,
ad.mx_attribute_category, ad.security_classification, dpc.depth, 1 AS src_pri
FROM deployed_package_chain dpc
INNER JOIN primitive_instance pi ON pi.package_id = dpc.package_id
INNER JOIN attribute_definition ad ON ad.primitive_definition_id = pi.primitive_definition_id
INNER JOIN gobject g ON g.gobject_id = dpc.gobject_id
INNER JOIN template_definition td ON td.template_definition_id = g.template_definition_id
WHERE td.category_id IN (1, 3, 4, 10, 11, 13, 17, 24, 26)
AND ad.attribute_name NOT LIKE '[_]%'
AND ad.attribute_name NOT LIKE '%.Description'
),
ranked AS (
SELECT c.*, ROW_NUMBER() OVER (
PARTITION BY c.gobject_id, c.attribute_name ORDER BY c.src_pri, c.depth) AS rn
FROM candidate c
)
SELECT
r.gobject_id, r.tag_name, r.attribute_name,
r.tag_name + '.' + r.attribute_name
+ CASE WHEN r.is_array = 1 THEN '[]' ELSE '' END AS full_tag_reference,
r.mx_data_type, dt.description AS data_type_name, r.is_array, r.array_dimension,
r.mx_attribute_category, r.security_classification,
CASE WHEN r.src_pri = 0 AND EXISTS (
SELECT 1 FROM deployed_package_chain dpc2
INNER JOIN primitive_instance pi ON pi.package_id = dpc2.package_id AND pi.primitive_name = r.attribute_name
INNER JOIN primitive_definition pd ON pd.primitive_definition_id = pi.primitive_definition_id AND pd.primitive_name = 'HistoryExtension'
WHERE dpc2.gobject_id = r.gobject_id
) THEN 1 ELSE 0 END AS is_historized,
CASE WHEN r.src_pri = 0 AND EXISTS (
SELECT 1 FROM deployed_package_chain dpc2
INNER JOIN primitive_instance pi ON pi.package_id = dpc2.package_id AND pi.primitive_name = r.attribute_name
INNER JOIN primitive_definition pd ON pd.primitive_definition_id = pi.primitive_definition_id AND pd.primitive_name = 'AlarmExtension'
WHERE dpc2.gobject_id = r.gobject_id
) THEN 1 ELSE 0 END AS is_alarm
FROM ranked r
LEFT JOIN data_type dt ON dt.mx_data_type = r.mx_data_type
WHERE r.rn = 1
ORDER BY r.tag_name, r.attribute_name";
}
@@ -0,0 +1,55 @@
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// Connection settings for the AVEVA System Platform Galaxy Repository database.
/// <para>
/// <see cref="SectionName"/> is a generic default; the DI extension accepts an explicit
/// configuration section path so a consumer can bind from its own section (e.g.
/// <c>HistorianGateway:Galaxy</c>).
/// </para>
/// </summary>
public sealed class GalaxyRepositoryOptions
{
/// <summary>
/// Generic default configuration section name. The DI extension accepts an explicit
/// section path, so a consumer may bind from a different section (e.g.
/// <c>HistorianGateway:Galaxy</c>).
/// </summary>
public const string SectionName = "GalaxyRepository";
/// <summary>
/// Default SQL Server connection string for the Galaxy Repository database.
/// Single source of truth shared with the integration-test fallback so the
/// production default and the live-test default cannot drift.
/// </summary>
public const string DefaultConnectionString =
"Server=localhost;Database=ZB;Integrated Security=True;TrustServerCertificate=True;Encrypt=False;";
/// <summary>The SQL Server connection string for the Galaxy Repository database.</summary>
public string ConnectionString { get; init; } = DefaultConnectionString;
/// <summary>The timeout in seconds for SQL commands executed against the Galaxy Repository.</summary>
public int CommandTimeoutSeconds { get; init; } = 60;
/// <summary>
/// Interval (seconds) between background refreshes of the dashboard Galaxy summary
/// cache. SQL is hit at most once per interval regardless of dashboard render rate.
/// </summary>
public int DashboardRefreshIntervalSeconds { get; init; } = 30;
/// <summary>
/// Whether the latest successful Galaxy browse dataset is persisted to disk. When
/// enabled, the cache reloads that snapshot at startup so clients can still browse
/// last-known data while the Galaxy database is unreachable.
/// </summary>
public bool PersistSnapshot { get; init; } = true;
/// <summary>
/// File path for the persisted Galaxy browse snapshot. Ignored when
/// <see cref="PersistSnapshot"/> is <see langword="false"/>. There is no built-in
/// default path — the consumer supplies a cross-platform-friendly path appropriate to
/// its host. When left empty and <see cref="PersistSnapshot"/> is enabled, the
/// snapshot store (a later task) decides where to write.
/// </summary>
public string SnapshotCachePath { get; init; } = string.Empty;
}
@@ -0,0 +1,16 @@
using ZB.MOM.WW.GalaxyRepository.Grpc;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// Resolution result for a tag address: the owning <see cref="GalaxyObject"/>, the
/// specific <see cref="GalaxyAttribute"/> when the address names an attribute (otherwise
/// <see langword="null"/>), and the object's contained path.
/// </summary>
/// <param name="Object">The Galaxy object that owns the looked-up address.</param>
/// <param name="Attribute">The matched attribute, or <see langword="null"/> when the address names an object.</param>
/// <param name="ContainedPath">The owning object's contained path.</param>
public sealed record GalaxyTagLookup(
GalaxyObject Object,
GalaxyAttribute? Attribute,
string ContainedPath);
@@ -0,0 +1,329 @@
using Google.Protobuf.WellKnownTypes;
using Grpc.Core;
using ProtoGalaxyRepository = ZB.MOM.WW.GalaxyRepository.Grpc.GalaxyRepository;
namespace ZB.MOM.WW.GalaxyRepository.Grpc;
/// <summary>
/// Reusable gRPC surface that exposes the Galaxy Repository to clients. Hosted by any
/// consuming gateway (e.g. MxAccessGateway or the HistorianGateway sidecar) via
/// <see cref="DependencyInjection.GalaxyRepositoryServiceCollectionExtensions.MapZbGalaxyRepository"/>.
/// <para>
/// <c>DiscoverHierarchy</c> and <c>GetLastDeployTime</c> serve from
/// <see cref="IGalaxyHierarchyCache"/> so many clients share a single SQL pull.
/// <c>WatchDeployEvents</c> streams events from <see cref="IGalaxyDeployNotifier"/>.
/// <c>TestConnection</c> remains a direct SQL probe since callers use it as a health check.
/// </para>
/// <para>
/// This service applies <b>no</b> per-identity browse-subtree filtering — the full
/// hierarchy is projected (<c>null</c> subtree globs). Authorization (including any
/// subtree scoping) is the responsibility of the hosting gateway's interceptor layer.
/// </para>
/// </summary>
/// <param name="repository">Direct SQL surface used by <c>TestConnection</c>.</param>
/// <param name="cache">Shared hierarchy cache that <c>DiscoverHierarchy</c>/<c>BrowseChildren</c>/<c>GetLastDeployTime</c> serve from.</param>
/// <param name="notifier">Deploy-event source streamed by <c>WatchDeployEvents</c>.</param>
public sealed class GalaxyRepositoryGrpcService(
IGalaxyRepository repository,
IGalaxyHierarchyCache cache,
IGalaxyDeployNotifier notifier) : ProtoGalaxyRepository.GalaxyRepositoryBase
{
private static readonly TimeSpan FirstLoadWaitBudget = TimeSpan.FromSeconds(5);
private const int DefaultDiscoverPageSize = 1000;
private const int MaxDiscoverPageSize = 5000;
private const int DefaultBrowsePageSize = 500;
// MaxBrowsePageSize reuses MaxDiscoverPageSize (5000) — same cap.
/// <inheritdoc />
public override async Task<TestConnectionReply> TestConnection(
TestConnectionRequest request,
ServerCallContext context)
{
bool ok = await repository.TestConnectionAsync(context.CancellationToken).ConfigureAwait(false);
return new TestConnectionReply { Ok = ok };
}
/// <inheritdoc />
public override async Task<GetLastDeployTimeReply> GetLastDeployTime(
GetLastDeployTimeRequest request,
ServerCallContext context)
{
await WaitForCacheBootstrap(context.CancellationToken).ConfigureAwait(false);
GalaxyHierarchyCacheEntry entry = cache.Current;
if (!entry.HasData)
{
throw new RpcException(new Status(
StatusCode.Unavailable,
ResolveUnavailableMessage(entry)));
}
GetLastDeployTimeReply reply = new() { Present = entry.LastDeployTime.HasValue };
if (entry.LastDeployTime.HasValue)
{
reply.TimeOfLastDeploy = Timestamp.FromDateTimeOffset(entry.LastDeployTime.Value);
}
return reply;
}
/// <inheritdoc />
public override async Task<DiscoverHierarchyReply> DiscoverHierarchy(
DiscoverHierarchyRequest request,
ServerCallContext context)
{
await WaitForCacheBootstrap(context.CancellationToken).ConfigureAwait(false);
GalaxyHierarchyCacheEntry entry = cache.Current;
if (!entry.HasData)
{
throw new RpcException(new Status(
StatusCode.Unavailable,
ResolveUnavailableMessage(entry)));
}
int pageSize = ResolvePageSize(request.PageSize);
// The shared library applies no per-identity subtree scoping; the hosting
// gateway enforces authorization at its interceptor layer.
string filterSignature = GalaxyHierarchyProjector.ComputeFilterSignature(request, browseSubtreeGlobs: null);
PageToken pageToken = ParsePageToken(request.PageToken, entry.Sequence, filterSignature);
GalaxyHierarchyQueryResult query = GalaxyHierarchyProjector.Project(
entry,
request,
browseSubtreeGlobs: null,
pageToken.Offset,
pageSize);
int offset = pageToken.Offset;
if (offset > query.TotalObjectCount)
{
throw new RpcException(new Status(
StatusCode.InvalidArgument,
"DiscoverHierarchy page_token is outside the current hierarchy."));
}
DiscoverHierarchyReply reply = new()
{
TotalObjectCount = query.TotalObjectCount,
};
reply.Objects.Add(query.Objects);
int nextOffset = offset + query.Objects.Count;
if (nextOffset < query.TotalObjectCount)
{
reply.NextPageToken = FormatPageToken(entry.Sequence, query.FilterSignature, nextOffset);
}
return reply;
}
/// <inheritdoc />
public override async Task<BrowseChildrenReply> BrowseChildren(
BrowseChildrenRequest request,
ServerCallContext context)
{
await WaitForCacheBootstrap(context.CancellationToken).ConfigureAwait(false);
GalaxyHierarchyCacheEntry entry = cache.Current;
if (!entry.HasData)
{
throw new RpcException(new Status(
StatusCode.Unavailable,
ResolveUnavailableMessage(entry)));
}
int pageSize = ResolveBrowsePageSize(request.PageSize);
// Resolve the parent id once so the page-token signature can include it
// and the projector sees the same resolved id when memoizing. The projector
// re-resolves internally; with the by-name/by-path indexes on
// GalaxyHierarchyIndex that second call is O(1), so the redundancy is cheap
// and keeps the projector self-contained.
int parentId = GalaxyBrowseProjector.ResolveParentId(entry, request);
string filterSignature = GalaxyBrowseProjector.ComputeFilterSignature(
request, browseSubtreeGlobs: null, parentId);
PageToken pageToken = ParsePageToken(request.PageToken, entry.Sequence, filterSignature);
GalaxyBrowseChildrenResult result = GalaxyBrowseProjector.ProjectChildren(
entry,
request,
browseSubtreeGlobs: null,
pageToken.Offset,
pageSize);
if (pageToken.Offset > result.TotalChildCount)
{
throw new RpcException(new Status(
StatusCode.InvalidArgument,
"BrowseChildren page_token is outside the current children set."));
}
BrowseChildrenReply reply = new()
{
TotalChildCount = result.TotalChildCount,
CacheSequence = (ulong)entry.Sequence,
};
reply.Children.Add(result.Children);
reply.ChildHasChildren.Add(result.ChildHasChildren);
int nextOffset = pageToken.Offset + result.Children.Count;
if (nextOffset < result.TotalChildCount)
{
reply.NextPageToken = FormatPageToken(entry.Sequence, result.FilterSignature, nextOffset);
}
return reply;
}
/// <inheritdoc />
public override async Task WatchDeployEvents(
WatchDeployEventsRequest request,
IServerStreamWriter<DeployEvent> responseStream,
ServerCallContext context)
{
DateTimeOffset? lastSeen = request.LastSeenDeployTime?.ToDateTimeOffset();
await foreach (GalaxyDeployEventInfo info in notifier
.SubscribeAsync(context.CancellationToken)
.ConfigureAwait(false))
{
// Suppress the initial bootstrap event when the client already knows about
// this deploy time. We only suppress the first one — subsequent events fire
// on actual changes, so they always pass.
if (lastSeen is { } seen && info.TimeOfLastDeploy == seen)
{
lastSeen = null;
continue;
}
lastSeen = null;
await responseStream.WriteAsync(MapDeployEvent(info), context.CancellationToken).ConfigureAwait(false);
}
}
private async Task WaitForCacheBootstrap(CancellationToken cancellationToken)
{
if (cache.Current.HasData || cache.Current.Status == GalaxyCacheStatus.Unavailable)
{
return;
}
using CancellationTokenSource budget = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
budget.CancelAfter(FirstLoadWaitBudget);
try
{
await cache.WaitForFirstLoadAsync(budget.Token).ConfigureAwait(false);
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
{
throw;
}
catch (OperationCanceledException)
{
// Budget elapsed; fall through and let the caller see the current
// (possibly Unknown/Unavailable) entry.
}
}
private static DeployEvent MapDeployEvent(GalaxyDeployEventInfo info)
{
DeployEvent ev = new()
{
Sequence = (ulong)info.Sequence,
ObservedAt = Timestamp.FromDateTimeOffset(info.ObservedAt),
ObjectCount = info.ObjectCount,
AttributeCount = info.AttributeCount,
TimeOfLastDeployPresent = info.TimeOfLastDeploy.HasValue,
};
if (info.TimeOfLastDeploy.HasValue)
{
ev.TimeOfLastDeploy = Timestamp.FromDateTimeOffset(info.TimeOfLastDeploy.Value);
}
return ev;
}
private static string ResolveUnavailableMessage(GalaxyHierarchyCacheEntry entry) => entry.Status switch
{
GalaxyCacheStatus.Unknown => "Galaxy cache has not completed its initial load yet.",
GalaxyCacheStatus.Unavailable => "Galaxy repository is unavailable.",
_ => "Galaxy cache has no data available.",
};
private static int ResolvePageSize(int requestedPageSize)
{
if (requestedPageSize < 0)
{
throw new RpcException(new Status(
StatusCode.InvalidArgument,
"DiscoverHierarchy page_size must be greater than zero when provided."));
}
int pageSize = requestedPageSize == 0 ? DefaultDiscoverPageSize : requestedPageSize;
return Math.Min(pageSize, MaxDiscoverPageSize);
}
private static int ResolveBrowsePageSize(int requested)
{
if (requested < 0)
{
throw new RpcException(new Status(
StatusCode.InvalidArgument,
"BrowseChildren page_size must be greater than zero when provided."));
}
int pageSize = requested == 0 ? DefaultBrowsePageSize : requested;
return Math.Min(pageSize, MaxDiscoverPageSize);
}
private static string FormatPageToken(long sequence, string filterSignature, int offset)
{
return string.Concat(
sequence.ToString(System.Globalization.CultureInfo.InvariantCulture),
":",
filterSignature,
":",
offset.ToString(System.Globalization.CultureInfo.InvariantCulture));
}
private static PageToken ParsePageToken(string pageToken, long currentSequence, string currentFilterSignature)
{
if (string.IsNullOrWhiteSpace(pageToken))
{
return new PageToken(currentSequence, currentFilterSignature, Offset: 0);
}
string[] parts = pageToken.Split(':', count: 3);
if (parts.Length != 3
|| !long.TryParse(
parts[0],
System.Globalization.NumberStyles.None,
System.Globalization.CultureInfo.InvariantCulture,
out long sequence)
|| !int.TryParse(
parts[2],
System.Globalization.NumberStyles.None,
System.Globalization.CultureInfo.InvariantCulture,
out int offset)
|| offset < 0)
{
throw new RpcException(new Status(
StatusCode.InvalidArgument,
"page_token is invalid."));
}
if (sequence != currentSequence)
{
throw new RpcException(new Status(
StatusCode.InvalidArgument,
"page_token is stale."));
}
if (!string.Equals(parts[1], currentFilterSignature, StringComparison.Ordinal))
{
throw new RpcException(new Status(
StatusCode.InvalidArgument,
"page_token does not match the current filters."));
}
return new PageToken(sequence, parts[1], offset);
}
private sealed record PageToken(long Sequence, string FilterSignature, int Offset);
}
@@ -0,0 +1,17 @@
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>Publishes Galaxy repository deploy events to subscribers.</summary>
public interface IGalaxyDeployNotifier
{
/// <summary>The most recently published event, or null if no event has fired yet.</summary>
GalaxyDeployEventInfo? Latest { get; }
/// <summary>Publishes a deploy event to all current subscribers and stores it as Latest.</summary>
/// <param name="info">The deploy event to publish.</param>
void Publish(GalaxyDeployEventInfo info);
/// <summary>Subscribes to deploy events. The sequence yields the latest event first (if available) then streams new events as they fire.</summary>
/// <param name="cancellationToken">Token to cancel the asynchronous operation.</param>
/// <returns>Async enumerable of deploy events.</returns>
IAsyncEnumerable<GalaxyDeployEventInfo> SubscribeAsync(CancellationToken cancellationToken);
}
@@ -0,0 +1,25 @@
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>Cache for Galaxy Repository hierarchy data.</summary>
public interface IGalaxyHierarchyCache
{
/// <summary>The latest cache entry. Status freshness is recomputed against the clock.</summary>
GalaxyHierarchyCacheEntry Current { get; }
/// <summary>
/// Forces a refresh against the Galaxy Repository. Performs a cheap
/// <c>time_of_last_deploy</c> probe first and only re-queries the heavy hierarchy +
/// attributes rowsets when the deploy time has changed since the last successful
/// refresh.
/// </summary>
/// <param name="cancellationToken">Token to cancel the asynchronous operation.</param>
Task RefreshAsync(CancellationToken cancellationToken);
/// <summary>
/// Awaits the first completed refresh attempt (success or failure). Useful for
/// gRPC handlers that want to serve from cache without returning Unavailable on the
/// very first request after the service starts.
/// </summary>
/// <param name="cancellationToken">Token to cancel the asynchronous operation.</param>
Task WaitForFirstLoadAsync(CancellationToken cancellationToken);
}
@@ -0,0 +1,28 @@
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// Persists the latest Galaxy Repository browse dataset to disk and reloads
/// it at startup. Lets <see cref="GalaxyHierarchyCache"/> serve last-known
/// browse data when the Galaxy database is unreachable on a cold start.
/// </summary>
public interface IGalaxyHierarchySnapshotStore
{
/// <summary>
/// Writes <paramref name="snapshot"/> to disk, replacing any previous
/// snapshot atomically. A no-op when snapshot persistence is disabled.
/// </summary>
/// <param name="snapshot">The browse dataset to persist.</param>
/// <param name="cancellationToken">Token to cancel the asynchronous operation.</param>
Task SaveAsync(GalaxyHierarchySnapshot snapshot, CancellationToken cancellationToken);
/// <summary>
/// Reads the persisted Galaxy browse dataset.
/// </summary>
/// <param name="cancellationToken">Token to cancel the asynchronous operation.</param>
/// <returns>
/// The persisted snapshot, or <see langword="null"/> when none exists,
/// persistence is disabled, or the on-disk file uses an unrecognized
/// schema version.
/// </returns>
Task<GalaxyHierarchySnapshot?> TryLoadAsync(CancellationToken cancellationToken);
}
@@ -0,0 +1,26 @@
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// Abstraction over <see cref="GalaxyRepository"/>: the read-only SQL surface over the
/// AVEVA System Platform Galaxy Repository database. Exists so consumers (and the cache
/// layer, a later task) can be unit-tested against an in-memory fake without standing up a
/// real <c>Microsoft.Data.SqlClient</c> <c>SqlConnection</c> against a bogus host/port.
/// </summary>
public interface IGalaxyRepository
{
/// <summary>Tests the connection to the Galaxy Repository database.</summary>
/// <param name="ct">Token to cancel the asynchronous operation.</param>
Task<bool> TestConnectionAsync(CancellationToken ct = default);
/// <summary>Retrieves the last deployment time from the Galaxy Repository.</summary>
/// <param name="ct">Token to cancel the asynchronous operation.</param>
Task<DateTime?> GetLastDeployTimeAsync(CancellationToken ct = default);
/// <summary>Retrieves the complete hierarchy of Galaxy objects from the repository.</summary>
/// <param name="ct">Token to cancel the asynchronous operation.</param>
Task<List<GalaxyHierarchyRow>> GetHierarchyAsync(CancellationToken ct = default);
/// <summary>Retrieves all attributes for Galaxy objects from the repository.</summary>
/// <param name="ct">Token to cancel the asynchronous operation.</param>
Task<List<GalaxyAttributeRow>> GetAttributesAsync(CancellationToken ct = default);
}
@@ -0,0 +1,190 @@
syntax = "proto3";
package galaxy_repository.v1;
option csharp_namespace = "ZB.MOM.WW.GalaxyRepository.Grpc";
import "google/protobuf/timestamp.proto";
import "google/protobuf/wrappers.proto";
// Wire-compatibility policy (ProtobufStyleGuide): this contract evolves
// additively only. Never renumber or repurpose an existing field number or
// enum value. When a field or enum value is removed, add a `reserved` range
// (and `reserved` name) covering it in the same change so a future editor
// cannot accidentally reuse the retired tag. There are no `reserved`
// declarations today because no field or enum value has ever been removed.
// Read-only browse over the AVEVA System Platform Galaxy Repository (ZB SQL
// database). Lets clients enumerate the deployed object hierarchy and each
// object's dynamic attributes so they know what tag references to subscribe
// to via the MxAccessGateway service.
service GalaxyRepository {
rpc TestConnection(TestConnectionRequest) returns (TestConnectionReply);
rpc GetLastDeployTime(GetLastDeployTimeRequest) returns (GetLastDeployTimeReply);
rpc DiscoverHierarchy(DiscoverHierarchyRequest) returns (DiscoverHierarchyReply);
// Server-stream of deploy events. The server emits the current state immediately
// on subscribe (so clients can bootstrap their cache without waiting for the next
// deploy), then emits one event each time the gateway's hierarchy cache observes
// a new galaxy.time_of_last_deploy. The sequence field is monotonically
// increasing per server start; gaps indicate the per-subscriber buffer dropped
// older events because the client was too slow.
rpc WatchDeployEvents(WatchDeployEventsRequest) returns (stream DeployEvent);
// Returns the direct children of a parent object (or the root objects when
// `parent` is unset). Designed for OPC UA-style lazy expand: clients walk
// one level at a time instead of paging the full hierarchy. Filters mirror
// DiscoverHierarchy exactly. Backed by the same shared hierarchy cache.
rpc BrowseChildren(BrowseChildrenRequest) returns (BrowseChildrenReply);
}
message TestConnectionRequest {}
message TestConnectionReply {
bool ok = 1;
}
message GetLastDeployTimeRequest {}
message GetLastDeployTimeReply {
bool present = 1;
google.protobuf.Timestamp time_of_last_deploy = 2;
}
message DiscoverHierarchyRequest {
// Maximum number of objects to return. The server applies its default when
// unset and rejects non-positive values.
int32 page_size = 1;
// Opaque token returned by a previous DiscoverHierarchy response.
string page_token = 2;
// Optional. When set, return only this object and its descendants.
// Empty = full hierarchy.
oneof root {
int32 root_gobject_id = 3;
string root_tag_name = 4;
string root_contained_path = 5;
}
// Optional. Cap on descendant depth from root. Zero returns only the root.
// Unset means unlimited depth.
google.protobuf.Int32Value max_depth = 6;
// Optional object category id filters.
repeated int32 category_ids = 7;
// Optional case-insensitive substring filters against template names.
repeated string template_chain_contains = 8;
// Optional anchored, case-insensitive glob over object tag_name.
string tag_name_glob = 9;
// Optional. Unset or true includes attributes. False returns object skeletons.
optional bool include_attributes = 10;
// Optional. Return only objects with at least one alarm-bearing attribute.
bool alarm_bearing_only = 11;
// Optional. Return only objects with at least one historized attribute.
bool historized_only = 12;
}
message DiscoverHierarchyReply {
repeated GalaxyObject objects = 1;
// Non-empty when another page is available.
string next_page_token = 2;
// Total number of objects in the cached hierarchy at the time of the call.
int32 total_object_count = 3;
}
message WatchDeployEventsRequest {
// Optional. When set, the bootstrap event is suppressed if the cached deploy
// time matches this value. Future events are still emitted normally.
google.protobuf.Timestamp last_seen_deploy_time = 1;
}
message DeployEvent {
// Monotonically increasing per server start. Gaps indicate dropped events.
uint64 sequence = 1;
// Server wall-clock when the cache observed the deploy.
google.protobuf.Timestamp observed_at = 2;
// Galaxy.time_of_last_deploy. Absent only when the Galaxy table reports null.
google.protobuf.Timestamp time_of_last_deploy = 3;
bool time_of_last_deploy_present = 4;
int32 object_count = 5;
int32 attribute_count = 6;
}
message GalaxyObject {
int32 gobject_id = 1;
string tag_name = 2;
string contained_name = 3;
string browse_name = 4;
int32 parent_gobject_id = 5;
bool is_area = 6;
int32 category_id = 7;
int32 hosted_by_gobject_id = 8;
repeated string template_chain = 9;
repeated GalaxyAttribute attributes = 10;
}
message GalaxyAttribute {
string attribute_name = 1;
string full_tag_reference = 2;
// Raw Galaxy SQL `dbo.data_type` identifier, passed through unchanged.
// This is NOT a member of `mxaccess_gateway.v1.MxDataType` — Galaxy's
// type enumeration is distinct from MXAccess's wire data-type enum and
// the two must not be cast or compared. The GalaxyRepository service is
// metadata-only and deliberately does not share types with
// mxaccess_gateway.proto. See docs/GalaxyRepository.md.
int32 mx_data_type = 3;
// Human-readable name from Galaxy's `dbo.data_type` table (e.g. "Float",
// "Integer", "Boolean"). Free-form Galaxy text; not a stable enum.
string data_type_name = 4;
bool is_array = 5;
int32 array_dimension = 6;
bool array_dimension_present = 7;
// Raw Galaxy SQL attribute-category identifier, passed through unchanged.
// Galaxy-specific; not mapped to any gateway enum. See
// docs/GalaxyRepository.md.
int32 mx_attribute_category = 8;
// Raw Galaxy SQL security-classification identifier, passed through
// unchanged. Galaxy-specific; not mapped to any gateway enum. See
// docs/GalaxyRepository.md.
int32 security_classification = 9;
bool is_historized = 10;
bool is_alarm = 11;
}
message BrowseChildrenRequest {
// Parent selector. Empty oneof returns root objects (parent_gobject_id == 0).
oneof parent {
int32 parent_gobject_id = 1;
string parent_tag_name = 2;
string parent_contained_path = 3;
}
// Maximum number of direct children to return. Server default 500; cap 5000.
int32 page_size = 4;
// Opaque token returned by a previous BrowseChildren response. Bound to the
// cache sequence, parent selector, and the filter set; a mismatch returns
// InvalidArgument.
string page_token = 5;
// --- Filter parity with DiscoverHierarchy. AND-combined. ---
repeated int32 category_ids = 6;
repeated string template_chain_contains = 7;
string tag_name_glob = 8;
optional bool include_attributes = 9;
bool alarm_bearing_only = 10;
bool historized_only = 11;
}
message BrowseChildrenReply {
// Direct children matching the filter, sorted areas-first then by
// case-insensitive display name (same order as the dashboard tree).
repeated GalaxyObject children = 1;
// Non-empty when another page of siblings is available.
string next_page_token = 2;
// Total matching direct children of the parent (post-filter).
int32 total_child_count = 3;
// Parallel array, indexed with `children`. True when the child has at least
// one matching descendant under the same filter set. Lets a UI choose
// whether to draw an expand triangle without an extra round trip.
repeated bool child_has_children = 4;
// Cache sequence this reply was projected from. Clients may pass it back as
// part of the page_token contract. Mismatch on the next page -> InvalidArgument.
uint64 cache_sequence = 5;
}
@@ -0,0 +1,30 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<IsPackable>true</IsPackable>
<PackageId>ZB.MOM.WW.GalaxyRepository</PackageId>
<Authors>ZB.MOM.WW</Authors>
<Description>Read-only Galaxy object-hierarchy browse library for the ZB.MOM.WW SCADA family. Provides a SQL provider for the Galaxy Repository database and a canonical gRPC service for exposing the hierarchy to modern .NET 10 clients — extracted from MxAccessGateway so any consumer can browse the Galaxy without loading 32-bit COM.</Description>
<PackageTags>galaxy;repository;browse;aveva;wonderware;system-platform;scada;grpc;sql;zb-mom-ww</PackageTags>
<PackageProjectUrl>https://gitea.dohertylan.com/dohertj2/zb-mom-ww-galaxyrepository</PackageProjectUrl>
<RepositoryUrl>https://gitea.dohertylan.com/dohertj2/zb-mom-ww-galaxyrepository</RepositoryUrl>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Data.SqlClient" />
<PackageReference Include="Grpc.AspNetCore" />
<PackageReference Include="Google.Protobuf" />
<PackageReference Include="Microsoft.Extensions.Hosting.Abstractions" />
<PackageReference Include="Microsoft.Extensions.Options.ConfigurationExtensions" />
<PackageReference Include="Grpc.Tools">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
</ItemGroup>
<!-- Proto files are added in Task 2; the empty glob is intentional and builds cleanly. -->
<ItemGroup>
<Protobuf Include="Protos\*.proto" GrpcServices="Server" />
</ItemGroup>
</Project>
@@ -0,0 +1,134 @@
using System.Runtime.CompilerServices;
using ZB.MOM.WW.GalaxyRepository;
namespace ZB.MOM.WW.GalaxyRepository.Tests;
/// <summary>
/// In-memory <see cref="IGalaxyRepository"/> returning canned rowsets. Counts the heavy
/// hierarchy/attribute reads so tests can assert deploy-gated skips, and can be flipped to
/// throw so the failure path is exercisable.
/// </summary>
internal sealed class FakeGalaxyRepository : IGalaxyRepository
{
private readonly IReadOnlyList<GalaxyHierarchyRow> _hierarchy;
private readonly IReadOnlyList<GalaxyAttributeRow> _attributes;
public FakeGalaxyRepository(
IReadOnlyList<GalaxyHierarchyRow> hierarchy,
IReadOnlyList<GalaxyAttributeRow> attributes,
DateTime? deployTime)
{
_hierarchy = hierarchy;
_attributes = attributes;
DeployTime = deployTime;
}
/// <summary>The deploy time returned by <see cref="GetLastDeployTimeAsync"/>; mutate to simulate a redeploy.</summary>
public DateTime? DeployTime { get; set; }
/// <summary>When set, every query throws this exception (simulates an unreachable database).</summary>
public Exception? ThrowOnQuery { get; set; }
public int HierarchyReadCount { get; private set; }
public int AttributeReadCount { get; private set; }
public Task<bool> TestConnectionAsync(CancellationToken ct = default) =>
ThrowOnQuery is null ? Task.FromResult(true) : throw ThrowOnQuery;
public Task<DateTime?> GetLastDeployTimeAsync(CancellationToken ct = default)
{
if (ThrowOnQuery is not null)
{
throw ThrowOnQuery;
}
return Task.FromResult(DeployTime);
}
public Task<List<GalaxyHierarchyRow>> GetHierarchyAsync(CancellationToken ct = default)
{
if (ThrowOnQuery is not null)
{
throw ThrowOnQuery;
}
HierarchyReadCount++;
return Task.FromResult(_hierarchy.ToList());
}
public Task<List<GalaxyAttributeRow>> GetAttributesAsync(CancellationToken ct = default)
{
if (ThrowOnQuery is not null)
{
throw ThrowOnQuery;
}
AttributeReadCount++;
return Task.FromResult(_attributes.ToList());
}
}
/// <summary>Records published deploy events so tests can assert publication.</summary>
internal sealed class RecordingDeployNotifier : IGalaxyDeployNotifier
{
public List<GalaxyDeployEventInfo> Published { get; } = [];
public GalaxyDeployEventInfo? Latest { get; private set; }
public void Publish(GalaxyDeployEventInfo info)
{
Published.Add(info);
Latest = info;
}
public async IAsyncEnumerable<GalaxyDeployEventInfo> SubscribeAsync(
[EnumeratorCancellation] CancellationToken cancellationToken)
{
if (Latest is { } latest)
{
yield return latest;
}
await Task.CompletedTask.ConfigureAwait(false);
}
}
/// <summary>
/// In-memory <see cref="IGalaxyHierarchySnapshotStore"/>. Pre-seed <see cref="Snapshot"/>
/// to exercise the restore path; reads <see cref="SaveAsync"/> back to assert persistence.
/// </summary>
internal sealed class FakeSnapshotStore : IGalaxyHierarchySnapshotStore
{
public GalaxyHierarchySnapshot? Snapshot { get; set; }
public int SaveCount { get; private set; }
public int LoadCount { get; private set; }
public Task SaveAsync(GalaxyHierarchySnapshot snapshot, CancellationToken cancellationToken)
{
SaveCount++;
Snapshot = snapshot;
return Task.CompletedTask;
}
public Task<GalaxyHierarchySnapshot?> TryLoadAsync(CancellationToken cancellationToken)
{
LoadCount++;
return Task.FromResult(Snapshot);
}
}
/// <summary>
/// A <see cref="TimeProvider"/> whose UTC clock is fixed (and advanceable) so the cache's
/// staleness projection (which fires after a 5-minute threshold) is deterministic.
/// </summary>
internal sealed class StubTimeProvider(DateTimeOffset start) : TimeProvider
{
private DateTimeOffset _now = start;
public override DateTimeOffset GetUtcNow() => _now;
public void Advance(TimeSpan delta) => _now += delta;
}
@@ -0,0 +1,236 @@
using ZB.MOM.WW.GalaxyRepository;
namespace ZB.MOM.WW.GalaxyRepository.Tests;
/// <summary>
/// Tests for <see cref="GalaxyHierarchyCache"/> first-load, deploy-gating, snapshot
/// restore, persistence, and status-transition behavior. Uses an in-memory
/// <see cref="IGalaxyRepository"/> and snapshot store plus a fixed
/// <see cref="StubTimeProvider"/> so no SQL is touched and no asserts are time-sensitive.
/// </summary>
public sealed class GalaxyHierarchyCacheTests
{
private static readonly DateTimeOffset FixedNow = new(2026, 1, 1, 12, 0, 0, TimeSpan.Zero);
private static readonly DateTime DeployTime = new(2026, 1, 1, 0, 0, 0, DateTimeKind.Utc);
private static List<GalaxyHierarchyRow> SampleHierarchy() =>
[
new() { GobjectId = 1, TagName = "Area1", ContainedName = "Area1", BrowseName = "Area1", IsArea = true },
new() { GobjectId = 2, TagName = "Pump01", ContainedName = "Pump01", BrowseName = "Pump01", ParentGobjectId = 1 },
];
private static List<GalaxyAttributeRow> SampleAttributes() =>
[
new() { GobjectId = 2, AttributeName = "PV", FullTagReference = "Pump01.PV", IsHistorized = true, IsAlarm = true },
];
[Fact]
public async Task RefreshAsync_FirstLoad_PopulatesCurrentWithDataAndUnblocksWaitForFirstLoad()
{
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime);
RecordingDeployNotifier notifier = new();
using GalaxyHierarchyCache cache = new(repository, notifier, new StubTimeProvider(FixedNow));
// Before refresh, the gate is unset and there is no data.
Assert.False(cache.Current.HasData);
Assert.Equal(GalaxyCacheStatus.Unknown, cache.Current.Status);
await cache.RefreshAsync(CancellationToken.None);
// First load completes (does not hang) and Current now holds usable data.
await cache.WaitForFirstLoadAsync(new CancellationTokenSource(TimeSpan.FromSeconds(5)).Token);
GalaxyHierarchyCacheEntry current = cache.Current;
Assert.True(current.HasData);
Assert.Equal(GalaxyCacheStatus.Healthy, current.Status);
Assert.Equal(2, current.ObjectCount);
Assert.Equal(1, current.AreaCount);
Assert.Equal(1, current.AttributeCount);
Assert.Equal(1, current.HistorizedAttributeCount);
Assert.Equal(1, current.AlarmAttributeCount);
// The heavy queries ran exactly once and a deploy event was published.
Assert.Equal(1, repository.HierarchyReadCount);
Assert.Equal(1, repository.AttributeReadCount);
GalaxyDeployEventInfo published = Assert.Single(notifier.Published);
Assert.Equal(2, published.ObjectCount);
Assert.Equal(1, published.AttributeCount);
}
[Fact]
public async Task RefreshAsync_NoDeployChange_SkipsHeavyQueriesOnSecondRefresh()
{
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime);
using GalaxyHierarchyCache cache = new(repository, new RecordingDeployNotifier(), new StubTimeProvider(FixedNow));
await cache.RefreshAsync(CancellationToken.None);
await cache.RefreshAsync(CancellationToken.None);
// Deploy time unchanged => the heavy hierarchy/attribute reads happened only once.
Assert.Equal(1, repository.HierarchyReadCount);
Assert.Equal(1, repository.AttributeReadCount);
Assert.True(cache.Current.HasData);
Assert.Equal(GalaxyCacheStatus.Healthy, cache.Current.Status);
}
[Fact]
public async Task RefreshAsync_DeployAdvances_RebuildsAndBumpsSequence()
{
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime);
RecordingDeployNotifier notifier = new();
using GalaxyHierarchyCache cache = new(repository, notifier, new StubTimeProvider(FixedNow));
await cache.RefreshAsync(CancellationToken.None);
long firstSequence = cache.Current.Sequence;
repository.DeployTime = DeployTime.AddHours(1);
await cache.RefreshAsync(CancellationToken.None);
Assert.Equal(2, repository.HierarchyReadCount);
Assert.Equal(firstSequence + 1, cache.Current.Sequence);
Assert.Equal(2, notifier.Published.Count);
}
[Fact]
public async Task RefreshAsync_FirstQueryFailsNoPriorData_StatusUnavailableButFirstLoadStillCompletes()
{
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime)
{
ThrowOnQuery = new TimeoutException("galaxy db unreachable"),
};
using GalaxyHierarchyCache cache = new(repository, new RecordingDeployNotifier(), new StubTimeProvider(FixedNow));
await cache.RefreshAsync(CancellationToken.None);
// First load must complete so callers do not hang, even though the query failed.
await cache.WaitForFirstLoadAsync(new CancellationTokenSource(TimeSpan.FromSeconds(5)).Token);
Assert.False(cache.Current.HasData);
Assert.Equal(GalaxyCacheStatus.Unavailable, cache.Current.Status);
Assert.Contains("unreachable", cache.Current.LastError);
}
[Fact]
public async Task RefreshAsync_QueryFailsAfterPriorData_DegradesToStaleAndKeepsData()
{
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime);
using GalaxyHierarchyCache cache = new(repository, new RecordingDeployNotifier(), new StubTimeProvider(FixedNow));
await cache.RefreshAsync(CancellationToken.None);
Assert.True(cache.Current.HasData);
// A later refresh fails: data is retained but flagged Stale.
repository.DeployTime = DeployTime.AddHours(1);
repository.ThrowOnQuery = new InvalidOperationException("transient");
await cache.RefreshAsync(CancellationToken.None);
Assert.True(cache.Current.HasData);
Assert.Equal(GalaxyCacheStatus.Stale, cache.Current.Status);
Assert.Equal(2, cache.Current.ObjectCount);
}
[Fact]
public async Task Current_AfterStalenessThreshold_ProjectsHealthyToStale()
{
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime);
StubTimeProvider clock = new(FixedNow);
using GalaxyHierarchyCache cache = new(repository, new RecordingDeployNotifier(), clock);
await cache.RefreshAsync(CancellationToken.None);
Assert.Equal(GalaxyCacheStatus.Healthy, cache.Current.Status);
// Advance past the 5-minute staleness threshold with no successful refresh.
clock.Advance(TimeSpan.FromMinutes(6));
Assert.Equal(GalaxyCacheStatus.Stale, cache.Current.Status);
// Data is still present — Stale means "old", not "gone".
Assert.True(cache.Current.HasData);
}
[Fact]
public async Task RefreshAsync_PersistsSnapshotAfterSuccessfulHeavyRefresh()
{
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime);
FakeSnapshotStore store = new();
using GalaxyHierarchyCache cache = new(
repository, new RecordingDeployNotifier(), new StubTimeProvider(FixedNow), logger: null, snapshotStore: store);
await cache.RefreshAsync(CancellationToken.None);
Assert.Equal(1, store.SaveCount);
Assert.NotNull(store.Snapshot);
Assert.Equal(2, store.Snapshot!.Hierarchy.Count);
Assert.Single(store.Snapshot.Attributes);
}
[Fact]
public async Task RefreshAsync_SnapshotRestore_ServesLastKnownDataAsStaleWhenDatabaseUnreachable()
{
// The snapshot store already holds a persisted dataset (last-known browse data).
FakeSnapshotStore store = new()
{
Snapshot = new GalaxyHierarchySnapshot(
LastDeployTime: DeployTime,
SavedAt: FixedNow.AddMinutes(-1),
Hierarchy: SampleHierarchy(),
Attributes: SampleAttributes()),
};
// The Galaxy database is unreachable on this cold start.
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime)
{
ThrowOnQuery = new TimeoutException("cold start, db down"),
};
RecordingDeployNotifier notifier = new();
using GalaxyHierarchyCache cache = new(
repository, notifier, new StubTimeProvider(FixedNow), logger: null, snapshotStore: store);
await cache.RefreshAsync(CancellationToken.None);
// First load is satisfied by the restored snapshot, not by SQL.
await cache.WaitForFirstLoadAsync(new CancellationTokenSource(TimeSpan.FromSeconds(5)).Token);
Assert.Equal(1, store.LoadCount);
GalaxyHierarchyCacheEntry current = cache.Current;
Assert.True(current.HasData);
// Restored data is "last-known", surfaced as Stale until the live DB confirms.
Assert.Equal(GalaxyCacheStatus.Stale, current.Status);
Assert.Equal(2, current.ObjectCount);
Assert.Equal(DeployTime, current.LastDeployTime!.Value.UtcDateTime);
// A deploy event was published for the restored data.
Assert.Single(notifier.Published);
}
[Fact]
public async Task RefreshAsync_SnapshotRestoreThenLiveQuery_PromotesRestoredDataToHealthy()
{
FakeSnapshotStore store = new()
{
Snapshot = new GalaxyHierarchySnapshot(
LastDeployTime: DeployTime,
SavedAt: FixedNow.AddMinutes(-1),
Hierarchy: SampleHierarchy(),
Attributes: SampleAttributes()),
};
// DB is reachable and reports the SAME deploy time the snapshot was pulled at.
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime);
using GalaxyHierarchyCache cache = new(
repository, new RecordingDeployNotifier(), new StubTimeProvider(FixedNow), logger: null, snapshotStore: store);
await cache.RefreshAsync(CancellationToken.None);
// Restore seeds Stale data; the same-deploy live query promotes it to Healthy
// without re-running the heavy hierarchy/attribute reads.
Assert.Equal(GalaxyCacheStatus.Healthy, cache.Current.Status);
Assert.Equal(0, repository.HierarchyReadCount);
Assert.True(cache.Current.HasData);
}
[Fact]
public void Dispose_CanBeCalledWithoutHavingRefreshed()
{
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime);
GalaxyHierarchyCache cache = new(repository, new RecordingDeployNotifier(), new StubTimeProvider(FixedNow));
// Dispose must be safe even when no refresh ever ran (semaphore never entered).
cache.Dispose();
}
}
@@ -0,0 +1,458 @@
using Grpc.Core;
using ZB.MOM.WW.GalaxyRepository;
using ZB.MOM.WW.GalaxyRepository.Grpc;
namespace ZB.MOM.WW.GalaxyRepository.Tests;
/// <summary>
/// Pure-logic tests for <see cref="GalaxyHierarchyProjector"/> and
/// <see cref="GalaxyBrowseProjector"/>. No SQL: the cache entry under test is built
/// from a small hand-made hierarchy through the same materialization the live cache
/// uses (a fake <see cref="IGalaxyRepository"/> driven through
/// <see cref="GalaxyHierarchyCache.RefreshAsync"/>), so the projectors are exercised
/// against a real <see cref="GalaxyHierarchyIndex"/>.
/// </summary>
public sealed class GalaxyHierarchyProjectorTests
{
/// <summary>
/// Builds a realistic cache entry by driving a fake repository through the cache's
/// own refresh path. This goes through <c>BuildEntry</c> + <see cref="GalaxyHierarchyIndex.Build"/>
/// exactly as production does, rather than reaching for an internal factory.
/// </summary>
private static GalaxyHierarchyCacheEntry BuildEntry(
IReadOnlyList<GalaxyHierarchyRow> hierarchy,
IReadOnlyList<GalaxyAttributeRow> attributes)
{
FakeGalaxyRepository repository = new(hierarchy, attributes, deployTime: new DateTime(2026, 1, 1, 0, 0, 0, DateTimeKind.Utc));
using GalaxyHierarchyCache cache = new(repository, new RecordingDeployNotifier());
cache.RefreshAsync(CancellationToken.None).GetAwaiter().GetResult();
GalaxyHierarchyCacheEntry entry = cache.Current;
Assert.True(entry.HasData);
return entry;
}
// A small but representative galaxy:
// PlantArea (area, id 1)
// ├─ LineA (area, id 2)
// │ ├─ Pump01 (id 10, template "Pump", historized+alarm attr)
// │ └─ Valve01 (id 11, template "Valve", plain attr)
// └─ Mixer01 (id 12, template "Mixer", alarm attr only)
// StandaloneTank (id 20, no parent — a root object)
private static GalaxyHierarchyCacheEntry BuildSampleEntry()
{
List<GalaxyHierarchyRow> hierarchy =
[
Hierarchy(1, "PlantArea", parent: 0, isArea: true, category: 100),
Hierarchy(2, "LineA", parent: 1, isArea: true, category: 100),
Hierarchy(10, "Pump01", parent: 2, category: 200, templates: ["$Pump", "$UserDefined"]),
Hierarchy(11, "Valve01", parent: 2, category: 201, templates: ["$Valve"]),
Hierarchy(12, "Mixer01", parent: 1, category: 202, templates: ["$Mixer"]),
Hierarchy(20, "StandaloneTank", parent: 0, category: 203, templates: ["$Tank"]),
];
List<GalaxyAttributeRow> attributes =
[
// Pump01: historized AND alarm-bearing.
Attribute(10, "Pump01.PV", historized: true, alarm: true),
Attribute(10, "Pump01.SP", historized: false, alarm: false),
// Valve01: plain.
Attribute(11, "Valve01.Cmd", historized: false, alarm: false),
// Mixer01: alarm only.
Attribute(12, "Mixer01.Fault", historized: false, alarm: true),
// StandaloneTank: historized only.
Attribute(20, "StandaloneTank.Level", historized: true, alarm: false),
];
return BuildEntry(hierarchy, attributes);
}
private static GalaxyHierarchyRow Hierarchy(
int id,
string tagName,
int parent,
bool isArea = false,
int category = 0,
IReadOnlyList<string>? templates = null) => new()
{
GobjectId = id,
TagName = tagName,
ContainedName = tagName,
BrowseName = tagName,
ParentGobjectId = parent,
IsArea = isArea,
CategoryId = category,
TemplateChain = templates ?? Array.Empty<string>(),
};
private static GalaxyAttributeRow Attribute(
int gobjectId,
string fullTagReference,
bool historized,
bool alarm) => new()
{
GobjectId = gobjectId,
AttributeName = fullTagReference.Split('.')[^1],
FullTagReference = fullTagReference,
IsHistorized = historized,
IsAlarm = alarm,
};
[Fact]
public void Project_NoFilters_ReturnsEveryObject()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, new DiscoverHierarchyRequest());
Assert.Equal(6, result.TotalObjectCount);
Assert.Equal(6, result.Objects.Count);
}
[Fact]
public void Project_PageSizeAndOffset_SlicesTheOrderedResult()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
DiscoverHierarchyRequest request = new();
GalaxyHierarchyQueryResult full = GalaxyHierarchyProjector.Project(entry, request, browseSubtreeGlobs: null, offset: 0, pageSize: int.MaxValue);
GalaxyHierarchyQueryResult page1 = GalaxyHierarchyProjector.Project(entry, request, browseSubtreeGlobs: null, offset: 0, pageSize: 2);
GalaxyHierarchyQueryResult page2 = GalaxyHierarchyProjector.Project(entry, request, browseSubtreeGlobs: null, offset: 2, pageSize: 2);
GalaxyHierarchyQueryResult page3 = GalaxyHierarchyProjector.Project(entry, request, browseSubtreeGlobs: null, offset: 4, pageSize: 2);
// Total is unaffected by paging.
Assert.Equal(6, page1.TotalObjectCount);
Assert.Equal(2, page1.Objects.Count);
Assert.Equal(2, page2.Objects.Count);
Assert.Equal(2, page3.Objects.Count);
// The three pages reconstruct the full ordered result with no gaps/dupes.
List<int> paged =
[
.. page1.Objects.Select(o => o.GobjectId),
.. page2.Objects.Select(o => o.GobjectId),
.. page3.Objects.Select(o => o.GobjectId),
];
Assert.Equal(full.Objects.Select(o => o.GobjectId), paged);
}
[Fact]
public void Project_OffsetPastEnd_ReturnsEmptyPageButRealTotal()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(
entry, new DiscoverHierarchyRequest(), browseSubtreeGlobs: null, offset: 999, pageSize: 10);
Assert.Empty(result.Objects);
Assert.Equal(6, result.TotalObjectCount);
}
[Fact]
public void Project_PageSignature_IsStableAcrossPagesAndMatchesComputeFilterSignature()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
DiscoverHierarchyRequest request = new() { TagNameGlob = "Pump*" };
string expected = GalaxyHierarchyProjector.ComputeFilterSignature(request, browseSubtreeGlobs: null);
GalaxyHierarchyQueryResult page1 = GalaxyHierarchyProjector.Project(entry, request, browseSubtreeGlobs: null, offset: 0, pageSize: 1);
GalaxyHierarchyQueryResult page2 = GalaxyHierarchyProjector.Project(entry, request, browseSubtreeGlobs: null, offset: 1, pageSize: 1);
// The signature a caller computes to mint a page token round-trips: the projector
// reports the same signature on every page of the same filter set.
Assert.Equal(expected, page1.FilterSignature);
Assert.Equal(expected, page2.FilterSignature);
}
[Fact]
public void ComputeFilterSignature_DiffersWhenAnyFilterChanges()
{
DiscoverHierarchyRequest baseRequest = new() { TagNameGlob = "Pump*" };
DiscoverHierarchyRequest differentGlob = new() { TagNameGlob = "Valve*" };
DiscoverHierarchyRequest differentAlarm = new() { TagNameGlob = "Pump*", AlarmBearingOnly = true };
string baseSig = GalaxyHierarchyProjector.ComputeFilterSignature(baseRequest, null);
Assert.NotEqual(baseSig, GalaxyHierarchyProjector.ComputeFilterSignature(differentGlob, null));
Assert.NotEqual(baseSig, GalaxyHierarchyProjector.ComputeFilterSignature(differentAlarm, null));
Assert.NotEqual(baseSig, GalaxyHierarchyProjector.ComputeFilterSignature(baseRequest, browseSubtreeGlobs: ["PlantArea/*"]));
// Same inputs => same signature (deterministic).
Assert.Equal(baseSig, GalaxyHierarchyProjector.ComputeFilterSignature(new DiscoverHierarchyRequest { TagNameGlob = "Pump*" }, null));
}
[Fact]
public void Project_MaxDepthZero_FromRoot_ReturnsOnlyTheRoot()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
DiscoverHierarchyRequest request = new() { RootGobjectId = 1, MaxDepth = 0 };
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, request);
GalaxyObject only = Assert.Single(result.Objects);
Assert.Equal(1, only.GobjectId);
}
[Fact]
public void Project_MaxDepthOne_FromRoot_ReturnsRootAndDirectChildrenOnly()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
// PlantArea(1) depth 0; LineA(2) and Mixer01(12) depth 1; Pump01/Valve01 depth 2.
DiscoverHierarchyRequest request = new() { RootGobjectId = 1, MaxDepth = 1 };
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, request);
Assert.Equal([1, 2, 12], result.Objects.Select(o => o.GobjectId).OrderBy(id => id));
}
[Fact]
public void Project_NegativeMaxDepth_ThrowsInvalidArgument()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
DiscoverHierarchyRequest request = new() { MaxDepth = -1 };
RpcException ex = Assert.Throws<RpcException>(() => GalaxyHierarchyProjector.Project(entry, request));
Assert.Equal(StatusCode.InvalidArgument, ex.StatusCode);
}
[Fact]
public void Project_UnknownRoot_ThrowsNotFound()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
DiscoverHierarchyRequest request = new() { RootGobjectId = 99999 };
RpcException ex = Assert.Throws<RpcException>(() => GalaxyHierarchyProjector.Project(entry, request));
Assert.Equal(StatusCode.NotFound, ex.StatusCode);
}
[Fact]
public void Project_HistorizedOnly_ReturnsOnlyObjectsWithAHistorizedAttribute()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
DiscoverHierarchyRequest request = new() { HistorizedOnly = true };
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, request);
// Pump01(10) and StandaloneTank(20) carry historized attributes.
Assert.Equal([10, 20], result.Objects.Select(o => o.GobjectId).OrderBy(id => id));
}
[Fact]
public void Project_AlarmBearingOnly_ReturnsOnlyObjectsWithAnAlarmAttribute()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
DiscoverHierarchyRequest request = new() { AlarmBearingOnly = true };
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, request);
// Pump01(10) and Mixer01(12) carry alarm attributes.
Assert.Equal([10, 12], result.Objects.Select(o => o.GobjectId).OrderBy(id => id));
}
[Fact]
public void Project_AlarmAndHistorizedTogether_RequiresBoth()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
DiscoverHierarchyRequest request = new() { AlarmBearingOnly = true, HistorizedOnly = true };
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, request);
// Only Pump01(10) carries an attribute set that is both historized and alarm-bearing.
GalaxyObject only = Assert.Single(result.Objects);
Assert.Equal(10, only.GobjectId);
}
[Fact]
public void Project_TagNameGlob_MatchesAnchoredCaseInsensitive()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
GalaxyHierarchyQueryResult prefix = GalaxyHierarchyProjector.Project(entry, new DiscoverHierarchyRequest { TagNameGlob = "Pump*" });
Assert.Equal([10], prefix.Objects.Select(o => o.GobjectId));
// Case-insensitive.
GalaxyHierarchyQueryResult lower = GalaxyHierarchyProjector.Project(entry, new DiscoverHierarchyRequest { TagNameGlob = "pump01" });
Assert.Equal([10], lower.Objects.Select(o => o.GobjectId));
// '?' single-char wildcard: "Pump0?" matches "Pump01".
GalaxyHierarchyQueryResult single = GalaxyHierarchyProjector.Project(entry, new DiscoverHierarchyRequest { TagNameGlob = "Pump0?" });
Assert.Equal([10], single.Objects.Select(o => o.GobjectId));
// Anchored: a bare substring that is not a prefix matches nothing.
GalaxyHierarchyQueryResult anchored = GalaxyHierarchyProjector.Project(entry, new DiscoverHierarchyRequest { TagNameGlob = "ump01" });
Assert.Empty(anchored.Objects);
}
[Fact]
public void Project_CategoryIds_FilterByObjectCategory()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
DiscoverHierarchyRequest request = new() { CategoryIds = { 200, 201 } };
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, request);
// category 200 = Pump01(10), category 201 = Valve01(11).
Assert.Equal([10, 11], result.Objects.Select(o => o.GobjectId).OrderBy(id => id));
}
[Fact]
public void Project_TemplateChainContains_IsSubstringAndCaseInsensitive()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
DiscoverHierarchyRequest request = new() { TemplateChainContains = { "pump" } };
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, request);
GalaxyObject only = Assert.Single(result.Objects);
Assert.Equal(10, only.GobjectId);
}
[Fact]
public void Project_IncludeAttributesDefault_CarriesAttributes()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
DiscoverHierarchyRequest request = new() { TagNameGlob = "Pump*" };
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, request);
GalaxyObject pump = Assert.Single(result.Objects);
Assert.Equal(2, pump.Attributes.Count);
}
[Fact]
public void Project_IncludeAttributesFalse_ReturnsSkeletons()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
DiscoverHierarchyRequest request = new() { TagNameGlob = "Pump*", IncludeAttributes = false };
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, request);
GalaxyObject pump = Assert.Single(result.Objects);
Assert.Empty(pump.Attributes);
}
[Fact]
public void Project_IncludeAttributesFalse_DoesNotMutateTheCachedEntry()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
// Project with attributes stripped, then again with attributes included.
GalaxyHierarchyProjector.Project(entry, new DiscoverHierarchyRequest { TagNameGlob = "Pump*", IncludeAttributes = false });
GalaxyHierarchyQueryResult included = GalaxyHierarchyProjector.Project(entry, new DiscoverHierarchyRequest { TagNameGlob = "Pump*" });
// The earlier strip cloned the object — the cached entry still holds the attributes.
GalaxyObject pump = Assert.Single(included.Objects);
Assert.Equal(2, pump.Attributes.Count);
}
[Fact]
public void Project_InvalidOffsetOrPageSize_Throws()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
Assert.Throws<ArgumentOutOfRangeException>(() =>
GalaxyHierarchyProjector.Project(entry, new DiscoverHierarchyRequest(), null, offset: -1, pageSize: 10));
Assert.Throws<ArgumentOutOfRangeException>(() =>
GalaxyHierarchyProjector.Project(entry, new DiscoverHierarchyRequest(), null, offset: 0, pageSize: 0));
}
// ---- GalaxyBrowseProjector ----
[Fact]
public void ProjectChildren_OfPlantArea_ReturnsDirectChildrenAreasFirst()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
BrowseChildrenRequest request = new() { ParentGobjectId = 1 };
GalaxyBrowseChildrenResult result = GalaxyBrowseProjector.ProjectChildren(entry, request, browseSubtreeGlobs: null, offset: 0, pageSize: 100);
// Direct children of PlantArea(1) are LineA(2, area) and Mixer01(12, non-area);
// areas sort first.
Assert.Equal([2, 12], result.Children.Select(c => c.GobjectId));
Assert.Equal(2, result.TotalChildCount);
}
[Fact]
public void ProjectChildren_ChildHasChildrenFlag_ReflectsPresenceOfChildren()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
BrowseChildrenRequest request = new() { ParentGobjectId = 1 };
GalaxyBrowseChildrenResult result = GalaxyBrowseProjector.ProjectChildren(entry, request, browseSubtreeGlobs: null, offset: 0, pageSize: 100);
Dictionary<int, bool> hasChildren = result.Children
.Select((child, index) => (child.GobjectId, result.ChildHasChildren[index]))
.ToDictionary(t => t.GobjectId, t => t.Item2);
// LineA(2) contains Pump01/Valve01 -> true; Mixer01(12) is a leaf -> false.
Assert.True(hasChildren[2]);
Assert.False(hasChildren[12]);
}
[Fact]
public void ProjectChildren_OfRoot_ReturnsTopLevelObjects()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
// Empty parent oneof => roots (parent id 0).
BrowseChildrenRequest request = new();
GalaxyBrowseChildrenResult result = GalaxyBrowseProjector.ProjectChildren(entry, request, browseSubtreeGlobs: null, offset: 0, pageSize: 100);
// Roots: PlantArea(1, area) and StandaloneTank(20, non-area); areas first.
Assert.Equal([1, 20], result.Children.Select(c => c.GobjectId));
}
[Fact]
public void ProjectChildren_FilterMatchingDescendant_SurfacesNonMatchingAncestor()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
// Pump01 lives two levels under PlantArea. Browsing PlantArea's children with a
// Pump glob should still surface LineA (which itself does not match) because it
// contains a matching descendant.
BrowseChildrenRequest request = new() { ParentGobjectId = 1, TagNameGlob = "Pump*" };
GalaxyBrowseChildrenResult result = GalaxyBrowseProjector.ProjectChildren(entry, request, browseSubtreeGlobs: null, offset: 0, pageSize: 100);
GalaxyObject surfaced = Assert.Single(result.Children);
Assert.Equal(2, surfaced.GobjectId);
Assert.True(result.ChildHasChildren[0]);
}
[Fact]
public void ProjectChildren_UnknownParent_ThrowsNotFound()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
BrowseChildrenRequest request = new() { ParentGobjectId = 99999 };
RpcException ex = Assert.Throws<RpcException>(() =>
GalaxyBrowseProjector.ProjectChildren(entry, request, null, 0, 100));
Assert.Equal(StatusCode.NotFound, ex.StatusCode);
}
[Fact]
public void ProjectChildren_Paging_SlicesAndPreservesTotal()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
// LineA(2) has two direct children: Pump01, Valve01.
BrowseChildrenRequest request = new() { ParentGobjectId = 2 };
GalaxyBrowseChildrenResult page1 = GalaxyBrowseProjector.ProjectChildren(entry, request, null, offset: 0, pageSize: 1);
GalaxyBrowseChildrenResult page2 = GalaxyBrowseProjector.ProjectChildren(entry, request, null, offset: 1, pageSize: 1);
Assert.Equal(2, page1.TotalChildCount);
Assert.Single(page1.Children);
Assert.Single(page2.Children);
Assert.NotEqual(page1.Children[0].GobjectId, page2.Children[0].GobjectId);
// Same filter+parent => same signature on both pages.
Assert.Equal(page1.FilterSignature, page2.FilterSignature);
}
[Fact]
public void ResolveParentId_ByTagName_ResolvesToGobjectId()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
BrowseChildrenRequest request = new() { ParentTagName = "LineA" };
int id = GalaxyBrowseProjector.ResolveParentId(entry, request);
Assert.Equal(2, id);
}
}
@@ -0,0 +1,84 @@
using Microsoft.Extensions.Options;
using ZB.MOM.WW.GalaxyRepository;
namespace ZB.MOM.WW.GalaxyRepository.Tests;
/// <summary>
/// Round-trip tests for the real <see cref="GalaxyHierarchySnapshotStore"/> over a temp
/// file path: save then load, no-op when persistence is disabled, and clean disposal.
/// </summary>
public sealed class GalaxyHierarchySnapshotStoreTests : IDisposable
{
private readonly string _path = Path.Combine(
Path.GetTempPath(),
$"galaxyrepo-snap-{Guid.NewGuid():N}.json");
public void Dispose()
{
if (File.Exists(_path))
{
File.Delete(_path);
}
}
private static GalaxyHierarchySnapshot SampleSnapshot() => new(
LastDeployTime: new DateTimeOffset(2026, 1, 1, 0, 0, 0, TimeSpan.Zero),
SavedAt: new DateTimeOffset(2026, 1, 1, 12, 0, 0, TimeSpan.Zero),
Hierarchy:
[
new GalaxyHierarchyRow { GobjectId = 1, TagName = "Area1", IsArea = true },
new GalaxyHierarchyRow { GobjectId = 2, TagName = "Pump01", ParentGobjectId = 1 },
],
Attributes:
[
new GalaxyAttributeRow { GobjectId = 2, AttributeName = "PV", FullTagReference = "Pump01.PV", IsHistorized = true },
]);
[Fact]
public async Task SaveThenLoad_RoundTripsTheSnapshot()
{
using GalaxyHierarchySnapshotStore store = new(
Options.Create(new GalaxyRepositoryOptions { PersistSnapshot = true, SnapshotCachePath = _path }));
await store.SaveAsync(SampleSnapshot(), CancellationToken.None);
GalaxyHierarchySnapshot? loaded = await store.TryLoadAsync(CancellationToken.None);
Assert.NotNull(loaded);
Assert.Equal(2, loaded!.Hierarchy.Count);
Assert.Single(loaded.Attributes);
Assert.Equal("Pump01.PV", loaded.Attributes[0].FullTagReference);
Assert.True(loaded.Attributes[0].IsHistorized);
Assert.Equal(SampleSnapshot().LastDeployTime, loaded.LastDeployTime);
}
[Fact]
public async Task SaveAndLoad_AreNoOps_WhenPersistenceDisabled()
{
using GalaxyHierarchySnapshotStore store = new(
Options.Create(new GalaxyRepositoryOptions { PersistSnapshot = false, SnapshotCachePath = _path }));
await store.SaveAsync(SampleSnapshot(), CancellationToken.None);
Assert.False(File.Exists(_path));
Assert.Null(await store.TryLoadAsync(CancellationToken.None));
}
[Fact]
public async Task TryLoad_ReturnsNull_WhenNoFileExists()
{
using GalaxyHierarchySnapshotStore store = new(
Options.Create(new GalaxyRepositoryOptions { PersistSnapshot = true, SnapshotCachePath = _path }));
Assert.Null(await store.TryLoadAsync(CancellationToken.None));
}
[Fact]
public async Task TryLoad_ReturnsNull_WhenFileIsNotValidJson()
{
await File.WriteAllTextAsync(_path, "{ this is not valid json");
using GalaxyHierarchySnapshotStore store = new(
Options.Create(new GalaxyRepositoryOptions { PersistSnapshot = true, SnapshotCachePath = _path }));
Assert.Null(await store.TryLoadAsync(CancellationToken.None));
}
}
@@ -0,0 +1,25 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<IsPackable>false</IsPackable>
<!-- Test project does not ship; no XML docs required (overrides Directory.Build.props). -->
<GenerateDocumentationFile>false</GenerateDocumentationFile>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="coverlet.collector" />
<PackageReference Include="Microsoft.NET.Test.Sdk" />
<PackageReference Include="xunit" />
<PackageReference Include="xunit.runner.visualstudio" />
<PackageReference Include="Microsoft.Data.SqlClient" />
</ItemGroup>
<ItemGroup>
<Using Include="Xunit" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\..\src\ZB.MOM.WW.GalaxyRepository\ZB.MOM.WW.GalaxyRepository.csproj" />
</ItemGroup>
</Project>
+27 -5
View File
@@ -10,6 +10,11 @@ All file references below point at the kit source under `src/ZB.MOM.WW.Theme/`.
> **RESOLVED in kit 0.3.0 (2026-06-05).** Issues 1, 2, 3, and 5 are fixed in the kit and > **RESOLVED in kit 0.3.0 (2026-06-05).** Issues 1, 2, 3, and 5 are fixed in the kit and
> redistributed; Issue 4 is an accepted, documented tradeoff (no code change). See > redistributed; Issue 4 is an accepted, documented tradeoff (no code change). See
> [Resolution](#resolution-kit-030) below for what changed and why. > [Resolution](#resolution-kit-030) below for what changed and why.
>
> **RESOLVED in kit 0.3.1 (2026-06-05).** Issue 6 (collapsible nav non-functional under
> interactive Blazor render) is fixed — CSS `display:none`-when-closed backstop +
> `MutationObserver` re-wire in `nav-state.js`. See the [Issue 6](#issue-6--collapsible-nav-is-non-functional-under-interactive-blazor-render-mode)
> resolution note. All six issues are now resolved (5 fixed, 1 accepted tradeoff).
## Summary ## Summary
@@ -20,7 +25,7 @@ All file references below point at the kit source under `src/ZB.MOM.WW.Theme/`.
| 3 | Medium | `nav-state.js` | Persistence wires once on `DOMContentLoaded`; not re-applied after Blazor enhanced navigation / dynamic re-render. | ✅ Fixed | | 3 | Medium | `nav-state.js` | Persistence wires once on `DOMContentLoaded`; not re-applied after Blazor enhanced navigation / dynamic re-render. | ✅ Fixed |
| 4 | Low | `NavRailSection` | Always-expanded SSR default causes a flash / layout shift of collapsed sections on load. | 📄 Accepted tradeoff (documented) | | 4 | Low | `NavRailSection` | Always-expanded SSR default causes a flash / layout shift of collapsed sections on load. | 📄 Accepted tradeoff (documented) |
| 5 | Low (optional) | `LoginCard` | Heading bakes the localizable `— sign in` suffix into the product title with no separate hook. | ✅ Fixed | | 5 | Low (optional) | `LoginCard` | Heading bakes the localizable `— sign in` suffix into the product title with no separate hook. | ✅ Fixed |
| 6 | High | `NavRailSection` / `nav-state.js` | Under **interactive** Blazor render mode the whole collapsible nav is non-functional: clicking a header doesn't hide items, and `nav-state.js` never wires (no aria sync, no persistence, no active-reveal). | ❌ Open (found in 0.3.0) | | 6 | High | `NavRailSection` / `nav-state.js` | Under **interactive** Blazor render mode the whole collapsible nav is non-functional: clicking a header doesn't hide items, and `nav-state.js` never wires (no aria sync, no persistence, no active-reveal). | ✅ Fixed (0.3.1) |
--- ---
@@ -234,7 +239,22 @@ or add an optional `Heading` parameter that, when set, replaces the default head
## Issue 6 — Collapsible nav is non-functional under interactive Blazor render mode ## Issue 6 — Collapsible nav is non-functional under interactive Blazor render mode
**Severity:** High · **Files:** `Components/NavRailSection.razor`, `wwwroot/js/nav-state.js`, **Severity:** High · **Files:** `Components/NavRailSection.razor`, `wwwroot/js/nav-state.js`,
`wwwroot/css/layout.css` · **Status:** Open (found in 0.3.0) `wwwroot/css/layout.css` · **Status:** ✅ Fixed in 0.3.1
> **Resolution (kit 0.3.1, 2026-06-05).** Both recommended parts shipped, so the collapsible
> nav now works under interactive render modes as well as static SSR:
> 1. **CSS robust collapse** — `layout.css` hides the body explicitly when closed instead of
> relying on the native `::details-content` content-hiding (which an interactive framework
> desyncs): `.rail-section:not([open]) > .rail-section-body { display: none; }`.
> 2. **Render-mode-agnostic re-wire** — `nav-state.js` adds a `MutationObserver` on
> `document.documentElement` (childList + subtree) that re-runs `apply()` whenever
> `details.rail-section` nodes are added/replaced, so the interactive runtime's re-render
> gets wired (aria sync, `data-zbnav-initialized`, localStorage persistence, active-reveal).
> The existing `enhancedload` hook (Issue 3) is kept for static-SSR enhanced navigation.
>
> Verified live in ScadaBridge Central UI (global `@rendermode InteractiveServer`): the
> Playwright `NavCollapseTests` (toggle-hides-items, persistence-survives-reload,
> deep-link-auto-reveal) now pass against 0.3.1.
> **This corrects Issue 3's note**, which claimed interactive Blazor Server consumers are > **This corrects Issue 3's note**, which claimed interactive Blazor Server consumers are
> "largely unaffected because the rail is patched in place." Direct observation of the live > "largely unaffected because the rail is patched in place." Direct observation of the live
@@ -311,9 +331,11 @@ silently does nothing.
summary's `aria-expanded` flips; `localStorage` gets a `zbnav:<key>` entry; the state survives summary's `aria-expanded` flips; `localStorage` gets a `zbnav:<key>` entry; the state survives
a reload; and deep-linking into a collapsed section reveals it. a reload; and deep-linking into a collapsed section reveals it.
**Consumer note (ScadaBridge).** Until the kit ships this, ScadaBridge's Central UI nav **Consumer note (ScadaBridge).** Resolved on 0.3.1: ScadaBridge's Central UI consumes
collapse is a no-op; the Playwright `NavCollapseTests` that exercise toggling, persistence, and `ZB.MOM.WW.Theme` 0.3.1, and the Playwright `NavCollapseTests` (toggling, persistence,
auto-reveal are therefore testing behavior the app does not currently have. auto-reveal) now pass — the `NavCollapseWiredAsync` gate (which waits for
`data-zbnav-initialized` on every `details.rail-section`) is satisfied under interactive
render, so those tests run unskipped and green.
--- ---
+19317
View File
File diff suppressed because it is too large Load Diff
+56
View File
@@ -0,0 +1,56 @@
Northwind Consumer Products — Unified Namespace
(generated from Galaxy DESKTOP-6JL3KKO\DEV; 40 machines, 1036 signals)
northwind
└─ birmingham
├─ filling/ (Filling & Capping; from Galaxy TestArea)
│ ├─ line-1/
│ │ ├─ rinser-01 [krones Hydra Srs3] ← TestMachine_001 (28 signals)
│ │ ├─ filler-02 [sidel SF300 Srs5] ← TestMachine_002 (28 signals)
│ │ ├─ capper-03 [khs Innofill Srs4] ← TestMachine_003 (28 signals)
│ │ ├─ labeler-04 [krones Contiroll Srs3] ← TestMachine_004 (28 signals)
│ │ ├─ inspector-05 [antares-vision Vmax Srs2] ← TestMachine_005 (28 signals)
│ │ ├─ coder-06 [videojet 1580 Srs4] ← TestMachine_006 (28 signals)
│ │ └─ rinser-07 [krones Hydra Srs3] ← TestMachine_007 (28 signals)
│ ├─ line-2/
│ │ ├─ rinser-08 [krones Hydra Srs3] ← TestMachine_008 (28 signals)
│ │ ├─ filler-09 [sidel SF300 Srs2] ← TestMachine_009 (28 signals)
│ │ ├─ capper-10 [khs Innofill Srs3] ← TestMachine_010 (28 signals)
│ │ ├─ labeler-11 [krones Contiroll Srs4] ← TestMachine_011 (28 signals)
│ │ ├─ inspector-12 [antares-vision Vmax Srs5] ← TestMachine_012 (28 signals)
│ │ └─ coder-13 [videojet 1580 Srs4] ← TestMachine_013 (28 signals)
│ └─ line-3/
│ ├─ rinser-14 [krones Hydra Srs4] ← TestMachine_014 (28 signals)
│ ├─ filler-15 [sidel SF300 Srs4] ← TestMachine_015 (28 signals)
│ ├─ capper-16 [khs Innofill Srs4] ← TestMachine_016 (28 signals)
│ ├─ labeler-17 [krones Contiroll Srs4] ← TestMachine_017 (28 signals)
│ ├─ inspector-18 [antares-vision Vmax Srs4] ← TestMachine_018 (28 signals)
│ └─ coder-19 [videojet 1580 Srs5] ← TestMachine_019 (28 signals)
├─ blending/ (Blending & CIP; from Galaxy TestArea2)
│ └─ cip-1/
│ └─ blender-20 [spx-flow APV-R5 Srs4] ← TestMachine_020 (24 signals)
└─ packaging/ (Packaging & Palletizing; from Galaxy TestArea3)
├─ pack-1/
│ ├─ cartoner-21 [marchesini MC820 Srs2] ← TestMachine_021 (24 signals)
│ ├─ case-packer-22 [bosch Elematic Srs4] ← TestMachine_022 (24 signals)
│ ├─ palletizer-23 [fanuc M410 Srs5] ← TestMachine_023 (24 signals)
│ ├─ stretch-wrapper-24 [lantech Q300 Srs4] ← TestMachine_024 (24 signals)
│ └─ checkweigher-25 [mettler-toledo C3570 Srs2] ← TestMachine_025 (24 signals)
├─ pack-2/
│ ├─ cartoner-26 [marchesini MC820 Srs2] ← TestMachine_026 (24 signals)
│ ├─ case-packer-27 [bosch Elematic Srs5] ← TestMachine_027 (24 signals)
│ ├─ palletizer-28 [fanuc M410 Srs5] ← TestMachine_028 (24 signals)
│ ├─ stretch-wrapper-29 [lantech Q300 Srs4] ← TestMachine_029 (24 signals)
│ └─ checkweigher-30 [mettler-toledo C3570 Srs5] ← TestMachine_030 (24 signals)
├─ pack-3/
│ ├─ cartoner-31 [marchesini MC820 Srs5] ← TestMachine_031 (24 signals)
│ ├─ case-packer-32 [bosch Elematic Srs5] ← TestMachine_032 (24 signals)
│ ├─ palletizer-33 [fanuc M410 Srs5] ← TestMachine_033 (24 signals)
│ ├─ stretch-wrapper-34 [lantech Q300 Srs4] ← TestMachine_034 (24 signals)
│ └─ checkweigher-35 [mettler-toledo C3570 Srs2] ← TestMachine_035 (24 signals)
└─ pack-4/
├─ cartoner-36 [marchesini MC820 Srs4] ← TestMachine_036 (24 signals)
├─ case-packer-37 [bosch Elematic Srs3] ← TestMachine_037 (24 signals)
├─ palletizer-38 [fanuc M410 Srs3] ← TestMachine_038 (24 signals)
├─ stretch-wrapper-39 [lantech Q300 Srs2] ← TestMachine_039 (24 signals)
└─ checkweigher-40 [mettler-toledo C3570 Srs5] ← TestMachine_040 (24 signals)
@@ -0,0 +1,122 @@
# ZB.MOM.WW.SPHistorianClient — Design
**Date:** 2026-06-19
**Status:** Approved — proceeding to implementation plan.
## Goal
Repackage the proven, pure-managed .NET 10 `AVEVA.Historian.Client` SDK (delivered in
`HistorianSDK_2023R2/histsdk-migration.zip` from `10.100.0.48`) as the family-branded shared
library **`ZB.MOM.WW.SPHistorianClient`** (System Platform Historian Client), following the same
conventions as the other `ZB.MOM.WW.*` shared libraries in this repo.
## Context — what the source bundle contains
`histsdk-migration.zip``histsdk-migration/`:
- `histsdk/` — the SDK git repo. `src/AVEVA.Historian.Client/` is a **pure-managed .NET 10** client
for AVEVA Historian (no `aahClientManaged.dll` / `aahClient.dll` / native AVEVA runtime — the wire
protocol is reverse-engineered and re-implemented in C#). ~165188 unit + gated-live tests pass.
- `analysis-2023r2/` — reverse-engineering analysis (recovered protos, decompiled stock contract,
transport writeup). **Kept separate from the repo on purpose.**
Two transport families exist in the SDK:
| Transport | Protocol | Platform | Verification |
|---|---|---|---|
| `LocalPipe`, `RemoteTcpIntegrated`, `RemoteTcpCertificate` | WCF/MDAS (2020) | **Windows-only** | **live-verified**: raw/aggregate(16 modes)/at-time/event reads, browse, metadata, status, `EnsureTag`/`DeleteTag` |
| `RemoteGrpc` | gRPC (2023 R2) | cross-platform (Grpc.Net.Client/.Web) | unit-tested; **not yet live-verified** against a real 2023 R2 server (`ExchangeKey` auth step unproven) |
## Decisions (locked)
1. **Approach: port + rebrand.** Copy the SDK source into `ZB.MOM.WW.SPHistorianClient`, rename the
root namespace, adopt ZB conventions, bring the unit tests, drop non-shippable artifacts. One
coherent shared library — a published package should not ship a third-party (AVEVA) namespace or
non-redistributable reverse-engineering artifacts.
2. **Transports: both WCF + gRPC.** Ship everything that works. WCF members keep
`[SupportedOSPlatform("windows")]`; the gRPC path runs anywhere. No working code discarded.
3. **Not a "component normalization."** There is no duplicated historian code across the three apps
to converge — this is a net-new shared library that simply follows ZB packaging conventions.
## Repository layout
Plain files committed into this repo (NOT a nested git repo — see the
`shared-libs-are-plain-files-not-nested-repos` convention):
```
ZB.MOM.WW.SPHistorianClient/
Directory.Build.props # net10.0, Nullable, ImplicitUsings, LangVersion latest, Version 0.1.0, central pkg mgmt
Directory.Packages.props # central PackageVersion entries
ZB.MOM.WW.SPHistorianClient.slnx
CLAUDE.md README.md .gitignore
src/ZB.MOM.WW.SPHistorianClient/ # the single package
HistorianClient.cs, HistorianClientOptions.cs, HistorianTransport.cs
Models/ Protocol/ Transport/ Wcf/ Wcf/Contracts/ Grpc/ Grpc/Protos/*.proto
DependencyInjection/AddZbSpHistorianClient (ZB-idiomatic DI extension)
tests/ZB.MOM.WW.SPHistorianClient.Tests/ # offline unit/golden-byte + gated-live integration
artifacts/ # dotnet pack output
```
## Port mechanics
- Copy `src/AVEVA.Historian.Client/` and `tests/AVEVA.Historian.Client.Tests/` from the bundle.
- Rename the C# root namespace `AVEVA.Historian.Client``ZB.MOM.WW.SPHistorianClient` across all
files: 74 `namespace` declarations spanning the root + 6 sub-namespaces
(`.Models`, `.Wcf`, `.Wcf.Contracts`, `.Protocol`, `.Transport`, `.Grpc`), all `using` directives,
and the `InternalsVisibleTo` to the test assembly. Drop the `InternalsVisibleTo` to
`AVEVA.Historian.ReverseEngineering` (tool not shipped).
- **Leave the proto wire contracts untouched:** the 6 `Grpc/Protos/*.proto` keep
`option csharp_namespace = "ArchestrA.Grpc.Contract.*"` — that is AVEVA's wire contract, not ours.
`Grpc.Tools` keeps generating the client stubs at build.
- Convert inline `PackageReference` versions to central management in `Directory.Packages.props`,
matching the `ZB.MOM.WW.Telemetry` template.
## Dependencies
- **Library:** `Google.Protobuf`, `Grpc.Net.Client`, `Grpc.Net.Client.Web`, `Grpc.Tools` (build-only,
`PrivateAssets=all`), `System.ServiceModel.NetNamedPipe`, `System.ServiceModel.NetTcp`,
`System.Security.Cryptography.Xml`. Add `Microsoft.Extensions.DependencyInjection.Abstractions` +
`Microsoft.Extensions.Options` for the DI extension.
- **Tests:** `xunit`, `xunit.runner.visualstudio`, `Microsoft.NET.Test.Sdk`, `coverlet.collector`,
`Microsoft.Data.SqlClient` (SQL post-check tests).
## Excluded (safety / non-redistributable / Windows-native)
- `tools/` reverse-engineering harnesses (.NET Framework, reference native AVEVA binaries).
- `analysis-2023r2/decompiled/` — proprietary AVEVA decompilations (not redistributable).
- `scripts/` — Frida / PowerShell / Python capture tooling.
- `docs/reverse-engineering/` — identity-bearing `.ndjson` / capture evidence.
**Kept:** the recovered `.proto` files (needed to build), the offline unit tests, and a sanitized
architecture/surface summary folded into `CLAUDE.md` / `README.md`. `.gitignore` blocks the
identity-bearing patterns (`*.ndjson`, `current/`, `aveva-install-*/`, `artifacts/`-raw, etc.).
## Public surface (preserved 1:1)
`HistorianClient` + `HistorianClientOptions` façade; `Models/*`; `HistorianTransport` enum
(`LocalPipe` / `RemoteTcpIntegrated` / `RemoteTcpCertificate` / `RemoteGrpc`); operations:
`ProbeAsync`, `ReadRawAsync` / `ReadAggregateAsync` / `ReadAtTimeAsync`, `ReadEventsAsync`,
`BrowseTagNamesAsync`, `GetTagMetadataAsync`, status calls, `EnsureTagAsync` / `DeleteTagAsync`.
**One ZB-idiomatic addition:** `AddZbSpHistorianClient(...)` DI extension mirroring `AddZbTelemetry`
— thin: binds `HistorianClientOptions` and registers `HistorianClient`. Optional to consumers.
## Cross-platform & testing posture
- WCF members already carry `[SupportedOSPlatform("windows")]`; the library builds and unit-tests on
macOS/Linux. gRPC path is portable.
- Offline unit/golden-byte tests run anywhere. Live integration tests stay gated by `HISTORIAN_*`
env vars and skip cleanly when unset.
- Verify `dotnet build` + `dotnet test` pass locally (macOS) before finishing.
## Packaging
`dotnet pack -c Release -o ./artifacts``ZB.MOM.WW.SPHistorianClient.0.1.0.nupkg`. Gitea URLs in
package metadata. **Not pushed/published** to any feed unless explicitly requested.
## Out of scope (this pass)
- Wiring `ZB.MOM.WW.SPHistorianClient` into any consumer (e.g. OtOpcUa Phase C HistoryRead) — a
separate follow-on.
- Live-verifying the gRPC `RemoteGrpc` path against a real 2023 R2 server.
- Writing samples (`AddS2`) — architecturally blocked in the source SDK; remains out of scope.
+569
View File
@@ -0,0 +1,569 @@
# ZB.MOM.WW.SPHistorianClient Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans (or subagent-driven-development) to implement this plan task-by-task.
**Goal:** Repackage the proven, pure-managed .NET 10 `AVEVA.Historian.Client` SDK from the migration bundle as the family-branded shared library `ZB.MOM.WW.SPHistorianClient`, following the same conventions as the other `ZB.MOM.WW.*` libraries in this repo.
**Architecture:** This is a **port + rebrand**, not a rewrite. Copy the SDK `src/` and `tests/` into a new `ZB.MOM.WW.SPHistorianClient/` directory, rewrite the C# root namespace `AVEVA.Historian.Client``ZB.MOM.WW.SPHistorianClient` (leaving the proto-generated `ArchestrA.Grpc.Contract.*` wire contracts untouched), adopt ZB conventions (`Directory.Build.props` / `Directory.Packages.props` central package management, `.slnx`, `CLAUDE.md`/`README.md`), drop the non-shippable reverse-engineering tooling and proprietary decompilations, add one ZB-idiomatic DI extension, then build/test/pack.
**Tech Stack:** .NET 10, C# (net10.0), WCF/MDAS (`System.ServiceModel.*`, Windows-only transports), gRPC (`Grpc.Net.Client` + `Grpc.Tools`, cross-platform 2023 R2 transport), xUnit. Central package management.
**Design doc:** `docs/plans/2026-06-19-sphistorianclient-design.md`
**Branch:** `feat/sphistorianclient` (already created; design doc already committed at `bbb7942`).
---
## Source bundle location (read-only inputs)
The SDK source lives in an extracted bundle under `/tmp`:
- Extracted root: `/tmp/histsdk/extracted/histsdk-migration/histsdk/`
- SDK source: `…/histsdk/src/AVEVA.Historian.Client/`**74 `.cs` + 6 `.proto`**
- SDK tests: `…/histsdk/tests/AVEVA.Historian.Client.Tests/`**25 `.cs`**
- Re-extract fallback (if `/tmp` was cleaned): `cd /tmp/histsdk && unzip -o -q histsdk-migration.zip -d extracted`
**Never copy:** `tools/` (RE harnesses, .NET Framework + native AVEVA refs), `analysis-2023r2/decompiled/` (proprietary, non-redistributable), `scripts/`, `docs/reverse-engineering/` (identity-bearing captures), `bin/`/`obj/`, the bundle's `.git/`, and the bundle's original `.csproj` files (we author fresh ZB ones).
**Gotchas baked into this plan (from prior repo experience):**
- Do **not** set `TreatWarningsAsErrors` — the WCF/SSPI code carries `[SupportedOSPlatform("windows")]` and will emit CA platform warnings on macOS that must stay warnings.
- Central package management means **no inline `Version=` on any `PackageReference`** (that is `NU1008`). All versions live in `Directory.Packages.props`.
- `Microsoft.Data.SqlClient` may surface an `NU1903` advisory on restore. Without `TreatWarningsAsErrors` it is a warning. If a restore ever hard-fails on it, add `-p:NuGetAudit=false` to the build/test command.
- macOS `sed -i` requires an explicit empty backup arg: `sed -i '' 's/…/…/g'`.
---
## Task 1: Scaffold the library skeleton
**Classification:** standard
**Estimated implement time:** ~4 min
**Parallelizable with:** none (every later task depends on this)
**Files:**
- Create: `ZB.MOM.WW.SPHistorianClient/Directory.Build.props`
- Create: `ZB.MOM.WW.SPHistorianClient/Directory.Packages.props`
- Create: `ZB.MOM.WW.SPHistorianClient/.gitignore`
- Create: `ZB.MOM.WW.SPHistorianClient/ZB.MOM.WW.SPHistorianClient.slnx`
- Create: `ZB.MOM.WW.SPHistorianClient/src/ZB.MOM.WW.SPHistorianClient/ZB.MOM.WW.SPHistorianClient.csproj`
- Create: `ZB.MOM.WW.SPHistorianClient/tests/ZB.MOM.WW.SPHistorianClient.Tests/ZB.MOM.WW.SPHistorianClient.Tests.csproj`
**Step 1: `Directory.Build.props`** (mirrors `ZB.MOM.WW.Telemetry/Directory.Build.props`)
```xml
<Project>
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<LangVersion>latest</LangVersion>
<Version>0.1.0</Version>
<ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
</PropertyGroup>
</Project>
```
**Step 2: `Directory.Packages.props`** (versions lifted verbatim from the bundle's two `.csproj` files)
```xml
<Project>
<PropertyGroup>
<ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
</PropertyGroup>
<ItemGroup>
<!-- Historian SDK runtime deps (WCF/MDAS transports — Windows-only at runtime) -->
<PackageVersion Include="System.Security.Cryptography.Xml" Version="10.0.7" />
<PackageVersion Include="System.ServiceModel.NetNamedPipe" Version="10.0.652802" />
<PackageVersion Include="System.ServiceModel.NetTcp" Version="10.0.652802" />
<!-- 2023 R2 gRPC transport (cross-platform) -->
<PackageVersion Include="Google.Protobuf" Version="3.24.4" />
<PackageVersion Include="Grpc.Net.Client" Version="2.58.0" />
<PackageVersion Include="Grpc.Net.Client.Web" Version="2.58.0" />
<PackageVersion Include="Grpc.Tools" Version="2.59.0" />
<!-- ZB-idiomatic DI extension (only non-BCL lib dependency) -->
<PackageVersion Include="Microsoft.Extensions.DependencyInjection.Abstractions" Version="10.0.7" />
<!-- Test -->
<PackageVersion Include="Microsoft.NET.Test.Sdk" Version="17.14.1" />
<PackageVersion Include="xunit" Version="2.9.3" />
<PackageVersion Include="xunit.runner.visualstudio" Version="3.1.4" />
<PackageVersion Include="coverlet.collector" Version="6.0.4" />
<PackageVersion Include="Microsoft.Data.SqlClient" Version="6.0.2" />
</ItemGroup>
</Project>
```
**Step 3: `.gitignore`**
```gitignore
bin/
obj/
# identity-bearing / non-redistributable — never commit
*.ndjson
current/
aveva-install-*/
```
**Step 4: `ZB.MOM.WW.SPHistorianClient.slnx`**
```xml
<Solution>
<Folder Name="/src/">
<Project Path="src/ZB.MOM.WW.SPHistorianClient/ZB.MOM.WW.SPHistorianClient.csproj" />
</Folder>
<Folder Name="/tests/">
<Project Path="tests/ZB.MOM.WW.SPHistorianClient.Tests/ZB.MOM.WW.SPHistorianClient.Tests.csproj" />
</Folder>
</Solution>
```
**Step 5: `src/ZB.MOM.WW.SPHistorianClient/ZB.MOM.WW.SPHistorianClient.csproj`**
(Derived from the bundle's `AVEVA.Historian.Client.csproj`: inline versions removed for central
management; ZB package metadata added; `InternalsVisibleTo` retargeted to the ZB test assembly and
the `…ReverseEngineering` one dropped; proto glob uses forward slashes for cross-platform MSBuild.)
```xml
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<IsPackable>true</IsPackable>
<PackageId>ZB.MOM.WW.SPHistorianClient</PackageId>
<Authors>ZB.MOM.WW</Authors>
<Description>Pure-managed .NET 10 client for AVEVA System Platform Historian (Wonderware) for the ZB.MOM.WW SCADA family. The wire protocol is reverse-engineered and re-implemented in C# — no native AVEVA runtime dependency. Surfaces history reads (raw / aggregate / at-time / event), tag browse + metadata, status, and tag create/delete over the WCF/MDAS transports (Windows) plus a cross-platform gRPC transport for 2023 R2.</Description>
<PackageTags>aveva;wonderware;historian;system-platform;scada;timeseries;grpc;wcf;zb-mom-ww</PackageTags>
<PackageProjectUrl>https://gitea.dohertylan.com/dohertj2/zb-mom-ww-sphistorianclient</PackageProjectUrl>
<RepositoryUrl>https://gitea.dohertylan.com/dohertj2/zb-mom-ww-sphistorianclient</RepositoryUrl>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="System.Security.Cryptography.Xml" />
<PackageReference Include="System.ServiceModel.NetNamedPipe" />
<PackageReference Include="System.ServiceModel.NetTcp" />
<PackageReference Include="Microsoft.Extensions.DependencyInjection.Abstractions" />
</ItemGroup>
<!-- 2023 R2 gRPC transport (RemoteGrpc). Pure-managed: Grpc.Net.Client + Google.Protobuf.
Grpc.Tools is build-only (PrivateAssets=all) and generates the client stubs from the
recovered contract under Grpc/Protos at build. -->
<ItemGroup>
<PackageReference Include="Google.Protobuf" />
<PackageReference Include="Grpc.Net.Client" />
<PackageReference Include="Grpc.Net.Client.Web" />
<PackageReference Include="Grpc.Tools">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
</ItemGroup>
<ItemGroup>
<Protobuf Include="Grpc/Protos/*.proto" GrpcServices="Client" ProtoRoot="Grpc/Protos" />
</ItemGroup>
<ItemGroup>
<AssemblyAttribute Include="System.Runtime.CompilerServices.InternalsVisibleToAttribute">
<_Parameter1>ZB.MOM.WW.SPHistorianClient.Tests</_Parameter1>
</AssemblyAttribute>
</ItemGroup>
</Project>
```
**Step 6: `tests/ZB.MOM.WW.SPHistorianClient.Tests/ZB.MOM.WW.SPHistorianClient.Tests.csproj`**
```xml
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<IsPackable>false</IsPackable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="coverlet.collector" />
<PackageReference Include="Microsoft.NET.Test.Sdk" />
<PackageReference Include="Microsoft.Data.SqlClient" />
<PackageReference Include="xunit" />
<PackageReference Include="xunit.runner.visualstudio" />
</ItemGroup>
<ItemGroup>
<Using Include="Xunit" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\..\src\ZB.MOM.WW.SPHistorianClient\ZB.MOM.WW.SPHistorianClient.csproj" />
</ItemGroup>
</Project>
```
**Step 7: Verify the skeleton is well-formed (build will fail — no sources yet — that is expected)**
Run: `cd ZB.MOM.WW.SPHistorianClient && dotnet restore ZB.MOM.WW.SPHistorianClient.slnx`
Expected: restore **succeeds** (proves the props/csproj XML and central package versions resolve). A
follow-up `dotnet build` would fail only because no `.cs` exist yet — do not build here.
**Step 8: Commit**
```bash
cd /Users/dohertj2/Desktop/scadaproj
git add ZB.MOM.WW.SPHistorianClient/
git commit -m "feat(sphistorianclient): scaffold shared library skeleton (props, csprojs, slnx)"
```
---
## Task 2: Port source + tests with namespace rewrite
**Classification:** standard
**Estimated implement time:** ~4 min
**Parallelizable with:** none
**Blocked by:** Task 1
**Files:**
- Create (scripted copy): `ZB.MOM.WW.SPHistorianClient/src/ZB.MOM.WW.SPHistorianClient/**/*.{cs,proto}` (74 `.cs` + 6 `.proto`)
- Create (scripted copy): `ZB.MOM.WW.SPHistorianClient/tests/ZB.MOM.WW.SPHistorianClient.Tests/**/*.cs` (25 `.cs`)
This task is a deterministic copy + namespace rewrite — run the script, then verify counts.
**Step 1: Copy + rewrite (single script)**
```bash
set -euo pipefail
BUNDLE=/tmp/histsdk/extracted/histsdk-migration/histsdk
DEST=/Users/dohertj2/Desktop/scadaproj/ZB.MOM.WW.SPHistorianClient
# Guard: re-extract if /tmp was cleaned
if [ ! -d "$BUNDLE/src/AVEVA.Historian.Client" ]; then
cd /tmp/histsdk && unzip -o -q histsdk-migration.zip -d extracted
fi
# --- src: copy .cs + .proto, preserving subdirs (NOT the old .csproj) ---
SRC="$DEST/src/ZB.MOM.WW.SPHistorianClient"
cd "$BUNDLE/src/AVEVA.Historian.Client"
find . \( -name '*.cs' -o -name '*.proto' \) | while read -r f; do
mkdir -p "$SRC/$(dirname "$f")"
cp "$f" "$SRC/$f"
done
# --- tests: copy .cs only (NOT the old .csproj) ---
TST="$DEST/tests/ZB.MOM.WW.SPHistorianClient.Tests"
cd "$BUNDLE/tests/AVEVA.Historian.Client.Tests"
find . -name '*.cs' | while read -r f; do
mkdir -p "$TST/$(dirname "$f")"
cp "$f" "$TST/$f"
done
# --- namespace rewrite in .cs ONLY (proto wire contracts stay ArchestrA.Grpc.Contract.*) ---
find "$SRC" "$TST" -name '*.cs' -print0 \
| xargs -0 sed -i '' 's/AVEVA\.Historian\.Client/ZB.MOM.WW.SPHistorianClient/g'
```
**Step 2: Verify counts and that the rename is total**
```bash
DEST=/Users/dohertj2/Desktop/scadaproj/ZB.MOM.WW.SPHistorianClient
echo "src cs: $(find "$DEST/src" -name '*.cs' | wc -l) (expect 74)"
echo "src proto: $(find "$DEST/src" -name '*.proto' | wc -l) (expect 6)"
echo "test cs: $(find "$DEST/tests" -name '*.cs' | wc -l) (expect 25)"
echo "leftover AVEVA.Historian.Client in .cs: $(grep -rl 'AVEVA\.Historian\.Client' "$DEST" --include='*.cs' | wc -l) (expect 0)"
echo "proto namespace preserved: $(grep -l 'ArchestrA.Grpc.Contract' "$DEST"/src/ZB.MOM.WW.SPHistorianClient/Grpc/Protos/*.proto | wc -l) (expect 6)"
```
Expected: `74`, `6`, `25`, `0`, `6`. If "leftover" is non-zero, inspect those files — the only legitimate
remaining mentions would be inside comments/strings that happen to differ in casing/spacing; a clean
port should show `0`.
**Step 3: Commit**
```bash
cd /Users/dohertj2/Desktop/scadaproj
git add ZB.MOM.WW.SPHistorianClient/src ZB.MOM.WW.SPHistorianClient/tests
git commit -m "feat(sphistorianclient): port SDK source + tests, rebrand namespace to ZB.MOM.WW.SPHistorianClient"
```
---
## Task 3: Build + test green
**Classification:** high-risk
**Estimated implement time:** ~5 min (plus restore/build wall-time)
**Parallelizable with:** none
**Blocked by:** Task 2
This is the integration gate. The port must compile and the offline test suite must pass on this macOS host.
**Files:**
- Modify (only if the build surfaces a defect): any ported file under `ZB.MOM.WW.SPHistorianClient/src` or `…/tests`, or the two `.csproj`.
**Step 1: Build**
Run: `cd ZB.MOM.WW.SPHistorianClient && dotnet build ZB.MOM.WW.SPHistorianClient.slnx`
Expected: **Build succeeded.** Platform-compatibility (CAxxxx `[SupportedOSPlatform("windows")]`) warnings
are acceptable and must remain warnings. If restore hard-fails on `NU1903`, re-run with
`-p:NuGetAudit=false`.
**Step 2: Test**
Run: `dotnet test ZB.MOM.WW.SPHistorianClient.slnx`
Expected: all tests pass; the live integration tests (`HistorianClientIntegrationTests`,
`HistorianGrpcIntegrationTests`, `RemoteTcpIntegrationTests`) **skip cleanly** because no `HISTORIAN_*`
env vars are set. The bundle's `MIGRATION-README.md` documents ~188 tests passing on macOS with the
live ones skipped — treat a comparable count with **zero failures** as success.
**Step 3: Triage rules (if not green)**
- Compile error referencing `AVEVA.Historian.Client` → a file was missed by the rewrite; re-run the
Task 2 sed on that file.
- `NU1008` (version on PackageReference) → an inline `Version=` slipped into a `.csproj`; remove it
(version belongs in `Directory.Packages.props`).
- Missing generated gRPC type (e.g. `ArchestrA.Grpc.Contract.*` not found) → confirm the `<Protobuf>`
glob in the src `.csproj` resolves the 6 `Grpc/Protos/*.proto` and that `Grpc.Tools` restored.
- A genuine test failure (not a skip) → this is a real port defect; fix the ported code, do **not**
delete/weaken the test.
**Step 4: Commit (only if Step 3 required edits)**
```bash
git add -A ZB.MOM.WW.SPHistorianClient/
git commit -m "fix(sphistorianclient): resolve port build/test fallout"
```
---
## Task 4: Add the `AddZbSpHistorianClient` DI extension (TDD)
**Classification:** standard
**Estimated implement time:** ~4 min
**Parallelizable with:** Task 5
**Blocked by:** Task 3
`HistorianClientOptions` uses `required` + `init`-only properties, so the extension takes a fully-built
options instance (not an `Action<T>` configurator). It depends only on
`Microsoft.Extensions.DependencyInjection.Abstractions`.
**Files:**
- Create: `ZB.MOM.WW.SPHistorianClient/src/ZB.MOM.WW.SPHistorianClient/DependencyInjection/ZbSpHistorianClientServiceCollectionExtensions.cs`
- Test: `ZB.MOM.WW.SPHistorianClient/tests/ZB.MOM.WW.SPHistorianClient.Tests/DependencyInjectionTests.cs`
**Step 1: Write the failing test**
```csharp
using Microsoft.Extensions.DependencyInjection;
using ZB.MOM.WW.SPHistorianClient;
namespace ZB.MOM.WW.SPHistorianClient.Tests;
public class DependencyInjectionTests
{
[Fact]
public void AddZbSpHistorianClient_resolves_client_and_options()
{
var services = new ServiceCollection();
var options = new HistorianClientOptions { Host = "localhost" };
services.AddZbSpHistorianClient(options);
using var sp = services.BuildServiceProvider();
Assert.Same(options, sp.GetRequiredService<HistorianClientOptions>());
Assert.NotNull(sp.GetRequiredService<HistorianClient>());
}
[Fact]
public void AddZbSpHistorianClient_throws_when_host_missing()
{
var services = new ServiceCollection();
var options = new HistorianClientOptions { Host = "" };
Assert.Throws<ArgumentException>(() => services.AddZbSpHistorianClient(options));
}
[Fact]
public void AddZbSpHistorianClient_throws_on_null_options()
{
var services = new ServiceCollection();
Assert.Throws<ArgumentNullException>(() => services.AddZbSpHistorianClient(null!));
}
}
```
**Step 2: Run — verify it fails to compile** (`AddZbSpHistorianClient` not defined)
Run: `dotnet test ZB.MOM.WW.SPHistorianClient.slnx --filter "FullyQualifiedName~DependencyInjectionTests"`
Expected: FAIL (does not compile / method missing).
**Step 3: Implement**
```csharp
using Microsoft.Extensions.DependencyInjection;
namespace ZB.MOM.WW.SPHistorianClient;
/// <summary>
/// ZB.MOM.WW DI registration for <see cref="HistorianClient"/>. Mirrors the family's
/// <c>AddZb*</c> convention. Because <see cref="HistorianClientOptions"/> is <c>required</c>/
/// <c>init</c>-only, callers pass a fully-built options instance (bind it from configuration in the
/// consuming app, e.g. <c>config.GetSection("Historian").Get&lt;HistorianClientOptions&gt;()</c>).
/// </summary>
public static class ZbSpHistorianClientServiceCollectionExtensions
{
public static IServiceCollection AddZbSpHistorianClient(
this IServiceCollection services,
HistorianClientOptions options)
{
ArgumentNullException.ThrowIfNull(services);
ArgumentNullException.ThrowIfNull(options);
if (string.IsNullOrWhiteSpace(options.Host))
{
throw new ArgumentException(
"HistorianClientOptions.Host must be set.", nameof(options));
}
services.AddSingleton(options);
// HistorianClient opens a fresh channel per operation and has a no-op DisposeAsync,
// so transient is safe and avoids assuming the shared dialect is concurrency-safe.
services.AddTransient<HistorianClient>();
return services;
}
}
```
**Step 4: Run — verify pass**
Run: `dotnet test ZB.MOM.WW.SPHistorianClient.slnx --filter "FullyQualifiedName~DependencyInjectionTests"`
Expected: PASS (3/3).
**Step 5: Commit**
```bash
git add ZB.MOM.WW.SPHistorianClient/src/ZB.MOM.WW.SPHistorianClient/DependencyInjection \
ZB.MOM.WW.SPHistorianClient/tests/ZB.MOM.WW.SPHistorianClient.Tests/DependencyInjectionTests.cs
git commit -m "feat(sphistorianclient): add AddZbSpHistorianClient DI extension"
```
---
## Task 5: Author `CLAUDE.md` + `README.md`
**Classification:** small
**Estimated implement time:** ~4 min
**Parallelizable with:** Task 4
**Blocked by:** Task 3
Sanitized docs only — **no hostnames, credentials, customer tag names, or capture data.** Model the
structure on `ZB.MOM.WW.Telemetry/CLAUDE.md` (overview, package table, build/test/pack commands,
status) but adapt to a single-package library.
**Files:**
- Create: `ZB.MOM.WW.SPHistorianClient/CLAUDE.md`
- Create: `ZB.MOM.WW.SPHistorianClient/README.md`
**`CLAUDE.md` must cover:**
- One-paragraph overview: pure-managed .NET 10 AVEVA System Platform Historian client, no native AVEVA
dependency, reverse-engineered wire protocol. Ported from the `histsdk` migration bundle.
- The supported operation surface table (copy the README table from the bundle:
`ProbeAsync`, `ReadRawAsync`, `ReadAggregateAsync` (16 modes), `ReadAtTimeAsync`, `ReadEventsAsync`,
`BrowseTagNamesAsync`, `GetTagMetadataAsync`, `GetConnectionStatusAsync`,
`GetStoreForwardStatusAsync`, `GetSystemParameterAsync`, `EnsureTagAsync`, `DeleteTagAsync`).
- Transport matrix: `LocalPipe` / `RemoteTcpIntegrated` / `RemoteTcpCertificate` (WCF, Windows-only,
live-verified) vs `RemoteGrpc` (2023 R2, cross-platform, **not yet live-verified**).
- Out of scope: writing samples (`AddS2` architecturally blocked), discrete/string tag creation.
- DI: the `AddZbSpHistorianClient(options)` extension + the bind-from-config note.
- Build/test/pack commands (from this dir):
`dotnet build ZB.MOM.WW.SPHistorianClient.slnx` / `dotnet test …` /
`dotnet pack ZB.MOM.WW.SPHistorianClient.slnx -c Release -o ./artifacts`.
- Live integration tests gated by `HISTORIAN_*` env vars (skip cleanly when unset). List the env vars.
**`README.md`:** a trimmed public-facing version — overview, quick-start snippet (the bundle's
`HistorianClient` usage example, namespace updated to `ZB.MOM.WW.SPHistorianClient`), supported surface
table, build/test commands.
**Commit:**
```bash
git add ZB.MOM.WW.SPHistorianClient/CLAUDE.md ZB.MOM.WW.SPHistorianClient/README.md
git commit -m "docs(sphistorianclient): add CLAUDE.md + README.md"
```
---
## Task 6: Pack verification
**Classification:** small
**Estimated implement time:** ~3 min
**Parallelizable with:** none
**Blocked by:** Task 4, Task 5
**Files:**
- Create (build output): `ZB.MOM.WW.SPHistorianClient/artifacts/ZB.MOM.WW.SPHistorianClient.0.1.0.nupkg`
**Step 1: Full green build + test once more, then pack**
```bash
cd /Users/dohertj2/Desktop/scadaproj/ZB.MOM.WW.SPHistorianClient
dotnet test ZB.MOM.WW.SPHistorianClient.slnx
dotnet pack ZB.MOM.WW.SPHistorianClient.slnx -c Release -o ./artifacts
```
Expected: tests pass (live ones skip); pack produces `artifacts/ZB.MOM.WW.SPHistorianClient.0.1.0.nupkg`.
**Step 2: Sanity-check the package contents**
```bash
unzip -l artifacts/ZB.MOM.WW.SPHistorianClient.0.1.0.nupkg | grep -E 'ZB.MOM.WW.SPHistorianClient.dll|.nuspec'
```
Expected: the lib DLL and nuspec are present.
**Step 3: Commit the nupkg** (matches the family convention — `ZB.MOM.WW.Telemetry` commits its `artifacts/*.nupkg`)
```bash
cd /Users/dohertj2/Desktop/scadaproj
git add -f ZB.MOM.WW.SPHistorianClient/artifacts/ZB.MOM.WW.SPHistorianClient.0.1.0.nupkg
git commit -m "build(sphistorianclient): pack 0.1.0 nupkg"
```
> **Do NOT push or publish** to the Gitea feed. Per repo experience, "published/adopted" claims must
> not be made without explicit user direction + feed verification.
---
## Task 7: Index the new library in the umbrella `CLAUDE.md` (optional)
**Classification:** trivial
**Estimated implement time:** ~2 min
**Parallelizable with:** none
**Blocked by:** Task 6
**Files:**
- Modify: `CLAUDE.md` (repo root umbrella index)
Add a short reference so the umbrella index reflects the newly-hosted library (the intro paragraph
that enumerates the hosted `ZB.MOM.WW.*` sources, and/or a one-line pointer near the component table
noting `ZB.MOM.WW.SPHistorianClient` is a net-new shared library — **not** a component normalization).
> **Caveat:** repo-root `CLAUDE.md` already has **pre-existing uncommitted edits** (unrelated to this
> work). Before editing, run `git diff CLAUDE.md` and make sure your commit message reflects that it
> may bundle those edits — or stage only the hunks you add. If this risks entangling unrelated changes,
> skip this task and leave it for the user.
**Commit:**
```bash
git add CLAUDE.md
git commit -m "docs: index ZB.MOM.WW.SPHistorianClient in umbrella CLAUDE.md"
```
---
## Done criteria
- `ZB.MOM.WW.SPHistorianClient/` exists with `src/`, `tests/`, props, `.slnx`, `CLAUDE.md`, `README.md`.
- `dotnet build` + `dotnet test` are green on macOS (live integration tests skip cleanly).
- `AddZbSpHistorianClient` DI extension present + tested.
- `artifacts/ZB.MOM.WW.SPHistorianClient.0.1.0.nupkg` produced.
- All work committed on `feat/sphistorianclient`. Not pushed/published.
@@ -0,0 +1,13 @@
{
"planPath": "docs/plans/2026-06-19-sphistorianclient.md",
"tasks": [
{"id": 1, "subject": "Task 1: Scaffold library skeleton", "status": "completed"},
{"id": 2, "subject": "Task 2: Port source + tests with namespace rewrite", "status": "completed", "blockedBy": [1]},
{"id": 3, "subject": "Task 3: Build + test green", "status": "completed", "blockedBy": [2]},
{"id": 4, "subject": "Task 4: Add AddZbSpHistorianClient DI extension (TDD)", "status": "completed", "blockedBy": [3]},
{"id": 5, "subject": "Task 5: Author CLAUDE.md + README.md", "status": "completed", "blockedBy": [3]},
{"id": 6, "subject": "Task 6: Pack verification", "status": "completed", "blockedBy": [4, 5]},
{"id": 7, "subject": "Task 7: Index new lib in umbrella CLAUDE.md (optional)", "status": "completed", "blockedBy": [6]}
],
"lastUpdated": "2026-06-19"
}
@@ -0,0 +1,216 @@
# ZB.MOM.WW.HistorianGateway — Design
**Date:** 2026-06-23
**Status:** Design approved (brainstorming complete) — ready for implementation planning
**Author:** brainstorming session (Joseph Doherty + Claude)
## 1. Summary
A new **full-feature sidecar** in the SCADA/OT sister-project family, modelled on
`MxAccessGateway` (`mxaccessgw`). It does two things:
1. **Read-only Galaxy metadata server** — exposes the AVEVA System Platform
("Wonderware") **Galaxy object hierarchy** (areas / objects / templates /
instances / attributes), sourced from the **Galaxy Repository SQL DB**, the same
data `mxaccessgw`'s `galaxy_repository` feature serves today.
2. **Full read/write gRPC API to the AVEVA (Wonderware) Historian** — reads (raw,
aggregate with all 15 retrieval modes, at-time, blocks, events, browse, metadata,
status) **and writes** (historical/backfill values, event send, tag-config
create/delete/rename/extended-properties, plus resilience helpers).
It reuses the family's **common shared packages and styles**: `ZB.MOM.WW.Auth`,
`ZB.MOM.WW.Theme`, `ZB.MOM.WW.Telemetry`(+`.Serilog`), `ZB.MOM.WW.Health`,
`ZB.MOM.WW.Configuration`, `ZB.MOM.WW.Audit`.
### Key reframing discovered during brainstorming
- The historian write surface lives in the **`histsdk`** repo
(`gitea.dohertylan.com/dohertj2/histsdk`, namespace `AVEVA.Historian.Client`),
which is **far ahead** of the stale `scadaproj/ZB.MOM.WW.SPHistorianClient` port
(2026-06-19 snapshot, reads + tag create/delete only, value-writes marked
"blocked"). `histsdk` has since added a **live-validated gRPC write surface**:
`AddHistoricalValuesAsync` (historical/backfill, gRPC-only), `SendEventAsync`
(events, both transports), `EnsureTags`/`DeleteTags`/`RenameTags`/
`AddTagExtendedProperties` (config writes, gRPC), plus higher-level
`HistorianStoreForwardWriter` (durable outbox) and a redundant-write cluster.
- **Hard server-side limit no client can lift:** `AddS2` *streaming live
process-sample* writes are GATED — the historian runtime cache only ingests from
configured IOServer / Application Server pipelines. "Live current value" writes are
therefore done via the **SQL path** (`aaAnalogTagInsert``INSERT INTO History`),
not gRPC.
- **No COM anywhere.** The historian SDK is pure-managed and Galaxy browse is plain
SQL, so — unlike `mxaccessgw` — this sidecar needs **no x86 worker and no
two-process split**. It is a single .NET 10 x64 process.
## 2. Decisions (locked during brainstorming)
| Decision | Choice |
|---|---|
| Purpose | **General-purpose gateway** — reusable modern façade over Historian + Galaxy metadata for any gRPC client (the way `mxaccessgw` is the façade for MXAccess). |
| Historian client code | **Vendor `histsdk`** (`AVEVA.Historian.Client`) into the sidecar repo as self-contained vendored source; namespace kept as-is to ease future re-sync. |
| Galaxy browse code | **New shared lib `ZB.MOM.WW.GalaxyRepository`** in scadaproj, extracted from `mxaccessgw`'s `galaxy_repository` browse; consumed by both `mxaccessgw` and this sidecar. |
| Dashboard | **Full Blazor dashboard** on `ZB.MOM.WW.Theme` (login + Galaxy browser + Historian console + API-key admin + status/health). |
| Historian write scope | **All:** historical/backfill writes, event send, tag-config writes, **and** resilience extras (store-and-forward outbox, redundant write fan-out, SQL live-value path). |
| Connection model | **Approach A — stateless gateway over pooled service-identity connections.** Clients authenticate to the gateway (ZB API key); the gateway owns historian credentials and reuses pooled, pre-authenticated connections. |
| Repo location | **New standalone sibling repo** `~/Desktop/HistorianGateway`, gitea remote `historiangw`. |
| Name / namespace | **`ZB.MOM.WW.HistorianGateway`**. |
## 3. Architecture & solution structure
Single .NET 10 x64 ASP.NET Core process.
```
~/Desktop/HistorianGateway/
src/
ZB.MOM.WW.HistorianGateway.Server/ ASP.NET Core host: gRPC services + Blazor dashboard + /healthz + /metrics
ZB.MOM.WW.HistorianGateway.Contracts/ the gateway's own .proto + generated types (for client codegen distribution)
vendor/AVEVA.Historian.Client/ VENDORED from histsdk; ArchestrA.Grpc.Contract.* protos + reads/writes/store-forward/redundancy
tests/
ZB.MOM.WW.HistorianGateway.Tests/ unit + env-gated live integration + bUnit dashboard
docs/plans/, CLAUDE.md, README.md
ZB.MOM.WW.HistorianGateway.slnx
```
**Cross-repo pieces:**
- **`scadaproj/ZB.MOM.WW.GalaxyRepository`** (new shared lib, plain files — NOT a
nested git repo): carries the **canonical `galaxy_repository.proto`** (adopted from
`mxaccessgw`'s existing contract so OtOpcUa's wire shape is not broken), the SQL
browse provider (connect to Galaxy Repository SQL → hierarchy model), and a
reusable gRPC service implementation both hosts can `MapGrpcService<>()`.
`mxaccessgw` adopting it is a **tracked follow-on** (same "built → adopted" pattern
as the other normalized components); this sidecar consumes it from the start.
- **Shared ZB packages consumed:** `ZB.MOM.WW.Auth`
(Abstractions+Ldap+ApiKeys+AspNetCore), `ZB.MOM.WW.Theme`, `ZB.MOM.WW.Telemetry`
(+`.Serilog`), `ZB.MOM.WW.Health`, `ZB.MOM.WW.Configuration`, `ZB.MOM.WW.Audit`.
## 4. gRPC API surface
Gateway's own curated contract (`ZB.MOM.WW.HistorianGateway.Grpc.V1`), grouped by
concern — not a 1:1 SDK dump. The vendored `ArchestrA.Grpc.Contract.*` protos stay
internal; clients see only the gateway contract.
| Service | RPCs | Notes |
|---|---|---|
| `HistorianRead` | `ReadRaw`, `ReadAggregate`, `ReadBlocks`, `ReadEvents` *(server-streaming)*; `ReadAtTime` | `ReadAggregate` exposes all 15 retrieval modes |
| `HistorianWrite` | `AddHistoricalValues`, `SendEvent`, `WriteLiveValues` | `WriteLiveValues` = SQL path (gRPC streaming is gated) |
| `HistorianTags` | `BrowseTagNames` *(streaming)*, `GetTagMetadata`, `EnsureTags`, `DeleteTags`, `RenameTags`, `AddTagExtendedProperties` | |
| `HistorianStatus` | `Probe`, `GetConnectionStatus`, `GetStoreForwardStatus`, `GetSystemParameter` | |
| `GalaxyRepository` | Browse areas / objects / templates / instances / attributes *(read-only)* | canonical proto from the shared lib |
**Authorization** is via **API-key scopes at the gateway** (Approach A trust
boundary): `historian:read`, `historian:write`, `historian:tags:write`,
`galaxy:read`.
## 5. Connection & data flow
```
gRPC client ──(ZB API key)──► HistorianGateway ──┬─ pooled, pre-authed gRPC conn ──► AVEVA Historian (RemoteGrpc 2023R2)
├─ store-forward outbox (SQLite) ─ replays writes on reconnect
├─ redundant-write fan-out ──────► historian members (All/Any ack)
├─ SqlConnection ──► Runtime DB (live-value writes via aaAnalogTagInsert/History)
└─ ZB.MOM.WW.GalaxyRepository ──► Galaxy Repository SQL (read-only browse)
```
- **Pooled connections:** the expensive auth handshake (`ValidateClientCredential` /
ECDH `ExchangeKey`) runs **once per connection on open**, then is reused across
requests; connections are health-checked with auto-reconnect. Write operations use
the write-enabled session mode (`0x401`).
- **Store-forward:** writes flow through the SDK's `HistorianStoreForwardWriter` — on
an unreachable historian, enqueue to durable SQLite; a background drain replays on
reconnect.
- **Redundancy:** `HistorianRedundantWriteResult` fan-out to configured members under
an All/Any ack policy; per-member result surfaced to the caller.
- **SQL live-write** and **Galaxy browse** are independent SQL paths, each with its
own validated connection config.
**Configuration** (all `ZB.MOM.WW.Configuration`-validated, aggregated by
`ConfigPreflight` at startup): historian (host, gRPC port 32565, transport=RemoteGrpc,
TLS, service identity/credentials), redundant members, store-forward path, Galaxy
Repository SQL connection string, Runtime DB connection string (SQL live-write),
Auth (LDAP + API-key pepper). Secrets live in the operator environment, never in repo.
## 6. Cross-cutting infrastructure + dashboard
- **Auth (`ZB.MOM.WW.Auth`):** gRPC clients use peppered-HMAC API keys (keyId/Bearer),
validated by a gRPC interceptor enforcing per-service scopes. Dashboard uses LDAP
login (`.Ldap`+`.AspNetCore`), cookie auth, `IGroupRoleMapper<TRole>`, canonical
`ZbClaimTypes`/`ZbCookieDefaults`, canonical-six roles, dev against the shared
GLAuth (`10.100.0.35:3893`, `dc=zb,dc=local`). `DisableLogin` dev/deploy switch.
- **Telemetry (`.Telemetry`+`.Serilog`):** `AddZbTelemetry` (Resource
`service.name=historian-gateway` + standard instrumentation + always-on Prometheus
`/metrics`, OTLP opt-in) + `AddZbSerilog`. App Meters: read/write counts + latency,
store-forward queue depth, pool connection state, redundancy ack outcomes.
- **Health (`.Health`):** three-tier ready/active/healthz + canonical JSON writer.
Probes: historian gRPC (`GrpcDependencyHealthCheck`), Galaxy Repository SQL +
Runtime DB (`DatabaseHealthCheck`), store-forward drain status.
- **Configuration (`.Configuration`):** `OptionsValidatorBase` / `ValidationBuilder` /
`AddValidatedOptions` / `ConfigPreflight` (§5).
- **Audit (`.Audit`, DEEP-adopt):** canonical `AuditEvent` + SQLite `IAuditWriter`
(MxGateway-style). Audited: tag-config writes, historical/event writes, API-key
admin, login/logout. `Actor` wired from the Auth principal via `IAuditActorAccessor`.
- **Dashboard (Blazor, `.Theme`):** Technical-Light side-rail shell + `LoginCard`
`/login`. Pages: **Status** (pool / store-forward / redundancy / version),
**Galaxy browser** (read-only hierarchy tree), **Historian console** (query with
raw/aggregate + mode picker + time range; role-gated write test for value insert /
event send), **API-key admin** (list/create/revoke keys + scopes), **Health**.
## 7. Error handling
- **gRPC status mapping:** `ProtocolEvidenceMissingException` (unsupported op/type —
e.g. non-analog tag, non-string event property) → `Unimplemented`/`FailedPrecondition`
with a clear "not in reverse-engineered surface" message; auth →
`Unauthenticated`/`PermissionDenied`; historian down → `Unavailable`; bad range /
unknown tag → `InvalidArgument`/`NotFound`.
- **Gated ops:** live streaming-sample writes (`AddS2`) are **not exposed** (no RPC);
live-value writes route through SQL `WriteLiveValues`.
- **Write resilience:** with store-forward enabled, an unreachable historian returns
*accepted + queued* (not an error); otherwise `Unavailable`. Redundancy surfaces a
per-member result; All-policy fails if any member fails, Any-policy succeeds on ≥1 ack.
- **Pool:** transient failures → reconnect + bounded retry; auth-handshake failure →
fail fast with diagnostic. No secrets/real hostnames in errors or logs (histsdk
safety rule).
## 8. Testing
- **Unit:** gRPC services against a faked historian-client seam + faked Galaxy
provider; scope/auth interceptor; config validators; SDK-model ↔ proto mapping.
- **Golden/protocol:** carry over `histsdk`'s golden byte tests for the vendored
client (historical "ON" buffer, event "OS" buffer, registration buffers) so the
vendored copy stays faithful.
- **Integration (env-gated, live, CI/macOS-safe):** real 2023 R2 historian + Galaxy
Repository SQL — read/write round-trips and browse via the self-cleaning
sandbox-tag lifecycle (`HISTORIAN_GRPC_WRITE_SANDBOX_TAG`); skipped when env vars
absent.
- **Dashboard:** bUnit component tests. **Smoke:** `/healthz`, `/metrics`, gRPC
`Probe`.
## 9. Out of scope / non-goals
- `AddS2` live streaming process-sample writes (GATED server-side; SQL path covers
live values instead).
- Non-analog tag creation, revision/edit writes, bit-faithful store-forward framing
(per `histsdk` capability matrix — `BOUNDED`/`HARD`/`GATED` items not selected).
- A two-process / x86 worker split (not needed — no COM).
- Re-syncing or replacing the existing stale `scadaproj/ZB.MOM.WW.SPHistorianClient`
port (we vendor `histsdk` instead; the stale port is left as-is).
## 10. Implementation components (high level)
1. **`ZB.MOM.WW.GalaxyRepository` shared lib** (scadaproj) — extract from
`mxaccessgw`, canonical proto + SQL browse provider + reusable gRPC service.
2. **Vendor `histsdk`** `AVEVA.Historian.Client` into the new repo + carry its golden
tests.
3. **Repo scaffold + host + shared-package wiring** (Auth/Telemetry/Health/
Configuration/Audit) + validated options + `ConfigPreflight`.
4. **gRPC contract + services** (Read / Write / Tags / Status / GalaxyRepository).
5. **Connection layer** — pooled pre-authed connections, store-forward, redundancy,
SQL live-write path.
6. **Auth** — API-key scope interceptor + LDAP dashboard auth + Audit wiring.
7. **Blazor dashboard** pages (Theme).
8. **Telemetry + Health** probes/meters.
9. **Tests** — unit / golden / env-gated integration / bUnit.
10. **Docs + repo/gitea setup**`CLAUDE.md`, `README.md`, gitea remote.
> `mxaccessgw` adoption of `ZB.MOM.WW.GalaxyRepository` is a separate tracked
> follow-on, not part of the initial sidecar delivery.
@@ -0,0 +1,523 @@
# ZB.MOM.WW.HistorianGateway Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans to implement this plan task-by-task.
**Goal:** Build a single .NET 10 x64 sidecar that exposes (1) a read-only Galaxy object-hierarchy metadata gRPC server and (2) a full read/write gRPC API to the AVEVA Historian, with a Blazor dashboard, reusing the family's shared `ZB.MOM.WW.*` packages.
**Architecture:** One ASP.NET Core process hosting gRPC services + Blazor (no COM, no x86 worker). The historian write/read surface comes from the **vendored `histsdk` client** (`AVEVA.Historian.Client`). The Galaxy browse comes from a **new shared lib `ZB.MOM.WW.GalaxyRepository`** in scadaproj (extracted from mxaccessgw, wire-compatible `galaxy_repository.v1`). Connection model: stateless gateway over a **pooled, pre-authenticated service-identity connection**; clients authenticate to the gateway via peppered-HMAC API keys with per-service scopes.
**Tech Stack:** .NET 10, ASP.NET Core, Grpc.AspNetCore 2.76, Grpc.Net.Client 2.58 (vendored), Google.Protobuf, Microsoft.Data.SqlClient, Microsoft.Data.Sqlite, Blazor InteractiveServer, `ZB.MOM.WW.Theme` 0.3.1, `ZB.MOM.WW.Auth` 0.1.2, `ZB.MOM.WW.Telemetry`/`.Serilog` 0.1.0, `ZB.MOM.WW.Health` 0.1.0, `ZB.MOM.WW.Audit` 0.1.0, `ZB.MOM.WW.Configuration` 0.1.0, xUnit, bUnit.
**Reference sources (read these for exact patterns — do NOT re-discover):**
- Design doc: `docs/plans/2026-06-23-historian-gateway-design.md`
- mxaccessgw (the model): `~/Desktop/MxAccessGateway/src/ZB.MOM.WW.MxGateway.Server/``GatewayApplication.cs` (host wiring), `Security/Authorization/*` (gRPC API-key interceptor + scope resolver), `Galaxy/GalaxyRepository.cs` (the SQL to extract), `Galaxy/GalaxyRepositoryOptions.cs`, `Galaxy/GalaxyHierarchyCache.cs`, `Galaxy/GalaxyRepositoryServiceCollectionExtensions.cs`, `Contracts/Protos/galaxy_repository.proto`, `Dashboard/Components/*` (Blazor + Theme).
- histsdk clone (to vendor): `/tmp/histsdk-explore/src/AVEVA.Historian.Client/` + `/tmp/histsdk-explore/tests/AVEVA.Historian.Client.Tests/`.
- Shared package signatures: captured in the design session; key paths under `~/Desktop/scadaproj/ZB.MOM.WW.{Telemetry,Health,Configuration,Audit,Auth,Theme}/`.
**Conventions for every task:** TDD where a seam exists (write the failing test first). Exact file paths in the `Files:` block ARE the implementer's contract. Commit after each task. Tests must stay green on macOS with no live historian/SQL (live tests are env-gated and skip when env vars are absent).
---
## Phase 0 — Shared `ZB.MOM.WW.GalaxyRepository` lib (in scadaproj)
> Built in `~/Desktop/scadaproj/ZB.MOM.WW.GalaxyRepository/` as plain files (NOT a nested git repo — see memory `shared-libs-are-plain-files-not-nested-repos`). Wire-compatible: keep proto `package galaxy_repository.v1` and all field numbers identical to mxaccessgw's so OtOpcUa is unaffected; only the C# `csharp_namespace` becomes neutral. mxaccessgw adoption of this lib is a separate follow-on, NOT in this plan.
### Task 1: Scaffold the GalaxyRepository lib project
**Classification:** small
**Estimated implement time:** ~3 min
**Parallelizable with:** Task 7 (vendoring), Task 9 (repo scaffold)
**Files:**
- Create: `~/Desktop/scadaproj/ZB.MOM.WW.GalaxyRepository/ZB.MOM.WW.GalaxyRepository.slnx`
- Create: `~/Desktop/scadaproj/ZB.MOM.WW.GalaxyRepository/src/ZB.MOM.WW.GalaxyRepository/ZB.MOM.WW.GalaxyRepository.csproj`
- Create: `~/Desktop/scadaproj/ZB.MOM.WW.GalaxyRepository/tests/ZB.MOM.WW.GalaxyRepository.Tests/ZB.MOM.WW.GalaxyRepository.Tests.csproj`
**Steps:**
1. Create the `.csproj` (net10.0, `Nullable`/`ImplicitUsings` enabled, packable, `PackageId=ZB.MOM.WW.GalaxyRepository`, `Version=0.1.0`). PackageReferences: `Microsoft.Data.SqlClient` 6.0.2, `Grpc.AspNetCore` 2.76.0, `Google.Protobuf`, `Microsoft.Extensions.Hosting.Abstractions`, `Microsoft.Extensions.Options.ConfigurationExtensions`. Add `<Protobuf Include="Protos\*.proto" GrpcServices="Server" />`.
2. Create the test `.csproj` (net10.0, `IsPackable=false`, xUnit 2.9.3 + `Microsoft.NET.Test.Sdk` 17.14.1 + `Microsoft.Data.SqlClient`), ProjectReference to the lib.
3. Create the `.slnx` listing both projects.
4. Run: `dotnet build ~/Desktop/scadaproj/ZB.MOM.WW.GalaxyRepository/ZB.MOM.WW.GalaxyRepository.slnx` — Expected: builds (no sources yet, 0 warnings).
5. Commit: `git -C ~/Desktop/scadaproj add ZB.MOM.WW.GalaxyRepository && git -C ~/Desktop/scadaproj commit -m "feat(galaxyrepo): scaffold ZB.MOM.WW.GalaxyRepository shared lib"`
### Task 2: Port the canonical galaxy_repository.proto (neutral namespace)
**Classification:** standard
**Estimated implement time:** ~4 min
**Parallelizable with:** none (Task 3+ depend on generated types)
**Files:**
- Create: `~/Desktop/scadaproj/ZB.MOM.WW.GalaxyRepository/src/ZB.MOM.WW.GalaxyRepository/Protos/galaxy_repository.proto`
**Steps:**
1. Copy mxaccessgw's `Contracts/Protos/galaxy_repository.proto` verbatim, changing ONLY `option csharp_namespace` to `"ZB.MOM.WW.GalaxyRepository.Grpc"`. Keep `package galaxy_repository.v1`, all services (`TestConnection`, `GetLastDeployTime`, `DiscoverHierarchy`, `WatchDeployEvents`, `BrowseChildren`), and every message/field number identical (wire compatibility).
2. Run: `dotnet build .../ZB.MOM.WW.GalaxyRepository.slnx` — Expected: PASS; generated `GalaxyRepository.GalaxyRepositoryBase`, `GalaxyObject`, `GalaxyAttribute`, etc. appear under namespace `ZB.MOM.WW.GalaxyRepository.Grpc`.
3. Commit: `feat(galaxyrepo): canonical galaxy_repository.v1 proto (neutral namespace)`
### Task 3: Port the SQL browse provider (`GalaxyRepository` + rows + options)
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** none
**Files:**
- Create: `.../src/ZB.MOM.WW.GalaxyRepository/GalaxyRepositoryOptions.cs`
- Create: `.../src/ZB.MOM.WW.GalaxyRepository/GalaxyHierarchyRow.cs`
- Create: `.../src/ZB.MOM.WW.GalaxyRepository/GalaxyAttributeRow.cs`
- Create: `.../src/ZB.MOM.WW.GalaxyRepository/IGalaxyRepository.cs`
- Create: `.../src/ZB.MOM.WW.GalaxyRepository/GalaxyRepository.cs`
**Steps:**
1. Port `GalaxyRepositoryOptions` from mxaccessgw `Galaxy/GalaxyRepositoryOptions.cs` — rename section const to `ZB.MOM.WW.GalaxyRepository` (the consuming app picks its own section path at registration), drop MxGateway-specific defaults. Keep `ConnectionString`, `CommandTimeoutSeconds`, `DashboardRefreshIntervalSeconds`, `PersistSnapshot`, `SnapshotCachePath`.
2. Port `GalaxyHierarchyRow` / `GalaxyAttributeRow` DTOs and the `IGalaxyRepository` interface (`TestConnectionAsync`, `GetLastDeployTimeAsync`, `GetHierarchyAsync`, `GetAttributesAsync`).
3. Port `GalaxyRepository.cs` **verbatim** including the two SQL blocks (`HierarchySql`, `AttributesSql`) and the `SqlConnection`/`SqlDataReader` mapping loops — these are validated reverse-engineered queries; do NOT modify the SQL.
4. Run: `dotnet build` — Expected: PASS.
5. Commit: `feat(galaxyrepo): SQL browse provider (hierarchy + attributes)`
### Task 4: Port the in-memory hierarchy cache + snapshot + deploy notifier + refresh service
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** none
**Files:**
- Create: `.../GalaxyHierarchyCacheEntry.cs`, `.../IGalaxyHierarchyCache.cs`, `.../GalaxyHierarchyCache.cs`
- Create: `.../IGalaxyDeployNotifier.cs`, `.../GalaxyDeployNotifier.cs`
- Create: `.../IGalaxyHierarchySnapshotStore.cs`, `.../GalaxyHierarchySnapshotStore.cs`
- Create: `.../GalaxyHierarchyRefreshService.cs` (`BackgroundService`)
- Create: `.../GalaxyHierarchyProjector.cs` (paging/filter projection used by the gRPC service)
**Steps:**
1. Port these from mxaccessgw's `Galaxy/` folder, adjusting namespaces to `ZB.MOM.WW.GalaxyRepository`. Keep the cache's first-load gate, refresh semaphore, snapshot restore, and deploy-poll refresh trigger.
2. Port `GalaxyHierarchyProjector` (the `Project(...)` + `ComputeFilterSignature(...)` used by `DiscoverHierarchy`/`BrowseChildren` paging).
3. Run: `dotnet build` — Expected: PASS.
4. Commit: `feat(galaxyrepo): hierarchy cache + snapshot + refresh service`
### Task 5: Port the reusable gRPC service + DI extension
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** none
**Files:**
- Create: `.../Grpc/GalaxyRepositoryGrpcService.cs`
- Create: `.../DependencyInjection/GalaxyRepositoryServiceCollectionExtensions.cs`
**Steps:**
1. Port `GalaxyRepositoryGrpcService` from mxaccessgw's `Grpc/GalaxyRepositoryGrpcService.cs`, but REMOVE the mxaccessgw-specific `IGatewayRequestIdentityAccessor`/`ApiKeyConstraints` browse-subtree filtering (the gateway will apply its own auth at the interceptor layer). Keep `DiscoverHierarchy`, `BrowseChildren`, `TestConnection`, `GetLastDeployTime`, `WatchDeployEvents`. Base class: `ZB.MOM.WW.GalaxyRepository.Grpc.GalaxyRepository.GalaxyRepositoryBase`.
2. Write `AddZbGalaxyRepository(this IServiceCollection, IConfiguration, string sectionPath)` modeled on mxaccessgw's `AddGalaxyRepository` — bind options from `sectionPath`, register `GalaxyRepository`/`IGalaxyRepository`, notifier, snapshot store, cache, and the refresh `HostedService`. Add a companion `MapZbGalaxyRepository(this IEndpointRouteBuilder)` that `MapGrpcService<GalaxyRepositoryGrpcService>()`.
3. Run: `dotnet build` — Expected: PASS.
4. Commit: `feat(galaxyrepo): reusable gRPC service + AddZbGalaxyRepository DI`
### Task 6: Unit tests for the projector + DI smoke; pack
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** none
**Files:**
- Create: `.../tests/ZB.MOM.WW.GalaxyRepository.Tests/GalaxyHierarchyProjectorTests.cs`
- Create: `.../tests/ZB.MOM.WW.GalaxyRepository.Tests/GalaxyHierarchyCacheTests.cs`
**Steps:**
1. **Write failing tests first:** projector paging (page_token round-trip, max_depth, `historized_only`/`alarm_bearing_only` filters, attribute include toggle) against a hand-built `GalaxyHierarchyCacheEntry` fixture; cache first-load gate + snapshot restore using a fake `IGalaxyRepository`. (SQL provider itself is exercised by env-gated integration later — no live DB in unit tests.)
2. Run: `dotnet test .../ZB.MOM.WW.GalaxyRepository.slnx` — Expected: FAIL (types/asserts).
3. Implement any small helper gaps surfaced; re-run — Expected: PASS.
4. Run: `dotnet pack .../src/ZB.MOM.WW.GalaxyRepository/ZB.MOM.WW.GalaxyRepository.csproj -c Release` — Expected: `ZB.MOM.WW.GalaxyRepository.0.1.0.nupkg` produced.
5. Commit: `test(galaxyrepo): projector + cache tests; pack 0.1.0`
---
## Phase 1 — Sidecar repo scaffold + vendor histsdk
### Task 7: Vendor the histsdk client + its golden tests
**Classification:** high-risk
**Estimated implement time:** ~5 min
**Parallelizable with:** Task 1
**Files:**
- Create: `~/Desktop/HistorianGateway/src/vendor/AVEVA.Historian.Client/**` (copied)
- Create: `~/Desktop/HistorianGateway/tests/AVEVA.Historian.Client.Tests/**` (copied)
- Create: `~/Desktop/HistorianGateway/src/vendor/AVEVA.Historian.Client/VENDORING.md`
**Steps:**
1. `mkdir -p ~/Desktop/HistorianGateway/src/vendor ~/Desktop/HistorianGateway/tests`. Copy `/tmp/histsdk-explore/src/AVEVA.Historian.Client/` and `/tmp/histsdk-explore/tests/AVEVA.Historian.Client.Tests/` into those locations.
2. In the vendored test `.csproj`, REMOVE the `ProjectReference` to `tools/AVEVA.Historian.ReverseEngineering` (not vendored) and delete any test classes that depend on that tooling namespace (the RE-sanitizer tests). KEEP the protocol/golden tests: `HistorianTagWriteProtocolTests`, `HistorianEventRowProtocolTests`, `GrpcEventSendProtocolTests`, `WcfDataQueryProtocolTests`, `StoreForwardOutboxTests`, `RedundancyTests`, version-gate tests. Fix the surviving test `.csproj` ProjectReference path to the new vendored client location.
3. Keep namespace `AVEVA.Historian.Client` as-is (eases re-sync). Write `VENDORING.md` recording: source repo `gitea.dohertylan.com/dohertj2/histsdk`, the commit/date of the snapshot, and "do not hand-edit; re-vendor from upstream."
4. Run: `dotnet build ~/Desktop/HistorianGateway/src/vendor/AVEVA.Historian.Client/AVEVA.Historian.Client.csproj` then `dotnet test ~/Desktop/HistorianGateway/tests/AVEVA.Historian.Client.Tests/` — Expected: build PASS; golden/offline tests PASS (live env-gated tests skip).
5. Commit (in the new repo, after Task 8 inits it — if running before Task 8, defer the commit): `chore(vendor): vendor histsdk AVEVA.Historian.Client + golden tests`
### Task 8: Initialize the sidecar repo + solution + Directory.Build.props
**Classification:** small
**Estimated implement time:** ~3 min
**Parallelizable with:** none (Task 7 output is added here)
**Files:**
- Create: `~/Desktop/HistorianGateway/.gitignore`
- Create: `~/Desktop/HistorianGateway/Directory.Build.props`
- Create: `~/Desktop/HistorianGateway/ZB.MOM.WW.HistorianGateway.slnx`
**Steps:**
1. `git -C ~/Desktop/HistorianGateway init` (this IS its own app repo — unlike shared libs). Add a .NET `.gitignore`.
2. `Directory.Build.props`: `net10.0`, `Nullable`/`ImplicitUsings` enable, `<Platforms>x64</Platforms>`, `<PlatformTarget>x64</PlatformTarget>`, common `LangVersion`.
3. Create `.slnx` referencing: `src/vendor/AVEVA.Historian.Client`, `tests/AVEVA.Historian.Client.Tests` (and the projects added in later phases — add them as created).
4. Run: `dotnet build ~/Desktop/HistorianGateway/ZB.MOM.WW.HistorianGateway.slnx` — Expected: PASS.
5. Commit: `chore: init repo + solution + Directory.Build.props` (then re-commit Task 7's vendored tree if it was deferred).
---
## Phase 2 — Host + configuration + shared-package wiring
### Task 9: Create the Contracts project + historian_gateway.proto skeleton
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** Task 1
**Files:**
- Create: `~/Desktop/HistorianGateway/src/ZB.MOM.WW.HistorianGateway.Contracts/ZB.MOM.WW.HistorianGateway.Contracts.csproj`
- Create: `.../Contracts/Protos/historian_gateway.proto`
**Steps:**
1. `.csproj` net10.0, `Grpc.AspNetCore` 2.76.0, `<Protobuf Include="Protos\*.proto" GrpcServices="Both" />`.
2. Author `historian_gateway.proto` (`package historian_gateway.v1; option csharp_namespace = "ZB.MOM.WW.HistorianGateway.Contracts.Grpc";`) with the **service stubs and message shells** for the 4 historian services: `HistorianRead` (ReadRaw/ReadAggregate/ReadBlocks/ReadEvents server-streaming, ReadAtTime unary), `HistorianWrite` (AddHistoricalValues, SendEvent, WriteLiveValues), `HistorianTags` (BrowseTagNames streaming, GetTagMetadata, EnsureTags, DeleteTags, RenameTags, AddTagExtendedProperties), `HistorianStatus` (Probe, GetConnectionStatus, GetStoreForwardStatus, GetSystemParameter). Map the message fields to the vendored `HistorianSample`/`HistorianAggregateSample`/`HistorianEvent`/`HistorianTagMetadata`/`HistorianHistoricalValue` shapes (timestamps as `google.protobuf.Timestamp`, `RetrievalMode` as an enum mirroring the SDK's 15 modes).
3. Run: `dotnet build` — Expected: PASS; gateway gRPC base classes generated. Add project to `.slnx`.
4. Commit: `feat(contracts): historian_gateway.v1 proto + generated types`
### Task 10: Create the Server project + minimal boot
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** none
**Files:**
- Create: `.../src/ZB.MOM.WW.HistorianGateway.Server/ZB.MOM.WW.HistorianGateway.Server.csproj`
- Create: `.../Server/Program.cs`
- Create: `.../Server/appsettings.json`, `.../Server/appsettings.Development.json`
**Steps:**
1. `.csproj` (Sdk `Microsoft.NET.Sdk.Web`): PackageReferences exactly mirroring mxaccessgw's Server csproj versions — `Grpc.AspNetCore` 2.76.0, `ZB.MOM.WW.Auth.{Abstractions,Ldap,ApiKeys,AspNetCore}` 0.1.2, `ZB.MOM.WW.Audit` 0.1.0, `ZB.MOM.WW.Theme` 0.3.1, `ZB.MOM.WW.Configuration` 0.1.0, `ZB.MOM.WW.Health` 0.1.0, `ZB.MOM.WW.Telemetry`+`.Serilog` 0.1.0, `Serilog.AspNetCore`/`.Sinks.Console`/`.Sinks.File`, `Microsoft.Data.Sqlite` 10.0.7, `Microsoft.Data.SqlClient` 6.0.2, `Polly.Core` 8.6.6. ProjectReferences: Contracts + vendored `AVEVA.Historian.Client` + `ZB.MOM.WW.GalaxyRepository` (project ref to the scadaproj lib, or pkg ref to its 0.1.0 nupkg).
2. `Program.cs`: minimal `WebApplication` that calls `AddZbSerilog`/`AddZbTelemetry` (ServiceName `historian-gateway`), `builder.Services.AddGrpc()`, maps `/healthz` + `/metrics` via `MapZbHealth`/`MapZbMetrics`, boots. (Subsystems wired in later tasks.)
3. Run: `dotnet build` then `dotnet run --project .../Server` and `curl -s localhost:<port>/healthz` — Expected: 200; `curl /metrics` returns Prometheus text. Add project to `.slnx`.
4. Commit: `feat(server): host scaffold + telemetry/serilog/health boot`
### Task 11: Configuration options + validators + ConfigPreflight
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** none
**Files:**
- Create: `.../Server/Configuration/HistorianOptions.cs` + `HistorianOptionsValidator.cs`
- Create: `.../Server/Configuration/GalaxyOptions.cs` (thin wrapper / reuse `GalaxyRepositoryOptions`)
- Create: `.../Server/Configuration/RuntimeDbOptions.cs` + validator (SQL live-write)
- Create: `.../Server/Configuration/RedundancyOptions.cs` + validator
- Create: `.../Server/Configuration/StoreForwardOptions.cs` + validator
- Modify: `.../Server/Program.cs` (register `AddValidatedOptions<,>` + run `ConfigPreflight`)
- Test: `.../tests/ZB.MOM.WW.HistorianGateway.Tests/Configuration/ValidatorTests.cs`
**Steps:**
1. **Write failing validator tests first** using `OptionsValidatorBase`/`ValidationBuilder` semantics (e.g., missing `Historian:Host` → failure; bad port → failure; `Transport` one-of; redundancy `MinCount(members,1)` when enabled). Run — Expected: FAIL.
2. Implement options records + validators (subclass `OptionsValidatorBase<T>`, use `ValidationBuilder.Required/Port/HostPort/OneOf/PositiveTimeSpan/MinCount`). Map `HistorianOptions` → vendored `HistorianClientOptions` (Host, Port default 32565, `Transport=RemoteGrpc`, `GrpcUseTls`, credentials, `AllowUntrustedServerCertificate`).
3. In `Program.cs`, `AddValidatedOptions<,>` each, and run a `ConfigPreflight` (RequireValue host, RequirePort) before host build.
4. Run: `dotnet test` — Expected: PASS.
5. Commit: `feat(server): validated options + ConfigPreflight`
---
## Phase 3 — Connection layer (vendored client → gateway)
### Task 12: `IHistorianClient` seam over the vendored client
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** none
**Files:**
- Create: `.../Server/Historian/IHistorianClient.cs` (interface mirroring the read/write methods the services need)
- Create: `.../Server/Historian/VendoredHistorianClient.cs` (adapts `AVEVA.Historian.Client.HistorianClient`)
- Test: `.../tests/.../Historian/HistorianClientSeamTests.cs`
**Steps:**
1. **Write failing test** that a `FakeHistorianClient : IHistorianClient` can be substituted and returns canned samples (this seam is what makes the gRPC services unit-testable without a live historian). Run — Expected: FAIL.
2. Define `IHistorianClient` with the methods the services call (ReadRaw/ReadAggregate/ReadAtTime/ReadBlocks/ReadEvents/BrowseTagNames/GetTagMetadata/Probe/GetConnectionStatus/GetStoreForwardStatus/GetSystemParameter/AddHistoricalValues/SendEvent/EnsureTag/DeleteTag/RenameTags/AddTagExtendedProperties). Implement `VendoredHistorianClient` delegating to the real `HistorianClient`.
3. Run: `dotnet test` — Expected: PASS.
4. Commit: `feat(historian): IHistorianClient seam + vendored adapter`
### Task 13: Connection pool (pre-authenticated, reused, health-checked)
**Classification:** high-risk
**Estimated implement time:** ~5 min
**Parallelizable with:** none
**Files:**
- Create: `.../Server/Historian/HistorianConnectionPool.cs` (+ `IHistorianConnectionPool`)
- Modify: `.../Server/Program.cs` (DI singleton)
- Test: `.../tests/.../Historian/HistorianConnectionPoolTests.cs`
**Steps:**
1. **Write failing test** asserting the pool opens/authenticates a connection once and reuses it across N borrow calls (count handshakes via a fake transport/lease factory), and that a faulted connection is evicted + re-created. Run — Expected: FAIL.
2. Implement a lease-based pool keyed by target; lazy-open with the auth handshake once; reuse; `SemaphoreSlim`-guarded reconnect on fault; expose `Lease()` returning a pooled `IHistorianClient`. (The vendored client is `IAsyncDisposable`; the pool owns lifecycle.)
3. Run: `dotnet test` — Expected: PASS.
4. Commit: `feat(historian): pooled pre-authenticated connection pool`
### Task 14: Store-forward + redundancy + SQL live-write wiring
**Classification:** high-risk
**Estimated implement time:** ~5 min
**Parallelizable with:** none
**Files:**
- Create: `.../Server/Historian/HistorianWriteCoordinator.cs` (routes writes → pool, store-forward, or redundancy per config)
- Create: `.../Server/Historian/SqlLiveValueWriter.cs` (`WriteLiveValues` via `aaAnalogTagInsert` + `INSERT INTO History`)
- Modify: `.../Server/Program.cs`
- Test: `.../tests/.../Historian/HistorianWriteCoordinatorTests.cs`, `.../SqlLiveValueWriterTests.cs`
**Steps:**
1. **Write failing tests:** (a) when store-forward enabled + historian unreachable, the coordinator enqueues (uses vendored `HistorianStoreForwardWriter` over a fake sink) and reports `Queued`; (b) when redundancy configured, it fans out via `HistorianRedundantClient` and returns per-member results under All/Any; (c) `SqlLiveValueWriter` builds the correct parameterized command sequence (assert against a fake `IDbCommand` recorder — no live SQL). Run — Expected: FAIL.
2. Implement the coordinator (compose vendored `HistorianStoreForwardWriter` + `HistorianRedundantClient` from config) and `SqlLiveValueWriter` (omit the server-managed `Quality` column; honor the storage-activation note from the SQL reference memory).
3. Run: `dotnet test` — Expected: PASS.
4. Commit: `feat(historian): write coordinator (store-forward + redundancy) + SQL live-write`
---
## Phase 4 — gRPC services + auth interceptor
### Task 15: `HistorianRead` service (representative TDD task; sets the pattern)
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** Task 17 after the mapper exists
**Files:**
- Create: `.../Server/Grpc/HistorianReadService.cs`
- Create: `.../Server/Grpc/HistorianProtoMapper.cs` (SDK model ↔ proto)
- Modify: `.../Server/Program.cs` (`MapGrpcService`)
- Test: `.../tests/.../Grpc/HistorianReadServiceTests.cs`
**Steps:**
1. **Write failing test:** with a `FakeHistorianClient` yielding 3 `HistorianSample`s, calling `ReadRaw` streams 3 mapped proto rows; `ReadAggregate` passes the right `RetrievalMode`+interval; an unknown tag → `RpcException(NotFound)`; bad time range → `InvalidArgument`. Use an in-memory `IServerStreamWriter<T>` capture. Run — Expected: FAIL.
2. Implement `HistorianReadService : HistorianRead.HistorianReadBase` consuming `IHistorianConnectionPool.Lease()`; implement `HistorianProtoMapper` (Timestamp conversions, RetrievalMode enum map). Map exceptions per design §7.
3. Run: `dotnet test` — Expected: PASS.
4. Commit: `feat(grpc): HistorianRead service + proto mapper`
### Task 16: `HistorianWrite` service
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** Task 17, Task 18 (no file overlap)
**Files:** Create `.../Server/Grpc/HistorianWriteService.cs`; Modify `Program.cs`; Test `.../Grpc/HistorianWriteServiceTests.cs`
**Steps:** TDD per the Task 15 pattern. `AddHistoricalValues`/`SendEvent` route through `HistorianWriteCoordinator`; `WriteLiveValues` through `SqlLiveValueWriter`. Map `ProtocolEvidenceMissingException``Unimplemented`, unreachable+store-forward → `OK` with `Queued` status, redundancy per-member results into the reply. Commit: `feat(grpc): HistorianWrite service`.
### Task 17: `HistorianTags` service
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** Task 16, Task 18
**Files:** Create `.../Server/Grpc/HistorianTagsService.cs`; Modify `Program.cs`; Test `.../Grpc/HistorianTagsServiceTests.cs`
**Steps:** TDD. `BrowseTagNames` (streaming), `GetTagMetadata`, `EnsureTags`/`DeleteTags`/`RenameTags`/`AddTagExtendedProperties` via the seam/pool. Map unsupported tag types (`ProtocolEvidenceMissingException`) → `FailedPrecondition`. Commit: `feat(grpc): HistorianTags service`.
### Task 18: `HistorianStatus` service
**Classification:** small
**Estimated implement time:** ~4 min
**Parallelizable with:** Task 16, Task 17
**Files:** Create `.../Server/Grpc/HistorianStatusService.cs`; Modify `Program.cs`; Test `.../Grpc/HistorianStatusServiceTests.cs`
**Steps:** TDD. `Probe`/`GetConnectionStatus`/`GetStoreForwardStatus`/`GetSystemParameter`. Commit: `feat(grpc): HistorianStatus service`.
### Task 19: Galaxy gRPC wiring (consume the shared lib)
**Classification:** small
**Estimated implement time:** ~3 min
**Parallelizable with:** Task 1618
**Files:** Modify `.../Server/Program.cs` (`AddZbGalaxyRepository(config, "Galaxy")` + `MapZbGalaxyRepository()`); Modify `appsettings.json`
**Steps:** Register the shared lib's service + refresh hosted service; add `Galaxy:ConnectionString` config. Run: `dotnet run` + grpcurl `DiscoverHierarchy` against a fake/empty config returns `Unavailable` until cache loads (no live DB needed to prove wiring). Commit: `feat(server): wire shared GalaxyRepository gRPC service`.
### Task 20: API-key auth interceptor + scope resolver
**Classification:** high-risk
**Estimated implement time:** ~5 min
**Parallelizable with:** none
**Files:**
- Create: `.../Server/Security/GatewayGrpcScopeResolver.cs` (maps request type → scope)
- Create: `.../Server/Security/GatewayGrpcAuthorizationInterceptor.cs`
- Create: `.../Server/Security/GatewayScopes.cs` (`historian:read|write`, `historian:tags:write`, `galaxy:read`)
- Modify: `.../Server/Program.cs` (`AddZbApiKeyAuth` + `AddGrpc(o => o.Interceptors.Add<...>())`)
- Test: `.../tests/.../Security/GrpcAuthorizationTests.cs`
**Steps:**
1. **Write failing tests:** missing/invalid key → `Unauthenticated`; valid key without the required scope → `PermissionDenied`; valid key with scope → continuation runs. Fake `IApiKeyVerifier`. Run — Expected: FAIL.
2. Implement modeled on mxaccessgw's `GatewayGrpcAuthorizationInterceptor` + `GatewayGrpcScopeResolver` (switch on request type → scope), using shared `IApiKeyVerifier.VerifyAsync`. Respect a `Disabled` auth mode for dev.
3. Run: `dotnet test` — Expected: PASS.
4. Commit: `feat(security): gRPC API-key interceptor + scope enforcement`
---
## Phase 5 — Audit
### Task 21: Canonical SQLite audit writer + actor accessor + wiring
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** Task 22 (dashboard auth) after interfaces exist
**Files:**
- Create: `.../Server/Audit/SqliteAuditWriter.cs` (`IAuditWriter`), `.../Server/Audit/HttpAuditActorAccessor.cs` (`IAuditActorAccessor`)
- Modify: write services (Tasks 16,17) + interceptor (Task 20) to emit `AuditEvent`s
- Modify: `.../Server/Program.cs` (`AddZbAudit` + register writer/actor)
- Test: `.../tests/.../Audit/SqliteAuditWriterTests.cs`
**Steps:**
1. **Write failing test:** writing an `AuditEvent` persists a row with the canonical 9 fields (`EventId`/`OccurredAtUtc`/`Actor`/`Action`/`Outcome`/`Category`/`Target`/`SourceNode`/`DetailsJson`), domain fields in `DetailsJson`; writer swallows internal errors. Use an in-memory SQLite. Run — Expected: FAIL.
2. Implement the SQLite writer (table create-if-missing) modeled on MxGateway's audit store; `HttpAuditActorAccessor` reads the Auth principal. Emit audit at tag/value/event writes, API-key admin, login/logout, with `Actor` from the accessor.
3. Run: `dotnet test` — Expected: PASS.
4. Commit: `feat(audit): canonical SQLite audit writer + actor wiring`
---
## Phase 6 — Blazor dashboard
### Task 22: Dashboard shell, LDAP cookie auth, login/logout
**Classification:** high-risk
**Estimated implement time:** ~5 min
**Parallelizable with:** none
**Files:**
- Create: `.../Server/Dashboard/Components/{App,Routes,_Imports}.razor`, `Layout/{MainLayout,LoginLayout}.razor`, `Pages/Login.razor`
- Create: `.../Server/Dashboard/DashboardServiceCollectionExtensions.cs`, `.../Dashboard/DashboardEndpointRouteBuilderExtensions.cs`, `.../Dashboard/DashboardAuthenticator.cs`, `.../Dashboard/DashboardGroupRoleMapper.cs`
- Modify: `Program.cs` (`AddGatewayDashboard` + `MapRazorComponents<App>` + auth/antiforgery middleware)
- Test: `.../tests/ZB.MOM.WW.HistorianGateway.Tests/bUnit/LayoutRenderTests.cs`
**Steps:**
1. **Write failing bUnit test** that `MainLayout` renders `<ThemeShell>` with the nav rail and `LoginCard` renders on the login page. Run — Expected: FAIL.
2. Port the dashboard shell from mxaccessgw (`App.razor` with `ThemeHead`/`ThemeScripts`, `MainLayout` with `ThemeShell`+`NavRailSection`/`NavRailItem`, `Login.razor` using `LoginCard` posting to `/auth/login`). Wire `AddZbLdapAuth(config,"Ldap")`, cookie auth via `ZbCookieDefaults.Apply`, `IGroupRoleMapper<CanonicalRole>`, `DisableLogin` switch, `IAuditActorAccessor`.
3. Run: `dotnet test` (bUnit) then `dotnet run` and load `/login` in a browser/curl — Expected: tests PASS; login page renders themed.
4. Commit: `feat(dashboard): Theme shell + LDAP cookie auth + login`
### Task 23: Status + Health pages
**Classification:** small
**Estimated implement time:** ~4 min
**Parallelizable with:** Task 24, Task 25
**Files:** Create `.../Dashboard/Components/Pages/{StatusPage,HealthPage}.razor` (+ a `DashboardStatusService`); Test bUnit render.
**Steps:** TDD bUnit render. Status shows pool state, store-forward queue depth, redundancy members, version (from a status service reading the pool/coordinator). Commit: `feat(dashboard): status + health pages`.
### Task 24: Galaxy browser page
**Classification:** small
**Estimated implement time:** ~5 min
**Parallelizable with:** Task 23, Task 25
**Files:** Create `.../Dashboard/Components/Pages/GalaxyBrowserPage.razor` + tree node view (port mxaccessgw `BrowsePage`/`BrowseTreeNodeView`, read-only, no add-tag); Test bUnit.
**Steps:** TDD bUnit render against the shared lib's cache. Commit: `feat(dashboard): read-only Galaxy browser`.
### Task 25: Historian console page (query + role-gated write test)
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** Task 23, Task 24
**Files:** Create `.../Dashboard/Components/Pages/HistorianConsolePage.razor` (+ `DashboardHistorianService` calling the seam/pool); Test bUnit.
**Steps:** TDD bUnit. Query form (tag, time range, raw/aggregate + mode picker) renders results; write-test panel (historical value insert / event send) visible only to Engineer+ roles via `AuthorizeView`. Commit: `feat(dashboard): historian query + role-gated write console`.
### Task 26: API-key admin page
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** Task 2325
**Files:** Create `.../Dashboard/Components/Pages/ApiKeysPage.razor` (+ `DashboardApiKeyManagementService` over the shared ApiKeys store); Test bUnit.
**Steps:** TDD bUnit. List/create (show secret once)/revoke keys with scope selection. Commit: `feat(dashboard): API-key admin`.
---
## Phase 7 — Telemetry meters + Health probes
### Task 27: App meters
**Classification:** small
**Estimated implement time:** ~4 min
**Parallelizable with:** Task 28
**Files:** Create `.../Server/Observability/GatewayMetrics.cs`; Modify services/coordinator/pool to record; Modify `Program.cs` (`o.Meters=[GatewayMetrics.MeterName]`); Test `.../Observability/GatewayMetricsTests.cs`.
**Steps:** TDD with `MeterListener`. Counters/histograms: read/write counts + latency, store-forward queue depth (observable gauge), pool connection state, redundancy ack outcomes. Commit: `feat(obs): gateway meters`.
### Task 28: Health probes
**Classification:** small
**Estimated implement time:** ~4 min
**Parallelizable with:** Task 27
**Files:** Create `.../Server/Health/{HistorianConnectionHealthCheck,StoreForwardDrainHealthCheck}.cs`; Modify `Program.cs` (`AddHealthChecks` with `GrpcDependencyHealthCheck` for historian, SQL checks for Galaxy + Runtime DB, custom checks, tagged `ZbHealthTags.Ready`); Test health-check unit tests.
**Steps:** TDD. Probes flip Unhealthy when a dependency is down (fake deps). Commit: `feat(health): historian/galaxy/runtime-db/store-forward probes`.
---
## Phase 8 — Integration, docs, repo
### Task 29: Env-gated live integration tests
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** none
**Files:** Create `.../tests/.../Integration/{HistorianRoundTripTests,GalaxyBrowseTests}.cs`
**Steps:** Gated on `HISTORIAN_GRPC_HOST`/`HISTORIAN_GRPC_WRITE_SANDBOX_TAG` and a Galaxy SQL connection env var; `Skip` when absent. Cover read→write→read-back via the self-cleaning sandbox-tag lifecycle and a Galaxy `DiscoverHierarchy`. Run `dotnet test` (skips locally). Commit: `test: env-gated live integration`.
### Task 30: Full-suite green gate + smoke
**Classification:** small
**Estimated implement time:** ~3 min
**Parallelizable with:** none
**Steps:** Run `dotnet build ZB.MOM.WW.HistorianGateway.slnx` + `dotnet test` (whole solution) on macOS with no live env — Expected: ALL green, live tests skipped. `dotnet run` + curl `/healthz` (200), `/metrics` (text), grpcurl `HistorianStatus/Probe`. Fix any gaps. Commit: `chore: green gate + smoke`.
### Task 31: CLAUDE.md + README + gitea remote + scadaproj index
**Classification:** small
**Estimated implement time:** ~5 min
**Parallelizable with:** none
**Files:** Create `~/Desktop/HistorianGateway/{CLAUDE.md,README.md}`; copy the two design/plan docs into its `docs/plans/`; Modify `~/Desktop/scadaproj/CLAUDE.md` (index the new sidecar + note the GalaxyRepository follow-on for mxaccessgw).
**Steps:**
1. Write `CLAUDE.md` (overview, build/run/test commands, the no-COM single-process note, the vendored-histsdk + shared-GalaxyRepository dependencies, config sections, env vars) and `README.md`.
2. Create the gitea repo `historiangw` and push: `git -C ~/Desktop/HistorianGateway remote add origin https://gitea.dohertylan.com/dohertj2/historiangw.git && git push -u origin main` (confirm remote name/visibility with the user first).
3. Update scadaproj's umbrella `CLAUDE.md` runtime/implementation table with the new project row; commit scadaproj separately.
4. Commit: `docs: CLAUDE.md + README; index in scadaproj`.
---
## Dependency summary (for parallel dispatch)
- **Foundational, no blockers:** Task 1 (galaxy lib scaffold), Task 7 (vendor histsdk), Task 8 (repo init) — Task 8 consumes Task 7's tree.
- **Galaxy lib chain:** 2→3→4→5→6 (sequential; share files).
- **Sidecar chain:** 8→9→10→11→12→13→14, then gRPC services 15→(16,17,18 parallel),19, then 20, then 21.
- **Dashboard:** 22→(23,24,25,26 parallel) after Task 20 (auth) + Task 13/14 (data) + Task 5/19 (galaxy).
- **Obs:** 27,28 parallel after Task 14.
- **Close-out:** 29→30→31 after everything.
## Notes / non-goals (from design §9)
- No `AddS2` live streaming-sample writes (GATED) — live values only via SQL `WriteLiveValues`.
- No two-process/x86 worker (no COM).
- mxaccessgw adopting `ZB.MOM.WW.GalaxyRepository` is a tracked follow-on, NOT in this plan.
@@ -0,0 +1,37 @@
{
"planPath": "docs/plans/2026-06-23-historian-gateway-implementation.md",
"tasks": [
{"id": 1, "subject": "Task 1: Scaffold the GalaxyRepository lib project", "status": "pending"},
{"id": 2, "subject": "Task 2: Port the canonical galaxy_repository.proto (neutral namespace)", "status": "pending", "blockedBy": [1]},
{"id": 3, "subject": "Task 3: Port the SQL browse provider", "status": "pending", "blockedBy": [2]},
{"id": 4, "subject": "Task 4: Port the in-memory hierarchy cache + snapshot + refresh service", "status": "pending", "blockedBy": [3]},
{"id": 5, "subject": "Task 5: Port the reusable gRPC service + DI extension", "status": "pending", "blockedBy": [4]},
{"id": 6, "subject": "Task 6: Unit tests for projector/cache; pack 0.1.0", "status": "pending", "blockedBy": [5]},
{"id": 7, "subject": "Task 7: Vendor histsdk AVEVA.Historian.Client + golden tests", "status": "pending"},
{"id": 8, "subject": "Task 8: Init sidecar repo + solution + Directory.Build.props", "status": "pending", "blockedBy": [7]},
{"id": 9, "subject": "Task 9: Contracts project + historian_gateway.proto skeleton", "status": "pending", "blockedBy": [8]},
{"id": 10, "subject": "Task 10: Server project + minimal boot (telemetry/serilog/health)", "status": "pending", "blockedBy": [9, 7, 6]},
{"id": 11, "subject": "Task 11: Configuration options + validators + ConfigPreflight", "status": "pending", "blockedBy": [10]},
{"id": 12, "subject": "Task 12: IHistorianClient seam over vendored client", "status": "pending", "blockedBy": [10]},
{"id": 13, "subject": "Task 13: Connection pool (pre-authenticated, reused)", "status": "pending", "blockedBy": [12]},
{"id": 14, "subject": "Task 14: Write coordinator (store-forward + redundancy) + SQL live-write", "status": "pending", "blockedBy": [13]},
{"id": 15, "subject": "Task 15: HistorianRead service + proto mapper", "status": "pending", "blockedBy": [13]},
{"id": 16, "subject": "Task 16: HistorianWrite service", "status": "pending", "blockedBy": [14, 15]},
{"id": 17, "subject": "Task 17: HistorianTags service", "status": "pending", "blockedBy": [13, 15]},
{"id": 18, "subject": "Task 18: HistorianStatus service", "status": "pending", "blockedBy": [13, 15]},
{"id": 19, "subject": "Task 19: Galaxy gRPC wiring (consume shared lib)", "status": "pending", "blockedBy": [10, 5]},
{"id": 20, "subject": "Task 20: API-key auth interceptor + scope resolver", "status": "pending", "blockedBy": [15]},
{"id": 21, "subject": "Task 21: SQLite audit writer + actor accessor + wiring", "status": "pending", "blockedBy": [16, 17, 20]},
{"id": 22, "subject": "Task 22: Dashboard shell + LDAP cookie auth + login", "status": "pending", "blockedBy": [20]},
{"id": 23, "subject": "Task 23: Status + Health pages", "status": "pending", "blockedBy": [22, 14]},
{"id": 24, "subject": "Task 24: Galaxy browser page", "status": "pending", "blockedBy": [22, 19]},
{"id": 25, "subject": "Task 25: Historian console page (role-gated write)", "status": "pending", "blockedBy": [22, 15]},
{"id": 26, "subject": "Task 26: API-key admin page", "status": "pending", "blockedBy": [22]},
{"id": 27, "subject": "Task 27: App meters", "status": "pending", "blockedBy": [14]},
{"id": 28, "subject": "Task 28: Health probes", "status": "pending", "blockedBy": [14, 19]},
{"id": 29, "subject": "Task 29: Env-gated live integration tests", "status": "pending", "blockedBy": [16, 17, 18, 19]},
{"id": 30, "subject": "Task 30: Full-suite green gate + smoke", "status": "pending", "blockedBy": [21, 26, 27, 28, 29]},
{"id": 31, "subject": "Task 31: CLAUDE.md + README + gitea remote + scadaproj index", "status": "pending", "blockedBy": [30]}
],
"lastUpdated": "2026-06-23"
}
+114605
View File
File diff suppressed because it is too large Load Diff
+324
View File
@@ -0,0 +1,324 @@
#!/usr/bin/env python3
"""
Generate a fake company's Unified Namespace (UNS) for OtOpcUa, grounded in the
REAL AVEVA Galaxy "DEV" hierarchy pulled from the MxAccess gateway
(galaxy-hierarchy.json).
Mapping onto OtOpcUa's config-DB model (ServerCluster -> Namespace(Kind=Equipment)
-> UnsArea -> UnsLine -> Equipment -> Tag), with the canonical UNS path:
Enterprise / Site / Area / Line / Equipment / Signal
Real grounding:
* The 40 $TestMachine instances (TestMachine_001..040) become Equipment ("machines").
* The Galaxy areas they are deployed in (TestArea / TestArea2 / TestArea3)
become UNS Areas.
* Each machine's real DELMIA + MES receiver children (resolved from Galaxy
containment) become tag sub-folders ("delmia" / "mes") whose Tags carry the
REAL MXAccess fullTagReference.
* Process Tags map the real $TestMachine UDAs (TestChangingInt, TestDouble, ...)
to believable signal names while preserving the real reference.
Synthesized ("fake company") parts: enterprise/site/area/line names, machine
archetypes, and asset metadata (manufacturer/model/serial/SAP id). Everything
synthetic is clearly under generated identifiers; every node keeps a `source`
block linking back to the real Galaxy object.
OtOpcUa UNS-segment rule enforced: Area/Line/Equipment names match ^[a-z0-9-]{1,32}$.
"""
import json, re, collections, hashlib
SRC = "galaxy-hierarchy.json"
SEG = re.compile(r"^[a-z0-9-]{1,32}$")
def seg(s):
assert SEG.match(s), f"invalid UNS segment: {s!r}"
return s
# ---------------------------------------------------------------- load source
gal = json.load(open(SRC))
objs = gal["objects"]
byid = {o["gobjectId"]: o for o in objs}
def name(i): return byid[i]["tagName"] if i in byid else None
def children(pid): return [o for o in objs if o.get("parentGobjectId") == pid]
def chain_has(o, frag):
return any(frag in c for c in o.get("templateChain", []))
# the 40 top-level machines, grouped by their Galaxy area
AREAS_GAL = {} # galaxy area tag -> list of machine objects (sorted by number)
for o in objs:
if "$TestMachine" in o.get("templateChain", []): # exactly $TestMachine (not .DelmiaReceiver etc.)
area = name(o.get("parentGobjectId"))
AREAS_GAL.setdefault(area, []).append(o)
for a in AREAS_GAL:
AREAS_GAL[a].sort(key=lambda o: o["tagName"])
# resolve each machine's real DELMIA / MES receiver child tag names
def receiver_refs(machine):
delmia = mes = None
for c in children(machine["gobjectId"]):
if chain_has(c, "DelmiaReceiver"): delmia = c
elif chain_has(c, "MESReceiver"): mes = c
return delmia, mes
# ---------------------------------------------------------------- fake company
ENTERPRISE = "northwind" # fictional consumer-goods manufacturer
SITE = "birmingham" # fictional plant (maps to Galaxy node/galaxy DEV)
GALAXY_NODE = "DESKTOP-6JL3KKO"
GALAXY = "DEV"
# Galaxy area -> (uns area segment, friendly label, line plan)
# line plan: list of (line-segment, count) consuming machines in order.
AREA_PLAN = {
"TestArea": ("filling", "Filling & Capping", [("line-1", 7), ("line-2", 6), ("line-3", 6)]),
"TestArea2": ("blending", "Blending & CIP", [("cip-1", 1)]),
"TestArea3": ("packaging", "Packaging & Palletizing",
[("pack-1", 5), ("pack-2", 5), ("pack-3", 5), ("pack-4", 5)]),
}
# believable machine archetypes cycled within a line, per area
ARCHETYPES = {
"filling": [("rinser","krones","Hydra"),("filler","sidel","SF300"),
("capper","khs","Innofill"),("labeler","krones","Contiroll"),
("inspector","antares-vision","Vmax"),("coder","videojet","1580")],
"blending": [("blender","spx-flow","APV-R5")],
"packaging": [("cartoner","marchesini","MC820"),("case-packer","bosch","Elematic"),
("palletizer","fanuc","M410"),("stretch-wrapper","lantech","Q300"),
("checkweigher","mettler-toledo","C3570")],
}
def asset(area_seg, archetype, model_series, gtag, gidx):
mfr_model = {a[0]:(a[1],a[2]) for a in ARCHETYPES[area_seg]}
mfr, model = mfr_model[archetype]
h = int(hashlib.sha1(gtag.encode()).hexdigest(), 16)
return {
"manufacturer": mfr,
"model": f"{model} Srs{2 + h % 4}",
"serialNumber": f"SN26{gidx:05d}",
"sapId": f"100{200000 + h % 700000}",
"zTag": f"Z{area_seg[:3].upper()}{gidx:04d}",
"hardwareRevision": f"H{1 + h % 4}.{h % 10}",
"softwareRevision": f"S{2 + h % 3}.{(h>>4) % 10}",
"equipmentClassRef": f"urn:northwind:equipclass:{archetype}",
}
# ----------------------------------------------------- signal (Tag) templates
def dtype(galaxy_type):
return {
"Boolean":"Boolean","Integer":"Int32","Double":"Double","Float":"Float",
"String":"String","Time":"DateTime","ElapsedTime":"Double",
"InternationalizedString":"String",
}.get(galaxy_type, "String")
# process signals: (uns tag name, real $TestMachine UDA, galaxy type, access, flags)
PROCESS_SIGNALS = [
("speed-rpm", "TestChangingInt", "Integer", "ReadOnly", {}),
("production-count", "TestHistoryValue","Integer", "ReadOnly", {"historize": True}),
("temperature-c", "TestDouble", "Double", "ReadOnly", {}),
("pressure-bar", "TestFloat", "Float", "ReadOnly", {}),
("cycle-time-s", "TestDuration", "ElapsedTime","ReadOnly",{}),
("last-cycle-ts", "TestDateTime", "Time", "ReadOnly", {}),
("safety-interlock", "ProtectedValue", "Boolean", "ReadWrite", {"secured": True}),
("maint-lockout", "ProtectedValue1", "Boolean", "ReadWrite", {"secured": True}),
("motor-fault", "TestAlarm001", "Boolean", "ReadOnly", {"alarm": True}),
("over-temp", "TestAlarm002", "Boolean", "ReadOnly", {"alarm": True}),
("jam-detected", "TestAlarm003", "Boolean", "ReadOnly", {"alarm": True}),
("in-alarm", "InAlarm", "Boolean", "ReadOnly", {}),
]
# DELMIA receiver signals (real $DelmiaReceiver attributes)
DELMIA_SIGNALS = [
("work-order", "WorkOrderNumber", "String", "ReadWrite", {}),
("part-number", "PartNumber", "String", "ReadWrite", {}),
("job-step", "JobStepNumber", "String", "ReadWrite", {}),
("recipe-path", "DownloadPath", "String", "ReadWrite", {}),
("recipe-dl-req", "RecipeDownloadFlag","Boolean","ReadWrite", {}),
("recipe-done", "RecipeProcessedFlag","Boolean","ReadOnly", {}),
("recipe-result", "RecipeProcessResult","Boolean","ReadOnly", {}),
("ready", "ReadyFlag", "Boolean","ReadOnly", {}),
]
# MES receiver signals (real $MESReceiver attributes)
MES_SIGNALS = [
("move-in-req", "MoveInFlag", "Boolean","ReadWrite", {}),
("move-in-batch", "MoveInBatchID", "Integer","ReadWrite", {}),
("move-in-operator","MoveInOperatorName", "String", "ReadOnly", {}),
("move-in-ok", "MoveInSuccessFlag", "Boolean","ReadOnly", {}),
("move-out-req", "MoveOutFlag", "Boolean","ReadWrite", {}),
("move-out-batch", "MoveOutBatchID", "Integer","ReadWrite", {}),
("move-out-ok", "MoveOutSuccessfulFlag","Boolean","ReadOnly", {}),
("container-id", "MoveOutMesContainerNum","String","ReadOnly", {}),
]
def attr_set(obj):
return {a["attributeName"] for a in obj["attributes"]} if obj else set()
def make_tags(equipment_id, machine, delmia, mes):
tags = []
def add(folder, signals, ref_obj):
if ref_obj is None:
return
ref_tag = ref_obj["tagName"]
have = attr_set(ref_obj)
for uns_name, attr, gtype, access, flags in signals:
if attr not in have:
continue # only emit signals whose real attribute exists on THIS instance
full = f"{ref_tag}.{attr}"
tid = f"tag-{equipment_id}-{(folder+'-' if folder else '')}{uns_name}"
t = {
"tagId": tid,
"name": uns_name,
"folderPath": folder,
"dataType": dtype(gtype),
"accessLevel": access,
"historize": bool(flags.get("historize", False)),
"tagConfig": {
"isAlarm": bool(flags.get("alarm", False)),
"secured": bool(flags.get("secured", False)),
},
"source": {
"namespaceKind": "SystemPlatform",
"fullTagReference": full,
"galaxyType": gtype,
},
}
tags.append(t)
add("", PROCESS_SIGNALS, machine)
add("delmia", DELMIA_SIGNALS, delmia)
add("mes", MES_SIGNALS, mes)
return tags
# ---------------------------------------------------------------- build UNS
cluster_id = f"{ENTERPRISE}-{SITE}"
uns_areas, uns_lines, equipment = [], [], []
src_hierarchy = {"galaxyNode": GALAXY_NODE, "galaxy": GALAXY, "areas": []}
gidx = 0
for gal_area, (area_seg, area_label, line_plan) in AREA_PLAN.items():
machines = AREAS_GAL.get(gal_area, [])
area_id = f"area-{area_seg}"
uns_areas.append({
"unsAreaId": area_id, "clusterId": cluster_id, "name": seg(area_seg),
"notes": area_label,
"source": {"galaxyArea": gal_area,
"galaxyGobjectId": next((o["gobjectId"] for o in objs
if o["tagName"] == gal_area), None)},
})
src_area = {"galaxyArea": gal_area, "machines": []}
# consume machines line-by-line per the plan
cursor = 0
for line_seg, count in line_plan:
line_id = f"line-{area_seg}-{line_seg}"
uns_lines.append({
"unsLineId": line_id, "unsAreaId": area_id, "name": seg(line_seg),
"notes": f"{area_label} {line_seg}",
})
archetypes = ARCHETYPES[area_seg]
for j in range(count):
if cursor >= len(machines): break
m = machines[cursor]; cursor += 1; gidx += 1
archetype, _, model_series = archetypes[j % len(archetypes)]
eq_seg = seg(f"{archetype}-{gidx:02d}")
eq_id = f"eq-{area_seg}-{line_seg}-{eq_seg}"
delmia, mes = receiver_refs(m)
a = asset(area_seg, archetype, model_series, m["tagName"], gidx)
tags = make_tags(eq_id, m, delmia, mes)
equipment.append({
"equipmentId": eq_id, "unsLineId": line_id, "name": eq_seg,
"machineCode": m["tagName"],
"manufacturer": a["manufacturer"], "model": a["model"],
"serialNumber": a["serialNumber"], "sapId": a["sapId"],
"zTag": a["zTag"], "hardwareRevision": a["hardwareRevision"],
"softwareRevision": a["softwareRevision"],
"assetLocation": f"{ENTERPRISE}/{SITE}/{area_seg}/{line_seg}",
"equipmentClassRef": a["equipmentClassRef"],
"unsPath": f"{ENTERPRISE}/{SITE}/{area_seg}/{line_seg}/{eq_seg}",
"source": {
"namespaceKind": "SystemPlatform",
"galaxyTag": m["tagName"], "galaxyGobjectId": m["gobjectId"],
"templateChain": m["templateChain"], "galaxyArea": gal_area,
"delmiaReceiverTag": delmia["tagName"] if delmia else None,
"mesReceiverTag": mes["tagName"] if mes else None,
},
"tags": tags,
})
src_area["machines"].append({
"tag": m["tagName"], "gobjectId": m["gobjectId"],
"unsEquipment": f"{ENTERPRISE}/{SITE}/{area_seg}/{line_seg}/{eq_seg}",
"delmiaReceiver": delmia["tagName"] if delmia else None,
"mesReceiver": mes["tagName"] if mes else None,
})
src_hierarchy["areas"].append(src_area)
tag_count = sum(len(e["tags"]) for e in equipment)
doc = {
"$schema": "internal://otopcua/uns-export/v1",
"kind": "fake-company-uns",
"description": ("Fictional company UNS generated around the real AVEVA Galaxy "
"'DEV' TestMachine instances and the areas they are deployed in. "
"Modeled on OtOpcUa's Cluster->Namespace->UnsArea->UnsLine->"
"Equipment->Tag schema. Synthetic naming/asset metadata; every "
"node links back to its real Galaxy source."),
"generatedFrom": {
"file": SRC, "galaxyNode": GALAXY_NODE, "galaxy": GALAXY,
"gatewayEndpoint": "http://10.100.0.48:5120 (MxAccess gateway, gRPC)",
"sourceObjectCount": len(objs),
},
"company": {"name": "Northwind Consumer Products",
"enterprise": ENTERPRISE, "site": SITE,
"siteDescription": "Birmingham bottling & packaging plant"},
"cluster": {"clusterId": cluster_id, "name": f"{ENTERPRISE}-{SITE}-uns",
"enterprise": ENTERPRISE, "site": SITE,
"redundancyMode": "Hot", "nodeCount": 2},
"namespace": {"namespaceId": f"{cluster_id}-equipment", "clusterId": cluster_id,
"kind": "Equipment",
"namespaceUri": f"urn:{ENTERPRISE}:{SITE}:uns"},
"uns": {"unsAreas": uns_areas, "unsLines": uns_lines, "equipment": equipment},
"stats": {"areas": len(uns_areas), "lines": len(uns_lines),
"equipment": len(equipment), "tags": tag_count},
"sourceGalaxyHierarchy": src_hierarchy,
}
json.dump(doc, open("company-uns.json", "w"), indent=2)
# ---------------------------------------------------------------- tree view
lines = []
lines.append(f"Northwind Consumer Products — Unified Namespace")
lines.append(f"(generated from Galaxy {GALAXY_NODE}\\{GALAXY}; "
f"{len(equipment)} machines, {tag_count} signals)")
lines.append("")
lines.append(f"{ENTERPRISE}")
lines.append(f"└─ {SITE}")
area_by_id = {a["unsAreaId"]: a for a in uns_areas}
lines_by_area = collections.defaultdict(list)
for l in uns_lines: lines_by_area[l["unsAreaId"]].append(l)
eq_by_line = collections.defaultdict(list)
for e in equipment: eq_by_line[e["unsLineId"]].append(e)
for ai, a in enumerate(uns_areas):
a_last = ai == len(uns_areas) - 1
a_pre = " " + ("└─ " if a_last else "├─ ")
lines.append(f" {'└─' if a_last else '├─'} {a['name']}/ ({a['notes']}; from Galaxy {a['source']['galaxyArea']})")
als = lines_by_area[a["unsAreaId"]]
a_cont = " " if a_last else ""
for li, l in enumerate(als):
l_last = li == len(als) - 1
lines.append(f"{a_cont}{'└─' if l_last else '├─'} {l['name']}/")
eqs = eq_by_line[l["unsLineId"]]
l_cont = a_cont + (" " if l_last else "")
for ei, e in enumerate(eqs):
e_last = ei == len(eqs) - 1
ntags = len(e["tags"])
lines.append(f"{l_cont}{'└─' if e_last else '├─'} {e['name']} "
f"[{e['manufacturer']} {e['model']}] ← {e['source']['galaxyTag']} ({ntags} signals)")
open("company-uns.tree.txt", "w").write("\n".join(lines) + "\n")
print(json.dumps(doc["stats"], indent=2))
print("\nsample equipment:")
e = equipment[0]
print(f" {e['unsPath']} machineCode={e['machineCode']} delmia={e['source']['delmiaReceiverTag']} mes={e['source']['mesReceiverTag']}")
print(" sample tags:")
for t in e["tags"][:3] + [e["tags"][12]] + [e["tags"][-1]]:
print(f" {t['folderPath'] or '.'}/{t['name']:18s} {t['dataType']:8s} {t['accessLevel']:9s} -> {t['source']['fullTagReference']}")
print("\nwrote: company-uns.json, company-uns.tree.txt")
+377
View File
@@ -0,0 +1,377 @@
# MES and Delmia-DNC integrations — API & MXAccess write specification
Documents two existing Wonderware integrations hosted on the `ZimmerBiomet` Gitea org
(`http://wonder-app-vd03.zmr.zimmer.com:3000`):
| Integration | Repo | What it does | Who does the MXAccess write |
|---|---|---|---|
| **MES** | [`ZimmerBiomet/MESAPI`](http://wonder-app-vd03.zmr.zimmer.com:3000/ZimmerBiomet/MESAPI) (solution `WWSupport`) | REST API the Camstar MES calls to move-in / move-out work orders against a machine, and to read machine alarm status | **The service itself** (`MesNotifier`, in-repo) |
| **Delmia DNC** | [`ZimmerBiomet/DelmiaIntegration`](http://wonder-app-vd03.zmr.zimmer.com:3000/ZimmerBiomet/DelmiaIntegration) | Pull an NC/recipe document from the DELMIA/Apriso (Intercim) DNC server and push the resulting recipe-download notification into Wonderware | **An external receiver service** at `wonder-app-vd01:9001/notify`; the actual flag handshake is implemented in the Galaxy `$DelmiaReceiver` object (`ProcessRecipe`/`Reset` scripts) — *not* in this repo |
Both integrations talk to AVEVA System Platform ("Galaxy") through **MXAccess COM**
(`ArchestrA.MxAccess.LMXProxyServerClass`) and use the same general pattern:
> **Handshake pattern** — read a *ready* flag → write the data tags → set a *trigger* flag
> → wait (bounded by a timeout) for a *complete* flag → read a *success* flag + *error text*
> → return the result and unsubscribe.
All facts below are taken verbatim from source at the repo `master` branch (cloned
2026-06-17) unless explicitly marked **(inferred)**.
---
## 1. MES integration — `MESAPI` / `WWSupport`
### 1.1 Topology & hosting
```
Camstar MES ──HTTPS/JSON──▶ WWSupport API (ServiceStack self-host)
│ ├─ SQL Server "BT" (machine lookup by SAPID, alarm catalog)
│ └─ MXAccess COM (LMXProxyServerClass, client "MesNotifier")
Galaxy object {MachineCode}.MesReceiver.* (move-in / move-out tags)
Galaxy object {MachineCode}.{AlarmName}.* (alarm attributes)
```
- **Framework:** ServiceStack, self-hosted via `AppSelfHostBase` (`AppHost : base("APIServer", typeof(MesServices).Assembly)`), .NET Framework, run as a Windows service.
- **Listen URL (per environment, `App.config``HttpListener`):**
- DEV `http://*:9501/` · QA `http://*:9500/` · PROD `http://*:9500/`
- **Database (`App.config` → connection `BatchDB`, DB `BT`):**
- DEV `wonder-sql-vd01.zmr.zimmer.com` · QA `wondersqlqa.zmr.zimmer.com` · PROD (same form). User `wonderapp`.
- **Auth:** every operation is decorated `[Authenticate]` + `[RequiredRole("MESAPI")]` (`MesServices.cs`).
`AppHost` registers an `AuthFeature` with two providers: `ApiKeyAuthProvider` and `LdapAuthProvider`.
- Unauthenticated → **401**; authenticated without the `MESAPI` role → **403**.
- **Serialization:** `JsConfig.IncludeNullValues = true` (null fields ARE emitted in JSON). `PostmanFeature` + `OpenApiFeature` (Swagger) are enabled.
- **MES counterpart object:** the live Galaxy attribute listing for the receiver object is in
[`mesrec.md`](mesrec.md) (`$MESReceiver` template). Note the API binds tags under the contained
name `MesReceiver` (i.e. `{MachineCode}.MesReceiver.<tag>`).
### 1.2 Endpoints (inputs / outputs)
Routes come from `[Route(...)]` on the request DTOs; all are **POST**, JSON in / JSON out, handled by `MesServices.Any(...)` which resolves a per-request `MesNotifier`.
#### `POST /mes/movein` → `MoveInResponse`
Request `MoveInRequest`:
| Field | Type | Notes |
|---|---|---|
| `SAPID` | string | machine key; looked up in `BT.Machine` to get `Machine.Code` |
| `OperatorName` | string | |
| `JobSequenceNumber` | string | |
| `WorkOrders` | `List<WorkOrderInfo>` | each = `{ WorkOrderNumber: string, PartNumber: string }` |
Response `MoveInResponse`: `WasSuccessful` (bool), `ErrorText` (string), `BatchID` (int?, only set if machine returns non-zero).
#### `POST /mes/moveout` → `MoveOutResponse`
Request `MoveOutRequest`: `SAPID` (string), `OperatorName` (string), `WorkOrders` (`List<WorkOrderInfo>`).
*(Move-out has no `JobSequenceNumber`.)*
Response `MoveOutResponse`: identical shape to `MoveInResponse` (`WasSuccessful`, `ErrorText`, `BatchID`).
#### `POST /mes/alarmstatus` → `AlarmStatusResponse`
Request `AlarmStatusRequest`:
- `MachineFilter` = `{ MachineID: int?, SAPID: string, ZTag: string, Code: string }` (any one identifies the machine)
- `AlarmFilter` = `{ NameFilter: string, MinSeverity: int?, MaxSeverity: int?, IncludeTriggered: bool=true, IncludeAcked: bool=true, FlaggedOnly: bool=false }`
#### `POST /mes/simplealarmstatus` → `AlarmStatusResponse`
Request `SimpleAlarmStatusRequest`: `SAPID` (string). Internally loads only alarms with `FlaggedForMES == true` for that machine.
Response `AlarmStatusResponse` (both alarm endpoints): `WasSuccessful` (bool), `ErrorText` (string), `Alarms` (`List<AlarmInfo>`).
`AlarmInfo`: `Name` (string), `HierarchicalName` (string, `{Code}.{AlarmName}`), `Description` (string), `IsFlaggedForMES` (bool), `Severity` (int), `StatusCode` (string — `"Triggered"` or `"Triggered.Acked"`), `TriggeredDT` (DateTime), `AckDT` (DateTime?), `AckComment` (string).
### 1.3 MXAccess connection model
`MesNotifier` owns one MXAccess proxy for the request:
```csharp
_lmxProxy = new ArchestrA.MxAccess.LMXProxyServerClass();
_lmxHandle = _lmxProxy.Register("MesNotifier");
_lmxProxy.OnDataChange += ...; // value updates resolve pending read/OnValue tasks
_lmxProxy.OnWriteComplete += ...; // write acks resolve pending write tasks
```
Tags are added with `AddItem` + `AdviseSupervisory` (subscribe), updated via `OnDataChange`,
and removed with `UnAdvise` + `RemoveItem` on cleanup. Reads/writes are wrapped as `Task<bool>`
that complete on the corresponding callback or **fail (`false`) on cancellation/timeout**.
A read is considered valid only if MXAccess **quality == 192** ("good").
**Target selection:** request `SAPID``db.Single<Machine>(x => x.SAPID == SAPID)``Machine.Code`
becomes the tag prefix. Move tags live under `{Code}.MesReceiver.*`; alarm tags under
`{Code}.{MachineAlarm.Name}.*` (alarm catalog from `db.Select<MachineAlarm>(...)`).
### 1.4 Tag mappings
**Move-in** (`MesMoveInTagset`, all `{Code}.MesReceiver.<tag>`):
| Tag | Type | Dir | Role | Source field |
|---|---|---|---|---|
| `MoveInReadyFlag` | bool | read | gate — must be `true` before writing | — |
| `MoveInFlag` | bool | **write** | **trigger** — set `true` last | — |
| `MoveInCompleteFlag` | bool | read | completion — handshake waits on this | — |
| `MoveInSuccessfulFlag` | bool | read | result | → `response.WasSuccessful` |
| `MoveInErrorText` | string | read | result | → `response.ErrorText` |
| `MoveInBatchID` | int | read | result | → `response.BatchID` (if ≠ 0) |
| `MoveInOperatorName` | string | write | data | `request.OperatorName` |
| `MoveInJobSequenceNumber` | string | write | data | `request.JobSequenceNumber` |
| `MoveInNumberWorkOrders` | int | write | data | `request.WorkOrders.Count` |
| `MoveInWorkOrderNumbers[]` | string[] | write | data (fixed length 50) | `WorkOrders.Select(w => w.WorkOrderNumber)` |
| `MoveInPartNumbers[]` | string[] | write | data (fixed length 50) | `WorkOrders.Select(w => w.PartNumber)` |
**Move-out** (`MesMoveOutTagset`): identical set with `MoveOut` prefix, **minus** `JobSequenceNumber`
(`MoveOutReadyFlag`, `MoveOutFlag`, `MoveOutCompleteFlag`, `MoveOutSuccessfulFlag`, `MoveOutErrorText`,
`MoveOutBatchID`, `MoveOutOperatorName`, `MoveOutNumberWorkOrders`, `MoveOutWorkOrderNumbers[]`, `MoveOutPartNumbers[]`).
**Alarms** (`AlarmTagset`, all `{Code}.{AlarmName}.<attr>`): `Quality` (int), `InAlarm` (bool),
`TimeAlarmOn` (DateTime), `DescAttrName` (string), `Acked` (bool), `TimeAlarmAcked` (DateTime?), `AckMsg` (string).
### 1.5 The handshake — `MesNotifier.MoveIn` (move-out is identical with `MoveOut*` tags)
Whole operation is bounded by **`new CancellationTokenSource(30000)` = 30 s**.
1. **Look up machine** by `SAPID`. Not found → `WasSuccessful=false`, `ErrorText="Failed to find machine with SAPID '{SAPID}'"`, return.
2. **Subscribe** to every move-in tag (`Advise(t, cts)`), `await Task.WhenAll(...)`. Any subscription that fails / quality ≠ 192 → `ErrorText="Failed to connect to machine"`.
3. **Check ready flag:** `if (!MoveInReadyFlag.Value)``ErrorText="Machine move in ready flag not set to true"`, stop.
4. **Arm completion watch:** `Task<bool> flagTask = MoveInCompleteFlag.OnValue(true, cts);` (completes when the flag goes `true`, or `false` on the 30 s timeout).
5. **Write data + trigger (in parallel, trigger last):** `MoveInOperatorName`, `MoveInJobSequenceNumber`, `MoveInNumberWorkOrders`, `MoveInPartNumbers[]` (padded to 50), `MoveInWorkOrderNumbers[]` (padded to 50), then `MoveInFlag = true`. `await Task.WhenAll(writeTasks)`; any write `!= true``ErrorText="Failed to write move in information to machine"`.
6. **Wait for completion:** `await Task.WhenAll(flagTask)`.
- `flagTask.Result == true` → read results: `WasSuccessful = MoveInSuccessfulFlag.Value`, `ErrorText = MoveInErrorText.Value`, `BatchID = MoveInBatchID.Value` (if ≠ 0).
- `flagTask.Result == false` (timed out) → `WasSuccessful=false`, `ErrorText="Timeout waiting for move in information to be processed"`.
7. **Cleanup:** `Tags.ForEach(Unadvise)` and return.
```csharp
using (var cts = new CancellationTokenSource(30000)) { // 30 s budget
...
if (!moveInTagset.MoveInReadyFlag.Value) { /* not-ready error */ }
var flagTask = moveInTagset.MoveInCompleteFlag.OnValue(true, cts); // arm completion watch
var writeTasks = new List<Task<bool>> {
Write(moveInTagset.MoveInOperatorName.Handle, request.OperatorName, cts),
Write(moveInTagset.MoveInJobSequenceNumber.Handle, request.JobSequenceNumber, cts),
Write(moveInTagset.MoveInNumberWorkOrders.Handle, request.WorkOrders.Count, cts),
Write(moveInTagset.MoveInPartNumbers.Handle, request.WorkOrders.Select(wo => wo.PartNumber).ToFixedLength(50), cts),
Write(moveInTagset.MoveInWorkOrderNumbers.Handle, request.WorkOrders.Select(wo => wo.WorkOrderNumber).ToFixedLength(50), cts),
Write(moveInTagset.MoveInFlag.Handle, true, cts) // TRIGGER — set last
};
await Task.WhenAll(writeTasks);
await Task.WhenAll(flagTask);
if (flagTask.Result) {
response.WasSuccessful = moveInTagset.MoveInSuccessfulFlag.Value;
response.ErrorText = moveInTagset.MoveInErrorText.Value;
if (moveInTagset.MoveInBatchID.Value != 0) response.BatchID = moveInTagset.MoveInBatchID.Value;
} else {
response.WasSuccessful = false;
response.ErrorText = "Timeout waiting for move in information to be processed";
}
moveInTagset.Tags.ForEach(Unadvise);
}
```
> There is **no busy-poll loop**: completion is event-driven via the MXAccess `OnDataChange`
> callback; the 30 s `CancellationTokenSource` is the only timeout.
### 1.6 Alarm-status path
1. Resolve machine from `MachineFilter` (`SAPID` / `Code` / `ZTag` / `MachineID`) — wrong/missing → error.
2. Load `MachineAlarm` rows for the machine; apply filters (`FlaggedOnly`, `MinSeverity`, `MaxSeverity`, case-insensitive `NameFilter.Contains`). *(`IncludeTriggered` is read but not used in the filter.)*
3. Subscribe + read each alarm's `Quality` and `InAlarm` (30 s budget). Bad quality / read failure → `ErrorText="Failed to read machine alarm status"`.
4. For alarms where `InAlarm == true`, additionally read `TimeAlarmOn`, `DescAttrName`, `Acked`, `TimeAlarmAcked`, `AckMsg`.
5. Build `AlarmInfo` per triggered alarm; `StatusCode = "Triggered.Acked"` if acked else `"Triggered"`. If `AlarmFilter.IncludeAcked == false`, acked alarms are skipped.
6. Unsubscribe; on failure `Alarms` is cleared.
### 1.7 Outputs / error handling (MES)
- **Transport status is always 200** for handled responses — success/failure is carried by the body's `WasSuccessful` flag + `ErrorText`. (401/403 only from the auth layer.)
- Success: `{ "WasSuccessful": true, "ErrorText": null, "BatchID": <int|null> }`.
- Failure/timeout: `{ "WasSuccessful": false, "ErrorText": "<message>", "BatchID": null }`.
- Distinct `ErrorText` values: machine-not-found, "Failed to connect to machine", "…ready flag not set to true", "Failed to write … to machine", "Timeout waiting for … to be processed", "Failed to read machine alarm status".
---
## 2. Delmia-DNC integration — `DelmiaIntegration` (+ Galaxy `$DelmiaReceiver`)
### 2.1 Topology — three hops
```
Operator (DelmiaIntegration.exe WinForms)
│ ① DelmiaClient ──HTTP POST (form-url-encoded)──▶ DELMIA/Apriso DNC "Downloader.asmx"
│ (e.g. http://dnc-app-vd01.zmr.zimmer.com/IntercimService/Downloader.asmx)
│ ◀── XML (SearchResults / DownloadResult, ns http://intercim.com/ruleset) ──
│ ② recipe file written to disk; WWNotifier.exe launched with CLI args
WWNotifier.exe ──HTTP POST (JSON RecipeDownload)──▶ WW receiver service (http://wonder-app-vd01:9001/notify)
◀── JSON RecipeDownloadResult ── │ ③ MXAccess COM write
Galaxy object {machine}.$DelmiaReceiver.* (recipe tags + flags)
+ ArchestrA scripts ProcessRecipe / Reset
```
**Assemblies in the repo:** `DelmiaContracts` (XML DTO library), `DelmiaIntegration` (`DelmiaClient` + WinForms UI),
`WWNotifier.exe` (console notifier), plus test harnesses (`AdminTestUtil`, `DownloadTestUtil`, `TestUI`).
> **Scope note.** Hops ① and ② are fully in this repo. Hop ③ — the service at `:9001/notify`
> that actually performs the MXAccess write — is **not** in the `ZimmerBiomet` Gitea org; only the
> JSON contract (below) and the Galaxy-side `$DelmiaReceiver` object (scripts + attributes, exported
> under `AA_EXPORT/`) are available. `WWNotificationSystem` *also* uses MXAccess but is an unrelated
> tag→email alerting service (port `:9876`, client name `WWNotifierMonitor`) — **not** the recipe receiver.
### 2.2 DNC server interface (`DelmiaClient`, hop ①)
- **Transport:** `HttpClient.PostAsync` with `FormUrlEncodedContent`; response is XML deserialized
with `XmlSerializer`. Base URL is `DelmiaClient.URL`; per-call `Timeout` default **30 s**.
Action is appended to the base URL (`URL.TrimEnd('/') + "/<Action>"`).
- **Base URL (from `AdminTestUtil` `DefaultURL`):** `http://dnc-app-vd01.zmr.zimmer.com/IntercimService/Downloader.asmx`.
- On any exception the client returns a result object with the error in `ErrorMessage` /
`TransferSuccessful=false` (it does not throw to the caller).
| Method (sync + `…Async`) | POST action | Form fields | Returns |
|---|---|---|---|
| `Search` | `/Search` | `username, machineID, partNumber, operationNumber` | `SearchResults` |
| `RequestProvenDocument` | `/RequestProvenDocument` | `username, machineID, partNumber, operationNumber, workOrderNumber` | `DownloadResult` |
| `RequestDocument` | `/RequestDocument` | `username, machineID, partNumber, operationNumber, workOrderNumber, documentKey` | `DownloadResult` |
DTO field lists (`DelmiaContracts`, XML namespace `http://intercim.com/ruleset`):
- **`SearchResults`**: `Results` (`List<SearchResult>`), `ErrorMessage` (string).
- **`SearchResult`**: `ShopOrderKey` (int), `ShopOrderID` (string), `ShopOrderStatus` (string),
`ShopOrderOperKey` (int), `ShopOrderOperID` (string), `ShopOrderOperStatus` (string),
`DocumentKey` (int), `DocumentObjectID` (int, with `…Specified` flag), `DocumentName` (string),
`DocumentRev` (string), `DocumentStatus` (string), `DocumentURL` (string), `PartID` (string), `PartRev` (string).
- **`DownloadResult`**: `UserKey` (int), `UserName` (string), `UserSite` (string), `MachineKey` (int),
`MachineID` (string), `MachineSite` (string), `WorkOrderNumber` (string), `ShopOrderKey` (int),
`ShopOrderID` (string), `ShopOrderStatus` (string), `ShopOrderOperKey` (int), `ShopOrderOperID` (string),
`ShopOrderOperStatus` (string), `DocumentKey` (int), `DocumentName` (string), `DocumentRev` (string),
`DocumentStatus` (string), `PartID` (string), `PartRev` (string), **`TransferSuccessful` (bool)**,
**`ErrorMessage` (string)**.
- **`MachineInfo`** (contract present; not called by current code): `MachineKey` (int), `MachineID` (string),
`MachineName` (string), `DownloadPath` (string), `MachineDescription` (string), `MachineSite` (string), `MachineStatus` (string).
- **`UserInfo`**: `UserKey` (int), `UserName` (string), `UserSite` (string), `IsActive` (bool).
**Recipe file:** the downloaded document is a key/value recipe file parsed by `DelmiaIntegration/Models/RecipeSet.cs`
(`KEY,VALUE` lines; typed accessors `GetString/GetInt/GetBool/GetFloat/GetDouble/...`). It is written to disk;
its path is what gets handed to `WWNotifier` (`--downloadpath`).
### 2.3 WWNotifier (hop ②) — invocation & handoff contract
`WWNotifier.exe` (uses `CommandLineParser`); CLI options (`CommandLineOptions`):
| Short | Long | Required | Field |
|---|---|---|---|
| `-d` | `--downloadpath` | yes | `DownloadPath` (recipe file path) |
| `-m` | `--machine` | yes | `MachineCode` |
| `-w` | `--workorder` | yes | `WorkOrderNumber` |
| `-p` | `--partnumber` | yes | `PartNumber` |
| `-s` | `--seqop` | no | `JobStepNumber` |
| `-u` | `--username` | no | `Username` |
Config (`WWNotifier/App.config`): `NotifyURL = http://wonder-app-vd01.zmr.zimmer.com:9001/notify`
(comma-separated list allowed — tried in order until one succeeds), `NotifyTimeout = 30` (seconds,
applied as the global Flurl HTTP timeout).
Handoff (Flurl): `url.PostJsonAsync(recipeDownload).ReceiveJson<RecipeDownloadResult>()`.
- **Request body `RecipeDownload`** (JSON): `MachineCode`, `DownloadPath`, `WorkOrderNumber`, `PartNumber`, `JobStepNumber`, `Username` (all string).
- **Response body `RecipeDownloadResult`** (JSON): `Result` (bool), `ResultText` (string).
- **Outputs:** prints `YES` and exit code `0` on success; prints `NO` + a message and sets exit code `-1`
on failure (parse error, missing `NotifyURL`/`NotifyTimeout`, `Result==false`, or HTTP exception).
*(Caveat: on a caught exception it logs `error.InnerException.Message`, which throws a NRE when there is no inner exception — so bare transport errors surface only as a generic failure.)*
### 2.4 MXAccess write — Galaxy `$DelmiaReceiver` object (hop ③)
The receiver service maps `RecipeDownload` fields onto the `$DelmiaReceiver` object instance selected by
`MachineCode`. Object attributes (from `AA_EXPORT/.../$DelmiaReceiver` export) and their roles:
| Attribute | Type | Role | Maps from |
|---|---|---|---|
| `ReadyFlag` | Boolean | gate — receiver expects `true` before writing | — |
| `DownloadPath` | String | data | `RecipeDownload.DownloadPath` |
| `WorkOrderNumber` | String | data | `RecipeDownload.WorkOrderNumber` |
| `PartNumber` | String | data | `RecipeDownload.PartNumber` |
| `JobStepNumber` | String | data | `RecipeDownload.JobStepNumber` |
| `Username` | String | data | `RecipeDownload.Username` |
| `RecipeDownloadFlag` | Boolean | **trigger** — set `true` to start processing | — |
| `RecipeProcessedFlag` | Boolean | completion — handshake waits on this | — |
| `RecipeProcessResult` | Boolean | result | → `RecipeDownloadResult.Result` |
| `RecipeProcessResultText` | String | result | → `RecipeDownloadResult.ResultText` |
*(`MachineCode` selects which receiver instance; it is not itself a written attribute.)*
**Galaxy-side handshake (authoritative — ArchestrA scripts on `$DelmiaReceiver`):**
`ProcessRecipe` (runs when `RecipeDownloadFlag` is set):
```
Me.RecipeDownloadFlag = false; ' clear trigger
Me.ReadyFlag = false; ' clear ready
try
Me.RecipeProcessResult = true;
Me.RecipeProcessResultText = "Success";
catch
Me.RecipeProcessResult = false;
Me.RecipeProcessResultText = "Failed to read recipe file";
endtry;
Me.RecipeProcessedFlag = true; ' signal completion
```
`Reset` (clears the slot for the next download):
```
Me.RecipeDownloadFlag = false; Me.RecipeProcessedFlag = false;
Me.RecipeProcessResult = false; Me.RecipeProcessResultText = "";
Me.DownloadPath = ""; Me.WorkOrderNumber = ""; Me.PartNumber = "";
Me.JobStepNumber = ""; Me.Username = "";
```
**Receiver-side sequence (inferred** — mirrors the MES handshake and is driven by the flags above; the C# source is not in the repo**):**
1. resolve the `$DelmiaReceiver` instance from `MachineCode`;
2. (optionally) verify `ReadyFlag == true`;
3. write `DownloadPath`, `WorkOrderNumber`, `PartNumber`, `JobStepNumber`, `Username`;
4. set `RecipeDownloadFlag = true` (trigger) → Galaxy `ProcessRecipe` fires;
5. wait for `RecipeProcessedFlag == true`, bounded by the request timeout;
6. read `RecipeProcessResult``Result`, `RecipeProcessResultText``ResultText`; return the `RecipeDownloadResult` JSON;
7. `Reset` the object.
### 2.5 Outputs / error handling (Delmia)
- **DNC server call:** failures are swallowed into the returned DTO — `SearchResults.ErrorMessage`, or
`DownloadResult.TransferSuccessful=false` + `ErrorMessage="Failed to call Delmia web service at '<URL>'."`.
- **Notify handoff:** `RecipeDownloadResult.Result` (bool) + `ResultText` (string); `WWNotifier` exit code
`0` (`YES`) / `-1` (`NO`).
- **Galaxy script:** `RecipeProcessResultText` is `"Success"` or `"Failed to read recipe file"`.
---
## 3. Side-by-side summary
| | MES (`MESAPI`) | Delmia DNC (`DelmiaIntegration`) |
|---|---|---|
| Caller | Camstar MES (HTTP/JSON) | Operator UI → DELMIA DNC server, then WWNotifier |
| API style | ServiceStack REST, `POST /mes/*` | DNC = form-url-encoded → XML; notify = JSON POST |
| Who writes MXAccess | the service (`MesNotifier`, in-repo) | external `:9001/notify` receiver (source not in repo) |
| MXAccess client | `LMXProxyServerClass`, register `"MesNotifier"` | `LMXProxyServerClass` (receiver), Galaxy `$DelmiaReceiver` scripts |
| Target object | `{MachineCode}.MesReceiver.*` | `{MachineCode}.$DelmiaReceiver` instance |
| Ready / trigger / complete | `MoveInReadyFlag` / `MoveInFlag` / `MoveInCompleteFlag` | `ReadyFlag` / `RecipeDownloadFlag` / `RecipeProcessedFlag` |
| Result / error | `MoveInSuccessfulFlag` / `MoveInErrorText` (+`MoveInBatchID`) | `RecipeProcessResult` / `RecipeProcessResultText` |
| Timeout | 30 s (`CancellationTokenSource(30000)`), event-driven | 30 s HTTP (`NotifyTimeout`); Galaxy wait at receiver |
---
## 4. Sources & open gaps
**Repos (Gitea `ZimmerBiomet`, `master`):**
- MES: `MESAPI``APIServer.ServiceInterface/MesServices.cs`, `MesNotifier.cs`,
`Mes{MoveIn,MoveOut}Tagset.cs`, `AlarmTagset.cs`, `Tag.cs`/`OnValueTask.cs`;
`APIServer.ServiceModel/Types/*`; `APIServer/AppHost.cs`, `App.config`.
- Delmia: `DelmiaIntegration``DelmiaIntegration/DelmiaClient.cs`, `Models/RecipeSet.cs`;
`WWNotifier/Program.cs`, `CommandLineOptions.cs`, `Models/RecipeDownload(Result).cs`, `App.config`;
`DelmiaContracts/*`.
**Galaxy export (`~/Desktop/AA_EXPORT/EXTRACTED/$DelmiaReceiver`):** `scripts/ProcessRecipe.txt`,
`scripts/Reset.txt`, `$DelmiaReceiver.top_level_attributes.csv`. MES receiver object attributes: [`mesrec.md`](mesrec.md).
**Open gaps / to verify against the box:**
1. The Delmia recipe **`/notify` receiver service** source (the actual MXAccess writer at `wonder-app-vd01:9001`)
was not found in the Gitea org — §2.4 receiver steps are inferred from the contract + Galaxy scripts.
2. MES tag prefix in code is `{Code}.MesReceiver.*`, while the live probe in `mesrec.md` shows a top-level
`MESReceiver_002` instance — confirm the exact contained-name/instance convention on the live Galaxy.
3. PROD `HttpListener`/DB host values should be read from the deployed `App.config`, not assumed.
+77
View File
@@ -0,0 +1,77 @@
# MESReceiver object — attributes (wonder-app-vd03 Galaxy)
Source: live AVEVA Galaxy DB **`ZB`** on **wonder-app-vd03** (the MxAccessGateway box),
read via the gateway's own `AttributesSql` (recursive `deployed_package_chain` over
`dynamic_attribute` / `attribute_definition`) run with `sqlcmd` over ssh — `2026-06-16`.
## Instance probed
- **TagName:** `MESReceiver_002` (GobjectId `5909`, ParentGobjectId `5908`, HostedBy `5049`)
- **TemplateChain:** `$MESDemo.MESReceiver``$MESReceiver``$gUserDefined``$UserDefined`
- **CategoryId:** 10, `IsArea = false`
- There are many MESReceiver instances on this Galaxy (1,253 `MESReceiver` references in the
`galaxy-snapshot.json` hierarchy cache). `MESReceiver_002` is representative of the template.
> Note: the cached `galaxy-snapshot.json` (gateway Server dir) holds the **object hierarchy only**
> — it carries **no attributes**, so attribute discovery requires the DB query (or gRPC `galaxy-discover`).
## Container number
There is **no attribute literally named `ContainerNumber`**. The MES container number is carried by:
- **`MoveInMesContainerNum`** — String (move-in interface)
- **`MoveOutMesContainerNum`** — String (move-out interface)
Full tag references: `MESReceiver_002.MoveInMesContainerNum`, `MESReceiver_002.MoveOutMesContainerNum`.
## MES interface UDAs (attribute-category 10, security-classification 1 = secured/operate)
### Move-In
| Attribute | Type | Full tag reference |
|---|---|---|
| **MoveInMesContainerNum** | String | `MESReceiver_002.MoveInMesContainerNum` |
| MoveInBatchID | Integer | `MESReceiver_002.MoveInBatchID` |
| MoveInJobSequenceNumber | String | `MESReceiver_002.MoveInJobSequenceNumber` |
| MoveInNumberWorkOrders | Integer | `MESReceiver_002.MoveInNumberWorkOrders` |
| MoveInWorkOrderNumbers | String[] | `MESReceiver_002.MoveInWorkOrderNumbers[]` |
| MoveInPartNumbers | String[] | `MESReceiver_002.MoveInPartNumbers[]` |
| MoveInOperatorName | String | `MESReceiver_002.MoveInOperatorName` |
| MoveInFlag | Boolean | `MESReceiver_002.MoveInFlag` |
| MoveInReadyFlag | Boolean | `MESReceiver_002.MoveInReadyFlag` |
| MoveInCompleteFlag | Boolean | `MESReceiver_002.MoveInCompleteFlag` |
| MoveInSuccessfulFlag | Boolean | `MESReceiver_002.MoveInSuccessfulFlag` |
| MoveInErrorText | String | `MESReceiver_002.MoveInErrorText` |
### Move-Out (symmetric; no JobSequenceNumber)
| Attribute | Type | Full tag reference |
|---|---|---|
| **MoveOutMesContainerNum** | String | `MESReceiver_002.MoveOutMesContainerNum` |
| MoveOutBatchID | Integer | `MESReceiver_002.MoveOutBatchID` |
| MoveOutNumberWorkOrders | Integer | `MESReceiver_002.MoveOutNumberWorkOrders` |
| MoveOutWorkOrderNumbers | String[] | `MESReceiver_002.MoveOutWorkOrderNumbers[]` |
| MoveOutPartNumbers | String[] | `MESReceiver_002.MoveOutPartNumbers[]` |
| MoveOutOperatorName | String | `MESReceiver_002.MoveOutOperatorName` |
| MoveOutFlag | Boolean | `MESReceiver_002.MoveOutFlag` |
| MoveOutReadyFlag | Boolean | `MESReceiver_002.MoveOutReadyFlag` |
| MoveOutCompleteFlag | Boolean | `MESReceiver_002.MoveOutCompleteFlag` |
| MoveOutSuccessfulFlag | Boolean | `MESReceiver_002.MoveOutSuccessfulFlag` |
| MoveOutErrorText | String | `MESReceiver_002.MoveOutErrorText` |
## Standard ArchestrA `$UserDefined` / system attributes (also present)
`AlarmCntsBySeverity[]`, `AlarmCntsBySeverityEnableShelved[]`, `AlarmInhibit`, `AlarmMode`,
`AlarmModeCmd`, `AlarmMostUrgentAcked`, `AlarmMostUrgentInAlarm`, `AlarmMostUrgentMode`,
`AlarmMostUrgentSeverity`, `AlarmMostUrgentShelved`, `AliasName`, `Area`, `CmdData`, `CodeBase`,
`ConfigVersion`, `ContainedName`, `Container`, `Errors[]`, `ExecutionRelatedObject`,
`ExecutionRelativeOrder`, `Extensions`, `HierarchicalName`, `Host`, `InAlarm`, `MinorVersion`,
`PropagatedAlarmInhibit`, `ScanState`, `ScanStateCmd`, `SecurityGroup`, `ShortDesc`, `Tagname`,
`UDAs`, `UserAttrData`, `PropagatedAlarmInhibit`.
## How to reproduce
1. `MESReceiver` instances are in the gateway hierarchy cache
`E:\ApiInstall\MxGateway\Server\galaxy-snapshot.json` (objects only, no attrs).
2. Attribute list: run the gateway's `AttributesSql`
(`src/.../Server/Galaxy/GalaxyRepository.cs`) scoped to one instance via
`... WHERE g.is_template = 0 AND g.deployed_package_id <> 0 AND g.tag_name = 'MESReceiver_002'`
in the `deployed_package_chain` anchor, then `sqlcmd -S (local) -d ZB -U wwadmin -P <pwd>`
on the box (Galaxy creds from `MxGateway.Galaxy.ConnectionString` in the gateway appsettings).
3. Live values would need an MXAccess read through the gateway (gRPC), not the repo SQL.
In OtOpcUa (Galaxy-as-standard-driver model) these bind as ordinary equipment tags:
`Tag{ DriverInstanceId = GalaxyMxGateway, TagConfig = {"FullName":"MESReceiver_002.MoveInMesContainerNum"} }`.
+212
View File
@@ -0,0 +1,212 @@
# New Jersey — DARS reactors Z28061 / Z28062 (wonder Galaxy)
Source: live AVEVA Galaxy DB **`ZB`** on **wonder-app-vd03** (MxAccessGateway box), via the gateway's `AttributesSql` over `sqlcmd`/ssh — `2026-06-16`. Lists the **meaningful (user-defined, cat-10) attributes**; ArchestrA system attributes (alarm framework, identity, security) and per-field config sub-attributes (`.EngUnits`, `.TrendHi`, `.Dev.*`, history/alarm settings) and scripts are omitted.
## Hierarchy
```
NewJersey (area)
└─ CVDAisle_1 ($CVDAisle, gobject 7243)
├─ Z28061 ($DARSReactor, gobject 7171)
│ ├─ Left (Left_002, $DARSReactor.Left, gobject 7172)
│ └─ Right (Right_002, $DARSReactor.Right, gobject 7173)
└─ Z28062 ($DARSReactor, gobject 7202)
├─ Left (Left_003, $DARSReactor.Left, gobject 7203)
└─ Right (Right_003, $DARSReactor.Right, gobject 7204)
```
(`Z28061Sim`, gobject 7146, is a simulator sibling of Z28061.)
## Objects
| Object | GobjectId | Template | Compound path | UDA attrs | Total attrs |
|---|---|---|---|---|---|
| Left_002 | 7172 | `$DARSReactor.Left` | Z28061.Left | 72 | 1907 |
| Left_003 | 7203 | `$DARSReactor.Left` | Z28062.Left | 72 | 1907 |
| Right_002 | 7173 | `$DARSReactor.Right` | Z28061.Right | 72 | 1907 |
| Right_003 | 7204 | `$DARSReactor.Right` | Z28062.Right | 72 | 1907 |
| Z28061 | 7171 | `$DARSReactor` | Z28061 | 16 | 195 |
| Z28062 | 7202 | `$DARSReactor` | Z28062 | 16 | 195 |
> Attribute sets are template-defined: `Z28061``Z28062`, `Left_002``Left_003`, `Right_002``Right_003`. Listed once per template.
## $DARSReactor — Z28061 / Z28062 (reactor body)
**Instances:** `Z28061` (Z28061, gobject 7171), `Z28062` (Z28062, gobject 7202)
**Template:** `$DARSReactor` · **16 meaningful (UDA) attributes** · 195 total incl. system + config sub-attributes.
| Attribute | Type | Hist | Alarm |
|---|---|---|---|
| ChlorineFlow | Integer | ✓ | |
| ConfirmTimeoutSec | Integer | | |
| Deadband | Double | | |
| HeartbeatInterval | Integer | | |
| HeartbeatRequest | Boolean | | |
| HeartbeatTimeout | Integer | | |
| HeartbeatTimeoutAlarm | Boolean | | ✓ |
| HydrogenFlow | Integer | ✓ | |
| LeakTestMaxDelta | Integer | | |
| LeakTestMinDuration | Integer | | |
| LeakTestTimeout | Integer | | |
| MachineCode | String | | |
| MachineDescription | String | | |
| MachineID | String | | |
| Runtime_Alert | Boolean | | |
| TrunkPressure | Float | ✓ | |
## $DARSReactor.Left — Left_002 / Left_003 (left chamber)
**Instances:** `Left_002` (Z28061.Left, gobject 7172), `Left_003` (Z28062.Left, gobject 7203)
**Template:** `$DARSReactor.Left` · **72 meaningful (UDA) attributes** · 1907 total incl. system + config sub-attributes.
| Attribute | Type | Hist | Alarm |
|---|---|---|---|
| AIR_SVC | String | | |
| AirAvg | Integer | ✓ | |
| AirFlow | Integer | ✓ | |
| AirFlowDevAlertPcnt | Float | ✓ | |
| AR_SVC | String | | |
| ArAvg | Integer | ✓ | |
| ArgonFlow | Integer | ✓ | |
| ChlorineFlowR | Integer | | |
| CL2_SVC | String | | |
| Cl2Avg | Integer | ✓ | |
| ClFlowDevAlertPcnt | Float | ✓ | |
| CoilDiameter | Boolean | ✓ | |
| CoilHeight | Integer | ✓ | |
| ContainerID | String | ✓ | |
| ContainerLoaded | Boolean | ✓ | |
| CurrentStep | Integer | | |
| CycleEndConfirm | Boolean | | |
| CycleEndNotify | Boolean | | |
| CycleEndTimeoutAlarm | Boolean | | |
| CycleRunning | Boolean | ✓ | |
| CycleStartConfirm | Boolean | | |
| CycleStartNotify | Boolean | | |
| CycleStartTimeoutAlarm | Boolean | | |
| Deadband | Double | | |
| DelmiaJobStep | Integer | | |
| FRR_Reset | Boolean | ✓ | |
| FRR_Runtime | Integer | ✓ | |
| FRR_Warning | Boolean | ✓ | ✓ |
| FurnaceTemp | Integer | ✓ | |
| FurnaceTempOverAlarmDev | Integer | ✓ | |
| FurnaceTempOverAlertDev | Integer | ✓ | |
| FurnaceTempUnderAlarmDev | Integer | ✓ | |
| FurnaceTempUnderAlertDev | Integer | ✓ | |
| H2_SVC | String | | |
| H2Avg | Integer | ✓ | |
| HydrogenFlowR | Integer | | |
| HyFlowDevAlertPcnt | Float | ✓ | |
| MachineBatchID | Integer | | |
| MachineBatchWOID | Integer | | |
| MachineCycleID | Integer | | |
| MantleTemp | Integer | ✓ | |
| MoveInReady | Boolean | | |
| MoveOutReady | Boolean | | |
| MTA_Lockout | Boolean | ✓ | ✓ |
| MTA_Reset | Boolean | ✓ | |
| MTA_Runtime | Integer | ✓ | |
| MTA_Warning | Boolean | ✓ | ✓ |
| NumOfParts | Integer | ✓ | |
| NumOfTurns | Integer | ✓ | |
| OperatorID | String | ✓ | |
| PartNumber | String | ✓ | |
| PID_Alarm | Boolean | | ✓ |
| PID_AlarmString | String | | |
| PID_CV | Integer | ✓ | |
| PotNumber | String | ✓ | |
| PotTempAvg | Integer | ✓ | |
| PreviousStep | Integer | | |
| ReactorTempAvg | Integer | ✓ | |
| RunDuration | Integer | | |
| RunEndTime | Time | | |
| RunStartTime | Time | | |
| SampleCount | Integer | | |
| StartingWeight | Integer | ✓ | |
| TableSide | Boolean | | |
| TableStatus | Integer | | |
| TC_Furnace_ID | String | | |
| TC_Mantle_ID | String | | |
| TC_Spare_ID | String | | |
| Vacuum | Float | ✓ | |
| VacuumAvg | Float | ✓ | |
| VacuumMode | Boolean | ✓ | |
| WorkOrder | String | | |
## $DARSReactor.Right — Right_002 / Right_003 (right chamber)
**Instances:** `Right_002` (Z28061.Right, gobject 7173), `Right_003` (Z28062.Right, gobject 7204)
**Template:** `$DARSReactor.Right` · **72 meaningful (UDA) attributes** · 1907 total incl. system + config sub-attributes.
| Attribute | Type | Hist | Alarm |
|---|---|---|---|
| AIR_SVC | String | | |
| AirAvg | Integer | ✓ | |
| AirFlow | Integer | ✓ | |
| AirFlowDevAlertPcnt | Float | ✓ | |
| AR_SVC | String | | |
| ArAvg | Integer | ✓ | |
| ArgonFlow | Integer | ✓ | |
| ChlorineFlowR | Integer | | |
| CL2_SVC | String | | |
| Cl2Avg | Integer | ✓ | |
| ClFlowDevAlertPcnt | Float | ✓ | |
| CoilDiameter | Boolean | ✓ | |
| CoilHeight | Integer | ✓ | |
| ContainerID | String | ✓ | |
| ContainerLoaded | Boolean | ✓ | |
| CurrentStep | Integer | | |
| CycleEndConfirm | Boolean | | |
| CycleEndNotify | Boolean | | |
| CycleEndTimeoutAlarm | Boolean | | |
| CycleRunning | Boolean | ✓ | |
| CycleStartConfirm | Boolean | | |
| CycleStartNotify | Boolean | | |
| CycleStartTimeoutAlarm | Boolean | | |
| Deadband | Double | | |
| DelmiaJobStep | Integer | | |
| FRR_Reset | Boolean | ✓ | |
| FRR_Runtime | Integer | ✓ | |
| FRR_Warning | Boolean | ✓ | ✓ |
| FurnaceTemp | Integer | ✓ | |
| FurnaceTempOverAlarmDev | Integer | ✓ | |
| FurnaceTempOverAlertDev | Integer | ✓ | |
| FurnaceTempUnderAlarmDev | Integer | ✓ | |
| FurnaceTempUnderAlertDev | Integer | ✓ | |
| H2_SVC | String | | |
| H2Avg | Integer | ✓ | |
| HydrogenFlowR | Integer | | |
| HyFlowDevAlertPcnt | Float | ✓ | |
| MachineBatchID | Integer | | |
| MachineBatchWOID | Integer | | |
| MachineCycleID | Integer | | |
| MantleTemp | Integer | ✓ | |
| MoveInReady | Boolean | | |
| MoveOutReady | Boolean | | |
| MTA_Lockout | Boolean | ✓ | ✓ |
| MTA_Reset | Boolean | ✓ | |
| MTA_Runtime | Integer | ✓ | |
| MTA_Warning | Boolean | ✓ | ✓ |
| NumOfParts | Integer | ✓ | |
| NumOfTurns | Integer | ✓ | |
| OperatorID | String | ✓ | |
| PartNumber | String | ✓ | |
| PID_Alarm | Boolean | | ✓ |
| PID_AlarmString | String | | |
| PID_CV | Integer | ✓ | |
| PotNumber | String | ✓ | |
| PotTempAvg | Integer | ✓ | |
| PreviousStep | Integer | | |
| ReactorTempAvg | Integer | ✓ | |
| RunDuration | Integer | | |
| RunEndTime | Time | | |
| RunStartTime | Time | | |
| SampleCount | Integer | | |
| StartingWeight | Integer | ✓ | |
| TableSide | Boolean | | |
| TableStatus | Integer | | |
| TC_Furnace_ID | String | | |
| TC_Mantle_ID | String | | |
| TC_Spare_ID | String | | |
| Vacuum | Float | ✓ | |
| VacuumAvg | Float | ✓ | |
| VacuumMode | Boolean | ✓ | |
| WorkOrder | String | | |
+7
View File
@@ -0,0 +1,7 @@
# Python virtualenv + caches (recreate via: python3 -m venv .venv && ./.venv/bin/pip install -r requirements.txt)
.venv/
__pycache__/
*.pyc
# Generated by `otopcua_uns.py generate` from ../galaxy-hierarchy.json
load-plan.json
+157
View File
@@ -0,0 +1,157 @@
# otopcua-uns-loader
A **reloadable** populate-and-verify tool for the OtOpcUa galaxy Unified Namespace.
Recreates a UNS load grounded in the real AVEVA Galaxy **DEV** hierarchy (the 40
`TestMachine` instances) and verifies it streams **live values** on OPC UA — so
you can rebuild the OtOpcUa docker-dev instance and get the namespace back with
one populate + one deploy click.
## What it loads
One **SystemPlatform** `Tag` per `(machine, signal)` bound to the existing
`GalaxyMxGateway` driver (`MAIN-galaxy-mxgw`). Each tag's `FolderPath` is the
Galaxy object and its `Name` the attribute, so the materialised OPC UA variable
`OtOpcUa/<machine>/<signal>` has a NodeId equal to the MXAccess reference the
driver subscribes to — which is what makes the value go live.
Signals mirrored per machine (only those the instance actually has): the
`$TestMachine` process UDAs — `TestChangingInt`, `TestHistoryValue`,
`TestDouble/Float/Duration/DateTime`, `ProtectedValue(1)`, `TestAlarm001..003`,
`InAlarm`**396 tags across 40 machines**.
Every row carries the `nw-mirror-` `TagId` prefix, so `clean` removes exactly
what the tool created (adopting any pre-existing seed row for the same ref).
## Why a deploy click is in the middle
OtOpcUa applies config only from **sealed Deployment snapshots**, and the only
way to seal one is the AdminUI **"Deploy current configuration"** button
(`http://localhost:9200/deployments`) — there is no SQL/REST/CLI trigger (it's
an in-cluster Akka operation). So the flow is:
```
populate ──SQL──▶ live config tables
│ (you click Deploy at :9200, sign in multi-role/password)
driver applies ▶ materialises variables ▶ SubscribeBulk ▶ live values
verify ──OPC UA──▶ browse + read Good values on :4840
```
`populate` and `clean` print the reminder; `verify --wait` polls until the
deploy lands.
> Live values depend on the driver **SubscribeBulk** pass
> (OtOpcUa `master` ≥ commit `c1ce583`). On older builds variables materialise
> but stay `BadWaitingForInitialData`.
## Setup
```bash
cd otopcua-uns-loader
python3 -m venv .venv
./.venv/bin/pip install -r requirements.txt
```
## Use
```bash
./.venv/bin/python otopcua_uns.py generate # build load-plan.json from galaxy-hierarchy.json
./.venv/bin/python otopcua_uns.py populate # upsert the 396 mirror Tag rows (idempotent)
# → open http://localhost:9200/deployments, sign in, click "Deploy current configuration"
./.venv/bin/python otopcua_uns.py verify --wait # poll until live values are Good on :4840
./.venv/bin/python otopcua_uns.py status # config-DB + address-space snapshot
./.venv/bin/python otopcua_uns.py clean # remove all nw-mirror-* tags (then Deploy again)
```
### Rebuild recovery (the point of the tool)
After the docker-dev instance is rebuilt (DB wiped):
1. Ensure the schema + clusters + the `MAIN-galaxy-mxgw` driver exist
(`dotnet ef database update` + the docker-dev `cluster-seed`; see the OtOpcUa
`docker-dev/README.md`).
2. `populate`**Deploy** at the AdminUI → `verify --wait`.
## Troubleshooting
**`verify` stays INCOMPLETE / deployment "Sealed" but drivers never applied it.**
If you recreate the **admin/coordinator** node (`admin-a`) around the same time
you click Deploy, the dispatch broadcast can be lost — the deployment seals but
`NodeDeploymentState` shows no row for the driver nodes, and the address space
keeps the old content. Recover by: restart the driver nodes
(`docker restart otopcua-dev-driver-a-1 otopcua-dev-driver-b-1`) so they cleanly
re-subscribe, then Deploy again. If a no-op "NoChanges" blocks the re-deploy,
delete the orphan sealed `Deployment` row that no node applied (it has no
`NodeDeploymentState` children) so Deploy sees drift again.
**A rebuilt/restarted node serves an empty address space until the next Deploy.**
On bootstrap a node recovers to its last-*applied* revision and does **not**
re-materialise until a new deployment is dispatched — so after any node restart,
click Deploy once (a config change bumps the revision) to repopulate + re-subscribe.
## Configuration
Defaults target docker-dev; override via flags or env:
| Flag | Env | Default |
|---|---|---|
| `--sql-host/-port/-user/-password/-db` | `OTOPCUA_SQL_*` | `localhost:14330` sa / `OtOpcUa!Dev123` / `OtOpcUa` |
| `--opcua-endpoint` | `OTOPCUA_OPCUA_ENDPOINT` | `opc.tcp://localhost:4840` |
| `--driver` | `OTOPCUA_GALAXY_DRIVER` | `MAIN-galaxy-mxgw` |
| `--galaxy-json` | `OTOPCUA_GALAXY_JSON` | `../galaxy-hierarchy.json` |
| `--deploy-url` | — | `http://localhost:9200/deployments` |
## Files
- `otopcua_uns.py` — the CLI (generate / populate / verify / status / clean)
- `load-plan.json` — generated load plan (machine → signal → MXAccess ref)
- `../galaxy-hierarchy.json` — the source of truth, pulled live from the gateway
- `requirements.txt`, `.venv/`
## Company-shape overlay (`populate-equipment`)
Besides the galaxy-native mirror, the tool can load the **Northwind company
shape** (`filling / line-1 / rinser-01 / speed-rpm`) as a second, **Equipment**-kind
namespace (`nw-uns`, in cluster `MAIN`) from `../company-uns.json`. Each company
signal is a **VirtualTag** (+ a `Script`) whose script simply mirrors the live
galaxy-mirror tag for that signal:
```csharp
return ctx.GetTag("TestMachine_001.TestDouble").Value;
```
so the company shape carries live **VALUES** driven off the same Galaxy source — no
driver, no `BadWaitingForInitialData` once the galaxy mirror is up. The `ctx.GetTag`
literal is the signal's `source.fullTagReference`; the engine's `DependencyExtractor`
harvests it and subscribes the VirtualTag to that galaxy-mirror tag. This needs
OtOpcUa `master` ≥ the Equipment-namespace VirtualTag materialisation milestone (WS-3),
which materialises `VirtualTag`/`Script` rows on deploy and added the **headless
deploy** endpoint.
```bash
./.venv/bin/python otopcua_uns.py populate-equipment # 3 areas / 8 lines / 40 equipment / 1036 VirtualTags
curl -s -X POST http://localhost:9200/api/deployments -H 'X-Api-Key: docker-dev-deploy-key' # headless deploy
./.venv/bin/python otopcua_uns.py verify-equipment --expect 1036 --require-good 396 --wait --wait-seconds 300 # structure + live values
```
> **Verified live 2026-06-07** (OtOpcUa `feat/equipment-namespace-live-values`): galaxy mirror
> **396/396 Good**, company overlay **396 Good** on `opc.tcp://localhost:4840`, `VERIFY-EQUIPMENT: PASS`.
> Why 396 of 1036? The shipped `company-uns.json` invents **1036 distinct** `ctx.GetTag` refs, but only
> **396** of them match a real galaxy-mirror tag — so 396 signals are backed by a live source (and all 396
> go Good); the other 640 cite synthetic refs with no galaxy tag (`BadNodeIdUnknown`). That ratio is a
> property of the company model, not the streaming path — **every signal with a resolvable live source
> streams Good.** So `--require-good 396` is the meaningful gate for the current model. Survives a node
> restart with no re-deploy (the bootstrap-restore path re-materialises + re-applies the VirtualTags).
UNS folders carry the friendly **DisplayName** (`filling`); the BrowseName/NodeId
stay the stable logical Id (`nw-area-filling`) — standard OPC UA. **No driver:** the
company signals are VirtualTags (which link to Equipment + a Script, not a driver); a
placeholder `nw-uns-modbus` driver is kept only because an Equipment namespace is
expected to have one, but no `Tag` binds to it. `verify-equipment --require-good N`
reads each leaf's value and asserts at least N are Good (default `0` = structure-only,
back-compat); `--wait` polls until the deploy + change-triggered evaluations land.
Tracked in `OtOpcUa/docs/plans/2026-06-06-equipment-namespace-materialization-scope.md` (WS-3).
`clean` removes both the mirror tags and the company overlay (the `VirtualTag` +
`Script` rows, in FK-safe order, plus the namespace/driver/equipment/areas/lines).
+601
View File
@@ -0,0 +1,601 @@
#!/usr/bin/env python3
"""
otopcua_uns.py reloadable populate + verify for the OtOpcUa galaxy UNS.
Recreates and verifies an OtOpcUa Unified-Namespace load grounded in the real
AVEVA Galaxy "DEV" hierarchy (the 40 TestMachine instances). Designed to be
re-run after the OtOpcUa docker-dev instance is rebuilt.
Pipeline (see scadaproj/memory otopcua-uns-deploy-and-value-streaming):
populate SQL live config tables (Tag rows, nw-* prefix)
you click "Deploy current configuration" at :9200
driver applies materialises OtOpcUa/<machine>/<signal> SubscribeBulk
verify OPC UA browse + read live values on :4840
What it loads: one SystemPlatform Tag per (machine, signal) bound to the
existing GalaxyMxGateway driver. Each tag's FolderPath is the Galaxy object and
its Name the attribute, so the materialised variable NodeId is exactly the
MXAccess ref the driver subscribes to giving live values. Every row carries
the `nw-` id prefix so `clean` can remove them without touching other config.
Idempotent: populate upserts by TagId; re-running is a no-op when unchanged.
There are TWO overlays:
the galaxy-native mirror (`populate`) SystemPlatform driver Tags, 396 tags;
the Northwind company shape (`populate-equipment`) an Equipment-kind namespace
whose 1036 signals are VirtualTags. Each VirtualTag's Script simply mirrors the
live galaxy-mirror tag (`return ctx.GetTag("<fullTagReference>").Value;`), so the
company shape carries live VALUES driven off the same Galaxy source.
Subcommands:
generate Build the load plan from galaxy-hierarchy.json (writes load-plan.json)
populate Upsert the SystemPlatform mirror Tag rows into the config DB
populate-equipment Load the company shape as VirtualTag+Script rows (mirror the galaxy tags)
verify Check DB rows present + live OPC UA values are Good on :4840
verify-equipment Browse the company tree; --require-good asserts live values
status Show config-DB + address-space state
clean Delete all nw-* mirror Tags + the company VirtualTag/Script overlay
Deploy is a human-gated AdminUI action (no SQL/REST trigger exists); populate
and clean print the reminder and `verify --wait` polls until it lands.
Deps: pymssql, asyncua (see requirements.txt; use the bundled .venv).
"""
import argparse
import hashlib
import json
import os
import re
import sys
import time
import uuid
# ── config (overridable via env / flags) ───────────────────────────────────
DEF_MSSQL = dict(
host=os.environ.get("OTOPCUA_SQL_HOST", "localhost"),
port=int(os.environ.get("OTOPCUA_SQL_PORT", "14330")),
user=os.environ.get("OTOPCUA_SQL_USER", "sa"),
password=os.environ.get("OTOPCUA_SQL_PASSWORD", "OtOpcUa!Dev123"),
database=os.environ.get("OTOPCUA_SQL_DB", "OtOpcUa"),
)
DEF_OPCUA = os.environ.get("OTOPCUA_OPCUA_ENDPOINT", "opc.tcp://localhost:4840")
DEF_DRIVER = os.environ.get("OTOPCUA_GALAXY_DRIVER", "MAIN-galaxy-mxgw")
DEF_GALAXY_JSON = os.environ.get(
"OTOPCUA_GALAXY_JSON",
os.path.join(os.path.dirname(__file__), "..", "galaxy-hierarchy.json"),
)
DEF_COMPANY_JSON = os.environ.get(
"OTOPCUA_COMPANY_JSON",
os.path.join(os.path.dirname(__file__), "..", "company-uns.json"),
)
ID_PREFIX = "nw-mirror-" # SystemPlatform galaxy-mirror TagId prefix
LOAD_PLAN = os.path.join(os.path.dirname(__file__), "load-plan.json")
# Equipment-overlay (company-shape) object ids — all carry the nw- prefix so
# `clean` can remove them. The Equipment namespace is a SECOND namespace loaded
# alongside the galaxy mirror. Each company signal is a VirtualTag (+ Script) whose
# script mirrors the live SystemPlatform galaxy-mirror tag for that signal — so the
# overlay carries live VALUES (scope doc WS-3), not just structure.
EQ_CLUSTER = os.environ.get("OTOPCUA_EQ_CLUSTER", "MAIN")
EQ_NS = "nw-uns"
EQ_ID_PREFIX = "nweq-" # VirtualTag/Script logical-id prefix (cleanup by prefix scan)
# galaxy dataTypeName / gen_uns dtype → valid OtOpcUa DriverDataType
_DTYPE_FIX = {"Double": "Float64", "Float": "Float32"}
_ACCESS = {"ReadOnly": "0", "Read": "0", "ReadWrite": "1"}
# ── the value signals we mirror, per $TestMachine instance ──────────────────
# (galaxy attribute name, OtOpcUa DriverDataType, access '0'=Read/'1'=ReadWrite)
SIGNALS = [
("TestChangingInt", "Int32", "0"),
("TestHistoryValue", "Int32", "0"),
("TestDouble", "Float64", "0"),
("TestFloat", "Float32", "0"),
("TestDuration", "Float64", "0"),
("TestDateTime", "DateTime", "0"),
("ProtectedValue", "Boolean", "1"),
("ProtectedValue1", "Boolean", "1"),
("TestAlarm001", "Boolean", "0"),
("TestAlarm002", "Boolean", "0"),
("TestAlarm003", "Boolean", "0"),
("InAlarm", "Boolean", "0"),
]
# ── plan generation (grounded in the real galaxy) ───────────────────────────
def build_plan(galaxy_json, driver):
with open(galaxy_json) as f:
gal = json.load(f)
machines = [
o for o in gal["objects"]
if "$TestMachine" in o.get("templateChain", [])
]
machines.sort(key=lambda o: o["tagName"])
rows = []
for m in machines:
have = {a["attributeName"] for a in m["attributes"]}
for attr, dtype, access in SIGNALS:
if attr not in have:
continue # only mirror attributes this instance really has
rows.append({
"tag_id": f"{ID_PREFIX}{m['tagName']}-{attr}".lower(),
"driver_instance_id": driver,
"name": attr,
"folder_path": m["tagName"], # → folder; ref = folder.name
"data_type": dtype,
"access_level": access,
"mxaccess_ref": f"{m['tagName']}.{attr}",
})
return {
"source": galaxy_json,
"driver_instance_id": driver,
"machines": len(machines),
"tags": len(rows),
"rows": rows,
}
# ── DB helpers ──────────────────────────────────────────────────────────────
def connect(cfg):
import pymssql
conn = pymssql.connect(
server=cfg["host"], port=str(cfg["port"]), user=cfg["user"],
password=cfg["password"], database=cfg["database"], autocommit=False,
)
cur = conn.cursor()
# The Tag table has filtered indexes / computed columns; writes require this.
cur.execute("SET QUOTED_IDENTIFIER ON; SET ANSI_NULLS ON;")
return conn, cur
def driver_exists(cur, driver):
cur.execute(
"SELECT n.Kind FROM dbo.DriverInstance d "
"JOIN dbo.Namespace n ON n.NamespaceId = d.NamespaceId "
"WHERE d.DriverInstanceId = %s", (driver,))
r = cur.fetchone()
return r[0] if r else None
# ── commands ────────────────────────────────────────────────────────────────
def cmd_generate(args):
plan = build_plan(args.galaxy_json, args.driver)
with open(LOAD_PLAN, "w") as f:
json.dump(plan, f, indent=2)
print(f"plan: {plan['machines']} machines → {plan['tags']} mirror tags (driver {plan['driver_instance_id']})")
print(f"wrote {LOAD_PLAN}")
return 0
def cmd_populate(args):
plan = build_plan(args.galaxy_json, args.driver)
conn, cur = connect(args.mssql)
kind = driver_exists(cur, args.driver)
if kind is None:
print(f"ERROR: driver instance '{args.driver}' not found in config DB.", file=sys.stderr)
return 2
if kind != "SystemPlatform":
print(f"ERROR: driver '{args.driver}' is in a {kind} namespace; the galaxy mirror needs a "
f"SystemPlatform/GalaxyMxGateway driver.", file=sys.stderr)
return 2
inserted = updated = 0
for r in plan["rows"]:
# Upsert by the SystemPlatform natural key (DriverInstanceId, FolderPath, Name)
# — the UX_Tag_FolderPath unique index. This adopts any pre-existing seed row for
# the same ref into our nw-* set (so `clean` can remove it) and stays idempotent on
# re-run. RowId/RowVersion are server-managed.
cur.execute(
"SELECT TagId FROM dbo.Tag WHERE DriverInstanceId=%s AND FolderPath=%s "
"AND Name=%s AND EquipmentId IS NULL",
(r["driver_instance_id"], r["folder_path"], r["name"]))
if cur.fetchone():
cur.execute(
"UPDATE dbo.Tag SET TagId=%s, DataType=%s, AccessLevel=%s, "
"WriteIdempotent=0, TagConfig='{}' "
"WHERE DriverInstanceId=%s AND FolderPath=%s AND Name=%s AND EquipmentId IS NULL",
(r["tag_id"], r["data_type"], r["access_level"],
r["driver_instance_id"], r["folder_path"], r["name"]))
updated += 1
else:
cur.execute(
"INSERT INTO dbo.Tag (TagRowId, TagId, DriverInstanceId, EquipmentId, "
"Name, FolderPath, DataType, AccessLevel, WriteIdempotent, TagConfig) "
"VALUES (NEWID(), %s, %s, NULL, %s, %s, %s, %s, 0, '{}')",
(r["tag_id"], r["driver_instance_id"], r["name"], r["folder_path"],
r["data_type"], r["access_level"]))
inserted += 1
conn.commit()
conn.close()
print(f"populated: {inserted} inserted, {updated} updated "
f"({plan['tags']} mirror tags across {plan['machines']} machines)")
print()
print(">>> NEXT: open the AdminUI, sign in, and click "
"'Deploy current configuration' to seal + serve the load:")
print(f" {args.deploy_url}")
print(">>> then run: otopcua_uns.py verify --wait")
return 0
def _eq_signal_ids(equipment_id, folder, name):
"""Deterministic (VirtualTagId, ScriptId) for a company signal. Both carry the
EQ_ID_PREFIX so `clean` removes exactly what was created. The two ids share the
same per-signal hash but differ by a kind token so they never collide across the
global UX_VirtualTag_LogicalId / UX_Script_LogicalId unique indexes. Capped at the
64-char id column width."""
base = hashlib.sha1(f"{equipment_id}|{folder}|{name}".encode()).hexdigest()[:20]
return EQ_ID_PREFIX + "vt-" + base, EQ_ID_PREFIX + "sc-" + base
def cmd_populate_equipment(args):
"""Load the company-shape Equipment namespace from company-uns.json: a second
(Equipment-kind) namespace alongside the galaxy mirror, with the Northwind
Area/Line/Equipment/Signal tree. Each signal is a VirtualTag whose Script mirrors
the live galaxy-mirror tag for that signal `return ctx.GetTag("<ref>").Value;`
so the company shape streams live VALUES off the same Galaxy source (no driver,
no BadWaitingForInitialData once the galaxy mirror is up). Idempotent:
drop-and-recreate of the nw- overlay rows."""
with open(args.company_json) as f:
doc = json.load(f)
u = doc["uns"]
conn, cur = connect(args.mssql)
# Drop any prior overlay (child rows first), then recreate. VirtualTag/Script go
# before Equipment (VirtualTag.EquipmentId logical-FKs Equipment). Equipment is
# scoped by its overlay UnsLine ('nw-line-%') — NOT by the EquipmentId, which is now the
# canonical 'EQ-'+uuid form (see DraftValidator) and no longer carries an 'nw-' prefix.
cur.execute("DELETE FROM dbo.VirtualTag WHERE VirtualTagId LIKE %s", (EQ_ID_PREFIX + "%",))
cur.execute("DELETE FROM dbo.Script WHERE ScriptId LIKE %s", (EQ_ID_PREFIX + "%",))
cur.execute("DELETE FROM dbo.Equipment WHERE UnsLineId LIKE 'nw-line-%'")
cur.execute("DELETE FROM dbo.UnsLine WHERE UnsLineId LIKE 'nw-line-%'")
cur.execute("DELETE FROM dbo.UnsArea WHERE UnsAreaId LIKE 'nw-area-%'")
# Equipment is now driver-less, but purge any driver still bound to the overlay namespace —
# self-heals environments that ran an older loader which created the 'nw-uns-modbus' placeholder.
cur.execute("DELETE FROM dbo.DriverInstance WHERE NamespaceId=%s", (EQ_NS,))
cur.execute("DELETE FROM dbo.Namespace WHERE NamespaceId=%s", (EQ_NS,))
cur.execute(
"INSERT INTO dbo.Namespace (NamespaceRowId, NamespaceId, ClusterId, Kind, NamespaceUri, Enabled) "
"VALUES (NEWID(), %s, %s, 'Equipment', %s, 1)",
(EQ_NS, EQ_CLUSTER, doc.get("namespace", {}).get("namespaceUri", "urn:northwind:birmingham:uns")))
for a in u["unsAreas"]:
cur.execute("INSERT INTO dbo.UnsArea (UnsAreaRowId, UnsAreaId, ClusterId, Name) VALUES (NEWID(), %s, %s, %s)",
("nw-" + a["unsAreaId"], EQ_CLUSTER, a["name"]))
for l in u["unsLines"]:
cur.execute("INSERT INTO dbo.UnsLine (UnsLineRowId, UnsLineId, UnsAreaId, Name) VALUES (NEWID(), %s, %s, %s)",
("nw-" + l["unsLineId"], "nw-" + l["unsAreaId"], l["name"]))
eq_n = vt_n = 0
for e in u["equipment"]:
eq_uuid = uuid.uuid5(uuid.NAMESPACE_URL, "otopcua-nw-eq/" + e["equipmentId"])
# Canonical EquipmentId: matches OtOpcUa DraftValidator.DeriveEquipmentId
# ("EQ-" + EquipmentUuid.ToString("N")[..12].ToLowerInvariant()). uuid.UUID.hex is
# already lowercase, 32 hex chars, no dashes — .hex[:12] is the first 12.
eq_id = "EQ-" + eq_uuid.hex[:12].lower()
eq_uuid = str(eq_uuid)
cur.execute(
"INSERT INTO dbo.Equipment (EquipmentRowId, EquipmentId, EquipmentUuid, UnsLineId, "
"Name, MachineCode, Manufacturer, Model, Enabled) VALUES (NEWID(), %s, %s, %s, %s, %s, %s, %s, 1)",
(eq_id, eq_uuid, "nw-" + e["unsLineId"], e["name"], e["machineCode"],
e.get("manufacturer"), e.get("model")))
eq_n += 1
for t in e["tags"]:
dtype = _DTYPE_FIX.get(t["dataType"], t["dataType"])
folder = t.get("folderPath")
# The galaxy-mirror MXAccess ref (e.g. TestMachine_001.TestDouble) is the upstream
# the VirtualTag mirrors. DependencyExtractor harvests the literal in ctx.GetTag(),
# so the engine subscribes to exactly this path on the galaxy-mirror driver.
full = t["source"]["fullTagReference"]
vt_id, sc_id = _eq_signal_ids(e["equipmentId"], folder, t["name"])
source_code = f'return ctx.GetTag("{full}").Value;'
source_hash = hashlib.sha256(source_code.encode()).hexdigest()
cur.execute(
"INSERT INTO dbo.Script (ScriptRowId, ScriptId, Name, SourceCode, SourceHash, Language) "
"VALUES (NEWID(), %s, %s, %s, %s, 'CSharp')",
(sc_id, t["name"], source_code, source_hash))
cur.execute(
"INSERT INTO dbo.VirtualTag (VirtualTagRowId, VirtualTagId, EquipmentId, Name, DataType, "
"ScriptId, ChangeTriggered, TimerIntervalMs, Historize, Enabled) "
"VALUES (NEWID(), %s, %s, %s, %s, %s, 1, NULL, %s, 1)",
(vt_id, eq_id, t["name"], dtype, sc_id, 1 if t.get("historize") else 0))
vt_n += 1
conn.commit()
conn.close()
print(f"populated equipment overlay: namespace {EQ_NS} ({EQ_CLUSTER}), "
f"{len(u['unsAreas'])} areas, {len(u['unsLines'])} lines, {eq_n} equipment, "
f"{vt_n} VirtualTags (+ {vt_n} mirror Scripts)")
print()
print(f">>> NEXT: deploy (headless) — curl -s -X POST {args.deploy_url.replace('/deployments','')}/api/deployments "
f"-H 'X-Api-Key: {args.deploy_key}'")
print(">>> then run: otopcua_uns.py verify-equipment --require-good 1036 --wait")
return 0
def cmd_clean(args):
conn, cur = connect(args.mssql)
cur.execute("DELETE FROM dbo.Tag WHERE TagId LIKE %s", (ID_PREFIX + "%",))
n = cur.rowcount
# Also drop the company-shape Equipment overlay (child rows first): VirtualTag and
# Script (both nweq-*) before Equipment, then the rest.
# Equipment is scoped by its overlay UnsLine ('nw-line-%') — NOT by the EquipmentId,
# which is now the canonical 'EQ-'+uuid form (see DraftValidator) with no 'nw-' prefix.
cur.execute("DELETE FROM dbo.VirtualTag WHERE VirtualTagId LIKE %s", (EQ_ID_PREFIX + "%",))
cur.execute("DELETE FROM dbo.Script WHERE ScriptId LIKE %s", (EQ_ID_PREFIX + "%",))
cur.execute("DELETE FROM dbo.Equipment WHERE UnsLineId LIKE 'nw-line-%'")
cur.execute("DELETE FROM dbo.UnsLine WHERE UnsLineId LIKE 'nw-line-%'")
cur.execute("DELETE FROM dbo.UnsArea WHERE UnsAreaId LIKE 'nw-area-%'")
# Purge any driver still bound to the overlay namespace (e.g. the legacy 'nw-uns-modbus'
# placeholder created by an older loader) so 'clean' fully removes the overlay.
cur.execute("DELETE FROM dbo.DriverInstance WHERE NamespaceId=%s", (EQ_NS,))
cur.execute("DELETE FROM dbo.Namespace WHERE NamespaceId=%s", (EQ_NS,))
conn.commit()
conn.close()
print(f"removed {n} nw-* mirror tag(s) + the {EQ_NS} equipment overlay. "
f"Deploy again at {args.deploy_url} to drop them from the address space.")
return 0
def cmd_status(args):
conn, cur = connect(args.mssql)
cur.execute("SELECT COUNT(*) FROM dbo.Tag WHERE TagId LIKE %s", (ID_PREFIX + "%",))
db_tags = cur.fetchone()[0]
# Deployment.Status: 2 = Sealed (the snapshot driver nodes apply).
cur.execute("SELECT TOP 1 RevisionHash, SealedAtUtc FROM dbo.Deployment "
"WHERE Status=2 ORDER BY SealedAtUtc DESC")
dep = cur.fetchone()
conn.close()
print(f"config DB : {db_tags} mirror tags (nw-*) present")
print(f"last sealed : {('rev '+dep[0][:12]+'… @ '+str(dep[1])) if dep else '(none)'}")
folders, variables, good = browse_summary(args.opcua_endpoint)
print(f"address space : {folders} machine folder(s), {variables} variable(s), {good} value(s) Good on {args.opcua_endpoint}")
return 0
def cmd_verify(args):
plan = build_plan(args.galaxy_json, args.driver)
expected = plan["tags"]
deadline = time.time() + (args.wait_seconds if args.wait else 0)
while True:
folders, variables, good = browse_summary(args.opcua_endpoint)
ok = variables >= expected and good >= max(1, int(expected * 0.5))
if ok or time.time() >= deadline:
break
print(f" waiting for deploy… ({variables}/{expected} vars, {good} Good)")
time.sleep(5)
print(f"expected mirror tags : {expected}")
print(f"address-space vars : {variables} (in {folders} folders)")
print(f"values Good (live) : {good}")
sample = sample_values(args.opcua_endpoint, 6)
for nm, val, sc in sample:
print(f" {nm} = {val} [{sc}]")
passed = variables >= expected and good >= max(1, int(expected * 0.5))
print("VERIFY:", "PASS — UNS loaded and live" if passed else "INCOMPLETE — did you Deploy at the AdminUI?")
return 0 if passed else 1
# ── OPC UA helpers (asyncua) ────────────────────────────────────────────────
def browse_summary(endpoint):
import asyncio
from asyncua import Client
async def run():
folders = variables = good = 0
async with Client(endpoint) as c:
for k in await c.nodes.objects.get_children():
if (await k.read_browse_name()).Name != "OtOpcUa":
continue
for f in await k.get_children():
folders += 1
for v in await f.get_children():
variables += 1
try:
dv = await v.read_data_value()
if dv.StatusCode and dv.StatusCode.is_good():
good += 1
except Exception:
pass
return folders, variables, good
try:
return asyncio.run(run())
except Exception as e:
return (f"<{type(e).__name__}>", 0, 0)
def sample_values(endpoint, n):
import asyncio
from asyncua import Client
async def run():
out = []
async with Client(endpoint) as c:
for k in await c.nodes.objects.get_children():
if (await k.read_browse_name()).Name != "OtOpcUa":
continue
for f in await k.get_children():
for v in await f.get_children():
try:
dv = await v.read_data_value()
sc = dv.StatusCode.name if dv.StatusCode else "?"
out.append(((await v.read_browse_name()).Name, dv.Value.Value, sc))
except Exception as e:
out.append(((await v.read_browse_name()).Name, f"<{type(e).__name__}>", "?"))
if len(out) >= n:
return out
return out
try:
return asyncio.run(run())
except Exception as e:
return [("<browse error>", str(e), "?")]
def browse_tree(endpoint, max_depth=8, top_prefix=None, read_values=False):
"""Recursively descend the OtOpcUa address space and count leaf variables, returning
(folder_count, leaf_count, leaf_paths, good_count). A node with no children is a leaf
signal this correctly handles the DEEP Equipment UNS tree
(Area/Line/Equipment/[FolderPath]/Signal), unlike browse_summary which assumes the flat
2-level Galaxy hierarchy. When top_prefix is set, only top-level OtOpcUa folders whose
browse name starts with it are counted (e.g. 'nw-area-' scopes to the company Equipment
overlay, excluding the Galaxy mirror folders). When read_values is True, each leaf's value
is read and good_count tallies the Good-quality ones (else good_count is 0)."""
import asyncio
from asyncua import Client
async def maybe_good(node, acc):
if not read_values:
return
try:
dv = await node.read_data_value()
if dv.StatusCode and dv.StatusCode.is_good():
acc["good"] += 1
except Exception:
pass
async def walk(node, path, depth, acc):
if depth >= max_depth:
return
for ch in await node.get_children():
try:
name = (await ch.read_browse_name()).Name
except Exception:
continue
child_path = path + "/" + name
grandkids = await ch.get_children()
if grandkids:
acc["folders"] += 1
await walk(ch, child_path, depth + 1, acc)
else:
acc["leaves"] += 1
acc["paths"].append(child_path)
await maybe_good(ch, acc)
async def run():
acc = {"folders": 0, "leaves": 0, "paths": [], "good": 0}
async with Client(endpoint) as c:
for k in await c.nodes.objects.get_children():
if (await k.read_browse_name()).Name != "OtOpcUa":
continue
for top in await k.get_children():
tn = (await top.read_browse_name()).Name
if top_prefix and not tn.startswith(top_prefix):
continue
if await top.get_children():
acc["folders"] += 1
await walk(top, "OtOpcUa/" + tn, 1, acc)
else:
acc["leaves"] += 1
acc["paths"].append("OtOpcUa/" + tn)
await maybe_good(top, acc)
return acc["folders"], acc["leaves"], acc["paths"], acc["good"]
try:
return asyncio.run(run())
except Exception as e:
return (f"<{type(e).__name__}: {e}>", 0, [], 0)
def cmd_verify_equipment(args):
"""Browse the full UNS tree by friendly Area/Line/Equipment/Signal names and report the leaf
signal count. With --expect N, exit non-zero unless exactly N leaf signals are present (the
equipment-namespace structure-materialisation check). With --require-good N (>0), also read
each leaf's value and require at least N Good ones (the live-VALUE check for the VirtualTag
overlay) back-compat default 0 = structure-only. --wait polls so it can wait for the deploy
+ change-triggered VirtualTag evaluations to land."""
top_prefix = None if args.all else "nw-area-"
scope = "whole address space" if args.all else "company overlay (nw-area-*)"
read_values = args.require_good > 0
deadline = time.time() + (args.wait_seconds if args.wait else 0)
while True:
folders, leaves, paths, good = browse_tree(
args.opcua_endpoint, top_prefix=top_prefix, read_values=read_values)
struct_ok = args.expect is None or leaves == args.expect
good_ok = good >= args.require_good
if (struct_ok and good_ok) or time.time() >= deadline:
break
print(f" waiting for deploy/values… ({leaves} leaves"
+ (f", {good} Good" if read_values else "") + ")")
time.sleep(5)
suffix = f", {good} Good value(s)" if read_values else ""
print(f"equipment tree : {folders} folder(s), {leaves} leaf signal(s){suffix} "
f"on {args.opcua_endpoint} [{scope}]")
for p in sorted(paths)[:args.show]:
print(f" {p}")
if len(paths) > args.show:
print(f" … and {len(paths) - args.show} more")
passed = True
if args.expect is not None:
struct_ok = leaves == args.expect
passed = passed and struct_ok
print(" structure :",
f"PASS ({leaves} == {args.expect})" if struct_ok
else f"FAIL (expected {args.expect}, found {leaves})")
if args.require_good > 0:
good_ok = good >= args.require_good
passed = passed and good_ok
print(" live good :",
f"PASS ({good} >= {args.require_good})" if good_ok
else f"FAIL (expected >= {args.require_good} Good, found {good})")
if args.expect is None and args.require_good == 0:
return 0
print("VERIFY-EQUIPMENT:", "PASS" if passed else "FAIL")
return 0 if passed else 1
# ── arg parsing ─────────────────────────────────────────────────────────────
def main(argv):
p = argparse.ArgumentParser(description="Reloadable populate + verify for the OtOpcUa galaxy UNS.")
p.add_argument("--galaxy-json", default=DEF_GALAXY_JSON)
p.add_argument("--driver", default=DEF_DRIVER, help="SystemPlatform GalaxyMxGateway driver instance id")
p.add_argument("--opcua-endpoint", default=DEF_OPCUA)
p.add_argument("--deploy-url", default="http://localhost:9200/deployments")
p.add_argument("--deploy-key", default=os.environ.get("OTOPCUA_DEPLOY_KEY", "docker-dev-deploy-key"),
help="X-Api-Key for the headless POST /api/deployments endpoint")
p.add_argument("--company-json", default=DEF_COMPANY_JSON)
p.add_argument("--sql-host", default=DEF_MSSQL["host"])
p.add_argument("--sql-port", type=int, default=DEF_MSSQL["port"])
p.add_argument("--sql-user", default=DEF_MSSQL["user"])
p.add_argument("--sql-password", default=DEF_MSSQL["password"])
p.add_argument("--sql-db", default=DEF_MSSQL["database"])
sub = p.add_subparsers(dest="cmd", required=True)
sub.add_parser("generate")
sub.add_parser("populate")
sub.add_parser("populate-equipment",
help="load the company-shape Equipment namespace from company-uns.json (structure-only)")
sub.add_parser("clean")
sub.add_parser("status")
vp = sub.add_parser("verify")
vp.add_argument("--wait", action="store_true", help="poll until the deploy lands")
vp.add_argument("--wait-seconds", type=int, default=120)
ep = sub.add_parser("verify-equipment",
help="recursively browse the Equipment UNS tree + count leaf signals "
"(+ optionally assert live Good values)")
ep.add_argument("--expect", type=int, default=None, help="assert exactly N leaf signals")
ep.add_argument("--require-good", type=int, default=0,
help="read each leaf's value and require >= N Good ones (0 = structure-only, default)")
ep.add_argument("--show", type=int, default=20, help="how many leaf paths to print")
ep.add_argument("--all", action="store_true",
help="count the whole address space (default: only the nw-area-* company overlay)")
ep.add_argument("--wait", action="store_true",
help="poll until the deploy lands + (with --require-good) values go Good")
ep.add_argument("--wait-seconds", type=int, default=120)
a = p.parse_args(argv)
a.mssql = dict(host=a.sql_host, port=a.sql_port, user=a.sql_user,
password=a.sql_password, database=a.sql_db)
return {
"generate": cmd_generate, "populate": cmd_populate,
"populate-equipment": cmd_populate_equipment, "clean": cmd_clean,
"status": cmd_status, "verify": cmd_verify, "verify-equipment": cmd_verify_equipment,
}[a.cmd](a)
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
+2
View File
@@ -0,0 +1,2 @@
pymssql>=2.3
asyncua>=2.0