Compare commits

...

32 Commits

Author SHA1 Message Date
Joseph Doherty b80abbb14b docs(index): HistorianGateway store-forward now FasterLog-backed; refresh test count (702/681) 2026-06-24 11:19:56 -04:00
Joseph Doherty 6c2d16d4af docs: refresh HistorianGateway + GalaxyRepository status in index
HistorianGateway is now pushed to gitea (gitea.dohertylan.com/dohertj2/historiangw), and
ZB.MOM.WW.GalaxyRepository is published to the Gitea feed and consumed as a PackageReference
(no longer a cross-repo ProjectReference). Updates the sister-project row, the component
table, and the GalaxyRepository narrative; test figure 584 green -> 590 total (584 on macOS).
2026-06-24 07:28:25 -04:00
Joseph Doherty a08ddab9dd chore: retire unused ZB.MOM.WW.SPHistorianClient (stale partial port; superseded by histsdk vendored in HistorianGateway; no consumers, not on feed) 2026-06-24 06:45:10 -04:00
Joseph Doherty 744eb090ac docs(scadaproj): index ZB.MOM.WW.HistorianGateway sidecar + GalaxyRepository shared lib
- Add HistorianGateway to the Runtime/implementation table (single-process
  .NET 10 x64 gRPC sidecar; no COM/x86; 584 tests; local only, not yet
  pushed to gitea)
- Update "What this repository is" count (five → six pieces of source;
  add GalaxyRepository)
- Add HistorianGateway paragraph to Cross-project relationships / Net effect
  (independent sidecar; no runtime coupling to the other three; depends on
  shared GalaxyRepository lib via ProjectReference)
- Add ZB.MOM.WW.GalaxyRepository row to Component normalization table +
  full description paragraph (built 0.1.0; consumed by HistorianGateway;
  mxaccessgw adoption is a follow-on; not yet published to Gitea feed)
- Add HistorianGateway primary commands block (build/test/run/live-integration)
- Extend Shared GLAuth note to cover HistorianGateway
2026-06-24 00:41:29 -04:00
Joseph Doherty 94512acf1f fix(galaxyrepo): drop no-op ValidateOnStart (consumer owns validation) 2026-06-23 20:36:28 -04:00
Joseph Doherty 2c6c764d3c test(galaxyrepo): projector + cache tests; dispose semaphores; pack 0.1.0 2026-06-23 20:34:32 -04:00
Joseph Doherty a30f8551e9 feat(galaxyrepo): reusable gRPC service + AddZbGalaxyRepository DI 2026-06-23 20:26:59 -04:00
Joseph Doherty afd0287f54 feat(galaxyrepo): hierarchy cache + snapshot + refresh service + projector 2026-06-23 20:22:35 -04:00
Joseph Doherty 1041f87b59 feat(galaxyrepo): SQL browse provider (hierarchy + attributes) 2026-06-23 20:12:33 -04:00
Joseph Doherty 5572edda85 feat(galaxyrepo): canonical galaxy_repository.v1 proto (neutral namespace) 2026-06-23 20:05:39 -04:00
Joseph Doherty aff7264df8 feat(galaxyrepo): scaffold ZB.MOM.WW.GalaxyRepository shared lib 2026-06-23 19:48:43 -04:00
Joseph Doherty 510b0010d6 docs(historian-gateway): implementation plan + task ledger (31 tasks) 2026-06-23 19:43:08 -04:00
Joseph Doherty 42ad31aded docs(historian-gateway): brainstormed design for ZB.MOM.WW.HistorianGateway sidecar 2026-06-23 19:31:54 -04:00
Joseph Doherty e3c0503a4f docs(sphistorianclient): mark RemoteGrpc (2023 R2) live-verified 2026-06-19 06:57:06 -04:00
Joseph Doherty a0527f9b5a fix(sphistorianclient): gRPC auth handshake uses StorageService.ValidateClientCredential
The RemoteGrpc orchestrator drove the SSPI/NTLM token loop through
HistoryService.ExchangeKey, which the 2023 R2 contract analysis shows is a
separate key-exchange/cert op — not the credential handshake. The server
rejected the NTLM Type-1 token at round 0. The Negotiate loop belongs on
StorageService.ValidateClientCredential (Handle/InBuff -> Status/OutBuff;
field names match the 2020 native contract). Live-verified end-to-end against
a 2023 R2 Historian (wonder-sql-vd03): SysTimeSec raw read returns correct
timestamped values.
2026-06-19 06:56:44 -04:00
Joseph Doherty 5f7d7e1b58 docs(sphistorianclient): document HISTORIAN_PORT env var; mark plan tasks complete 2026-06-19 06:09:43 -04:00
Joseph Doherty 78418346df build(sphistorianclient): pack 0.1.0 nupkg 2026-06-19 06:02:05 -04:00
Joseph Doherty 4920b89666 docs(sphistorianclient): correct retrieval-mode count (15) + EnsureTag verification scope 2026-06-19 06:01:07 -04:00
Joseph Doherty 989db9317d docs(sphistorianclient): add CLAUDE.md + README.md 2026-06-19 05:58:13 -04:00
Joseph Doherty 81bf7322f0 feat(sphistorianclient): add AddZbSpHistorianClient DI extension 2026-06-19 05:53:56 -04:00
Joseph Doherty 8033a7f12d fix(sphistorianclient): resolve port build/test fallout 2026-06-19 05:49:22 -04:00
Joseph Doherty 63cddfb65b feat(sphistorianclient): port SDK source + tests, rebrand namespace to ZB.MOM.WW.SPHistorianClient 2026-06-19 05:45:06 -04:00
Joseph Doherty 965f5006f2 feat(sphistorianclient): scaffold shared library skeleton (props, csprojs, slnx) 2026-06-19 05:40:10 -04:00
Joseph Doherty 294da8b2db docs(sphistorianclient): implementation plan + task tracking 2026-06-19 05:36:24 -04:00
Joseph Doherty bbb7942788 docs(sphistorianclient): approved design for ZB.MOM.WW.SPHistorianClient port 2026-06-19 05:29:51 -04:00
Joseph Doherty d5b134b117 docs: add MES + Delmia-DNC integration API/MXAccess specs
mes-delmia-integration-api.md: endpoints, request/response DTOs, and the MXAccess flag handshake for MESAPI (in-repo MesNotifier) and DelmiaIntegration (DNC Downloader.asmx -> WWNotifier /notify -> Galaxy $DelmiaReceiver). mesrec.md / nj.md: live Galaxy receiver + reactor attribute references.
2026-06-17 06:52:36 -04:00
Joseph Doherty eb8b44c29d loader: purge legacy driver in overlay namespace on teardown (self-heal nw-uns-modbus placeholder) 2026-06-08 07:07:22 -04:00
Joseph Doherty a6fa36043a loader: equipment is driver-less (drop Modbus placeholder, NULL DriverInstanceId) 2026-06-08 06:42:31 -04:00
Joseph Doherty 05a4a547f4 feat(loader): canonical EQ-+uuid EquipmentIds (passes OtOpcUa full DraftValidator); clean by UnsLine scope 2026-06-07 11:18:39 -04:00
Joseph Doherty 4d57e34ff3 docs(loader): record live-values verification + 396/1036 explanation for company overlay 2026-06-07 06:08:36 -04:00
Joseph Doherty b3d8990a0f fix(loader): keep empty folderPath distinct in vtag ids; dedupe verify args; readme wait-seconds 2026-06-07 05:07:00 -04:00
Joseph Doherty 5655b75fe6 feat(loader): company overlay as VirtualTags mirroring the galaxy mirror + verify --require-good 2026-06-07 04:59:51 -04:00
49 changed files with 6318 additions and 88 deletions
+49 -11
View File
@@ -6,12 +6,13 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
`scadaproj` is primarily an umbrella/index workspace that aggregates a family of `scadaproj` is primarily an umbrella/index workspace that aggregates a family of
related SCADA / OT / Wonderware / OPC UA "sister projects" that live as **sibling related SCADA / OT / Wonderware / OPC UA "sister projects" that live as **sibling
directories under `~/Desktop/`**. It now also **hosts five pieces of source itself** directories under `~/Desktop/`**. It now also **hosts six pieces of source itself**
the shared [`ZB.MOM.WW.Auth/`](ZB.MOM.WW.Auth/) library, the shared the shared [`ZB.MOM.WW.Auth/`](ZB.MOM.WW.Auth/) library, the shared
[`ZB.MOM.WW.Theme/`](ZB.MOM.WW.Theme/) UI kit, the shared [`ZB.MOM.WW.Theme/`](ZB.MOM.WW.Theme/) UI kit, the shared
[`ZB.MOM.WW.Health/`](ZB.MOM.WW.Health/) health-check library, the shared [`ZB.MOM.WW.Health/`](ZB.MOM.WW.Health/) health-check library, the shared
[`ZB.MOM.WW.Telemetry/`](ZB.MOM.WW.Telemetry/) observability library, and the shared [`ZB.MOM.WW.Telemetry/`](ZB.MOM.WW.Telemetry/) observability library, the shared
[`ZB.MOM.WW.Configuration/`](ZB.MOM.WW.Configuration/) config-validation library — all the realized output of their [`ZB.MOM.WW.Configuration/`](ZB.MOM.WW.Configuration/) config-validation library, and the new
[`ZB.MOM.WW.GalaxyRepository/`](ZB.MOM.WW.GalaxyRepository/) Galaxy browse library — all the realized output of their
respective component normalizations (see [Component normalization](#component-normalization)). respective component normalizations (see [Component normalization](#component-normalization)).
The point of this file is to give a high-level scan of each sister project — its purpose, The point of this file is to give a high-level scan of each sister project — its purpose,
location, stack, and primary commands — so a fresh Claude Code session can orient across location, stack, and primary commands — so a fresh Claude Code session can orient across
@@ -30,9 +31,10 @@ own `CLAUDE.md` for the full picture. See [Refreshing this index](#refreshing-th
| Project | Location | Stack | Repo | Summary | | Project | Location | Stack | Repo | Summary |
|---|---|---|---|---| |---|---|---|---|---|
| **OtOpcUa** | `~/Desktop/OtOpcUa` | .NET 10, OPC UA, gRPC | `gitea.dohertylan.com/dohertj2/lmxopcua` | OPC UA server that exposes AVEVA System Platform (Wonderware) Galaxy tags as an OPC UA address space. Galaxy access flows through an in-process `GalaxyDriver` → gRPC → the **mxaccessgw** gateway. | | **OtOpcUa** | `~/Desktop/OtOpcUa` | .NET 10, OPC UA, gRPC | `gitea.dohertylan.com/dohertj2/lmxopcua` | OPC UA server that exposes industrial data sources under a **unified Equipment-based address space** — native-protocol drivers (Modbus, S7, AB CIP/Legacy, TwinCAT, FOCAS, OpcUaClient) **and AVEVA System Platform (Wonderware) Galaxy, now a standard Equipment-kind driver** (the old SystemPlatform mirror / alias-tag model was retired ~2026-06-12). Galaxy access flows through the in-process `GalaxyDriver` → gRPC → the **mxaccessgw** gateway. Surfaces live read + authorized write, native OPC UA Part 9 alarms, and server-side HistoryRead. |
| **MxAccessGateway** (`mxaccessgw`) | `~/Desktop/MxAccessGateway` | .NET 10 gateway (x64) + .NET 4.8 worker (**x86**), gRPC | `gitea.dohertylan.com/dohertj2/mxaccessgw` | gRPC gateway giving modern clients full MXAccess parity without loading 32-bit COM. Two-process: gateway (ASP.NET Core gRPC + Blazor dashboard) + per-session x86 worker that owns the MXAccess COM STA. **OtOpcUa depends on this.** | | **MxAccessGateway** (`mxaccessgw`) | `~/Desktop/MxAccessGateway` | .NET 10 gateway (x64) + .NET 4.8 worker (**x86**), gRPC | `gitea.dohertylan.com/dohertj2/mxaccessgw` | gRPC gateway giving modern clients full MXAccess parity without loading 32-bit COM. Two-process: gateway (ASP.NET Core gRPC + Blazor dashboard) + per-session x86 worker that owns the MXAccess COM STA. **OtOpcUa depends on this.** |
| **ScadaBridge** | `~/Desktop/ScadaBridge` | .NET 10, Akka.NET, Docker | _git_ | Full implementation of the distributed SCADA platform — hub-and-spoke (1 central cluster + N site clusters). Projects prefixed `ZB.MOM.WW.ScadaBridge.*`; solution `ZB.MOM.WW.ScadaBridge.slnx`. Ships `src/`, `tests/`, `docker/` topology, and the design docs that are the spec. | | **ScadaBridge** | `~/Desktop/ScadaBridge` | .NET 10, Akka.NET, Docker | _git_ | Full implementation of the distributed SCADA platform — hub-and-spoke (1 central cluster + N site clusters). Projects prefixed `ZB.MOM.WW.ScadaBridge.*`; solution `ZB.MOM.WW.ScadaBridge.slnx`. Ships `src/`, `tests/`, `docker/` topology, and the design docs that are the spec. |
| **HistorianGateway** | `~/Desktop/HistorianGateway` | .NET 10 x64, gRPC, Blazor | `gitea.dohertylan.com/dohertj2/historiangw` | Single-process gRPC sidecar exposing (1) full read/write API to the AVEVA Historian (5 gRPC services; 15 retrieval modes; historical/backfill writes; tag-config lifecycle; SQL live-value path; store-forward + redundancy resilience; all default-disabled) and (2) read-only Galaxy object-hierarchy browse via the shared `ZB.MOM.WW.GalaxyRepository` lib (consumed as a Gitea-feed package). No COM, no x86 worker. Dashboard on `:5220` (HTTP/1.1); gRPC h2c on `:5221`. Vendors `AVEVA.Historian.Client` from `histsdk`. Store-forward uses a crash-safe FasterLog append-only outbox (`Microsoft.FASTER.Core` 2.6.5; `CommitMode` PerEntry/Periodic), not SQLite. 702 tests total — 681 green on macOS; the env-gated live historian + Galaxy integration suite (21 tests) skips without a live server. |
## Cross-project relationships ## Cross-project relationships
@@ -84,8 +86,10 @@ the gateway uses `MxGateway.*`). The common subject is **AVEVA System Platform (
`GalaxyRepositoryClient` for the static hierarchy, and an MXAccess session `GalaxyRepositoryClient` for the static hierarchy, and an MXAccess session
(`MxCommand`/`MxEvent` protos) for live read/write/subscribe. A `DeployWatcher` polls the (`MxCommand`/`MxEvent` protos) for live read/write/subscribe. A `DeployWatcher` polls the
gateway's deploy-event signal to rebuild the OPC UA address space on Galaxy redeploy. gateway's deploy-event signal to rebuild the OPC UA address space on Galaxy redeploy.
OtOpcUa's job is purely a **protocol bridge**: it republishes Galaxy as an OPC UA address OtOpcUa's job is a **protocol bridge**: it republishes Galaxy — now bound as a *standard
space for *any* OPC UA client. Equipment-kind driver* alongside its native-protocol drivers, not a special SystemPlatform
mirror — as an OPC UA address space (live values, Part 9 alarms, HistoryRead) for *any* OPC
UA client.
- **ScadaBridge → OPC UA** (OPC UA client). ScadaBridge's DCL has an OPC UA adapter that - **ScadaBridge → OPC UA** (OPC UA client). ScadaBridge's DCL has an OPC UA adapter that
collects data and mirrors native OPC UA Alarms & Conditions. OtOpcUa is exactly such a collects data and mirrors native OPC UA Alarms & Conditions. OtOpcUa is exactly such a
server, so ScadaBridge can ingest Wonderware data **indirectly via OtOpcUa**. server, so ScadaBridge can ingest Wonderware data **indirectly via OtOpcUa**.
@@ -101,15 +105,21 @@ the gateway uses `MxGateway.*`). The common subject is **AVEVA System Platform (
- ScadaBridge has **two paths** to the same Wonderware data: (1) OPC UA → OtOpcUa → - ScadaBridge has **two paths** to the same Wonderware data: (1) OPC UA → OtOpcUa →
gateway, or (2) MxGateway adapter → gateway directly. Path 1 gives standards-based OPC UA gateway, or (2) MxGateway adapter → gateway directly. Path 1 gives standards-based OPC UA
decoupling; path 2 gives a more direct/native feed. decoupling; path 2 gives a more direct/native feed.
- **HistorianGateway is a new, independent sidecar** (no runtime coupling to the three above).
It reaches the Historian via its vendored gRPC client and the Galaxy Repository SQL DB directly,
not through `mxaccessgw`. It consumes the shared `ZB.MOM.WW.GalaxyRepository` lib
(cross-repo `ProjectReference`). Any client that needs Historian data or Galaxy browse can
target HistorianGateway independently; it is not a dependency of OtOpcUa or ScadaBridge today.
- Coupling is loose: each repo references the others only as **sibling context** (the - Coupling is loose: each repo references the others only as **sibling context** (the
`## Sister Projects` note in ScadaBridge's own `CLAUDE.md` lists `MxAccessGateway` and `## Sister Projects` note in ScadaBridge's own `CLAUDE.md` lists `MxAccessGateway` and
`OtOpcUa` with their Gitea URLs but states they are *not part of its solution*). `OtOpcUa` with their Gitea URLs but states they are *not part of its solution*).
- **The break surface is the wire contracts, not code.** Because coupling is by network - **The break surface is the wire contracts, not code.** Because coupling is by network
protocol, the things that break across repo boundaries are: the gateway's `.proto` files protocol, the things that break across repo boundaries are: the gateway's `.proto` files
(`mxaccess_gateway.proto`, `mxaccess_worker.proto`, `galaxy_repository.proto`), and the (`mxaccess_gateway.proto`, `mxaccess_worker.proto`, `galaxy_repository.proto`), the
OPC UA address-space shape OtOpcUa publishes (browse paths, node IDs, A&C alarm model). `historian_gateway.v1` proto (HistorianGateway's own contract), and the OPC UA address-space
Changes to any of these must be coordinated across the affected repos — a green build in shape OtOpcUa publishes (browse paths, node IDs, A&C alarm model). Changes to any of these
one repo does not prove the others still interoperate. must be coordinated across the affected repos — a green build in one repo does not prove the
others still interoperate.
## Component normalization ## Component normalization
@@ -126,6 +136,7 @@ each project's **code-verified current state**, and the **gaps** between. See
| Observability (metrics / traces / logs) | Built (lib `0.1.0`) | Shared `ZB.MOM.WW.Telemetry` lib + `.Serilog` | [`components/observability/`](components/observability/) | [`ZB.MOM.WW.Telemetry/`](ZB.MOM.WW.Telemetry/) | | Observability (metrics / traces / logs) | Built (lib `0.1.0`) | Shared `ZB.MOM.WW.Telemetry` lib + `.Serilog` | [`components/observability/`](components/observability/) | [`ZB.MOM.WW.Telemetry/`](ZB.MOM.WW.Telemetry/) |
| Config + validation (options / startup validation) | Adopted (lib `0.1.0`; all 3 apps, local) | Shared `ZB.MOM.WW.Configuration` lib | [`components/configuration/`](components/configuration/) | [`ZB.MOM.WW.Configuration/`](ZB.MOM.WW.Configuration/) | | Config + validation (options / startup validation) | Adopted (lib `0.1.0`; all 3 apps, local) | Shared `ZB.MOM.WW.Configuration` lib | [`components/configuration/`](components/configuration/) | [`ZB.MOM.WW.Configuration/`](ZB.MOM.WW.Configuration/) |
| Audit (event model + writer seam) | Adopted (lib `0.1.0`; all 3 apps, merged to **local default** main/master + **pushed to origin** (gitea)) | Shared `ZB.MOM.WW.Audit` lib | [`components/audit/`](components/audit/) | [`ZB.MOM.WW.Audit/`](ZB.MOM.WW.Audit/) | | Audit (event model + writer seam) | Adopted (lib `0.1.0`; all 3 apps, merged to **local default** main/master + **pushed to origin** (gitea)) | Shared `ZB.MOM.WW.Audit` lib | [`components/audit/`](components/audit/) | [`ZB.MOM.WW.Audit/`](ZB.MOM.WW.Audit/) |
| Galaxy Repository (object-hierarchy SQL browse + gRPC service) | Built (lib `0.1.0`, **published to the Gitea feed**; consumed by HistorianGateway as a feed `PackageReference`) | Shared `ZB.MOM.WW.GalaxyRepository` lib | _(design in histsdk + design doc 2026-06-23)_ | [`ZB.MOM.WW.GalaxyRepository/`](ZB.MOM.WW.GalaxyRepository/) |
The auth component is fully populated: a normalized [`spec`](components/auth/spec/SPEC.md), a The auth component is fully populated: a normalized [`spec`](components/auth/spec/SPEC.md), a
proposed [`shared-contract`](components/auth/shared-contract/ZB.MOM.WW.Auth.md), three proposed [`shared-contract`](components/auth/shared-contract/ZB.MOM.WW.Auth.md), three
@@ -261,6 +272,25 @@ migration, MSSQL-verified). Phase 3 wires `Actor` from the Auth principal at aut
Build/test from `ZB.MOM.WW.Audit/`: `dotnet test`. Consumer matrix: all three apps consume the single Build/test from `ZB.MOM.WW.Audit/`: `dotnet test`. Consumer matrix: all three apps consume the single
`ZB.MOM.WW.Audit` package (OtOpcUa, MxAccessGateway, ScadaBridge — DEEP-adopted as the canonical record). `ZB.MOM.WW.Audit` package (OtOpcUa, MxAccessGateway, ScadaBridge — DEEP-adopted as the canonical record).
The Galaxy Repository component normalizes the **Galaxy object-hierarchy SQL browse + reusable gRPC service**
that was previously embedded in `mxaccessgw`. Shared = canonical `galaxy_repository.v1` proto (wire-compatible
with `mxaccessgw`'s existing contract so OtOpcUa's `GalaxyRepositoryClient` is unaffected), the SQL browse
provider (`HierarchySql` / `AttributesSql` validated reverse-engineered queries), in-memory hierarchy cache +
snapshot + deploy-poll refresh `BackgroundService`, `GalaxyHierarchyProjector`, and `AddZbGalaxyRepository` /
`MapZbGalaxyRepository` DI extension. Left per-consumer = section path, subtree auth filtering, and any
app-specific paging defaults.
The shared library is **built and lives in this repo** at [`ZB.MOM.WW.GalaxyRepository/`](ZB.MOM.WW.GalaxyRepository/)
(.NET 10; single package `ZB.MOM.WW.GalaxyRepository`; `dotnet pack` → 1 nupkg @ 0.1.0, **published to
the Gitea NuGet feed** `gitea.dohertylan.com/api/packages/dohertj2/nuget`). The design doc is at
[`docs/plans/2026-06-23-historian-gateway-design.md`](docs/plans/2026-06-23-historian-gateway-design.md) (§10, component 1).
**Consumed by HistorianGateway** as a `PackageReference` from that Gitea feed, pinned at `0.1.0` (originally a
cross-repo `ProjectReference` to this scadaproj tree; switched to the feed package 2026-06-24).
**mxaccessgw adoption is a tracked follow-on** — once adopted, mxaccessgw's inline Galaxy browse code is replaced
by the shared lib (the `galaxy_repository.v1` wire contract is already identical, so OtOpcUa and ScadaBridge
clients are unaffected). Build/test from `ZB.MOM.WW.GalaxyRepository/`: `dotnet test`.
Consumer matrix: HistorianGateway (initial); mxaccessgw (follow-on adoption).
## Per-project primary commands ## Per-project primary commands
Run these from inside each project directory (not from `scadaproj`). Run these from inside each project directory (not from `scadaproj`).
@@ -282,9 +312,17 @@ dotnet run --project src/MxGateway.Server/MxGateway.Server.csproj
dotnet build ZB.MOM.WW.ScadaBridge.slnx dotnet build ZB.MOM.WW.ScadaBridge.slnx
bash docker/deploy.sh # rebuild + redeploy the 8-node cluster bash docker/deploy.sh # rebuild + redeploy the 8-node cluster
cd infra && docker compose up -d # local test services (SQL, OPC UA, SMTP, REST, Traefik) — LDAP is NOT here cd infra && docker compose up -d # local test services (SQL, OPC UA, SMTP, REST, Traefik) — LDAP is NOT here
# HistorianGateway (~/Desktop/HistorianGateway)
dotnet build ZB.MOM.WW.HistorianGateway.slnx
dotnet test ZB.MOM.WW.HistorianGateway.slnx # unit + golden; live integration tests skip without env vars
dotnet run --project src/ZB.MOM.WW.HistorianGateway.Server/ZB.MOM.WW.HistorianGateway.Server.csproj
# dashboard on :5220, gRPC h2c on :5221
# Live integration (need HISTORIAN_GRPC_HOST + HISTORIAN_GRPC_WRITE_SANDBOX_TAG + GALAXY_SQL_CONNSTR set)
dotnet test ZB.MOM.WW.HistorianGateway.slnx --filter "Category=LiveIntegration"
``` ```
> **Shared GLAuth (all three apps):** LDAP auth for every local dev/test stack is provided by a > **Shared GLAuth (all three apps + HistorianGateway):** LDAP auth for every local dev/test stack is provided by a
> single `zb-shared-glauth` container on the Linux fixture host **`10.100.0.35:3893`** > single `zb-shared-glauth` container on the Linux fixture host **`10.100.0.35:3893`**
> (`baseDN dc=zb,dc=local`, Transport=None). Source of truth and deploy runbook: > (`baseDN dc=zb,dc=local`, Transport=None). Source of truth and deploy runbook:
> [`scadaproj/infra/glauth/`](infra/glauth/) (`config.toml` + `docker-compose.yml` + `README.md`). > [`scadaproj/infra/glauth/`](infra/glauth/) (`config.toml` + `docker-compose.yml` + `README.md`).
@@ -0,0 +1,11 @@
<Project>
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<LangVersion>latest</LangVersion>
<Version>0.1.0</Version>
<ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
</PropertyGroup>
</Project>
@@ -0,0 +1,24 @@
<Project>
<PropertyGroup>
<ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
</PropertyGroup>
<ItemGroup>
<!-- Library -->
<PackageVersion Include="Microsoft.Data.SqlClient" Version="6.0.2" />
<PackageVersion Include="Grpc.AspNetCore" Version="2.76.0" />
<!-- Google.Protobuf and Grpc.Tools must be >= the minimums Grpc.AspNetCore 2.76.0 requires -->
<PackageVersion Include="Google.Protobuf" Version="3.31.1" />
<PackageVersion Include="Microsoft.Extensions.Hosting.Abstractions" Version="10.0.0" />
<PackageVersion Include="Microsoft.Extensions.Options.ConfigurationExtensions" Version="10.0.0" />
<PackageVersion Include="Grpc.Tools" Version="2.76.0" />
<!-- Test -->
<PackageVersion Include="xunit" Version="2.9.3" />
<PackageVersion Include="xunit.runner.visualstudio" Version="3.1.4" />
<PackageVersion Include="Microsoft.NET.Test.Sdk" Version="17.14.1" />
<PackageVersion Include="coverlet.collector" Version="6.0.4" />
</ItemGroup>
</Project>
@@ -0,0 +1,8 @@
<Solution>
<Folder Name="/src/">
<Project Path="src/ZB.MOM.WW.GalaxyRepository/ZB.MOM.WW.GalaxyRepository.csproj" />
</Folder>
<Folder Name="/tests/">
<Project Path="tests/ZB.MOM.WW.GalaxyRepository.Tests/ZB.MOM.WW.GalaxyRepository.Tests.csproj" />
</Folder>
</Solution>
@@ -0,0 +1,71 @@
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Routing;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Options;
using ZB.MOM.WW.GalaxyRepository.Grpc;
namespace ZB.MOM.WW.GalaxyRepository.DependencyInjection;
/// <summary>
/// Dependency-injection and endpoint-routing extensions that register the reusable
/// Galaxy Repository services and map the canonical gRPC service. A consuming gateway
/// calls <see cref="AddZbGalaxyRepository"/> during service registration and
/// <see cref="MapZbGalaxyRepository"/> while building its endpoint pipeline.
/// </summary>
public static class GalaxyRepositoryServiceCollectionExtensions
{
/// <summary>
/// Registers the Galaxy Repository SQL provider, shared hierarchy cache, deploy
/// notifier, on-disk snapshot store, and the background refresh service, binding
/// <see cref="GalaxyRepositoryOptions"/> from the supplied configuration section.
/// </summary>
/// <param name="services">The service collection to add registrations to.</param>
/// <param name="configuration">The application configuration root.</param>
/// <param name="sectionPath">
/// The configuration section path to bind <see cref="GalaxyRepositoryOptions"/> from
/// (for example <c>MxGateway:Galaxy</c> or <c>HistorianGateway:Galaxy</c>).
/// </param>
/// <returns>The service collection for chaining.</returns>
public static IServiceCollection AddZbGalaxyRepository(
this IServiceCollection services,
IConfiguration configuration,
string sectionPath)
{
ArgumentNullException.ThrowIfNull(services);
ArgumentNullException.ThrowIfNull(configuration);
ArgumentException.ThrowIfNullOrWhiteSpace(sectionPath);
// Bind only — this shared lib ships no validator, so a .ValidateOnStart() here
// would be a silent no-op. The consuming application owns option validation
// (e.g. the sidecar's ConfigPreflight / validated-options layer).
services
.AddOptions<GalaxyRepositoryOptions>()
.Bind(configuration.GetSection(sectionPath));
services.AddSingleton(sp =>
new GalaxyRepository(sp.GetRequiredService<IOptions<GalaxyRepositoryOptions>>().Value));
services.AddSingleton<IGalaxyRepository>(sp => sp.GetRequiredService<GalaxyRepository>());
services.AddSingleton<IGalaxyDeployNotifier, GalaxyDeployNotifier>();
services.AddSingleton<IGalaxyHierarchySnapshotStore, GalaxyHierarchySnapshotStore>();
services.AddSingleton<IGalaxyHierarchyCache, GalaxyHierarchyCache>();
services.AddHostedService<GalaxyHierarchyRefreshService>();
return services;
}
/// <summary>
/// Maps the canonical <see cref="GalaxyRepositoryGrpcService"/> onto the consuming
/// application's endpoint pipeline. Call after <see cref="AddZbGalaxyRepository"/> and
/// after gRPC has been added to the application's services.
/// </summary>
/// <param name="endpoints">The endpoint route builder to map the gRPC service onto.</param>
/// <returns>The endpoint route builder for chaining.</returns>
public static IEndpointRouteBuilder MapZbGalaxyRepository(this IEndpointRouteBuilder endpoints)
{
ArgumentNullException.ThrowIfNull(endpoints);
endpoints.MapGrpcService<GalaxyRepositoryGrpcService>();
return endpoints;
}
}
@@ -0,0 +1,41 @@
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>One row from <see cref="GalaxyRepository.GetAttributesAsync"/>.</summary>
public sealed class GalaxyAttributeRow
{
/// <summary>Gets the Galaxy object identifier.</summary>
public int GobjectId { get; init; }
/// <summary>Gets the tag name.</summary>
public string TagName { get; init; } = string.Empty;
/// <summary>Gets the attribute name.</summary>
public string AttributeName { get; init; } = string.Empty;
/// <summary>Gets the full tag reference.</summary>
public string FullTagReference { get; init; } = string.Empty;
/// <summary>Gets the MXAccess data type code.</summary>
public int MxDataType { get; init; }
/// <summary>Gets the data type name.</summary>
public string? DataTypeName { get; init; }
/// <summary>Gets a value indicating whether this is an array.</summary>
public bool IsArray { get; init; }
/// <summary>Gets the array dimension, if applicable.</summary>
public int? ArrayDimension { get; init; }
/// <summary>Gets the MXAccess attribute category code.</summary>
public int MxAttributeCategory { get; init; }
/// <summary>Gets the security classification code.</summary>
public int SecurityClassification { get; init; }
/// <summary>Gets a value indicating whether this is historized.</summary>
public bool IsHistorized { get; init; }
/// <summary>Gets a value indicating whether this is an alarm.</summary>
public bool IsAlarm { get; init; }
}
@@ -0,0 +1,19 @@
using ZB.MOM.WW.GalaxyRepository.Grpc;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// Result of one <see cref="GalaxyBrowseProjector.ProjectChildren"/> call. Holds a
/// materialized page of direct children for the requested parent, along with a
/// parallel-indexed <see cref="ChildHasChildren"/> hint and the total post-filter
/// sibling count for paging.
/// </summary>
/// <param name="Children">The page of direct children, sorted areas-first then by display name.</param>
/// <param name="ChildHasChildren">Parallel array indicating whether each child has at least one matching descendant under the same filter set.</param>
/// <param name="TotalChildCount">Total matching direct children of the parent (post-filter).</param>
/// <param name="FilterSignature">Stable signature of the filter and parent selector, used to bind page tokens.</param>
public sealed record GalaxyBrowseChildrenResult(
IReadOnlyList<GalaxyObject> Children,
IReadOnlyList<bool> ChildHasChildren,
int TotalChildCount,
string FilterSignature);
@@ -0,0 +1,281 @@
using System.Collections.Concurrent;
using System.Runtime.CompilerServices;
using System.Security.Cryptography;
using System.Text;
using Grpc.Core;
using ZB.MOM.WW.GalaxyRepository.Grpc;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// Projects one level of children of a parent object out of an immutable
/// <see cref="GalaxyHierarchyCacheEntry"/>. Pure and side-effect free. Memoizes the
/// filtered child list per cache-entry instance so repeated paging is an O(pageSize)
/// slice rather than an O(siblings) filter scan per page. The memo is keyed on the
/// immutable cache entry, so when the cache publishes a new entry the stale memo
/// becomes unreachable and is reclaimed with it.
/// </summary>
public static class GalaxyBrowseProjector
{
private static readonly ConditionalWeakTable<
GalaxyHierarchyCacheEntry,
ConcurrentDictionary<string, FilteredChildren>> FilteredChildrenCache = new();
/// <summary>Projects one page of direct children of the resolved parent.</summary>
/// <param name="entry">The Galaxy hierarchy cache entry to query.</param>
/// <param name="request">The browse-children request.</param>
/// <param name="browseSubtreeGlobs">Optional API-key browse-subtree constraints.</param>
/// <param name="offset">Zero-based offset into the filtered child list.</param>
/// <param name="pageSize">Maximum number of children to return.</param>
public static GalaxyBrowseChildrenResult ProjectChildren(
GalaxyHierarchyCacheEntry entry,
BrowseChildrenRequest request,
IReadOnlyList<string>? browseSubtreeGlobs,
int offset,
int pageSize)
{
ArgumentNullException.ThrowIfNull(entry);
ArgumentNullException.ThrowIfNull(request);
if (offset < 0)
{
throw new ArgumentOutOfRangeException(nameof(offset), offset, "Offset must be greater than or equal to zero.");
}
if (pageSize <= 0)
{
throw new ArgumentOutOfRangeException(nameof(pageSize), pageSize, "Page size must be greater than zero.");
}
int parentId = ResolveParentId(entry, request);
string filterSignature = ComputeFilterSignature(request, browseSubtreeGlobs, parentId);
FilteredChildren filtered = GetFilteredChildren(entry, request, browseSubtreeGlobs, parentId, filterSignature);
bool includeAttributes = IncludeAttributes(request);
int end = (int)Math.Min((long)offset + pageSize, filtered.Children.Count);
List<GalaxyObject> page = new(Math.Max(0, end - offset));
List<bool> hasChildren = new(Math.Max(0, end - offset));
for (int index = offset; index < end; index++)
{
page.Add(CloneObject(filtered.Children[index].Object, includeAttributes));
hasChildren.Add(filtered.HasMatchingDescendant[index]);
}
return new GalaxyBrowseChildrenResult(page, hasChildren, filtered.Children.Count, filterSignature);
}
/// <summary>
/// Resolves the request's parent oneof to a gobject id, throwing
/// <see cref="RpcException"/> with <see cref="StatusCode.NotFound"/> when the
/// parent does not exist. Public so the gRPC handler can compute the same
/// parent id (needed for the page-token signature) without reimplementing the
/// resolution rules.
/// </summary>
/// <param name="entry">The Galaxy hierarchy cache entry to query.</param>
/// <param name="request">The browse-children request.</param>
public static int ResolveParentId(GalaxyHierarchyCacheEntry entry, BrowseChildrenRequest request)
{
switch (request.ParentCase)
{
case BrowseChildrenRequest.ParentOneofCase.None:
return 0;
case BrowseChildrenRequest.ParentOneofCase.ParentGobjectId:
if (request.ParentGobjectId == 0)
{
return 0;
}
if (!entry.Index.ObjectViewsById.ContainsKey(request.ParentGobjectId))
{
throw new RpcException(new Status(StatusCode.NotFound, "BrowseChildren parent was not found."));
}
return request.ParentGobjectId;
case BrowseChildrenRequest.ParentOneofCase.ParentTagName:
{
if (!entry.Index.ObjectViewsByTagName.TryGetValue(request.ParentTagName, out GalaxyObjectView? match))
{
throw new RpcException(new Status(StatusCode.NotFound, "BrowseChildren parent was not found."));
}
return match.Object.GobjectId;
}
case BrowseChildrenRequest.ParentOneofCase.ParentContainedPath:
{
if (!entry.Index.ObjectViewsByContainedPath.TryGetValue(request.ParentContainedPath, out GalaxyObjectView? match))
{
throw new RpcException(new Status(StatusCode.NotFound, "BrowseChildren parent was not found."));
}
return match.Object.GobjectId;
}
default:
return 0;
}
}
private static FilteredChildren GetFilteredChildren(
GalaxyHierarchyCacheEntry entry,
BrowseChildrenRequest request,
IReadOnlyList<string>? browseSubtreeGlobs,
int parentId,
string filterSignature)
{
ConcurrentDictionary<string, FilteredChildren> memo =
FilteredChildrenCache.GetValue(entry, static _ => new ConcurrentDictionary<string, FilteredChildren>(StringComparer.Ordinal));
return memo.GetOrAdd(
filterSignature,
static (_, state) =>
{
IReadOnlyDictionary<int, IReadOnlyList<GalaxyObjectView>> map = state.Entry.Index.ChildrenByParent;
IReadOnlyList<GalaxyObjectView> directChildren = map.TryGetValue(state.ParentId, out IReadOnlyList<GalaxyObjectView>? list)
? list
: Array.Empty<GalaxyObjectView>();
List<GalaxyObjectView> matched = [];
List<bool> hasMatching = [];
foreach (GalaxyObjectView view in directChildren)
{
if (!MatchesBrowseSubtrees(view, state.BrowseSubtreeGlobs))
{
continue;
}
if (!MatchesFilters(view.Object, state.Request))
{
// Even if the direct child itself fails the filter, a matching
// descendant should still surface its ancestor — but only when
// there is one. Mirror the dashboard browse-tree semantics: if a
// descendant matches, include the parent with has-children true.
if (HasMatchingDescendant(view, state.Entry.Index, state.Request, state.BrowseSubtreeGlobs))
{
matched.Add(view);
hasMatching.Add(true);
}
continue;
}
matched.Add(view);
hasMatching.Add(HasMatchingDescendant(view, state.Entry.Index, state.Request, state.BrowseSubtreeGlobs));
}
return new FilteredChildren(matched, hasMatching);
},
(Entry: entry, ParentId: parentId, Request: request, BrowseSubtreeGlobs: browseSubtreeGlobs));
}
private static bool HasMatchingDescendant(
GalaxyObjectView parent,
GalaxyHierarchyIndex index,
BrowseChildrenRequest request,
IReadOnlyList<string>? browseSubtreeGlobs)
{
if (!index.ChildrenByParent.TryGetValue(parent.Object.GobjectId, out IReadOnlyList<GalaxyObjectView>? children))
{
return false;
}
// Defend against pathological cycles in Galaxy data (e.g. a corrupt A→B→A chain).
// BuildContainedPath uses the same visited-id pattern; mirror it so this walk
// terminates even when ChildrenByParent forms a cycle.
HashSet<int> visited = new() { parent.Object.GobjectId };
Stack<GalaxyObjectView> stack = new();
foreach (GalaxyObjectView child in children)
{
if (visited.Add(child.Object.GobjectId))
{
stack.Push(child);
}
}
while (stack.Count > 0)
{
GalaxyObjectView candidate = stack.Pop();
if (MatchesBrowseSubtrees(candidate, browseSubtreeGlobs)
&& MatchesFilters(candidate.Object, request))
{
return true;
}
if (index.ChildrenByParent.TryGetValue(candidate.Object.GobjectId, out IReadOnlyList<GalaxyObjectView>? grandchildren))
{
foreach (GalaxyObjectView grandchild in grandchildren)
{
if (visited.Add(grandchild.Object.GobjectId))
{
stack.Push(grandchild);
}
}
}
}
return false;
}
private static bool MatchesBrowseSubtrees(GalaxyObjectView view, IReadOnlyList<string>? browseSubtreeGlobs)
{
return browseSubtreeGlobs is null
|| browseSubtreeGlobs.Count == 0
|| browseSubtreeGlobs.Any(glob => GalaxyGlobMatcher.IsMatch(view.ContainedPath, glob));
}
private static bool MatchesFilters(GalaxyObject obj, BrowseChildrenRequest request)
{
if (request.CategoryIds.Count > 0 && !request.CategoryIds.Contains(obj.CategoryId))
{
return false;
}
foreach (string templateFilter in request.TemplateChainContains)
{
if (!obj.TemplateChain.Any(template => template.Contains(templateFilter, StringComparison.OrdinalIgnoreCase)))
{
return false;
}
}
if (!string.IsNullOrWhiteSpace(request.TagNameGlob)
&& !GalaxyGlobMatcher.IsMatch(obj.TagName, request.TagNameGlob))
{
return false;
}
if (request.AlarmBearingOnly && !obj.Attributes.Any(attribute => attribute.IsAlarm))
{
return false;
}
if (request.HistorizedOnly && !obj.Attributes.Any(attribute => attribute.IsHistorized))
{
return false;
}
return true;
}
private static bool IncludeAttributes(BrowseChildrenRequest request)
{
return !request.HasIncludeAttributes || request.IncludeAttributes;
}
private static GalaxyObject CloneObject(GalaxyObject source, bool includeAttributes)
{
GalaxyObject clone = source.Clone();
if (!includeAttributes)
{
clone.Attributes.Clear();
}
return clone;
}
/// <summary>Computes a stable filter signature for memoization purposes.</summary>
/// <param name="request">The browse-children request.</param>
/// <param name="browseSubtreeGlobs">Optional API-key browse-subtree constraints.</param>
/// <param name="parentId">Resolved parent gobject id (0 for roots).</param>
public static string ComputeFilterSignature(
BrowseChildrenRequest request,
IReadOnlyList<string>? browseSubtreeGlobs,
int parentId)
{
StringBuilder builder = new();
builder.Append("parent=").Append(parentId.ToString(System.Globalization.CultureInfo.InvariantCulture));
builder.Append("|cat=").AppendJoin(',', request.CategoryIds.Order());
builder.Append("|tpl=").AppendJoin(',', request.TemplateChainContains.Order(StringComparer.OrdinalIgnoreCase));
builder.Append("|glob=").Append(request.TagNameGlob);
builder.Append("|attrs=").Append(request.HasIncludeAttributes ? request.IncludeAttributes.ToString() : "unset");
builder.Append("|alarm=").Append(request.AlarmBearingOnly);
builder.Append("|hist=").Append(request.HistorizedOnly);
builder.Append("|browse=").AppendJoin(',', (browseSubtreeGlobs ?? Array.Empty<string>()).Order(StringComparer.OrdinalIgnoreCase));
byte[] hash = SHA256.HashData(Encoding.UTF8.GetBytes(builder.ToString()));
return Convert.ToHexString(hash, 0, 12);
}
private sealed record FilteredChildren(
IReadOnlyList<GalaxyObjectView> Children,
IReadOnlyList<bool> HasMatchingDescendant);
}
@@ -0,0 +1,18 @@
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>Freshness state of the shared Galaxy hierarchy cache entry.</summary>
public enum GalaxyCacheStatus
{
/// <summary>Cache has never completed a refresh.</summary>
Unknown = 0,
/// <summary>Cache holds data from a recent successful refresh.</summary>
Healthy = 1,
/// <summary>Cache holds data, but the most recent refresh attempt failed
/// or no successful refresh has happened within the staleness threshold.</summary>
Stale = 2,
/// <summary>Latest refresh failed and no prior data is available.</summary>
Unavailable = 3,
}
@@ -0,0 +1,19 @@
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// A single Galaxy deploy notification. Published by <see cref="GalaxyHierarchyCache"/>
/// whenever a refresh detects that <c>galaxy.time_of_last_deploy</c> has changed (or on
/// the first successful refresh). Consumed by <see cref="IGalaxyDeployNotifier"/>
/// subscribers (the streaming gRPC RPC).
/// </summary>
/// <param name="Sequence">Monotonically increasing per process start; gaps indicate dropped events.</param>
/// <param name="ObservedAt">Server wall-clock when the cache observed the deploy.</param>
/// <param name="TimeOfLastDeploy">The <c>galaxy.time_of_last_deploy</c> value, or <see langword="null"/> when the Galaxy table reports none.</param>
/// <param name="ObjectCount">Number of objects in the hierarchy at the time of the event.</param>
/// <param name="AttributeCount">Number of attributes in the hierarchy at the time of the event.</param>
public sealed record GalaxyDeployEventInfo(
long Sequence,
DateTimeOffset ObservedAt,
DateTimeOffset? TimeOfLastDeploy,
int ObjectCount,
int AttributeCount);
@@ -0,0 +1,79 @@
using System.Collections.Concurrent;
using System.Runtime.CompilerServices;
using System.Threading.Channels;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// Channel-based fan-out of Galaxy deploy events to streaming gRPC subscribers. Each
/// subscriber gets a private bounded channel so a slow client cannot back-pressure
/// other subscribers or the publisher. When a subscriber's channel is full the oldest
/// event is dropped — clients use the sequence field to detect gaps.
/// </summary>
public sealed class GalaxyDeployNotifier : IGalaxyDeployNotifier
{
private const int SubscriberQueueCapacity = 16;
private readonly ConcurrentDictionary<Guid, Channel<GalaxyDeployEventInfo>> _subscribers = new();
private GalaxyDeployEventInfo? _latest;
/// <summary>
/// The most recent deploy event, or null if none has been published.
/// </summary>
public GalaxyDeployEventInfo? Latest => Volatile.Read(ref _latest);
/// <inheritdoc />
public void Publish(GalaxyDeployEventInfo info)
{
ArgumentNullException.ThrowIfNull(info);
Volatile.Write(ref _latest, info);
foreach (Channel<GalaxyDeployEventInfo> channel in _subscribers.Values)
{
// BoundedChannelFullMode.DropOldest -> writes never wait; we only fail if the
// channel was completed by the subscriber side, which we ignore.
channel.Writer.TryWrite(info);
}
}
/// <inheritdoc />
public async IAsyncEnumerable<GalaxyDeployEventInfo> SubscribeAsync(
[EnumeratorCancellation] CancellationToken cancellationToken)
{
Guid subscriberId = Guid.NewGuid();
Channel<GalaxyDeployEventInfo> channel = Channel.CreateBounded<GalaxyDeployEventInfo>(
new BoundedChannelOptions(SubscriberQueueCapacity)
{
FullMode = BoundedChannelFullMode.DropOldest,
SingleReader = true,
SingleWriter = false,
});
_subscribers[subscriberId] = channel;
// Bootstrap: emit the latest known event so subscribers don't need to wait for
// the next deploy to know current state.
GalaxyDeployEventInfo? bootstrap = Volatile.Read(ref _latest);
if (bootstrap is not null)
{
channel.Writer.TryWrite(bootstrap);
}
try
{
while (await channel.Reader.WaitToReadAsync(cancellationToken).ConfigureAwait(false))
{
while (channel.Reader.TryRead(out GalaxyDeployEventInfo? next))
{
yield return next;
}
}
}
finally
{
_subscribers.TryRemove(subscriberId, out _);
channel.Writer.TryComplete();
}
}
}
@@ -0,0 +1,131 @@
using System.Collections.Concurrent;
using System.Text;
using System.Text.RegularExpressions;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// Anchored, case-insensitive glob matcher (<c>*</c> and <c>?</c> wildcards) used by the
/// hierarchy and browse projectors to filter object tag names and browse subtrees.
/// Compiled regexes are cached and the cache is bounded so an unbounded stream of distinct
/// client-supplied globs cannot grow memory without limit.
/// </summary>
public static class GalaxyGlobMatcher
{
/// <summary>
/// Maximum number of compiled-regex entries retained in <see cref="RegexCache"/>.
/// The cache is keyed by glob pattern and patterns flow in from two sources:
/// admin-controlled API-key constraints (naturally bounded) and the
/// client-supplied <c>DiscoverHierarchyRequest.TagNameGlob</c> (unbounded — a
/// client can iterate through generated names and create millions of distinct
/// globs over the process lifetime). Capping the cache bounds memory while
/// keeping the hot working set hit-cached.
/// </summary>
internal const int RegexCacheCapacity = 256;
/// <summary>
/// Bounded compiled-regex cache keyed by glob pattern. <c>IsMatch</c> is called
/// once per object per <c>DiscoverHierarchy</c>/<c>WatchDeployEvents</c>
/// evaluation, so the same handful of glob patterns are translated
/// repeatedly; caching avoids rebuilding and recompiling the regex on every
/// call. Beyond <see cref="RegexCacheCapacity"/> entries the oldest insertion
/// is evicted so a client cannot grow the cache without bound by submitting
/// unique patterns. Eviction is approximate (FIFO over insertion order, not
/// true LRU) because we only need the bound, not exact recency tracking.
/// </summary>
private static readonly ConcurrentDictionary<string, Regex> RegexCache = new(StringComparer.Ordinal);
/// <summary>
/// Insertion-order queue used to evict the oldest cache entry when the cache
/// exceeds <see cref="RegexCacheCapacity"/>. A separate queue keeps the
/// <see cref="RegexCache"/> reads lock-free; the lock below only guards the
/// eviction path.
/// </summary>
private static readonly ConcurrentQueue<string> InsertionOrder = new();
private static readonly object EvictionLock = new();
/// <summary>
/// Current cache size, exposed for tests asserting the cap is honoured.
/// </summary>
internal static int CurrentCacheSize => RegexCache.Count;
/// <summary>Determines whether a value matches a glob pattern (with * and ? wildcards).</summary>
/// <param name="value">The value to test against the glob pattern.</param>
/// <param name="glob">The glob pattern with * and ? wildcards.</param>
public static bool IsMatch(string value, string glob)
{
if (string.IsNullOrWhiteSpace(glob))
{
return true;
}
return GetOrCreateRegex(glob).IsMatch(value ?? string.Empty);
}
private static Regex GetOrCreateRegex(string glob)
{
if (RegexCache.TryGetValue(glob, out Regex? existing))
{
return existing;
}
Regex compiled = new(
BuildRegex(glob),
RegexOptions.CultureInvariant | RegexOptions.IgnoreCase | RegexOptions.Compiled,
TimeSpan.FromMilliseconds(100));
// GetOrAdd atomically returns whichever instance is in the cache after the
// call — either the locally-compiled regex (we won the race) or the regex
// another thread inserted (we lost). It also avoids the TryAdd-then-indexer
// pattern where the key could be evicted between the failed TryAdd and the
// indexer read, producing a KeyNotFoundException under contention near the cap.
Regex result = RegexCache.GetOrAdd(glob, compiled);
if (ReferenceEquals(result, compiled))
{
// We were the inserter — track for FIFO eviction and bound the cache.
InsertionOrder.Enqueue(glob);
EvictIfOverCapacity();
}
return result;
}
private static void EvictIfOverCapacity()
{
if (RegexCache.Count <= RegexCacheCapacity)
{
return;
}
// Serialize eviction so two threads do not race past the cap together.
lock (EvictionLock)
{
while (RegexCache.Count > RegexCacheCapacity && InsertionOrder.TryDequeue(out string? oldest))
{
RegexCache.TryRemove(oldest, out _);
}
}
}
private static string BuildRegex(string glob)
{
StringBuilder builder = new("^", glob.Length + 2);
foreach (char character in glob)
{
switch (character)
{
case '*':
builder.Append(".*");
break;
case '?':
builder.Append('.');
break;
default:
builder.Append(Regex.Escape(character.ToString()));
break;
}
}
builder.Append('$');
return builder.ToString();
}
}
@@ -0,0 +1,365 @@
using Microsoft.Extensions.Logging;
using ZB.MOM.WW.GalaxyRepository.Grpc;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// Server-side cache of Galaxy Repository browse data. All gRPC clients share the same
/// entry — the materialized object list is produced once per refresh and reused across
/// requests. Refreshes are deploy-time gated: every tick queries
/// <c>galaxy.time_of_last_deploy</c> (cheap), and the heavy hierarchy + attributes rowsets
/// are pulled only when that timestamp has advanced.
/// Each successful heavy refresh is persisted to disk through
/// <see cref="IGalaxyHierarchySnapshotStore"/>; the first refresh restores that
/// snapshot (as <see cref="GalaxyCacheStatus.Stale"/>) so clients can browse
/// last-known data when the Galaxy database is unreachable on a cold start.
/// </summary>
public sealed class GalaxyHierarchyCache : IGalaxyHierarchyCache, IDisposable
{
private static readonly TimeSpan StaleThreshold = TimeSpan.FromMinutes(5);
private readonly IGalaxyRepository _repository;
private readonly IGalaxyDeployNotifier _notifier;
private readonly IGalaxyHierarchySnapshotStore? _snapshotStore;
private readonly TimeProvider _timeProvider;
private readonly ILogger<GalaxyHierarchyCache>? _logger;
private readonly TaskCompletionSource _firstLoad = new(TaskCreationOptions.RunContinuationsAsynchronously);
private readonly SemaphoreSlim _refreshGate = new(1, 1);
private GalaxyHierarchyCacheEntry _current = GalaxyHierarchyCacheEntry.Empty;
private bool _restoreAttempted;
/// <summary>Initializes a new instance of the <see cref="GalaxyHierarchyCache"/> class.</summary>
/// <param name="repository">Galaxy Repository client for SQL queries.</param>
/// <param name="notifier">Galaxy deploy event notifier.</param>
/// <param name="timeProvider">Provider for current time; defaults to system time.</param>
/// <param name="logger">Optional logger for diagnostic output.</param>
/// <param name="snapshotStore">
/// Optional on-disk snapshot store. When supplied, the cache persists each
/// successful refresh and restores the last snapshot on first load.
/// </param>
public GalaxyHierarchyCache(
IGalaxyRepository repository,
IGalaxyDeployNotifier notifier,
TimeProvider? timeProvider = null,
ILogger<GalaxyHierarchyCache>? logger = null,
IGalaxyHierarchySnapshotStore? snapshotStore = null)
{
_repository = repository;
_notifier = notifier;
_timeProvider = timeProvider ?? TimeProvider.System;
_logger = logger;
_snapshotStore = snapshotStore;
}
/// <summary>Gets the current Galaxy hierarchy cache entry with projected status.</summary>
public GalaxyHierarchyCacheEntry Current
{
get
{
GalaxyHierarchyCacheEntry snapshot = Volatile.Read(ref _current);
GalaxyCacheStatus projected = ProjectStatus(snapshot);
return projected == snapshot.Status
? snapshot
: snapshot with { Status = projected };
}
}
/// <summary>Refreshes the Galaxy hierarchy cache if the deploy time has advanced.</summary>
/// <param name="cancellationToken">Token to cancel the asynchronous operation.</param>
/// <returns>Asynchronous task representing the refresh operation.</returns>
public async Task RefreshAsync(CancellationToken cancellationToken)
{
await _refreshGate.WaitAsync(cancellationToken).ConfigureAwait(false);
try
{
await RefreshCoreAsync(cancellationToken).ConfigureAwait(false);
}
finally
{
_refreshGate.Release();
}
}
/// <summary>Waits for the Galaxy hierarchy cache to complete its first load.</summary>
/// <param name="cancellationToken">Token to cancel the asynchronous operation.</param>
/// <returns>Asynchronous task representing the wait operation.</returns>
public Task WaitForFirstLoadAsync(CancellationToken cancellationToken)
{
return _firstLoad.Task.WaitAsync(cancellationToken);
}
/// <summary>
/// Disposes the refresh gate. As a DI singleton the cache is disposed once at host
/// shutdown, after the refresh <see cref="GalaxyHierarchyRefreshService"/> has stopped,
/// so no in-flight refresh can be holding the gate.
/// </summary>
public void Dispose()
{
_refreshGate.Dispose();
}
private async Task RefreshCoreAsync(CancellationToken cancellationToken)
{
// First refresh only: seed the cache from the on-disk snapshot before
// querying SQL, so a cold start with an unreachable Galaxy database can
// still serve last-known browse data. Runs under the refresh gate.
if (!_restoreAttempted)
{
_restoreAttempted = true;
await TryRestoreFromDiskAsync(cancellationToken).ConfigureAwait(false);
}
GalaxyHierarchyCacheEntry previous = Volatile.Read(ref _current);
DateTimeOffset queriedAt = _timeProvider.GetUtcNow();
try
{
DateTime? deployRaw = await _repository.GetLastDeployTimeAsync(cancellationToken).ConfigureAwait(false);
DateTimeOffset? deployTime = deployRaw.HasValue
? new DateTimeOffset(DateTime.SpecifyKind(deployRaw.Value, DateTimeKind.Utc))
: null;
bool hasPriorData = previous.HasData;
bool deployChanged = !hasPriorData || deployTime != previous.LastDeployTime;
if (!deployChanged)
{
// No deploy change — skip heavy queries; just bump LastSuccessAt.
GalaxyHierarchyCacheEntry refreshed = previous with
{
Status = GalaxyCacheStatus.Healthy,
LastQueriedAt = queriedAt,
LastSuccessAt = queriedAt,
LastError = null,
};
Volatile.Write(ref _current, refreshed);
_firstLoad.TrySetResult();
return;
}
Task<List<GalaxyHierarchyRow>> hierarchyTask = _repository.GetHierarchyAsync(cancellationToken);
Task<List<GalaxyAttributeRow>> attributesTask = _repository.GetAttributesAsync(cancellationToken);
await Task.WhenAll(hierarchyTask, attributesTask).ConfigureAwait(false);
List<GalaxyHierarchyRow> hierarchy = hierarchyTask.Result;
List<GalaxyAttributeRow> attributes = attributesTask.Result;
long nextSequence = previous.Sequence + 1;
GalaxyHierarchyCacheEntry next = BuildEntry(
status: GalaxyCacheStatus.Healthy,
sequence: nextSequence,
lastQueriedAt: queriedAt,
lastSuccessAt: queriedAt,
lastDeployTime: deployTime,
lastError: null,
hierarchy: hierarchy,
attributes: attributes);
Volatile.Write(ref _current, next);
_firstLoad.TrySetResult();
_notifier.Publish(new GalaxyDeployEventInfo(
Sequence: nextSequence,
ObservedAt: queriedAt,
TimeOfLastDeploy: deployTime,
ObjectCount: hierarchy.Count,
AttributeCount: attributes.Count));
await PersistSnapshotAsync(deployTime, queriedAt, hierarchy, attributes, cancellationToken).ConfigureAwait(false);
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
{
throw;
}
catch (Exception exception)
{
// Catch every non-cancellation failure — not just SqlException /
// InvalidOperationException. A TimeoutException or Win32Exception
// from connection establishment, or another DbException subtype,
// must still degrade gracefully to Stale/Unavailable and complete
// _firstLoad rather than escape and fault the refresh BackgroundService.
_logger?.LogWarning(exception, "Galaxy hierarchy cache refresh failed.");
GalaxyHierarchyCacheEntry failed = previous with
{
Status = previous.HasData ? GalaxyCacheStatus.Stale : GalaxyCacheStatus.Unavailable,
LastQueriedAt = queriedAt,
LastError = exception.Message,
};
Volatile.Write(ref _current, failed);
_firstLoad.TrySetResult();
}
}
/// <summary>
/// Materializes a complete <see cref="GalaxyHierarchyCacheEntry"/> from raw
/// hierarchy and attribute rowsets. Shared by the live refresh path and the
/// on-disk restore path so both produce an identical object list and index.
/// </summary>
private static GalaxyHierarchyCacheEntry BuildEntry(
GalaxyCacheStatus status,
long sequence,
DateTimeOffset? lastQueriedAt,
DateTimeOffset? lastSuccessAt,
DateTimeOffset? lastDeployTime,
string? lastError,
IReadOnlyList<GalaxyHierarchyRow> hierarchy,
IReadOnlyList<GalaxyAttributeRow> attributes)
{
IReadOnlyList<GalaxyObject> objects = BuildObjects(hierarchy, attributes);
GalaxyHierarchyIndex index = GalaxyHierarchyIndex.Build(objects);
int areaCount = hierarchy.Count(row => row.IsArea);
int historized = attributes.Count(row => row.IsHistorized);
int alarms = attributes.Count(row => row.IsAlarm);
return new GalaxyHierarchyCacheEntry(
Status: status,
Sequence: sequence,
LastQueriedAt: lastQueriedAt,
LastSuccessAt: lastSuccessAt,
LastDeployTime: lastDeployTime,
LastError: lastError,
Objects: objects,
Index: index,
ObjectCount: hierarchy.Count,
AreaCount: areaCount,
AttributeCount: attributes.Count,
HistorizedAttributeCount: historized,
AlarmAttributeCount: alarms);
}
/// <summary>
/// Seeds the cache from the on-disk snapshot when no live data has loaded yet.
/// The restored entry is marked <see cref="GalaxyCacheStatus.Stale"/> — it is
/// last-known data, not live. A later refresh that observes the same deploy
/// time promotes it to healthy; one that observes a newer deploy replaces it.
/// </summary>
private async Task TryRestoreFromDiskAsync(CancellationToken cancellationToken)
{
if (_snapshotStore is null)
{
return;
}
if (Volatile.Read(ref _current).HasData)
{
return;
}
GalaxyHierarchySnapshot? snapshot;
try
{
snapshot = await _snapshotStore.TryLoadAsync(cancellationToken).ConfigureAwait(false);
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
{
throw;
}
catch (Exception exception)
{
_logger?.LogWarning(exception, "Failed to restore the Galaxy hierarchy from the on-disk snapshot.");
return;
}
if (snapshot is null)
{
return;
}
long sequence = Volatile.Read(ref _current).Sequence + 1;
GalaxyHierarchyCacheEntry restored = BuildEntry(
status: GalaxyCacheStatus.Stale,
sequence: sequence,
lastQueriedAt: snapshot.SavedAt,
lastSuccessAt: snapshot.SavedAt,
lastDeployTime: snapshot.LastDeployTime,
lastError: null,
hierarchy: snapshot.Hierarchy,
attributes: snapshot.Attributes);
Volatile.Write(ref _current, restored);
// Restored data is a valid completed first load: unblock callers waiting on
// the bootstrap gate immediately, rather than making them wait out the full
// wait budget for a live query that — when the database is unreachable, the
// scenario this restore exists for — may not return for seconds.
_firstLoad.TrySetResult();
_notifier.Publish(new GalaxyDeployEventInfo(
Sequence: sequence,
ObservedAt: _timeProvider.GetUtcNow(),
TimeOfLastDeploy: snapshot.LastDeployTime,
ObjectCount: snapshot.Hierarchy.Count,
AttributeCount: snapshot.Attributes.Count));
_logger?.LogInformation(
"Restored Galaxy hierarchy from on-disk snapshot saved {SavedAt:o}: {ObjectCount} objects, {AttributeCount} attributes (status Stale until the Galaxy database confirms).",
snapshot.SavedAt,
snapshot.Hierarchy.Count,
snapshot.Attributes.Count);
}
/// <summary>
/// Persists a successful refresh to disk. Persistence failures are logged and
/// swallowed — a cache that cannot write its backup is still fully usable.
/// </summary>
private async Task PersistSnapshotAsync(
DateTimeOffset? deployTime,
DateTimeOffset savedAt,
IReadOnlyList<GalaxyHierarchyRow> hierarchy,
IReadOnlyList<GalaxyAttributeRow> attributes,
CancellationToken cancellationToken)
{
if (_snapshotStore is null)
{
return;
}
try
{
await _snapshotStore.SaveAsync(
new GalaxyHierarchySnapshot(deployTime, savedAt, hierarchy, attributes),
cancellationToken).ConfigureAwait(false);
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
{
// The refresh was cancelled (service shutdown) before the write finished.
// That is not a persistence failure — do not log it as a warning.
}
catch (Exception exception)
{
_logger?.LogWarning(exception, "Failed to persist the Galaxy hierarchy snapshot to disk.");
}
}
private static IReadOnlyList<GalaxyObject> BuildObjects(
IReadOnlyList<GalaxyHierarchyRow> hierarchy,
IReadOnlyList<GalaxyAttributeRow> attributes)
{
Dictionary<int, List<GalaxyAttributeRow>> attributesByGobjectId = attributes
.GroupBy(a => a.GobjectId)
.ToDictionary(g => g.Key, g => g.ToList());
List<GalaxyObject> objects = new(hierarchy.Count);
foreach (GalaxyHierarchyRow row in hierarchy)
{
objects.Add(GalaxyProtoMapper.MapObject(row, attributesByGobjectId));
}
return objects;
}
private GalaxyCacheStatus ProjectStatus(GalaxyHierarchyCacheEntry snapshot)
{
if (snapshot.Status is GalaxyCacheStatus.Unknown or GalaxyCacheStatus.Unavailable)
{
return snapshot.Status;
}
if (snapshot.LastSuccessAt is { } success
&& _timeProvider.GetUtcNow() - success > StaleThreshold)
{
return GalaxyCacheStatus.Stale;
}
return snapshot.Status;
}
}
@@ -0,0 +1,56 @@
using ZB.MOM.WW.GalaxyRepository.Grpc;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// Immutable snapshot of the Galaxy Repository browse data held by
/// <see cref="GalaxyHierarchyCache"/>. Multiple gRPC clients share the same
/// materialized object list and precomputed hierarchy index.
/// </summary>
/// <param name="Status">The cache freshness state at the time the entry was produced.</param>
/// <param name="Sequence">Monotonically increasing per process start; bumped on each heavy refresh.</param>
/// <param name="LastQueriedAt">UTC wall-clock of the most recent refresh attempt.</param>
/// <param name="LastSuccessAt">UTC wall-clock of the most recent successful refresh.</param>
/// <param name="LastDeployTime">The <c>galaxy.time_of_last_deploy</c> the data was pulled at.</param>
/// <param name="LastError">The most recent refresh error message, or <see langword="null"/>.</param>
/// <param name="Objects">The materialized Galaxy object list.</param>
/// <param name="Index">Precomputed lookup structures over <paramref name="Objects"/>.</param>
/// <param name="ObjectCount">Number of objects in the hierarchy.</param>
/// <param name="AreaCount">Number of area objects in the hierarchy.</param>
/// <param name="AttributeCount">Number of attributes across all objects.</param>
/// <param name="HistorizedAttributeCount">Number of historized attributes.</param>
/// <param name="AlarmAttributeCount">Number of alarm-bearing attributes.</param>
public sealed record GalaxyHierarchyCacheEntry(
GalaxyCacheStatus Status,
long Sequence,
DateTimeOffset? LastQueriedAt,
DateTimeOffset? LastSuccessAt,
DateTimeOffset? LastDeployTime,
string? LastError,
IReadOnlyList<GalaxyObject> Objects,
GalaxyHierarchyIndex Index,
int ObjectCount,
int AreaCount,
int AttributeCount,
int HistorizedAttributeCount,
int AlarmAttributeCount)
{
/// <summary>Gets an empty Galaxy hierarchy cache entry.</summary>
public static GalaxyHierarchyCacheEntry Empty { get; } = new(
Status: GalaxyCacheStatus.Unknown,
Sequence: 0,
LastQueriedAt: null,
LastSuccessAt: null,
LastDeployTime: null,
LastError: null,
Objects: Array.Empty<GalaxyObject>(),
Index: GalaxyHierarchyIndex.Empty,
ObjectCount: 0,
AreaCount: 0,
AttributeCount: 0,
HistorizedAttributeCount: 0,
AlarmAttributeCount: 0);
/// <summary>Gets a value indicating whether the cache entry contains usable data.</summary>
public bool HasData => Status is GalaxyCacheStatus.Healthy or GalaxyCacheStatus.Stale;
}
@@ -0,0 +1,206 @@
using ZB.MOM.WW.GalaxyRepository.Grpc;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// Precomputed lookup structures over a materialized Galaxy object list. Built once per
/// cache entry so browse/discover handlers can resolve roots/parents by id, tag name, or
/// contained path in O(1), enumerate direct children, and resolve tag addresses to objects
/// or attributes without rescanning the full object list.
/// </summary>
public sealed class GalaxyHierarchyIndex
{
private GalaxyHierarchyIndex(
IReadOnlyList<GalaxyObjectView> objectViews,
IReadOnlyDictionary<int, GalaxyObjectView> objectViewsById,
IReadOnlyDictionary<string, GalaxyTagLookup> tagsByAddress,
IReadOnlyDictionary<int, IReadOnlyList<GalaxyObjectView>> childrenByParent,
IReadOnlyDictionary<string, GalaxyObjectView> objectViewsByTagName,
IReadOnlyDictionary<string, GalaxyObjectView> objectViewsByContainedPath)
{
ObjectViews = objectViews;
ObjectViewsById = objectViewsById;
TagsByAddress = tagsByAddress;
ChildrenByParent = childrenByParent;
ObjectViewsByTagName = objectViewsByTagName;
ObjectViewsByContainedPath = objectViewsByContainedPath;
}
/// <summary>Gets an empty Galaxy hierarchy index.</summary>
public static GalaxyHierarchyIndex Empty { get; } = new(
Array.Empty<GalaxyObjectView>(),
new Dictionary<int, GalaxyObjectView>(),
new Dictionary<string, GalaxyTagLookup>(StringComparer.OrdinalIgnoreCase),
new Dictionary<int, IReadOnlyList<GalaxyObjectView>>(),
new Dictionary<string, GalaxyObjectView>(StringComparer.OrdinalIgnoreCase),
new Dictionary<string, GalaxyObjectView>(StringComparer.OrdinalIgnoreCase));
/// <summary>Gets the object views.</summary>
public IReadOnlyList<GalaxyObjectView> ObjectViews { get; }
/// <summary>Gets the object views indexed by gobject id.</summary>
public IReadOnlyDictionary<int, GalaxyObjectView> ObjectViewsById { get; }
/// <summary>Gets tags indexed by address.</summary>
public IReadOnlyDictionary<string, GalaxyTagLookup> TagsByAddress { get; }
/// <summary>Gets direct children grouped by parent gobject id. Root objects (no parent, or self-parented) live under key 0. Each list is sorted areas-first, then by display name (OrdinalIgnoreCase).</summary>
public IReadOnlyDictionary<int, IReadOnlyList<GalaxyObjectView>> ChildrenByParent { get; }
/// <summary>Gets object views indexed by <see cref="GalaxyObject.TagName"/> (OrdinalIgnoreCase). Lets browse/discover handlers resolve parents/roots by tag name in O(1) instead of scanning <see cref="ObjectViews"/>.</summary>
public IReadOnlyDictionary<string, GalaxyObjectView> ObjectViewsByTagName { get; }
/// <summary>Gets object views indexed by contained path (OrdinalIgnoreCase). Lets browse/discover handlers resolve parents/roots by path in O(1) instead of scanning <see cref="ObjectViews"/>.</summary>
public IReadOnlyDictionary<string, GalaxyObjectView> ObjectViewsByContainedPath { get; }
/// <summary>Builds a Galaxy hierarchy index from the given objects.</summary>
/// <param name="objects">The Galaxy objects to index.</param>
/// <returns>A new Galaxy hierarchy index.</returns>
public static GalaxyHierarchyIndex Build(IReadOnlyList<GalaxyObject> objects)
{
if (objects.Count == 0)
{
return Empty;
}
Dictionary<int, GalaxyObject> objectsById = new();
foreach (GalaxyObject obj in objects)
{
objectsById.TryAdd(obj.GobjectId, obj);
}
List<GalaxyObjectView> views = new(objects.Count);
Dictionary<int, GalaxyObjectView> viewsById = new();
Dictionary<string, GalaxyTagLookup> tagsByAddress = new(StringComparer.OrdinalIgnoreCase);
Dictionary<string, GalaxyObjectView> viewsByTagName = new(StringComparer.OrdinalIgnoreCase);
Dictionary<string, GalaxyObjectView> viewsByContainedPath = new(StringComparer.OrdinalIgnoreCase);
foreach (GalaxyObject obj in objects)
{
string path = BuildContainedPath(obj, objectsById);
int depth = string.IsNullOrWhiteSpace(path) ? 0 : path.Count(character => character == '/');
GalaxyObjectView view = new(obj, path, depth);
views.Add(view);
viewsById.TryAdd(obj.GobjectId, view);
if (!string.IsNullOrWhiteSpace(obj.TagName))
{
tagsByAddress.TryAdd(obj.TagName, new GalaxyTagLookup(obj, Attribute: null, path));
viewsByTagName.TryAdd(obj.TagName, view);
}
if (!string.IsNullOrWhiteSpace(path))
{
viewsByContainedPath.TryAdd(path, view);
}
foreach (GalaxyAttribute attribute in obj.Attributes)
{
if (!string.IsNullOrWhiteSpace(attribute.FullTagReference))
{
tagsByAddress.TryAdd(attribute.FullTagReference, new GalaxyTagLookup(obj, attribute, path));
}
}
}
Dictionary<int, List<GalaxyObjectView>> childrenByParent = new();
foreach (GalaxyObjectView view in views)
{
int parentKey = view.Object.ParentGobjectId;
// Treat self-parented (corrupt) rows as roots.
if (parentKey == view.Object.GobjectId)
{
parentKey = 0;
}
// Re-root orphans whose parent object is absent from the set (e.g. a deleted or
// never-loaded container area). Otherwise they bucket under a phantom parent id
// that is never reached from the root, so they vanish from browse entirely.
else if (parentKey != 0 && !objectsById.ContainsKey(parentKey))
{
parentKey = 0;
}
if (!childrenByParent.TryGetValue(parentKey, out List<GalaxyObjectView>? bucket))
{
bucket = [];
childrenByParent[parentKey] = bucket;
}
bucket.Add(view);
}
foreach (List<GalaxyObjectView> bucket in childrenByParent.Values)
{
bucket.Sort(CompareByAreaThenDisplayName);
}
Dictionary<int, IReadOnlyList<GalaxyObjectView>> readOnlyChildren = new(childrenByParent.Count);
foreach (KeyValuePair<int, List<GalaxyObjectView>> kvp in childrenByParent)
{
readOnlyChildren[kvp.Key] = kvp.Value;
}
return new GalaxyHierarchyIndex(
views,
viewsById,
tagsByAddress,
readOnlyChildren,
viewsByTagName,
viewsByContainedPath);
}
private static string BuildContainedPath(
GalaxyObject obj,
IReadOnlyDictionary<int, GalaxyObject> objectsById)
{
Stack<string> names = new();
HashSet<int> seen = [];
GalaxyObject? current = obj;
while (current is not null && seen.Add(current.GobjectId))
{
names.Push(ResolvePathSegment(current));
current = current.ParentGobjectId != 0
&& objectsById.TryGetValue(current.ParentGobjectId, out GalaxyObject? parent)
? parent
: null;
}
return string.Join('/', names.Where(name => !string.IsNullOrWhiteSpace(name)));
}
private static string ResolvePathSegment(GalaxyObject obj)
{
if (!string.IsNullOrWhiteSpace(obj.ContainedName))
{
return obj.ContainedName;
}
if (!string.IsNullOrWhiteSpace(obj.BrowseName))
{
return obj.BrowseName;
}
return obj.TagName;
}
private static int CompareByAreaThenDisplayName(GalaxyObjectView left, GalaxyObjectView right)
{
if (left.Object.IsArea != right.Object.IsArea)
{
return left.Object.IsArea ? -1 : 1;
}
return string.Compare(DisplayNameOf(left), DisplayNameOf(right), StringComparison.OrdinalIgnoreCase);
}
private static string DisplayNameOf(GalaxyObjectView view)
{
GalaxyObject obj = view.Object;
if (!string.IsNullOrWhiteSpace(obj.BrowseName))
{
return obj.BrowseName;
}
if (!string.IsNullOrWhiteSpace(obj.ContainedName))
{
return obj.ContainedName;
}
return obj.TagName;
}
}
@@ -0,0 +1,317 @@
using System.Collections.Concurrent;
using System.Runtime.CompilerServices;
using System.Security.Cryptography;
using System.Text;
using Grpc.Core;
using ZB.MOM.WW.GalaxyRepository.Grpc;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// Projects a <c>DiscoverHierarchy</c> request against an immutable
/// <see cref="GalaxyHierarchyCacheEntry"/>: applies the root/depth/category/template/glob
/// filters, pages the result, and memoizes the filtered list per cache-entry instance so
/// paging is O(pageSize) rather than O(total) per page. Pure and side-effect free.
/// </summary>
public static class GalaxyHierarchyProjector
{
/// <summary>
/// Per-cache-entry memo of filtered, ordered <see cref="GalaxyObjectView"/> lists
/// keyed by filter signature. Without it, paging through a large hierarchy
/// re-applies every filter and re-scans the full <see cref="GalaxyHierarchyIndex.ObjectViews"/>
/// collection on every page — O(total) per page, O(total²/pageSize) end-to-end.
/// With it, the first page builds the filtered list and each subsequent page is an
/// O(pageSize) slice. The table is keyed on the immutable cache-entry instance, so
/// when the cache publishes a new entry the stale memo becomes unreachable and is
/// reclaimed with it — no explicit invalidation needed.
/// </summary>
private static readonly ConditionalWeakTable<GalaxyHierarchyCacheEntry, ConcurrentDictionary<string, IReadOnlyList<GalaxyObjectView>>> FilteredViewCache = new();
/// <summary>Projects a discovery request against a cache entry and returns all matching objects.</summary>
/// <param name="entry">The Galaxy hierarchy cache entry.</param>
/// <param name="request">The discovery hierarchy request.</param>
/// <param name="browseSubtreeGlobs">Optional glob patterns to filter browse subtrees.</param>
public static GalaxyHierarchyQueryResult Project(
GalaxyHierarchyCacheEntry entry,
DiscoverHierarchyRequest request,
IReadOnlyList<string>? browseSubtreeGlobs = null)
{
return Project(
entry,
request,
browseSubtreeGlobs,
offset: 0,
pageSize: int.MaxValue);
}
/// <summary>Projects a discovery request with paging against a cache entry and returns a page of matching objects.</summary>
/// <param name="entry">The Galaxy hierarchy cache entry.</param>
/// <param name="request">The discovery hierarchy request.</param>
/// <param name="browseSubtreeGlobs">Optional glob patterns to filter browse subtrees.</param>
/// <param name="offset">The zero-based offset into the result set.</param>
/// <param name="pageSize">The maximum number of results to return.</param>
public static GalaxyHierarchyQueryResult Project(
GalaxyHierarchyCacheEntry entry,
DiscoverHierarchyRequest request,
IReadOnlyList<string>? browseSubtreeGlobs,
int offset,
int pageSize)
{
ArgumentNullException.ThrowIfNull(entry);
ArgumentNullException.ThrowIfNull(request);
if (offset < 0)
{
throw new ArgumentOutOfRangeException(nameof(offset), offset, "Offset must be greater than or equal to zero.");
}
if (pageSize <= 0)
{
throw new ArgumentOutOfRangeException(nameof(pageSize), pageSize, "Page size must be greater than zero.");
}
int? maxDepth = request.MaxDepth;
if (maxDepth < 0)
{
throw new RpcException(new Status(
StatusCode.InvalidArgument,
"DiscoverHierarchy max_depth must be greater than or equal to zero when provided."));
}
string filterSignature = ComputeFilterSignature(request, browseSubtreeGlobs);
IReadOnlyList<GalaxyObjectView> matchedViews = GetFilteredViews(
entry,
request,
browseSubtreeGlobs,
maxDepth,
filterSignature);
bool includeAttributes = IncludeAttributes(request);
List<GalaxyObject> page = new(Math.Min(pageSize, Math.Max(0, matchedViews.Count - offset)));
int end = (int)Math.Min((long)offset + pageSize, matchedViews.Count);
for (int index = offset; index < end; index++)
{
page.Add(CloneObject(matchedViews[index].Object, includeAttributes));
}
return new GalaxyHierarchyQueryResult(
page,
matchedViews.Count,
filterSignature);
}
private static IReadOnlyList<GalaxyObjectView> GetFilteredViews(
GalaxyHierarchyCacheEntry entry,
DiscoverHierarchyRequest request,
IReadOnlyList<string>? browseSubtreeGlobs,
int? maxDepth,
string filterSignature)
{
// ResolveRoot can throw RpcException(NotFound); run it before consulting the
// memo so a bad root surfaces consistently regardless of cache state.
IReadOnlyList<GalaxyObjectView> views = entry.Index.ObjectViews;
GalaxyObjectView? root = ResolveRoot(request, entry.Index);
ConcurrentDictionary<string, IReadOnlyList<GalaxyObjectView>> memo =
FilteredViewCache.GetValue(entry, static _ => new ConcurrentDictionary<string, IReadOnlyList<GalaxyObjectView>>(StringComparer.Ordinal));
return memo.GetOrAdd(
filterSignature,
static (_, state) =>
{
List<GalaxyObjectView> matched = [];
foreach (GalaxyObjectView view in state.Views)
{
if (MatchesRoot(view, state.Root, state.MaxDepth)
&& MatchesBrowseSubtrees(view, state.BrowseSubtreeGlobs)
&& MatchesFilters(view.Object, state.Request))
{
matched.Add(view);
}
}
return matched;
},
(Views: views, Root: root, MaxDepth: maxDepth, BrowseSubtreeGlobs: browseSubtreeGlobs, Request: request));
}
/// <summary>Finds an object in the hierarchy by its tag address.</summary>
/// <param name="entry">The Galaxy hierarchy cache entry.</param>
/// <param name="tagAddress">The tag address to search for.</param>
public static GalaxyObject? FindObjectForTag(
GalaxyHierarchyCacheEntry entry,
string tagAddress)
{
if (string.IsNullOrWhiteSpace(tagAddress))
{
return null;
}
return entry.Index.TagsByAddress.TryGetValue(tagAddress, out GalaxyTagLookup? lookup)
? lookup.Object
: null;
}
/// <summary>Finds an attribute in the hierarchy by its tag address.</summary>
/// <param name="entry">The Galaxy hierarchy cache entry.</param>
/// <param name="tagAddress">The tag address to search for.</param>
public static GalaxyAttribute? FindAttributeForTag(
GalaxyHierarchyCacheEntry entry,
string tagAddress)
{
if (string.IsNullOrWhiteSpace(tagAddress))
{
return null;
}
return entry.Index.TagsByAddress.TryGetValue(tagAddress, out GalaxyTagLookup? lookup)
? lookup.Attribute
: null;
}
/// <summary>Gets the contained path for an object by its gobject ID.</summary>
/// <param name="entry">The Galaxy hierarchy cache entry.</param>
/// <param name="gobjectId">The Galaxy object ID.</param>
public static string GetContainedPath(
GalaxyHierarchyCacheEntry entry,
int gobjectId)
{
return entry.Index.ObjectViewsById.TryGetValue(gobjectId, out GalaxyObjectView? view)
? view.ContainedPath
: string.Empty;
}
private static GalaxyObjectView? ResolveRoot(
DiscoverHierarchyRequest request,
GalaxyHierarchyIndex index)
{
GalaxyObjectView? root = request.RootCase switch
{
DiscoverHierarchyRequest.RootOneofCase.None => null,
DiscoverHierarchyRequest.RootOneofCase.RootGobjectId =>
index.ObjectViewsById.TryGetValue(request.RootGobjectId, out GalaxyObjectView? byId) ? byId : null,
DiscoverHierarchyRequest.RootOneofCase.RootTagName =>
index.ObjectViewsByTagName.TryGetValue(request.RootTagName, out GalaxyObjectView? byTag) ? byTag : null,
DiscoverHierarchyRequest.RootOneofCase.RootContainedPath =>
index.ObjectViewsByContainedPath.TryGetValue(request.RootContainedPath, out GalaxyObjectView? byPath) ? byPath : null,
_ => null,
};
if (request.RootCase != DiscoverHierarchyRequest.RootOneofCase.None && root is null)
{
throw new RpcException(new Status(StatusCode.NotFound, "DiscoverHierarchy root was not found."));
}
return root;
}
private static bool MatchesRoot(
GalaxyObjectView view,
GalaxyObjectView? root,
int? maxDepth)
{
if (root is null)
{
return true;
}
bool isRoot = view.Object.GobjectId == root.Object.GobjectId;
bool isDescendant = view.ContainedPath.StartsWith(root.ContainedPath + "/", StringComparison.OrdinalIgnoreCase);
if (!isRoot && !isDescendant)
{
return false;
}
return maxDepth is null || view.Depth - root.Depth <= maxDepth.Value;
}
private static bool MatchesBrowseSubtrees(
GalaxyObjectView view,
IReadOnlyList<string>? browseSubtreeGlobs)
{
return browseSubtreeGlobs is null
|| browseSubtreeGlobs.Count == 0
|| browseSubtreeGlobs.Any(glob => GalaxyGlobMatcher.IsMatch(view.ContainedPath, glob));
}
private static bool MatchesFilters(
GalaxyObject obj,
DiscoverHierarchyRequest request)
{
if (request.CategoryIds.Count > 0 && !request.CategoryIds.Contains(obj.CategoryId))
{
return false;
}
foreach (string templateFilter in request.TemplateChainContains)
{
if (!obj.TemplateChain.Any(template => template.Contains(templateFilter, StringComparison.OrdinalIgnoreCase)))
{
return false;
}
}
if (!string.IsNullOrWhiteSpace(request.TagNameGlob)
&& !GalaxyGlobMatcher.IsMatch(obj.TagName, request.TagNameGlob))
{
return false;
}
if (request.AlarmBearingOnly && !obj.Attributes.Any(attribute => attribute.IsAlarm))
{
return false;
}
if (request.HistorizedOnly && !obj.Attributes.Any(attribute => attribute.IsHistorized))
{
return false;
}
return true;
}
private static bool IncludeAttributes(DiscoverHierarchyRequest request)
{
return !request.HasIncludeAttributes || request.IncludeAttributes;
}
private static GalaxyObject CloneObject(GalaxyObject source, bool includeAttributes)
{
GalaxyObject clone = source.Clone();
if (!includeAttributes)
{
clone.Attributes.Clear();
}
return clone;
}
/// <summary>Computes a stable filter signature for memoization purposes.</summary>
/// <param name="request">The discovery hierarchy request.</param>
/// <param name="browseSubtreeGlobs">Optional glob patterns to filter browse subtrees.</param>
public static string ComputeFilterSignature(
DiscoverHierarchyRequest request,
IReadOnlyList<string>? browseSubtreeGlobs)
{
StringBuilder builder = new();
builder.Append("root=").Append(request.RootCase).Append('|');
builder.Append(request.RootCase switch
{
DiscoverHierarchyRequest.RootOneofCase.RootGobjectId => request.RootGobjectId.ToString(
System.Globalization.CultureInfo.InvariantCulture),
DiscoverHierarchyRequest.RootOneofCase.RootTagName => request.RootTagName,
DiscoverHierarchyRequest.RootOneofCase.RootContainedPath => request.RootContainedPath,
_ => string.Empty,
});
builder.Append("|max=").Append(request.MaxDepth?.ToString(System.Globalization.CultureInfo.InvariantCulture) ?? "");
builder.Append("|cat=").AppendJoin(',', request.CategoryIds.Order());
builder.Append("|tpl=").AppendJoin(',', request.TemplateChainContains.Order(StringComparer.OrdinalIgnoreCase));
builder.Append("|glob=").Append(request.TagNameGlob);
builder.Append("|attrs=").Append(request.HasIncludeAttributes ? request.IncludeAttributes.ToString() : "unset");
builder.Append("|alarm=").Append(request.AlarmBearingOnly);
builder.Append("|hist=").Append(request.HistorizedOnly);
builder.Append("|browse=").AppendJoin(',', (browseSubtreeGlobs ?? Array.Empty<string>()).Order(StringComparer.OrdinalIgnoreCase));
byte[] hash = SHA256.HashData(Encoding.UTF8.GetBytes(builder.ToString()));
return Convert.ToHexString(hash, 0, 12);
}
}
@@ -0,0 +1,16 @@
using ZB.MOM.WW.GalaxyRepository.Grpc;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// Result of one <see cref="GalaxyHierarchyProjector.Project(GalaxyHierarchyCacheEntry, DiscoverHierarchyRequest, System.Collections.Generic.IReadOnlyList{string}, int, int)"/>
/// call: a materialized page of matching objects, the total post-filter object count, and
/// the stable filter signature used to bind page tokens.
/// </summary>
/// <param name="Objects">The page of matching objects.</param>
/// <param name="TotalObjectCount">Total matching objects across the whole hierarchy (post-filter).</param>
/// <param name="FilterSignature">Stable signature of the filter set, used to bind page tokens.</param>
public sealed record GalaxyHierarchyQueryResult(
IReadOnlyList<GalaxyObject> Objects,
int TotalObjectCount,
string FilterSignature);
@@ -0,0 +1,62 @@
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>Background service that periodically refreshes the Galaxy Repository hierarchy cache off the request path.</summary>
public sealed class GalaxyHierarchyRefreshService(
IGalaxyHierarchyCache cache,
IOptions<GalaxyRepositoryOptions> options,
ILogger<GalaxyHierarchyRefreshService> logger,
TimeProvider? timeProvider = null) : BackgroundService
{
private readonly TimeProvider _timeProvider = timeProvider ?? TimeProvider.System;
/// <inheritdoc />
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
TimeSpan interval = TimeSpan.FromSeconds(Math.Max(1, options.Value.DashboardRefreshIntervalSeconds));
try
{
await cache.RefreshAsync(stoppingToken).ConfigureAwait(false);
}
catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
{
return;
}
catch (Exception exception)
{
// A transient first-load failure (e.g. a TimeoutException or
// Win32Exception from connection establishment, or a DbException
// subtype the cache does not catch) must not fault this
// BackgroundService and stop the whole host. The cache records
// its own Unavailable/Stale status; the periodic tick below retries.
logger.LogWarning(exception, "Initial Galaxy hierarchy cache load failed; will retry on the refresh interval.");
}
using PeriodicTimer timer = new(interval, _timeProvider);
try
{
while (await timer.WaitForNextTickAsync(stoppingToken).ConfigureAwait(false))
{
try
{
await cache.RefreshAsync(stoppingToken).ConfigureAwait(false);
}
catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
{
return;
}
catch (Exception exception)
{
logger.LogWarning(exception, "Galaxy hierarchy cache refresh tick failed.");
}
}
}
catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
{
}
}
}
@@ -0,0 +1,35 @@
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// One row from <see cref="GalaxyRepository.GetHierarchyAsync"/>: a deployed Galaxy
/// <c>gobject</c> with its hierarchy parent and template-derivation chain.
/// </summary>
public sealed class GalaxyHierarchyRow
{
/// <summary>Gets the Galaxy object identifier.</summary>
public int GobjectId { get; init; }
/// <summary>Gets the tag name.</summary>
public string TagName { get; init; } = string.Empty;
/// <summary>Gets the contained name.</summary>
public string ContainedName { get; init; } = string.Empty;
/// <summary>Gets the browse name.</summary>
public string BrowseName { get; init; } = string.Empty;
/// <summary>Gets the parent Galaxy object identifier.</summary>
public int ParentGobjectId { get; init; }
/// <summary>Gets a value indicating whether this is an area.</summary>
public bool IsArea { get; init; }
/// <summary>Gets the category identifier.</summary>
public int CategoryId { get; init; }
/// <summary>Gets the Galaxy object identifier of the host.</summary>
public int HostedByGobjectId { get; init; }
/// <summary>Gets the template derivation chain.</summary>
public IReadOnlyList<string> TemplateChain { get; init; } = Array.Empty<string>();
}
@@ -0,0 +1,24 @@
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// A serializable point-in-time copy of the Galaxy Repository browse data.
/// Holds the raw hierarchy and attribute rowsets — not the materialized
/// protobuf objects — so the restore path runs the exact same
/// materialization as a live refresh. Persisted by
/// <see cref="IGalaxyHierarchySnapshotStore"/> after a successful refresh
/// and reloaded at startup when the Galaxy database is unreachable.
/// </summary>
/// <param name="LastDeployTime">
/// The <c>galaxy.time_of_last_deploy</c> the rowsets were pulled at, or
/// <see langword="null"/> when the Galaxy table reported no deploy. A later
/// live refresh that observes this same timestamp can promote the restored
/// entry to healthy without re-running the heavy queries.
/// </param>
/// <param name="SavedAt">UTC wall-clock when the snapshot was written to disk.</param>
/// <param name="Hierarchy">The persisted object-hierarchy rowset.</param>
/// <param name="Attributes">The persisted attribute rowset.</param>
public sealed record GalaxyHierarchySnapshot(
DateTimeOffset? LastDeployTime,
DateTimeOffset SavedAt,
IReadOnlyList<GalaxyHierarchyRow> Hierarchy,
IReadOnlyList<GalaxyAttributeRow> Attributes);
@@ -0,0 +1,152 @@
using System.Text.Json;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// JSON-file implementation of <see cref="IGalaxyHierarchySnapshotStore"/>.
/// Writes the on-disk snapshot atomically (temp file + rename) so a crash
/// mid-write can never leave a torn file, and ignores files whose schema
/// version it does not recognize. When
/// <see cref="GalaxyRepositoryOptions.PersistSnapshot"/> is <see langword="false"/>
/// — or <see cref="GalaxyRepositoryOptions.SnapshotCachePath"/> is empty —
/// both operations are no-ops. The snapshot path is fully consumer-supplied;
/// this store imposes no platform-specific default, so it is cross-platform.
/// </summary>
public sealed class GalaxyHierarchySnapshotStore : IGalaxyHierarchySnapshotStore, IDisposable
{
/// <summary>
/// On-disk format version. Bump this whenever the persisted shape changes
/// in a way an older or newer consumer cannot read; a mismatched file is
/// ignored rather than misparsed.
/// </summary>
private const int CurrentSchemaVersion = 1;
private static readonly JsonSerializerOptions SerializerOptions = new()
{
WriteIndented = false,
};
private readonly string? _path;
private readonly TimeSpan _writeTimeout;
private readonly ILogger<GalaxyHierarchySnapshotStore>? _logger;
private readonly SemaphoreSlim _ioGate = new(1, 1);
/// <summary>Initializes a new instance of the <see cref="GalaxyHierarchySnapshotStore"/> class.</summary>
/// <param name="options">Galaxy repository options carrying the snapshot path and enable flag.</param>
/// <param name="logger">Optional logger for diagnostic output.</param>
public GalaxyHierarchySnapshotStore(
IOptions<GalaxyRepositoryOptions> options,
ILogger<GalaxyHierarchySnapshotStore>? logger = null)
{
GalaxyRepositoryOptions value = options.Value;
_path = value.PersistSnapshot && !string.IsNullOrWhiteSpace(value.SnapshotCachePath)
? value.SnapshotCachePath
: null;
_writeTimeout = TimeSpan.FromSeconds(Math.Max(1, value.CommandTimeoutSeconds));
_logger = logger;
}
/// <inheritdoc />
public async Task SaveAsync(GalaxyHierarchySnapshot snapshot, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(snapshot);
if (_path is null)
{
return;
}
PersistedFile file = new(CurrentSchemaVersion, snapshot);
await _ioGate.WaitAsync(cancellationToken).ConfigureAwait(false);
try
{
// Bound the write so a stuck disk — e.g. a SnapshotCachePath on an
// unresponsive network share — cannot stall the caller. On the cache
// refresh path that would otherwise pin the whole refresh loop.
using CancellationTokenSource writeCts = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
writeCts.CancelAfter(_writeTimeout);
string? directory = Path.GetDirectoryName(_path);
if (!string.IsNullOrEmpty(directory))
{
Directory.CreateDirectory(directory);
}
string tempPath = _path + ".tmp";
await using (FileStream stream = new(tempPath, FileMode.Create, FileAccess.Write, FileShare.None))
{
await JsonSerializer.SerializeAsync(stream, file, SerializerOptions, writeCts.Token).ConfigureAwait(false);
}
File.Move(tempPath, _path, overwrite: true);
_logger?.LogDebug(
"Persisted Galaxy hierarchy snapshot to {Path} ({ObjectCount} objects, {AttributeCount} attributes).",
_path,
snapshot.Hierarchy.Count,
snapshot.Attributes.Count);
}
finally
{
_ioGate.Release();
}
}
/// <inheritdoc />
public async Task<GalaxyHierarchySnapshot?> TryLoadAsync(CancellationToken cancellationToken)
{
if (_path is null || !File.Exists(_path))
{
return null;
}
await _ioGate.WaitAsync(cancellationToken).ConfigureAwait(false);
try
{
PersistedFile? file;
await using (FileStream stream = new(_path, FileMode.Open, FileAccess.Read, FileShare.Read))
{
file = await JsonSerializer.DeserializeAsync<PersistedFile>(
stream, SerializerOptions, cancellationToken).ConfigureAwait(false);
}
if (file is null || file.SchemaVersion != CurrentSchemaVersion || file.Snapshot is null)
{
_logger?.LogWarning(
"Ignoring Galaxy hierarchy snapshot at {Path}: unrecognized or empty schema version.",
_path);
return null;
}
return file.Snapshot;
}
catch (Exception exception) when (exception is JsonException or IOException or UnauthorizedAccessException)
{
// A corrupt, truncated, locked, or access-denied snapshot file is an
// expected failure mode for a disk cache — honor the Try contract and
// return null rather than throwing.
_logger?.LogWarning(
exception,
"Ignoring Galaxy hierarchy snapshot at {Path}: the file is unreadable or not valid JSON.",
_path);
return null;
}
finally
{
_ioGate.Release();
}
}
/// <summary>
/// Disposes the I/O gate. As a DI singleton the store is disposed once at host
/// shutdown, by which point no save/load is in flight.
/// </summary>
public void Dispose()
{
_ioGate.Dispose();
}
/// <summary>On-disk envelope: a schema version plus the snapshot payload.</summary>
private sealed record PersistedFile(int SchemaVersion, GalaxyHierarchySnapshot? Snapshot);
}
@@ -0,0 +1,16 @@
using ZB.MOM.WW.GalaxyRepository.Grpc;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// A <see cref="GalaxyObject"/> paired with its computed contained path and hierarchy
/// depth. Materialized once per cache entry by <see cref="GalaxyHierarchyIndex"/> so
/// browse/discover projection can filter and page without recomputing paths.
/// </summary>
/// <param name="Object">The projected Galaxy object.</param>
/// <param name="ContainedPath">The slash-delimited contained path from the hierarchy root.</param>
/// <param name="Depth">The number of path segments from the root (zero for top-level objects).</param>
public sealed record GalaxyObjectView(
GalaxyObject Object,
string ContainedPath,
int Depth);
@@ -0,0 +1,76 @@
using ZB.MOM.WW.GalaxyRepository.Grpc;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// Maps <see cref="GalaxyHierarchyRow"/> + <see cref="GalaxyAttributeRow"/> rows produced
/// by <see cref="GalaxyRepository"/> into <c>galaxy_repository.v1</c> proto messages.
/// Pure function, separated so it can be unit-tested without a SQL connection.
/// </summary>
public static class GalaxyProtoMapper
{
/// <summary>Maps Galaxy hierarchy and attribute rows to Galaxy object protos.</summary>
/// <param name="hierarchy">Hierarchy rows from Galaxy Repository.</param>
/// <param name="attributes">Attribute rows from Galaxy Repository.</param>
public static IEnumerable<GalaxyObject> MapHierarchy(
IReadOnlyList<GalaxyHierarchyRow> hierarchy,
IReadOnlyList<GalaxyAttributeRow> attributes)
{
Dictionary<int, List<GalaxyAttributeRow>> attributesByGobjectId = attributes
.GroupBy(a => a.GobjectId)
.ToDictionary(g => g.Key, g => g.ToList());
foreach (GalaxyHierarchyRow row in hierarchy)
{
yield return MapObject(row, attributesByGobjectId);
}
}
/// <summary>Maps a Galaxy hierarchy row to a Galaxy object proto.</summary>
/// <param name="row">Hierarchy row from Galaxy Repository.</param>
/// <param name="attributesByGobjectId">Attributes indexed by gobject ID.</param>
public static GalaxyObject MapObject(
GalaxyHierarchyRow row,
IReadOnlyDictionary<int, List<GalaxyAttributeRow>> attributesByGobjectId)
{
GalaxyObject obj = new()
{
GobjectId = row.GobjectId,
TagName = row.TagName,
ContainedName = row.ContainedName,
BrowseName = row.BrowseName,
ParentGobjectId = row.ParentGobjectId,
IsArea = row.IsArea,
CategoryId = row.CategoryId,
HostedByGobjectId = row.HostedByGobjectId,
};
obj.TemplateChain.AddRange(row.TemplateChain);
if (attributesByGobjectId.TryGetValue(row.GobjectId, out List<GalaxyAttributeRow>? attrs))
{
foreach (GalaxyAttributeRow attr in attrs)
{
obj.Attributes.Add(MapAttribute(attr));
}
}
return obj;
}
/// <summary>Maps a Galaxy attribute row to a Galaxy attribute proto.</summary>
/// <param name="row">Attribute row from Galaxy Repository.</param>
public static GalaxyAttribute MapAttribute(GalaxyAttributeRow row) => new()
{
AttributeName = row.AttributeName,
FullTagReference = row.FullTagReference,
MxDataType = row.MxDataType,
DataTypeName = row.DataTypeName ?? string.Empty,
IsArray = row.IsArray,
ArrayDimension = row.ArrayDimension ?? 0,
ArrayDimensionPresent = row.ArrayDimension.HasValue,
MxAttributeCategory = row.MxAttributeCategory,
SecurityClassification = row.SecurityClassification,
IsHistorized = row.IsHistorized,
IsAlarm = row.IsAlarm,
};
}
@@ -0,0 +1,257 @@
using Microsoft.Data.SqlClient;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// SQL access to the AVEVA System Platform Galaxy Repository database.
/// <para>
/// <see cref="HierarchySql" /> is the query originally ported from the OtOpcUa
/// project. <see cref="AttributesSql" /> has diverged: it additionally enumerates the
/// built-in attributes contributed by each object's primitives (from
/// <c>attribute_definition</c> via <c>primitive_instance</c>), so engine/platform objects
/// and extension sub-attributes (e.g. <c>TestAlarm001.Acked</c>) are surfaced. The
/// OtOpcUa query is not kept in sync.
/// </para>
/// </summary>
public sealed class GalaxyRepository(GalaxyRepositoryOptions options) : IGalaxyRepository
{
/// <summary>Tests the connection to the Galaxy Repository database.</summary>
/// <param name="ct">Token to cancel the asynchronous operation.</param>
public async Task<bool> TestConnectionAsync(CancellationToken ct = default)
{
try
{
using SqlConnection conn = new(options.ConnectionString);
await conn.OpenAsync(ct).ConfigureAwait(false);
using SqlCommand cmd = new("SELECT 1", conn) { CommandTimeout = options.CommandTimeoutSeconds };
object? result = await cmd.ExecuteScalarAsync(ct).ConfigureAwait(false);
return result is int i && i == 1;
}
catch (SqlException) { return false; }
catch (InvalidOperationException) { return false; }
}
/// <summary>Retrieves the last deployment time from the Galaxy Repository.</summary>
/// <param name="ct">Token to cancel the asynchronous operation.</param>
public async Task<DateTime?> GetLastDeployTimeAsync(CancellationToken ct = default)
{
using SqlConnection conn = new(options.ConnectionString);
await conn.OpenAsync(ct).ConfigureAwait(false);
using SqlCommand cmd = new("SELECT time_of_last_deploy FROM galaxy", conn)
{ CommandTimeout = options.CommandTimeoutSeconds };
object? result = await cmd.ExecuteScalarAsync(ct).ConfigureAwait(false);
return result is DateTime dt ? dt : null;
}
/// <summary>Retrieves the complete hierarchy of Galaxy objects from the repository.</summary>
/// <param name="ct">Token to cancel the asynchronous operation.</param>
public async Task<List<GalaxyHierarchyRow>> GetHierarchyAsync(CancellationToken ct = default)
{
List<GalaxyHierarchyRow> rows = new();
using SqlConnection conn = new(options.ConnectionString);
await conn.OpenAsync(ct).ConfigureAwait(false);
using SqlCommand cmd = new(HierarchySql, conn) { CommandTimeout = options.CommandTimeoutSeconds };
using SqlDataReader reader = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false);
while (await reader.ReadAsync(ct).ConfigureAwait(false))
{
string templateChainRaw = reader.IsDBNull(8) ? string.Empty : reader.GetString(8);
string[] templateChain = templateChainRaw.Length == 0
? Array.Empty<string>()
: templateChainRaw.Split(['|'], StringSplitOptions.RemoveEmptyEntries)
.Select(s => s.Trim())
.Where(s => s.Length > 0)
.ToArray();
rows.Add(new GalaxyHierarchyRow
{
GobjectId = Convert.ToInt32(reader.GetValue(0)),
TagName = reader.GetString(1),
ContainedName = reader.IsDBNull(2) ? string.Empty : reader.GetString(2),
BrowseName = reader.GetString(3),
ParentGobjectId = Convert.ToInt32(reader.GetValue(4)),
IsArea = Convert.ToInt32(reader.GetValue(5)) == 1,
CategoryId = Convert.ToInt32(reader.GetValue(6)),
HostedByGobjectId = Convert.ToInt32(reader.GetValue(7)),
TemplateChain = templateChain,
});
}
return rows;
}
/// <summary>Retrieves all attributes for Galaxy objects from the repository.</summary>
/// <param name="ct">Token to cancel the asynchronous operation.</param>
public async Task<List<GalaxyAttributeRow>> GetAttributesAsync(CancellationToken ct = default)
{
List<GalaxyAttributeRow> rows = new();
using SqlConnection conn = new(options.ConnectionString);
await conn.OpenAsync(ct).ConfigureAwait(false);
using SqlCommand cmd = new(AttributesSql, conn) { CommandTimeout = options.CommandTimeoutSeconds };
using SqlDataReader reader = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false);
while (await reader.ReadAsync(ct).ConfigureAwait(false))
{
rows.Add(new GalaxyAttributeRow
{
GobjectId = Convert.ToInt32(reader.GetValue(0)),
TagName = reader.GetString(1),
AttributeName = reader.GetString(2),
FullTagReference = reader.GetString(3),
MxDataType = Convert.ToInt32(reader.GetValue(4)),
DataTypeName = reader.IsDBNull(5) ? null : reader.GetString(5),
IsArray = Convert.ToInt32(reader.GetValue(6)) == 1,
ArrayDimension = reader.IsDBNull(7) ? null : Convert.ToInt32(reader.GetValue(7)),
MxAttributeCategory = Convert.ToInt32(reader.GetValue(8)),
SecurityClassification = Convert.ToInt32(reader.GetValue(9)),
IsHistorized = Convert.ToInt32(reader.GetValue(10)) == 1,
IsAlarm = Convert.ToInt32(reader.GetValue(11)) == 1,
});
}
return rows;
}
// Area objects (category 13) are returned even when undeployed (deployed_package_id = 0):
// they are organizational/model nodes that group deployed objects, so excluding them
// orphans every area whose containing area is not itself deployed. All non-area objects
// still require deployment. Orphans left by a missing/deleted parent area are re-rooted
// by GalaxyHierarchyIndex.Build so nothing disappears from browse.
private const string HierarchySql = @"
;WITH template_chain AS (
SELECT g.gobject_id AS instance_gobject_id, t.gobject_id AS template_gobject_id,
t.tag_name AS template_tag_name, t.derived_from_gobject_id, 0 AS depth
FROM gobject g
INNER JOIN gobject t ON t.gobject_id = g.derived_from_gobject_id
WHERE g.is_template = 0 AND g.deployed_package_id <> 0 AND g.derived_from_gobject_id <> 0
UNION ALL
SELECT tc.instance_gobject_id, t.gobject_id, t.tag_name, t.derived_from_gobject_id, tc.depth + 1
FROM template_chain tc
INNER JOIN gobject t ON t.gobject_id = tc.derived_from_gobject_id
WHERE tc.derived_from_gobject_id <> 0 AND tc.depth < 10
)
SELECT DISTINCT
g.gobject_id,
g.tag_name,
g.contained_name,
CASE WHEN g.contained_name IS NULL OR g.contained_name = ''
THEN g.tag_name
ELSE g.contained_name
END AS browse_name,
CASE WHEN g.contained_by_gobject_id = 0
THEN g.area_gobject_id
ELSE g.contained_by_gobject_id
END AS parent_gobject_id,
CASE WHEN td.category_id = 13
THEN 1
ELSE 0
END AS is_area,
td.category_id AS category_id,
g.hosted_by_gobject_id AS hosted_by_gobject_id,
ISNULL(
STUFF((
SELECT '|' + tc.template_tag_name
FROM template_chain tc
WHERE tc.instance_gobject_id = g.gobject_id
ORDER BY tc.depth
FOR XML PATH('')
), 1, 1, ''),
''
) AS template_chain
FROM gobject g
INNER JOIN template_definition td
ON g.template_definition_id = td.template_definition_id
WHERE td.category_id IN (1, 3, 4, 10, 11, 13, 17, 24, 26)
AND g.is_template = 0
AND (g.deployed_package_id <> 0 OR td.category_id = 13)
ORDER BY parent_gobject_id, g.tag_name";
// Unlike HierarchySql, this query has diverged from the OtOpcUa original. It returns two
// kinds of attribute: user-configured dynamic attributes (the original `dynamic_attribute`
// body, src_pri 0) and the built-in attributes every object inherits from its primitives
// (`attribute_definition` joined through `primitive_instance`, src_pri 1). Built-in
// attributes are why engine/platform objects and extension sub-attributes such as
// `TestAlarm001.Acked` show up at all. Built-in rows carry no category filter (the
// `attribute_definition` category numbering differs from `dynamic_attribute`'s — only the
// `_`-prefix and `.Description` name exclusions apply) and are never flagged
// `is_historized`/`is_alarm`: those flags describe a user attribute that anchors an
// extension, not the extension's machinery leaves.
private const string AttributesSql = @"
;WITH deployed_package_chain AS (
SELECT g.gobject_id, p.package_id, p.derived_from_package_id, 0 AS depth
FROM gobject g
INNER JOIN package p ON p.package_id = g.deployed_package_id
WHERE g.is_template = 0 AND g.deployed_package_id <> 0
UNION ALL
SELECT dpc.gobject_id, p.package_id, p.derived_from_package_id, dpc.depth + 1
FROM deployed_package_chain dpc
INNER JOIN package p ON p.package_id = dpc.derived_from_package_id
WHERE dpc.derived_from_package_id <> 0 AND dpc.depth < 10
),
candidate AS (
SELECT
dpc.gobject_id, g.tag_name, da.attribute_name, da.mx_data_type, da.is_array,
CASE WHEN da.is_array = 1
THEN CONVERT(int, CONVERT(varbinary(2),
SUBSTRING(da.mx_value, 15, 2) + SUBSTRING(da.mx_value, 13, 2), 2))
ELSE NULL END AS array_dimension,
da.mx_attribute_category, da.security_classification, dpc.depth, 0 AS src_pri
FROM deployed_package_chain dpc
INNER JOIN dynamic_attribute da ON da.package_id = dpc.package_id
INNER JOIN gobject g ON g.gobject_id = dpc.gobject_id
INNER JOIN template_definition td ON td.template_definition_id = g.template_definition_id
WHERE td.category_id IN (1, 3, 4, 10, 11, 13, 17, 24, 26)
AND da.attribute_name NOT LIKE '[_]%'
AND da.attribute_name NOT LIKE '%.Description'
AND da.mx_attribute_category IN (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 24)
UNION ALL
SELECT
dpc.gobject_id, g.tag_name,
CASE WHEN pi.primitive_name IS NULL OR pi.primitive_name = ''
THEN ad.attribute_name
ELSE pi.primitive_name + '.' + ad.attribute_name END AS attribute_name,
ad.mx_data_type, ad.is_array,
CASE WHEN ad.is_array = 1
THEN CONVERT(int, CONVERT(varbinary(2),
SUBSTRING(ad.mx_value, 15, 2) + SUBSTRING(ad.mx_value, 13, 2), 2))
ELSE NULL END AS array_dimension,
ad.mx_attribute_category, ad.security_classification, dpc.depth, 1 AS src_pri
FROM deployed_package_chain dpc
INNER JOIN primitive_instance pi ON pi.package_id = dpc.package_id
INNER JOIN attribute_definition ad ON ad.primitive_definition_id = pi.primitive_definition_id
INNER JOIN gobject g ON g.gobject_id = dpc.gobject_id
INNER JOIN template_definition td ON td.template_definition_id = g.template_definition_id
WHERE td.category_id IN (1, 3, 4, 10, 11, 13, 17, 24, 26)
AND ad.attribute_name NOT LIKE '[_]%'
AND ad.attribute_name NOT LIKE '%.Description'
),
ranked AS (
SELECT c.*, ROW_NUMBER() OVER (
PARTITION BY c.gobject_id, c.attribute_name ORDER BY c.src_pri, c.depth) AS rn
FROM candidate c
)
SELECT
r.gobject_id, r.tag_name, r.attribute_name,
r.tag_name + '.' + r.attribute_name
+ CASE WHEN r.is_array = 1 THEN '[]' ELSE '' END AS full_tag_reference,
r.mx_data_type, dt.description AS data_type_name, r.is_array, r.array_dimension,
r.mx_attribute_category, r.security_classification,
CASE WHEN r.src_pri = 0 AND EXISTS (
SELECT 1 FROM deployed_package_chain dpc2
INNER JOIN primitive_instance pi ON pi.package_id = dpc2.package_id AND pi.primitive_name = r.attribute_name
INNER JOIN primitive_definition pd ON pd.primitive_definition_id = pi.primitive_definition_id AND pd.primitive_name = 'HistoryExtension'
WHERE dpc2.gobject_id = r.gobject_id
) THEN 1 ELSE 0 END AS is_historized,
CASE WHEN r.src_pri = 0 AND EXISTS (
SELECT 1 FROM deployed_package_chain dpc2
INNER JOIN primitive_instance pi ON pi.package_id = dpc2.package_id AND pi.primitive_name = r.attribute_name
INNER JOIN primitive_definition pd ON pd.primitive_definition_id = pi.primitive_definition_id AND pd.primitive_name = 'AlarmExtension'
WHERE dpc2.gobject_id = r.gobject_id
) THEN 1 ELSE 0 END AS is_alarm
FROM ranked r
LEFT JOIN data_type dt ON dt.mx_data_type = r.mx_data_type
WHERE r.rn = 1
ORDER BY r.tag_name, r.attribute_name";
}
@@ -0,0 +1,55 @@
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// Connection settings for the AVEVA System Platform Galaxy Repository database.
/// <para>
/// <see cref="SectionName"/> is a generic default; the DI extension accepts an explicit
/// configuration section path so a consumer can bind from its own section (e.g.
/// <c>HistorianGateway:Galaxy</c>).
/// </para>
/// </summary>
public sealed class GalaxyRepositoryOptions
{
/// <summary>
/// Generic default configuration section name. The DI extension accepts an explicit
/// section path, so a consumer may bind from a different section (e.g.
/// <c>HistorianGateway:Galaxy</c>).
/// </summary>
public const string SectionName = "GalaxyRepository";
/// <summary>
/// Default SQL Server connection string for the Galaxy Repository database.
/// Single source of truth shared with the integration-test fallback so the
/// production default and the live-test default cannot drift.
/// </summary>
public const string DefaultConnectionString =
"Server=localhost;Database=ZB;Integrated Security=True;TrustServerCertificate=True;Encrypt=False;";
/// <summary>The SQL Server connection string for the Galaxy Repository database.</summary>
public string ConnectionString { get; init; } = DefaultConnectionString;
/// <summary>The timeout in seconds for SQL commands executed against the Galaxy Repository.</summary>
public int CommandTimeoutSeconds { get; init; } = 60;
/// <summary>
/// Interval (seconds) between background refreshes of the dashboard Galaxy summary
/// cache. SQL is hit at most once per interval regardless of dashboard render rate.
/// </summary>
public int DashboardRefreshIntervalSeconds { get; init; } = 30;
/// <summary>
/// Whether the latest successful Galaxy browse dataset is persisted to disk. When
/// enabled, the cache reloads that snapshot at startup so clients can still browse
/// last-known data while the Galaxy database is unreachable.
/// </summary>
public bool PersistSnapshot { get; init; } = true;
/// <summary>
/// File path for the persisted Galaxy browse snapshot. Ignored when
/// <see cref="PersistSnapshot"/> is <see langword="false"/>. There is no built-in
/// default path — the consumer supplies a cross-platform-friendly path appropriate to
/// its host. When left empty and <see cref="PersistSnapshot"/> is enabled, the
/// snapshot store (a later task) decides where to write.
/// </summary>
public string SnapshotCachePath { get; init; } = string.Empty;
}
@@ -0,0 +1,16 @@
using ZB.MOM.WW.GalaxyRepository.Grpc;
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// Resolution result for a tag address: the owning <see cref="GalaxyObject"/>, the
/// specific <see cref="GalaxyAttribute"/> when the address names an attribute (otherwise
/// <see langword="null"/>), and the object's contained path.
/// </summary>
/// <param name="Object">The Galaxy object that owns the looked-up address.</param>
/// <param name="Attribute">The matched attribute, or <see langword="null"/> when the address names an object.</param>
/// <param name="ContainedPath">The owning object's contained path.</param>
public sealed record GalaxyTagLookup(
GalaxyObject Object,
GalaxyAttribute? Attribute,
string ContainedPath);
@@ -0,0 +1,329 @@
using Google.Protobuf.WellKnownTypes;
using Grpc.Core;
using ProtoGalaxyRepository = ZB.MOM.WW.GalaxyRepository.Grpc.GalaxyRepository;
namespace ZB.MOM.WW.GalaxyRepository.Grpc;
/// <summary>
/// Reusable gRPC surface that exposes the Galaxy Repository to clients. Hosted by any
/// consuming gateway (e.g. MxAccessGateway or the HistorianGateway sidecar) via
/// <see cref="DependencyInjection.GalaxyRepositoryServiceCollectionExtensions.MapZbGalaxyRepository"/>.
/// <para>
/// <c>DiscoverHierarchy</c> and <c>GetLastDeployTime</c> serve from
/// <see cref="IGalaxyHierarchyCache"/> so many clients share a single SQL pull.
/// <c>WatchDeployEvents</c> streams events from <see cref="IGalaxyDeployNotifier"/>.
/// <c>TestConnection</c> remains a direct SQL probe since callers use it as a health check.
/// </para>
/// <para>
/// This service applies <b>no</b> per-identity browse-subtree filtering — the full
/// hierarchy is projected (<c>null</c> subtree globs). Authorization (including any
/// subtree scoping) is the responsibility of the hosting gateway's interceptor layer.
/// </para>
/// </summary>
/// <param name="repository">Direct SQL surface used by <c>TestConnection</c>.</param>
/// <param name="cache">Shared hierarchy cache that <c>DiscoverHierarchy</c>/<c>BrowseChildren</c>/<c>GetLastDeployTime</c> serve from.</param>
/// <param name="notifier">Deploy-event source streamed by <c>WatchDeployEvents</c>.</param>
public sealed class GalaxyRepositoryGrpcService(
IGalaxyRepository repository,
IGalaxyHierarchyCache cache,
IGalaxyDeployNotifier notifier) : ProtoGalaxyRepository.GalaxyRepositoryBase
{
private static readonly TimeSpan FirstLoadWaitBudget = TimeSpan.FromSeconds(5);
private const int DefaultDiscoverPageSize = 1000;
private const int MaxDiscoverPageSize = 5000;
private const int DefaultBrowsePageSize = 500;
// MaxBrowsePageSize reuses MaxDiscoverPageSize (5000) — same cap.
/// <inheritdoc />
public override async Task<TestConnectionReply> TestConnection(
TestConnectionRequest request,
ServerCallContext context)
{
bool ok = await repository.TestConnectionAsync(context.CancellationToken).ConfigureAwait(false);
return new TestConnectionReply { Ok = ok };
}
/// <inheritdoc />
public override async Task<GetLastDeployTimeReply> GetLastDeployTime(
GetLastDeployTimeRequest request,
ServerCallContext context)
{
await WaitForCacheBootstrap(context.CancellationToken).ConfigureAwait(false);
GalaxyHierarchyCacheEntry entry = cache.Current;
if (!entry.HasData)
{
throw new RpcException(new Status(
StatusCode.Unavailable,
ResolveUnavailableMessage(entry)));
}
GetLastDeployTimeReply reply = new() { Present = entry.LastDeployTime.HasValue };
if (entry.LastDeployTime.HasValue)
{
reply.TimeOfLastDeploy = Timestamp.FromDateTimeOffset(entry.LastDeployTime.Value);
}
return reply;
}
/// <inheritdoc />
public override async Task<DiscoverHierarchyReply> DiscoverHierarchy(
DiscoverHierarchyRequest request,
ServerCallContext context)
{
await WaitForCacheBootstrap(context.CancellationToken).ConfigureAwait(false);
GalaxyHierarchyCacheEntry entry = cache.Current;
if (!entry.HasData)
{
throw new RpcException(new Status(
StatusCode.Unavailable,
ResolveUnavailableMessage(entry)));
}
int pageSize = ResolvePageSize(request.PageSize);
// The shared library applies no per-identity subtree scoping; the hosting
// gateway enforces authorization at its interceptor layer.
string filterSignature = GalaxyHierarchyProjector.ComputeFilterSignature(request, browseSubtreeGlobs: null);
PageToken pageToken = ParsePageToken(request.PageToken, entry.Sequence, filterSignature);
GalaxyHierarchyQueryResult query = GalaxyHierarchyProjector.Project(
entry,
request,
browseSubtreeGlobs: null,
pageToken.Offset,
pageSize);
int offset = pageToken.Offset;
if (offset > query.TotalObjectCount)
{
throw new RpcException(new Status(
StatusCode.InvalidArgument,
"DiscoverHierarchy page_token is outside the current hierarchy."));
}
DiscoverHierarchyReply reply = new()
{
TotalObjectCount = query.TotalObjectCount,
};
reply.Objects.Add(query.Objects);
int nextOffset = offset + query.Objects.Count;
if (nextOffset < query.TotalObjectCount)
{
reply.NextPageToken = FormatPageToken(entry.Sequence, query.FilterSignature, nextOffset);
}
return reply;
}
/// <inheritdoc />
public override async Task<BrowseChildrenReply> BrowseChildren(
BrowseChildrenRequest request,
ServerCallContext context)
{
await WaitForCacheBootstrap(context.CancellationToken).ConfigureAwait(false);
GalaxyHierarchyCacheEntry entry = cache.Current;
if (!entry.HasData)
{
throw new RpcException(new Status(
StatusCode.Unavailable,
ResolveUnavailableMessage(entry)));
}
int pageSize = ResolveBrowsePageSize(request.PageSize);
// Resolve the parent id once so the page-token signature can include it
// and the projector sees the same resolved id when memoizing. The projector
// re-resolves internally; with the by-name/by-path indexes on
// GalaxyHierarchyIndex that second call is O(1), so the redundancy is cheap
// and keeps the projector self-contained.
int parentId = GalaxyBrowseProjector.ResolveParentId(entry, request);
string filterSignature = GalaxyBrowseProjector.ComputeFilterSignature(
request, browseSubtreeGlobs: null, parentId);
PageToken pageToken = ParsePageToken(request.PageToken, entry.Sequence, filterSignature);
GalaxyBrowseChildrenResult result = GalaxyBrowseProjector.ProjectChildren(
entry,
request,
browseSubtreeGlobs: null,
pageToken.Offset,
pageSize);
if (pageToken.Offset > result.TotalChildCount)
{
throw new RpcException(new Status(
StatusCode.InvalidArgument,
"BrowseChildren page_token is outside the current children set."));
}
BrowseChildrenReply reply = new()
{
TotalChildCount = result.TotalChildCount,
CacheSequence = (ulong)entry.Sequence,
};
reply.Children.Add(result.Children);
reply.ChildHasChildren.Add(result.ChildHasChildren);
int nextOffset = pageToken.Offset + result.Children.Count;
if (nextOffset < result.TotalChildCount)
{
reply.NextPageToken = FormatPageToken(entry.Sequence, result.FilterSignature, nextOffset);
}
return reply;
}
/// <inheritdoc />
public override async Task WatchDeployEvents(
WatchDeployEventsRequest request,
IServerStreamWriter<DeployEvent> responseStream,
ServerCallContext context)
{
DateTimeOffset? lastSeen = request.LastSeenDeployTime?.ToDateTimeOffset();
await foreach (GalaxyDeployEventInfo info in notifier
.SubscribeAsync(context.CancellationToken)
.ConfigureAwait(false))
{
// Suppress the initial bootstrap event when the client already knows about
// this deploy time. We only suppress the first one — subsequent events fire
// on actual changes, so they always pass.
if (lastSeen is { } seen && info.TimeOfLastDeploy == seen)
{
lastSeen = null;
continue;
}
lastSeen = null;
await responseStream.WriteAsync(MapDeployEvent(info), context.CancellationToken).ConfigureAwait(false);
}
}
private async Task WaitForCacheBootstrap(CancellationToken cancellationToken)
{
if (cache.Current.HasData || cache.Current.Status == GalaxyCacheStatus.Unavailable)
{
return;
}
using CancellationTokenSource budget = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
budget.CancelAfter(FirstLoadWaitBudget);
try
{
await cache.WaitForFirstLoadAsync(budget.Token).ConfigureAwait(false);
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
{
throw;
}
catch (OperationCanceledException)
{
// Budget elapsed; fall through and let the caller see the current
// (possibly Unknown/Unavailable) entry.
}
}
private static DeployEvent MapDeployEvent(GalaxyDeployEventInfo info)
{
DeployEvent ev = new()
{
Sequence = (ulong)info.Sequence,
ObservedAt = Timestamp.FromDateTimeOffset(info.ObservedAt),
ObjectCount = info.ObjectCount,
AttributeCount = info.AttributeCount,
TimeOfLastDeployPresent = info.TimeOfLastDeploy.HasValue,
};
if (info.TimeOfLastDeploy.HasValue)
{
ev.TimeOfLastDeploy = Timestamp.FromDateTimeOffset(info.TimeOfLastDeploy.Value);
}
return ev;
}
private static string ResolveUnavailableMessage(GalaxyHierarchyCacheEntry entry) => entry.Status switch
{
GalaxyCacheStatus.Unknown => "Galaxy cache has not completed its initial load yet.",
GalaxyCacheStatus.Unavailable => "Galaxy repository is unavailable.",
_ => "Galaxy cache has no data available.",
};
private static int ResolvePageSize(int requestedPageSize)
{
if (requestedPageSize < 0)
{
throw new RpcException(new Status(
StatusCode.InvalidArgument,
"DiscoverHierarchy page_size must be greater than zero when provided."));
}
int pageSize = requestedPageSize == 0 ? DefaultDiscoverPageSize : requestedPageSize;
return Math.Min(pageSize, MaxDiscoverPageSize);
}
private static int ResolveBrowsePageSize(int requested)
{
if (requested < 0)
{
throw new RpcException(new Status(
StatusCode.InvalidArgument,
"BrowseChildren page_size must be greater than zero when provided."));
}
int pageSize = requested == 0 ? DefaultBrowsePageSize : requested;
return Math.Min(pageSize, MaxDiscoverPageSize);
}
private static string FormatPageToken(long sequence, string filterSignature, int offset)
{
return string.Concat(
sequence.ToString(System.Globalization.CultureInfo.InvariantCulture),
":",
filterSignature,
":",
offset.ToString(System.Globalization.CultureInfo.InvariantCulture));
}
private static PageToken ParsePageToken(string pageToken, long currentSequence, string currentFilterSignature)
{
if (string.IsNullOrWhiteSpace(pageToken))
{
return new PageToken(currentSequence, currentFilterSignature, Offset: 0);
}
string[] parts = pageToken.Split(':', count: 3);
if (parts.Length != 3
|| !long.TryParse(
parts[0],
System.Globalization.NumberStyles.None,
System.Globalization.CultureInfo.InvariantCulture,
out long sequence)
|| !int.TryParse(
parts[2],
System.Globalization.NumberStyles.None,
System.Globalization.CultureInfo.InvariantCulture,
out int offset)
|| offset < 0)
{
throw new RpcException(new Status(
StatusCode.InvalidArgument,
"page_token is invalid."));
}
if (sequence != currentSequence)
{
throw new RpcException(new Status(
StatusCode.InvalidArgument,
"page_token is stale."));
}
if (!string.Equals(parts[1], currentFilterSignature, StringComparison.Ordinal))
{
throw new RpcException(new Status(
StatusCode.InvalidArgument,
"page_token does not match the current filters."));
}
return new PageToken(sequence, parts[1], offset);
}
private sealed record PageToken(long Sequence, string FilterSignature, int Offset);
}
@@ -0,0 +1,17 @@
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>Publishes Galaxy repository deploy events to subscribers.</summary>
public interface IGalaxyDeployNotifier
{
/// <summary>The most recently published event, or null if no event has fired yet.</summary>
GalaxyDeployEventInfo? Latest { get; }
/// <summary>Publishes a deploy event to all current subscribers and stores it as Latest.</summary>
/// <param name="info">The deploy event to publish.</param>
void Publish(GalaxyDeployEventInfo info);
/// <summary>Subscribes to deploy events. The sequence yields the latest event first (if available) then streams new events as they fire.</summary>
/// <param name="cancellationToken">Token to cancel the asynchronous operation.</param>
/// <returns>Async enumerable of deploy events.</returns>
IAsyncEnumerable<GalaxyDeployEventInfo> SubscribeAsync(CancellationToken cancellationToken);
}
@@ -0,0 +1,25 @@
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>Cache for Galaxy Repository hierarchy data.</summary>
public interface IGalaxyHierarchyCache
{
/// <summary>The latest cache entry. Status freshness is recomputed against the clock.</summary>
GalaxyHierarchyCacheEntry Current { get; }
/// <summary>
/// Forces a refresh against the Galaxy Repository. Performs a cheap
/// <c>time_of_last_deploy</c> probe first and only re-queries the heavy hierarchy +
/// attributes rowsets when the deploy time has changed since the last successful
/// refresh.
/// </summary>
/// <param name="cancellationToken">Token to cancel the asynchronous operation.</param>
Task RefreshAsync(CancellationToken cancellationToken);
/// <summary>
/// Awaits the first completed refresh attempt (success or failure). Useful for
/// gRPC handlers that want to serve from cache without returning Unavailable on the
/// very first request after the service starts.
/// </summary>
/// <param name="cancellationToken">Token to cancel the asynchronous operation.</param>
Task WaitForFirstLoadAsync(CancellationToken cancellationToken);
}
@@ -0,0 +1,28 @@
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// Persists the latest Galaxy Repository browse dataset to disk and reloads
/// it at startup. Lets <see cref="GalaxyHierarchyCache"/> serve last-known
/// browse data when the Galaxy database is unreachable on a cold start.
/// </summary>
public interface IGalaxyHierarchySnapshotStore
{
/// <summary>
/// Writes <paramref name="snapshot"/> to disk, replacing any previous
/// snapshot atomically. A no-op when snapshot persistence is disabled.
/// </summary>
/// <param name="snapshot">The browse dataset to persist.</param>
/// <param name="cancellationToken">Token to cancel the asynchronous operation.</param>
Task SaveAsync(GalaxyHierarchySnapshot snapshot, CancellationToken cancellationToken);
/// <summary>
/// Reads the persisted Galaxy browse dataset.
/// </summary>
/// <param name="cancellationToken">Token to cancel the asynchronous operation.</param>
/// <returns>
/// The persisted snapshot, or <see langword="null"/> when none exists,
/// persistence is disabled, or the on-disk file uses an unrecognized
/// schema version.
/// </returns>
Task<GalaxyHierarchySnapshot?> TryLoadAsync(CancellationToken cancellationToken);
}
@@ -0,0 +1,26 @@
namespace ZB.MOM.WW.GalaxyRepository;
/// <summary>
/// Abstraction over <see cref="GalaxyRepository"/>: the read-only SQL surface over the
/// AVEVA System Platform Galaxy Repository database. Exists so consumers (and the cache
/// layer, a later task) can be unit-tested against an in-memory fake without standing up a
/// real <c>Microsoft.Data.SqlClient</c> <c>SqlConnection</c> against a bogus host/port.
/// </summary>
public interface IGalaxyRepository
{
/// <summary>Tests the connection to the Galaxy Repository database.</summary>
/// <param name="ct">Token to cancel the asynchronous operation.</param>
Task<bool> TestConnectionAsync(CancellationToken ct = default);
/// <summary>Retrieves the last deployment time from the Galaxy Repository.</summary>
/// <param name="ct">Token to cancel the asynchronous operation.</param>
Task<DateTime?> GetLastDeployTimeAsync(CancellationToken ct = default);
/// <summary>Retrieves the complete hierarchy of Galaxy objects from the repository.</summary>
/// <param name="ct">Token to cancel the asynchronous operation.</param>
Task<List<GalaxyHierarchyRow>> GetHierarchyAsync(CancellationToken ct = default);
/// <summary>Retrieves all attributes for Galaxy objects from the repository.</summary>
/// <param name="ct">Token to cancel the asynchronous operation.</param>
Task<List<GalaxyAttributeRow>> GetAttributesAsync(CancellationToken ct = default);
}
@@ -0,0 +1,190 @@
syntax = "proto3";
package galaxy_repository.v1;
option csharp_namespace = "ZB.MOM.WW.GalaxyRepository.Grpc";
import "google/protobuf/timestamp.proto";
import "google/protobuf/wrappers.proto";
// Wire-compatibility policy (ProtobufStyleGuide): this contract evolves
// additively only. Never renumber or repurpose an existing field number or
// enum value. When a field or enum value is removed, add a `reserved` range
// (and `reserved` name) covering it in the same change so a future editor
// cannot accidentally reuse the retired tag. There are no `reserved`
// declarations today because no field or enum value has ever been removed.
// Read-only browse over the AVEVA System Platform Galaxy Repository (ZB SQL
// database). Lets clients enumerate the deployed object hierarchy and each
// object's dynamic attributes so they know what tag references to subscribe
// to via the MxAccessGateway service.
service GalaxyRepository {
rpc TestConnection(TestConnectionRequest) returns (TestConnectionReply);
rpc GetLastDeployTime(GetLastDeployTimeRequest) returns (GetLastDeployTimeReply);
rpc DiscoverHierarchy(DiscoverHierarchyRequest) returns (DiscoverHierarchyReply);
// Server-stream of deploy events. The server emits the current state immediately
// on subscribe (so clients can bootstrap their cache without waiting for the next
// deploy), then emits one event each time the gateway's hierarchy cache observes
// a new galaxy.time_of_last_deploy. The sequence field is monotonically
// increasing per server start; gaps indicate the per-subscriber buffer dropped
// older events because the client was too slow.
rpc WatchDeployEvents(WatchDeployEventsRequest) returns (stream DeployEvent);
// Returns the direct children of a parent object (or the root objects when
// `parent` is unset). Designed for OPC UA-style lazy expand: clients walk
// one level at a time instead of paging the full hierarchy. Filters mirror
// DiscoverHierarchy exactly. Backed by the same shared hierarchy cache.
rpc BrowseChildren(BrowseChildrenRequest) returns (BrowseChildrenReply);
}
message TestConnectionRequest {}
message TestConnectionReply {
bool ok = 1;
}
message GetLastDeployTimeRequest {}
message GetLastDeployTimeReply {
bool present = 1;
google.protobuf.Timestamp time_of_last_deploy = 2;
}
message DiscoverHierarchyRequest {
// Maximum number of objects to return. The server applies its default when
// unset and rejects non-positive values.
int32 page_size = 1;
// Opaque token returned by a previous DiscoverHierarchy response.
string page_token = 2;
// Optional. When set, return only this object and its descendants.
// Empty = full hierarchy.
oneof root {
int32 root_gobject_id = 3;
string root_tag_name = 4;
string root_contained_path = 5;
}
// Optional. Cap on descendant depth from root. Zero returns only the root.
// Unset means unlimited depth.
google.protobuf.Int32Value max_depth = 6;
// Optional object category id filters.
repeated int32 category_ids = 7;
// Optional case-insensitive substring filters against template names.
repeated string template_chain_contains = 8;
// Optional anchored, case-insensitive glob over object tag_name.
string tag_name_glob = 9;
// Optional. Unset or true includes attributes. False returns object skeletons.
optional bool include_attributes = 10;
// Optional. Return only objects with at least one alarm-bearing attribute.
bool alarm_bearing_only = 11;
// Optional. Return only objects with at least one historized attribute.
bool historized_only = 12;
}
message DiscoverHierarchyReply {
repeated GalaxyObject objects = 1;
// Non-empty when another page is available.
string next_page_token = 2;
// Total number of objects in the cached hierarchy at the time of the call.
int32 total_object_count = 3;
}
message WatchDeployEventsRequest {
// Optional. When set, the bootstrap event is suppressed if the cached deploy
// time matches this value. Future events are still emitted normally.
google.protobuf.Timestamp last_seen_deploy_time = 1;
}
message DeployEvent {
// Monotonically increasing per server start. Gaps indicate dropped events.
uint64 sequence = 1;
// Server wall-clock when the cache observed the deploy.
google.protobuf.Timestamp observed_at = 2;
// Galaxy.time_of_last_deploy. Absent only when the Galaxy table reports null.
google.protobuf.Timestamp time_of_last_deploy = 3;
bool time_of_last_deploy_present = 4;
int32 object_count = 5;
int32 attribute_count = 6;
}
message GalaxyObject {
int32 gobject_id = 1;
string tag_name = 2;
string contained_name = 3;
string browse_name = 4;
int32 parent_gobject_id = 5;
bool is_area = 6;
int32 category_id = 7;
int32 hosted_by_gobject_id = 8;
repeated string template_chain = 9;
repeated GalaxyAttribute attributes = 10;
}
message GalaxyAttribute {
string attribute_name = 1;
string full_tag_reference = 2;
// Raw Galaxy SQL `dbo.data_type` identifier, passed through unchanged.
// This is NOT a member of `mxaccess_gateway.v1.MxDataType` — Galaxy's
// type enumeration is distinct from MXAccess's wire data-type enum and
// the two must not be cast or compared. The GalaxyRepository service is
// metadata-only and deliberately does not share types with
// mxaccess_gateway.proto. See docs/GalaxyRepository.md.
int32 mx_data_type = 3;
// Human-readable name from Galaxy's `dbo.data_type` table (e.g. "Float",
// "Integer", "Boolean"). Free-form Galaxy text; not a stable enum.
string data_type_name = 4;
bool is_array = 5;
int32 array_dimension = 6;
bool array_dimension_present = 7;
// Raw Galaxy SQL attribute-category identifier, passed through unchanged.
// Galaxy-specific; not mapped to any gateway enum. See
// docs/GalaxyRepository.md.
int32 mx_attribute_category = 8;
// Raw Galaxy SQL security-classification identifier, passed through
// unchanged. Galaxy-specific; not mapped to any gateway enum. See
// docs/GalaxyRepository.md.
int32 security_classification = 9;
bool is_historized = 10;
bool is_alarm = 11;
}
message BrowseChildrenRequest {
// Parent selector. Empty oneof returns root objects (parent_gobject_id == 0).
oneof parent {
int32 parent_gobject_id = 1;
string parent_tag_name = 2;
string parent_contained_path = 3;
}
// Maximum number of direct children to return. Server default 500; cap 5000.
int32 page_size = 4;
// Opaque token returned by a previous BrowseChildren response. Bound to the
// cache sequence, parent selector, and the filter set; a mismatch returns
// InvalidArgument.
string page_token = 5;
// --- Filter parity with DiscoverHierarchy. AND-combined. ---
repeated int32 category_ids = 6;
repeated string template_chain_contains = 7;
string tag_name_glob = 8;
optional bool include_attributes = 9;
bool alarm_bearing_only = 10;
bool historized_only = 11;
}
message BrowseChildrenReply {
// Direct children matching the filter, sorted areas-first then by
// case-insensitive display name (same order as the dashboard tree).
repeated GalaxyObject children = 1;
// Non-empty when another page of siblings is available.
string next_page_token = 2;
// Total matching direct children of the parent (post-filter).
int32 total_child_count = 3;
// Parallel array, indexed with `children`. True when the child has at least
// one matching descendant under the same filter set. Lets a UI choose
// whether to draw an expand triangle without an extra round trip.
repeated bool child_has_children = 4;
// Cache sequence this reply was projected from. Clients may pass it back as
// part of the page_token contract. Mismatch on the next page -> InvalidArgument.
uint64 cache_sequence = 5;
}
@@ -0,0 +1,30 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<IsPackable>true</IsPackable>
<PackageId>ZB.MOM.WW.GalaxyRepository</PackageId>
<Authors>ZB.MOM.WW</Authors>
<Description>Read-only Galaxy object-hierarchy browse library for the ZB.MOM.WW SCADA family. Provides a SQL provider for the Galaxy Repository database and a canonical gRPC service for exposing the hierarchy to modern .NET 10 clients — extracted from MxAccessGateway so any consumer can browse the Galaxy without loading 32-bit COM.</Description>
<PackageTags>galaxy;repository;browse;aveva;wonderware;system-platform;scada;grpc;sql;zb-mom-ww</PackageTags>
<PackageProjectUrl>https://gitea.dohertylan.com/dohertj2/zb-mom-ww-galaxyrepository</PackageProjectUrl>
<RepositoryUrl>https://gitea.dohertylan.com/dohertj2/zb-mom-ww-galaxyrepository</RepositoryUrl>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Data.SqlClient" />
<PackageReference Include="Grpc.AspNetCore" />
<PackageReference Include="Google.Protobuf" />
<PackageReference Include="Microsoft.Extensions.Hosting.Abstractions" />
<PackageReference Include="Microsoft.Extensions.Options.ConfigurationExtensions" />
<PackageReference Include="Grpc.Tools">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
</ItemGroup>
<!-- Proto files are added in Task 2; the empty glob is intentional and builds cleanly. -->
<ItemGroup>
<Protobuf Include="Protos\*.proto" GrpcServices="Server" />
</ItemGroup>
</Project>
@@ -0,0 +1,134 @@
using System.Runtime.CompilerServices;
using ZB.MOM.WW.GalaxyRepository;
namespace ZB.MOM.WW.GalaxyRepository.Tests;
/// <summary>
/// In-memory <see cref="IGalaxyRepository"/> returning canned rowsets. Counts the heavy
/// hierarchy/attribute reads so tests can assert deploy-gated skips, and can be flipped to
/// throw so the failure path is exercisable.
/// </summary>
internal sealed class FakeGalaxyRepository : IGalaxyRepository
{
private readonly IReadOnlyList<GalaxyHierarchyRow> _hierarchy;
private readonly IReadOnlyList<GalaxyAttributeRow> _attributes;
public FakeGalaxyRepository(
IReadOnlyList<GalaxyHierarchyRow> hierarchy,
IReadOnlyList<GalaxyAttributeRow> attributes,
DateTime? deployTime)
{
_hierarchy = hierarchy;
_attributes = attributes;
DeployTime = deployTime;
}
/// <summary>The deploy time returned by <see cref="GetLastDeployTimeAsync"/>; mutate to simulate a redeploy.</summary>
public DateTime? DeployTime { get; set; }
/// <summary>When set, every query throws this exception (simulates an unreachable database).</summary>
public Exception? ThrowOnQuery { get; set; }
public int HierarchyReadCount { get; private set; }
public int AttributeReadCount { get; private set; }
public Task<bool> TestConnectionAsync(CancellationToken ct = default) =>
ThrowOnQuery is null ? Task.FromResult(true) : throw ThrowOnQuery;
public Task<DateTime?> GetLastDeployTimeAsync(CancellationToken ct = default)
{
if (ThrowOnQuery is not null)
{
throw ThrowOnQuery;
}
return Task.FromResult(DeployTime);
}
public Task<List<GalaxyHierarchyRow>> GetHierarchyAsync(CancellationToken ct = default)
{
if (ThrowOnQuery is not null)
{
throw ThrowOnQuery;
}
HierarchyReadCount++;
return Task.FromResult(_hierarchy.ToList());
}
public Task<List<GalaxyAttributeRow>> GetAttributesAsync(CancellationToken ct = default)
{
if (ThrowOnQuery is not null)
{
throw ThrowOnQuery;
}
AttributeReadCount++;
return Task.FromResult(_attributes.ToList());
}
}
/// <summary>Records published deploy events so tests can assert publication.</summary>
internal sealed class RecordingDeployNotifier : IGalaxyDeployNotifier
{
public List<GalaxyDeployEventInfo> Published { get; } = [];
public GalaxyDeployEventInfo? Latest { get; private set; }
public void Publish(GalaxyDeployEventInfo info)
{
Published.Add(info);
Latest = info;
}
public async IAsyncEnumerable<GalaxyDeployEventInfo> SubscribeAsync(
[EnumeratorCancellation] CancellationToken cancellationToken)
{
if (Latest is { } latest)
{
yield return latest;
}
await Task.CompletedTask.ConfigureAwait(false);
}
}
/// <summary>
/// In-memory <see cref="IGalaxyHierarchySnapshotStore"/>. Pre-seed <see cref="Snapshot"/>
/// to exercise the restore path; reads <see cref="SaveAsync"/> back to assert persistence.
/// </summary>
internal sealed class FakeSnapshotStore : IGalaxyHierarchySnapshotStore
{
public GalaxyHierarchySnapshot? Snapshot { get; set; }
public int SaveCount { get; private set; }
public int LoadCount { get; private set; }
public Task SaveAsync(GalaxyHierarchySnapshot snapshot, CancellationToken cancellationToken)
{
SaveCount++;
Snapshot = snapshot;
return Task.CompletedTask;
}
public Task<GalaxyHierarchySnapshot?> TryLoadAsync(CancellationToken cancellationToken)
{
LoadCount++;
return Task.FromResult(Snapshot);
}
}
/// <summary>
/// A <see cref="TimeProvider"/> whose UTC clock is fixed (and advanceable) so the cache's
/// staleness projection (which fires after a 5-minute threshold) is deterministic.
/// </summary>
internal sealed class StubTimeProvider(DateTimeOffset start) : TimeProvider
{
private DateTimeOffset _now = start;
public override DateTimeOffset GetUtcNow() => _now;
public void Advance(TimeSpan delta) => _now += delta;
}
@@ -0,0 +1,236 @@
using ZB.MOM.WW.GalaxyRepository;
namespace ZB.MOM.WW.GalaxyRepository.Tests;
/// <summary>
/// Tests for <see cref="GalaxyHierarchyCache"/> first-load, deploy-gating, snapshot
/// restore, persistence, and status-transition behavior. Uses an in-memory
/// <see cref="IGalaxyRepository"/> and snapshot store plus a fixed
/// <see cref="StubTimeProvider"/> so no SQL is touched and no asserts are time-sensitive.
/// </summary>
public sealed class GalaxyHierarchyCacheTests
{
private static readonly DateTimeOffset FixedNow = new(2026, 1, 1, 12, 0, 0, TimeSpan.Zero);
private static readonly DateTime DeployTime = new(2026, 1, 1, 0, 0, 0, DateTimeKind.Utc);
private static List<GalaxyHierarchyRow> SampleHierarchy() =>
[
new() { GobjectId = 1, TagName = "Area1", ContainedName = "Area1", BrowseName = "Area1", IsArea = true },
new() { GobjectId = 2, TagName = "Pump01", ContainedName = "Pump01", BrowseName = "Pump01", ParentGobjectId = 1 },
];
private static List<GalaxyAttributeRow> SampleAttributes() =>
[
new() { GobjectId = 2, AttributeName = "PV", FullTagReference = "Pump01.PV", IsHistorized = true, IsAlarm = true },
];
[Fact]
public async Task RefreshAsync_FirstLoad_PopulatesCurrentWithDataAndUnblocksWaitForFirstLoad()
{
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime);
RecordingDeployNotifier notifier = new();
using GalaxyHierarchyCache cache = new(repository, notifier, new StubTimeProvider(FixedNow));
// Before refresh, the gate is unset and there is no data.
Assert.False(cache.Current.HasData);
Assert.Equal(GalaxyCacheStatus.Unknown, cache.Current.Status);
await cache.RefreshAsync(CancellationToken.None);
// First load completes (does not hang) and Current now holds usable data.
await cache.WaitForFirstLoadAsync(new CancellationTokenSource(TimeSpan.FromSeconds(5)).Token);
GalaxyHierarchyCacheEntry current = cache.Current;
Assert.True(current.HasData);
Assert.Equal(GalaxyCacheStatus.Healthy, current.Status);
Assert.Equal(2, current.ObjectCount);
Assert.Equal(1, current.AreaCount);
Assert.Equal(1, current.AttributeCount);
Assert.Equal(1, current.HistorizedAttributeCount);
Assert.Equal(1, current.AlarmAttributeCount);
// The heavy queries ran exactly once and a deploy event was published.
Assert.Equal(1, repository.HierarchyReadCount);
Assert.Equal(1, repository.AttributeReadCount);
GalaxyDeployEventInfo published = Assert.Single(notifier.Published);
Assert.Equal(2, published.ObjectCount);
Assert.Equal(1, published.AttributeCount);
}
[Fact]
public async Task RefreshAsync_NoDeployChange_SkipsHeavyQueriesOnSecondRefresh()
{
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime);
using GalaxyHierarchyCache cache = new(repository, new RecordingDeployNotifier(), new StubTimeProvider(FixedNow));
await cache.RefreshAsync(CancellationToken.None);
await cache.RefreshAsync(CancellationToken.None);
// Deploy time unchanged => the heavy hierarchy/attribute reads happened only once.
Assert.Equal(1, repository.HierarchyReadCount);
Assert.Equal(1, repository.AttributeReadCount);
Assert.True(cache.Current.HasData);
Assert.Equal(GalaxyCacheStatus.Healthy, cache.Current.Status);
}
[Fact]
public async Task RefreshAsync_DeployAdvances_RebuildsAndBumpsSequence()
{
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime);
RecordingDeployNotifier notifier = new();
using GalaxyHierarchyCache cache = new(repository, notifier, new StubTimeProvider(FixedNow));
await cache.RefreshAsync(CancellationToken.None);
long firstSequence = cache.Current.Sequence;
repository.DeployTime = DeployTime.AddHours(1);
await cache.RefreshAsync(CancellationToken.None);
Assert.Equal(2, repository.HierarchyReadCount);
Assert.Equal(firstSequence + 1, cache.Current.Sequence);
Assert.Equal(2, notifier.Published.Count);
}
[Fact]
public async Task RefreshAsync_FirstQueryFailsNoPriorData_StatusUnavailableButFirstLoadStillCompletes()
{
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime)
{
ThrowOnQuery = new TimeoutException("galaxy db unreachable"),
};
using GalaxyHierarchyCache cache = new(repository, new RecordingDeployNotifier(), new StubTimeProvider(FixedNow));
await cache.RefreshAsync(CancellationToken.None);
// First load must complete so callers do not hang, even though the query failed.
await cache.WaitForFirstLoadAsync(new CancellationTokenSource(TimeSpan.FromSeconds(5)).Token);
Assert.False(cache.Current.HasData);
Assert.Equal(GalaxyCacheStatus.Unavailable, cache.Current.Status);
Assert.Contains("unreachable", cache.Current.LastError);
}
[Fact]
public async Task RefreshAsync_QueryFailsAfterPriorData_DegradesToStaleAndKeepsData()
{
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime);
using GalaxyHierarchyCache cache = new(repository, new RecordingDeployNotifier(), new StubTimeProvider(FixedNow));
await cache.RefreshAsync(CancellationToken.None);
Assert.True(cache.Current.HasData);
// A later refresh fails: data is retained but flagged Stale.
repository.DeployTime = DeployTime.AddHours(1);
repository.ThrowOnQuery = new InvalidOperationException("transient");
await cache.RefreshAsync(CancellationToken.None);
Assert.True(cache.Current.HasData);
Assert.Equal(GalaxyCacheStatus.Stale, cache.Current.Status);
Assert.Equal(2, cache.Current.ObjectCount);
}
[Fact]
public async Task Current_AfterStalenessThreshold_ProjectsHealthyToStale()
{
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime);
StubTimeProvider clock = new(FixedNow);
using GalaxyHierarchyCache cache = new(repository, new RecordingDeployNotifier(), clock);
await cache.RefreshAsync(CancellationToken.None);
Assert.Equal(GalaxyCacheStatus.Healthy, cache.Current.Status);
// Advance past the 5-minute staleness threshold with no successful refresh.
clock.Advance(TimeSpan.FromMinutes(6));
Assert.Equal(GalaxyCacheStatus.Stale, cache.Current.Status);
// Data is still present — Stale means "old", not "gone".
Assert.True(cache.Current.HasData);
}
[Fact]
public async Task RefreshAsync_PersistsSnapshotAfterSuccessfulHeavyRefresh()
{
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime);
FakeSnapshotStore store = new();
using GalaxyHierarchyCache cache = new(
repository, new RecordingDeployNotifier(), new StubTimeProvider(FixedNow), logger: null, snapshotStore: store);
await cache.RefreshAsync(CancellationToken.None);
Assert.Equal(1, store.SaveCount);
Assert.NotNull(store.Snapshot);
Assert.Equal(2, store.Snapshot!.Hierarchy.Count);
Assert.Single(store.Snapshot.Attributes);
}
[Fact]
public async Task RefreshAsync_SnapshotRestore_ServesLastKnownDataAsStaleWhenDatabaseUnreachable()
{
// The snapshot store already holds a persisted dataset (last-known browse data).
FakeSnapshotStore store = new()
{
Snapshot = new GalaxyHierarchySnapshot(
LastDeployTime: DeployTime,
SavedAt: FixedNow.AddMinutes(-1),
Hierarchy: SampleHierarchy(),
Attributes: SampleAttributes()),
};
// The Galaxy database is unreachable on this cold start.
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime)
{
ThrowOnQuery = new TimeoutException("cold start, db down"),
};
RecordingDeployNotifier notifier = new();
using GalaxyHierarchyCache cache = new(
repository, notifier, new StubTimeProvider(FixedNow), logger: null, snapshotStore: store);
await cache.RefreshAsync(CancellationToken.None);
// First load is satisfied by the restored snapshot, not by SQL.
await cache.WaitForFirstLoadAsync(new CancellationTokenSource(TimeSpan.FromSeconds(5)).Token);
Assert.Equal(1, store.LoadCount);
GalaxyHierarchyCacheEntry current = cache.Current;
Assert.True(current.HasData);
// Restored data is "last-known", surfaced as Stale until the live DB confirms.
Assert.Equal(GalaxyCacheStatus.Stale, current.Status);
Assert.Equal(2, current.ObjectCount);
Assert.Equal(DeployTime, current.LastDeployTime!.Value.UtcDateTime);
// A deploy event was published for the restored data.
Assert.Single(notifier.Published);
}
[Fact]
public async Task RefreshAsync_SnapshotRestoreThenLiveQuery_PromotesRestoredDataToHealthy()
{
FakeSnapshotStore store = new()
{
Snapshot = new GalaxyHierarchySnapshot(
LastDeployTime: DeployTime,
SavedAt: FixedNow.AddMinutes(-1),
Hierarchy: SampleHierarchy(),
Attributes: SampleAttributes()),
};
// DB is reachable and reports the SAME deploy time the snapshot was pulled at.
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime);
using GalaxyHierarchyCache cache = new(
repository, new RecordingDeployNotifier(), new StubTimeProvider(FixedNow), logger: null, snapshotStore: store);
await cache.RefreshAsync(CancellationToken.None);
// Restore seeds Stale data; the same-deploy live query promotes it to Healthy
// without re-running the heavy hierarchy/attribute reads.
Assert.Equal(GalaxyCacheStatus.Healthy, cache.Current.Status);
Assert.Equal(0, repository.HierarchyReadCount);
Assert.True(cache.Current.HasData);
}
[Fact]
public void Dispose_CanBeCalledWithoutHavingRefreshed()
{
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime);
GalaxyHierarchyCache cache = new(repository, new RecordingDeployNotifier(), new StubTimeProvider(FixedNow));
// Dispose must be safe even when no refresh ever ran (semaphore never entered).
cache.Dispose();
}
}
@@ -0,0 +1,458 @@
using Grpc.Core;
using ZB.MOM.WW.GalaxyRepository;
using ZB.MOM.WW.GalaxyRepository.Grpc;
namespace ZB.MOM.WW.GalaxyRepository.Tests;
/// <summary>
/// Pure-logic tests for <see cref="GalaxyHierarchyProjector"/> and
/// <see cref="GalaxyBrowseProjector"/>. No SQL: the cache entry under test is built
/// from a small hand-made hierarchy through the same materialization the live cache
/// uses (a fake <see cref="IGalaxyRepository"/> driven through
/// <see cref="GalaxyHierarchyCache.RefreshAsync"/>), so the projectors are exercised
/// against a real <see cref="GalaxyHierarchyIndex"/>.
/// </summary>
public sealed class GalaxyHierarchyProjectorTests
{
/// <summary>
/// Builds a realistic cache entry by driving a fake repository through the cache's
/// own refresh path. This goes through <c>BuildEntry</c> + <see cref="GalaxyHierarchyIndex.Build"/>
/// exactly as production does, rather than reaching for an internal factory.
/// </summary>
private static GalaxyHierarchyCacheEntry BuildEntry(
IReadOnlyList<GalaxyHierarchyRow> hierarchy,
IReadOnlyList<GalaxyAttributeRow> attributes)
{
FakeGalaxyRepository repository = new(hierarchy, attributes, deployTime: new DateTime(2026, 1, 1, 0, 0, 0, DateTimeKind.Utc));
using GalaxyHierarchyCache cache = new(repository, new RecordingDeployNotifier());
cache.RefreshAsync(CancellationToken.None).GetAwaiter().GetResult();
GalaxyHierarchyCacheEntry entry = cache.Current;
Assert.True(entry.HasData);
return entry;
}
// A small but representative galaxy:
// PlantArea (area, id 1)
// ├─ LineA (area, id 2)
// │ ├─ Pump01 (id 10, template "Pump", historized+alarm attr)
// │ └─ Valve01 (id 11, template "Valve", plain attr)
// └─ Mixer01 (id 12, template "Mixer", alarm attr only)
// StandaloneTank (id 20, no parent — a root object)
private static GalaxyHierarchyCacheEntry BuildSampleEntry()
{
List<GalaxyHierarchyRow> hierarchy =
[
Hierarchy(1, "PlantArea", parent: 0, isArea: true, category: 100),
Hierarchy(2, "LineA", parent: 1, isArea: true, category: 100),
Hierarchy(10, "Pump01", parent: 2, category: 200, templates: ["$Pump", "$UserDefined"]),
Hierarchy(11, "Valve01", parent: 2, category: 201, templates: ["$Valve"]),
Hierarchy(12, "Mixer01", parent: 1, category: 202, templates: ["$Mixer"]),
Hierarchy(20, "StandaloneTank", parent: 0, category: 203, templates: ["$Tank"]),
];
List<GalaxyAttributeRow> attributes =
[
// Pump01: historized AND alarm-bearing.
Attribute(10, "Pump01.PV", historized: true, alarm: true),
Attribute(10, "Pump01.SP", historized: false, alarm: false),
// Valve01: plain.
Attribute(11, "Valve01.Cmd", historized: false, alarm: false),
// Mixer01: alarm only.
Attribute(12, "Mixer01.Fault", historized: false, alarm: true),
// StandaloneTank: historized only.
Attribute(20, "StandaloneTank.Level", historized: true, alarm: false),
];
return BuildEntry(hierarchy, attributes);
}
private static GalaxyHierarchyRow Hierarchy(
int id,
string tagName,
int parent,
bool isArea = false,
int category = 0,
IReadOnlyList<string>? templates = null) => new()
{
GobjectId = id,
TagName = tagName,
ContainedName = tagName,
BrowseName = tagName,
ParentGobjectId = parent,
IsArea = isArea,
CategoryId = category,
TemplateChain = templates ?? Array.Empty<string>(),
};
private static GalaxyAttributeRow Attribute(
int gobjectId,
string fullTagReference,
bool historized,
bool alarm) => new()
{
GobjectId = gobjectId,
AttributeName = fullTagReference.Split('.')[^1],
FullTagReference = fullTagReference,
IsHistorized = historized,
IsAlarm = alarm,
};
[Fact]
public void Project_NoFilters_ReturnsEveryObject()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, new DiscoverHierarchyRequest());
Assert.Equal(6, result.TotalObjectCount);
Assert.Equal(6, result.Objects.Count);
}
[Fact]
public void Project_PageSizeAndOffset_SlicesTheOrderedResult()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
DiscoverHierarchyRequest request = new();
GalaxyHierarchyQueryResult full = GalaxyHierarchyProjector.Project(entry, request, browseSubtreeGlobs: null, offset: 0, pageSize: int.MaxValue);
GalaxyHierarchyQueryResult page1 = GalaxyHierarchyProjector.Project(entry, request, browseSubtreeGlobs: null, offset: 0, pageSize: 2);
GalaxyHierarchyQueryResult page2 = GalaxyHierarchyProjector.Project(entry, request, browseSubtreeGlobs: null, offset: 2, pageSize: 2);
GalaxyHierarchyQueryResult page3 = GalaxyHierarchyProjector.Project(entry, request, browseSubtreeGlobs: null, offset: 4, pageSize: 2);
// Total is unaffected by paging.
Assert.Equal(6, page1.TotalObjectCount);
Assert.Equal(2, page1.Objects.Count);
Assert.Equal(2, page2.Objects.Count);
Assert.Equal(2, page3.Objects.Count);
// The three pages reconstruct the full ordered result with no gaps/dupes.
List<int> paged =
[
.. page1.Objects.Select(o => o.GobjectId),
.. page2.Objects.Select(o => o.GobjectId),
.. page3.Objects.Select(o => o.GobjectId),
];
Assert.Equal(full.Objects.Select(o => o.GobjectId), paged);
}
[Fact]
public void Project_OffsetPastEnd_ReturnsEmptyPageButRealTotal()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(
entry, new DiscoverHierarchyRequest(), browseSubtreeGlobs: null, offset: 999, pageSize: 10);
Assert.Empty(result.Objects);
Assert.Equal(6, result.TotalObjectCount);
}
[Fact]
public void Project_PageSignature_IsStableAcrossPagesAndMatchesComputeFilterSignature()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
DiscoverHierarchyRequest request = new() { TagNameGlob = "Pump*" };
string expected = GalaxyHierarchyProjector.ComputeFilterSignature(request, browseSubtreeGlobs: null);
GalaxyHierarchyQueryResult page1 = GalaxyHierarchyProjector.Project(entry, request, browseSubtreeGlobs: null, offset: 0, pageSize: 1);
GalaxyHierarchyQueryResult page2 = GalaxyHierarchyProjector.Project(entry, request, browseSubtreeGlobs: null, offset: 1, pageSize: 1);
// The signature a caller computes to mint a page token round-trips: the projector
// reports the same signature on every page of the same filter set.
Assert.Equal(expected, page1.FilterSignature);
Assert.Equal(expected, page2.FilterSignature);
}
[Fact]
public void ComputeFilterSignature_DiffersWhenAnyFilterChanges()
{
DiscoverHierarchyRequest baseRequest = new() { TagNameGlob = "Pump*" };
DiscoverHierarchyRequest differentGlob = new() { TagNameGlob = "Valve*" };
DiscoverHierarchyRequest differentAlarm = new() { TagNameGlob = "Pump*", AlarmBearingOnly = true };
string baseSig = GalaxyHierarchyProjector.ComputeFilterSignature(baseRequest, null);
Assert.NotEqual(baseSig, GalaxyHierarchyProjector.ComputeFilterSignature(differentGlob, null));
Assert.NotEqual(baseSig, GalaxyHierarchyProjector.ComputeFilterSignature(differentAlarm, null));
Assert.NotEqual(baseSig, GalaxyHierarchyProjector.ComputeFilterSignature(baseRequest, browseSubtreeGlobs: ["PlantArea/*"]));
// Same inputs => same signature (deterministic).
Assert.Equal(baseSig, GalaxyHierarchyProjector.ComputeFilterSignature(new DiscoverHierarchyRequest { TagNameGlob = "Pump*" }, null));
}
[Fact]
public void Project_MaxDepthZero_FromRoot_ReturnsOnlyTheRoot()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
DiscoverHierarchyRequest request = new() { RootGobjectId = 1, MaxDepth = 0 };
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, request);
GalaxyObject only = Assert.Single(result.Objects);
Assert.Equal(1, only.GobjectId);
}
[Fact]
public void Project_MaxDepthOne_FromRoot_ReturnsRootAndDirectChildrenOnly()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
// PlantArea(1) depth 0; LineA(2) and Mixer01(12) depth 1; Pump01/Valve01 depth 2.
DiscoverHierarchyRequest request = new() { RootGobjectId = 1, MaxDepth = 1 };
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, request);
Assert.Equal([1, 2, 12], result.Objects.Select(o => o.GobjectId).OrderBy(id => id));
}
[Fact]
public void Project_NegativeMaxDepth_ThrowsInvalidArgument()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
DiscoverHierarchyRequest request = new() { MaxDepth = -1 };
RpcException ex = Assert.Throws<RpcException>(() => GalaxyHierarchyProjector.Project(entry, request));
Assert.Equal(StatusCode.InvalidArgument, ex.StatusCode);
}
[Fact]
public void Project_UnknownRoot_ThrowsNotFound()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
DiscoverHierarchyRequest request = new() { RootGobjectId = 99999 };
RpcException ex = Assert.Throws<RpcException>(() => GalaxyHierarchyProjector.Project(entry, request));
Assert.Equal(StatusCode.NotFound, ex.StatusCode);
}
[Fact]
public void Project_HistorizedOnly_ReturnsOnlyObjectsWithAHistorizedAttribute()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
DiscoverHierarchyRequest request = new() { HistorizedOnly = true };
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, request);
// Pump01(10) and StandaloneTank(20) carry historized attributes.
Assert.Equal([10, 20], result.Objects.Select(o => o.GobjectId).OrderBy(id => id));
}
[Fact]
public void Project_AlarmBearingOnly_ReturnsOnlyObjectsWithAnAlarmAttribute()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
DiscoverHierarchyRequest request = new() { AlarmBearingOnly = true };
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, request);
// Pump01(10) and Mixer01(12) carry alarm attributes.
Assert.Equal([10, 12], result.Objects.Select(o => o.GobjectId).OrderBy(id => id));
}
[Fact]
public void Project_AlarmAndHistorizedTogether_RequiresBoth()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
DiscoverHierarchyRequest request = new() { AlarmBearingOnly = true, HistorizedOnly = true };
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, request);
// Only Pump01(10) carries an attribute set that is both historized and alarm-bearing.
GalaxyObject only = Assert.Single(result.Objects);
Assert.Equal(10, only.GobjectId);
}
[Fact]
public void Project_TagNameGlob_MatchesAnchoredCaseInsensitive()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
GalaxyHierarchyQueryResult prefix = GalaxyHierarchyProjector.Project(entry, new DiscoverHierarchyRequest { TagNameGlob = "Pump*" });
Assert.Equal([10], prefix.Objects.Select(o => o.GobjectId));
// Case-insensitive.
GalaxyHierarchyQueryResult lower = GalaxyHierarchyProjector.Project(entry, new DiscoverHierarchyRequest { TagNameGlob = "pump01" });
Assert.Equal([10], lower.Objects.Select(o => o.GobjectId));
// '?' single-char wildcard: "Pump0?" matches "Pump01".
GalaxyHierarchyQueryResult single = GalaxyHierarchyProjector.Project(entry, new DiscoverHierarchyRequest { TagNameGlob = "Pump0?" });
Assert.Equal([10], single.Objects.Select(o => o.GobjectId));
// Anchored: a bare substring that is not a prefix matches nothing.
GalaxyHierarchyQueryResult anchored = GalaxyHierarchyProjector.Project(entry, new DiscoverHierarchyRequest { TagNameGlob = "ump01" });
Assert.Empty(anchored.Objects);
}
[Fact]
public void Project_CategoryIds_FilterByObjectCategory()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
DiscoverHierarchyRequest request = new() { CategoryIds = { 200, 201 } };
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, request);
// category 200 = Pump01(10), category 201 = Valve01(11).
Assert.Equal([10, 11], result.Objects.Select(o => o.GobjectId).OrderBy(id => id));
}
[Fact]
public void Project_TemplateChainContains_IsSubstringAndCaseInsensitive()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
DiscoverHierarchyRequest request = new() { TemplateChainContains = { "pump" } };
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, request);
GalaxyObject only = Assert.Single(result.Objects);
Assert.Equal(10, only.GobjectId);
}
[Fact]
public void Project_IncludeAttributesDefault_CarriesAttributes()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
DiscoverHierarchyRequest request = new() { TagNameGlob = "Pump*" };
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, request);
GalaxyObject pump = Assert.Single(result.Objects);
Assert.Equal(2, pump.Attributes.Count);
}
[Fact]
public void Project_IncludeAttributesFalse_ReturnsSkeletons()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
DiscoverHierarchyRequest request = new() { TagNameGlob = "Pump*", IncludeAttributes = false };
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, request);
GalaxyObject pump = Assert.Single(result.Objects);
Assert.Empty(pump.Attributes);
}
[Fact]
public void Project_IncludeAttributesFalse_DoesNotMutateTheCachedEntry()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
// Project with attributes stripped, then again with attributes included.
GalaxyHierarchyProjector.Project(entry, new DiscoverHierarchyRequest { TagNameGlob = "Pump*", IncludeAttributes = false });
GalaxyHierarchyQueryResult included = GalaxyHierarchyProjector.Project(entry, new DiscoverHierarchyRequest { TagNameGlob = "Pump*" });
// The earlier strip cloned the object — the cached entry still holds the attributes.
GalaxyObject pump = Assert.Single(included.Objects);
Assert.Equal(2, pump.Attributes.Count);
}
[Fact]
public void Project_InvalidOffsetOrPageSize_Throws()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
Assert.Throws<ArgumentOutOfRangeException>(() =>
GalaxyHierarchyProjector.Project(entry, new DiscoverHierarchyRequest(), null, offset: -1, pageSize: 10));
Assert.Throws<ArgumentOutOfRangeException>(() =>
GalaxyHierarchyProjector.Project(entry, new DiscoverHierarchyRequest(), null, offset: 0, pageSize: 0));
}
// ---- GalaxyBrowseProjector ----
[Fact]
public void ProjectChildren_OfPlantArea_ReturnsDirectChildrenAreasFirst()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
BrowseChildrenRequest request = new() { ParentGobjectId = 1 };
GalaxyBrowseChildrenResult result = GalaxyBrowseProjector.ProjectChildren(entry, request, browseSubtreeGlobs: null, offset: 0, pageSize: 100);
// Direct children of PlantArea(1) are LineA(2, area) and Mixer01(12, non-area);
// areas sort first.
Assert.Equal([2, 12], result.Children.Select(c => c.GobjectId));
Assert.Equal(2, result.TotalChildCount);
}
[Fact]
public void ProjectChildren_ChildHasChildrenFlag_ReflectsPresenceOfChildren()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
BrowseChildrenRequest request = new() { ParentGobjectId = 1 };
GalaxyBrowseChildrenResult result = GalaxyBrowseProjector.ProjectChildren(entry, request, browseSubtreeGlobs: null, offset: 0, pageSize: 100);
Dictionary<int, bool> hasChildren = result.Children
.Select((child, index) => (child.GobjectId, result.ChildHasChildren[index]))
.ToDictionary(t => t.GobjectId, t => t.Item2);
// LineA(2) contains Pump01/Valve01 -> true; Mixer01(12) is a leaf -> false.
Assert.True(hasChildren[2]);
Assert.False(hasChildren[12]);
}
[Fact]
public void ProjectChildren_OfRoot_ReturnsTopLevelObjects()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
// Empty parent oneof => roots (parent id 0).
BrowseChildrenRequest request = new();
GalaxyBrowseChildrenResult result = GalaxyBrowseProjector.ProjectChildren(entry, request, browseSubtreeGlobs: null, offset: 0, pageSize: 100);
// Roots: PlantArea(1, area) and StandaloneTank(20, non-area); areas first.
Assert.Equal([1, 20], result.Children.Select(c => c.GobjectId));
}
[Fact]
public void ProjectChildren_FilterMatchingDescendant_SurfacesNonMatchingAncestor()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
// Pump01 lives two levels under PlantArea. Browsing PlantArea's children with a
// Pump glob should still surface LineA (which itself does not match) because it
// contains a matching descendant.
BrowseChildrenRequest request = new() { ParentGobjectId = 1, TagNameGlob = "Pump*" };
GalaxyBrowseChildrenResult result = GalaxyBrowseProjector.ProjectChildren(entry, request, browseSubtreeGlobs: null, offset: 0, pageSize: 100);
GalaxyObject surfaced = Assert.Single(result.Children);
Assert.Equal(2, surfaced.GobjectId);
Assert.True(result.ChildHasChildren[0]);
}
[Fact]
public void ProjectChildren_UnknownParent_ThrowsNotFound()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
BrowseChildrenRequest request = new() { ParentGobjectId = 99999 };
RpcException ex = Assert.Throws<RpcException>(() =>
GalaxyBrowseProjector.ProjectChildren(entry, request, null, 0, 100));
Assert.Equal(StatusCode.NotFound, ex.StatusCode);
}
[Fact]
public void ProjectChildren_Paging_SlicesAndPreservesTotal()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
// LineA(2) has two direct children: Pump01, Valve01.
BrowseChildrenRequest request = new() { ParentGobjectId = 2 };
GalaxyBrowseChildrenResult page1 = GalaxyBrowseProjector.ProjectChildren(entry, request, null, offset: 0, pageSize: 1);
GalaxyBrowseChildrenResult page2 = GalaxyBrowseProjector.ProjectChildren(entry, request, null, offset: 1, pageSize: 1);
Assert.Equal(2, page1.TotalChildCount);
Assert.Single(page1.Children);
Assert.Single(page2.Children);
Assert.NotEqual(page1.Children[0].GobjectId, page2.Children[0].GobjectId);
// Same filter+parent => same signature on both pages.
Assert.Equal(page1.FilterSignature, page2.FilterSignature);
}
[Fact]
public void ResolveParentId_ByTagName_ResolvesToGobjectId()
{
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
BrowseChildrenRequest request = new() { ParentTagName = "LineA" };
int id = GalaxyBrowseProjector.ResolveParentId(entry, request);
Assert.Equal(2, id);
}
}
@@ -0,0 +1,84 @@
using Microsoft.Extensions.Options;
using ZB.MOM.WW.GalaxyRepository;
namespace ZB.MOM.WW.GalaxyRepository.Tests;
/// <summary>
/// Round-trip tests for the real <see cref="GalaxyHierarchySnapshotStore"/> over a temp
/// file path: save then load, no-op when persistence is disabled, and clean disposal.
/// </summary>
public sealed class GalaxyHierarchySnapshotStoreTests : IDisposable
{
private readonly string _path = Path.Combine(
Path.GetTempPath(),
$"galaxyrepo-snap-{Guid.NewGuid():N}.json");
public void Dispose()
{
if (File.Exists(_path))
{
File.Delete(_path);
}
}
private static GalaxyHierarchySnapshot SampleSnapshot() => new(
LastDeployTime: new DateTimeOffset(2026, 1, 1, 0, 0, 0, TimeSpan.Zero),
SavedAt: new DateTimeOffset(2026, 1, 1, 12, 0, 0, TimeSpan.Zero),
Hierarchy:
[
new GalaxyHierarchyRow { GobjectId = 1, TagName = "Area1", IsArea = true },
new GalaxyHierarchyRow { GobjectId = 2, TagName = "Pump01", ParentGobjectId = 1 },
],
Attributes:
[
new GalaxyAttributeRow { GobjectId = 2, AttributeName = "PV", FullTagReference = "Pump01.PV", IsHistorized = true },
]);
[Fact]
public async Task SaveThenLoad_RoundTripsTheSnapshot()
{
using GalaxyHierarchySnapshotStore store = new(
Options.Create(new GalaxyRepositoryOptions { PersistSnapshot = true, SnapshotCachePath = _path }));
await store.SaveAsync(SampleSnapshot(), CancellationToken.None);
GalaxyHierarchySnapshot? loaded = await store.TryLoadAsync(CancellationToken.None);
Assert.NotNull(loaded);
Assert.Equal(2, loaded!.Hierarchy.Count);
Assert.Single(loaded.Attributes);
Assert.Equal("Pump01.PV", loaded.Attributes[0].FullTagReference);
Assert.True(loaded.Attributes[0].IsHistorized);
Assert.Equal(SampleSnapshot().LastDeployTime, loaded.LastDeployTime);
}
[Fact]
public async Task SaveAndLoad_AreNoOps_WhenPersistenceDisabled()
{
using GalaxyHierarchySnapshotStore store = new(
Options.Create(new GalaxyRepositoryOptions { PersistSnapshot = false, SnapshotCachePath = _path }));
await store.SaveAsync(SampleSnapshot(), CancellationToken.None);
Assert.False(File.Exists(_path));
Assert.Null(await store.TryLoadAsync(CancellationToken.None));
}
[Fact]
public async Task TryLoad_ReturnsNull_WhenNoFileExists()
{
using GalaxyHierarchySnapshotStore store = new(
Options.Create(new GalaxyRepositoryOptions { PersistSnapshot = true, SnapshotCachePath = _path }));
Assert.Null(await store.TryLoadAsync(CancellationToken.None));
}
[Fact]
public async Task TryLoad_ReturnsNull_WhenFileIsNotValidJson()
{
await File.WriteAllTextAsync(_path, "{ this is not valid json");
using GalaxyHierarchySnapshotStore store = new(
Options.Create(new GalaxyRepositoryOptions { PersistSnapshot = true, SnapshotCachePath = _path }));
Assert.Null(await store.TryLoadAsync(CancellationToken.None));
}
}
@@ -0,0 +1,25 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<IsPackable>false</IsPackable>
<!-- Test project does not ship; no XML docs required (overrides Directory.Build.props). -->
<GenerateDocumentationFile>false</GenerateDocumentationFile>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="coverlet.collector" />
<PackageReference Include="Microsoft.NET.Test.Sdk" />
<PackageReference Include="xunit" />
<PackageReference Include="xunit.runner.visualstudio" />
<PackageReference Include="Microsoft.Data.SqlClient" />
</ItemGroup>
<ItemGroup>
<Using Include="Xunit" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\..\src\ZB.MOM.WW.GalaxyRepository\ZB.MOM.WW.GalaxyRepository.csproj" />
</ItemGroup>
</Project>
@@ -0,0 +1,122 @@
# ZB.MOM.WW.SPHistorianClient — Design
**Date:** 2026-06-19
**Status:** Approved — proceeding to implementation plan.
## Goal
Repackage the proven, pure-managed .NET 10 `AVEVA.Historian.Client` SDK (delivered in
`HistorianSDK_2023R2/histsdk-migration.zip` from `10.100.0.48`) as the family-branded shared
library **`ZB.MOM.WW.SPHistorianClient`** (System Platform Historian Client), following the same
conventions as the other `ZB.MOM.WW.*` shared libraries in this repo.
## Context — what the source bundle contains
`histsdk-migration.zip``histsdk-migration/`:
- `histsdk/` — the SDK git repo. `src/AVEVA.Historian.Client/` is a **pure-managed .NET 10** client
for AVEVA Historian (no `aahClientManaged.dll` / `aahClient.dll` / native AVEVA runtime — the wire
protocol is reverse-engineered and re-implemented in C#). ~165188 unit + gated-live tests pass.
- `analysis-2023r2/` — reverse-engineering analysis (recovered protos, decompiled stock contract,
transport writeup). **Kept separate from the repo on purpose.**
Two transport families exist in the SDK:
| Transport | Protocol | Platform | Verification |
|---|---|---|---|
| `LocalPipe`, `RemoteTcpIntegrated`, `RemoteTcpCertificate` | WCF/MDAS (2020) | **Windows-only** | **live-verified**: raw/aggregate(16 modes)/at-time/event reads, browse, metadata, status, `EnsureTag`/`DeleteTag` |
| `RemoteGrpc` | gRPC (2023 R2) | cross-platform (Grpc.Net.Client/.Web) | unit-tested; **not yet live-verified** against a real 2023 R2 server (`ExchangeKey` auth step unproven) |
## Decisions (locked)
1. **Approach: port + rebrand.** Copy the SDK source into `ZB.MOM.WW.SPHistorianClient`, rename the
root namespace, adopt ZB conventions, bring the unit tests, drop non-shippable artifacts. One
coherent shared library — a published package should not ship a third-party (AVEVA) namespace or
non-redistributable reverse-engineering artifacts.
2. **Transports: both WCF + gRPC.** Ship everything that works. WCF members keep
`[SupportedOSPlatform("windows")]`; the gRPC path runs anywhere. No working code discarded.
3. **Not a "component normalization."** There is no duplicated historian code across the three apps
to converge — this is a net-new shared library that simply follows ZB packaging conventions.
## Repository layout
Plain files committed into this repo (NOT a nested git repo — see the
`shared-libs-are-plain-files-not-nested-repos` convention):
```
ZB.MOM.WW.SPHistorianClient/
Directory.Build.props # net10.0, Nullable, ImplicitUsings, LangVersion latest, Version 0.1.0, central pkg mgmt
Directory.Packages.props # central PackageVersion entries
ZB.MOM.WW.SPHistorianClient.slnx
CLAUDE.md README.md .gitignore
src/ZB.MOM.WW.SPHistorianClient/ # the single package
HistorianClient.cs, HistorianClientOptions.cs, HistorianTransport.cs
Models/ Protocol/ Transport/ Wcf/ Wcf/Contracts/ Grpc/ Grpc/Protos/*.proto
DependencyInjection/AddZbSpHistorianClient (ZB-idiomatic DI extension)
tests/ZB.MOM.WW.SPHistorianClient.Tests/ # offline unit/golden-byte + gated-live integration
artifacts/ # dotnet pack output
```
## Port mechanics
- Copy `src/AVEVA.Historian.Client/` and `tests/AVEVA.Historian.Client.Tests/` from the bundle.
- Rename the C# root namespace `AVEVA.Historian.Client``ZB.MOM.WW.SPHistorianClient` across all
files: 74 `namespace` declarations spanning the root + 6 sub-namespaces
(`.Models`, `.Wcf`, `.Wcf.Contracts`, `.Protocol`, `.Transport`, `.Grpc`), all `using` directives,
and the `InternalsVisibleTo` to the test assembly. Drop the `InternalsVisibleTo` to
`AVEVA.Historian.ReverseEngineering` (tool not shipped).
- **Leave the proto wire contracts untouched:** the 6 `Grpc/Protos/*.proto` keep
`option csharp_namespace = "ArchestrA.Grpc.Contract.*"` — that is AVEVA's wire contract, not ours.
`Grpc.Tools` keeps generating the client stubs at build.
- Convert inline `PackageReference` versions to central management in `Directory.Packages.props`,
matching the `ZB.MOM.WW.Telemetry` template.
## Dependencies
- **Library:** `Google.Protobuf`, `Grpc.Net.Client`, `Grpc.Net.Client.Web`, `Grpc.Tools` (build-only,
`PrivateAssets=all`), `System.ServiceModel.NetNamedPipe`, `System.ServiceModel.NetTcp`,
`System.Security.Cryptography.Xml`. Add `Microsoft.Extensions.DependencyInjection.Abstractions` +
`Microsoft.Extensions.Options` for the DI extension.
- **Tests:** `xunit`, `xunit.runner.visualstudio`, `Microsoft.NET.Test.Sdk`, `coverlet.collector`,
`Microsoft.Data.SqlClient` (SQL post-check tests).
## Excluded (safety / non-redistributable / Windows-native)
- `tools/` reverse-engineering harnesses (.NET Framework, reference native AVEVA binaries).
- `analysis-2023r2/decompiled/` — proprietary AVEVA decompilations (not redistributable).
- `scripts/` — Frida / PowerShell / Python capture tooling.
- `docs/reverse-engineering/` — identity-bearing `.ndjson` / capture evidence.
**Kept:** the recovered `.proto` files (needed to build), the offline unit tests, and a sanitized
architecture/surface summary folded into `CLAUDE.md` / `README.md`. `.gitignore` blocks the
identity-bearing patterns (`*.ndjson`, `current/`, `aveva-install-*/`, `artifacts/`-raw, etc.).
## Public surface (preserved 1:1)
`HistorianClient` + `HistorianClientOptions` façade; `Models/*`; `HistorianTransport` enum
(`LocalPipe` / `RemoteTcpIntegrated` / `RemoteTcpCertificate` / `RemoteGrpc`); operations:
`ProbeAsync`, `ReadRawAsync` / `ReadAggregateAsync` / `ReadAtTimeAsync`, `ReadEventsAsync`,
`BrowseTagNamesAsync`, `GetTagMetadataAsync`, status calls, `EnsureTagAsync` / `DeleteTagAsync`.
**One ZB-idiomatic addition:** `AddZbSpHistorianClient(...)` DI extension mirroring `AddZbTelemetry`
— thin: binds `HistorianClientOptions` and registers `HistorianClient`. Optional to consumers.
## Cross-platform & testing posture
- WCF members already carry `[SupportedOSPlatform("windows")]`; the library builds and unit-tests on
macOS/Linux. gRPC path is portable.
- Offline unit/golden-byte tests run anywhere. Live integration tests stay gated by `HISTORIAN_*`
env vars and skip cleanly when unset.
- Verify `dotnet build` + `dotnet test` pass locally (macOS) before finishing.
## Packaging
`dotnet pack -c Release -o ./artifacts``ZB.MOM.WW.SPHistorianClient.0.1.0.nupkg`. Gitea URLs in
package metadata. **Not pushed/published** to any feed unless explicitly requested.
## Out of scope (this pass)
- Wiring `ZB.MOM.WW.SPHistorianClient` into any consumer (e.g. OtOpcUa Phase C HistoryRead) — a
separate follow-on.
- Live-verifying the gRPC `RemoteGrpc` path against a real 2023 R2 server.
- Writing samples (`AddS2`) — architecturally blocked in the source SDK; remains out of scope.
+569
View File
@@ -0,0 +1,569 @@
# ZB.MOM.WW.SPHistorianClient Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans (or subagent-driven-development) to implement this plan task-by-task.
**Goal:** Repackage the proven, pure-managed .NET 10 `AVEVA.Historian.Client` SDK from the migration bundle as the family-branded shared library `ZB.MOM.WW.SPHistorianClient`, following the same conventions as the other `ZB.MOM.WW.*` libraries in this repo.
**Architecture:** This is a **port + rebrand**, not a rewrite. Copy the SDK `src/` and `tests/` into a new `ZB.MOM.WW.SPHistorianClient/` directory, rewrite the C# root namespace `AVEVA.Historian.Client``ZB.MOM.WW.SPHistorianClient` (leaving the proto-generated `ArchestrA.Grpc.Contract.*` wire contracts untouched), adopt ZB conventions (`Directory.Build.props` / `Directory.Packages.props` central package management, `.slnx`, `CLAUDE.md`/`README.md`), drop the non-shippable reverse-engineering tooling and proprietary decompilations, add one ZB-idiomatic DI extension, then build/test/pack.
**Tech Stack:** .NET 10, C# (net10.0), WCF/MDAS (`System.ServiceModel.*`, Windows-only transports), gRPC (`Grpc.Net.Client` + `Grpc.Tools`, cross-platform 2023 R2 transport), xUnit. Central package management.
**Design doc:** `docs/plans/2026-06-19-sphistorianclient-design.md`
**Branch:** `feat/sphistorianclient` (already created; design doc already committed at `bbb7942`).
---
## Source bundle location (read-only inputs)
The SDK source lives in an extracted bundle under `/tmp`:
- Extracted root: `/tmp/histsdk/extracted/histsdk-migration/histsdk/`
- SDK source: `…/histsdk/src/AVEVA.Historian.Client/`**74 `.cs` + 6 `.proto`**
- SDK tests: `…/histsdk/tests/AVEVA.Historian.Client.Tests/`**25 `.cs`**
- Re-extract fallback (if `/tmp` was cleaned): `cd /tmp/histsdk && unzip -o -q histsdk-migration.zip -d extracted`
**Never copy:** `tools/` (RE harnesses, .NET Framework + native AVEVA refs), `analysis-2023r2/decompiled/` (proprietary, non-redistributable), `scripts/`, `docs/reverse-engineering/` (identity-bearing captures), `bin/`/`obj/`, the bundle's `.git/`, and the bundle's original `.csproj` files (we author fresh ZB ones).
**Gotchas baked into this plan (from prior repo experience):**
- Do **not** set `TreatWarningsAsErrors` — the WCF/SSPI code carries `[SupportedOSPlatform("windows")]` and will emit CA platform warnings on macOS that must stay warnings.
- Central package management means **no inline `Version=` on any `PackageReference`** (that is `NU1008`). All versions live in `Directory.Packages.props`.
- `Microsoft.Data.SqlClient` may surface an `NU1903` advisory on restore. Without `TreatWarningsAsErrors` it is a warning. If a restore ever hard-fails on it, add `-p:NuGetAudit=false` to the build/test command.
- macOS `sed -i` requires an explicit empty backup arg: `sed -i '' 's/…/…/g'`.
---
## Task 1: Scaffold the library skeleton
**Classification:** standard
**Estimated implement time:** ~4 min
**Parallelizable with:** none (every later task depends on this)
**Files:**
- Create: `ZB.MOM.WW.SPHistorianClient/Directory.Build.props`
- Create: `ZB.MOM.WW.SPHistorianClient/Directory.Packages.props`
- Create: `ZB.MOM.WW.SPHistorianClient/.gitignore`
- Create: `ZB.MOM.WW.SPHistorianClient/ZB.MOM.WW.SPHistorianClient.slnx`
- Create: `ZB.MOM.WW.SPHistorianClient/src/ZB.MOM.WW.SPHistorianClient/ZB.MOM.WW.SPHistorianClient.csproj`
- Create: `ZB.MOM.WW.SPHistorianClient/tests/ZB.MOM.WW.SPHistorianClient.Tests/ZB.MOM.WW.SPHistorianClient.Tests.csproj`
**Step 1: `Directory.Build.props`** (mirrors `ZB.MOM.WW.Telemetry/Directory.Build.props`)
```xml
<Project>
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<LangVersion>latest</LangVersion>
<Version>0.1.0</Version>
<ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
</PropertyGroup>
</Project>
```
**Step 2: `Directory.Packages.props`** (versions lifted verbatim from the bundle's two `.csproj` files)
```xml
<Project>
<PropertyGroup>
<ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
</PropertyGroup>
<ItemGroup>
<!-- Historian SDK runtime deps (WCF/MDAS transports — Windows-only at runtime) -->
<PackageVersion Include="System.Security.Cryptography.Xml" Version="10.0.7" />
<PackageVersion Include="System.ServiceModel.NetNamedPipe" Version="10.0.652802" />
<PackageVersion Include="System.ServiceModel.NetTcp" Version="10.0.652802" />
<!-- 2023 R2 gRPC transport (cross-platform) -->
<PackageVersion Include="Google.Protobuf" Version="3.24.4" />
<PackageVersion Include="Grpc.Net.Client" Version="2.58.0" />
<PackageVersion Include="Grpc.Net.Client.Web" Version="2.58.0" />
<PackageVersion Include="Grpc.Tools" Version="2.59.0" />
<!-- ZB-idiomatic DI extension (only non-BCL lib dependency) -->
<PackageVersion Include="Microsoft.Extensions.DependencyInjection.Abstractions" Version="10.0.7" />
<!-- Test -->
<PackageVersion Include="Microsoft.NET.Test.Sdk" Version="17.14.1" />
<PackageVersion Include="xunit" Version="2.9.3" />
<PackageVersion Include="xunit.runner.visualstudio" Version="3.1.4" />
<PackageVersion Include="coverlet.collector" Version="6.0.4" />
<PackageVersion Include="Microsoft.Data.SqlClient" Version="6.0.2" />
</ItemGroup>
</Project>
```
**Step 3: `.gitignore`**
```gitignore
bin/
obj/
# identity-bearing / non-redistributable — never commit
*.ndjson
current/
aveva-install-*/
```
**Step 4: `ZB.MOM.WW.SPHistorianClient.slnx`**
```xml
<Solution>
<Folder Name="/src/">
<Project Path="src/ZB.MOM.WW.SPHistorianClient/ZB.MOM.WW.SPHistorianClient.csproj" />
</Folder>
<Folder Name="/tests/">
<Project Path="tests/ZB.MOM.WW.SPHistorianClient.Tests/ZB.MOM.WW.SPHistorianClient.Tests.csproj" />
</Folder>
</Solution>
```
**Step 5: `src/ZB.MOM.WW.SPHistorianClient/ZB.MOM.WW.SPHistorianClient.csproj`**
(Derived from the bundle's `AVEVA.Historian.Client.csproj`: inline versions removed for central
management; ZB package metadata added; `InternalsVisibleTo` retargeted to the ZB test assembly and
the `…ReverseEngineering` one dropped; proto glob uses forward slashes for cross-platform MSBuild.)
```xml
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<IsPackable>true</IsPackable>
<PackageId>ZB.MOM.WW.SPHistorianClient</PackageId>
<Authors>ZB.MOM.WW</Authors>
<Description>Pure-managed .NET 10 client for AVEVA System Platform Historian (Wonderware) for the ZB.MOM.WW SCADA family. The wire protocol is reverse-engineered and re-implemented in C# — no native AVEVA runtime dependency. Surfaces history reads (raw / aggregate / at-time / event), tag browse + metadata, status, and tag create/delete over the WCF/MDAS transports (Windows) plus a cross-platform gRPC transport for 2023 R2.</Description>
<PackageTags>aveva;wonderware;historian;system-platform;scada;timeseries;grpc;wcf;zb-mom-ww</PackageTags>
<PackageProjectUrl>https://gitea.dohertylan.com/dohertj2/zb-mom-ww-sphistorianclient</PackageProjectUrl>
<RepositoryUrl>https://gitea.dohertylan.com/dohertj2/zb-mom-ww-sphistorianclient</RepositoryUrl>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="System.Security.Cryptography.Xml" />
<PackageReference Include="System.ServiceModel.NetNamedPipe" />
<PackageReference Include="System.ServiceModel.NetTcp" />
<PackageReference Include="Microsoft.Extensions.DependencyInjection.Abstractions" />
</ItemGroup>
<!-- 2023 R2 gRPC transport (RemoteGrpc). Pure-managed: Grpc.Net.Client + Google.Protobuf.
Grpc.Tools is build-only (PrivateAssets=all) and generates the client stubs from the
recovered contract under Grpc/Protos at build. -->
<ItemGroup>
<PackageReference Include="Google.Protobuf" />
<PackageReference Include="Grpc.Net.Client" />
<PackageReference Include="Grpc.Net.Client.Web" />
<PackageReference Include="Grpc.Tools">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
</ItemGroup>
<ItemGroup>
<Protobuf Include="Grpc/Protos/*.proto" GrpcServices="Client" ProtoRoot="Grpc/Protos" />
</ItemGroup>
<ItemGroup>
<AssemblyAttribute Include="System.Runtime.CompilerServices.InternalsVisibleToAttribute">
<_Parameter1>ZB.MOM.WW.SPHistorianClient.Tests</_Parameter1>
</AssemblyAttribute>
</ItemGroup>
</Project>
```
**Step 6: `tests/ZB.MOM.WW.SPHistorianClient.Tests/ZB.MOM.WW.SPHistorianClient.Tests.csproj`**
```xml
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<IsPackable>false</IsPackable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="coverlet.collector" />
<PackageReference Include="Microsoft.NET.Test.Sdk" />
<PackageReference Include="Microsoft.Data.SqlClient" />
<PackageReference Include="xunit" />
<PackageReference Include="xunit.runner.visualstudio" />
</ItemGroup>
<ItemGroup>
<Using Include="Xunit" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\..\src\ZB.MOM.WW.SPHistorianClient\ZB.MOM.WW.SPHistorianClient.csproj" />
</ItemGroup>
</Project>
```
**Step 7: Verify the skeleton is well-formed (build will fail — no sources yet — that is expected)**
Run: `cd ZB.MOM.WW.SPHistorianClient && dotnet restore ZB.MOM.WW.SPHistorianClient.slnx`
Expected: restore **succeeds** (proves the props/csproj XML and central package versions resolve). A
follow-up `dotnet build` would fail only because no `.cs` exist yet — do not build here.
**Step 8: Commit**
```bash
cd /Users/dohertj2/Desktop/scadaproj
git add ZB.MOM.WW.SPHistorianClient/
git commit -m "feat(sphistorianclient): scaffold shared library skeleton (props, csprojs, slnx)"
```
---
## Task 2: Port source + tests with namespace rewrite
**Classification:** standard
**Estimated implement time:** ~4 min
**Parallelizable with:** none
**Blocked by:** Task 1
**Files:**
- Create (scripted copy): `ZB.MOM.WW.SPHistorianClient/src/ZB.MOM.WW.SPHistorianClient/**/*.{cs,proto}` (74 `.cs` + 6 `.proto`)
- Create (scripted copy): `ZB.MOM.WW.SPHistorianClient/tests/ZB.MOM.WW.SPHistorianClient.Tests/**/*.cs` (25 `.cs`)
This task is a deterministic copy + namespace rewrite — run the script, then verify counts.
**Step 1: Copy + rewrite (single script)**
```bash
set -euo pipefail
BUNDLE=/tmp/histsdk/extracted/histsdk-migration/histsdk
DEST=/Users/dohertj2/Desktop/scadaproj/ZB.MOM.WW.SPHistorianClient
# Guard: re-extract if /tmp was cleaned
if [ ! -d "$BUNDLE/src/AVEVA.Historian.Client" ]; then
cd /tmp/histsdk && unzip -o -q histsdk-migration.zip -d extracted
fi
# --- src: copy .cs + .proto, preserving subdirs (NOT the old .csproj) ---
SRC="$DEST/src/ZB.MOM.WW.SPHistorianClient"
cd "$BUNDLE/src/AVEVA.Historian.Client"
find . \( -name '*.cs' -o -name '*.proto' \) | while read -r f; do
mkdir -p "$SRC/$(dirname "$f")"
cp "$f" "$SRC/$f"
done
# --- tests: copy .cs only (NOT the old .csproj) ---
TST="$DEST/tests/ZB.MOM.WW.SPHistorianClient.Tests"
cd "$BUNDLE/tests/AVEVA.Historian.Client.Tests"
find . -name '*.cs' | while read -r f; do
mkdir -p "$TST/$(dirname "$f")"
cp "$f" "$TST/$f"
done
# --- namespace rewrite in .cs ONLY (proto wire contracts stay ArchestrA.Grpc.Contract.*) ---
find "$SRC" "$TST" -name '*.cs' -print0 \
| xargs -0 sed -i '' 's/AVEVA\.Historian\.Client/ZB.MOM.WW.SPHistorianClient/g'
```
**Step 2: Verify counts and that the rename is total**
```bash
DEST=/Users/dohertj2/Desktop/scadaproj/ZB.MOM.WW.SPHistorianClient
echo "src cs: $(find "$DEST/src" -name '*.cs' | wc -l) (expect 74)"
echo "src proto: $(find "$DEST/src" -name '*.proto' | wc -l) (expect 6)"
echo "test cs: $(find "$DEST/tests" -name '*.cs' | wc -l) (expect 25)"
echo "leftover AVEVA.Historian.Client in .cs: $(grep -rl 'AVEVA\.Historian\.Client' "$DEST" --include='*.cs' | wc -l) (expect 0)"
echo "proto namespace preserved: $(grep -l 'ArchestrA.Grpc.Contract' "$DEST"/src/ZB.MOM.WW.SPHistorianClient/Grpc/Protos/*.proto | wc -l) (expect 6)"
```
Expected: `74`, `6`, `25`, `0`, `6`. If "leftover" is non-zero, inspect those files — the only legitimate
remaining mentions would be inside comments/strings that happen to differ in casing/spacing; a clean
port should show `0`.
**Step 3: Commit**
```bash
cd /Users/dohertj2/Desktop/scadaproj
git add ZB.MOM.WW.SPHistorianClient/src ZB.MOM.WW.SPHistorianClient/tests
git commit -m "feat(sphistorianclient): port SDK source + tests, rebrand namespace to ZB.MOM.WW.SPHistorianClient"
```
---
## Task 3: Build + test green
**Classification:** high-risk
**Estimated implement time:** ~5 min (plus restore/build wall-time)
**Parallelizable with:** none
**Blocked by:** Task 2
This is the integration gate. The port must compile and the offline test suite must pass on this macOS host.
**Files:**
- Modify (only if the build surfaces a defect): any ported file under `ZB.MOM.WW.SPHistorianClient/src` or `…/tests`, or the two `.csproj`.
**Step 1: Build**
Run: `cd ZB.MOM.WW.SPHistorianClient && dotnet build ZB.MOM.WW.SPHistorianClient.slnx`
Expected: **Build succeeded.** Platform-compatibility (CAxxxx `[SupportedOSPlatform("windows")]`) warnings
are acceptable and must remain warnings. If restore hard-fails on `NU1903`, re-run with
`-p:NuGetAudit=false`.
**Step 2: Test**
Run: `dotnet test ZB.MOM.WW.SPHistorianClient.slnx`
Expected: all tests pass; the live integration tests (`HistorianClientIntegrationTests`,
`HistorianGrpcIntegrationTests`, `RemoteTcpIntegrationTests`) **skip cleanly** because no `HISTORIAN_*`
env vars are set. The bundle's `MIGRATION-README.md` documents ~188 tests passing on macOS with the
live ones skipped — treat a comparable count with **zero failures** as success.
**Step 3: Triage rules (if not green)**
- Compile error referencing `AVEVA.Historian.Client` → a file was missed by the rewrite; re-run the
Task 2 sed on that file.
- `NU1008` (version on PackageReference) → an inline `Version=` slipped into a `.csproj`; remove it
(version belongs in `Directory.Packages.props`).
- Missing generated gRPC type (e.g. `ArchestrA.Grpc.Contract.*` not found) → confirm the `<Protobuf>`
glob in the src `.csproj` resolves the 6 `Grpc/Protos/*.proto` and that `Grpc.Tools` restored.
- A genuine test failure (not a skip) → this is a real port defect; fix the ported code, do **not**
delete/weaken the test.
**Step 4: Commit (only if Step 3 required edits)**
```bash
git add -A ZB.MOM.WW.SPHistorianClient/
git commit -m "fix(sphistorianclient): resolve port build/test fallout"
```
---
## Task 4: Add the `AddZbSpHistorianClient` DI extension (TDD)
**Classification:** standard
**Estimated implement time:** ~4 min
**Parallelizable with:** Task 5
**Blocked by:** Task 3
`HistorianClientOptions` uses `required` + `init`-only properties, so the extension takes a fully-built
options instance (not an `Action<T>` configurator). It depends only on
`Microsoft.Extensions.DependencyInjection.Abstractions`.
**Files:**
- Create: `ZB.MOM.WW.SPHistorianClient/src/ZB.MOM.WW.SPHistorianClient/DependencyInjection/ZbSpHistorianClientServiceCollectionExtensions.cs`
- Test: `ZB.MOM.WW.SPHistorianClient/tests/ZB.MOM.WW.SPHistorianClient.Tests/DependencyInjectionTests.cs`
**Step 1: Write the failing test**
```csharp
using Microsoft.Extensions.DependencyInjection;
using ZB.MOM.WW.SPHistorianClient;
namespace ZB.MOM.WW.SPHistorianClient.Tests;
public class DependencyInjectionTests
{
[Fact]
public void AddZbSpHistorianClient_resolves_client_and_options()
{
var services = new ServiceCollection();
var options = new HistorianClientOptions { Host = "localhost" };
services.AddZbSpHistorianClient(options);
using var sp = services.BuildServiceProvider();
Assert.Same(options, sp.GetRequiredService<HistorianClientOptions>());
Assert.NotNull(sp.GetRequiredService<HistorianClient>());
}
[Fact]
public void AddZbSpHistorianClient_throws_when_host_missing()
{
var services = new ServiceCollection();
var options = new HistorianClientOptions { Host = "" };
Assert.Throws<ArgumentException>(() => services.AddZbSpHistorianClient(options));
}
[Fact]
public void AddZbSpHistorianClient_throws_on_null_options()
{
var services = new ServiceCollection();
Assert.Throws<ArgumentNullException>(() => services.AddZbSpHistorianClient(null!));
}
}
```
**Step 2: Run — verify it fails to compile** (`AddZbSpHistorianClient` not defined)
Run: `dotnet test ZB.MOM.WW.SPHistorianClient.slnx --filter "FullyQualifiedName~DependencyInjectionTests"`
Expected: FAIL (does not compile / method missing).
**Step 3: Implement**
```csharp
using Microsoft.Extensions.DependencyInjection;
namespace ZB.MOM.WW.SPHistorianClient;
/// <summary>
/// ZB.MOM.WW DI registration for <see cref="HistorianClient"/>. Mirrors the family's
/// <c>AddZb*</c> convention. Because <see cref="HistorianClientOptions"/> is <c>required</c>/
/// <c>init</c>-only, callers pass a fully-built options instance (bind it from configuration in the
/// consuming app, e.g. <c>config.GetSection("Historian").Get&lt;HistorianClientOptions&gt;()</c>).
/// </summary>
public static class ZbSpHistorianClientServiceCollectionExtensions
{
public static IServiceCollection AddZbSpHistorianClient(
this IServiceCollection services,
HistorianClientOptions options)
{
ArgumentNullException.ThrowIfNull(services);
ArgumentNullException.ThrowIfNull(options);
if (string.IsNullOrWhiteSpace(options.Host))
{
throw new ArgumentException(
"HistorianClientOptions.Host must be set.", nameof(options));
}
services.AddSingleton(options);
// HistorianClient opens a fresh channel per operation and has a no-op DisposeAsync,
// so transient is safe and avoids assuming the shared dialect is concurrency-safe.
services.AddTransient<HistorianClient>();
return services;
}
}
```
**Step 4: Run — verify pass**
Run: `dotnet test ZB.MOM.WW.SPHistorianClient.slnx --filter "FullyQualifiedName~DependencyInjectionTests"`
Expected: PASS (3/3).
**Step 5: Commit**
```bash
git add ZB.MOM.WW.SPHistorianClient/src/ZB.MOM.WW.SPHistorianClient/DependencyInjection \
ZB.MOM.WW.SPHistorianClient/tests/ZB.MOM.WW.SPHistorianClient.Tests/DependencyInjectionTests.cs
git commit -m "feat(sphistorianclient): add AddZbSpHistorianClient DI extension"
```
---
## Task 5: Author `CLAUDE.md` + `README.md`
**Classification:** small
**Estimated implement time:** ~4 min
**Parallelizable with:** Task 4
**Blocked by:** Task 3
Sanitized docs only — **no hostnames, credentials, customer tag names, or capture data.** Model the
structure on `ZB.MOM.WW.Telemetry/CLAUDE.md` (overview, package table, build/test/pack commands,
status) but adapt to a single-package library.
**Files:**
- Create: `ZB.MOM.WW.SPHistorianClient/CLAUDE.md`
- Create: `ZB.MOM.WW.SPHistorianClient/README.md`
**`CLAUDE.md` must cover:**
- One-paragraph overview: pure-managed .NET 10 AVEVA System Platform Historian client, no native AVEVA
dependency, reverse-engineered wire protocol. Ported from the `histsdk` migration bundle.
- The supported operation surface table (copy the README table from the bundle:
`ProbeAsync`, `ReadRawAsync`, `ReadAggregateAsync` (16 modes), `ReadAtTimeAsync`, `ReadEventsAsync`,
`BrowseTagNamesAsync`, `GetTagMetadataAsync`, `GetConnectionStatusAsync`,
`GetStoreForwardStatusAsync`, `GetSystemParameterAsync`, `EnsureTagAsync`, `DeleteTagAsync`).
- Transport matrix: `LocalPipe` / `RemoteTcpIntegrated` / `RemoteTcpCertificate` (WCF, Windows-only,
live-verified) vs `RemoteGrpc` (2023 R2, cross-platform, **not yet live-verified**).
- Out of scope: writing samples (`AddS2` architecturally blocked), discrete/string tag creation.
- DI: the `AddZbSpHistorianClient(options)` extension + the bind-from-config note.
- Build/test/pack commands (from this dir):
`dotnet build ZB.MOM.WW.SPHistorianClient.slnx` / `dotnet test …` /
`dotnet pack ZB.MOM.WW.SPHistorianClient.slnx -c Release -o ./artifacts`.
- Live integration tests gated by `HISTORIAN_*` env vars (skip cleanly when unset). List the env vars.
**`README.md`:** a trimmed public-facing version — overview, quick-start snippet (the bundle's
`HistorianClient` usage example, namespace updated to `ZB.MOM.WW.SPHistorianClient`), supported surface
table, build/test commands.
**Commit:**
```bash
git add ZB.MOM.WW.SPHistorianClient/CLAUDE.md ZB.MOM.WW.SPHistorianClient/README.md
git commit -m "docs(sphistorianclient): add CLAUDE.md + README.md"
```
---
## Task 6: Pack verification
**Classification:** small
**Estimated implement time:** ~3 min
**Parallelizable with:** none
**Blocked by:** Task 4, Task 5
**Files:**
- Create (build output): `ZB.MOM.WW.SPHistorianClient/artifacts/ZB.MOM.WW.SPHistorianClient.0.1.0.nupkg`
**Step 1: Full green build + test once more, then pack**
```bash
cd /Users/dohertj2/Desktop/scadaproj/ZB.MOM.WW.SPHistorianClient
dotnet test ZB.MOM.WW.SPHistorianClient.slnx
dotnet pack ZB.MOM.WW.SPHistorianClient.slnx -c Release -o ./artifacts
```
Expected: tests pass (live ones skip); pack produces `artifacts/ZB.MOM.WW.SPHistorianClient.0.1.0.nupkg`.
**Step 2: Sanity-check the package contents**
```bash
unzip -l artifacts/ZB.MOM.WW.SPHistorianClient.0.1.0.nupkg | grep -E 'ZB.MOM.WW.SPHistorianClient.dll|.nuspec'
```
Expected: the lib DLL and nuspec are present.
**Step 3: Commit the nupkg** (matches the family convention — `ZB.MOM.WW.Telemetry` commits its `artifacts/*.nupkg`)
```bash
cd /Users/dohertj2/Desktop/scadaproj
git add -f ZB.MOM.WW.SPHistorianClient/artifacts/ZB.MOM.WW.SPHistorianClient.0.1.0.nupkg
git commit -m "build(sphistorianclient): pack 0.1.0 nupkg"
```
> **Do NOT push or publish** to the Gitea feed. Per repo experience, "published/adopted" claims must
> not be made without explicit user direction + feed verification.
---
## Task 7: Index the new library in the umbrella `CLAUDE.md` (optional)
**Classification:** trivial
**Estimated implement time:** ~2 min
**Parallelizable with:** none
**Blocked by:** Task 6
**Files:**
- Modify: `CLAUDE.md` (repo root umbrella index)
Add a short reference so the umbrella index reflects the newly-hosted library (the intro paragraph
that enumerates the hosted `ZB.MOM.WW.*` sources, and/or a one-line pointer near the component table
noting `ZB.MOM.WW.SPHistorianClient` is a net-new shared library — **not** a component normalization).
> **Caveat:** repo-root `CLAUDE.md` already has **pre-existing uncommitted edits** (unrelated to this
> work). Before editing, run `git diff CLAUDE.md` and make sure your commit message reflects that it
> may bundle those edits — or stage only the hunks you add. If this risks entangling unrelated changes,
> skip this task and leave it for the user.
**Commit:**
```bash
git add CLAUDE.md
git commit -m "docs: index ZB.MOM.WW.SPHistorianClient in umbrella CLAUDE.md"
```
---
## Done criteria
- `ZB.MOM.WW.SPHistorianClient/` exists with `src/`, `tests/`, props, `.slnx`, `CLAUDE.md`, `README.md`.
- `dotnet build` + `dotnet test` are green on macOS (live integration tests skip cleanly).
- `AddZbSpHistorianClient` DI extension present + tested.
- `artifacts/ZB.MOM.WW.SPHistorianClient.0.1.0.nupkg` produced.
- All work committed on `feat/sphistorianclient`. Not pushed/published.
@@ -0,0 +1,13 @@
{
"planPath": "docs/plans/2026-06-19-sphistorianclient.md",
"tasks": [
{"id": 1, "subject": "Task 1: Scaffold library skeleton", "status": "completed"},
{"id": 2, "subject": "Task 2: Port source + tests with namespace rewrite", "status": "completed", "blockedBy": [1]},
{"id": 3, "subject": "Task 3: Build + test green", "status": "completed", "blockedBy": [2]},
{"id": 4, "subject": "Task 4: Add AddZbSpHistorianClient DI extension (TDD)", "status": "completed", "blockedBy": [3]},
{"id": 5, "subject": "Task 5: Author CLAUDE.md + README.md", "status": "completed", "blockedBy": [3]},
{"id": 6, "subject": "Task 6: Pack verification", "status": "completed", "blockedBy": [4, 5]},
{"id": 7, "subject": "Task 7: Index new lib in umbrella CLAUDE.md (optional)", "status": "completed", "blockedBy": [6]}
],
"lastUpdated": "2026-06-19"
}
@@ -0,0 +1,216 @@
# ZB.MOM.WW.HistorianGateway — Design
**Date:** 2026-06-23
**Status:** Design approved (brainstorming complete) — ready for implementation planning
**Author:** brainstorming session (Joseph Doherty + Claude)
## 1. Summary
A new **full-feature sidecar** in the SCADA/OT sister-project family, modelled on
`MxAccessGateway` (`mxaccessgw`). It does two things:
1. **Read-only Galaxy metadata server** — exposes the AVEVA System Platform
("Wonderware") **Galaxy object hierarchy** (areas / objects / templates /
instances / attributes), sourced from the **Galaxy Repository SQL DB**, the same
data `mxaccessgw`'s `galaxy_repository` feature serves today.
2. **Full read/write gRPC API to the AVEVA (Wonderware) Historian** — reads (raw,
aggregate with all 15 retrieval modes, at-time, blocks, events, browse, metadata,
status) **and writes** (historical/backfill values, event send, tag-config
create/delete/rename/extended-properties, plus resilience helpers).
It reuses the family's **common shared packages and styles**: `ZB.MOM.WW.Auth`,
`ZB.MOM.WW.Theme`, `ZB.MOM.WW.Telemetry`(+`.Serilog`), `ZB.MOM.WW.Health`,
`ZB.MOM.WW.Configuration`, `ZB.MOM.WW.Audit`.
### Key reframing discovered during brainstorming
- The historian write surface lives in the **`histsdk`** repo
(`gitea.dohertylan.com/dohertj2/histsdk`, namespace `AVEVA.Historian.Client`),
which is **far ahead** of the stale `scadaproj/ZB.MOM.WW.SPHistorianClient` port
(2026-06-19 snapshot, reads + tag create/delete only, value-writes marked
"blocked"). `histsdk` has since added a **live-validated gRPC write surface**:
`AddHistoricalValuesAsync` (historical/backfill, gRPC-only), `SendEventAsync`
(events, both transports), `EnsureTags`/`DeleteTags`/`RenameTags`/
`AddTagExtendedProperties` (config writes, gRPC), plus higher-level
`HistorianStoreForwardWriter` (durable outbox) and a redundant-write cluster.
- **Hard server-side limit no client can lift:** `AddS2` *streaming live
process-sample* writes are GATED — the historian runtime cache only ingests from
configured IOServer / Application Server pipelines. "Live current value" writes are
therefore done via the **SQL path** (`aaAnalogTagInsert``INSERT INTO History`),
not gRPC.
- **No COM anywhere.** The historian SDK is pure-managed and Galaxy browse is plain
SQL, so — unlike `mxaccessgw` — this sidecar needs **no x86 worker and no
two-process split**. It is a single .NET 10 x64 process.
## 2. Decisions (locked during brainstorming)
| Decision | Choice |
|---|---|
| Purpose | **General-purpose gateway** — reusable modern façade over Historian + Galaxy metadata for any gRPC client (the way `mxaccessgw` is the façade for MXAccess). |
| Historian client code | **Vendor `histsdk`** (`AVEVA.Historian.Client`) into the sidecar repo as self-contained vendored source; namespace kept as-is to ease future re-sync. |
| Galaxy browse code | **New shared lib `ZB.MOM.WW.GalaxyRepository`** in scadaproj, extracted from `mxaccessgw`'s `galaxy_repository` browse; consumed by both `mxaccessgw` and this sidecar. |
| Dashboard | **Full Blazor dashboard** on `ZB.MOM.WW.Theme` (login + Galaxy browser + Historian console + API-key admin + status/health). |
| Historian write scope | **All:** historical/backfill writes, event send, tag-config writes, **and** resilience extras (store-and-forward outbox, redundant write fan-out, SQL live-value path). |
| Connection model | **Approach A — stateless gateway over pooled service-identity connections.** Clients authenticate to the gateway (ZB API key); the gateway owns historian credentials and reuses pooled, pre-authenticated connections. |
| Repo location | **New standalone sibling repo** `~/Desktop/HistorianGateway`, gitea remote `historiangw`. |
| Name / namespace | **`ZB.MOM.WW.HistorianGateway`**. |
## 3. Architecture & solution structure
Single .NET 10 x64 ASP.NET Core process.
```
~/Desktop/HistorianGateway/
src/
ZB.MOM.WW.HistorianGateway.Server/ ASP.NET Core host: gRPC services + Blazor dashboard + /healthz + /metrics
ZB.MOM.WW.HistorianGateway.Contracts/ the gateway's own .proto + generated types (for client codegen distribution)
vendor/AVEVA.Historian.Client/ VENDORED from histsdk; ArchestrA.Grpc.Contract.* protos + reads/writes/store-forward/redundancy
tests/
ZB.MOM.WW.HistorianGateway.Tests/ unit + env-gated live integration + bUnit dashboard
docs/plans/, CLAUDE.md, README.md
ZB.MOM.WW.HistorianGateway.slnx
```
**Cross-repo pieces:**
- **`scadaproj/ZB.MOM.WW.GalaxyRepository`** (new shared lib, plain files — NOT a
nested git repo): carries the **canonical `galaxy_repository.proto`** (adopted from
`mxaccessgw`'s existing contract so OtOpcUa's wire shape is not broken), the SQL
browse provider (connect to Galaxy Repository SQL → hierarchy model), and a
reusable gRPC service implementation both hosts can `MapGrpcService<>()`.
`mxaccessgw` adopting it is a **tracked follow-on** (same "built → adopted" pattern
as the other normalized components); this sidecar consumes it from the start.
- **Shared ZB packages consumed:** `ZB.MOM.WW.Auth`
(Abstractions+Ldap+ApiKeys+AspNetCore), `ZB.MOM.WW.Theme`, `ZB.MOM.WW.Telemetry`
(+`.Serilog`), `ZB.MOM.WW.Health`, `ZB.MOM.WW.Configuration`, `ZB.MOM.WW.Audit`.
## 4. gRPC API surface
Gateway's own curated contract (`ZB.MOM.WW.HistorianGateway.Grpc.V1`), grouped by
concern — not a 1:1 SDK dump. The vendored `ArchestrA.Grpc.Contract.*` protos stay
internal; clients see only the gateway contract.
| Service | RPCs | Notes |
|---|---|---|
| `HistorianRead` | `ReadRaw`, `ReadAggregate`, `ReadBlocks`, `ReadEvents` *(server-streaming)*; `ReadAtTime` | `ReadAggregate` exposes all 15 retrieval modes |
| `HistorianWrite` | `AddHistoricalValues`, `SendEvent`, `WriteLiveValues` | `WriteLiveValues` = SQL path (gRPC streaming is gated) |
| `HistorianTags` | `BrowseTagNames` *(streaming)*, `GetTagMetadata`, `EnsureTags`, `DeleteTags`, `RenameTags`, `AddTagExtendedProperties` | |
| `HistorianStatus` | `Probe`, `GetConnectionStatus`, `GetStoreForwardStatus`, `GetSystemParameter` | |
| `GalaxyRepository` | Browse areas / objects / templates / instances / attributes *(read-only)* | canonical proto from the shared lib |
**Authorization** is via **API-key scopes at the gateway** (Approach A trust
boundary): `historian:read`, `historian:write`, `historian:tags:write`,
`galaxy:read`.
## 5. Connection & data flow
```
gRPC client ──(ZB API key)──► HistorianGateway ──┬─ pooled, pre-authed gRPC conn ──► AVEVA Historian (RemoteGrpc 2023R2)
├─ store-forward outbox (SQLite) ─ replays writes on reconnect
├─ redundant-write fan-out ──────► historian members (All/Any ack)
├─ SqlConnection ──► Runtime DB (live-value writes via aaAnalogTagInsert/History)
└─ ZB.MOM.WW.GalaxyRepository ──► Galaxy Repository SQL (read-only browse)
```
- **Pooled connections:** the expensive auth handshake (`ValidateClientCredential` /
ECDH `ExchangeKey`) runs **once per connection on open**, then is reused across
requests; connections are health-checked with auto-reconnect. Write operations use
the write-enabled session mode (`0x401`).
- **Store-forward:** writes flow through the SDK's `HistorianStoreForwardWriter` — on
an unreachable historian, enqueue to durable SQLite; a background drain replays on
reconnect.
- **Redundancy:** `HistorianRedundantWriteResult` fan-out to configured members under
an All/Any ack policy; per-member result surfaced to the caller.
- **SQL live-write** and **Galaxy browse** are independent SQL paths, each with its
own validated connection config.
**Configuration** (all `ZB.MOM.WW.Configuration`-validated, aggregated by
`ConfigPreflight` at startup): historian (host, gRPC port 32565, transport=RemoteGrpc,
TLS, service identity/credentials), redundant members, store-forward path, Galaxy
Repository SQL connection string, Runtime DB connection string (SQL live-write),
Auth (LDAP + API-key pepper). Secrets live in the operator environment, never in repo.
## 6. Cross-cutting infrastructure + dashboard
- **Auth (`ZB.MOM.WW.Auth`):** gRPC clients use peppered-HMAC API keys (keyId/Bearer),
validated by a gRPC interceptor enforcing per-service scopes. Dashboard uses LDAP
login (`.Ldap`+`.AspNetCore`), cookie auth, `IGroupRoleMapper<TRole>`, canonical
`ZbClaimTypes`/`ZbCookieDefaults`, canonical-six roles, dev against the shared
GLAuth (`10.100.0.35:3893`, `dc=zb,dc=local`). `DisableLogin` dev/deploy switch.
- **Telemetry (`.Telemetry`+`.Serilog`):** `AddZbTelemetry` (Resource
`service.name=historian-gateway` + standard instrumentation + always-on Prometheus
`/metrics`, OTLP opt-in) + `AddZbSerilog`. App Meters: read/write counts + latency,
store-forward queue depth, pool connection state, redundancy ack outcomes.
- **Health (`.Health`):** three-tier ready/active/healthz + canonical JSON writer.
Probes: historian gRPC (`GrpcDependencyHealthCheck`), Galaxy Repository SQL +
Runtime DB (`DatabaseHealthCheck`), store-forward drain status.
- **Configuration (`.Configuration`):** `OptionsValidatorBase` / `ValidationBuilder` /
`AddValidatedOptions` / `ConfigPreflight` (§5).
- **Audit (`.Audit`, DEEP-adopt):** canonical `AuditEvent` + SQLite `IAuditWriter`
(MxGateway-style). Audited: tag-config writes, historical/event writes, API-key
admin, login/logout. `Actor` wired from the Auth principal via `IAuditActorAccessor`.
- **Dashboard (Blazor, `.Theme`):** Technical-Light side-rail shell + `LoginCard`
`/login`. Pages: **Status** (pool / store-forward / redundancy / version),
**Galaxy browser** (read-only hierarchy tree), **Historian console** (query with
raw/aggregate + mode picker + time range; role-gated write test for value insert /
event send), **API-key admin** (list/create/revoke keys + scopes), **Health**.
## 7. Error handling
- **gRPC status mapping:** `ProtocolEvidenceMissingException` (unsupported op/type —
e.g. non-analog tag, non-string event property) → `Unimplemented`/`FailedPrecondition`
with a clear "not in reverse-engineered surface" message; auth →
`Unauthenticated`/`PermissionDenied`; historian down → `Unavailable`; bad range /
unknown tag → `InvalidArgument`/`NotFound`.
- **Gated ops:** live streaming-sample writes (`AddS2`) are **not exposed** (no RPC);
live-value writes route through SQL `WriteLiveValues`.
- **Write resilience:** with store-forward enabled, an unreachable historian returns
*accepted + queued* (not an error); otherwise `Unavailable`. Redundancy surfaces a
per-member result; All-policy fails if any member fails, Any-policy succeeds on ≥1 ack.
- **Pool:** transient failures → reconnect + bounded retry; auth-handshake failure →
fail fast with diagnostic. No secrets/real hostnames in errors or logs (histsdk
safety rule).
## 8. Testing
- **Unit:** gRPC services against a faked historian-client seam + faked Galaxy
provider; scope/auth interceptor; config validators; SDK-model ↔ proto mapping.
- **Golden/protocol:** carry over `histsdk`'s golden byte tests for the vendored
client (historical "ON" buffer, event "OS" buffer, registration buffers) so the
vendored copy stays faithful.
- **Integration (env-gated, live, CI/macOS-safe):** real 2023 R2 historian + Galaxy
Repository SQL — read/write round-trips and browse via the self-cleaning
sandbox-tag lifecycle (`HISTORIAN_GRPC_WRITE_SANDBOX_TAG`); skipped when env vars
absent.
- **Dashboard:** bUnit component tests. **Smoke:** `/healthz`, `/metrics`, gRPC
`Probe`.
## 9. Out of scope / non-goals
- `AddS2` live streaming process-sample writes (GATED server-side; SQL path covers
live values instead).
- Non-analog tag creation, revision/edit writes, bit-faithful store-forward framing
(per `histsdk` capability matrix — `BOUNDED`/`HARD`/`GATED` items not selected).
- A two-process / x86 worker split (not needed — no COM).
- Re-syncing or replacing the existing stale `scadaproj/ZB.MOM.WW.SPHistorianClient`
port (we vendor `histsdk` instead; the stale port is left as-is).
## 10. Implementation components (high level)
1. **`ZB.MOM.WW.GalaxyRepository` shared lib** (scadaproj) — extract from
`mxaccessgw`, canonical proto + SQL browse provider + reusable gRPC service.
2. **Vendor `histsdk`** `AVEVA.Historian.Client` into the new repo + carry its golden
tests.
3. **Repo scaffold + host + shared-package wiring** (Auth/Telemetry/Health/
Configuration/Audit) + validated options + `ConfigPreflight`.
4. **gRPC contract + services** (Read / Write / Tags / Status / GalaxyRepository).
5. **Connection layer** — pooled pre-authed connections, store-forward, redundancy,
SQL live-write path.
6. **Auth** — API-key scope interceptor + LDAP dashboard auth + Audit wiring.
7. **Blazor dashboard** pages (Theme).
8. **Telemetry + Health** probes/meters.
9. **Tests** — unit / golden / env-gated integration / bUnit.
10. **Docs + repo/gitea setup**`CLAUDE.md`, `README.md`, gitea remote.
> `mxaccessgw` adoption of `ZB.MOM.WW.GalaxyRepository` is a separate tracked
> follow-on, not part of the initial sidecar delivery.
@@ -0,0 +1,523 @@
# ZB.MOM.WW.HistorianGateway Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans to implement this plan task-by-task.
**Goal:** Build a single .NET 10 x64 sidecar that exposes (1) a read-only Galaxy object-hierarchy metadata gRPC server and (2) a full read/write gRPC API to the AVEVA Historian, with a Blazor dashboard, reusing the family's shared `ZB.MOM.WW.*` packages.
**Architecture:** One ASP.NET Core process hosting gRPC services + Blazor (no COM, no x86 worker). The historian write/read surface comes from the **vendored `histsdk` client** (`AVEVA.Historian.Client`). The Galaxy browse comes from a **new shared lib `ZB.MOM.WW.GalaxyRepository`** in scadaproj (extracted from mxaccessgw, wire-compatible `galaxy_repository.v1`). Connection model: stateless gateway over a **pooled, pre-authenticated service-identity connection**; clients authenticate to the gateway via peppered-HMAC API keys with per-service scopes.
**Tech Stack:** .NET 10, ASP.NET Core, Grpc.AspNetCore 2.76, Grpc.Net.Client 2.58 (vendored), Google.Protobuf, Microsoft.Data.SqlClient, Microsoft.Data.Sqlite, Blazor InteractiveServer, `ZB.MOM.WW.Theme` 0.3.1, `ZB.MOM.WW.Auth` 0.1.2, `ZB.MOM.WW.Telemetry`/`.Serilog` 0.1.0, `ZB.MOM.WW.Health` 0.1.0, `ZB.MOM.WW.Audit` 0.1.0, `ZB.MOM.WW.Configuration` 0.1.0, xUnit, bUnit.
**Reference sources (read these for exact patterns — do NOT re-discover):**
- Design doc: `docs/plans/2026-06-23-historian-gateway-design.md`
- mxaccessgw (the model): `~/Desktop/MxAccessGateway/src/ZB.MOM.WW.MxGateway.Server/``GatewayApplication.cs` (host wiring), `Security/Authorization/*` (gRPC API-key interceptor + scope resolver), `Galaxy/GalaxyRepository.cs` (the SQL to extract), `Galaxy/GalaxyRepositoryOptions.cs`, `Galaxy/GalaxyHierarchyCache.cs`, `Galaxy/GalaxyRepositoryServiceCollectionExtensions.cs`, `Contracts/Protos/galaxy_repository.proto`, `Dashboard/Components/*` (Blazor + Theme).
- histsdk clone (to vendor): `/tmp/histsdk-explore/src/AVEVA.Historian.Client/` + `/tmp/histsdk-explore/tests/AVEVA.Historian.Client.Tests/`.
- Shared package signatures: captured in the design session; key paths under `~/Desktop/scadaproj/ZB.MOM.WW.{Telemetry,Health,Configuration,Audit,Auth,Theme}/`.
**Conventions for every task:** TDD where a seam exists (write the failing test first). Exact file paths in the `Files:` block ARE the implementer's contract. Commit after each task. Tests must stay green on macOS with no live historian/SQL (live tests are env-gated and skip when env vars are absent).
---
## Phase 0 — Shared `ZB.MOM.WW.GalaxyRepository` lib (in scadaproj)
> Built in `~/Desktop/scadaproj/ZB.MOM.WW.GalaxyRepository/` as plain files (NOT a nested git repo — see memory `shared-libs-are-plain-files-not-nested-repos`). Wire-compatible: keep proto `package galaxy_repository.v1` and all field numbers identical to mxaccessgw's so OtOpcUa is unaffected; only the C# `csharp_namespace` becomes neutral. mxaccessgw adoption of this lib is a separate follow-on, NOT in this plan.
### Task 1: Scaffold the GalaxyRepository lib project
**Classification:** small
**Estimated implement time:** ~3 min
**Parallelizable with:** Task 7 (vendoring), Task 9 (repo scaffold)
**Files:**
- Create: `~/Desktop/scadaproj/ZB.MOM.WW.GalaxyRepository/ZB.MOM.WW.GalaxyRepository.slnx`
- Create: `~/Desktop/scadaproj/ZB.MOM.WW.GalaxyRepository/src/ZB.MOM.WW.GalaxyRepository/ZB.MOM.WW.GalaxyRepository.csproj`
- Create: `~/Desktop/scadaproj/ZB.MOM.WW.GalaxyRepository/tests/ZB.MOM.WW.GalaxyRepository.Tests/ZB.MOM.WW.GalaxyRepository.Tests.csproj`
**Steps:**
1. Create the `.csproj` (net10.0, `Nullable`/`ImplicitUsings` enabled, packable, `PackageId=ZB.MOM.WW.GalaxyRepository`, `Version=0.1.0`). PackageReferences: `Microsoft.Data.SqlClient` 6.0.2, `Grpc.AspNetCore` 2.76.0, `Google.Protobuf`, `Microsoft.Extensions.Hosting.Abstractions`, `Microsoft.Extensions.Options.ConfigurationExtensions`. Add `<Protobuf Include="Protos\*.proto" GrpcServices="Server" />`.
2. Create the test `.csproj` (net10.0, `IsPackable=false`, xUnit 2.9.3 + `Microsoft.NET.Test.Sdk` 17.14.1 + `Microsoft.Data.SqlClient`), ProjectReference to the lib.
3. Create the `.slnx` listing both projects.
4. Run: `dotnet build ~/Desktop/scadaproj/ZB.MOM.WW.GalaxyRepository/ZB.MOM.WW.GalaxyRepository.slnx` — Expected: builds (no sources yet, 0 warnings).
5. Commit: `git -C ~/Desktop/scadaproj add ZB.MOM.WW.GalaxyRepository && git -C ~/Desktop/scadaproj commit -m "feat(galaxyrepo): scaffold ZB.MOM.WW.GalaxyRepository shared lib"`
### Task 2: Port the canonical galaxy_repository.proto (neutral namespace)
**Classification:** standard
**Estimated implement time:** ~4 min
**Parallelizable with:** none (Task 3+ depend on generated types)
**Files:**
- Create: `~/Desktop/scadaproj/ZB.MOM.WW.GalaxyRepository/src/ZB.MOM.WW.GalaxyRepository/Protos/galaxy_repository.proto`
**Steps:**
1. Copy mxaccessgw's `Contracts/Protos/galaxy_repository.proto` verbatim, changing ONLY `option csharp_namespace` to `"ZB.MOM.WW.GalaxyRepository.Grpc"`. Keep `package galaxy_repository.v1`, all services (`TestConnection`, `GetLastDeployTime`, `DiscoverHierarchy`, `WatchDeployEvents`, `BrowseChildren`), and every message/field number identical (wire compatibility).
2. Run: `dotnet build .../ZB.MOM.WW.GalaxyRepository.slnx` — Expected: PASS; generated `GalaxyRepository.GalaxyRepositoryBase`, `GalaxyObject`, `GalaxyAttribute`, etc. appear under namespace `ZB.MOM.WW.GalaxyRepository.Grpc`.
3. Commit: `feat(galaxyrepo): canonical galaxy_repository.v1 proto (neutral namespace)`
### Task 3: Port the SQL browse provider (`GalaxyRepository` + rows + options)
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** none
**Files:**
- Create: `.../src/ZB.MOM.WW.GalaxyRepository/GalaxyRepositoryOptions.cs`
- Create: `.../src/ZB.MOM.WW.GalaxyRepository/GalaxyHierarchyRow.cs`
- Create: `.../src/ZB.MOM.WW.GalaxyRepository/GalaxyAttributeRow.cs`
- Create: `.../src/ZB.MOM.WW.GalaxyRepository/IGalaxyRepository.cs`
- Create: `.../src/ZB.MOM.WW.GalaxyRepository/GalaxyRepository.cs`
**Steps:**
1. Port `GalaxyRepositoryOptions` from mxaccessgw `Galaxy/GalaxyRepositoryOptions.cs` — rename section const to `ZB.MOM.WW.GalaxyRepository` (the consuming app picks its own section path at registration), drop MxGateway-specific defaults. Keep `ConnectionString`, `CommandTimeoutSeconds`, `DashboardRefreshIntervalSeconds`, `PersistSnapshot`, `SnapshotCachePath`.
2. Port `GalaxyHierarchyRow` / `GalaxyAttributeRow` DTOs and the `IGalaxyRepository` interface (`TestConnectionAsync`, `GetLastDeployTimeAsync`, `GetHierarchyAsync`, `GetAttributesAsync`).
3. Port `GalaxyRepository.cs` **verbatim** including the two SQL blocks (`HierarchySql`, `AttributesSql`) and the `SqlConnection`/`SqlDataReader` mapping loops — these are validated reverse-engineered queries; do NOT modify the SQL.
4. Run: `dotnet build` — Expected: PASS.
5. Commit: `feat(galaxyrepo): SQL browse provider (hierarchy + attributes)`
### Task 4: Port the in-memory hierarchy cache + snapshot + deploy notifier + refresh service
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** none
**Files:**
- Create: `.../GalaxyHierarchyCacheEntry.cs`, `.../IGalaxyHierarchyCache.cs`, `.../GalaxyHierarchyCache.cs`
- Create: `.../IGalaxyDeployNotifier.cs`, `.../GalaxyDeployNotifier.cs`
- Create: `.../IGalaxyHierarchySnapshotStore.cs`, `.../GalaxyHierarchySnapshotStore.cs`
- Create: `.../GalaxyHierarchyRefreshService.cs` (`BackgroundService`)
- Create: `.../GalaxyHierarchyProjector.cs` (paging/filter projection used by the gRPC service)
**Steps:**
1. Port these from mxaccessgw's `Galaxy/` folder, adjusting namespaces to `ZB.MOM.WW.GalaxyRepository`. Keep the cache's first-load gate, refresh semaphore, snapshot restore, and deploy-poll refresh trigger.
2. Port `GalaxyHierarchyProjector` (the `Project(...)` + `ComputeFilterSignature(...)` used by `DiscoverHierarchy`/`BrowseChildren` paging).
3. Run: `dotnet build` — Expected: PASS.
4. Commit: `feat(galaxyrepo): hierarchy cache + snapshot + refresh service`
### Task 5: Port the reusable gRPC service + DI extension
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** none
**Files:**
- Create: `.../Grpc/GalaxyRepositoryGrpcService.cs`
- Create: `.../DependencyInjection/GalaxyRepositoryServiceCollectionExtensions.cs`
**Steps:**
1. Port `GalaxyRepositoryGrpcService` from mxaccessgw's `Grpc/GalaxyRepositoryGrpcService.cs`, but REMOVE the mxaccessgw-specific `IGatewayRequestIdentityAccessor`/`ApiKeyConstraints` browse-subtree filtering (the gateway will apply its own auth at the interceptor layer). Keep `DiscoverHierarchy`, `BrowseChildren`, `TestConnection`, `GetLastDeployTime`, `WatchDeployEvents`. Base class: `ZB.MOM.WW.GalaxyRepository.Grpc.GalaxyRepository.GalaxyRepositoryBase`.
2. Write `AddZbGalaxyRepository(this IServiceCollection, IConfiguration, string sectionPath)` modeled on mxaccessgw's `AddGalaxyRepository` — bind options from `sectionPath`, register `GalaxyRepository`/`IGalaxyRepository`, notifier, snapshot store, cache, and the refresh `HostedService`. Add a companion `MapZbGalaxyRepository(this IEndpointRouteBuilder)` that `MapGrpcService<GalaxyRepositoryGrpcService>()`.
3. Run: `dotnet build` — Expected: PASS.
4. Commit: `feat(galaxyrepo): reusable gRPC service + AddZbGalaxyRepository DI`
### Task 6: Unit tests for the projector + DI smoke; pack
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** none
**Files:**
- Create: `.../tests/ZB.MOM.WW.GalaxyRepository.Tests/GalaxyHierarchyProjectorTests.cs`
- Create: `.../tests/ZB.MOM.WW.GalaxyRepository.Tests/GalaxyHierarchyCacheTests.cs`
**Steps:**
1. **Write failing tests first:** projector paging (page_token round-trip, max_depth, `historized_only`/`alarm_bearing_only` filters, attribute include toggle) against a hand-built `GalaxyHierarchyCacheEntry` fixture; cache first-load gate + snapshot restore using a fake `IGalaxyRepository`. (SQL provider itself is exercised by env-gated integration later — no live DB in unit tests.)
2. Run: `dotnet test .../ZB.MOM.WW.GalaxyRepository.slnx` — Expected: FAIL (types/asserts).
3. Implement any small helper gaps surfaced; re-run — Expected: PASS.
4. Run: `dotnet pack .../src/ZB.MOM.WW.GalaxyRepository/ZB.MOM.WW.GalaxyRepository.csproj -c Release` — Expected: `ZB.MOM.WW.GalaxyRepository.0.1.0.nupkg` produced.
5. Commit: `test(galaxyrepo): projector + cache tests; pack 0.1.0`
---
## Phase 1 — Sidecar repo scaffold + vendor histsdk
### Task 7: Vendor the histsdk client + its golden tests
**Classification:** high-risk
**Estimated implement time:** ~5 min
**Parallelizable with:** Task 1
**Files:**
- Create: `~/Desktop/HistorianGateway/src/vendor/AVEVA.Historian.Client/**` (copied)
- Create: `~/Desktop/HistorianGateway/tests/AVEVA.Historian.Client.Tests/**` (copied)
- Create: `~/Desktop/HistorianGateway/src/vendor/AVEVA.Historian.Client/VENDORING.md`
**Steps:**
1. `mkdir -p ~/Desktop/HistorianGateway/src/vendor ~/Desktop/HistorianGateway/tests`. Copy `/tmp/histsdk-explore/src/AVEVA.Historian.Client/` and `/tmp/histsdk-explore/tests/AVEVA.Historian.Client.Tests/` into those locations.
2. In the vendored test `.csproj`, REMOVE the `ProjectReference` to `tools/AVEVA.Historian.ReverseEngineering` (not vendored) and delete any test classes that depend on that tooling namespace (the RE-sanitizer tests). KEEP the protocol/golden tests: `HistorianTagWriteProtocolTests`, `HistorianEventRowProtocolTests`, `GrpcEventSendProtocolTests`, `WcfDataQueryProtocolTests`, `StoreForwardOutboxTests`, `RedundancyTests`, version-gate tests. Fix the surviving test `.csproj` ProjectReference path to the new vendored client location.
3. Keep namespace `AVEVA.Historian.Client` as-is (eases re-sync). Write `VENDORING.md` recording: source repo `gitea.dohertylan.com/dohertj2/histsdk`, the commit/date of the snapshot, and "do not hand-edit; re-vendor from upstream."
4. Run: `dotnet build ~/Desktop/HistorianGateway/src/vendor/AVEVA.Historian.Client/AVEVA.Historian.Client.csproj` then `dotnet test ~/Desktop/HistorianGateway/tests/AVEVA.Historian.Client.Tests/` — Expected: build PASS; golden/offline tests PASS (live env-gated tests skip).
5. Commit (in the new repo, after Task 8 inits it — if running before Task 8, defer the commit): `chore(vendor): vendor histsdk AVEVA.Historian.Client + golden tests`
### Task 8: Initialize the sidecar repo + solution + Directory.Build.props
**Classification:** small
**Estimated implement time:** ~3 min
**Parallelizable with:** none (Task 7 output is added here)
**Files:**
- Create: `~/Desktop/HistorianGateway/.gitignore`
- Create: `~/Desktop/HistorianGateway/Directory.Build.props`
- Create: `~/Desktop/HistorianGateway/ZB.MOM.WW.HistorianGateway.slnx`
**Steps:**
1. `git -C ~/Desktop/HistorianGateway init` (this IS its own app repo — unlike shared libs). Add a .NET `.gitignore`.
2. `Directory.Build.props`: `net10.0`, `Nullable`/`ImplicitUsings` enable, `<Platforms>x64</Platforms>`, `<PlatformTarget>x64</PlatformTarget>`, common `LangVersion`.
3. Create `.slnx` referencing: `src/vendor/AVEVA.Historian.Client`, `tests/AVEVA.Historian.Client.Tests` (and the projects added in later phases — add them as created).
4. Run: `dotnet build ~/Desktop/HistorianGateway/ZB.MOM.WW.HistorianGateway.slnx` — Expected: PASS.
5. Commit: `chore: init repo + solution + Directory.Build.props` (then re-commit Task 7's vendored tree if it was deferred).
---
## Phase 2 — Host + configuration + shared-package wiring
### Task 9: Create the Contracts project + historian_gateway.proto skeleton
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** Task 1
**Files:**
- Create: `~/Desktop/HistorianGateway/src/ZB.MOM.WW.HistorianGateway.Contracts/ZB.MOM.WW.HistorianGateway.Contracts.csproj`
- Create: `.../Contracts/Protos/historian_gateway.proto`
**Steps:**
1. `.csproj` net10.0, `Grpc.AspNetCore` 2.76.0, `<Protobuf Include="Protos\*.proto" GrpcServices="Both" />`.
2. Author `historian_gateway.proto` (`package historian_gateway.v1; option csharp_namespace = "ZB.MOM.WW.HistorianGateway.Contracts.Grpc";`) with the **service stubs and message shells** for the 4 historian services: `HistorianRead` (ReadRaw/ReadAggregate/ReadBlocks/ReadEvents server-streaming, ReadAtTime unary), `HistorianWrite` (AddHistoricalValues, SendEvent, WriteLiveValues), `HistorianTags` (BrowseTagNames streaming, GetTagMetadata, EnsureTags, DeleteTags, RenameTags, AddTagExtendedProperties), `HistorianStatus` (Probe, GetConnectionStatus, GetStoreForwardStatus, GetSystemParameter). Map the message fields to the vendored `HistorianSample`/`HistorianAggregateSample`/`HistorianEvent`/`HistorianTagMetadata`/`HistorianHistoricalValue` shapes (timestamps as `google.protobuf.Timestamp`, `RetrievalMode` as an enum mirroring the SDK's 15 modes).
3. Run: `dotnet build` — Expected: PASS; gateway gRPC base classes generated. Add project to `.slnx`.
4. Commit: `feat(contracts): historian_gateway.v1 proto + generated types`
### Task 10: Create the Server project + minimal boot
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** none
**Files:**
- Create: `.../src/ZB.MOM.WW.HistorianGateway.Server/ZB.MOM.WW.HistorianGateway.Server.csproj`
- Create: `.../Server/Program.cs`
- Create: `.../Server/appsettings.json`, `.../Server/appsettings.Development.json`
**Steps:**
1. `.csproj` (Sdk `Microsoft.NET.Sdk.Web`): PackageReferences exactly mirroring mxaccessgw's Server csproj versions — `Grpc.AspNetCore` 2.76.0, `ZB.MOM.WW.Auth.{Abstractions,Ldap,ApiKeys,AspNetCore}` 0.1.2, `ZB.MOM.WW.Audit` 0.1.0, `ZB.MOM.WW.Theme` 0.3.1, `ZB.MOM.WW.Configuration` 0.1.0, `ZB.MOM.WW.Health` 0.1.0, `ZB.MOM.WW.Telemetry`+`.Serilog` 0.1.0, `Serilog.AspNetCore`/`.Sinks.Console`/`.Sinks.File`, `Microsoft.Data.Sqlite` 10.0.7, `Microsoft.Data.SqlClient` 6.0.2, `Polly.Core` 8.6.6. ProjectReferences: Contracts + vendored `AVEVA.Historian.Client` + `ZB.MOM.WW.GalaxyRepository` (project ref to the scadaproj lib, or pkg ref to its 0.1.0 nupkg).
2. `Program.cs`: minimal `WebApplication` that calls `AddZbSerilog`/`AddZbTelemetry` (ServiceName `historian-gateway`), `builder.Services.AddGrpc()`, maps `/healthz` + `/metrics` via `MapZbHealth`/`MapZbMetrics`, boots. (Subsystems wired in later tasks.)
3. Run: `dotnet build` then `dotnet run --project .../Server` and `curl -s localhost:<port>/healthz` — Expected: 200; `curl /metrics` returns Prometheus text. Add project to `.slnx`.
4. Commit: `feat(server): host scaffold + telemetry/serilog/health boot`
### Task 11: Configuration options + validators + ConfigPreflight
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** none
**Files:**
- Create: `.../Server/Configuration/HistorianOptions.cs` + `HistorianOptionsValidator.cs`
- Create: `.../Server/Configuration/GalaxyOptions.cs` (thin wrapper / reuse `GalaxyRepositoryOptions`)
- Create: `.../Server/Configuration/RuntimeDbOptions.cs` + validator (SQL live-write)
- Create: `.../Server/Configuration/RedundancyOptions.cs` + validator
- Create: `.../Server/Configuration/StoreForwardOptions.cs` + validator
- Modify: `.../Server/Program.cs` (register `AddValidatedOptions<,>` + run `ConfigPreflight`)
- Test: `.../tests/ZB.MOM.WW.HistorianGateway.Tests/Configuration/ValidatorTests.cs`
**Steps:**
1. **Write failing validator tests first** using `OptionsValidatorBase`/`ValidationBuilder` semantics (e.g., missing `Historian:Host` → failure; bad port → failure; `Transport` one-of; redundancy `MinCount(members,1)` when enabled). Run — Expected: FAIL.
2. Implement options records + validators (subclass `OptionsValidatorBase<T>`, use `ValidationBuilder.Required/Port/HostPort/OneOf/PositiveTimeSpan/MinCount`). Map `HistorianOptions` → vendored `HistorianClientOptions` (Host, Port default 32565, `Transport=RemoteGrpc`, `GrpcUseTls`, credentials, `AllowUntrustedServerCertificate`).
3. In `Program.cs`, `AddValidatedOptions<,>` each, and run a `ConfigPreflight` (RequireValue host, RequirePort) before host build.
4. Run: `dotnet test` — Expected: PASS.
5. Commit: `feat(server): validated options + ConfigPreflight`
---
## Phase 3 — Connection layer (vendored client → gateway)
### Task 12: `IHistorianClient` seam over the vendored client
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** none
**Files:**
- Create: `.../Server/Historian/IHistorianClient.cs` (interface mirroring the read/write methods the services need)
- Create: `.../Server/Historian/VendoredHistorianClient.cs` (adapts `AVEVA.Historian.Client.HistorianClient`)
- Test: `.../tests/.../Historian/HistorianClientSeamTests.cs`
**Steps:**
1. **Write failing test** that a `FakeHistorianClient : IHistorianClient` can be substituted and returns canned samples (this seam is what makes the gRPC services unit-testable without a live historian). Run — Expected: FAIL.
2. Define `IHistorianClient` with the methods the services call (ReadRaw/ReadAggregate/ReadAtTime/ReadBlocks/ReadEvents/BrowseTagNames/GetTagMetadata/Probe/GetConnectionStatus/GetStoreForwardStatus/GetSystemParameter/AddHistoricalValues/SendEvent/EnsureTag/DeleteTag/RenameTags/AddTagExtendedProperties). Implement `VendoredHistorianClient` delegating to the real `HistorianClient`.
3. Run: `dotnet test` — Expected: PASS.
4. Commit: `feat(historian): IHistorianClient seam + vendored adapter`
### Task 13: Connection pool (pre-authenticated, reused, health-checked)
**Classification:** high-risk
**Estimated implement time:** ~5 min
**Parallelizable with:** none
**Files:**
- Create: `.../Server/Historian/HistorianConnectionPool.cs` (+ `IHistorianConnectionPool`)
- Modify: `.../Server/Program.cs` (DI singleton)
- Test: `.../tests/.../Historian/HistorianConnectionPoolTests.cs`
**Steps:**
1. **Write failing test** asserting the pool opens/authenticates a connection once and reuses it across N borrow calls (count handshakes via a fake transport/lease factory), and that a faulted connection is evicted + re-created. Run — Expected: FAIL.
2. Implement a lease-based pool keyed by target; lazy-open with the auth handshake once; reuse; `SemaphoreSlim`-guarded reconnect on fault; expose `Lease()` returning a pooled `IHistorianClient`. (The vendored client is `IAsyncDisposable`; the pool owns lifecycle.)
3. Run: `dotnet test` — Expected: PASS.
4. Commit: `feat(historian): pooled pre-authenticated connection pool`
### Task 14: Store-forward + redundancy + SQL live-write wiring
**Classification:** high-risk
**Estimated implement time:** ~5 min
**Parallelizable with:** none
**Files:**
- Create: `.../Server/Historian/HistorianWriteCoordinator.cs` (routes writes → pool, store-forward, or redundancy per config)
- Create: `.../Server/Historian/SqlLiveValueWriter.cs` (`WriteLiveValues` via `aaAnalogTagInsert` + `INSERT INTO History`)
- Modify: `.../Server/Program.cs`
- Test: `.../tests/.../Historian/HistorianWriteCoordinatorTests.cs`, `.../SqlLiveValueWriterTests.cs`
**Steps:**
1. **Write failing tests:** (a) when store-forward enabled + historian unreachable, the coordinator enqueues (uses vendored `HistorianStoreForwardWriter` over a fake sink) and reports `Queued`; (b) when redundancy configured, it fans out via `HistorianRedundantClient` and returns per-member results under All/Any; (c) `SqlLiveValueWriter` builds the correct parameterized command sequence (assert against a fake `IDbCommand` recorder — no live SQL). Run — Expected: FAIL.
2. Implement the coordinator (compose vendored `HistorianStoreForwardWriter` + `HistorianRedundantClient` from config) and `SqlLiveValueWriter` (omit the server-managed `Quality` column; honor the storage-activation note from the SQL reference memory).
3. Run: `dotnet test` — Expected: PASS.
4. Commit: `feat(historian): write coordinator (store-forward + redundancy) + SQL live-write`
---
## Phase 4 — gRPC services + auth interceptor
### Task 15: `HistorianRead` service (representative TDD task; sets the pattern)
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** Task 17 after the mapper exists
**Files:**
- Create: `.../Server/Grpc/HistorianReadService.cs`
- Create: `.../Server/Grpc/HistorianProtoMapper.cs` (SDK model ↔ proto)
- Modify: `.../Server/Program.cs` (`MapGrpcService`)
- Test: `.../tests/.../Grpc/HistorianReadServiceTests.cs`
**Steps:**
1. **Write failing test:** with a `FakeHistorianClient` yielding 3 `HistorianSample`s, calling `ReadRaw` streams 3 mapped proto rows; `ReadAggregate` passes the right `RetrievalMode`+interval; an unknown tag → `RpcException(NotFound)`; bad time range → `InvalidArgument`. Use an in-memory `IServerStreamWriter<T>` capture. Run — Expected: FAIL.
2. Implement `HistorianReadService : HistorianRead.HistorianReadBase` consuming `IHistorianConnectionPool.Lease()`; implement `HistorianProtoMapper` (Timestamp conversions, RetrievalMode enum map). Map exceptions per design §7.
3. Run: `dotnet test` — Expected: PASS.
4. Commit: `feat(grpc): HistorianRead service + proto mapper`
### Task 16: `HistorianWrite` service
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** Task 17, Task 18 (no file overlap)
**Files:** Create `.../Server/Grpc/HistorianWriteService.cs`; Modify `Program.cs`; Test `.../Grpc/HistorianWriteServiceTests.cs`
**Steps:** TDD per the Task 15 pattern. `AddHistoricalValues`/`SendEvent` route through `HistorianWriteCoordinator`; `WriteLiveValues` through `SqlLiveValueWriter`. Map `ProtocolEvidenceMissingException``Unimplemented`, unreachable+store-forward → `OK` with `Queued` status, redundancy per-member results into the reply. Commit: `feat(grpc): HistorianWrite service`.
### Task 17: `HistorianTags` service
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** Task 16, Task 18
**Files:** Create `.../Server/Grpc/HistorianTagsService.cs`; Modify `Program.cs`; Test `.../Grpc/HistorianTagsServiceTests.cs`
**Steps:** TDD. `BrowseTagNames` (streaming), `GetTagMetadata`, `EnsureTags`/`DeleteTags`/`RenameTags`/`AddTagExtendedProperties` via the seam/pool. Map unsupported tag types (`ProtocolEvidenceMissingException`) → `FailedPrecondition`. Commit: `feat(grpc): HistorianTags service`.
### Task 18: `HistorianStatus` service
**Classification:** small
**Estimated implement time:** ~4 min
**Parallelizable with:** Task 16, Task 17
**Files:** Create `.../Server/Grpc/HistorianStatusService.cs`; Modify `Program.cs`; Test `.../Grpc/HistorianStatusServiceTests.cs`
**Steps:** TDD. `Probe`/`GetConnectionStatus`/`GetStoreForwardStatus`/`GetSystemParameter`. Commit: `feat(grpc): HistorianStatus service`.
### Task 19: Galaxy gRPC wiring (consume the shared lib)
**Classification:** small
**Estimated implement time:** ~3 min
**Parallelizable with:** Task 1618
**Files:** Modify `.../Server/Program.cs` (`AddZbGalaxyRepository(config, "Galaxy")` + `MapZbGalaxyRepository()`); Modify `appsettings.json`
**Steps:** Register the shared lib's service + refresh hosted service; add `Galaxy:ConnectionString` config. Run: `dotnet run` + grpcurl `DiscoverHierarchy` against a fake/empty config returns `Unavailable` until cache loads (no live DB needed to prove wiring). Commit: `feat(server): wire shared GalaxyRepository gRPC service`.
### Task 20: API-key auth interceptor + scope resolver
**Classification:** high-risk
**Estimated implement time:** ~5 min
**Parallelizable with:** none
**Files:**
- Create: `.../Server/Security/GatewayGrpcScopeResolver.cs` (maps request type → scope)
- Create: `.../Server/Security/GatewayGrpcAuthorizationInterceptor.cs`
- Create: `.../Server/Security/GatewayScopes.cs` (`historian:read|write`, `historian:tags:write`, `galaxy:read`)
- Modify: `.../Server/Program.cs` (`AddZbApiKeyAuth` + `AddGrpc(o => o.Interceptors.Add<...>())`)
- Test: `.../tests/.../Security/GrpcAuthorizationTests.cs`
**Steps:**
1. **Write failing tests:** missing/invalid key → `Unauthenticated`; valid key without the required scope → `PermissionDenied`; valid key with scope → continuation runs. Fake `IApiKeyVerifier`. Run — Expected: FAIL.
2. Implement modeled on mxaccessgw's `GatewayGrpcAuthorizationInterceptor` + `GatewayGrpcScopeResolver` (switch on request type → scope), using shared `IApiKeyVerifier.VerifyAsync`. Respect a `Disabled` auth mode for dev.
3. Run: `dotnet test` — Expected: PASS.
4. Commit: `feat(security): gRPC API-key interceptor + scope enforcement`
---
## Phase 5 — Audit
### Task 21: Canonical SQLite audit writer + actor accessor + wiring
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** Task 22 (dashboard auth) after interfaces exist
**Files:**
- Create: `.../Server/Audit/SqliteAuditWriter.cs` (`IAuditWriter`), `.../Server/Audit/HttpAuditActorAccessor.cs` (`IAuditActorAccessor`)
- Modify: write services (Tasks 16,17) + interceptor (Task 20) to emit `AuditEvent`s
- Modify: `.../Server/Program.cs` (`AddZbAudit` + register writer/actor)
- Test: `.../tests/.../Audit/SqliteAuditWriterTests.cs`
**Steps:**
1. **Write failing test:** writing an `AuditEvent` persists a row with the canonical 9 fields (`EventId`/`OccurredAtUtc`/`Actor`/`Action`/`Outcome`/`Category`/`Target`/`SourceNode`/`DetailsJson`), domain fields in `DetailsJson`; writer swallows internal errors. Use an in-memory SQLite. Run — Expected: FAIL.
2. Implement the SQLite writer (table create-if-missing) modeled on MxGateway's audit store; `HttpAuditActorAccessor` reads the Auth principal. Emit audit at tag/value/event writes, API-key admin, login/logout, with `Actor` from the accessor.
3. Run: `dotnet test` — Expected: PASS.
4. Commit: `feat(audit): canonical SQLite audit writer + actor wiring`
---
## Phase 6 — Blazor dashboard
### Task 22: Dashboard shell, LDAP cookie auth, login/logout
**Classification:** high-risk
**Estimated implement time:** ~5 min
**Parallelizable with:** none
**Files:**
- Create: `.../Server/Dashboard/Components/{App,Routes,_Imports}.razor`, `Layout/{MainLayout,LoginLayout}.razor`, `Pages/Login.razor`
- Create: `.../Server/Dashboard/DashboardServiceCollectionExtensions.cs`, `.../Dashboard/DashboardEndpointRouteBuilderExtensions.cs`, `.../Dashboard/DashboardAuthenticator.cs`, `.../Dashboard/DashboardGroupRoleMapper.cs`
- Modify: `Program.cs` (`AddGatewayDashboard` + `MapRazorComponents<App>` + auth/antiforgery middleware)
- Test: `.../tests/ZB.MOM.WW.HistorianGateway.Tests/bUnit/LayoutRenderTests.cs`
**Steps:**
1. **Write failing bUnit test** that `MainLayout` renders `<ThemeShell>` with the nav rail and `LoginCard` renders on the login page. Run — Expected: FAIL.
2. Port the dashboard shell from mxaccessgw (`App.razor` with `ThemeHead`/`ThemeScripts`, `MainLayout` with `ThemeShell`+`NavRailSection`/`NavRailItem`, `Login.razor` using `LoginCard` posting to `/auth/login`). Wire `AddZbLdapAuth(config,"Ldap")`, cookie auth via `ZbCookieDefaults.Apply`, `IGroupRoleMapper<CanonicalRole>`, `DisableLogin` switch, `IAuditActorAccessor`.
3. Run: `dotnet test` (bUnit) then `dotnet run` and load `/login` in a browser/curl — Expected: tests PASS; login page renders themed.
4. Commit: `feat(dashboard): Theme shell + LDAP cookie auth + login`
### Task 23: Status + Health pages
**Classification:** small
**Estimated implement time:** ~4 min
**Parallelizable with:** Task 24, Task 25
**Files:** Create `.../Dashboard/Components/Pages/{StatusPage,HealthPage}.razor` (+ a `DashboardStatusService`); Test bUnit render.
**Steps:** TDD bUnit render. Status shows pool state, store-forward queue depth, redundancy members, version (from a status service reading the pool/coordinator). Commit: `feat(dashboard): status + health pages`.
### Task 24: Galaxy browser page
**Classification:** small
**Estimated implement time:** ~5 min
**Parallelizable with:** Task 23, Task 25
**Files:** Create `.../Dashboard/Components/Pages/GalaxyBrowserPage.razor` + tree node view (port mxaccessgw `BrowsePage`/`BrowseTreeNodeView`, read-only, no add-tag); Test bUnit.
**Steps:** TDD bUnit render against the shared lib's cache. Commit: `feat(dashboard): read-only Galaxy browser`.
### Task 25: Historian console page (query + role-gated write test)
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** Task 23, Task 24
**Files:** Create `.../Dashboard/Components/Pages/HistorianConsolePage.razor` (+ `DashboardHistorianService` calling the seam/pool); Test bUnit.
**Steps:** TDD bUnit. Query form (tag, time range, raw/aggregate + mode picker) renders results; write-test panel (historical value insert / event send) visible only to Engineer+ roles via `AuthorizeView`. Commit: `feat(dashboard): historian query + role-gated write console`.
### Task 26: API-key admin page
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** Task 2325
**Files:** Create `.../Dashboard/Components/Pages/ApiKeysPage.razor` (+ `DashboardApiKeyManagementService` over the shared ApiKeys store); Test bUnit.
**Steps:** TDD bUnit. List/create (show secret once)/revoke keys with scope selection. Commit: `feat(dashboard): API-key admin`.
---
## Phase 7 — Telemetry meters + Health probes
### Task 27: App meters
**Classification:** small
**Estimated implement time:** ~4 min
**Parallelizable with:** Task 28
**Files:** Create `.../Server/Observability/GatewayMetrics.cs`; Modify services/coordinator/pool to record; Modify `Program.cs` (`o.Meters=[GatewayMetrics.MeterName]`); Test `.../Observability/GatewayMetricsTests.cs`.
**Steps:** TDD with `MeterListener`. Counters/histograms: read/write counts + latency, store-forward queue depth (observable gauge), pool connection state, redundancy ack outcomes. Commit: `feat(obs): gateway meters`.
### Task 28: Health probes
**Classification:** small
**Estimated implement time:** ~4 min
**Parallelizable with:** Task 27
**Files:** Create `.../Server/Health/{HistorianConnectionHealthCheck,StoreForwardDrainHealthCheck}.cs`; Modify `Program.cs` (`AddHealthChecks` with `GrpcDependencyHealthCheck` for historian, SQL checks for Galaxy + Runtime DB, custom checks, tagged `ZbHealthTags.Ready`); Test health-check unit tests.
**Steps:** TDD. Probes flip Unhealthy when a dependency is down (fake deps). Commit: `feat(health): historian/galaxy/runtime-db/store-forward probes`.
---
## Phase 8 — Integration, docs, repo
### Task 29: Env-gated live integration tests
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** none
**Files:** Create `.../tests/.../Integration/{HistorianRoundTripTests,GalaxyBrowseTests}.cs`
**Steps:** Gated on `HISTORIAN_GRPC_HOST`/`HISTORIAN_GRPC_WRITE_SANDBOX_TAG` and a Galaxy SQL connection env var; `Skip` when absent. Cover read→write→read-back via the self-cleaning sandbox-tag lifecycle and a Galaxy `DiscoverHierarchy`. Run `dotnet test` (skips locally). Commit: `test: env-gated live integration`.
### Task 30: Full-suite green gate + smoke
**Classification:** small
**Estimated implement time:** ~3 min
**Parallelizable with:** none
**Steps:** Run `dotnet build ZB.MOM.WW.HistorianGateway.slnx` + `dotnet test` (whole solution) on macOS with no live env — Expected: ALL green, live tests skipped. `dotnet run` + curl `/healthz` (200), `/metrics` (text), grpcurl `HistorianStatus/Probe`. Fix any gaps. Commit: `chore: green gate + smoke`.
### Task 31: CLAUDE.md + README + gitea remote + scadaproj index
**Classification:** small
**Estimated implement time:** ~5 min
**Parallelizable with:** none
**Files:** Create `~/Desktop/HistorianGateway/{CLAUDE.md,README.md}`; copy the two design/plan docs into its `docs/plans/`; Modify `~/Desktop/scadaproj/CLAUDE.md` (index the new sidecar + note the GalaxyRepository follow-on for mxaccessgw).
**Steps:**
1. Write `CLAUDE.md` (overview, build/run/test commands, the no-COM single-process note, the vendored-histsdk + shared-GalaxyRepository dependencies, config sections, env vars) and `README.md`.
2. Create the gitea repo `historiangw` and push: `git -C ~/Desktop/HistorianGateway remote add origin https://gitea.dohertylan.com/dohertj2/historiangw.git && git push -u origin main` (confirm remote name/visibility with the user first).
3. Update scadaproj's umbrella `CLAUDE.md` runtime/implementation table with the new project row; commit scadaproj separately.
4. Commit: `docs: CLAUDE.md + README; index in scadaproj`.
---
## Dependency summary (for parallel dispatch)
- **Foundational, no blockers:** Task 1 (galaxy lib scaffold), Task 7 (vendor histsdk), Task 8 (repo init) — Task 8 consumes Task 7's tree.
- **Galaxy lib chain:** 2→3→4→5→6 (sequential; share files).
- **Sidecar chain:** 8→9→10→11→12→13→14, then gRPC services 15→(16,17,18 parallel),19, then 20, then 21.
- **Dashboard:** 22→(23,24,25,26 parallel) after Task 20 (auth) + Task 13/14 (data) + Task 5/19 (galaxy).
- **Obs:** 27,28 parallel after Task 14.
- **Close-out:** 29→30→31 after everything.
## Notes / non-goals (from design §9)
- No `AddS2` live streaming-sample writes (GATED) — live values only via SQL `WriteLiveValues`.
- No two-process/x86 worker (no COM).
- mxaccessgw adopting `ZB.MOM.WW.GalaxyRepository` is a tracked follow-on, NOT in this plan.
@@ -0,0 +1,37 @@
{
"planPath": "docs/plans/2026-06-23-historian-gateway-implementation.md",
"tasks": [
{"id": 1, "subject": "Task 1: Scaffold the GalaxyRepository lib project", "status": "pending"},
{"id": 2, "subject": "Task 2: Port the canonical galaxy_repository.proto (neutral namespace)", "status": "pending", "blockedBy": [1]},
{"id": 3, "subject": "Task 3: Port the SQL browse provider", "status": "pending", "blockedBy": [2]},
{"id": 4, "subject": "Task 4: Port the in-memory hierarchy cache + snapshot + refresh service", "status": "pending", "blockedBy": [3]},
{"id": 5, "subject": "Task 5: Port the reusable gRPC service + DI extension", "status": "pending", "blockedBy": [4]},
{"id": 6, "subject": "Task 6: Unit tests for projector/cache; pack 0.1.0", "status": "pending", "blockedBy": [5]},
{"id": 7, "subject": "Task 7: Vendor histsdk AVEVA.Historian.Client + golden tests", "status": "pending"},
{"id": 8, "subject": "Task 8: Init sidecar repo + solution + Directory.Build.props", "status": "pending", "blockedBy": [7]},
{"id": 9, "subject": "Task 9: Contracts project + historian_gateway.proto skeleton", "status": "pending", "blockedBy": [8]},
{"id": 10, "subject": "Task 10: Server project + minimal boot (telemetry/serilog/health)", "status": "pending", "blockedBy": [9, 7, 6]},
{"id": 11, "subject": "Task 11: Configuration options + validators + ConfigPreflight", "status": "pending", "blockedBy": [10]},
{"id": 12, "subject": "Task 12: IHistorianClient seam over vendored client", "status": "pending", "blockedBy": [10]},
{"id": 13, "subject": "Task 13: Connection pool (pre-authenticated, reused)", "status": "pending", "blockedBy": [12]},
{"id": 14, "subject": "Task 14: Write coordinator (store-forward + redundancy) + SQL live-write", "status": "pending", "blockedBy": [13]},
{"id": 15, "subject": "Task 15: HistorianRead service + proto mapper", "status": "pending", "blockedBy": [13]},
{"id": 16, "subject": "Task 16: HistorianWrite service", "status": "pending", "blockedBy": [14, 15]},
{"id": 17, "subject": "Task 17: HistorianTags service", "status": "pending", "blockedBy": [13, 15]},
{"id": 18, "subject": "Task 18: HistorianStatus service", "status": "pending", "blockedBy": [13, 15]},
{"id": 19, "subject": "Task 19: Galaxy gRPC wiring (consume shared lib)", "status": "pending", "blockedBy": [10, 5]},
{"id": 20, "subject": "Task 20: API-key auth interceptor + scope resolver", "status": "pending", "blockedBy": [15]},
{"id": 21, "subject": "Task 21: SQLite audit writer + actor accessor + wiring", "status": "pending", "blockedBy": [16, 17, 20]},
{"id": 22, "subject": "Task 22: Dashboard shell + LDAP cookie auth + login", "status": "pending", "blockedBy": [20]},
{"id": 23, "subject": "Task 23: Status + Health pages", "status": "pending", "blockedBy": [22, 14]},
{"id": 24, "subject": "Task 24: Galaxy browser page", "status": "pending", "blockedBy": [22, 19]},
{"id": 25, "subject": "Task 25: Historian console page (role-gated write)", "status": "pending", "blockedBy": [22, 15]},
{"id": 26, "subject": "Task 26: API-key admin page", "status": "pending", "blockedBy": [22]},
{"id": 27, "subject": "Task 27: App meters", "status": "pending", "blockedBy": [14]},
{"id": 28, "subject": "Task 28: Health probes", "status": "pending", "blockedBy": [14, 19]},
{"id": 29, "subject": "Task 29: Env-gated live integration tests", "status": "pending", "blockedBy": [16, 17, 18, 19]},
{"id": 30, "subject": "Task 30: Full-suite green gate + smoke", "status": "pending", "blockedBy": [21, 26, 27, 28, 29]},
{"id": 31, "subject": "Task 31: CLAUDE.md + README + gitea remote + scadaproj index", "status": "pending", "blockedBy": [30]}
],
"lastUpdated": "2026-06-23"
}
+377
View File
@@ -0,0 +1,377 @@
# MES and Delmia-DNC integrations — API & MXAccess write specification
Documents two existing Wonderware integrations hosted on the `ZimmerBiomet` Gitea org
(`http://wonder-app-vd03.zmr.zimmer.com:3000`):
| Integration | Repo | What it does | Who does the MXAccess write |
|---|---|---|---|
| **MES** | [`ZimmerBiomet/MESAPI`](http://wonder-app-vd03.zmr.zimmer.com:3000/ZimmerBiomet/MESAPI) (solution `WWSupport`) | REST API the Camstar MES calls to move-in / move-out work orders against a machine, and to read machine alarm status | **The service itself** (`MesNotifier`, in-repo) |
| **Delmia DNC** | [`ZimmerBiomet/DelmiaIntegration`](http://wonder-app-vd03.zmr.zimmer.com:3000/ZimmerBiomet/DelmiaIntegration) | Pull an NC/recipe document from the DELMIA/Apriso (Intercim) DNC server and push the resulting recipe-download notification into Wonderware | **An external receiver service** at `wonder-app-vd01:9001/notify`; the actual flag handshake is implemented in the Galaxy `$DelmiaReceiver` object (`ProcessRecipe`/`Reset` scripts) — *not* in this repo |
Both integrations talk to AVEVA System Platform ("Galaxy") through **MXAccess COM**
(`ArchestrA.MxAccess.LMXProxyServerClass`) and use the same general pattern:
> **Handshake pattern** — read a *ready* flag → write the data tags → set a *trigger* flag
> → wait (bounded by a timeout) for a *complete* flag → read a *success* flag + *error text*
> → return the result and unsubscribe.
All facts below are taken verbatim from source at the repo `master` branch (cloned
2026-06-17) unless explicitly marked **(inferred)**.
---
## 1. MES integration — `MESAPI` / `WWSupport`
### 1.1 Topology & hosting
```
Camstar MES ──HTTPS/JSON──▶ WWSupport API (ServiceStack self-host)
│ ├─ SQL Server "BT" (machine lookup by SAPID, alarm catalog)
│ └─ MXAccess COM (LMXProxyServerClass, client "MesNotifier")
Galaxy object {MachineCode}.MesReceiver.* (move-in / move-out tags)
Galaxy object {MachineCode}.{AlarmName}.* (alarm attributes)
```
- **Framework:** ServiceStack, self-hosted via `AppSelfHostBase` (`AppHost : base("APIServer", typeof(MesServices).Assembly)`), .NET Framework, run as a Windows service.
- **Listen URL (per environment, `App.config``HttpListener`):**
- DEV `http://*:9501/` · QA `http://*:9500/` · PROD `http://*:9500/`
- **Database (`App.config` → connection `BatchDB`, DB `BT`):**
- DEV `wonder-sql-vd01.zmr.zimmer.com` · QA `wondersqlqa.zmr.zimmer.com` · PROD (same form). User `wonderapp`.
- **Auth:** every operation is decorated `[Authenticate]` + `[RequiredRole("MESAPI")]` (`MesServices.cs`).
`AppHost` registers an `AuthFeature` with two providers: `ApiKeyAuthProvider` and `LdapAuthProvider`.
- Unauthenticated → **401**; authenticated without the `MESAPI` role → **403**.
- **Serialization:** `JsConfig.IncludeNullValues = true` (null fields ARE emitted in JSON). `PostmanFeature` + `OpenApiFeature` (Swagger) are enabled.
- **MES counterpart object:** the live Galaxy attribute listing for the receiver object is in
[`mesrec.md`](mesrec.md) (`$MESReceiver` template). Note the API binds tags under the contained
name `MesReceiver` (i.e. `{MachineCode}.MesReceiver.<tag>`).
### 1.2 Endpoints (inputs / outputs)
Routes come from `[Route(...)]` on the request DTOs; all are **POST**, JSON in / JSON out, handled by `MesServices.Any(...)` which resolves a per-request `MesNotifier`.
#### `POST /mes/movein` → `MoveInResponse`
Request `MoveInRequest`:
| Field | Type | Notes |
|---|---|---|
| `SAPID` | string | machine key; looked up in `BT.Machine` to get `Machine.Code` |
| `OperatorName` | string | |
| `JobSequenceNumber` | string | |
| `WorkOrders` | `List<WorkOrderInfo>` | each = `{ WorkOrderNumber: string, PartNumber: string }` |
Response `MoveInResponse`: `WasSuccessful` (bool), `ErrorText` (string), `BatchID` (int?, only set if machine returns non-zero).
#### `POST /mes/moveout` → `MoveOutResponse`
Request `MoveOutRequest`: `SAPID` (string), `OperatorName` (string), `WorkOrders` (`List<WorkOrderInfo>`).
*(Move-out has no `JobSequenceNumber`.)*
Response `MoveOutResponse`: identical shape to `MoveInResponse` (`WasSuccessful`, `ErrorText`, `BatchID`).
#### `POST /mes/alarmstatus` → `AlarmStatusResponse`
Request `AlarmStatusRequest`:
- `MachineFilter` = `{ MachineID: int?, SAPID: string, ZTag: string, Code: string }` (any one identifies the machine)
- `AlarmFilter` = `{ NameFilter: string, MinSeverity: int?, MaxSeverity: int?, IncludeTriggered: bool=true, IncludeAcked: bool=true, FlaggedOnly: bool=false }`
#### `POST /mes/simplealarmstatus` → `AlarmStatusResponse`
Request `SimpleAlarmStatusRequest`: `SAPID` (string). Internally loads only alarms with `FlaggedForMES == true` for that machine.
Response `AlarmStatusResponse` (both alarm endpoints): `WasSuccessful` (bool), `ErrorText` (string), `Alarms` (`List<AlarmInfo>`).
`AlarmInfo`: `Name` (string), `HierarchicalName` (string, `{Code}.{AlarmName}`), `Description` (string), `IsFlaggedForMES` (bool), `Severity` (int), `StatusCode` (string — `"Triggered"` or `"Triggered.Acked"`), `TriggeredDT` (DateTime), `AckDT` (DateTime?), `AckComment` (string).
### 1.3 MXAccess connection model
`MesNotifier` owns one MXAccess proxy for the request:
```csharp
_lmxProxy = new ArchestrA.MxAccess.LMXProxyServerClass();
_lmxHandle = _lmxProxy.Register("MesNotifier");
_lmxProxy.OnDataChange += ...; // value updates resolve pending read/OnValue tasks
_lmxProxy.OnWriteComplete += ...; // write acks resolve pending write tasks
```
Tags are added with `AddItem` + `AdviseSupervisory` (subscribe), updated via `OnDataChange`,
and removed with `UnAdvise` + `RemoveItem` on cleanup. Reads/writes are wrapped as `Task<bool>`
that complete on the corresponding callback or **fail (`false`) on cancellation/timeout**.
A read is considered valid only if MXAccess **quality == 192** ("good").
**Target selection:** request `SAPID``db.Single<Machine>(x => x.SAPID == SAPID)``Machine.Code`
becomes the tag prefix. Move tags live under `{Code}.MesReceiver.*`; alarm tags under
`{Code}.{MachineAlarm.Name}.*` (alarm catalog from `db.Select<MachineAlarm>(...)`).
### 1.4 Tag mappings
**Move-in** (`MesMoveInTagset`, all `{Code}.MesReceiver.<tag>`):
| Tag | Type | Dir | Role | Source field |
|---|---|---|---|---|
| `MoveInReadyFlag` | bool | read | gate — must be `true` before writing | — |
| `MoveInFlag` | bool | **write** | **trigger** — set `true` last | — |
| `MoveInCompleteFlag` | bool | read | completion — handshake waits on this | — |
| `MoveInSuccessfulFlag` | bool | read | result | → `response.WasSuccessful` |
| `MoveInErrorText` | string | read | result | → `response.ErrorText` |
| `MoveInBatchID` | int | read | result | → `response.BatchID` (if ≠ 0) |
| `MoveInOperatorName` | string | write | data | `request.OperatorName` |
| `MoveInJobSequenceNumber` | string | write | data | `request.JobSequenceNumber` |
| `MoveInNumberWorkOrders` | int | write | data | `request.WorkOrders.Count` |
| `MoveInWorkOrderNumbers[]` | string[] | write | data (fixed length 50) | `WorkOrders.Select(w => w.WorkOrderNumber)` |
| `MoveInPartNumbers[]` | string[] | write | data (fixed length 50) | `WorkOrders.Select(w => w.PartNumber)` |
**Move-out** (`MesMoveOutTagset`): identical set with `MoveOut` prefix, **minus** `JobSequenceNumber`
(`MoveOutReadyFlag`, `MoveOutFlag`, `MoveOutCompleteFlag`, `MoveOutSuccessfulFlag`, `MoveOutErrorText`,
`MoveOutBatchID`, `MoveOutOperatorName`, `MoveOutNumberWorkOrders`, `MoveOutWorkOrderNumbers[]`, `MoveOutPartNumbers[]`).
**Alarms** (`AlarmTagset`, all `{Code}.{AlarmName}.<attr>`): `Quality` (int), `InAlarm` (bool),
`TimeAlarmOn` (DateTime), `DescAttrName` (string), `Acked` (bool), `TimeAlarmAcked` (DateTime?), `AckMsg` (string).
### 1.5 The handshake — `MesNotifier.MoveIn` (move-out is identical with `MoveOut*` tags)
Whole operation is bounded by **`new CancellationTokenSource(30000)` = 30 s**.
1. **Look up machine** by `SAPID`. Not found → `WasSuccessful=false`, `ErrorText="Failed to find machine with SAPID '{SAPID}'"`, return.
2. **Subscribe** to every move-in tag (`Advise(t, cts)`), `await Task.WhenAll(...)`. Any subscription that fails / quality ≠ 192 → `ErrorText="Failed to connect to machine"`.
3. **Check ready flag:** `if (!MoveInReadyFlag.Value)``ErrorText="Machine move in ready flag not set to true"`, stop.
4. **Arm completion watch:** `Task<bool> flagTask = MoveInCompleteFlag.OnValue(true, cts);` (completes when the flag goes `true`, or `false` on the 30 s timeout).
5. **Write data + trigger (in parallel, trigger last):** `MoveInOperatorName`, `MoveInJobSequenceNumber`, `MoveInNumberWorkOrders`, `MoveInPartNumbers[]` (padded to 50), `MoveInWorkOrderNumbers[]` (padded to 50), then `MoveInFlag = true`. `await Task.WhenAll(writeTasks)`; any write `!= true``ErrorText="Failed to write move in information to machine"`.
6. **Wait for completion:** `await Task.WhenAll(flagTask)`.
- `flagTask.Result == true` → read results: `WasSuccessful = MoveInSuccessfulFlag.Value`, `ErrorText = MoveInErrorText.Value`, `BatchID = MoveInBatchID.Value` (if ≠ 0).
- `flagTask.Result == false` (timed out) → `WasSuccessful=false`, `ErrorText="Timeout waiting for move in information to be processed"`.
7. **Cleanup:** `Tags.ForEach(Unadvise)` and return.
```csharp
using (var cts = new CancellationTokenSource(30000)) { // 30 s budget
...
if (!moveInTagset.MoveInReadyFlag.Value) { /* not-ready error */ }
var flagTask = moveInTagset.MoveInCompleteFlag.OnValue(true, cts); // arm completion watch
var writeTasks = new List<Task<bool>> {
Write(moveInTagset.MoveInOperatorName.Handle, request.OperatorName, cts),
Write(moveInTagset.MoveInJobSequenceNumber.Handle, request.JobSequenceNumber, cts),
Write(moveInTagset.MoveInNumberWorkOrders.Handle, request.WorkOrders.Count, cts),
Write(moveInTagset.MoveInPartNumbers.Handle, request.WorkOrders.Select(wo => wo.PartNumber).ToFixedLength(50), cts),
Write(moveInTagset.MoveInWorkOrderNumbers.Handle, request.WorkOrders.Select(wo => wo.WorkOrderNumber).ToFixedLength(50), cts),
Write(moveInTagset.MoveInFlag.Handle, true, cts) // TRIGGER — set last
};
await Task.WhenAll(writeTasks);
await Task.WhenAll(flagTask);
if (flagTask.Result) {
response.WasSuccessful = moveInTagset.MoveInSuccessfulFlag.Value;
response.ErrorText = moveInTagset.MoveInErrorText.Value;
if (moveInTagset.MoveInBatchID.Value != 0) response.BatchID = moveInTagset.MoveInBatchID.Value;
} else {
response.WasSuccessful = false;
response.ErrorText = "Timeout waiting for move in information to be processed";
}
moveInTagset.Tags.ForEach(Unadvise);
}
```
> There is **no busy-poll loop**: completion is event-driven via the MXAccess `OnDataChange`
> callback; the 30 s `CancellationTokenSource` is the only timeout.
### 1.6 Alarm-status path
1. Resolve machine from `MachineFilter` (`SAPID` / `Code` / `ZTag` / `MachineID`) — wrong/missing → error.
2. Load `MachineAlarm` rows for the machine; apply filters (`FlaggedOnly`, `MinSeverity`, `MaxSeverity`, case-insensitive `NameFilter.Contains`). *(`IncludeTriggered` is read but not used in the filter.)*
3. Subscribe + read each alarm's `Quality` and `InAlarm` (30 s budget). Bad quality / read failure → `ErrorText="Failed to read machine alarm status"`.
4. For alarms where `InAlarm == true`, additionally read `TimeAlarmOn`, `DescAttrName`, `Acked`, `TimeAlarmAcked`, `AckMsg`.
5. Build `AlarmInfo` per triggered alarm; `StatusCode = "Triggered.Acked"` if acked else `"Triggered"`. If `AlarmFilter.IncludeAcked == false`, acked alarms are skipped.
6. Unsubscribe; on failure `Alarms` is cleared.
### 1.7 Outputs / error handling (MES)
- **Transport status is always 200** for handled responses — success/failure is carried by the body's `WasSuccessful` flag + `ErrorText`. (401/403 only from the auth layer.)
- Success: `{ "WasSuccessful": true, "ErrorText": null, "BatchID": <int|null> }`.
- Failure/timeout: `{ "WasSuccessful": false, "ErrorText": "<message>", "BatchID": null }`.
- Distinct `ErrorText` values: machine-not-found, "Failed to connect to machine", "…ready flag not set to true", "Failed to write … to machine", "Timeout waiting for … to be processed", "Failed to read machine alarm status".
---
## 2. Delmia-DNC integration — `DelmiaIntegration` (+ Galaxy `$DelmiaReceiver`)
### 2.1 Topology — three hops
```
Operator (DelmiaIntegration.exe WinForms)
│ ① DelmiaClient ──HTTP POST (form-url-encoded)──▶ DELMIA/Apriso DNC "Downloader.asmx"
│ (e.g. http://dnc-app-vd01.zmr.zimmer.com/IntercimService/Downloader.asmx)
│ ◀── XML (SearchResults / DownloadResult, ns http://intercim.com/ruleset) ──
│ ② recipe file written to disk; WWNotifier.exe launched with CLI args
WWNotifier.exe ──HTTP POST (JSON RecipeDownload)──▶ WW receiver service (http://wonder-app-vd01:9001/notify)
◀── JSON RecipeDownloadResult ── │ ③ MXAccess COM write
Galaxy object {machine}.$DelmiaReceiver.* (recipe tags + flags)
+ ArchestrA scripts ProcessRecipe / Reset
```
**Assemblies in the repo:** `DelmiaContracts` (XML DTO library), `DelmiaIntegration` (`DelmiaClient` + WinForms UI),
`WWNotifier.exe` (console notifier), plus test harnesses (`AdminTestUtil`, `DownloadTestUtil`, `TestUI`).
> **Scope note.** Hops ① and ② are fully in this repo. Hop ③ — the service at `:9001/notify`
> that actually performs the MXAccess write — is **not** in the `ZimmerBiomet` Gitea org; only the
> JSON contract (below) and the Galaxy-side `$DelmiaReceiver` object (scripts + attributes, exported
> under `AA_EXPORT/`) are available. `WWNotificationSystem` *also* uses MXAccess but is an unrelated
> tag→email alerting service (port `:9876`, client name `WWNotifierMonitor`) — **not** the recipe receiver.
### 2.2 DNC server interface (`DelmiaClient`, hop ①)
- **Transport:** `HttpClient.PostAsync` with `FormUrlEncodedContent`; response is XML deserialized
with `XmlSerializer`. Base URL is `DelmiaClient.URL`; per-call `Timeout` default **30 s**.
Action is appended to the base URL (`URL.TrimEnd('/') + "/<Action>"`).
- **Base URL (from `AdminTestUtil` `DefaultURL`):** `http://dnc-app-vd01.zmr.zimmer.com/IntercimService/Downloader.asmx`.
- On any exception the client returns a result object with the error in `ErrorMessage` /
`TransferSuccessful=false` (it does not throw to the caller).
| Method (sync + `…Async`) | POST action | Form fields | Returns |
|---|---|---|---|
| `Search` | `/Search` | `username, machineID, partNumber, operationNumber` | `SearchResults` |
| `RequestProvenDocument` | `/RequestProvenDocument` | `username, machineID, partNumber, operationNumber, workOrderNumber` | `DownloadResult` |
| `RequestDocument` | `/RequestDocument` | `username, machineID, partNumber, operationNumber, workOrderNumber, documentKey` | `DownloadResult` |
DTO field lists (`DelmiaContracts`, XML namespace `http://intercim.com/ruleset`):
- **`SearchResults`**: `Results` (`List<SearchResult>`), `ErrorMessage` (string).
- **`SearchResult`**: `ShopOrderKey` (int), `ShopOrderID` (string), `ShopOrderStatus` (string),
`ShopOrderOperKey` (int), `ShopOrderOperID` (string), `ShopOrderOperStatus` (string),
`DocumentKey` (int), `DocumentObjectID` (int, with `…Specified` flag), `DocumentName` (string),
`DocumentRev` (string), `DocumentStatus` (string), `DocumentURL` (string), `PartID` (string), `PartRev` (string).
- **`DownloadResult`**: `UserKey` (int), `UserName` (string), `UserSite` (string), `MachineKey` (int),
`MachineID` (string), `MachineSite` (string), `WorkOrderNumber` (string), `ShopOrderKey` (int),
`ShopOrderID` (string), `ShopOrderStatus` (string), `ShopOrderOperKey` (int), `ShopOrderOperID` (string),
`ShopOrderOperStatus` (string), `DocumentKey` (int), `DocumentName` (string), `DocumentRev` (string),
`DocumentStatus` (string), `PartID` (string), `PartRev` (string), **`TransferSuccessful` (bool)**,
**`ErrorMessage` (string)**.
- **`MachineInfo`** (contract present; not called by current code): `MachineKey` (int), `MachineID` (string),
`MachineName` (string), `DownloadPath` (string), `MachineDescription` (string), `MachineSite` (string), `MachineStatus` (string).
- **`UserInfo`**: `UserKey` (int), `UserName` (string), `UserSite` (string), `IsActive` (bool).
**Recipe file:** the downloaded document is a key/value recipe file parsed by `DelmiaIntegration/Models/RecipeSet.cs`
(`KEY,VALUE` lines; typed accessors `GetString/GetInt/GetBool/GetFloat/GetDouble/...`). It is written to disk;
its path is what gets handed to `WWNotifier` (`--downloadpath`).
### 2.3 WWNotifier (hop ②) — invocation & handoff contract
`WWNotifier.exe` (uses `CommandLineParser`); CLI options (`CommandLineOptions`):
| Short | Long | Required | Field |
|---|---|---|---|
| `-d` | `--downloadpath` | yes | `DownloadPath` (recipe file path) |
| `-m` | `--machine` | yes | `MachineCode` |
| `-w` | `--workorder` | yes | `WorkOrderNumber` |
| `-p` | `--partnumber` | yes | `PartNumber` |
| `-s` | `--seqop` | no | `JobStepNumber` |
| `-u` | `--username` | no | `Username` |
Config (`WWNotifier/App.config`): `NotifyURL = http://wonder-app-vd01.zmr.zimmer.com:9001/notify`
(comma-separated list allowed — tried in order until one succeeds), `NotifyTimeout = 30` (seconds,
applied as the global Flurl HTTP timeout).
Handoff (Flurl): `url.PostJsonAsync(recipeDownload).ReceiveJson<RecipeDownloadResult>()`.
- **Request body `RecipeDownload`** (JSON): `MachineCode`, `DownloadPath`, `WorkOrderNumber`, `PartNumber`, `JobStepNumber`, `Username` (all string).
- **Response body `RecipeDownloadResult`** (JSON): `Result` (bool), `ResultText` (string).
- **Outputs:** prints `YES` and exit code `0` on success; prints `NO` + a message and sets exit code `-1`
on failure (parse error, missing `NotifyURL`/`NotifyTimeout`, `Result==false`, or HTTP exception).
*(Caveat: on a caught exception it logs `error.InnerException.Message`, which throws a NRE when there is no inner exception — so bare transport errors surface only as a generic failure.)*
### 2.4 MXAccess write — Galaxy `$DelmiaReceiver` object (hop ③)
The receiver service maps `RecipeDownload` fields onto the `$DelmiaReceiver` object instance selected by
`MachineCode`. Object attributes (from `AA_EXPORT/.../$DelmiaReceiver` export) and their roles:
| Attribute | Type | Role | Maps from |
|---|---|---|---|
| `ReadyFlag` | Boolean | gate — receiver expects `true` before writing | — |
| `DownloadPath` | String | data | `RecipeDownload.DownloadPath` |
| `WorkOrderNumber` | String | data | `RecipeDownload.WorkOrderNumber` |
| `PartNumber` | String | data | `RecipeDownload.PartNumber` |
| `JobStepNumber` | String | data | `RecipeDownload.JobStepNumber` |
| `Username` | String | data | `RecipeDownload.Username` |
| `RecipeDownloadFlag` | Boolean | **trigger** — set `true` to start processing | — |
| `RecipeProcessedFlag` | Boolean | completion — handshake waits on this | — |
| `RecipeProcessResult` | Boolean | result | → `RecipeDownloadResult.Result` |
| `RecipeProcessResultText` | String | result | → `RecipeDownloadResult.ResultText` |
*(`MachineCode` selects which receiver instance; it is not itself a written attribute.)*
**Galaxy-side handshake (authoritative — ArchestrA scripts on `$DelmiaReceiver`):**
`ProcessRecipe` (runs when `RecipeDownloadFlag` is set):
```
Me.RecipeDownloadFlag = false; ' clear trigger
Me.ReadyFlag = false; ' clear ready
try
Me.RecipeProcessResult = true;
Me.RecipeProcessResultText = "Success";
catch
Me.RecipeProcessResult = false;
Me.RecipeProcessResultText = "Failed to read recipe file";
endtry;
Me.RecipeProcessedFlag = true; ' signal completion
```
`Reset` (clears the slot for the next download):
```
Me.RecipeDownloadFlag = false; Me.RecipeProcessedFlag = false;
Me.RecipeProcessResult = false; Me.RecipeProcessResultText = "";
Me.DownloadPath = ""; Me.WorkOrderNumber = ""; Me.PartNumber = "";
Me.JobStepNumber = ""; Me.Username = "";
```
**Receiver-side sequence (inferred** — mirrors the MES handshake and is driven by the flags above; the C# source is not in the repo**):**
1. resolve the `$DelmiaReceiver` instance from `MachineCode`;
2. (optionally) verify `ReadyFlag == true`;
3. write `DownloadPath`, `WorkOrderNumber`, `PartNumber`, `JobStepNumber`, `Username`;
4. set `RecipeDownloadFlag = true` (trigger) → Galaxy `ProcessRecipe` fires;
5. wait for `RecipeProcessedFlag == true`, bounded by the request timeout;
6. read `RecipeProcessResult``Result`, `RecipeProcessResultText``ResultText`; return the `RecipeDownloadResult` JSON;
7. `Reset` the object.
### 2.5 Outputs / error handling (Delmia)
- **DNC server call:** failures are swallowed into the returned DTO — `SearchResults.ErrorMessage`, or
`DownloadResult.TransferSuccessful=false` + `ErrorMessage="Failed to call Delmia web service at '<URL>'."`.
- **Notify handoff:** `RecipeDownloadResult.Result` (bool) + `ResultText` (string); `WWNotifier` exit code
`0` (`YES`) / `-1` (`NO`).
- **Galaxy script:** `RecipeProcessResultText` is `"Success"` or `"Failed to read recipe file"`.
---
## 3. Side-by-side summary
| | MES (`MESAPI`) | Delmia DNC (`DelmiaIntegration`) |
|---|---|---|
| Caller | Camstar MES (HTTP/JSON) | Operator UI → DELMIA DNC server, then WWNotifier |
| API style | ServiceStack REST, `POST /mes/*` | DNC = form-url-encoded → XML; notify = JSON POST |
| Who writes MXAccess | the service (`MesNotifier`, in-repo) | external `:9001/notify` receiver (source not in repo) |
| MXAccess client | `LMXProxyServerClass`, register `"MesNotifier"` | `LMXProxyServerClass` (receiver), Galaxy `$DelmiaReceiver` scripts |
| Target object | `{MachineCode}.MesReceiver.*` | `{MachineCode}.$DelmiaReceiver` instance |
| Ready / trigger / complete | `MoveInReadyFlag` / `MoveInFlag` / `MoveInCompleteFlag` | `ReadyFlag` / `RecipeDownloadFlag` / `RecipeProcessedFlag` |
| Result / error | `MoveInSuccessfulFlag` / `MoveInErrorText` (+`MoveInBatchID`) | `RecipeProcessResult` / `RecipeProcessResultText` |
| Timeout | 30 s (`CancellationTokenSource(30000)`), event-driven | 30 s HTTP (`NotifyTimeout`); Galaxy wait at receiver |
---
## 4. Sources & open gaps
**Repos (Gitea `ZimmerBiomet`, `master`):**
- MES: `MESAPI``APIServer.ServiceInterface/MesServices.cs`, `MesNotifier.cs`,
`Mes{MoveIn,MoveOut}Tagset.cs`, `AlarmTagset.cs`, `Tag.cs`/`OnValueTask.cs`;
`APIServer.ServiceModel/Types/*`; `APIServer/AppHost.cs`, `App.config`.
- Delmia: `DelmiaIntegration``DelmiaIntegration/DelmiaClient.cs`, `Models/RecipeSet.cs`;
`WWNotifier/Program.cs`, `CommandLineOptions.cs`, `Models/RecipeDownload(Result).cs`, `App.config`;
`DelmiaContracts/*`.
**Galaxy export (`~/Desktop/AA_EXPORT/EXTRACTED/$DelmiaReceiver`):** `scripts/ProcessRecipe.txt`,
`scripts/Reset.txt`, `$DelmiaReceiver.top_level_attributes.csv`. MES receiver object attributes: [`mesrec.md`](mesrec.md).
**Open gaps / to verify against the box:**
1. The Delmia recipe **`/notify` receiver service** source (the actual MXAccess writer at `wonder-app-vd01:9001`)
was not found in the Gitea org — §2.4 receiver steps are inferred from the contract + Galaxy scripts.
2. MES tag prefix in code is `{Code}.MesReceiver.*`, while the live probe in `mesrec.md` shows a top-level
`MESReceiver_002` instance — confirm the exact contained-name/instance convention on the live Galaxy.
3. PROD `HttpListener`/DB host values should be read from the deployed `App.config`, not assumed.
+77
View File
@@ -0,0 +1,77 @@
# MESReceiver object — attributes (wonder-app-vd03 Galaxy)
Source: live AVEVA Galaxy DB **`ZB`** on **wonder-app-vd03** (the MxAccessGateway box),
read via the gateway's own `AttributesSql` (recursive `deployed_package_chain` over
`dynamic_attribute` / `attribute_definition`) run with `sqlcmd` over ssh — `2026-06-16`.
## Instance probed
- **TagName:** `MESReceiver_002` (GobjectId `5909`, ParentGobjectId `5908`, HostedBy `5049`)
- **TemplateChain:** `$MESDemo.MESReceiver``$MESReceiver``$gUserDefined``$UserDefined`
- **CategoryId:** 10, `IsArea = false`
- There are many MESReceiver instances on this Galaxy (1,253 `MESReceiver` references in the
`galaxy-snapshot.json` hierarchy cache). `MESReceiver_002` is representative of the template.
> Note: the cached `galaxy-snapshot.json` (gateway Server dir) holds the **object hierarchy only**
> — it carries **no attributes**, so attribute discovery requires the DB query (or gRPC `galaxy-discover`).
## Container number
There is **no attribute literally named `ContainerNumber`**. The MES container number is carried by:
- **`MoveInMesContainerNum`** — String (move-in interface)
- **`MoveOutMesContainerNum`** — String (move-out interface)
Full tag references: `MESReceiver_002.MoveInMesContainerNum`, `MESReceiver_002.MoveOutMesContainerNum`.
## MES interface UDAs (attribute-category 10, security-classification 1 = secured/operate)
### Move-In
| Attribute | Type | Full tag reference |
|---|---|---|
| **MoveInMesContainerNum** | String | `MESReceiver_002.MoveInMesContainerNum` |
| MoveInBatchID | Integer | `MESReceiver_002.MoveInBatchID` |
| MoveInJobSequenceNumber | String | `MESReceiver_002.MoveInJobSequenceNumber` |
| MoveInNumberWorkOrders | Integer | `MESReceiver_002.MoveInNumberWorkOrders` |
| MoveInWorkOrderNumbers | String[] | `MESReceiver_002.MoveInWorkOrderNumbers[]` |
| MoveInPartNumbers | String[] | `MESReceiver_002.MoveInPartNumbers[]` |
| MoveInOperatorName | String | `MESReceiver_002.MoveInOperatorName` |
| MoveInFlag | Boolean | `MESReceiver_002.MoveInFlag` |
| MoveInReadyFlag | Boolean | `MESReceiver_002.MoveInReadyFlag` |
| MoveInCompleteFlag | Boolean | `MESReceiver_002.MoveInCompleteFlag` |
| MoveInSuccessfulFlag | Boolean | `MESReceiver_002.MoveInSuccessfulFlag` |
| MoveInErrorText | String | `MESReceiver_002.MoveInErrorText` |
### Move-Out (symmetric; no JobSequenceNumber)
| Attribute | Type | Full tag reference |
|---|---|---|
| **MoveOutMesContainerNum** | String | `MESReceiver_002.MoveOutMesContainerNum` |
| MoveOutBatchID | Integer | `MESReceiver_002.MoveOutBatchID` |
| MoveOutNumberWorkOrders | Integer | `MESReceiver_002.MoveOutNumberWorkOrders` |
| MoveOutWorkOrderNumbers | String[] | `MESReceiver_002.MoveOutWorkOrderNumbers[]` |
| MoveOutPartNumbers | String[] | `MESReceiver_002.MoveOutPartNumbers[]` |
| MoveOutOperatorName | String | `MESReceiver_002.MoveOutOperatorName` |
| MoveOutFlag | Boolean | `MESReceiver_002.MoveOutFlag` |
| MoveOutReadyFlag | Boolean | `MESReceiver_002.MoveOutReadyFlag` |
| MoveOutCompleteFlag | Boolean | `MESReceiver_002.MoveOutCompleteFlag` |
| MoveOutSuccessfulFlag | Boolean | `MESReceiver_002.MoveOutSuccessfulFlag` |
| MoveOutErrorText | String | `MESReceiver_002.MoveOutErrorText` |
## Standard ArchestrA `$UserDefined` / system attributes (also present)
`AlarmCntsBySeverity[]`, `AlarmCntsBySeverityEnableShelved[]`, `AlarmInhibit`, `AlarmMode`,
`AlarmModeCmd`, `AlarmMostUrgentAcked`, `AlarmMostUrgentInAlarm`, `AlarmMostUrgentMode`,
`AlarmMostUrgentSeverity`, `AlarmMostUrgentShelved`, `AliasName`, `Area`, `CmdData`, `CodeBase`,
`ConfigVersion`, `ContainedName`, `Container`, `Errors[]`, `ExecutionRelatedObject`,
`ExecutionRelativeOrder`, `Extensions`, `HierarchicalName`, `Host`, `InAlarm`, `MinorVersion`,
`PropagatedAlarmInhibit`, `ScanState`, `ScanStateCmd`, `SecurityGroup`, `ShortDesc`, `Tagname`,
`UDAs`, `UserAttrData`, `PropagatedAlarmInhibit`.
## How to reproduce
1. `MESReceiver` instances are in the gateway hierarchy cache
`E:\ApiInstall\MxGateway\Server\galaxy-snapshot.json` (objects only, no attrs).
2. Attribute list: run the gateway's `AttributesSql`
(`src/.../Server/Galaxy/GalaxyRepository.cs`) scoped to one instance via
`... WHERE g.is_template = 0 AND g.deployed_package_id <> 0 AND g.tag_name = 'MESReceiver_002'`
in the `deployed_package_chain` anchor, then `sqlcmd -S (local) -d ZB -U wwadmin -P <pwd>`
on the box (Galaxy creds from `MxGateway.Galaxy.ConnectionString` in the gateway appsettings).
3. Live values would need an MXAccess read through the gateway (gRPC), not the repo SQL.
In OtOpcUa (Galaxy-as-standard-driver model) these bind as ordinary equipment tags:
`Tag{ DriverInstanceId = GalaxyMxGateway, TagConfig = {"FullName":"MESReceiver_002.MoveInMesContainerNum"} }`.
+212
View File
@@ -0,0 +1,212 @@
# New Jersey — DARS reactors Z28061 / Z28062 (wonder Galaxy)
Source: live AVEVA Galaxy DB **`ZB`** on **wonder-app-vd03** (MxAccessGateway box), via the gateway's `AttributesSql` over `sqlcmd`/ssh — `2026-06-16`. Lists the **meaningful (user-defined, cat-10) attributes**; ArchestrA system attributes (alarm framework, identity, security) and per-field config sub-attributes (`.EngUnits`, `.TrendHi`, `.Dev.*`, history/alarm settings) and scripts are omitted.
## Hierarchy
```
NewJersey (area)
└─ CVDAisle_1 ($CVDAisle, gobject 7243)
├─ Z28061 ($DARSReactor, gobject 7171)
│ ├─ Left (Left_002, $DARSReactor.Left, gobject 7172)
│ └─ Right (Right_002, $DARSReactor.Right, gobject 7173)
└─ Z28062 ($DARSReactor, gobject 7202)
├─ Left (Left_003, $DARSReactor.Left, gobject 7203)
└─ Right (Right_003, $DARSReactor.Right, gobject 7204)
```
(`Z28061Sim`, gobject 7146, is a simulator sibling of Z28061.)
## Objects
| Object | GobjectId | Template | Compound path | UDA attrs | Total attrs |
|---|---|---|---|---|---|
| Left_002 | 7172 | `$DARSReactor.Left` | Z28061.Left | 72 | 1907 |
| Left_003 | 7203 | `$DARSReactor.Left` | Z28062.Left | 72 | 1907 |
| Right_002 | 7173 | `$DARSReactor.Right` | Z28061.Right | 72 | 1907 |
| Right_003 | 7204 | `$DARSReactor.Right` | Z28062.Right | 72 | 1907 |
| Z28061 | 7171 | `$DARSReactor` | Z28061 | 16 | 195 |
| Z28062 | 7202 | `$DARSReactor` | Z28062 | 16 | 195 |
> Attribute sets are template-defined: `Z28061`≡`Z28062`, `Left_002`≡`Left_003`, `Right_002`≡`Right_003`. Listed once per template.
## $DARSReactor — Z28061 / Z28062 (reactor body)
**Instances:** `Z28061` (Z28061, gobject 7171), `Z28062` (Z28062, gobject 7202)
**Template:** `$DARSReactor` · **16 meaningful (UDA) attributes** · 195 total incl. system + config sub-attributes.
| Attribute | Type | Hist | Alarm |
|---|---|---|---|
| ChlorineFlow | Integer | ✓ | |
| ConfirmTimeoutSec | Integer | | |
| Deadband | Double | | |
| HeartbeatInterval | Integer | | |
| HeartbeatRequest | Boolean | | |
| HeartbeatTimeout | Integer | | |
| HeartbeatTimeoutAlarm | Boolean | | ✓ |
| HydrogenFlow | Integer | ✓ | |
| LeakTestMaxDelta | Integer | | |
| LeakTestMinDuration | Integer | | |
| LeakTestTimeout | Integer | | |
| MachineCode | String | | |
| MachineDescription | String | | |
| MachineID | String | | |
| Runtime_Alert | Boolean | | |
| TrunkPressure | Float | ✓ | |
## $DARSReactor.Left — Left_002 / Left_003 (left chamber)
**Instances:** `Left_002` (Z28061.Left, gobject 7172), `Left_003` (Z28062.Left, gobject 7203)
**Template:** `$DARSReactor.Left` · **72 meaningful (UDA) attributes** · 1907 total incl. system + config sub-attributes.
| Attribute | Type | Hist | Alarm |
|---|---|---|---|
| AIR_SVC | String | | |
| AirAvg | Integer | ✓ | |
| AirFlow | Integer | ✓ | |
| AirFlowDevAlertPcnt | Float | ✓ | |
| AR_SVC | String | | |
| ArAvg | Integer | ✓ | |
| ArgonFlow | Integer | ✓ | |
| ChlorineFlowR | Integer | | |
| CL2_SVC | String | | |
| Cl2Avg | Integer | ✓ | |
| ClFlowDevAlertPcnt | Float | ✓ | |
| CoilDiameter | Boolean | ✓ | |
| CoilHeight | Integer | ✓ | |
| ContainerID | String | ✓ | |
| ContainerLoaded | Boolean | ✓ | |
| CurrentStep | Integer | | |
| CycleEndConfirm | Boolean | | |
| CycleEndNotify | Boolean | | |
| CycleEndTimeoutAlarm | Boolean | | |
| CycleRunning | Boolean | ✓ | |
| CycleStartConfirm | Boolean | | |
| CycleStartNotify | Boolean | | |
| CycleStartTimeoutAlarm | Boolean | | |
| Deadband | Double | | |
| DelmiaJobStep | Integer | | |
| FRR_Reset | Boolean | ✓ | |
| FRR_Runtime | Integer | ✓ | |
| FRR_Warning | Boolean | ✓ | ✓ |
| FurnaceTemp | Integer | ✓ | |
| FurnaceTempOverAlarmDev | Integer | ✓ | |
| FurnaceTempOverAlertDev | Integer | ✓ | |
| FurnaceTempUnderAlarmDev | Integer | ✓ | |
| FurnaceTempUnderAlertDev | Integer | ✓ | |
| H2_SVC | String | | |
| H2Avg | Integer | ✓ | |
| HydrogenFlowR | Integer | | |
| HyFlowDevAlertPcnt | Float | ✓ | |
| MachineBatchID | Integer | | |
| MachineBatchWOID | Integer | | |
| MachineCycleID | Integer | | |
| MantleTemp | Integer | ✓ | |
| MoveInReady | Boolean | | |
| MoveOutReady | Boolean | | |
| MTA_Lockout | Boolean | ✓ | ✓ |
| MTA_Reset | Boolean | ✓ | |
| MTA_Runtime | Integer | ✓ | |
| MTA_Warning | Boolean | ✓ | ✓ |
| NumOfParts | Integer | ✓ | |
| NumOfTurns | Integer | ✓ | |
| OperatorID | String | ✓ | |
| PartNumber | String | ✓ | |
| PID_Alarm | Boolean | | ✓ |
| PID_AlarmString | String | | |
| PID_CV | Integer | ✓ | |
| PotNumber | String | ✓ | |
| PotTempAvg | Integer | ✓ | |
| PreviousStep | Integer | | |
| ReactorTempAvg | Integer | ✓ | |
| RunDuration | Integer | | |
| RunEndTime | Time | | |
| RunStartTime | Time | | |
| SampleCount | Integer | | |
| StartingWeight | Integer | ✓ | |
| TableSide | Boolean | | |
| TableStatus | Integer | | |
| TC_Furnace_ID | String | | |
| TC_Mantle_ID | String | | |
| TC_Spare_ID | String | | |
| Vacuum | Float | ✓ | |
| VacuumAvg | Float | ✓ | |
| VacuumMode | Boolean | ✓ | |
| WorkOrder | String | | |
## $DARSReactor.Right — Right_002 / Right_003 (right chamber)
**Instances:** `Right_002` (Z28061.Right, gobject 7173), `Right_003` (Z28062.Right, gobject 7204)
**Template:** `$DARSReactor.Right` · **72 meaningful (UDA) attributes** · 1907 total incl. system + config sub-attributes.
| Attribute | Type | Hist | Alarm |
|---|---|---|---|
| AIR_SVC | String | | |
| AirAvg | Integer | ✓ | |
| AirFlow | Integer | ✓ | |
| AirFlowDevAlertPcnt | Float | ✓ | |
| AR_SVC | String | | |
| ArAvg | Integer | ✓ | |
| ArgonFlow | Integer | ✓ | |
| ChlorineFlowR | Integer | | |
| CL2_SVC | String | | |
| Cl2Avg | Integer | ✓ | |
| ClFlowDevAlertPcnt | Float | ✓ | |
| CoilDiameter | Boolean | ✓ | |
| CoilHeight | Integer | ✓ | |
| ContainerID | String | ✓ | |
| ContainerLoaded | Boolean | ✓ | |
| CurrentStep | Integer | | |
| CycleEndConfirm | Boolean | | |
| CycleEndNotify | Boolean | | |
| CycleEndTimeoutAlarm | Boolean | | |
| CycleRunning | Boolean | ✓ | |
| CycleStartConfirm | Boolean | | |
| CycleStartNotify | Boolean | | |
| CycleStartTimeoutAlarm | Boolean | | |
| Deadband | Double | | |
| DelmiaJobStep | Integer | | |
| FRR_Reset | Boolean | ✓ | |
| FRR_Runtime | Integer | ✓ | |
| FRR_Warning | Boolean | ✓ | ✓ |
| FurnaceTemp | Integer | ✓ | |
| FurnaceTempOverAlarmDev | Integer | ✓ | |
| FurnaceTempOverAlertDev | Integer | ✓ | |
| FurnaceTempUnderAlarmDev | Integer | ✓ | |
| FurnaceTempUnderAlertDev | Integer | ✓ | |
| H2_SVC | String | | |
| H2Avg | Integer | ✓ | |
| HydrogenFlowR | Integer | | |
| HyFlowDevAlertPcnt | Float | ✓ | |
| MachineBatchID | Integer | | |
| MachineBatchWOID | Integer | | |
| MachineCycleID | Integer | | |
| MantleTemp | Integer | ✓ | |
| MoveInReady | Boolean | | |
| MoveOutReady | Boolean | | |
| MTA_Lockout | Boolean | ✓ | ✓ |
| MTA_Reset | Boolean | ✓ | |
| MTA_Runtime | Integer | ✓ | |
| MTA_Warning | Boolean | ✓ | ✓ |
| NumOfParts | Integer | ✓ | |
| NumOfTurns | Integer | ✓ | |
| OperatorID | String | ✓ | |
| PartNumber | String | ✓ | |
| PID_Alarm | Boolean | | ✓ |
| PID_AlarmString | String | | |
| PID_CV | Integer | ✓ | |
| PotNumber | String | ✓ | |
| PotTempAvg | Integer | ✓ | |
| PreviousStep | Integer | | |
| ReactorTempAvg | Integer | ✓ | |
| RunDuration | Integer | | |
| RunEndTime | Time | | |
| RunStartTime | Time | | |
| SampleCount | Integer | | |
| StartingWeight | Integer | ✓ | |
| TableSide | Boolean | | |
| TableStatus | Integer | | |
| TC_Furnace_ID | String | | |
| TC_Mantle_ID | String | | |
| TC_Spare_ID | String | | |
| Vacuum | Float | ✓ | |
| VacuumAvg | Float | ✓ | |
| VacuumMode | Boolean | ✓ | |
| WorkOrder | String | | |
+38 -15
View File
@@ -113,22 +113,45 @@ Defaults target docker-dev; override via flags or env:
Besides the galaxy-native mirror, the tool can load the **Northwind company Besides the galaxy-native mirror, the tool can load the **Northwind company
shape** (`filling / line-1 / rinser-01 / speed-rpm`) as a second, **Equipment**-kind shape** (`filling / line-1 / rinser-01 / speed-rpm`) as a second, **Equipment**-kind
namespace (`nw-uns`, in cluster `MAIN`) from `../company-uns.json`. This needs namespace (`nw-uns`, in cluster `MAIN`) from `../company-uns.json`. Each company
OtOpcUa `master` ≥ the Equipment-namespace structure milestone signal is a **VirtualTag** (+ a `Script`) whose script simply mirrors the live
(`febe462…9a67ebc`), which materialises Equipment `Tag`/`VirtualTag` rows on galaxy-mirror tag for that signal:
deploy and added a **headless deploy** endpoint.
```bash ```csharp
./.venv/bin/python otopcua_uns.py populate-equipment # 3 areas / 8 lines / 40 equipment / 1036 signals return ctx.GetTag("TestMachine_001.TestDouble").Value;
curl -s -X POST http://localhost:9200/api/deployments -H 'X-Api-Key: docker-dev-deploy-key' # headless deploy
./.venv/bin/python otopcua_uns.py verify-equipment --expect 1036 # browse the company tree (nw-area-* scope)
``` ```
UNS folders carry the friendly **DisplayName** (`filling`); the BrowseName/NodeId so the company shape carries live **VALUES** driven off the same Galaxy source — no
stay the stable logical Id (`nw-area-filling`) — standard OPC UA. **Structure-only:** driver, no `BadWaitingForInitialData` once the galaxy mirror is up. The `ctx.GetTag`
the company leaves materialise as `BadWaitingForInitialData` — live **values** in literal is the signal's `source.fullTagReference`; the engine's `DependencyExtractor`
the company shape are the next OtOpcUa milestone (driver/VirtualTag source), tracked harvests it and subscribes the VirtualTag to that galaxy-mirror tag. This needs
in `OtOpcUa/docs/plans/2026-06-06-equipment-namespace-materialization-scope.md` (WS-3). OtOpcUa `master` ≥ the Equipment-namespace VirtualTag materialisation milestone (WS-3),
The galaxy-native mirror (`populate`) still carries live values. which materialises `VirtualTag`/`Script` rows on deploy and added the **headless
deploy** endpoint.
`clean` removes both the mirror tags and the company overlay. ```bash
./.venv/bin/python otopcua_uns.py populate-equipment # 3 areas / 8 lines / 40 equipment / 1036 VirtualTags
curl -s -X POST http://localhost:9200/api/deployments -H 'X-Api-Key: docker-dev-deploy-key' # headless deploy
./.venv/bin/python otopcua_uns.py verify-equipment --expect 1036 --require-good 396 --wait --wait-seconds 300 # structure + live values
```
> **Verified live 2026-06-07** (OtOpcUa `feat/equipment-namespace-live-values`): galaxy mirror
> **396/396 Good**, company overlay **396 Good** on `opc.tcp://localhost:4840`, `VERIFY-EQUIPMENT: PASS`.
> Why 396 of 1036? The shipped `company-uns.json` invents **1036 distinct** `ctx.GetTag` refs, but only
> **396** of them match a real galaxy-mirror tag — so 396 signals are backed by a live source (and all 396
> go Good); the other 640 cite synthetic refs with no galaxy tag (`BadNodeIdUnknown`). That ratio is a
> property of the company model, not the streaming path — **every signal with a resolvable live source
> streams Good.** So `--require-good 396` is the meaningful gate for the current model. Survives a node
> restart with no re-deploy (the bootstrap-restore path re-materialises + re-applies the VirtualTags).
UNS folders carry the friendly **DisplayName** (`filling`); the BrowseName/NodeId
stay the stable logical Id (`nw-area-filling`) — standard OPC UA. **No driver:** the
company signals are VirtualTags (which link to Equipment + a Script, not a driver); a
placeholder `nw-uns-modbus` driver is kept only because an Equipment namespace is
expected to have one, but no `Tag` binds to it. `verify-equipment --require-good N`
reads each leaf's value and asserts at least N are Good (default `0` = structure-only,
back-compat); `--wait` polls until the deploy + change-triggered evaluations land.
Tracked in `OtOpcUa/docs/plans/2026-06-06-equipment-namespace-materialization-scope.md` (WS-3).
`clean` removes both the mirror tags and the company overlay (the `VirtualTag` +
`Script` rows, in FK-safe order, plus the namespace/driver/equipment/areas/lines).
+148 -62
View File
@@ -25,12 +25,21 @@ the `nw-` id prefix so `clean` can remove them without touching other config.
Idempotent: populate upserts by TagId; re-running is a no-op when unchanged. Idempotent: populate upserts by TagId; re-running is a no-op when unchanged.
There are TWO overlays:
the galaxy-native mirror (`populate`) SystemPlatform driver Tags, 396 tags;
the Northwind company shape (`populate-equipment`) an Equipment-kind namespace
whose 1036 signals are VirtualTags. Each VirtualTag's Script simply mirrors the
live galaxy-mirror tag (`return ctx.GetTag("<fullTagReference>").Value;`), so the
company shape carries live VALUES driven off the same Galaxy source.
Subcommands: Subcommands:
generate Build the load plan from galaxy-hierarchy.json (writes load-plan.json) generate Build the load plan from galaxy-hierarchy.json (writes load-plan.json)
populate Upsert the SystemPlatform mirror Tag rows into the config DB populate Upsert the SystemPlatform mirror Tag rows into the config DB
verify Check DB rows present + live OPC UA values are Good on :4840 populate-equipment Load the company shape as VirtualTag+Script rows (mirror the galaxy tags)
status Show config-DB + address-space state verify Check DB rows present + live OPC UA values are Good on :4840
clean Delete all nw-* mirror Tag rows verify-equipment Browse the company tree; --require-good asserts live values
status Show config-DB + address-space state
clean Delete all nw-* mirror Tags + the company VirtualTag/Script overlay
Deploy is a human-gated AdminUI action (no SQL/REST trigger exists); populate Deploy is a human-gated AdminUI action (no SQL/REST trigger exists); populate
and clean print the reminder and `verify --wait` polls until it lands. and clean print the reminder and `verify --wait` polls until it lands.
@@ -69,11 +78,12 @@ LOAD_PLAN = os.path.join(os.path.dirname(__file__), "load-plan.json")
# Equipment-overlay (company-shape) object ids — all carry the nw- prefix so # Equipment-overlay (company-shape) object ids — all carry the nw- prefix so
# `clean` can remove them. The Equipment namespace is a SECOND namespace loaded # `clean` can remove them. The Equipment namespace is a SECOND namespace loaded
# alongside the galaxy mirror; its leaves stay BadWaitingForInitialData until the # alongside the galaxy mirror. Each company signal is a VirtualTag (+ Script) whose
# value milestone (scope doc WS-3) wires a driver/VirtualTag source. # script mirrors the live SystemPlatform galaxy-mirror tag for that signal — so the
# overlay carries live VALUES (scope doc WS-3), not just structure.
EQ_CLUSTER = os.environ.get("OTOPCUA_EQ_CLUSTER", "MAIN") EQ_CLUSTER = os.environ.get("OTOPCUA_EQ_CLUSTER", "MAIN")
EQ_NS = "nw-uns" EQ_NS = "nw-uns"
EQ_DRIVER = "nw-uns-modbus" # non-Galaxy FK driver (structure-only; doesn't stream) EQ_ID_PREFIX = "nweq-" # VirtualTag/Script logical-id prefix (cleanup by prefix scan)
# galaxy dataTypeName / gen_uns dtype → valid OtOpcUa DriverDataType # galaxy dataTypeName / gen_uns dtype → valid OtOpcUa DriverDataType
_DTYPE_FIX = {"Double": "Float64", "Float": "Float32"} _DTYPE_FIX = {"Double": "Float64", "Float": "Float32"}
@@ -212,33 +222,47 @@ def cmd_populate(args):
return 0 return 0
def _eq_signal_ids(equipment_id, folder, name):
"""Deterministic (VirtualTagId, ScriptId) for a company signal. Both carry the
EQ_ID_PREFIX so `clean` removes exactly what was created. The two ids share the
same per-signal hash but differ by a kind token so they never collide across the
global UX_VirtualTag_LogicalId / UX_Script_LogicalId unique indexes. Capped at the
64-char id column width."""
base = hashlib.sha1(f"{equipment_id}|{folder}|{name}".encode()).hexdigest()[:20]
return EQ_ID_PREFIX + "vt-" + base, EQ_ID_PREFIX + "sc-" + base
def cmd_populate_equipment(args): def cmd_populate_equipment(args):
"""Load the company-shape Equipment namespace from company-uns.json: a second """Load the company-shape Equipment namespace from company-uns.json: a second
(Equipment-kind) namespace alongside the galaxy mirror, with the Northwind (Equipment-kind) namespace alongside the galaxy mirror, with the Northwind
Area/Line/Equipment/Signal tree. Structure-only leaves materialise as Area/Line/Equipment/Signal tree. Each signal is a VirtualTag whose Script mirrors
BadWaitingForInitialData (the value milestone is separate). Idempotent: the live galaxy-mirror tag for that signal `return ctx.GetTag("<ref>").Value;`
so the company shape streams live VALUES off the same Galaxy source (no driver,
no BadWaitingForInitialData once the galaxy mirror is up). Idempotent:
drop-and-recreate of the nw- overlay rows.""" drop-and-recreate of the nw- overlay rows."""
with open(args.company_json) as f: with open(args.company_json) as f:
doc = json.load(f) doc = json.load(f)
u = doc["uns"] u = doc["uns"]
conn, cur = connect(args.mssql) conn, cur = connect(args.mssql)
# Drop any prior overlay (child rows first), then recreate. # Drop any prior overlay (child rows first), then recreate. VirtualTag/Script go
cur.execute("DELETE FROM dbo.Tag WHERE DriverInstanceId=%s", (EQ_DRIVER,)) # before Equipment (VirtualTag.EquipmentId logical-FKs Equipment). Equipment is
cur.execute("DELETE FROM dbo.Equipment WHERE DriverInstanceId=%s", (EQ_DRIVER,)) # scoped by its overlay UnsLine ('nw-line-%') — NOT by the EquipmentId, which is now the
# canonical 'EQ-'+uuid form (see DraftValidator) and no longer carries an 'nw-' prefix.
cur.execute("DELETE FROM dbo.VirtualTag WHERE VirtualTagId LIKE %s", (EQ_ID_PREFIX + "%",))
cur.execute("DELETE FROM dbo.Script WHERE ScriptId LIKE %s", (EQ_ID_PREFIX + "%",))
cur.execute("DELETE FROM dbo.Equipment WHERE UnsLineId LIKE 'nw-line-%'")
cur.execute("DELETE FROM dbo.UnsLine WHERE UnsLineId LIKE 'nw-line-%'") cur.execute("DELETE FROM dbo.UnsLine WHERE UnsLineId LIKE 'nw-line-%'")
cur.execute("DELETE FROM dbo.UnsArea WHERE UnsAreaId LIKE 'nw-area-%'") cur.execute("DELETE FROM dbo.UnsArea WHERE UnsAreaId LIKE 'nw-area-%'")
cur.execute("DELETE FROM dbo.DriverInstance WHERE DriverInstanceId=%s", (EQ_DRIVER,)) # Equipment is now driver-less, but purge any driver still bound to the overlay namespace —
# self-heals environments that ran an older loader which created the 'nw-uns-modbus' placeholder.
cur.execute("DELETE FROM dbo.DriverInstance WHERE NamespaceId=%s", (EQ_NS,))
cur.execute("DELETE FROM dbo.Namespace WHERE NamespaceId=%s", (EQ_NS,)) cur.execute("DELETE FROM dbo.Namespace WHERE NamespaceId=%s", (EQ_NS,))
cur.execute( cur.execute(
"INSERT INTO dbo.Namespace (NamespaceRowId, NamespaceId, ClusterId, Kind, NamespaceUri, Enabled) " "INSERT INTO dbo.Namespace (NamespaceRowId, NamespaceId, ClusterId, Kind, NamespaceUri, Enabled) "
"VALUES (NEWID(), %s, %s, 'Equipment', %s, 1)", "VALUES (NEWID(), %s, %s, 'Equipment', %s, 1)",
(EQ_NS, EQ_CLUSTER, doc.get("namespace", {}).get("namespaceUri", "urn:northwind:birmingham:uns"))) (EQ_NS, EQ_CLUSTER, doc.get("namespace", {}).get("namespaceUri", "urn:northwind:birmingham:uns")))
cur.execute(
"INSERT INTO dbo.DriverInstance (DriverInstanceRowId, DriverInstanceId, ClusterId, NamespaceId, "
"Name, DriverType, Enabled, DriverConfig) VALUES (NEWID(), %s, %s, %s, 'Northwind UNS placeholder', 'Modbus', 1, '{}')",
(EQ_DRIVER, EQ_CLUSTER, EQ_NS))
for a in u["unsAreas"]: for a in u["unsAreas"]:
cur.execute("INSERT INTO dbo.UnsArea (UnsAreaRowId, UnsAreaId, ClusterId, Name) VALUES (NEWID(), %s, %s, %s)", cur.execute("INSERT INTO dbo.UnsArea (UnsAreaRowId, UnsAreaId, ClusterId, Name) VALUES (NEWID(), %s, %s, %s)",
@@ -247,42 +271,50 @@ def cmd_populate_equipment(args):
cur.execute("INSERT INTO dbo.UnsLine (UnsLineRowId, UnsLineId, UnsAreaId, Name) VALUES (NEWID(), %s, %s, %s)", cur.execute("INSERT INTO dbo.UnsLine (UnsLineRowId, UnsLineId, UnsAreaId, Name) VALUES (NEWID(), %s, %s, %s)",
("nw-" + l["unsLineId"], "nw-" + l["unsAreaId"], l["name"])) ("nw-" + l["unsLineId"], "nw-" + l["unsAreaId"], l["name"]))
eq_n = tag_n = 0 eq_n = vt_n = 0
for e in u["equipment"]: for e in u["equipment"]:
eq_id = "nw-" + e["equipmentId"] eq_uuid = uuid.uuid5(uuid.NAMESPACE_URL, "otopcua-nw-eq/" + e["equipmentId"])
eq_uuid = str(uuid.uuid5(uuid.NAMESPACE_URL, "otopcua-nw-eq/" + e["equipmentId"])) # Canonical EquipmentId: matches OtOpcUa DraftValidator.DeriveEquipmentId
# ("EQ-" + EquipmentUuid.ToString("N")[..12].ToLowerInvariant()). uuid.UUID.hex is
# already lowercase, 32 hex chars, no dashes — .hex[:12] is the first 12.
eq_id = "EQ-" + eq_uuid.hex[:12].lower()
eq_uuid = str(eq_uuid)
cur.execute( cur.execute(
"INSERT INTO dbo.Equipment (EquipmentRowId, EquipmentId, EquipmentUuid, DriverInstanceId, UnsLineId, " "INSERT INTO dbo.Equipment (EquipmentRowId, EquipmentId, EquipmentUuid, UnsLineId, "
"Name, MachineCode, Manufacturer, Model, Enabled) VALUES (NEWID(), %s, %s, %s, %s, %s, %s, %s, %s, 1)", "Name, MachineCode, Manufacturer, Model, Enabled) VALUES (NEWID(), %s, %s, %s, %s, %s, %s, %s, 1)",
(eq_id, eq_uuid, EQ_DRIVER, "nw-" + e["unsLineId"], e["name"], e["machineCode"], (eq_id, eq_uuid, "nw-" + e["unsLineId"], e["name"], e["machineCode"],
e.get("manufacturer"), e.get("model"))) e.get("manufacturer"), e.get("model")))
eq_n += 1 eq_n += 1
for t in e["tags"]: for t in e["tags"]:
dtype = _DTYPE_FIX.get(t["dataType"], t["dataType"]) dtype = _DTYPE_FIX.get(t["dataType"], t["dataType"])
access = _ACCESS.get(t["accessLevel"], "0") folder = t.get("folderPath")
folder = t.get("folderPath") or None # The galaxy-mirror MXAccess ref (e.g. TestMachine_001.TestDouble) is the upstream
# Local NodeId == TagConfig.FullName; prefix with nw: so it never collides with the # the VirtualTag mirrors. DependencyExtractor harvests the literal in ctx.GetTag(),
# galaxy-mirror SystemPlatform NodeIds (which use the bare MXAccess ref). # so the engine subscribes to exactly this path on the galaxy-mirror driver.
full = "nw:" + t["source"]["fullTagReference"] full = t["source"]["fullTagReference"]
# TagId is capped at 64 chars; a short stable hash keeps it unique. Cleanup is by vt_id, sc_id = _eq_signal_ids(e["equipmentId"], folder, t["name"])
# DriverInstanceId (not TagId), so no prefix scan is needed. source_code = f'return ctx.GetTag("{full}").Value;'
tag_id = "nweq-" + hashlib.sha1( source_hash = hashlib.sha256(source_code.encode()).hexdigest()
f"{e['equipmentId']}|{folder}|{t['name']}".encode()).hexdigest()[:20]
cfg = json.dumps({"FullName": full, "DataType": dtype})
cur.execute( cur.execute(
"INSERT INTO dbo.Tag (TagRowId, TagId, DriverInstanceId, EquipmentId, Name, FolderPath, " "INSERT INTO dbo.Script (ScriptRowId, ScriptId, Name, SourceCode, SourceHash, Language) "
"DataType, AccessLevel, WriteIdempotent, TagConfig) VALUES (NEWID(), %s, %s, %s, %s, %s, %s, %s, 0, %s)", "VALUES (NEWID(), %s, %s, %s, %s, 'CSharp')",
(tag_id, EQ_DRIVER, eq_id, t["name"], folder, dtype, access, cfg)) (sc_id, t["name"], source_code, source_hash))
tag_n += 1 cur.execute(
"INSERT INTO dbo.VirtualTag (VirtualTagRowId, VirtualTagId, EquipmentId, Name, DataType, "
"ScriptId, ChangeTriggered, TimerIntervalMs, Historize, Enabled) "
"VALUES (NEWID(), %s, %s, %s, %s, %s, 1, NULL, %s, 1)",
(vt_id, eq_id, t["name"], dtype, sc_id, 1 if t.get("historize") else 0))
vt_n += 1
conn.commit() conn.commit()
conn.close() conn.close()
print(f"populated equipment overlay: namespace {EQ_NS} ({EQ_CLUSTER}), " print(f"populated equipment overlay: namespace {EQ_NS} ({EQ_CLUSTER}), "
f"{len(u['unsAreas'])} areas, {len(u['unsLines'])} lines, {eq_n} equipment, {tag_n} signals") f"{len(u['unsAreas'])} areas, {len(u['unsLines'])} lines, {eq_n} equipment, "
f"{vt_n} VirtualTags (+ {vt_n} mirror Scripts)")
print() print()
print(f">>> NEXT: deploy (headless) — curl -s -X POST {args.deploy_url.replace('/deployments','')}/api/deployments " print(f">>> NEXT: deploy (headless) — curl -s -X POST {args.deploy_url.replace('/deployments','')}/api/deployments "
f"-H 'X-Api-Key: {args.deploy_key}'") f"-H 'X-Api-Key: {args.deploy_key}'")
print(">>> then run: otopcua_uns.py verify-equipment") print(">>> then run: otopcua_uns.py verify-equipment --require-good 1036 --wait")
return 0 return 0
@@ -290,12 +322,18 @@ def cmd_clean(args):
conn, cur = connect(args.mssql) conn, cur = connect(args.mssql)
cur.execute("DELETE FROM dbo.Tag WHERE TagId LIKE %s", (ID_PREFIX + "%",)) cur.execute("DELETE FROM dbo.Tag WHERE TagId LIKE %s", (ID_PREFIX + "%",))
n = cur.rowcount n = cur.rowcount
# Also drop the company-shape Equipment overlay (child rows first). # Also drop the company-shape Equipment overlay (child rows first): VirtualTag and
cur.execute("DELETE FROM dbo.Tag WHERE DriverInstanceId=%s", (EQ_DRIVER,)) # Script (both nweq-*) before Equipment, then the rest.
cur.execute("DELETE FROM dbo.Equipment WHERE DriverInstanceId=%s", (EQ_DRIVER,)) # Equipment is scoped by its overlay UnsLine ('nw-line-%') — NOT by the EquipmentId,
# which is now the canonical 'EQ-'+uuid form (see DraftValidator) with no 'nw-' prefix.
cur.execute("DELETE FROM dbo.VirtualTag WHERE VirtualTagId LIKE %s", (EQ_ID_PREFIX + "%",))
cur.execute("DELETE FROM dbo.Script WHERE ScriptId LIKE %s", (EQ_ID_PREFIX + "%",))
cur.execute("DELETE FROM dbo.Equipment WHERE UnsLineId LIKE 'nw-line-%'")
cur.execute("DELETE FROM dbo.UnsLine WHERE UnsLineId LIKE 'nw-line-%'") cur.execute("DELETE FROM dbo.UnsLine WHERE UnsLineId LIKE 'nw-line-%'")
cur.execute("DELETE FROM dbo.UnsArea WHERE UnsAreaId LIKE 'nw-area-%'") cur.execute("DELETE FROM dbo.UnsArea WHERE UnsAreaId LIKE 'nw-area-%'")
cur.execute("DELETE FROM dbo.DriverInstance WHERE DriverInstanceId=%s", (EQ_DRIVER,)) # Purge any driver still bound to the overlay namespace (e.g. the legacy 'nw-uns-modbus'
# placeholder created by an older loader) so 'clean' fully removes the overlay.
cur.execute("DELETE FROM dbo.DriverInstance WHERE NamespaceId=%s", (EQ_NS,))
cur.execute("DELETE FROM dbo.Namespace WHERE NamespaceId=%s", (EQ_NS,)) cur.execute("DELETE FROM dbo.Namespace WHERE NamespaceId=%s", (EQ_NS,))
conn.commit() conn.commit()
conn.close() conn.close()
@@ -398,16 +436,28 @@ def sample_values(endpoint, n):
return [("<browse error>", str(e), "?")] return [("<browse error>", str(e), "?")]
def browse_tree(endpoint, max_depth=8, top_prefix=None): def browse_tree(endpoint, max_depth=8, top_prefix=None, read_values=False):
"""Recursively descend the OtOpcUa address space and count leaf variables, returning """Recursively descend the OtOpcUa address space and count leaf variables, returning
(folder_count, leaf_count, leaf_paths). A node with no children is a leaf signal this (folder_count, leaf_count, leaf_paths, good_count). A node with no children is a leaf
correctly handles the DEEP Equipment UNS tree (Area/Line/Equipment/[FolderPath]/Signal), signal this correctly handles the DEEP Equipment UNS tree
unlike browse_summary which assumes the flat 2-level Galaxy hierarchy. When top_prefix is (Area/Line/Equipment/[FolderPath]/Signal), unlike browse_summary which assumes the flat
set, only top-level OtOpcUa folders whose browse name starts with it are counted (e.g. 2-level Galaxy hierarchy. When top_prefix is set, only top-level OtOpcUa folders whose
'nw-area-' scopes to the company Equipment overlay, excluding the Galaxy mirror folders).""" browse name starts with it are counted (e.g. 'nw-area-' scopes to the company Equipment
overlay, excluding the Galaxy mirror folders). When read_values is True, each leaf's value
is read and good_count tallies the Good-quality ones (else good_count is 0)."""
import asyncio import asyncio
from asyncua import Client from asyncua import Client
async def maybe_good(node, acc):
if not read_values:
return
try:
dv = await node.read_data_value()
if dv.StatusCode and dv.StatusCode.is_good():
acc["good"] += 1
except Exception:
pass
async def walk(node, path, depth, acc): async def walk(node, path, depth, acc):
if depth >= max_depth: if depth >= max_depth:
return return
@@ -424,9 +474,10 @@ def browse_tree(endpoint, max_depth=8, top_prefix=None):
else: else:
acc["leaves"] += 1 acc["leaves"] += 1
acc["paths"].append(child_path) acc["paths"].append(child_path)
await maybe_good(ch, acc)
async def run(): async def run():
acc = {"folders": 0, "leaves": 0, "paths": []} acc = {"folders": 0, "leaves": 0, "paths": [], "good": 0}
async with Client(endpoint) as c: async with Client(endpoint) as c:
for k in await c.nodes.objects.get_children(): for k in await c.nodes.objects.get_children():
if (await k.read_browse_name()).Name != "OtOpcUa": if (await k.read_browse_name()).Name != "OtOpcUa":
@@ -441,32 +492,61 @@ def browse_tree(endpoint, max_depth=8, top_prefix=None):
else: else:
acc["leaves"] += 1 acc["leaves"] += 1
acc["paths"].append("OtOpcUa/" + tn) acc["paths"].append("OtOpcUa/" + tn)
return acc["folders"], acc["leaves"], acc["paths"] await maybe_good(top, acc)
return acc["folders"], acc["leaves"], acc["paths"], acc["good"]
try: try:
return asyncio.run(run()) return asyncio.run(run())
except Exception as e: except Exception as e:
return (f"<{type(e).__name__}: {e}>", 0, []) return (f"<{type(e).__name__}: {e}>", 0, [], 0)
def cmd_verify_equipment(args): def cmd_verify_equipment(args):
"""Browse the full UNS tree by friendly Area/Line/Equipment/Signal names and report the leaf """Browse the full UNS tree by friendly Area/Line/Equipment/Signal names and report the leaf
signal count. With --expect N, exit non-zero unless exactly N leaf signals are present (the signal count. With --expect N, exit non-zero unless exactly N leaf signals are present (the
equipment-namespace structure-materialisation check).""" equipment-namespace structure-materialisation check). With --require-good N (>0), also read
each leaf's value and require at least N Good ones (the live-VALUE check for the VirtualTag
overlay) back-compat default 0 = structure-only. --wait polls so it can wait for the deploy
+ change-triggered VirtualTag evaluations to land."""
top_prefix = None if args.all else "nw-area-" top_prefix = None if args.all else "nw-area-"
folders, leaves, paths = browse_tree(args.opcua_endpoint, top_prefix=top_prefix)
scope = "whole address space" if args.all else "company overlay (nw-area-*)" scope = "whole address space" if args.all else "company overlay (nw-area-*)"
print(f"equipment tree : {folders} folder(s), {leaves} leaf signal(s) on {args.opcua_endpoint} [{scope}]") read_values = args.require_good > 0
deadline = time.time() + (args.wait_seconds if args.wait else 0)
while True:
folders, leaves, paths, good = browse_tree(
args.opcua_endpoint, top_prefix=top_prefix, read_values=read_values)
struct_ok = args.expect is None or leaves == args.expect
good_ok = good >= args.require_good
if (struct_ok and good_ok) or time.time() >= deadline:
break
print(f" waiting for deploy/values… ({leaves} leaves"
+ (f", {good} Good" if read_values else "") + ")")
time.sleep(5)
suffix = f", {good} Good value(s)" if read_values else ""
print(f"equipment tree : {folders} folder(s), {leaves} leaf signal(s){suffix} "
f"on {args.opcua_endpoint} [{scope}]")
for p in sorted(paths)[:args.show]: for p in sorted(paths)[:args.show]:
print(f" {p}") print(f" {p}")
if len(paths) > args.show: if len(paths) > args.show:
print(f" … and {len(paths) - args.show} more") print(f" … and {len(paths) - args.show} more")
passed = True
if args.expect is not None: if args.expect is not None:
passed = leaves == args.expect struct_ok = leaves == args.expect
print("VERIFY-EQUIPMENT:", passed = passed and struct_ok
f"PASS ({leaves} == {args.expect})" if passed print(" structure :",
f"PASS ({leaves} == {args.expect})" if struct_ok
else f"FAIL (expected {args.expect}, found {leaves})") else f"FAIL (expected {args.expect}, found {leaves})")
return 0 if passed else 1 if args.require_good > 0:
return 0 good_ok = good >= args.require_good
passed = passed and good_ok
print(" live good :",
f"PASS ({good} >= {args.require_good})" if good_ok
else f"FAIL (expected >= {args.require_good} Good, found {good})")
if args.expect is None and args.require_good == 0:
return 0
print("VERIFY-EQUIPMENT:", "PASS" if passed else "FAIL")
return 0 if passed else 1
# ── arg parsing ───────────────────────────────────────────────────────────── # ── arg parsing ─────────────────────────────────────────────────────────────
@@ -495,11 +575,17 @@ def main(argv):
vp.add_argument("--wait", action="store_true", help="poll until the deploy lands") vp.add_argument("--wait", action="store_true", help="poll until the deploy lands")
vp.add_argument("--wait-seconds", type=int, default=120) vp.add_argument("--wait-seconds", type=int, default=120)
ep = sub.add_parser("verify-equipment", ep = sub.add_parser("verify-equipment",
help="recursively browse the Equipment UNS tree + count leaf signals") help="recursively browse the Equipment UNS tree + count leaf signals "
"(+ optionally assert live Good values)")
ep.add_argument("--expect", type=int, default=None, help="assert exactly N leaf signals") ep.add_argument("--expect", type=int, default=None, help="assert exactly N leaf signals")
ep.add_argument("--require-good", type=int, default=0,
help="read each leaf's value and require >= N Good ones (0 = structure-only, default)")
ep.add_argument("--show", type=int, default=20, help="how many leaf paths to print") ep.add_argument("--show", type=int, default=20, help="how many leaf paths to print")
ep.add_argument("--all", action="store_true", ep.add_argument("--all", action="store_true",
help="count the whole address space (default: only the nw-area-* company overlay)") help="count the whole address space (default: only the nw-area-* company overlay)")
ep.add_argument("--wait", action="store_true",
help="poll until the deploy lands + (with --require-good) values go Good")
ep.add_argument("--wait-seconds", type=int, default=120)
a = p.parse_args(argv) a = p.parse_args(argv)
a.mssql = dict(host=a.sql_host, port=a.sql_port, user=a.sql_user, a.mssql = dict(host=a.sql_host, port=a.sql_port, user=a.sql_user,