Compare commits
34 Commits
eb26bf3248
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
| b80abbb14b | |||
| 6c2d16d4af | |||
| a08ddab9dd | |||
| 744eb090ac | |||
| 94512acf1f | |||
| 2c6c764d3c | |||
| a30f8551e9 | |||
| afd0287f54 | |||
| 1041f87b59 | |||
| 5572edda85 | |||
| aff7264df8 | |||
| 510b0010d6 | |||
| 42ad31aded | |||
| e3c0503a4f | |||
| a0527f9b5a | |||
| 5f7d7e1b58 | |||
| 78418346df | |||
| 4920b89666 | |||
| 989db9317d | |||
| 81bf7322f0 | |||
| 8033a7f12d | |||
| 63cddfb65b | |||
| 965f5006f2 | |||
| 294da8b2db | |||
| bbb7942788 | |||
| d5b134b117 | |||
| eb8b44c29d | |||
| a6fa36043a | |||
| 05a4a547f4 | |||
| 4d57e34ff3 | |||
| b3d8990a0f | |||
| 5655b75fe6 | |||
| dce6f83488 | |||
| fd34e25cb1 |
@@ -6,12 +6,13 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
|
||||
|
||||
`scadaproj` is primarily an umbrella/index workspace that aggregates a family of
|
||||
related SCADA / OT / Wonderware / OPC UA "sister projects" that live as **sibling
|
||||
directories under `~/Desktop/`**. It now also **hosts five pieces of source itself** —
|
||||
directories under `~/Desktop/`**. It now also **hosts six pieces of source itself** —
|
||||
the shared [`ZB.MOM.WW.Auth/`](ZB.MOM.WW.Auth/) library, the shared
|
||||
[`ZB.MOM.WW.Theme/`](ZB.MOM.WW.Theme/) UI kit, the shared
|
||||
[`ZB.MOM.WW.Health/`](ZB.MOM.WW.Health/) health-check library, the shared
|
||||
[`ZB.MOM.WW.Telemetry/`](ZB.MOM.WW.Telemetry/) observability library, and the shared
|
||||
[`ZB.MOM.WW.Configuration/`](ZB.MOM.WW.Configuration/) config-validation library — all the realized output of their
|
||||
[`ZB.MOM.WW.Telemetry/`](ZB.MOM.WW.Telemetry/) observability library, the shared
|
||||
[`ZB.MOM.WW.Configuration/`](ZB.MOM.WW.Configuration/) config-validation library, and the new
|
||||
[`ZB.MOM.WW.GalaxyRepository/`](ZB.MOM.WW.GalaxyRepository/) Galaxy browse library — all the realized output of their
|
||||
respective component normalizations (see [Component normalization](#component-normalization)).
|
||||
The point of this file is to give a high-level scan of each sister project — its purpose,
|
||||
location, stack, and primary commands — so a fresh Claude Code session can orient across
|
||||
@@ -30,9 +31,10 @@ own `CLAUDE.md` for the full picture. See [Refreshing this index](#refreshing-th
|
||||
|
||||
| Project | Location | Stack | Repo | Summary |
|
||||
|---|---|---|---|---|
|
||||
| **OtOpcUa** | `~/Desktop/OtOpcUa` | .NET 10, OPC UA, gRPC | `gitea.dohertylan.com/dohertj2/lmxopcua` | OPC UA server that exposes AVEVA System Platform (Wonderware) Galaxy tags as an OPC UA address space. Galaxy access flows through an in-process `GalaxyDriver` → gRPC → the **mxaccessgw** gateway. |
|
||||
| **OtOpcUa** | `~/Desktop/OtOpcUa` | .NET 10, OPC UA, gRPC | `gitea.dohertylan.com/dohertj2/lmxopcua` | OPC UA server that exposes industrial data sources under a **unified Equipment-based address space** — native-protocol drivers (Modbus, S7, AB CIP/Legacy, TwinCAT, FOCAS, OpcUaClient) **and AVEVA System Platform (Wonderware) Galaxy, now a standard Equipment-kind driver** (the old SystemPlatform mirror / alias-tag model was retired ~2026-06-12). Galaxy access flows through the in-process `GalaxyDriver` → gRPC → the **mxaccessgw** gateway. Surfaces live read + authorized write, native OPC UA Part 9 alarms, and server-side HistoryRead. |
|
||||
| **MxAccessGateway** (`mxaccessgw`) | `~/Desktop/MxAccessGateway` | .NET 10 gateway (x64) + .NET 4.8 worker (**x86**), gRPC | `gitea.dohertylan.com/dohertj2/mxaccessgw` | gRPC gateway giving modern clients full MXAccess parity without loading 32-bit COM. Two-process: gateway (ASP.NET Core gRPC + Blazor dashboard) + per-session x86 worker that owns the MXAccess COM STA. **OtOpcUa depends on this.** |
|
||||
| **ScadaBridge** | `~/Desktop/ScadaBridge` | .NET 10, Akka.NET, Docker | _git_ | Full implementation of the distributed SCADA platform — hub-and-spoke (1 central cluster + N site clusters). Projects prefixed `ZB.MOM.WW.ScadaBridge.*`; solution `ZB.MOM.WW.ScadaBridge.slnx`. Ships `src/`, `tests/`, `docker/` topology, and the design docs that are the spec. |
|
||||
| **HistorianGateway** | `~/Desktop/HistorianGateway` | .NET 10 x64, gRPC, Blazor | `gitea.dohertylan.com/dohertj2/historiangw` | Single-process gRPC sidecar exposing (1) full read/write API to the AVEVA Historian (5 gRPC services; 15 retrieval modes; historical/backfill writes; tag-config lifecycle; SQL live-value path; store-forward + redundancy resilience; all default-disabled) and (2) read-only Galaxy object-hierarchy browse via the shared `ZB.MOM.WW.GalaxyRepository` lib (consumed as a Gitea-feed package). No COM, no x86 worker. Dashboard on `:5220` (HTTP/1.1); gRPC h2c on `:5221`. Vendors `AVEVA.Historian.Client` from `histsdk`. Store-forward uses a crash-safe FasterLog append-only outbox (`Microsoft.FASTER.Core` 2.6.5; `CommitMode` PerEntry/Periodic), not SQLite. 702 tests total — 681 green on macOS; the env-gated live historian + Galaxy integration suite (21 tests) skips without a live server. |
|
||||
|
||||
## Cross-project relationships
|
||||
|
||||
@@ -84,8 +86,10 @@ the gateway uses `MxGateway.*`). The common subject is **AVEVA System Platform (
|
||||
`GalaxyRepositoryClient` for the static hierarchy, and an MXAccess session
|
||||
(`MxCommand`/`MxEvent` protos) for live read/write/subscribe. A `DeployWatcher` polls the
|
||||
gateway's deploy-event signal to rebuild the OPC UA address space on Galaxy redeploy.
|
||||
OtOpcUa's job is purely a **protocol bridge**: it republishes Galaxy as an OPC UA address
|
||||
space for *any* OPC UA client.
|
||||
OtOpcUa's job is a **protocol bridge**: it republishes Galaxy — now bound as a *standard
|
||||
Equipment-kind driver* alongside its native-protocol drivers, not a special SystemPlatform
|
||||
mirror — as an OPC UA address space (live values, Part 9 alarms, HistoryRead) for *any* OPC
|
||||
UA client.
|
||||
- **ScadaBridge → OPC UA** (OPC UA client). ScadaBridge's DCL has an OPC UA adapter that
|
||||
collects data and mirrors native OPC UA Alarms & Conditions. OtOpcUa is exactly such a
|
||||
server, so ScadaBridge can ingest Wonderware data **indirectly via OtOpcUa**.
|
||||
@@ -101,15 +105,21 @@ the gateway uses `MxGateway.*`). The common subject is **AVEVA System Platform (
|
||||
- ScadaBridge has **two paths** to the same Wonderware data: (1) OPC UA → OtOpcUa →
|
||||
gateway, or (2) MxGateway adapter → gateway directly. Path 1 gives standards-based OPC UA
|
||||
decoupling; path 2 gives a more direct/native feed.
|
||||
- **HistorianGateway is a new, independent sidecar** (no runtime coupling to the three above).
|
||||
It reaches the Historian via its vendored gRPC client and the Galaxy Repository SQL DB directly,
|
||||
not through `mxaccessgw`. It consumes the shared `ZB.MOM.WW.GalaxyRepository` lib
|
||||
(cross-repo `ProjectReference`). Any client that needs Historian data or Galaxy browse can
|
||||
target HistorianGateway independently; it is not a dependency of OtOpcUa or ScadaBridge today.
|
||||
- Coupling is loose: each repo references the others only as **sibling context** (the
|
||||
`## Sister Projects` note in ScadaBridge's own `CLAUDE.md` lists `MxAccessGateway` and
|
||||
`OtOpcUa` with their Gitea URLs but states they are *not part of its solution*).
|
||||
- **The break surface is the wire contracts, not code.** Because coupling is by network
|
||||
protocol, the things that break across repo boundaries are: the gateway's `.proto` files
|
||||
(`mxaccess_gateway.proto`, `mxaccess_worker.proto`, `galaxy_repository.proto`), and the
|
||||
OPC UA address-space shape OtOpcUa publishes (browse paths, node IDs, A&C alarm model).
|
||||
Changes to any of these must be coordinated across the affected repos — a green build in
|
||||
one repo does not prove the others still interoperate.
|
||||
(`mxaccess_gateway.proto`, `mxaccess_worker.proto`, `galaxy_repository.proto`), the
|
||||
`historian_gateway.v1` proto (HistorianGateway's own contract), and the OPC UA address-space
|
||||
shape OtOpcUa publishes (browse paths, node IDs, A&C alarm model). Changes to any of these
|
||||
must be coordinated across the affected repos — a green build in one repo does not prove the
|
||||
others still interoperate.
|
||||
|
||||
## Component normalization
|
||||
|
||||
@@ -126,6 +136,7 @@ each project's **code-verified current state**, and the **gaps** between. See
|
||||
| Observability (metrics / traces / logs) | Built (lib `0.1.0`) | Shared `ZB.MOM.WW.Telemetry` lib + `.Serilog` | [`components/observability/`](components/observability/) | [`ZB.MOM.WW.Telemetry/`](ZB.MOM.WW.Telemetry/) |
|
||||
| Config + validation (options / startup validation) | Adopted (lib `0.1.0`; all 3 apps, local) | Shared `ZB.MOM.WW.Configuration` lib | [`components/configuration/`](components/configuration/) | [`ZB.MOM.WW.Configuration/`](ZB.MOM.WW.Configuration/) |
|
||||
| Audit (event model + writer seam) | Adopted (lib `0.1.0`; all 3 apps, merged to **local default** main/master + **pushed to origin** (gitea)) | Shared `ZB.MOM.WW.Audit` lib | [`components/audit/`](components/audit/) | [`ZB.MOM.WW.Audit/`](ZB.MOM.WW.Audit/) |
|
||||
| Galaxy Repository (object-hierarchy SQL browse + gRPC service) | Built (lib `0.1.0`, **published to the Gitea feed**; consumed by HistorianGateway as a feed `PackageReference`) | Shared `ZB.MOM.WW.GalaxyRepository` lib | _(design in histsdk + design doc 2026-06-23)_ | [`ZB.MOM.WW.GalaxyRepository/`](ZB.MOM.WW.GalaxyRepository/) |
|
||||
|
||||
The auth component is fully populated: a normalized [`spec`](components/auth/spec/SPEC.md), a
|
||||
proposed [`shared-contract`](components/auth/shared-contract/ZB.MOM.WW.Auth.md), three
|
||||
@@ -261,6 +272,25 @@ migration, MSSQL-verified). Phase 3 wires `Actor` from the Auth principal at aut
|
||||
Build/test from `ZB.MOM.WW.Audit/`: `dotnet test`. Consumer matrix: all three apps consume the single
|
||||
`ZB.MOM.WW.Audit` package (OtOpcUa, MxAccessGateway, ScadaBridge — DEEP-adopted as the canonical record).
|
||||
|
||||
The Galaxy Repository component normalizes the **Galaxy object-hierarchy SQL browse + reusable gRPC service**
|
||||
that was previously embedded in `mxaccessgw`. Shared = canonical `galaxy_repository.v1` proto (wire-compatible
|
||||
with `mxaccessgw`'s existing contract so OtOpcUa's `GalaxyRepositoryClient` is unaffected), the SQL browse
|
||||
provider (`HierarchySql` / `AttributesSql` validated reverse-engineered queries), in-memory hierarchy cache +
|
||||
snapshot + deploy-poll refresh `BackgroundService`, `GalaxyHierarchyProjector`, and `AddZbGalaxyRepository` /
|
||||
`MapZbGalaxyRepository` DI extension. Left per-consumer = section path, subtree auth filtering, and any
|
||||
app-specific paging defaults.
|
||||
|
||||
The shared library is **built and lives in this repo** at [`ZB.MOM.WW.GalaxyRepository/`](ZB.MOM.WW.GalaxyRepository/)
|
||||
(.NET 10; single package `ZB.MOM.WW.GalaxyRepository`; `dotnet pack` → 1 nupkg @ 0.1.0, **published to
|
||||
the Gitea NuGet feed** `gitea.dohertylan.com/api/packages/dohertj2/nuget`). The design doc is at
|
||||
[`docs/plans/2026-06-23-historian-gateway-design.md`](docs/plans/2026-06-23-historian-gateway-design.md) (§10, component 1).
|
||||
**Consumed by HistorianGateway** as a `PackageReference` from that Gitea feed, pinned at `0.1.0` (originally a
|
||||
cross-repo `ProjectReference` to this scadaproj tree; switched to the feed package 2026-06-24).
|
||||
**mxaccessgw adoption is a tracked follow-on** — once adopted, mxaccessgw's inline Galaxy browse code is replaced
|
||||
by the shared lib (the `galaxy_repository.v1` wire contract is already identical, so OtOpcUa and ScadaBridge
|
||||
clients are unaffected). Build/test from `ZB.MOM.WW.GalaxyRepository/`: `dotnet test`.
|
||||
Consumer matrix: HistorianGateway (initial); mxaccessgw (follow-on adoption).
|
||||
|
||||
## Per-project primary commands
|
||||
|
||||
Run these from inside each project directory (not from `scadaproj`).
|
||||
@@ -282,9 +312,17 @@ dotnet run --project src/MxGateway.Server/MxGateway.Server.csproj
|
||||
dotnet build ZB.MOM.WW.ScadaBridge.slnx
|
||||
bash docker/deploy.sh # rebuild + redeploy the 8-node cluster
|
||||
cd infra && docker compose up -d # local test services (SQL, OPC UA, SMTP, REST, Traefik) — LDAP is NOT here
|
||||
|
||||
# HistorianGateway (~/Desktop/HistorianGateway)
|
||||
dotnet build ZB.MOM.WW.HistorianGateway.slnx
|
||||
dotnet test ZB.MOM.WW.HistorianGateway.slnx # unit + golden; live integration tests skip without env vars
|
||||
dotnet run --project src/ZB.MOM.WW.HistorianGateway.Server/ZB.MOM.WW.HistorianGateway.Server.csproj
|
||||
# dashboard on :5220, gRPC h2c on :5221
|
||||
# Live integration (need HISTORIAN_GRPC_HOST + HISTORIAN_GRPC_WRITE_SANDBOX_TAG + GALAXY_SQL_CONNSTR set)
|
||||
dotnet test ZB.MOM.WW.HistorianGateway.slnx --filter "Category=LiveIntegration"
|
||||
```
|
||||
|
||||
> **Shared GLAuth (all three apps):** LDAP auth for every local dev/test stack is provided by a
|
||||
> **Shared GLAuth (all three apps + HistorianGateway):** LDAP auth for every local dev/test stack is provided by a
|
||||
> single `zb-shared-glauth` container on the Linux fixture host **`10.100.0.35:3893`**
|
||||
> (`baseDN dc=zb,dc=local`, Transport=None). Source of truth and deploy runbook:
|
||||
> [`scadaproj/infra/glauth/`](infra/glauth/) (`config.toml` + `docker-compose.yml` + `README.md`).
|
||||
|
||||
@@ -0,0 +1,11 @@
|
||||
<Project>
|
||||
<PropertyGroup>
|
||||
<TargetFramework>net10.0</TargetFramework>
|
||||
<Nullable>enable</Nullable>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<LangVersion>latest</LangVersion>
|
||||
<Version>0.1.0</Version>
|
||||
<ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
|
||||
<GenerateDocumentationFile>true</GenerateDocumentationFile>
|
||||
</PropertyGroup>
|
||||
</Project>
|
||||
@@ -0,0 +1,24 @@
|
||||
<Project>
|
||||
|
||||
<PropertyGroup>
|
||||
<ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<!-- Library -->
|
||||
<PackageVersion Include="Microsoft.Data.SqlClient" Version="6.0.2" />
|
||||
<PackageVersion Include="Grpc.AspNetCore" Version="2.76.0" />
|
||||
<!-- Google.Protobuf and Grpc.Tools must be >= the minimums Grpc.AspNetCore 2.76.0 requires -->
|
||||
<PackageVersion Include="Google.Protobuf" Version="3.31.1" />
|
||||
<PackageVersion Include="Microsoft.Extensions.Hosting.Abstractions" Version="10.0.0" />
|
||||
<PackageVersion Include="Microsoft.Extensions.Options.ConfigurationExtensions" Version="10.0.0" />
|
||||
<PackageVersion Include="Grpc.Tools" Version="2.76.0" />
|
||||
|
||||
<!-- Test -->
|
||||
<PackageVersion Include="xunit" Version="2.9.3" />
|
||||
<PackageVersion Include="xunit.runner.visualstudio" Version="3.1.4" />
|
||||
<PackageVersion Include="Microsoft.NET.Test.Sdk" Version="17.14.1" />
|
||||
<PackageVersion Include="coverlet.collector" Version="6.0.4" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
@@ -0,0 +1,8 @@
|
||||
<Solution>
|
||||
<Folder Name="/src/">
|
||||
<Project Path="src/ZB.MOM.WW.GalaxyRepository/ZB.MOM.WW.GalaxyRepository.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/tests/">
|
||||
<Project Path="tests/ZB.MOM.WW.GalaxyRepository.Tests/ZB.MOM.WW.GalaxyRepository.Tests.csproj" />
|
||||
</Folder>
|
||||
</Solution>
|
||||
+71
@@ -0,0 +1,71 @@
|
||||
using Microsoft.AspNetCore.Builder;
|
||||
using Microsoft.AspNetCore.Routing;
|
||||
using Microsoft.Extensions.Configuration;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Microsoft.Extensions.Options;
|
||||
using ZB.MOM.WW.GalaxyRepository.Grpc;
|
||||
|
||||
namespace ZB.MOM.WW.GalaxyRepository.DependencyInjection;
|
||||
|
||||
/// <summary>
|
||||
/// Dependency-injection and endpoint-routing extensions that register the reusable
|
||||
/// Galaxy Repository services and map the canonical gRPC service. A consuming gateway
|
||||
/// calls <see cref="AddZbGalaxyRepository"/> during service registration and
|
||||
/// <see cref="MapZbGalaxyRepository"/> while building its endpoint pipeline.
|
||||
/// </summary>
|
||||
public static class GalaxyRepositoryServiceCollectionExtensions
|
||||
{
|
||||
/// <summary>
|
||||
/// Registers the Galaxy Repository SQL provider, shared hierarchy cache, deploy
|
||||
/// notifier, on-disk snapshot store, and the background refresh service, binding
|
||||
/// <see cref="GalaxyRepositoryOptions"/> from the supplied configuration section.
|
||||
/// </summary>
|
||||
/// <param name="services">The service collection to add registrations to.</param>
|
||||
/// <param name="configuration">The application configuration root.</param>
|
||||
/// <param name="sectionPath">
|
||||
/// The configuration section path to bind <see cref="GalaxyRepositoryOptions"/> from
|
||||
/// (for example <c>MxGateway:Galaxy</c> or <c>HistorianGateway:Galaxy</c>).
|
||||
/// </param>
|
||||
/// <returns>The service collection for chaining.</returns>
|
||||
public static IServiceCollection AddZbGalaxyRepository(
|
||||
this IServiceCollection services,
|
||||
IConfiguration configuration,
|
||||
string sectionPath)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(services);
|
||||
ArgumentNullException.ThrowIfNull(configuration);
|
||||
ArgumentException.ThrowIfNullOrWhiteSpace(sectionPath);
|
||||
|
||||
// Bind only — this shared lib ships no validator, so a .ValidateOnStart() here
|
||||
// would be a silent no-op. The consuming application owns option validation
|
||||
// (e.g. the sidecar's ConfigPreflight / validated-options layer).
|
||||
services
|
||||
.AddOptions<GalaxyRepositoryOptions>()
|
||||
.Bind(configuration.GetSection(sectionPath));
|
||||
|
||||
services.AddSingleton(sp =>
|
||||
new GalaxyRepository(sp.GetRequiredService<IOptions<GalaxyRepositoryOptions>>().Value));
|
||||
services.AddSingleton<IGalaxyRepository>(sp => sp.GetRequiredService<GalaxyRepository>());
|
||||
|
||||
services.AddSingleton<IGalaxyDeployNotifier, GalaxyDeployNotifier>();
|
||||
services.AddSingleton<IGalaxyHierarchySnapshotStore, GalaxyHierarchySnapshotStore>();
|
||||
services.AddSingleton<IGalaxyHierarchyCache, GalaxyHierarchyCache>();
|
||||
services.AddHostedService<GalaxyHierarchyRefreshService>();
|
||||
|
||||
return services;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Maps the canonical <see cref="GalaxyRepositoryGrpcService"/> onto the consuming
|
||||
/// application's endpoint pipeline. Call after <see cref="AddZbGalaxyRepository"/> and
|
||||
/// after gRPC has been added to the application's services.
|
||||
/// </summary>
|
||||
/// <param name="endpoints">The endpoint route builder to map the gRPC service onto.</param>
|
||||
/// <returns>The endpoint route builder for chaining.</returns>
|
||||
public static IEndpointRouteBuilder MapZbGalaxyRepository(this IEndpointRouteBuilder endpoints)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(endpoints);
|
||||
endpoints.MapGrpcService<GalaxyRepositoryGrpcService>();
|
||||
return endpoints;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
namespace ZB.MOM.WW.GalaxyRepository;
|
||||
|
||||
/// <summary>One row from <see cref="GalaxyRepository.GetAttributesAsync"/>.</summary>
|
||||
public sealed class GalaxyAttributeRow
|
||||
{
|
||||
/// <summary>Gets the Galaxy object identifier.</summary>
|
||||
public int GobjectId { get; init; }
|
||||
|
||||
/// <summary>Gets the tag name.</summary>
|
||||
public string TagName { get; init; } = string.Empty;
|
||||
|
||||
/// <summary>Gets the attribute name.</summary>
|
||||
public string AttributeName { get; init; } = string.Empty;
|
||||
|
||||
/// <summary>Gets the full tag reference.</summary>
|
||||
public string FullTagReference { get; init; } = string.Empty;
|
||||
|
||||
/// <summary>Gets the MXAccess data type code.</summary>
|
||||
public int MxDataType { get; init; }
|
||||
|
||||
/// <summary>Gets the data type name.</summary>
|
||||
public string? DataTypeName { get; init; }
|
||||
|
||||
/// <summary>Gets a value indicating whether this is an array.</summary>
|
||||
public bool IsArray { get; init; }
|
||||
|
||||
/// <summary>Gets the array dimension, if applicable.</summary>
|
||||
public int? ArrayDimension { get; init; }
|
||||
|
||||
/// <summary>Gets the MXAccess attribute category code.</summary>
|
||||
public int MxAttributeCategory { get; init; }
|
||||
|
||||
/// <summary>Gets the security classification code.</summary>
|
||||
public int SecurityClassification { get; init; }
|
||||
|
||||
/// <summary>Gets a value indicating whether this is historized.</summary>
|
||||
public bool IsHistorized { get; init; }
|
||||
|
||||
/// <summary>Gets a value indicating whether this is an alarm.</summary>
|
||||
public bool IsAlarm { get; init; }
|
||||
}
|
||||
+19
@@ -0,0 +1,19 @@
|
||||
using ZB.MOM.WW.GalaxyRepository.Grpc;
|
||||
|
||||
namespace ZB.MOM.WW.GalaxyRepository;
|
||||
|
||||
/// <summary>
|
||||
/// Result of one <see cref="GalaxyBrowseProjector.ProjectChildren"/> call. Holds a
|
||||
/// materialized page of direct children for the requested parent, along with a
|
||||
/// parallel-indexed <see cref="ChildHasChildren"/> hint and the total post-filter
|
||||
/// sibling count for paging.
|
||||
/// </summary>
|
||||
/// <param name="Children">The page of direct children, sorted areas-first then by display name.</param>
|
||||
/// <param name="ChildHasChildren">Parallel array indicating whether each child has at least one matching descendant under the same filter set.</param>
|
||||
/// <param name="TotalChildCount">Total matching direct children of the parent (post-filter).</param>
|
||||
/// <param name="FilterSignature">Stable signature of the filter and parent selector, used to bind page tokens.</param>
|
||||
public sealed record GalaxyBrowseChildrenResult(
|
||||
IReadOnlyList<GalaxyObject> Children,
|
||||
IReadOnlyList<bool> ChildHasChildren,
|
||||
int TotalChildCount,
|
||||
string FilterSignature);
|
||||
@@ -0,0 +1,281 @@
|
||||
using System.Collections.Concurrent;
|
||||
using System.Runtime.CompilerServices;
|
||||
using System.Security.Cryptography;
|
||||
using System.Text;
|
||||
using Grpc.Core;
|
||||
using ZB.MOM.WW.GalaxyRepository.Grpc;
|
||||
|
||||
namespace ZB.MOM.WW.GalaxyRepository;
|
||||
|
||||
/// <summary>
|
||||
/// Projects one level of children of a parent object out of an immutable
|
||||
/// <see cref="GalaxyHierarchyCacheEntry"/>. Pure and side-effect free. Memoizes the
|
||||
/// filtered child list per cache-entry instance so repeated paging is an O(pageSize)
|
||||
/// slice rather than an O(siblings) filter scan per page. The memo is keyed on the
|
||||
/// immutable cache entry, so when the cache publishes a new entry the stale memo
|
||||
/// becomes unreachable and is reclaimed with it.
|
||||
/// </summary>
|
||||
public static class GalaxyBrowseProjector
|
||||
{
|
||||
private static readonly ConditionalWeakTable<
|
||||
GalaxyHierarchyCacheEntry,
|
||||
ConcurrentDictionary<string, FilteredChildren>> FilteredChildrenCache = new();
|
||||
|
||||
/// <summary>Projects one page of direct children of the resolved parent.</summary>
|
||||
/// <param name="entry">The Galaxy hierarchy cache entry to query.</param>
|
||||
/// <param name="request">The browse-children request.</param>
|
||||
/// <param name="browseSubtreeGlobs">Optional API-key browse-subtree constraints.</param>
|
||||
/// <param name="offset">Zero-based offset into the filtered child list.</param>
|
||||
/// <param name="pageSize">Maximum number of children to return.</param>
|
||||
public static GalaxyBrowseChildrenResult ProjectChildren(
|
||||
GalaxyHierarchyCacheEntry entry,
|
||||
BrowseChildrenRequest request,
|
||||
IReadOnlyList<string>? browseSubtreeGlobs,
|
||||
int offset,
|
||||
int pageSize)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(entry);
|
||||
ArgumentNullException.ThrowIfNull(request);
|
||||
if (offset < 0)
|
||||
{
|
||||
throw new ArgumentOutOfRangeException(nameof(offset), offset, "Offset must be greater than or equal to zero.");
|
||||
}
|
||||
if (pageSize <= 0)
|
||||
{
|
||||
throw new ArgumentOutOfRangeException(nameof(pageSize), pageSize, "Page size must be greater than zero.");
|
||||
}
|
||||
|
||||
int parentId = ResolveParentId(entry, request);
|
||||
string filterSignature = ComputeFilterSignature(request, browseSubtreeGlobs, parentId);
|
||||
FilteredChildren filtered = GetFilteredChildren(entry, request, browseSubtreeGlobs, parentId, filterSignature);
|
||||
|
||||
bool includeAttributes = IncludeAttributes(request);
|
||||
int end = (int)Math.Min((long)offset + pageSize, filtered.Children.Count);
|
||||
List<GalaxyObject> page = new(Math.Max(0, end - offset));
|
||||
List<bool> hasChildren = new(Math.Max(0, end - offset));
|
||||
for (int index = offset; index < end; index++)
|
||||
{
|
||||
page.Add(CloneObject(filtered.Children[index].Object, includeAttributes));
|
||||
hasChildren.Add(filtered.HasMatchingDescendant[index]);
|
||||
}
|
||||
|
||||
return new GalaxyBrowseChildrenResult(page, hasChildren, filtered.Children.Count, filterSignature);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Resolves the request's parent oneof to a gobject id, throwing
|
||||
/// <see cref="RpcException"/> with <see cref="StatusCode.NotFound"/> when the
|
||||
/// parent does not exist. Public so the gRPC handler can compute the same
|
||||
/// parent id (needed for the page-token signature) without reimplementing the
|
||||
/// resolution rules.
|
||||
/// </summary>
|
||||
/// <param name="entry">The Galaxy hierarchy cache entry to query.</param>
|
||||
/// <param name="request">The browse-children request.</param>
|
||||
public static int ResolveParentId(GalaxyHierarchyCacheEntry entry, BrowseChildrenRequest request)
|
||||
{
|
||||
switch (request.ParentCase)
|
||||
{
|
||||
case BrowseChildrenRequest.ParentOneofCase.None:
|
||||
return 0;
|
||||
case BrowseChildrenRequest.ParentOneofCase.ParentGobjectId:
|
||||
if (request.ParentGobjectId == 0)
|
||||
{
|
||||
return 0;
|
||||
}
|
||||
if (!entry.Index.ObjectViewsById.ContainsKey(request.ParentGobjectId))
|
||||
{
|
||||
throw new RpcException(new Status(StatusCode.NotFound, "BrowseChildren parent was not found."));
|
||||
}
|
||||
return request.ParentGobjectId;
|
||||
case BrowseChildrenRequest.ParentOneofCase.ParentTagName:
|
||||
{
|
||||
if (!entry.Index.ObjectViewsByTagName.TryGetValue(request.ParentTagName, out GalaxyObjectView? match))
|
||||
{
|
||||
throw new RpcException(new Status(StatusCode.NotFound, "BrowseChildren parent was not found."));
|
||||
}
|
||||
return match.Object.GobjectId;
|
||||
}
|
||||
case BrowseChildrenRequest.ParentOneofCase.ParentContainedPath:
|
||||
{
|
||||
if (!entry.Index.ObjectViewsByContainedPath.TryGetValue(request.ParentContainedPath, out GalaxyObjectView? match))
|
||||
{
|
||||
throw new RpcException(new Status(StatusCode.NotFound, "BrowseChildren parent was not found."));
|
||||
}
|
||||
return match.Object.GobjectId;
|
||||
}
|
||||
default:
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
||||
private static FilteredChildren GetFilteredChildren(
|
||||
GalaxyHierarchyCacheEntry entry,
|
||||
BrowseChildrenRequest request,
|
||||
IReadOnlyList<string>? browseSubtreeGlobs,
|
||||
int parentId,
|
||||
string filterSignature)
|
||||
{
|
||||
ConcurrentDictionary<string, FilteredChildren> memo =
|
||||
FilteredChildrenCache.GetValue(entry, static _ => new ConcurrentDictionary<string, FilteredChildren>(StringComparer.Ordinal));
|
||||
|
||||
return memo.GetOrAdd(
|
||||
filterSignature,
|
||||
static (_, state) =>
|
||||
{
|
||||
IReadOnlyDictionary<int, IReadOnlyList<GalaxyObjectView>> map = state.Entry.Index.ChildrenByParent;
|
||||
IReadOnlyList<GalaxyObjectView> directChildren = map.TryGetValue(state.ParentId, out IReadOnlyList<GalaxyObjectView>? list)
|
||||
? list
|
||||
: Array.Empty<GalaxyObjectView>();
|
||||
|
||||
List<GalaxyObjectView> matched = [];
|
||||
List<bool> hasMatching = [];
|
||||
foreach (GalaxyObjectView view in directChildren)
|
||||
{
|
||||
if (!MatchesBrowseSubtrees(view, state.BrowseSubtreeGlobs))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
if (!MatchesFilters(view.Object, state.Request))
|
||||
{
|
||||
// Even if the direct child itself fails the filter, a matching
|
||||
// descendant should still surface its ancestor — but only when
|
||||
// there is one. Mirror the dashboard browse-tree semantics: if a
|
||||
// descendant matches, include the parent with has-children true.
|
||||
if (HasMatchingDescendant(view, state.Entry.Index, state.Request, state.BrowseSubtreeGlobs))
|
||||
{
|
||||
matched.Add(view);
|
||||
hasMatching.Add(true);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
matched.Add(view);
|
||||
hasMatching.Add(HasMatchingDescendant(view, state.Entry.Index, state.Request, state.BrowseSubtreeGlobs));
|
||||
}
|
||||
|
||||
return new FilteredChildren(matched, hasMatching);
|
||||
},
|
||||
(Entry: entry, ParentId: parentId, Request: request, BrowseSubtreeGlobs: browseSubtreeGlobs));
|
||||
}
|
||||
|
||||
private static bool HasMatchingDescendant(
|
||||
GalaxyObjectView parent,
|
||||
GalaxyHierarchyIndex index,
|
||||
BrowseChildrenRequest request,
|
||||
IReadOnlyList<string>? browseSubtreeGlobs)
|
||||
{
|
||||
if (!index.ChildrenByParent.TryGetValue(parent.Object.GobjectId, out IReadOnlyList<GalaxyObjectView>? children))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
// Defend against pathological cycles in Galaxy data (e.g. a corrupt A→B→A chain).
|
||||
// BuildContainedPath uses the same visited-id pattern; mirror it so this walk
|
||||
// terminates even when ChildrenByParent forms a cycle.
|
||||
HashSet<int> visited = new() { parent.Object.GobjectId };
|
||||
Stack<GalaxyObjectView> stack = new();
|
||||
foreach (GalaxyObjectView child in children)
|
||||
{
|
||||
if (visited.Add(child.Object.GobjectId))
|
||||
{
|
||||
stack.Push(child);
|
||||
}
|
||||
}
|
||||
while (stack.Count > 0)
|
||||
{
|
||||
GalaxyObjectView candidate = stack.Pop();
|
||||
if (MatchesBrowseSubtrees(candidate, browseSubtreeGlobs)
|
||||
&& MatchesFilters(candidate.Object, request))
|
||||
{
|
||||
return true;
|
||||
}
|
||||
if (index.ChildrenByParent.TryGetValue(candidate.Object.GobjectId, out IReadOnlyList<GalaxyObjectView>? grandchildren))
|
||||
{
|
||||
foreach (GalaxyObjectView grandchild in grandchildren)
|
||||
{
|
||||
if (visited.Add(grandchild.Object.GobjectId))
|
||||
{
|
||||
stack.Push(grandchild);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
private static bool MatchesBrowseSubtrees(GalaxyObjectView view, IReadOnlyList<string>? browseSubtreeGlobs)
|
||||
{
|
||||
return browseSubtreeGlobs is null
|
||||
|| browseSubtreeGlobs.Count == 0
|
||||
|| browseSubtreeGlobs.Any(glob => GalaxyGlobMatcher.IsMatch(view.ContainedPath, glob));
|
||||
}
|
||||
|
||||
private static bool MatchesFilters(GalaxyObject obj, BrowseChildrenRequest request)
|
||||
{
|
||||
if (request.CategoryIds.Count > 0 && !request.CategoryIds.Contains(obj.CategoryId))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
foreach (string templateFilter in request.TemplateChainContains)
|
||||
{
|
||||
if (!obj.TemplateChain.Any(template => template.Contains(templateFilter, StringComparison.OrdinalIgnoreCase)))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
}
|
||||
if (!string.IsNullOrWhiteSpace(request.TagNameGlob)
|
||||
&& !GalaxyGlobMatcher.IsMatch(obj.TagName, request.TagNameGlob))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
if (request.AlarmBearingOnly && !obj.Attributes.Any(attribute => attribute.IsAlarm))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
if (request.HistorizedOnly && !obj.Attributes.Any(attribute => attribute.IsHistorized))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
private static bool IncludeAttributes(BrowseChildrenRequest request)
|
||||
{
|
||||
return !request.HasIncludeAttributes || request.IncludeAttributes;
|
||||
}
|
||||
|
||||
private static GalaxyObject CloneObject(GalaxyObject source, bool includeAttributes)
|
||||
{
|
||||
GalaxyObject clone = source.Clone();
|
||||
if (!includeAttributes)
|
||||
{
|
||||
clone.Attributes.Clear();
|
||||
}
|
||||
return clone;
|
||||
}
|
||||
|
||||
/// <summary>Computes a stable filter signature for memoization purposes.</summary>
|
||||
/// <param name="request">The browse-children request.</param>
|
||||
/// <param name="browseSubtreeGlobs">Optional API-key browse-subtree constraints.</param>
|
||||
/// <param name="parentId">Resolved parent gobject id (0 for roots).</param>
|
||||
public static string ComputeFilterSignature(
|
||||
BrowseChildrenRequest request,
|
||||
IReadOnlyList<string>? browseSubtreeGlobs,
|
||||
int parentId)
|
||||
{
|
||||
StringBuilder builder = new();
|
||||
builder.Append("parent=").Append(parentId.ToString(System.Globalization.CultureInfo.InvariantCulture));
|
||||
builder.Append("|cat=").AppendJoin(',', request.CategoryIds.Order());
|
||||
builder.Append("|tpl=").AppendJoin(',', request.TemplateChainContains.Order(StringComparer.OrdinalIgnoreCase));
|
||||
builder.Append("|glob=").Append(request.TagNameGlob);
|
||||
builder.Append("|attrs=").Append(request.HasIncludeAttributes ? request.IncludeAttributes.ToString() : "unset");
|
||||
builder.Append("|alarm=").Append(request.AlarmBearingOnly);
|
||||
builder.Append("|hist=").Append(request.HistorizedOnly);
|
||||
builder.Append("|browse=").AppendJoin(',', (browseSubtreeGlobs ?? Array.Empty<string>()).Order(StringComparer.OrdinalIgnoreCase));
|
||||
byte[] hash = SHA256.HashData(Encoding.UTF8.GetBytes(builder.ToString()));
|
||||
return Convert.ToHexString(hash, 0, 12);
|
||||
}
|
||||
|
||||
private sealed record FilteredChildren(
|
||||
IReadOnlyList<GalaxyObjectView> Children,
|
||||
IReadOnlyList<bool> HasMatchingDescendant);
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
namespace ZB.MOM.WW.GalaxyRepository;
|
||||
|
||||
/// <summary>Freshness state of the shared Galaxy hierarchy cache entry.</summary>
|
||||
public enum GalaxyCacheStatus
|
||||
{
|
||||
/// <summary>Cache has never completed a refresh.</summary>
|
||||
Unknown = 0,
|
||||
|
||||
/// <summary>Cache holds data from a recent successful refresh.</summary>
|
||||
Healthy = 1,
|
||||
|
||||
/// <summary>Cache holds data, but the most recent refresh attempt failed
|
||||
/// or no successful refresh has happened within the staleness threshold.</summary>
|
||||
Stale = 2,
|
||||
|
||||
/// <summary>Latest refresh failed and no prior data is available.</summary>
|
||||
Unavailable = 3,
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
namespace ZB.MOM.WW.GalaxyRepository;
|
||||
|
||||
/// <summary>
|
||||
/// A single Galaxy deploy notification. Published by <see cref="GalaxyHierarchyCache"/>
|
||||
/// whenever a refresh detects that <c>galaxy.time_of_last_deploy</c> has changed (or on
|
||||
/// the first successful refresh). Consumed by <see cref="IGalaxyDeployNotifier"/>
|
||||
/// subscribers (the streaming gRPC RPC).
|
||||
/// </summary>
|
||||
/// <param name="Sequence">Monotonically increasing per process start; gaps indicate dropped events.</param>
|
||||
/// <param name="ObservedAt">Server wall-clock when the cache observed the deploy.</param>
|
||||
/// <param name="TimeOfLastDeploy">The <c>galaxy.time_of_last_deploy</c> value, or <see langword="null"/> when the Galaxy table reports none.</param>
|
||||
/// <param name="ObjectCount">Number of objects in the hierarchy at the time of the event.</param>
|
||||
/// <param name="AttributeCount">Number of attributes in the hierarchy at the time of the event.</param>
|
||||
public sealed record GalaxyDeployEventInfo(
|
||||
long Sequence,
|
||||
DateTimeOffset ObservedAt,
|
||||
DateTimeOffset? TimeOfLastDeploy,
|
||||
int ObjectCount,
|
||||
int AttributeCount);
|
||||
@@ -0,0 +1,79 @@
|
||||
using System.Collections.Concurrent;
|
||||
using System.Runtime.CompilerServices;
|
||||
using System.Threading.Channels;
|
||||
|
||||
namespace ZB.MOM.WW.GalaxyRepository;
|
||||
|
||||
/// <summary>
|
||||
/// Channel-based fan-out of Galaxy deploy events to streaming gRPC subscribers. Each
|
||||
/// subscriber gets a private bounded channel so a slow client cannot back-pressure
|
||||
/// other subscribers or the publisher. When a subscriber's channel is full the oldest
|
||||
/// event is dropped — clients use the sequence field to detect gaps.
|
||||
/// </summary>
|
||||
public sealed class GalaxyDeployNotifier : IGalaxyDeployNotifier
|
||||
{
|
||||
private const int SubscriberQueueCapacity = 16;
|
||||
|
||||
private readonly ConcurrentDictionary<Guid, Channel<GalaxyDeployEventInfo>> _subscribers = new();
|
||||
private GalaxyDeployEventInfo? _latest;
|
||||
|
||||
/// <summary>
|
||||
/// The most recent deploy event, or null if none has been published.
|
||||
/// </summary>
|
||||
public GalaxyDeployEventInfo? Latest => Volatile.Read(ref _latest);
|
||||
|
||||
/// <inheritdoc />
|
||||
public void Publish(GalaxyDeployEventInfo info)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(info);
|
||||
|
||||
Volatile.Write(ref _latest, info);
|
||||
|
||||
foreach (Channel<GalaxyDeployEventInfo> channel in _subscribers.Values)
|
||||
{
|
||||
// BoundedChannelFullMode.DropOldest -> writes never wait; we only fail if the
|
||||
// channel was completed by the subscriber side, which we ignore.
|
||||
channel.Writer.TryWrite(info);
|
||||
}
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
public async IAsyncEnumerable<GalaxyDeployEventInfo> SubscribeAsync(
|
||||
[EnumeratorCancellation] CancellationToken cancellationToken)
|
||||
{
|
||||
Guid subscriberId = Guid.NewGuid();
|
||||
Channel<GalaxyDeployEventInfo> channel = Channel.CreateBounded<GalaxyDeployEventInfo>(
|
||||
new BoundedChannelOptions(SubscriberQueueCapacity)
|
||||
{
|
||||
FullMode = BoundedChannelFullMode.DropOldest,
|
||||
SingleReader = true,
|
||||
SingleWriter = false,
|
||||
});
|
||||
|
||||
_subscribers[subscriberId] = channel;
|
||||
|
||||
// Bootstrap: emit the latest known event so subscribers don't need to wait for
|
||||
// the next deploy to know current state.
|
||||
GalaxyDeployEventInfo? bootstrap = Volatile.Read(ref _latest);
|
||||
if (bootstrap is not null)
|
||||
{
|
||||
channel.Writer.TryWrite(bootstrap);
|
||||
}
|
||||
|
||||
try
|
||||
{
|
||||
while (await channel.Reader.WaitToReadAsync(cancellationToken).ConfigureAwait(false))
|
||||
{
|
||||
while (channel.Reader.TryRead(out GalaxyDeployEventInfo? next))
|
||||
{
|
||||
yield return next;
|
||||
}
|
||||
}
|
||||
}
|
||||
finally
|
||||
{
|
||||
_subscribers.TryRemove(subscriberId, out _);
|
||||
channel.Writer.TryComplete();
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,131 @@
|
||||
using System.Collections.Concurrent;
|
||||
using System.Text;
|
||||
using System.Text.RegularExpressions;
|
||||
|
||||
namespace ZB.MOM.WW.GalaxyRepository;
|
||||
|
||||
/// <summary>
|
||||
/// Anchored, case-insensitive glob matcher (<c>*</c> and <c>?</c> wildcards) used by the
|
||||
/// hierarchy and browse projectors to filter object tag names and browse subtrees.
|
||||
/// Compiled regexes are cached and the cache is bounded so an unbounded stream of distinct
|
||||
/// client-supplied globs cannot grow memory without limit.
|
||||
/// </summary>
|
||||
public static class GalaxyGlobMatcher
|
||||
{
|
||||
/// <summary>
|
||||
/// Maximum number of compiled-regex entries retained in <see cref="RegexCache"/>.
|
||||
/// The cache is keyed by glob pattern and patterns flow in from two sources:
|
||||
/// admin-controlled API-key constraints (naturally bounded) and the
|
||||
/// client-supplied <c>DiscoverHierarchyRequest.TagNameGlob</c> (unbounded — a
|
||||
/// client can iterate through generated names and create millions of distinct
|
||||
/// globs over the process lifetime). Capping the cache bounds memory while
|
||||
/// keeping the hot working set hit-cached.
|
||||
/// </summary>
|
||||
internal const int RegexCacheCapacity = 256;
|
||||
|
||||
/// <summary>
|
||||
/// Bounded compiled-regex cache keyed by glob pattern. <c>IsMatch</c> is called
|
||||
/// once per object per <c>DiscoverHierarchy</c>/<c>WatchDeployEvents</c>
|
||||
/// evaluation, so the same handful of glob patterns are translated
|
||||
/// repeatedly; caching avoids rebuilding and recompiling the regex on every
|
||||
/// call. Beyond <see cref="RegexCacheCapacity"/> entries the oldest insertion
|
||||
/// is evicted so a client cannot grow the cache without bound by submitting
|
||||
/// unique patterns. Eviction is approximate (FIFO over insertion order, not
|
||||
/// true LRU) because we only need the bound, not exact recency tracking.
|
||||
/// </summary>
|
||||
private static readonly ConcurrentDictionary<string, Regex> RegexCache = new(StringComparer.Ordinal);
|
||||
|
||||
/// <summary>
|
||||
/// Insertion-order queue used to evict the oldest cache entry when the cache
|
||||
/// exceeds <see cref="RegexCacheCapacity"/>. A separate queue keeps the
|
||||
/// <see cref="RegexCache"/> reads lock-free; the lock below only guards the
|
||||
/// eviction path.
|
||||
/// </summary>
|
||||
private static readonly ConcurrentQueue<string> InsertionOrder = new();
|
||||
private static readonly object EvictionLock = new();
|
||||
|
||||
/// <summary>
|
||||
/// Current cache size, exposed for tests asserting the cap is honoured.
|
||||
/// </summary>
|
||||
internal static int CurrentCacheSize => RegexCache.Count;
|
||||
|
||||
/// <summary>Determines whether a value matches a glob pattern (with * and ? wildcards).</summary>
|
||||
/// <param name="value">The value to test against the glob pattern.</param>
|
||||
/// <param name="glob">The glob pattern with * and ? wildcards.</param>
|
||||
public static bool IsMatch(string value, string glob)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(glob))
|
||||
{
|
||||
return true;
|
||||
}
|
||||
|
||||
return GetOrCreateRegex(glob).IsMatch(value ?? string.Empty);
|
||||
}
|
||||
|
||||
private static Regex GetOrCreateRegex(string glob)
|
||||
{
|
||||
if (RegexCache.TryGetValue(glob, out Regex? existing))
|
||||
{
|
||||
return existing;
|
||||
}
|
||||
|
||||
Regex compiled = new(
|
||||
BuildRegex(glob),
|
||||
RegexOptions.CultureInvariant | RegexOptions.IgnoreCase | RegexOptions.Compiled,
|
||||
TimeSpan.FromMilliseconds(100));
|
||||
|
||||
// GetOrAdd atomically returns whichever instance is in the cache after the
|
||||
// call — either the locally-compiled regex (we won the race) or the regex
|
||||
// another thread inserted (we lost). It also avoids the TryAdd-then-indexer
|
||||
// pattern where the key could be evicted between the failed TryAdd and the
|
||||
// indexer read, producing a KeyNotFoundException under contention near the cap.
|
||||
Regex result = RegexCache.GetOrAdd(glob, compiled);
|
||||
if (ReferenceEquals(result, compiled))
|
||||
{
|
||||
// We were the inserter — track for FIFO eviction and bound the cache.
|
||||
InsertionOrder.Enqueue(glob);
|
||||
EvictIfOverCapacity();
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private static void EvictIfOverCapacity()
|
||||
{
|
||||
if (RegexCache.Count <= RegexCacheCapacity)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
// Serialize eviction so two threads do not race past the cap together.
|
||||
lock (EvictionLock)
|
||||
{
|
||||
while (RegexCache.Count > RegexCacheCapacity && InsertionOrder.TryDequeue(out string? oldest))
|
||||
{
|
||||
RegexCache.TryRemove(oldest, out _);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private static string BuildRegex(string glob)
|
||||
{
|
||||
StringBuilder builder = new("^", glob.Length + 2);
|
||||
foreach (char character in glob)
|
||||
{
|
||||
switch (character)
|
||||
{
|
||||
case '*':
|
||||
builder.Append(".*");
|
||||
break;
|
||||
case '?':
|
||||
builder.Append('.');
|
||||
break;
|
||||
default:
|
||||
builder.Append(Regex.Escape(character.ToString()));
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
builder.Append('$');
|
||||
return builder.ToString();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,365 @@
|
||||
using Microsoft.Extensions.Logging;
|
||||
using ZB.MOM.WW.GalaxyRepository.Grpc;
|
||||
|
||||
namespace ZB.MOM.WW.GalaxyRepository;
|
||||
|
||||
/// <summary>
|
||||
/// Server-side cache of Galaxy Repository browse data. All gRPC clients share the same
|
||||
/// entry — the materialized object list is produced once per refresh and reused across
|
||||
/// requests. Refreshes are deploy-time gated: every tick queries
|
||||
/// <c>galaxy.time_of_last_deploy</c> (cheap), and the heavy hierarchy + attributes rowsets
|
||||
/// are pulled only when that timestamp has advanced.
|
||||
/// Each successful heavy refresh is persisted to disk through
|
||||
/// <see cref="IGalaxyHierarchySnapshotStore"/>; the first refresh restores that
|
||||
/// snapshot (as <see cref="GalaxyCacheStatus.Stale"/>) so clients can browse
|
||||
/// last-known data when the Galaxy database is unreachable on a cold start.
|
||||
/// </summary>
|
||||
public sealed class GalaxyHierarchyCache : IGalaxyHierarchyCache, IDisposable
|
||||
{
|
||||
private static readonly TimeSpan StaleThreshold = TimeSpan.FromMinutes(5);
|
||||
|
||||
private readonly IGalaxyRepository _repository;
|
||||
private readonly IGalaxyDeployNotifier _notifier;
|
||||
private readonly IGalaxyHierarchySnapshotStore? _snapshotStore;
|
||||
private readonly TimeProvider _timeProvider;
|
||||
private readonly ILogger<GalaxyHierarchyCache>? _logger;
|
||||
private readonly TaskCompletionSource _firstLoad = new(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||
private readonly SemaphoreSlim _refreshGate = new(1, 1);
|
||||
private GalaxyHierarchyCacheEntry _current = GalaxyHierarchyCacheEntry.Empty;
|
||||
private bool _restoreAttempted;
|
||||
|
||||
/// <summary>Initializes a new instance of the <see cref="GalaxyHierarchyCache"/> class.</summary>
|
||||
/// <param name="repository">Galaxy Repository client for SQL queries.</param>
|
||||
/// <param name="notifier">Galaxy deploy event notifier.</param>
|
||||
/// <param name="timeProvider">Provider for current time; defaults to system time.</param>
|
||||
/// <param name="logger">Optional logger for diagnostic output.</param>
|
||||
/// <param name="snapshotStore">
|
||||
/// Optional on-disk snapshot store. When supplied, the cache persists each
|
||||
/// successful refresh and restores the last snapshot on first load.
|
||||
/// </param>
|
||||
public GalaxyHierarchyCache(
|
||||
IGalaxyRepository repository,
|
||||
IGalaxyDeployNotifier notifier,
|
||||
TimeProvider? timeProvider = null,
|
||||
ILogger<GalaxyHierarchyCache>? logger = null,
|
||||
IGalaxyHierarchySnapshotStore? snapshotStore = null)
|
||||
{
|
||||
_repository = repository;
|
||||
_notifier = notifier;
|
||||
_timeProvider = timeProvider ?? TimeProvider.System;
|
||||
_logger = logger;
|
||||
_snapshotStore = snapshotStore;
|
||||
}
|
||||
|
||||
/// <summary>Gets the current Galaxy hierarchy cache entry with projected status.</summary>
|
||||
public GalaxyHierarchyCacheEntry Current
|
||||
{
|
||||
get
|
||||
{
|
||||
GalaxyHierarchyCacheEntry snapshot = Volatile.Read(ref _current);
|
||||
GalaxyCacheStatus projected = ProjectStatus(snapshot);
|
||||
return projected == snapshot.Status
|
||||
? snapshot
|
||||
: snapshot with { Status = projected };
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Refreshes the Galaxy hierarchy cache if the deploy time has advanced.</summary>
|
||||
/// <param name="cancellationToken">Token to cancel the asynchronous operation.</param>
|
||||
/// <returns>Asynchronous task representing the refresh operation.</returns>
|
||||
public async Task RefreshAsync(CancellationToken cancellationToken)
|
||||
{
|
||||
await _refreshGate.WaitAsync(cancellationToken).ConfigureAwait(false);
|
||||
try
|
||||
{
|
||||
await RefreshCoreAsync(cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
finally
|
||||
{
|
||||
_refreshGate.Release();
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Waits for the Galaxy hierarchy cache to complete its first load.</summary>
|
||||
/// <param name="cancellationToken">Token to cancel the asynchronous operation.</param>
|
||||
/// <returns>Asynchronous task representing the wait operation.</returns>
|
||||
public Task WaitForFirstLoadAsync(CancellationToken cancellationToken)
|
||||
{
|
||||
return _firstLoad.Task.WaitAsync(cancellationToken);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Disposes the refresh gate. As a DI singleton the cache is disposed once at host
|
||||
/// shutdown, after the refresh <see cref="GalaxyHierarchyRefreshService"/> has stopped,
|
||||
/// so no in-flight refresh can be holding the gate.
|
||||
/// </summary>
|
||||
public void Dispose()
|
||||
{
|
||||
_refreshGate.Dispose();
|
||||
}
|
||||
|
||||
private async Task RefreshCoreAsync(CancellationToken cancellationToken)
|
||||
{
|
||||
// First refresh only: seed the cache from the on-disk snapshot before
|
||||
// querying SQL, so a cold start with an unreachable Galaxy database can
|
||||
// still serve last-known browse data. Runs under the refresh gate.
|
||||
if (!_restoreAttempted)
|
||||
{
|
||||
_restoreAttempted = true;
|
||||
await TryRestoreFromDiskAsync(cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
|
||||
GalaxyHierarchyCacheEntry previous = Volatile.Read(ref _current);
|
||||
DateTimeOffset queriedAt = _timeProvider.GetUtcNow();
|
||||
|
||||
try
|
||||
{
|
||||
DateTime? deployRaw = await _repository.GetLastDeployTimeAsync(cancellationToken).ConfigureAwait(false);
|
||||
DateTimeOffset? deployTime = deployRaw.HasValue
|
||||
? new DateTimeOffset(DateTime.SpecifyKind(deployRaw.Value, DateTimeKind.Utc))
|
||||
: null;
|
||||
|
||||
bool hasPriorData = previous.HasData;
|
||||
bool deployChanged = !hasPriorData || deployTime != previous.LastDeployTime;
|
||||
|
||||
if (!deployChanged)
|
||||
{
|
||||
// No deploy change — skip heavy queries; just bump LastSuccessAt.
|
||||
GalaxyHierarchyCacheEntry refreshed = previous with
|
||||
{
|
||||
Status = GalaxyCacheStatus.Healthy,
|
||||
LastQueriedAt = queriedAt,
|
||||
LastSuccessAt = queriedAt,
|
||||
LastError = null,
|
||||
};
|
||||
Volatile.Write(ref _current, refreshed);
|
||||
_firstLoad.TrySetResult();
|
||||
return;
|
||||
}
|
||||
|
||||
Task<List<GalaxyHierarchyRow>> hierarchyTask = _repository.GetHierarchyAsync(cancellationToken);
|
||||
Task<List<GalaxyAttributeRow>> attributesTask = _repository.GetAttributesAsync(cancellationToken);
|
||||
await Task.WhenAll(hierarchyTask, attributesTask).ConfigureAwait(false);
|
||||
|
||||
List<GalaxyHierarchyRow> hierarchy = hierarchyTask.Result;
|
||||
List<GalaxyAttributeRow> attributes = attributesTask.Result;
|
||||
|
||||
long nextSequence = previous.Sequence + 1;
|
||||
GalaxyHierarchyCacheEntry next = BuildEntry(
|
||||
status: GalaxyCacheStatus.Healthy,
|
||||
sequence: nextSequence,
|
||||
lastQueriedAt: queriedAt,
|
||||
lastSuccessAt: queriedAt,
|
||||
lastDeployTime: deployTime,
|
||||
lastError: null,
|
||||
hierarchy: hierarchy,
|
||||
attributes: attributes);
|
||||
|
||||
Volatile.Write(ref _current, next);
|
||||
_firstLoad.TrySetResult();
|
||||
|
||||
_notifier.Publish(new GalaxyDeployEventInfo(
|
||||
Sequence: nextSequence,
|
||||
ObservedAt: queriedAt,
|
||||
TimeOfLastDeploy: deployTime,
|
||||
ObjectCount: hierarchy.Count,
|
||||
AttributeCount: attributes.Count));
|
||||
|
||||
await PersistSnapshotAsync(deployTime, queriedAt, hierarchy, attributes, cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
|
||||
{
|
||||
throw;
|
||||
}
|
||||
catch (Exception exception)
|
||||
{
|
||||
// Catch every non-cancellation failure — not just SqlException /
|
||||
// InvalidOperationException. A TimeoutException or Win32Exception
|
||||
// from connection establishment, or another DbException subtype,
|
||||
// must still degrade gracefully to Stale/Unavailable and complete
|
||||
// _firstLoad rather than escape and fault the refresh BackgroundService.
|
||||
_logger?.LogWarning(exception, "Galaxy hierarchy cache refresh failed.");
|
||||
GalaxyHierarchyCacheEntry failed = previous with
|
||||
{
|
||||
Status = previous.HasData ? GalaxyCacheStatus.Stale : GalaxyCacheStatus.Unavailable,
|
||||
LastQueriedAt = queriedAt,
|
||||
LastError = exception.Message,
|
||||
};
|
||||
Volatile.Write(ref _current, failed);
|
||||
_firstLoad.TrySetResult();
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Materializes a complete <see cref="GalaxyHierarchyCacheEntry"/> from raw
|
||||
/// hierarchy and attribute rowsets. Shared by the live refresh path and the
|
||||
/// on-disk restore path so both produce an identical object list and index.
|
||||
/// </summary>
|
||||
private static GalaxyHierarchyCacheEntry BuildEntry(
|
||||
GalaxyCacheStatus status,
|
||||
long sequence,
|
||||
DateTimeOffset? lastQueriedAt,
|
||||
DateTimeOffset? lastSuccessAt,
|
||||
DateTimeOffset? lastDeployTime,
|
||||
string? lastError,
|
||||
IReadOnlyList<GalaxyHierarchyRow> hierarchy,
|
||||
IReadOnlyList<GalaxyAttributeRow> attributes)
|
||||
{
|
||||
IReadOnlyList<GalaxyObject> objects = BuildObjects(hierarchy, attributes);
|
||||
GalaxyHierarchyIndex index = GalaxyHierarchyIndex.Build(objects);
|
||||
|
||||
int areaCount = hierarchy.Count(row => row.IsArea);
|
||||
int historized = attributes.Count(row => row.IsHistorized);
|
||||
int alarms = attributes.Count(row => row.IsAlarm);
|
||||
|
||||
return new GalaxyHierarchyCacheEntry(
|
||||
Status: status,
|
||||
Sequence: sequence,
|
||||
LastQueriedAt: lastQueriedAt,
|
||||
LastSuccessAt: lastSuccessAt,
|
||||
LastDeployTime: lastDeployTime,
|
||||
LastError: lastError,
|
||||
Objects: objects,
|
||||
Index: index,
|
||||
ObjectCount: hierarchy.Count,
|
||||
AreaCount: areaCount,
|
||||
AttributeCount: attributes.Count,
|
||||
HistorizedAttributeCount: historized,
|
||||
AlarmAttributeCount: alarms);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Seeds the cache from the on-disk snapshot when no live data has loaded yet.
|
||||
/// The restored entry is marked <see cref="GalaxyCacheStatus.Stale"/> — it is
|
||||
/// last-known data, not live. A later refresh that observes the same deploy
|
||||
/// time promotes it to healthy; one that observes a newer deploy replaces it.
|
||||
/// </summary>
|
||||
private async Task TryRestoreFromDiskAsync(CancellationToken cancellationToken)
|
||||
{
|
||||
if (_snapshotStore is null)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
if (Volatile.Read(ref _current).HasData)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
GalaxyHierarchySnapshot? snapshot;
|
||||
try
|
||||
{
|
||||
snapshot = await _snapshotStore.TryLoadAsync(cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
|
||||
{
|
||||
throw;
|
||||
}
|
||||
catch (Exception exception)
|
||||
{
|
||||
_logger?.LogWarning(exception, "Failed to restore the Galaxy hierarchy from the on-disk snapshot.");
|
||||
return;
|
||||
}
|
||||
|
||||
if (snapshot is null)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
long sequence = Volatile.Read(ref _current).Sequence + 1;
|
||||
GalaxyHierarchyCacheEntry restored = BuildEntry(
|
||||
status: GalaxyCacheStatus.Stale,
|
||||
sequence: sequence,
|
||||
lastQueriedAt: snapshot.SavedAt,
|
||||
lastSuccessAt: snapshot.SavedAt,
|
||||
lastDeployTime: snapshot.LastDeployTime,
|
||||
lastError: null,
|
||||
hierarchy: snapshot.Hierarchy,
|
||||
attributes: snapshot.Attributes);
|
||||
Volatile.Write(ref _current, restored);
|
||||
|
||||
// Restored data is a valid completed first load: unblock callers waiting on
|
||||
// the bootstrap gate immediately, rather than making them wait out the full
|
||||
// wait budget for a live query that — when the database is unreachable, the
|
||||
// scenario this restore exists for — may not return for seconds.
|
||||
_firstLoad.TrySetResult();
|
||||
|
||||
_notifier.Publish(new GalaxyDeployEventInfo(
|
||||
Sequence: sequence,
|
||||
ObservedAt: _timeProvider.GetUtcNow(),
|
||||
TimeOfLastDeploy: snapshot.LastDeployTime,
|
||||
ObjectCount: snapshot.Hierarchy.Count,
|
||||
AttributeCount: snapshot.Attributes.Count));
|
||||
|
||||
_logger?.LogInformation(
|
||||
"Restored Galaxy hierarchy from on-disk snapshot saved {SavedAt:o}: {ObjectCount} objects, {AttributeCount} attributes (status Stale until the Galaxy database confirms).",
|
||||
snapshot.SavedAt,
|
||||
snapshot.Hierarchy.Count,
|
||||
snapshot.Attributes.Count);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Persists a successful refresh to disk. Persistence failures are logged and
|
||||
/// swallowed — a cache that cannot write its backup is still fully usable.
|
||||
/// </summary>
|
||||
private async Task PersistSnapshotAsync(
|
||||
DateTimeOffset? deployTime,
|
||||
DateTimeOffset savedAt,
|
||||
IReadOnlyList<GalaxyHierarchyRow> hierarchy,
|
||||
IReadOnlyList<GalaxyAttributeRow> attributes,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
if (_snapshotStore is null)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
try
|
||||
{
|
||||
await _snapshotStore.SaveAsync(
|
||||
new GalaxyHierarchySnapshot(deployTime, savedAt, hierarchy, attributes),
|
||||
cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
|
||||
{
|
||||
// The refresh was cancelled (service shutdown) before the write finished.
|
||||
// That is not a persistence failure — do not log it as a warning.
|
||||
}
|
||||
catch (Exception exception)
|
||||
{
|
||||
_logger?.LogWarning(exception, "Failed to persist the Galaxy hierarchy snapshot to disk.");
|
||||
}
|
||||
}
|
||||
|
||||
private static IReadOnlyList<GalaxyObject> BuildObjects(
|
||||
IReadOnlyList<GalaxyHierarchyRow> hierarchy,
|
||||
IReadOnlyList<GalaxyAttributeRow> attributes)
|
||||
{
|
||||
Dictionary<int, List<GalaxyAttributeRow>> attributesByGobjectId = attributes
|
||||
.GroupBy(a => a.GobjectId)
|
||||
.ToDictionary(g => g.Key, g => g.ToList());
|
||||
|
||||
List<GalaxyObject> objects = new(hierarchy.Count);
|
||||
foreach (GalaxyHierarchyRow row in hierarchy)
|
||||
{
|
||||
objects.Add(GalaxyProtoMapper.MapObject(row, attributesByGobjectId));
|
||||
}
|
||||
return objects;
|
||||
}
|
||||
|
||||
private GalaxyCacheStatus ProjectStatus(GalaxyHierarchyCacheEntry snapshot)
|
||||
{
|
||||
if (snapshot.Status is GalaxyCacheStatus.Unknown or GalaxyCacheStatus.Unavailable)
|
||||
{
|
||||
return snapshot.Status;
|
||||
}
|
||||
|
||||
if (snapshot.LastSuccessAt is { } success
|
||||
&& _timeProvider.GetUtcNow() - success > StaleThreshold)
|
||||
{
|
||||
return GalaxyCacheStatus.Stale;
|
||||
}
|
||||
|
||||
return snapshot.Status;
|
||||
}
|
||||
}
|
||||
+56
@@ -0,0 +1,56 @@
|
||||
using ZB.MOM.WW.GalaxyRepository.Grpc;
|
||||
|
||||
namespace ZB.MOM.WW.GalaxyRepository;
|
||||
|
||||
/// <summary>
|
||||
/// Immutable snapshot of the Galaxy Repository browse data held by
|
||||
/// <see cref="GalaxyHierarchyCache"/>. Multiple gRPC clients share the same
|
||||
/// materialized object list and precomputed hierarchy index.
|
||||
/// </summary>
|
||||
/// <param name="Status">The cache freshness state at the time the entry was produced.</param>
|
||||
/// <param name="Sequence">Monotonically increasing per process start; bumped on each heavy refresh.</param>
|
||||
/// <param name="LastQueriedAt">UTC wall-clock of the most recent refresh attempt.</param>
|
||||
/// <param name="LastSuccessAt">UTC wall-clock of the most recent successful refresh.</param>
|
||||
/// <param name="LastDeployTime">The <c>galaxy.time_of_last_deploy</c> the data was pulled at.</param>
|
||||
/// <param name="LastError">The most recent refresh error message, or <see langword="null"/>.</param>
|
||||
/// <param name="Objects">The materialized Galaxy object list.</param>
|
||||
/// <param name="Index">Precomputed lookup structures over <paramref name="Objects"/>.</param>
|
||||
/// <param name="ObjectCount">Number of objects in the hierarchy.</param>
|
||||
/// <param name="AreaCount">Number of area objects in the hierarchy.</param>
|
||||
/// <param name="AttributeCount">Number of attributes across all objects.</param>
|
||||
/// <param name="HistorizedAttributeCount">Number of historized attributes.</param>
|
||||
/// <param name="AlarmAttributeCount">Number of alarm-bearing attributes.</param>
|
||||
public sealed record GalaxyHierarchyCacheEntry(
|
||||
GalaxyCacheStatus Status,
|
||||
long Sequence,
|
||||
DateTimeOffset? LastQueriedAt,
|
||||
DateTimeOffset? LastSuccessAt,
|
||||
DateTimeOffset? LastDeployTime,
|
||||
string? LastError,
|
||||
IReadOnlyList<GalaxyObject> Objects,
|
||||
GalaxyHierarchyIndex Index,
|
||||
int ObjectCount,
|
||||
int AreaCount,
|
||||
int AttributeCount,
|
||||
int HistorizedAttributeCount,
|
||||
int AlarmAttributeCount)
|
||||
{
|
||||
/// <summary>Gets an empty Galaxy hierarchy cache entry.</summary>
|
||||
public static GalaxyHierarchyCacheEntry Empty { get; } = new(
|
||||
Status: GalaxyCacheStatus.Unknown,
|
||||
Sequence: 0,
|
||||
LastQueriedAt: null,
|
||||
LastSuccessAt: null,
|
||||
LastDeployTime: null,
|
||||
LastError: null,
|
||||
Objects: Array.Empty<GalaxyObject>(),
|
||||
Index: GalaxyHierarchyIndex.Empty,
|
||||
ObjectCount: 0,
|
||||
AreaCount: 0,
|
||||
AttributeCount: 0,
|
||||
HistorizedAttributeCount: 0,
|
||||
AlarmAttributeCount: 0);
|
||||
|
||||
/// <summary>Gets a value indicating whether the cache entry contains usable data.</summary>
|
||||
public bool HasData => Status is GalaxyCacheStatus.Healthy or GalaxyCacheStatus.Stale;
|
||||
}
|
||||
@@ -0,0 +1,206 @@
|
||||
using ZB.MOM.WW.GalaxyRepository.Grpc;
|
||||
|
||||
namespace ZB.MOM.WW.GalaxyRepository;
|
||||
|
||||
/// <summary>
|
||||
/// Precomputed lookup structures over a materialized Galaxy object list. Built once per
|
||||
/// cache entry so browse/discover handlers can resolve roots/parents by id, tag name, or
|
||||
/// contained path in O(1), enumerate direct children, and resolve tag addresses to objects
|
||||
/// or attributes without rescanning the full object list.
|
||||
/// </summary>
|
||||
public sealed class GalaxyHierarchyIndex
|
||||
{
|
||||
private GalaxyHierarchyIndex(
|
||||
IReadOnlyList<GalaxyObjectView> objectViews,
|
||||
IReadOnlyDictionary<int, GalaxyObjectView> objectViewsById,
|
||||
IReadOnlyDictionary<string, GalaxyTagLookup> tagsByAddress,
|
||||
IReadOnlyDictionary<int, IReadOnlyList<GalaxyObjectView>> childrenByParent,
|
||||
IReadOnlyDictionary<string, GalaxyObjectView> objectViewsByTagName,
|
||||
IReadOnlyDictionary<string, GalaxyObjectView> objectViewsByContainedPath)
|
||||
{
|
||||
ObjectViews = objectViews;
|
||||
ObjectViewsById = objectViewsById;
|
||||
TagsByAddress = tagsByAddress;
|
||||
ChildrenByParent = childrenByParent;
|
||||
ObjectViewsByTagName = objectViewsByTagName;
|
||||
ObjectViewsByContainedPath = objectViewsByContainedPath;
|
||||
}
|
||||
|
||||
/// <summary>Gets an empty Galaxy hierarchy index.</summary>
|
||||
public static GalaxyHierarchyIndex Empty { get; } = new(
|
||||
Array.Empty<GalaxyObjectView>(),
|
||||
new Dictionary<int, GalaxyObjectView>(),
|
||||
new Dictionary<string, GalaxyTagLookup>(StringComparer.OrdinalIgnoreCase),
|
||||
new Dictionary<int, IReadOnlyList<GalaxyObjectView>>(),
|
||||
new Dictionary<string, GalaxyObjectView>(StringComparer.OrdinalIgnoreCase),
|
||||
new Dictionary<string, GalaxyObjectView>(StringComparer.OrdinalIgnoreCase));
|
||||
|
||||
/// <summary>Gets the object views.</summary>
|
||||
public IReadOnlyList<GalaxyObjectView> ObjectViews { get; }
|
||||
|
||||
/// <summary>Gets the object views indexed by gobject id.</summary>
|
||||
public IReadOnlyDictionary<int, GalaxyObjectView> ObjectViewsById { get; }
|
||||
|
||||
/// <summary>Gets tags indexed by address.</summary>
|
||||
public IReadOnlyDictionary<string, GalaxyTagLookup> TagsByAddress { get; }
|
||||
|
||||
/// <summary>Gets direct children grouped by parent gobject id. Root objects (no parent, or self-parented) live under key 0. Each list is sorted areas-first, then by display name (OrdinalIgnoreCase).</summary>
|
||||
public IReadOnlyDictionary<int, IReadOnlyList<GalaxyObjectView>> ChildrenByParent { get; }
|
||||
|
||||
/// <summary>Gets object views indexed by <see cref="GalaxyObject.TagName"/> (OrdinalIgnoreCase). Lets browse/discover handlers resolve parents/roots by tag name in O(1) instead of scanning <see cref="ObjectViews"/>.</summary>
|
||||
public IReadOnlyDictionary<string, GalaxyObjectView> ObjectViewsByTagName { get; }
|
||||
|
||||
/// <summary>Gets object views indexed by contained path (OrdinalIgnoreCase). Lets browse/discover handlers resolve parents/roots by path in O(1) instead of scanning <see cref="ObjectViews"/>.</summary>
|
||||
public IReadOnlyDictionary<string, GalaxyObjectView> ObjectViewsByContainedPath { get; }
|
||||
|
||||
/// <summary>Builds a Galaxy hierarchy index from the given objects.</summary>
|
||||
/// <param name="objects">The Galaxy objects to index.</param>
|
||||
/// <returns>A new Galaxy hierarchy index.</returns>
|
||||
public static GalaxyHierarchyIndex Build(IReadOnlyList<GalaxyObject> objects)
|
||||
{
|
||||
if (objects.Count == 0)
|
||||
{
|
||||
return Empty;
|
||||
}
|
||||
|
||||
Dictionary<int, GalaxyObject> objectsById = new();
|
||||
foreach (GalaxyObject obj in objects)
|
||||
{
|
||||
objectsById.TryAdd(obj.GobjectId, obj);
|
||||
}
|
||||
|
||||
List<GalaxyObjectView> views = new(objects.Count);
|
||||
Dictionary<int, GalaxyObjectView> viewsById = new();
|
||||
Dictionary<string, GalaxyTagLookup> tagsByAddress = new(StringComparer.OrdinalIgnoreCase);
|
||||
Dictionary<string, GalaxyObjectView> viewsByTagName = new(StringComparer.OrdinalIgnoreCase);
|
||||
Dictionary<string, GalaxyObjectView> viewsByContainedPath = new(StringComparer.OrdinalIgnoreCase);
|
||||
|
||||
foreach (GalaxyObject obj in objects)
|
||||
{
|
||||
string path = BuildContainedPath(obj, objectsById);
|
||||
int depth = string.IsNullOrWhiteSpace(path) ? 0 : path.Count(character => character == '/');
|
||||
GalaxyObjectView view = new(obj, path, depth);
|
||||
views.Add(view);
|
||||
viewsById.TryAdd(obj.GobjectId, view);
|
||||
|
||||
if (!string.IsNullOrWhiteSpace(obj.TagName))
|
||||
{
|
||||
tagsByAddress.TryAdd(obj.TagName, new GalaxyTagLookup(obj, Attribute: null, path));
|
||||
viewsByTagName.TryAdd(obj.TagName, view);
|
||||
}
|
||||
|
||||
if (!string.IsNullOrWhiteSpace(path))
|
||||
{
|
||||
viewsByContainedPath.TryAdd(path, view);
|
||||
}
|
||||
|
||||
foreach (GalaxyAttribute attribute in obj.Attributes)
|
||||
{
|
||||
if (!string.IsNullOrWhiteSpace(attribute.FullTagReference))
|
||||
{
|
||||
tagsByAddress.TryAdd(attribute.FullTagReference, new GalaxyTagLookup(obj, attribute, path));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Dictionary<int, List<GalaxyObjectView>> childrenByParent = new();
|
||||
foreach (GalaxyObjectView view in views)
|
||||
{
|
||||
int parentKey = view.Object.ParentGobjectId;
|
||||
// Treat self-parented (corrupt) rows as roots.
|
||||
if (parentKey == view.Object.GobjectId)
|
||||
{
|
||||
parentKey = 0;
|
||||
}
|
||||
// Re-root orphans whose parent object is absent from the set (e.g. a deleted or
|
||||
// never-loaded container area). Otherwise they bucket under a phantom parent id
|
||||
// that is never reached from the root, so they vanish from browse entirely.
|
||||
else if (parentKey != 0 && !objectsById.ContainsKey(parentKey))
|
||||
{
|
||||
parentKey = 0;
|
||||
}
|
||||
if (!childrenByParent.TryGetValue(parentKey, out List<GalaxyObjectView>? bucket))
|
||||
{
|
||||
bucket = [];
|
||||
childrenByParent[parentKey] = bucket;
|
||||
}
|
||||
bucket.Add(view);
|
||||
}
|
||||
|
||||
foreach (List<GalaxyObjectView> bucket in childrenByParent.Values)
|
||||
{
|
||||
bucket.Sort(CompareByAreaThenDisplayName);
|
||||
}
|
||||
|
||||
Dictionary<int, IReadOnlyList<GalaxyObjectView>> readOnlyChildren = new(childrenByParent.Count);
|
||||
foreach (KeyValuePair<int, List<GalaxyObjectView>> kvp in childrenByParent)
|
||||
{
|
||||
readOnlyChildren[kvp.Key] = kvp.Value;
|
||||
}
|
||||
|
||||
return new GalaxyHierarchyIndex(
|
||||
views,
|
||||
viewsById,
|
||||
tagsByAddress,
|
||||
readOnlyChildren,
|
||||
viewsByTagName,
|
||||
viewsByContainedPath);
|
||||
}
|
||||
|
||||
private static string BuildContainedPath(
|
||||
GalaxyObject obj,
|
||||
IReadOnlyDictionary<int, GalaxyObject> objectsById)
|
||||
{
|
||||
Stack<string> names = new();
|
||||
HashSet<int> seen = [];
|
||||
GalaxyObject? current = obj;
|
||||
while (current is not null && seen.Add(current.GobjectId))
|
||||
{
|
||||
names.Push(ResolvePathSegment(current));
|
||||
current = current.ParentGobjectId != 0
|
||||
&& objectsById.TryGetValue(current.ParentGobjectId, out GalaxyObject? parent)
|
||||
? parent
|
||||
: null;
|
||||
}
|
||||
|
||||
return string.Join('/', names.Where(name => !string.IsNullOrWhiteSpace(name)));
|
||||
}
|
||||
|
||||
private static string ResolvePathSegment(GalaxyObject obj)
|
||||
{
|
||||
if (!string.IsNullOrWhiteSpace(obj.ContainedName))
|
||||
{
|
||||
return obj.ContainedName;
|
||||
}
|
||||
|
||||
if (!string.IsNullOrWhiteSpace(obj.BrowseName))
|
||||
{
|
||||
return obj.BrowseName;
|
||||
}
|
||||
|
||||
return obj.TagName;
|
||||
}
|
||||
|
||||
private static int CompareByAreaThenDisplayName(GalaxyObjectView left, GalaxyObjectView right)
|
||||
{
|
||||
if (left.Object.IsArea != right.Object.IsArea)
|
||||
{
|
||||
return left.Object.IsArea ? -1 : 1;
|
||||
}
|
||||
return string.Compare(DisplayNameOf(left), DisplayNameOf(right), StringComparison.OrdinalIgnoreCase);
|
||||
}
|
||||
|
||||
private static string DisplayNameOf(GalaxyObjectView view)
|
||||
{
|
||||
GalaxyObject obj = view.Object;
|
||||
if (!string.IsNullOrWhiteSpace(obj.BrowseName))
|
||||
{
|
||||
return obj.BrowseName;
|
||||
}
|
||||
if (!string.IsNullOrWhiteSpace(obj.ContainedName))
|
||||
{
|
||||
return obj.ContainedName;
|
||||
}
|
||||
return obj.TagName;
|
||||
}
|
||||
}
|
||||
+317
@@ -0,0 +1,317 @@
|
||||
using System.Collections.Concurrent;
|
||||
using System.Runtime.CompilerServices;
|
||||
using System.Security.Cryptography;
|
||||
using System.Text;
|
||||
using Grpc.Core;
|
||||
using ZB.MOM.WW.GalaxyRepository.Grpc;
|
||||
|
||||
namespace ZB.MOM.WW.GalaxyRepository;
|
||||
|
||||
/// <summary>
|
||||
/// Projects a <c>DiscoverHierarchy</c> request against an immutable
|
||||
/// <see cref="GalaxyHierarchyCacheEntry"/>: applies the root/depth/category/template/glob
|
||||
/// filters, pages the result, and memoizes the filtered list per cache-entry instance so
|
||||
/// paging is O(pageSize) rather than O(total) per page. Pure and side-effect free.
|
||||
/// </summary>
|
||||
public static class GalaxyHierarchyProjector
|
||||
{
|
||||
/// <summary>
|
||||
/// Per-cache-entry memo of filtered, ordered <see cref="GalaxyObjectView"/> lists
|
||||
/// keyed by filter signature. Without it, paging through a large hierarchy
|
||||
/// re-applies every filter and re-scans the full <see cref="GalaxyHierarchyIndex.ObjectViews"/>
|
||||
/// collection on every page — O(total) per page, O(total²/pageSize) end-to-end.
|
||||
/// With it, the first page builds the filtered list and each subsequent page is an
|
||||
/// O(pageSize) slice. The table is keyed on the immutable cache-entry instance, so
|
||||
/// when the cache publishes a new entry the stale memo becomes unreachable and is
|
||||
/// reclaimed with it — no explicit invalidation needed.
|
||||
/// </summary>
|
||||
private static readonly ConditionalWeakTable<GalaxyHierarchyCacheEntry, ConcurrentDictionary<string, IReadOnlyList<GalaxyObjectView>>> FilteredViewCache = new();
|
||||
|
||||
/// <summary>Projects a discovery request against a cache entry and returns all matching objects.</summary>
|
||||
/// <param name="entry">The Galaxy hierarchy cache entry.</param>
|
||||
/// <param name="request">The discovery hierarchy request.</param>
|
||||
/// <param name="browseSubtreeGlobs">Optional glob patterns to filter browse subtrees.</param>
|
||||
public static GalaxyHierarchyQueryResult Project(
|
||||
GalaxyHierarchyCacheEntry entry,
|
||||
DiscoverHierarchyRequest request,
|
||||
IReadOnlyList<string>? browseSubtreeGlobs = null)
|
||||
{
|
||||
return Project(
|
||||
entry,
|
||||
request,
|
||||
browseSubtreeGlobs,
|
||||
offset: 0,
|
||||
pageSize: int.MaxValue);
|
||||
}
|
||||
|
||||
/// <summary>Projects a discovery request with paging against a cache entry and returns a page of matching objects.</summary>
|
||||
/// <param name="entry">The Galaxy hierarchy cache entry.</param>
|
||||
/// <param name="request">The discovery hierarchy request.</param>
|
||||
/// <param name="browseSubtreeGlobs">Optional glob patterns to filter browse subtrees.</param>
|
||||
/// <param name="offset">The zero-based offset into the result set.</param>
|
||||
/// <param name="pageSize">The maximum number of results to return.</param>
|
||||
public static GalaxyHierarchyQueryResult Project(
|
||||
GalaxyHierarchyCacheEntry entry,
|
||||
DiscoverHierarchyRequest request,
|
||||
IReadOnlyList<string>? browseSubtreeGlobs,
|
||||
int offset,
|
||||
int pageSize)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(entry);
|
||||
ArgumentNullException.ThrowIfNull(request);
|
||||
if (offset < 0)
|
||||
{
|
||||
throw new ArgumentOutOfRangeException(nameof(offset), offset, "Offset must be greater than or equal to zero.");
|
||||
}
|
||||
|
||||
if (pageSize <= 0)
|
||||
{
|
||||
throw new ArgumentOutOfRangeException(nameof(pageSize), pageSize, "Page size must be greater than zero.");
|
||||
}
|
||||
|
||||
int? maxDepth = request.MaxDepth;
|
||||
if (maxDepth < 0)
|
||||
{
|
||||
throw new RpcException(new Status(
|
||||
StatusCode.InvalidArgument,
|
||||
"DiscoverHierarchy max_depth must be greater than or equal to zero when provided."));
|
||||
}
|
||||
|
||||
string filterSignature = ComputeFilterSignature(request, browseSubtreeGlobs);
|
||||
IReadOnlyList<GalaxyObjectView> matchedViews = GetFilteredViews(
|
||||
entry,
|
||||
request,
|
||||
browseSubtreeGlobs,
|
||||
maxDepth,
|
||||
filterSignature);
|
||||
|
||||
bool includeAttributes = IncludeAttributes(request);
|
||||
List<GalaxyObject> page = new(Math.Min(pageSize, Math.Max(0, matchedViews.Count - offset)));
|
||||
int end = (int)Math.Min((long)offset + pageSize, matchedViews.Count);
|
||||
for (int index = offset; index < end; index++)
|
||||
{
|
||||
page.Add(CloneObject(matchedViews[index].Object, includeAttributes));
|
||||
}
|
||||
|
||||
return new GalaxyHierarchyQueryResult(
|
||||
page,
|
||||
matchedViews.Count,
|
||||
filterSignature);
|
||||
}
|
||||
|
||||
private static IReadOnlyList<GalaxyObjectView> GetFilteredViews(
|
||||
GalaxyHierarchyCacheEntry entry,
|
||||
DiscoverHierarchyRequest request,
|
||||
IReadOnlyList<string>? browseSubtreeGlobs,
|
||||
int? maxDepth,
|
||||
string filterSignature)
|
||||
{
|
||||
// ResolveRoot can throw RpcException(NotFound); run it before consulting the
|
||||
// memo so a bad root surfaces consistently regardless of cache state.
|
||||
IReadOnlyList<GalaxyObjectView> views = entry.Index.ObjectViews;
|
||||
GalaxyObjectView? root = ResolveRoot(request, entry.Index);
|
||||
|
||||
ConcurrentDictionary<string, IReadOnlyList<GalaxyObjectView>> memo =
|
||||
FilteredViewCache.GetValue(entry, static _ => new ConcurrentDictionary<string, IReadOnlyList<GalaxyObjectView>>(StringComparer.Ordinal));
|
||||
|
||||
return memo.GetOrAdd(
|
||||
filterSignature,
|
||||
static (_, state) =>
|
||||
{
|
||||
List<GalaxyObjectView> matched = [];
|
||||
foreach (GalaxyObjectView view in state.Views)
|
||||
{
|
||||
if (MatchesRoot(view, state.Root, state.MaxDepth)
|
||||
&& MatchesBrowseSubtrees(view, state.BrowseSubtreeGlobs)
|
||||
&& MatchesFilters(view.Object, state.Request))
|
||||
{
|
||||
matched.Add(view);
|
||||
}
|
||||
}
|
||||
|
||||
return matched;
|
||||
},
|
||||
(Views: views, Root: root, MaxDepth: maxDepth, BrowseSubtreeGlobs: browseSubtreeGlobs, Request: request));
|
||||
}
|
||||
|
||||
/// <summary>Finds an object in the hierarchy by its tag address.</summary>
|
||||
/// <param name="entry">The Galaxy hierarchy cache entry.</param>
|
||||
/// <param name="tagAddress">The tag address to search for.</param>
|
||||
public static GalaxyObject? FindObjectForTag(
|
||||
GalaxyHierarchyCacheEntry entry,
|
||||
string tagAddress)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(tagAddress))
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
return entry.Index.TagsByAddress.TryGetValue(tagAddress, out GalaxyTagLookup? lookup)
|
||||
? lookup.Object
|
||||
: null;
|
||||
}
|
||||
|
||||
/// <summary>Finds an attribute in the hierarchy by its tag address.</summary>
|
||||
/// <param name="entry">The Galaxy hierarchy cache entry.</param>
|
||||
/// <param name="tagAddress">The tag address to search for.</param>
|
||||
public static GalaxyAttribute? FindAttributeForTag(
|
||||
GalaxyHierarchyCacheEntry entry,
|
||||
string tagAddress)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(tagAddress))
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
return entry.Index.TagsByAddress.TryGetValue(tagAddress, out GalaxyTagLookup? lookup)
|
||||
? lookup.Attribute
|
||||
: null;
|
||||
}
|
||||
|
||||
/// <summary>Gets the contained path for an object by its gobject ID.</summary>
|
||||
/// <param name="entry">The Galaxy hierarchy cache entry.</param>
|
||||
/// <param name="gobjectId">The Galaxy object ID.</param>
|
||||
public static string GetContainedPath(
|
||||
GalaxyHierarchyCacheEntry entry,
|
||||
int gobjectId)
|
||||
{
|
||||
return entry.Index.ObjectViewsById.TryGetValue(gobjectId, out GalaxyObjectView? view)
|
||||
? view.ContainedPath
|
||||
: string.Empty;
|
||||
}
|
||||
|
||||
private static GalaxyObjectView? ResolveRoot(
|
||||
DiscoverHierarchyRequest request,
|
||||
GalaxyHierarchyIndex index)
|
||||
{
|
||||
GalaxyObjectView? root = request.RootCase switch
|
||||
{
|
||||
DiscoverHierarchyRequest.RootOneofCase.None => null,
|
||||
DiscoverHierarchyRequest.RootOneofCase.RootGobjectId =>
|
||||
index.ObjectViewsById.TryGetValue(request.RootGobjectId, out GalaxyObjectView? byId) ? byId : null,
|
||||
DiscoverHierarchyRequest.RootOneofCase.RootTagName =>
|
||||
index.ObjectViewsByTagName.TryGetValue(request.RootTagName, out GalaxyObjectView? byTag) ? byTag : null,
|
||||
DiscoverHierarchyRequest.RootOneofCase.RootContainedPath =>
|
||||
index.ObjectViewsByContainedPath.TryGetValue(request.RootContainedPath, out GalaxyObjectView? byPath) ? byPath : null,
|
||||
_ => null,
|
||||
};
|
||||
|
||||
if (request.RootCase != DiscoverHierarchyRequest.RootOneofCase.None && root is null)
|
||||
{
|
||||
throw new RpcException(new Status(StatusCode.NotFound, "DiscoverHierarchy root was not found."));
|
||||
}
|
||||
|
||||
return root;
|
||||
}
|
||||
|
||||
private static bool MatchesRoot(
|
||||
GalaxyObjectView view,
|
||||
GalaxyObjectView? root,
|
||||
int? maxDepth)
|
||||
{
|
||||
if (root is null)
|
||||
{
|
||||
return true;
|
||||
}
|
||||
|
||||
bool isRoot = view.Object.GobjectId == root.Object.GobjectId;
|
||||
bool isDescendant = view.ContainedPath.StartsWith(root.ContainedPath + "/", StringComparison.OrdinalIgnoreCase);
|
||||
if (!isRoot && !isDescendant)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
return maxDepth is null || view.Depth - root.Depth <= maxDepth.Value;
|
||||
}
|
||||
|
||||
private static bool MatchesBrowseSubtrees(
|
||||
GalaxyObjectView view,
|
||||
IReadOnlyList<string>? browseSubtreeGlobs)
|
||||
{
|
||||
return browseSubtreeGlobs is null
|
||||
|| browseSubtreeGlobs.Count == 0
|
||||
|| browseSubtreeGlobs.Any(glob => GalaxyGlobMatcher.IsMatch(view.ContainedPath, glob));
|
||||
}
|
||||
|
||||
private static bool MatchesFilters(
|
||||
GalaxyObject obj,
|
||||
DiscoverHierarchyRequest request)
|
||||
{
|
||||
if (request.CategoryIds.Count > 0 && !request.CategoryIds.Contains(obj.CategoryId))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
foreach (string templateFilter in request.TemplateChainContains)
|
||||
{
|
||||
if (!obj.TemplateChain.Any(template => template.Contains(templateFilter, StringComparison.OrdinalIgnoreCase)))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if (!string.IsNullOrWhiteSpace(request.TagNameGlob)
|
||||
&& !GalaxyGlobMatcher.IsMatch(obj.TagName, request.TagNameGlob))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
if (request.AlarmBearingOnly && !obj.Attributes.Any(attribute => attribute.IsAlarm))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
if (request.HistorizedOnly && !obj.Attributes.Any(attribute => attribute.IsHistorized))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
private static bool IncludeAttributes(DiscoverHierarchyRequest request)
|
||||
{
|
||||
return !request.HasIncludeAttributes || request.IncludeAttributes;
|
||||
}
|
||||
|
||||
private static GalaxyObject CloneObject(GalaxyObject source, bool includeAttributes)
|
||||
{
|
||||
GalaxyObject clone = source.Clone();
|
||||
if (!includeAttributes)
|
||||
{
|
||||
clone.Attributes.Clear();
|
||||
}
|
||||
|
||||
return clone;
|
||||
}
|
||||
|
||||
/// <summary>Computes a stable filter signature for memoization purposes.</summary>
|
||||
/// <param name="request">The discovery hierarchy request.</param>
|
||||
/// <param name="browseSubtreeGlobs">Optional glob patterns to filter browse subtrees.</param>
|
||||
public static string ComputeFilterSignature(
|
||||
DiscoverHierarchyRequest request,
|
||||
IReadOnlyList<string>? browseSubtreeGlobs)
|
||||
{
|
||||
StringBuilder builder = new();
|
||||
builder.Append("root=").Append(request.RootCase).Append('|');
|
||||
builder.Append(request.RootCase switch
|
||||
{
|
||||
DiscoverHierarchyRequest.RootOneofCase.RootGobjectId => request.RootGobjectId.ToString(
|
||||
System.Globalization.CultureInfo.InvariantCulture),
|
||||
DiscoverHierarchyRequest.RootOneofCase.RootTagName => request.RootTagName,
|
||||
DiscoverHierarchyRequest.RootOneofCase.RootContainedPath => request.RootContainedPath,
|
||||
_ => string.Empty,
|
||||
});
|
||||
builder.Append("|max=").Append(request.MaxDepth?.ToString(System.Globalization.CultureInfo.InvariantCulture) ?? "");
|
||||
builder.Append("|cat=").AppendJoin(',', request.CategoryIds.Order());
|
||||
builder.Append("|tpl=").AppendJoin(',', request.TemplateChainContains.Order(StringComparer.OrdinalIgnoreCase));
|
||||
builder.Append("|glob=").Append(request.TagNameGlob);
|
||||
builder.Append("|attrs=").Append(request.HasIncludeAttributes ? request.IncludeAttributes.ToString() : "unset");
|
||||
builder.Append("|alarm=").Append(request.AlarmBearingOnly);
|
||||
builder.Append("|hist=").Append(request.HistorizedOnly);
|
||||
builder.Append("|browse=").AppendJoin(',', (browseSubtreeGlobs ?? Array.Empty<string>()).Order(StringComparer.OrdinalIgnoreCase));
|
||||
|
||||
byte[] hash = SHA256.HashData(Encoding.UTF8.GetBytes(builder.ToString()));
|
||||
return Convert.ToHexString(hash, 0, 12);
|
||||
}
|
||||
}
|
||||
+16
@@ -0,0 +1,16 @@
|
||||
using ZB.MOM.WW.GalaxyRepository.Grpc;
|
||||
|
||||
namespace ZB.MOM.WW.GalaxyRepository;
|
||||
|
||||
/// <summary>
|
||||
/// Result of one <see cref="GalaxyHierarchyProjector.Project(GalaxyHierarchyCacheEntry, DiscoverHierarchyRequest, System.Collections.Generic.IReadOnlyList{string}, int, int)"/>
|
||||
/// call: a materialized page of matching objects, the total post-filter object count, and
|
||||
/// the stable filter signature used to bind page tokens.
|
||||
/// </summary>
|
||||
/// <param name="Objects">The page of matching objects.</param>
|
||||
/// <param name="TotalObjectCount">Total matching objects across the whole hierarchy (post-filter).</param>
|
||||
/// <param name="FilterSignature">Stable signature of the filter set, used to bind page tokens.</param>
|
||||
public sealed record GalaxyHierarchyQueryResult(
|
||||
IReadOnlyList<GalaxyObject> Objects,
|
||||
int TotalObjectCount,
|
||||
string FilterSignature);
|
||||
+62
@@ -0,0 +1,62 @@
|
||||
using Microsoft.Extensions.Hosting;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
|
||||
namespace ZB.MOM.WW.GalaxyRepository;
|
||||
|
||||
/// <summary>Background service that periodically refreshes the Galaxy Repository hierarchy cache off the request path.</summary>
|
||||
public sealed class GalaxyHierarchyRefreshService(
|
||||
IGalaxyHierarchyCache cache,
|
||||
IOptions<GalaxyRepositoryOptions> options,
|
||||
ILogger<GalaxyHierarchyRefreshService> logger,
|
||||
TimeProvider? timeProvider = null) : BackgroundService
|
||||
{
|
||||
private readonly TimeProvider _timeProvider = timeProvider ?? TimeProvider.System;
|
||||
|
||||
/// <inheritdoc />
|
||||
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
|
||||
{
|
||||
TimeSpan interval = TimeSpan.FromSeconds(Math.Max(1, options.Value.DashboardRefreshIntervalSeconds));
|
||||
|
||||
try
|
||||
{
|
||||
await cache.RefreshAsync(stoppingToken).ConfigureAwait(false);
|
||||
}
|
||||
catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
|
||||
{
|
||||
return;
|
||||
}
|
||||
catch (Exception exception)
|
||||
{
|
||||
// A transient first-load failure (e.g. a TimeoutException or
|
||||
// Win32Exception from connection establishment, or a DbException
|
||||
// subtype the cache does not catch) must not fault this
|
||||
// BackgroundService and stop the whole host. The cache records
|
||||
// its own Unavailable/Stale status; the periodic tick below retries.
|
||||
logger.LogWarning(exception, "Initial Galaxy hierarchy cache load failed; will retry on the refresh interval.");
|
||||
}
|
||||
|
||||
using PeriodicTimer timer = new(interval, _timeProvider);
|
||||
try
|
||||
{
|
||||
while (await timer.WaitForNextTickAsync(stoppingToken).ConfigureAwait(false))
|
||||
{
|
||||
try
|
||||
{
|
||||
await cache.RefreshAsync(stoppingToken).ConfigureAwait(false);
|
||||
}
|
||||
catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
|
||||
{
|
||||
return;
|
||||
}
|
||||
catch (Exception exception)
|
||||
{
|
||||
logger.LogWarning(exception, "Galaxy hierarchy cache refresh tick failed.");
|
||||
}
|
||||
}
|
||||
}
|
||||
catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
|
||||
{
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,35 @@
|
||||
namespace ZB.MOM.WW.GalaxyRepository;
|
||||
|
||||
/// <summary>
|
||||
/// One row from <see cref="GalaxyRepository.GetHierarchyAsync"/>: a deployed Galaxy
|
||||
/// <c>gobject</c> with its hierarchy parent and template-derivation chain.
|
||||
/// </summary>
|
||||
public sealed class GalaxyHierarchyRow
|
||||
{
|
||||
/// <summary>Gets the Galaxy object identifier.</summary>
|
||||
public int GobjectId { get; init; }
|
||||
|
||||
/// <summary>Gets the tag name.</summary>
|
||||
public string TagName { get; init; } = string.Empty;
|
||||
|
||||
/// <summary>Gets the contained name.</summary>
|
||||
public string ContainedName { get; init; } = string.Empty;
|
||||
|
||||
/// <summary>Gets the browse name.</summary>
|
||||
public string BrowseName { get; init; } = string.Empty;
|
||||
|
||||
/// <summary>Gets the parent Galaxy object identifier.</summary>
|
||||
public int ParentGobjectId { get; init; }
|
||||
|
||||
/// <summary>Gets a value indicating whether this is an area.</summary>
|
||||
public bool IsArea { get; init; }
|
||||
|
||||
/// <summary>Gets the category identifier.</summary>
|
||||
public int CategoryId { get; init; }
|
||||
|
||||
/// <summary>Gets the Galaxy object identifier of the host.</summary>
|
||||
public int HostedByGobjectId { get; init; }
|
||||
|
||||
/// <summary>Gets the template derivation chain.</summary>
|
||||
public IReadOnlyList<string> TemplateChain { get; init; } = Array.Empty<string>();
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
namespace ZB.MOM.WW.GalaxyRepository;
|
||||
|
||||
/// <summary>
|
||||
/// A serializable point-in-time copy of the Galaxy Repository browse data.
|
||||
/// Holds the raw hierarchy and attribute rowsets — not the materialized
|
||||
/// protobuf objects — so the restore path runs the exact same
|
||||
/// materialization as a live refresh. Persisted by
|
||||
/// <see cref="IGalaxyHierarchySnapshotStore"/> after a successful refresh
|
||||
/// and reloaded at startup when the Galaxy database is unreachable.
|
||||
/// </summary>
|
||||
/// <param name="LastDeployTime">
|
||||
/// The <c>galaxy.time_of_last_deploy</c> the rowsets were pulled at, or
|
||||
/// <see langword="null"/> when the Galaxy table reported no deploy. A later
|
||||
/// live refresh that observes this same timestamp can promote the restored
|
||||
/// entry to healthy without re-running the heavy queries.
|
||||
/// </param>
|
||||
/// <param name="SavedAt">UTC wall-clock when the snapshot was written to disk.</param>
|
||||
/// <param name="Hierarchy">The persisted object-hierarchy rowset.</param>
|
||||
/// <param name="Attributes">The persisted attribute rowset.</param>
|
||||
public sealed record GalaxyHierarchySnapshot(
|
||||
DateTimeOffset? LastDeployTime,
|
||||
DateTimeOffset SavedAt,
|
||||
IReadOnlyList<GalaxyHierarchyRow> Hierarchy,
|
||||
IReadOnlyList<GalaxyAttributeRow> Attributes);
|
||||
+152
@@ -0,0 +1,152 @@
|
||||
using System.Text.Json;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
|
||||
namespace ZB.MOM.WW.GalaxyRepository;
|
||||
|
||||
/// <summary>
|
||||
/// JSON-file implementation of <see cref="IGalaxyHierarchySnapshotStore"/>.
|
||||
/// Writes the on-disk snapshot atomically (temp file + rename) so a crash
|
||||
/// mid-write can never leave a torn file, and ignores files whose schema
|
||||
/// version it does not recognize. When
|
||||
/// <see cref="GalaxyRepositoryOptions.PersistSnapshot"/> is <see langword="false"/>
|
||||
/// — or <see cref="GalaxyRepositoryOptions.SnapshotCachePath"/> is empty —
|
||||
/// both operations are no-ops. The snapshot path is fully consumer-supplied;
|
||||
/// this store imposes no platform-specific default, so it is cross-platform.
|
||||
/// </summary>
|
||||
public sealed class GalaxyHierarchySnapshotStore : IGalaxyHierarchySnapshotStore, IDisposable
|
||||
{
|
||||
/// <summary>
|
||||
/// On-disk format version. Bump this whenever the persisted shape changes
|
||||
/// in a way an older or newer consumer cannot read; a mismatched file is
|
||||
/// ignored rather than misparsed.
|
||||
/// </summary>
|
||||
private const int CurrentSchemaVersion = 1;
|
||||
|
||||
private static readonly JsonSerializerOptions SerializerOptions = new()
|
||||
{
|
||||
WriteIndented = false,
|
||||
};
|
||||
|
||||
private readonly string? _path;
|
||||
private readonly TimeSpan _writeTimeout;
|
||||
private readonly ILogger<GalaxyHierarchySnapshotStore>? _logger;
|
||||
private readonly SemaphoreSlim _ioGate = new(1, 1);
|
||||
|
||||
/// <summary>Initializes a new instance of the <see cref="GalaxyHierarchySnapshotStore"/> class.</summary>
|
||||
/// <param name="options">Galaxy repository options carrying the snapshot path and enable flag.</param>
|
||||
/// <param name="logger">Optional logger for diagnostic output.</param>
|
||||
public GalaxyHierarchySnapshotStore(
|
||||
IOptions<GalaxyRepositoryOptions> options,
|
||||
ILogger<GalaxyHierarchySnapshotStore>? logger = null)
|
||||
{
|
||||
GalaxyRepositoryOptions value = options.Value;
|
||||
_path = value.PersistSnapshot && !string.IsNullOrWhiteSpace(value.SnapshotCachePath)
|
||||
? value.SnapshotCachePath
|
||||
: null;
|
||||
_writeTimeout = TimeSpan.FromSeconds(Math.Max(1, value.CommandTimeoutSeconds));
|
||||
_logger = logger;
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
public async Task SaveAsync(GalaxyHierarchySnapshot snapshot, CancellationToken cancellationToken)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(snapshot);
|
||||
if (_path is null)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
PersistedFile file = new(CurrentSchemaVersion, snapshot);
|
||||
|
||||
await _ioGate.WaitAsync(cancellationToken).ConfigureAwait(false);
|
||||
try
|
||||
{
|
||||
// Bound the write so a stuck disk — e.g. a SnapshotCachePath on an
|
||||
// unresponsive network share — cannot stall the caller. On the cache
|
||||
// refresh path that would otherwise pin the whole refresh loop.
|
||||
using CancellationTokenSource writeCts = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
|
||||
writeCts.CancelAfter(_writeTimeout);
|
||||
|
||||
string? directory = Path.GetDirectoryName(_path);
|
||||
if (!string.IsNullOrEmpty(directory))
|
||||
{
|
||||
Directory.CreateDirectory(directory);
|
||||
}
|
||||
|
||||
string tempPath = _path + ".tmp";
|
||||
await using (FileStream stream = new(tempPath, FileMode.Create, FileAccess.Write, FileShare.None))
|
||||
{
|
||||
await JsonSerializer.SerializeAsync(stream, file, SerializerOptions, writeCts.Token).ConfigureAwait(false);
|
||||
}
|
||||
|
||||
File.Move(tempPath, _path, overwrite: true);
|
||||
_logger?.LogDebug(
|
||||
"Persisted Galaxy hierarchy snapshot to {Path} ({ObjectCount} objects, {AttributeCount} attributes).",
|
||||
_path,
|
||||
snapshot.Hierarchy.Count,
|
||||
snapshot.Attributes.Count);
|
||||
}
|
||||
finally
|
||||
{
|
||||
_ioGate.Release();
|
||||
}
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
public async Task<GalaxyHierarchySnapshot?> TryLoadAsync(CancellationToken cancellationToken)
|
||||
{
|
||||
if (_path is null || !File.Exists(_path))
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
await _ioGate.WaitAsync(cancellationToken).ConfigureAwait(false);
|
||||
try
|
||||
{
|
||||
PersistedFile? file;
|
||||
await using (FileStream stream = new(_path, FileMode.Open, FileAccess.Read, FileShare.Read))
|
||||
{
|
||||
file = await JsonSerializer.DeserializeAsync<PersistedFile>(
|
||||
stream, SerializerOptions, cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
|
||||
if (file is null || file.SchemaVersion != CurrentSchemaVersion || file.Snapshot is null)
|
||||
{
|
||||
_logger?.LogWarning(
|
||||
"Ignoring Galaxy hierarchy snapshot at {Path}: unrecognized or empty schema version.",
|
||||
_path);
|
||||
return null;
|
||||
}
|
||||
|
||||
return file.Snapshot;
|
||||
}
|
||||
catch (Exception exception) when (exception is JsonException or IOException or UnauthorizedAccessException)
|
||||
{
|
||||
// A corrupt, truncated, locked, or access-denied snapshot file is an
|
||||
// expected failure mode for a disk cache — honor the Try contract and
|
||||
// return null rather than throwing.
|
||||
_logger?.LogWarning(
|
||||
exception,
|
||||
"Ignoring Galaxy hierarchy snapshot at {Path}: the file is unreadable or not valid JSON.",
|
||||
_path);
|
||||
return null;
|
||||
}
|
||||
finally
|
||||
{
|
||||
_ioGate.Release();
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Disposes the I/O gate. As a DI singleton the store is disposed once at host
|
||||
/// shutdown, by which point no save/load is in flight.
|
||||
/// </summary>
|
||||
public void Dispose()
|
||||
{
|
||||
_ioGate.Dispose();
|
||||
}
|
||||
|
||||
/// <summary>On-disk envelope: a schema version plus the snapshot payload.</summary>
|
||||
private sealed record PersistedFile(int SchemaVersion, GalaxyHierarchySnapshot? Snapshot);
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
using ZB.MOM.WW.GalaxyRepository.Grpc;
|
||||
|
||||
namespace ZB.MOM.WW.GalaxyRepository;
|
||||
|
||||
/// <summary>
|
||||
/// A <see cref="GalaxyObject"/> paired with its computed contained path and hierarchy
|
||||
/// depth. Materialized once per cache entry by <see cref="GalaxyHierarchyIndex"/> so
|
||||
/// browse/discover projection can filter and page without recomputing paths.
|
||||
/// </summary>
|
||||
/// <param name="Object">The projected Galaxy object.</param>
|
||||
/// <param name="ContainedPath">The slash-delimited contained path from the hierarchy root.</param>
|
||||
/// <param name="Depth">The number of path segments from the root (zero for top-level objects).</param>
|
||||
public sealed record GalaxyObjectView(
|
||||
GalaxyObject Object,
|
||||
string ContainedPath,
|
||||
int Depth);
|
||||
@@ -0,0 +1,76 @@
|
||||
using ZB.MOM.WW.GalaxyRepository.Grpc;
|
||||
|
||||
namespace ZB.MOM.WW.GalaxyRepository;
|
||||
|
||||
/// <summary>
|
||||
/// Maps <see cref="GalaxyHierarchyRow"/> + <see cref="GalaxyAttributeRow"/> rows produced
|
||||
/// by <see cref="GalaxyRepository"/> into <c>galaxy_repository.v1</c> proto messages.
|
||||
/// Pure function, separated so it can be unit-tested without a SQL connection.
|
||||
/// </summary>
|
||||
public static class GalaxyProtoMapper
|
||||
{
|
||||
/// <summary>Maps Galaxy hierarchy and attribute rows to Galaxy object protos.</summary>
|
||||
/// <param name="hierarchy">Hierarchy rows from Galaxy Repository.</param>
|
||||
/// <param name="attributes">Attribute rows from Galaxy Repository.</param>
|
||||
public static IEnumerable<GalaxyObject> MapHierarchy(
|
||||
IReadOnlyList<GalaxyHierarchyRow> hierarchy,
|
||||
IReadOnlyList<GalaxyAttributeRow> attributes)
|
||||
{
|
||||
Dictionary<int, List<GalaxyAttributeRow>> attributesByGobjectId = attributes
|
||||
.GroupBy(a => a.GobjectId)
|
||||
.ToDictionary(g => g.Key, g => g.ToList());
|
||||
|
||||
foreach (GalaxyHierarchyRow row in hierarchy)
|
||||
{
|
||||
yield return MapObject(row, attributesByGobjectId);
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Maps a Galaxy hierarchy row to a Galaxy object proto.</summary>
|
||||
/// <param name="row">Hierarchy row from Galaxy Repository.</param>
|
||||
/// <param name="attributesByGobjectId">Attributes indexed by gobject ID.</param>
|
||||
public static GalaxyObject MapObject(
|
||||
GalaxyHierarchyRow row,
|
||||
IReadOnlyDictionary<int, List<GalaxyAttributeRow>> attributesByGobjectId)
|
||||
{
|
||||
GalaxyObject obj = new()
|
||||
{
|
||||
GobjectId = row.GobjectId,
|
||||
TagName = row.TagName,
|
||||
ContainedName = row.ContainedName,
|
||||
BrowseName = row.BrowseName,
|
||||
ParentGobjectId = row.ParentGobjectId,
|
||||
IsArea = row.IsArea,
|
||||
CategoryId = row.CategoryId,
|
||||
HostedByGobjectId = row.HostedByGobjectId,
|
||||
};
|
||||
obj.TemplateChain.AddRange(row.TemplateChain);
|
||||
|
||||
if (attributesByGobjectId.TryGetValue(row.GobjectId, out List<GalaxyAttributeRow>? attrs))
|
||||
{
|
||||
foreach (GalaxyAttributeRow attr in attrs)
|
||||
{
|
||||
obj.Attributes.Add(MapAttribute(attr));
|
||||
}
|
||||
}
|
||||
|
||||
return obj;
|
||||
}
|
||||
|
||||
/// <summary>Maps a Galaxy attribute row to a Galaxy attribute proto.</summary>
|
||||
/// <param name="row">Attribute row from Galaxy Repository.</param>
|
||||
public static GalaxyAttribute MapAttribute(GalaxyAttributeRow row) => new()
|
||||
{
|
||||
AttributeName = row.AttributeName,
|
||||
FullTagReference = row.FullTagReference,
|
||||
MxDataType = row.MxDataType,
|
||||
DataTypeName = row.DataTypeName ?? string.Empty,
|
||||
IsArray = row.IsArray,
|
||||
ArrayDimension = row.ArrayDimension ?? 0,
|
||||
ArrayDimensionPresent = row.ArrayDimension.HasValue,
|
||||
MxAttributeCategory = row.MxAttributeCategory,
|
||||
SecurityClassification = row.SecurityClassification,
|
||||
IsHistorized = row.IsHistorized,
|
||||
IsAlarm = row.IsAlarm,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,257 @@
|
||||
using Microsoft.Data.SqlClient;
|
||||
|
||||
namespace ZB.MOM.WW.GalaxyRepository;
|
||||
|
||||
/// <summary>
|
||||
/// SQL access to the AVEVA System Platform Galaxy Repository database.
|
||||
/// <para>
|
||||
/// <see cref="HierarchySql" /> is the query originally ported from the OtOpcUa
|
||||
/// project. <see cref="AttributesSql" /> has diverged: it additionally enumerates the
|
||||
/// built-in attributes contributed by each object's primitives (from
|
||||
/// <c>attribute_definition</c> via <c>primitive_instance</c>), so engine/platform objects
|
||||
/// and extension sub-attributes (e.g. <c>TestAlarm001.Acked</c>) are surfaced. The
|
||||
/// OtOpcUa query is not kept in sync.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
public sealed class GalaxyRepository(GalaxyRepositoryOptions options) : IGalaxyRepository
|
||||
{
|
||||
/// <summary>Tests the connection to the Galaxy Repository database.</summary>
|
||||
/// <param name="ct">Token to cancel the asynchronous operation.</param>
|
||||
public async Task<bool> TestConnectionAsync(CancellationToken ct = default)
|
||||
{
|
||||
try
|
||||
{
|
||||
using SqlConnection conn = new(options.ConnectionString);
|
||||
await conn.OpenAsync(ct).ConfigureAwait(false);
|
||||
using SqlCommand cmd = new("SELECT 1", conn) { CommandTimeout = options.CommandTimeoutSeconds };
|
||||
object? result = await cmd.ExecuteScalarAsync(ct).ConfigureAwait(false);
|
||||
return result is int i && i == 1;
|
||||
}
|
||||
catch (SqlException) { return false; }
|
||||
catch (InvalidOperationException) { return false; }
|
||||
}
|
||||
|
||||
/// <summary>Retrieves the last deployment time from the Galaxy Repository.</summary>
|
||||
/// <param name="ct">Token to cancel the asynchronous operation.</param>
|
||||
public async Task<DateTime?> GetLastDeployTimeAsync(CancellationToken ct = default)
|
||||
{
|
||||
using SqlConnection conn = new(options.ConnectionString);
|
||||
await conn.OpenAsync(ct).ConfigureAwait(false);
|
||||
using SqlCommand cmd = new("SELECT time_of_last_deploy FROM galaxy", conn)
|
||||
{ CommandTimeout = options.CommandTimeoutSeconds };
|
||||
object? result = await cmd.ExecuteScalarAsync(ct).ConfigureAwait(false);
|
||||
return result is DateTime dt ? dt : null;
|
||||
}
|
||||
|
||||
/// <summary>Retrieves the complete hierarchy of Galaxy objects from the repository.</summary>
|
||||
/// <param name="ct">Token to cancel the asynchronous operation.</param>
|
||||
public async Task<List<GalaxyHierarchyRow>> GetHierarchyAsync(CancellationToken ct = default)
|
||||
{
|
||||
List<GalaxyHierarchyRow> rows = new();
|
||||
|
||||
using SqlConnection conn = new(options.ConnectionString);
|
||||
await conn.OpenAsync(ct).ConfigureAwait(false);
|
||||
|
||||
using SqlCommand cmd = new(HierarchySql, conn) { CommandTimeout = options.CommandTimeoutSeconds };
|
||||
using SqlDataReader reader = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false);
|
||||
|
||||
while (await reader.ReadAsync(ct).ConfigureAwait(false))
|
||||
{
|
||||
string templateChainRaw = reader.IsDBNull(8) ? string.Empty : reader.GetString(8);
|
||||
string[] templateChain = templateChainRaw.Length == 0
|
||||
? Array.Empty<string>()
|
||||
: templateChainRaw.Split(['|'], StringSplitOptions.RemoveEmptyEntries)
|
||||
.Select(s => s.Trim())
|
||||
.Where(s => s.Length > 0)
|
||||
.ToArray();
|
||||
|
||||
rows.Add(new GalaxyHierarchyRow
|
||||
{
|
||||
GobjectId = Convert.ToInt32(reader.GetValue(0)),
|
||||
TagName = reader.GetString(1),
|
||||
ContainedName = reader.IsDBNull(2) ? string.Empty : reader.GetString(2),
|
||||
BrowseName = reader.GetString(3),
|
||||
ParentGobjectId = Convert.ToInt32(reader.GetValue(4)),
|
||||
IsArea = Convert.ToInt32(reader.GetValue(5)) == 1,
|
||||
CategoryId = Convert.ToInt32(reader.GetValue(6)),
|
||||
HostedByGobjectId = Convert.ToInt32(reader.GetValue(7)),
|
||||
TemplateChain = templateChain,
|
||||
});
|
||||
}
|
||||
return rows;
|
||||
}
|
||||
|
||||
/// <summary>Retrieves all attributes for Galaxy objects from the repository.</summary>
|
||||
/// <param name="ct">Token to cancel the asynchronous operation.</param>
|
||||
public async Task<List<GalaxyAttributeRow>> GetAttributesAsync(CancellationToken ct = default)
|
||||
{
|
||||
List<GalaxyAttributeRow> rows = new();
|
||||
|
||||
using SqlConnection conn = new(options.ConnectionString);
|
||||
await conn.OpenAsync(ct).ConfigureAwait(false);
|
||||
|
||||
using SqlCommand cmd = new(AttributesSql, conn) { CommandTimeout = options.CommandTimeoutSeconds };
|
||||
using SqlDataReader reader = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false);
|
||||
|
||||
while (await reader.ReadAsync(ct).ConfigureAwait(false))
|
||||
{
|
||||
rows.Add(new GalaxyAttributeRow
|
||||
{
|
||||
GobjectId = Convert.ToInt32(reader.GetValue(0)),
|
||||
TagName = reader.GetString(1),
|
||||
AttributeName = reader.GetString(2),
|
||||
FullTagReference = reader.GetString(3),
|
||||
MxDataType = Convert.ToInt32(reader.GetValue(4)),
|
||||
DataTypeName = reader.IsDBNull(5) ? null : reader.GetString(5),
|
||||
IsArray = Convert.ToInt32(reader.GetValue(6)) == 1,
|
||||
ArrayDimension = reader.IsDBNull(7) ? null : Convert.ToInt32(reader.GetValue(7)),
|
||||
MxAttributeCategory = Convert.ToInt32(reader.GetValue(8)),
|
||||
SecurityClassification = Convert.ToInt32(reader.GetValue(9)),
|
||||
IsHistorized = Convert.ToInt32(reader.GetValue(10)) == 1,
|
||||
IsAlarm = Convert.ToInt32(reader.GetValue(11)) == 1,
|
||||
});
|
||||
}
|
||||
return rows;
|
||||
}
|
||||
|
||||
// Area objects (category 13) are returned even when undeployed (deployed_package_id = 0):
|
||||
// they are organizational/model nodes that group deployed objects, so excluding them
|
||||
// orphans every area whose containing area is not itself deployed. All non-area objects
|
||||
// still require deployment. Orphans left by a missing/deleted parent area are re-rooted
|
||||
// by GalaxyHierarchyIndex.Build so nothing disappears from browse.
|
||||
private const string HierarchySql = @"
|
||||
;WITH template_chain AS (
|
||||
SELECT g.gobject_id AS instance_gobject_id, t.gobject_id AS template_gobject_id,
|
||||
t.tag_name AS template_tag_name, t.derived_from_gobject_id, 0 AS depth
|
||||
FROM gobject g
|
||||
INNER JOIN gobject t ON t.gobject_id = g.derived_from_gobject_id
|
||||
WHERE g.is_template = 0 AND g.deployed_package_id <> 0 AND g.derived_from_gobject_id <> 0
|
||||
UNION ALL
|
||||
SELECT tc.instance_gobject_id, t.gobject_id, t.tag_name, t.derived_from_gobject_id, tc.depth + 1
|
||||
FROM template_chain tc
|
||||
INNER JOIN gobject t ON t.gobject_id = tc.derived_from_gobject_id
|
||||
WHERE tc.derived_from_gobject_id <> 0 AND tc.depth < 10
|
||||
)
|
||||
SELECT DISTINCT
|
||||
g.gobject_id,
|
||||
g.tag_name,
|
||||
g.contained_name,
|
||||
CASE WHEN g.contained_name IS NULL OR g.contained_name = ''
|
||||
THEN g.tag_name
|
||||
ELSE g.contained_name
|
||||
END AS browse_name,
|
||||
CASE WHEN g.contained_by_gobject_id = 0
|
||||
THEN g.area_gobject_id
|
||||
ELSE g.contained_by_gobject_id
|
||||
END AS parent_gobject_id,
|
||||
CASE WHEN td.category_id = 13
|
||||
THEN 1
|
||||
ELSE 0
|
||||
END AS is_area,
|
||||
td.category_id AS category_id,
|
||||
g.hosted_by_gobject_id AS hosted_by_gobject_id,
|
||||
ISNULL(
|
||||
STUFF((
|
||||
SELECT '|' + tc.template_tag_name
|
||||
FROM template_chain tc
|
||||
WHERE tc.instance_gobject_id = g.gobject_id
|
||||
ORDER BY tc.depth
|
||||
FOR XML PATH('')
|
||||
), 1, 1, ''),
|
||||
''
|
||||
) AS template_chain
|
||||
FROM gobject g
|
||||
INNER JOIN template_definition td
|
||||
ON g.template_definition_id = td.template_definition_id
|
||||
WHERE td.category_id IN (1, 3, 4, 10, 11, 13, 17, 24, 26)
|
||||
AND g.is_template = 0
|
||||
AND (g.deployed_package_id <> 0 OR td.category_id = 13)
|
||||
ORDER BY parent_gobject_id, g.tag_name";
|
||||
|
||||
// Unlike HierarchySql, this query has diverged from the OtOpcUa original. It returns two
|
||||
// kinds of attribute: user-configured dynamic attributes (the original `dynamic_attribute`
|
||||
// body, src_pri 0) and the built-in attributes every object inherits from its primitives
|
||||
// (`attribute_definition` joined through `primitive_instance`, src_pri 1). Built-in
|
||||
// attributes are why engine/platform objects and extension sub-attributes such as
|
||||
// `TestAlarm001.Acked` show up at all. Built-in rows carry no category filter (the
|
||||
// `attribute_definition` category numbering differs from `dynamic_attribute`'s — only the
|
||||
// `_`-prefix and `.Description` name exclusions apply) and are never flagged
|
||||
// `is_historized`/`is_alarm`: those flags describe a user attribute that anchors an
|
||||
// extension, not the extension's machinery leaves.
|
||||
private const string AttributesSql = @"
|
||||
;WITH deployed_package_chain AS (
|
||||
SELECT g.gobject_id, p.package_id, p.derived_from_package_id, 0 AS depth
|
||||
FROM gobject g
|
||||
INNER JOIN package p ON p.package_id = g.deployed_package_id
|
||||
WHERE g.is_template = 0 AND g.deployed_package_id <> 0
|
||||
UNION ALL
|
||||
SELECT dpc.gobject_id, p.package_id, p.derived_from_package_id, dpc.depth + 1
|
||||
FROM deployed_package_chain dpc
|
||||
INNER JOIN package p ON p.package_id = dpc.derived_from_package_id
|
||||
WHERE dpc.derived_from_package_id <> 0 AND dpc.depth < 10
|
||||
),
|
||||
candidate AS (
|
||||
SELECT
|
||||
dpc.gobject_id, g.tag_name, da.attribute_name, da.mx_data_type, da.is_array,
|
||||
CASE WHEN da.is_array = 1
|
||||
THEN CONVERT(int, CONVERT(varbinary(2),
|
||||
SUBSTRING(da.mx_value, 15, 2) + SUBSTRING(da.mx_value, 13, 2), 2))
|
||||
ELSE NULL END AS array_dimension,
|
||||
da.mx_attribute_category, da.security_classification, dpc.depth, 0 AS src_pri
|
||||
FROM deployed_package_chain dpc
|
||||
INNER JOIN dynamic_attribute da ON da.package_id = dpc.package_id
|
||||
INNER JOIN gobject g ON g.gobject_id = dpc.gobject_id
|
||||
INNER JOIN template_definition td ON td.template_definition_id = g.template_definition_id
|
||||
WHERE td.category_id IN (1, 3, 4, 10, 11, 13, 17, 24, 26)
|
||||
AND da.attribute_name NOT LIKE '[_]%'
|
||||
AND da.attribute_name NOT LIKE '%.Description'
|
||||
AND da.mx_attribute_category IN (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 24)
|
||||
UNION ALL
|
||||
SELECT
|
||||
dpc.gobject_id, g.tag_name,
|
||||
CASE WHEN pi.primitive_name IS NULL OR pi.primitive_name = ''
|
||||
THEN ad.attribute_name
|
||||
ELSE pi.primitive_name + '.' + ad.attribute_name END AS attribute_name,
|
||||
ad.mx_data_type, ad.is_array,
|
||||
CASE WHEN ad.is_array = 1
|
||||
THEN CONVERT(int, CONVERT(varbinary(2),
|
||||
SUBSTRING(ad.mx_value, 15, 2) + SUBSTRING(ad.mx_value, 13, 2), 2))
|
||||
ELSE NULL END AS array_dimension,
|
||||
ad.mx_attribute_category, ad.security_classification, dpc.depth, 1 AS src_pri
|
||||
FROM deployed_package_chain dpc
|
||||
INNER JOIN primitive_instance pi ON pi.package_id = dpc.package_id
|
||||
INNER JOIN attribute_definition ad ON ad.primitive_definition_id = pi.primitive_definition_id
|
||||
INNER JOIN gobject g ON g.gobject_id = dpc.gobject_id
|
||||
INNER JOIN template_definition td ON td.template_definition_id = g.template_definition_id
|
||||
WHERE td.category_id IN (1, 3, 4, 10, 11, 13, 17, 24, 26)
|
||||
AND ad.attribute_name NOT LIKE '[_]%'
|
||||
AND ad.attribute_name NOT LIKE '%.Description'
|
||||
),
|
||||
ranked AS (
|
||||
SELECT c.*, ROW_NUMBER() OVER (
|
||||
PARTITION BY c.gobject_id, c.attribute_name ORDER BY c.src_pri, c.depth) AS rn
|
||||
FROM candidate c
|
||||
)
|
||||
SELECT
|
||||
r.gobject_id, r.tag_name, r.attribute_name,
|
||||
r.tag_name + '.' + r.attribute_name
|
||||
+ CASE WHEN r.is_array = 1 THEN '[]' ELSE '' END AS full_tag_reference,
|
||||
r.mx_data_type, dt.description AS data_type_name, r.is_array, r.array_dimension,
|
||||
r.mx_attribute_category, r.security_classification,
|
||||
CASE WHEN r.src_pri = 0 AND EXISTS (
|
||||
SELECT 1 FROM deployed_package_chain dpc2
|
||||
INNER JOIN primitive_instance pi ON pi.package_id = dpc2.package_id AND pi.primitive_name = r.attribute_name
|
||||
INNER JOIN primitive_definition pd ON pd.primitive_definition_id = pi.primitive_definition_id AND pd.primitive_name = 'HistoryExtension'
|
||||
WHERE dpc2.gobject_id = r.gobject_id
|
||||
) THEN 1 ELSE 0 END AS is_historized,
|
||||
CASE WHEN r.src_pri = 0 AND EXISTS (
|
||||
SELECT 1 FROM deployed_package_chain dpc2
|
||||
INNER JOIN primitive_instance pi ON pi.package_id = dpc2.package_id AND pi.primitive_name = r.attribute_name
|
||||
INNER JOIN primitive_definition pd ON pd.primitive_definition_id = pi.primitive_definition_id AND pd.primitive_name = 'AlarmExtension'
|
||||
WHERE dpc2.gobject_id = r.gobject_id
|
||||
) THEN 1 ELSE 0 END AS is_alarm
|
||||
FROM ranked r
|
||||
LEFT JOIN data_type dt ON dt.mx_data_type = r.mx_data_type
|
||||
WHERE r.rn = 1
|
||||
ORDER BY r.tag_name, r.attribute_name";
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
namespace ZB.MOM.WW.GalaxyRepository;
|
||||
|
||||
/// <summary>
|
||||
/// Connection settings for the AVEVA System Platform Galaxy Repository database.
|
||||
/// <para>
|
||||
/// <see cref="SectionName"/> is a generic default; the DI extension accepts an explicit
|
||||
/// configuration section path so a consumer can bind from its own section (e.g.
|
||||
/// <c>HistorianGateway:Galaxy</c>).
|
||||
/// </para>
|
||||
/// </summary>
|
||||
public sealed class GalaxyRepositoryOptions
|
||||
{
|
||||
/// <summary>
|
||||
/// Generic default configuration section name. The DI extension accepts an explicit
|
||||
/// section path, so a consumer may bind from a different section (e.g.
|
||||
/// <c>HistorianGateway:Galaxy</c>).
|
||||
/// </summary>
|
||||
public const string SectionName = "GalaxyRepository";
|
||||
|
||||
/// <summary>
|
||||
/// Default SQL Server connection string for the Galaxy Repository database.
|
||||
/// Single source of truth shared with the integration-test fallback so the
|
||||
/// production default and the live-test default cannot drift.
|
||||
/// </summary>
|
||||
public const string DefaultConnectionString =
|
||||
"Server=localhost;Database=ZB;Integrated Security=True;TrustServerCertificate=True;Encrypt=False;";
|
||||
|
||||
/// <summary>The SQL Server connection string for the Galaxy Repository database.</summary>
|
||||
public string ConnectionString { get; init; } = DefaultConnectionString;
|
||||
|
||||
/// <summary>The timeout in seconds for SQL commands executed against the Galaxy Repository.</summary>
|
||||
public int CommandTimeoutSeconds { get; init; } = 60;
|
||||
|
||||
/// <summary>
|
||||
/// Interval (seconds) between background refreshes of the dashboard Galaxy summary
|
||||
/// cache. SQL is hit at most once per interval regardless of dashboard render rate.
|
||||
/// </summary>
|
||||
public int DashboardRefreshIntervalSeconds { get; init; } = 30;
|
||||
|
||||
/// <summary>
|
||||
/// Whether the latest successful Galaxy browse dataset is persisted to disk. When
|
||||
/// enabled, the cache reloads that snapshot at startup so clients can still browse
|
||||
/// last-known data while the Galaxy database is unreachable.
|
||||
/// </summary>
|
||||
public bool PersistSnapshot { get; init; } = true;
|
||||
|
||||
/// <summary>
|
||||
/// File path for the persisted Galaxy browse snapshot. Ignored when
|
||||
/// <see cref="PersistSnapshot"/> is <see langword="false"/>. There is no built-in
|
||||
/// default path — the consumer supplies a cross-platform-friendly path appropriate to
|
||||
/// its host. When left empty and <see cref="PersistSnapshot"/> is enabled, the
|
||||
/// snapshot store (a later task) decides where to write.
|
||||
/// </summary>
|
||||
public string SnapshotCachePath { get; init; } = string.Empty;
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
using ZB.MOM.WW.GalaxyRepository.Grpc;
|
||||
|
||||
namespace ZB.MOM.WW.GalaxyRepository;
|
||||
|
||||
/// <summary>
|
||||
/// Resolution result for a tag address: the owning <see cref="GalaxyObject"/>, the
|
||||
/// specific <see cref="GalaxyAttribute"/> when the address names an attribute (otherwise
|
||||
/// <see langword="null"/>), and the object's contained path.
|
||||
/// </summary>
|
||||
/// <param name="Object">The Galaxy object that owns the looked-up address.</param>
|
||||
/// <param name="Attribute">The matched attribute, or <see langword="null"/> when the address names an object.</param>
|
||||
/// <param name="ContainedPath">The owning object's contained path.</param>
|
||||
public sealed record GalaxyTagLookup(
|
||||
GalaxyObject Object,
|
||||
GalaxyAttribute? Attribute,
|
||||
string ContainedPath);
|
||||
+329
@@ -0,0 +1,329 @@
|
||||
using Google.Protobuf.WellKnownTypes;
|
||||
using Grpc.Core;
|
||||
using ProtoGalaxyRepository = ZB.MOM.WW.GalaxyRepository.Grpc.GalaxyRepository;
|
||||
|
||||
namespace ZB.MOM.WW.GalaxyRepository.Grpc;
|
||||
|
||||
/// <summary>
|
||||
/// Reusable gRPC surface that exposes the Galaxy Repository to clients. Hosted by any
|
||||
/// consuming gateway (e.g. MxAccessGateway or the HistorianGateway sidecar) via
|
||||
/// <see cref="DependencyInjection.GalaxyRepositoryServiceCollectionExtensions.MapZbGalaxyRepository"/>.
|
||||
/// <para>
|
||||
/// <c>DiscoverHierarchy</c> and <c>GetLastDeployTime</c> serve from
|
||||
/// <see cref="IGalaxyHierarchyCache"/> so many clients share a single SQL pull.
|
||||
/// <c>WatchDeployEvents</c> streams events from <see cref="IGalaxyDeployNotifier"/>.
|
||||
/// <c>TestConnection</c> remains a direct SQL probe since callers use it as a health check.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// This service applies <b>no</b> per-identity browse-subtree filtering — the full
|
||||
/// hierarchy is projected (<c>null</c> subtree globs). Authorization (including any
|
||||
/// subtree scoping) is the responsibility of the hosting gateway's interceptor layer.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
/// <param name="repository">Direct SQL surface used by <c>TestConnection</c>.</param>
|
||||
/// <param name="cache">Shared hierarchy cache that <c>DiscoverHierarchy</c>/<c>BrowseChildren</c>/<c>GetLastDeployTime</c> serve from.</param>
|
||||
/// <param name="notifier">Deploy-event source streamed by <c>WatchDeployEvents</c>.</param>
|
||||
public sealed class GalaxyRepositoryGrpcService(
|
||||
IGalaxyRepository repository,
|
||||
IGalaxyHierarchyCache cache,
|
||||
IGalaxyDeployNotifier notifier) : ProtoGalaxyRepository.GalaxyRepositoryBase
|
||||
{
|
||||
private static readonly TimeSpan FirstLoadWaitBudget = TimeSpan.FromSeconds(5);
|
||||
private const int DefaultDiscoverPageSize = 1000;
|
||||
private const int MaxDiscoverPageSize = 5000;
|
||||
private const int DefaultBrowsePageSize = 500;
|
||||
// MaxBrowsePageSize reuses MaxDiscoverPageSize (5000) — same cap.
|
||||
|
||||
/// <inheritdoc />
|
||||
public override async Task<TestConnectionReply> TestConnection(
|
||||
TestConnectionRequest request,
|
||||
ServerCallContext context)
|
||||
{
|
||||
bool ok = await repository.TestConnectionAsync(context.CancellationToken).ConfigureAwait(false);
|
||||
return new TestConnectionReply { Ok = ok };
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
public override async Task<GetLastDeployTimeReply> GetLastDeployTime(
|
||||
GetLastDeployTimeRequest request,
|
||||
ServerCallContext context)
|
||||
{
|
||||
await WaitForCacheBootstrap(context.CancellationToken).ConfigureAwait(false);
|
||||
GalaxyHierarchyCacheEntry entry = cache.Current;
|
||||
|
||||
if (!entry.HasData)
|
||||
{
|
||||
throw new RpcException(new Status(
|
||||
StatusCode.Unavailable,
|
||||
ResolveUnavailableMessage(entry)));
|
||||
}
|
||||
|
||||
GetLastDeployTimeReply reply = new() { Present = entry.LastDeployTime.HasValue };
|
||||
if (entry.LastDeployTime.HasValue)
|
||||
{
|
||||
reply.TimeOfLastDeploy = Timestamp.FromDateTimeOffset(entry.LastDeployTime.Value);
|
||||
}
|
||||
return reply;
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
public override async Task<DiscoverHierarchyReply> DiscoverHierarchy(
|
||||
DiscoverHierarchyRequest request,
|
||||
ServerCallContext context)
|
||||
{
|
||||
await WaitForCacheBootstrap(context.CancellationToken).ConfigureAwait(false);
|
||||
GalaxyHierarchyCacheEntry entry = cache.Current;
|
||||
|
||||
if (!entry.HasData)
|
||||
{
|
||||
throw new RpcException(new Status(
|
||||
StatusCode.Unavailable,
|
||||
ResolveUnavailableMessage(entry)));
|
||||
}
|
||||
|
||||
int pageSize = ResolvePageSize(request.PageSize);
|
||||
// The shared library applies no per-identity subtree scoping; the hosting
|
||||
// gateway enforces authorization at its interceptor layer.
|
||||
string filterSignature = GalaxyHierarchyProjector.ComputeFilterSignature(request, browseSubtreeGlobs: null);
|
||||
PageToken pageToken = ParsePageToken(request.PageToken, entry.Sequence, filterSignature);
|
||||
GalaxyHierarchyQueryResult query = GalaxyHierarchyProjector.Project(
|
||||
entry,
|
||||
request,
|
||||
browseSubtreeGlobs: null,
|
||||
pageToken.Offset,
|
||||
pageSize);
|
||||
int offset = pageToken.Offset;
|
||||
if (offset > query.TotalObjectCount)
|
||||
{
|
||||
throw new RpcException(new Status(
|
||||
StatusCode.InvalidArgument,
|
||||
"DiscoverHierarchy page_token is outside the current hierarchy."));
|
||||
}
|
||||
|
||||
DiscoverHierarchyReply reply = new()
|
||||
{
|
||||
TotalObjectCount = query.TotalObjectCount,
|
||||
};
|
||||
reply.Objects.Add(query.Objects);
|
||||
|
||||
int nextOffset = offset + query.Objects.Count;
|
||||
if (nextOffset < query.TotalObjectCount)
|
||||
{
|
||||
reply.NextPageToken = FormatPageToken(entry.Sequence, query.FilterSignature, nextOffset);
|
||||
}
|
||||
|
||||
return reply;
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
public override async Task<BrowseChildrenReply> BrowseChildren(
|
||||
BrowseChildrenRequest request,
|
||||
ServerCallContext context)
|
||||
{
|
||||
await WaitForCacheBootstrap(context.CancellationToken).ConfigureAwait(false);
|
||||
GalaxyHierarchyCacheEntry entry = cache.Current;
|
||||
|
||||
if (!entry.HasData)
|
||||
{
|
||||
throw new RpcException(new Status(
|
||||
StatusCode.Unavailable,
|
||||
ResolveUnavailableMessage(entry)));
|
||||
}
|
||||
|
||||
int pageSize = ResolveBrowsePageSize(request.PageSize);
|
||||
|
||||
// Resolve the parent id once so the page-token signature can include it
|
||||
// and the projector sees the same resolved id when memoizing. The projector
|
||||
// re-resolves internally; with the by-name/by-path indexes on
|
||||
// GalaxyHierarchyIndex that second call is O(1), so the redundancy is cheap
|
||||
// and keeps the projector self-contained.
|
||||
int parentId = GalaxyBrowseProjector.ResolveParentId(entry, request);
|
||||
string filterSignature = GalaxyBrowseProjector.ComputeFilterSignature(
|
||||
request, browseSubtreeGlobs: null, parentId);
|
||||
PageToken pageToken = ParsePageToken(request.PageToken, entry.Sequence, filterSignature);
|
||||
|
||||
GalaxyBrowseChildrenResult result = GalaxyBrowseProjector.ProjectChildren(
|
||||
entry,
|
||||
request,
|
||||
browseSubtreeGlobs: null,
|
||||
pageToken.Offset,
|
||||
pageSize);
|
||||
|
||||
if (pageToken.Offset > result.TotalChildCount)
|
||||
{
|
||||
throw new RpcException(new Status(
|
||||
StatusCode.InvalidArgument,
|
||||
"BrowseChildren page_token is outside the current children set."));
|
||||
}
|
||||
|
||||
BrowseChildrenReply reply = new()
|
||||
{
|
||||
TotalChildCount = result.TotalChildCount,
|
||||
CacheSequence = (ulong)entry.Sequence,
|
||||
};
|
||||
reply.Children.Add(result.Children);
|
||||
reply.ChildHasChildren.Add(result.ChildHasChildren);
|
||||
|
||||
int nextOffset = pageToken.Offset + result.Children.Count;
|
||||
if (nextOffset < result.TotalChildCount)
|
||||
{
|
||||
reply.NextPageToken = FormatPageToken(entry.Sequence, result.FilterSignature, nextOffset);
|
||||
}
|
||||
|
||||
return reply;
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
public override async Task WatchDeployEvents(
|
||||
WatchDeployEventsRequest request,
|
||||
IServerStreamWriter<DeployEvent> responseStream,
|
||||
ServerCallContext context)
|
||||
{
|
||||
DateTimeOffset? lastSeen = request.LastSeenDeployTime?.ToDateTimeOffset();
|
||||
|
||||
await foreach (GalaxyDeployEventInfo info in notifier
|
||||
.SubscribeAsync(context.CancellationToken)
|
||||
.ConfigureAwait(false))
|
||||
{
|
||||
// Suppress the initial bootstrap event when the client already knows about
|
||||
// this deploy time. We only suppress the first one — subsequent events fire
|
||||
// on actual changes, so they always pass.
|
||||
if (lastSeen is { } seen && info.TimeOfLastDeploy == seen)
|
||||
{
|
||||
lastSeen = null;
|
||||
continue;
|
||||
}
|
||||
lastSeen = null;
|
||||
|
||||
await responseStream.WriteAsync(MapDeployEvent(info), context.CancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
}
|
||||
|
||||
private async Task WaitForCacheBootstrap(CancellationToken cancellationToken)
|
||||
{
|
||||
if (cache.Current.HasData || cache.Current.Status == GalaxyCacheStatus.Unavailable)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
using CancellationTokenSource budget = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
|
||||
budget.CancelAfter(FirstLoadWaitBudget);
|
||||
try
|
||||
{
|
||||
await cache.WaitForFirstLoadAsync(budget.Token).ConfigureAwait(false);
|
||||
}
|
||||
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
|
||||
{
|
||||
throw;
|
||||
}
|
||||
catch (OperationCanceledException)
|
||||
{
|
||||
// Budget elapsed; fall through and let the caller see the current
|
||||
// (possibly Unknown/Unavailable) entry.
|
||||
}
|
||||
}
|
||||
|
||||
private static DeployEvent MapDeployEvent(GalaxyDeployEventInfo info)
|
||||
{
|
||||
DeployEvent ev = new()
|
||||
{
|
||||
Sequence = (ulong)info.Sequence,
|
||||
ObservedAt = Timestamp.FromDateTimeOffset(info.ObservedAt),
|
||||
ObjectCount = info.ObjectCount,
|
||||
AttributeCount = info.AttributeCount,
|
||||
TimeOfLastDeployPresent = info.TimeOfLastDeploy.HasValue,
|
||||
};
|
||||
if (info.TimeOfLastDeploy.HasValue)
|
||||
{
|
||||
ev.TimeOfLastDeploy = Timestamp.FromDateTimeOffset(info.TimeOfLastDeploy.Value);
|
||||
}
|
||||
return ev;
|
||||
}
|
||||
|
||||
private static string ResolveUnavailableMessage(GalaxyHierarchyCacheEntry entry) => entry.Status switch
|
||||
{
|
||||
GalaxyCacheStatus.Unknown => "Galaxy cache has not completed its initial load yet.",
|
||||
GalaxyCacheStatus.Unavailable => "Galaxy repository is unavailable.",
|
||||
_ => "Galaxy cache has no data available.",
|
||||
};
|
||||
|
||||
private static int ResolvePageSize(int requestedPageSize)
|
||||
{
|
||||
if (requestedPageSize < 0)
|
||||
{
|
||||
throw new RpcException(new Status(
|
||||
StatusCode.InvalidArgument,
|
||||
"DiscoverHierarchy page_size must be greater than zero when provided."));
|
||||
}
|
||||
|
||||
int pageSize = requestedPageSize == 0 ? DefaultDiscoverPageSize : requestedPageSize;
|
||||
return Math.Min(pageSize, MaxDiscoverPageSize);
|
||||
}
|
||||
|
||||
private static int ResolveBrowsePageSize(int requested)
|
||||
{
|
||||
if (requested < 0)
|
||||
{
|
||||
throw new RpcException(new Status(
|
||||
StatusCode.InvalidArgument,
|
||||
"BrowseChildren page_size must be greater than zero when provided."));
|
||||
}
|
||||
int pageSize = requested == 0 ? DefaultBrowsePageSize : requested;
|
||||
return Math.Min(pageSize, MaxDiscoverPageSize);
|
||||
}
|
||||
|
||||
private static string FormatPageToken(long sequence, string filterSignature, int offset)
|
||||
{
|
||||
return string.Concat(
|
||||
sequence.ToString(System.Globalization.CultureInfo.InvariantCulture),
|
||||
":",
|
||||
filterSignature,
|
||||
":",
|
||||
offset.ToString(System.Globalization.CultureInfo.InvariantCulture));
|
||||
}
|
||||
|
||||
private static PageToken ParsePageToken(string pageToken, long currentSequence, string currentFilterSignature)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(pageToken))
|
||||
{
|
||||
return new PageToken(currentSequence, currentFilterSignature, Offset: 0);
|
||||
}
|
||||
|
||||
string[] parts = pageToken.Split(':', count: 3);
|
||||
if (parts.Length != 3
|
||||
|| !long.TryParse(
|
||||
parts[0],
|
||||
System.Globalization.NumberStyles.None,
|
||||
System.Globalization.CultureInfo.InvariantCulture,
|
||||
out long sequence)
|
||||
|| !int.TryParse(
|
||||
parts[2],
|
||||
System.Globalization.NumberStyles.None,
|
||||
System.Globalization.CultureInfo.InvariantCulture,
|
||||
out int offset)
|
||||
|| offset < 0)
|
||||
{
|
||||
throw new RpcException(new Status(
|
||||
StatusCode.InvalidArgument,
|
||||
"page_token is invalid."));
|
||||
}
|
||||
|
||||
if (sequence != currentSequence)
|
||||
{
|
||||
throw new RpcException(new Status(
|
||||
StatusCode.InvalidArgument,
|
||||
"page_token is stale."));
|
||||
}
|
||||
|
||||
if (!string.Equals(parts[1], currentFilterSignature, StringComparison.Ordinal))
|
||||
{
|
||||
throw new RpcException(new Status(
|
||||
StatusCode.InvalidArgument,
|
||||
"page_token does not match the current filters."));
|
||||
}
|
||||
|
||||
return new PageToken(sequence, parts[1], offset);
|
||||
}
|
||||
|
||||
private sealed record PageToken(long Sequence, string FilterSignature, int Offset);
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
namespace ZB.MOM.WW.GalaxyRepository;
|
||||
|
||||
/// <summary>Publishes Galaxy repository deploy events to subscribers.</summary>
|
||||
public interface IGalaxyDeployNotifier
|
||||
{
|
||||
/// <summary>The most recently published event, or null if no event has fired yet.</summary>
|
||||
GalaxyDeployEventInfo? Latest { get; }
|
||||
|
||||
/// <summary>Publishes a deploy event to all current subscribers and stores it as Latest.</summary>
|
||||
/// <param name="info">The deploy event to publish.</param>
|
||||
void Publish(GalaxyDeployEventInfo info);
|
||||
|
||||
/// <summary>Subscribes to deploy events. The sequence yields the latest event first (if available) then streams new events as they fire.</summary>
|
||||
/// <param name="cancellationToken">Token to cancel the asynchronous operation.</param>
|
||||
/// <returns>Async enumerable of deploy events.</returns>
|
||||
IAsyncEnumerable<GalaxyDeployEventInfo> SubscribeAsync(CancellationToken cancellationToken);
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
namespace ZB.MOM.WW.GalaxyRepository;
|
||||
|
||||
/// <summary>Cache for Galaxy Repository hierarchy data.</summary>
|
||||
public interface IGalaxyHierarchyCache
|
||||
{
|
||||
/// <summary>The latest cache entry. Status freshness is recomputed against the clock.</summary>
|
||||
GalaxyHierarchyCacheEntry Current { get; }
|
||||
|
||||
/// <summary>
|
||||
/// Forces a refresh against the Galaxy Repository. Performs a cheap
|
||||
/// <c>time_of_last_deploy</c> probe first and only re-queries the heavy hierarchy +
|
||||
/// attributes rowsets when the deploy time has changed since the last successful
|
||||
/// refresh.
|
||||
/// </summary>
|
||||
/// <param name="cancellationToken">Token to cancel the asynchronous operation.</param>
|
||||
Task RefreshAsync(CancellationToken cancellationToken);
|
||||
|
||||
/// <summary>
|
||||
/// Awaits the first completed refresh attempt (success or failure). Useful for
|
||||
/// gRPC handlers that want to serve from cache without returning Unavailable on the
|
||||
/// very first request after the service starts.
|
||||
/// </summary>
|
||||
/// <param name="cancellationToken">Token to cancel the asynchronous operation.</param>
|
||||
Task WaitForFirstLoadAsync(CancellationToken cancellationToken);
|
||||
}
|
||||
+28
@@ -0,0 +1,28 @@
|
||||
namespace ZB.MOM.WW.GalaxyRepository;
|
||||
|
||||
/// <summary>
|
||||
/// Persists the latest Galaxy Repository browse dataset to disk and reloads
|
||||
/// it at startup. Lets <see cref="GalaxyHierarchyCache"/> serve last-known
|
||||
/// browse data when the Galaxy database is unreachable on a cold start.
|
||||
/// </summary>
|
||||
public interface IGalaxyHierarchySnapshotStore
|
||||
{
|
||||
/// <summary>
|
||||
/// Writes <paramref name="snapshot"/> to disk, replacing any previous
|
||||
/// snapshot atomically. A no-op when snapshot persistence is disabled.
|
||||
/// </summary>
|
||||
/// <param name="snapshot">The browse dataset to persist.</param>
|
||||
/// <param name="cancellationToken">Token to cancel the asynchronous operation.</param>
|
||||
Task SaveAsync(GalaxyHierarchySnapshot snapshot, CancellationToken cancellationToken);
|
||||
|
||||
/// <summary>
|
||||
/// Reads the persisted Galaxy browse dataset.
|
||||
/// </summary>
|
||||
/// <param name="cancellationToken">Token to cancel the asynchronous operation.</param>
|
||||
/// <returns>
|
||||
/// The persisted snapshot, or <see langword="null"/> when none exists,
|
||||
/// persistence is disabled, or the on-disk file uses an unrecognized
|
||||
/// schema version.
|
||||
/// </returns>
|
||||
Task<GalaxyHierarchySnapshot?> TryLoadAsync(CancellationToken cancellationToken);
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
namespace ZB.MOM.WW.GalaxyRepository;
|
||||
|
||||
/// <summary>
|
||||
/// Abstraction over <see cref="GalaxyRepository"/>: the read-only SQL surface over the
|
||||
/// AVEVA System Platform Galaxy Repository database. Exists so consumers (and the cache
|
||||
/// layer, a later task) can be unit-tested against an in-memory fake without standing up a
|
||||
/// real <c>Microsoft.Data.SqlClient</c> <c>SqlConnection</c> against a bogus host/port.
|
||||
/// </summary>
|
||||
public interface IGalaxyRepository
|
||||
{
|
||||
/// <summary>Tests the connection to the Galaxy Repository database.</summary>
|
||||
/// <param name="ct">Token to cancel the asynchronous operation.</param>
|
||||
Task<bool> TestConnectionAsync(CancellationToken ct = default);
|
||||
|
||||
/// <summary>Retrieves the last deployment time from the Galaxy Repository.</summary>
|
||||
/// <param name="ct">Token to cancel the asynchronous operation.</param>
|
||||
Task<DateTime?> GetLastDeployTimeAsync(CancellationToken ct = default);
|
||||
|
||||
/// <summary>Retrieves the complete hierarchy of Galaxy objects from the repository.</summary>
|
||||
/// <param name="ct">Token to cancel the asynchronous operation.</param>
|
||||
Task<List<GalaxyHierarchyRow>> GetHierarchyAsync(CancellationToken ct = default);
|
||||
|
||||
/// <summary>Retrieves all attributes for Galaxy objects from the repository.</summary>
|
||||
/// <param name="ct">Token to cancel the asynchronous operation.</param>
|
||||
Task<List<GalaxyAttributeRow>> GetAttributesAsync(CancellationToken ct = default);
|
||||
}
|
||||
+190
@@ -0,0 +1,190 @@
|
||||
syntax = "proto3";
|
||||
|
||||
package galaxy_repository.v1;
|
||||
|
||||
option csharp_namespace = "ZB.MOM.WW.GalaxyRepository.Grpc";
|
||||
|
||||
import "google/protobuf/timestamp.proto";
|
||||
import "google/protobuf/wrappers.proto";
|
||||
|
||||
// Wire-compatibility policy (ProtobufStyleGuide): this contract evolves
|
||||
// additively only. Never renumber or repurpose an existing field number or
|
||||
// enum value. When a field or enum value is removed, add a `reserved` range
|
||||
// (and `reserved` name) covering it in the same change so a future editor
|
||||
// cannot accidentally reuse the retired tag. There are no `reserved`
|
||||
// declarations today because no field or enum value has ever been removed.
|
||||
|
||||
// Read-only browse over the AVEVA System Platform Galaxy Repository (ZB SQL
|
||||
// database). Lets clients enumerate the deployed object hierarchy and each
|
||||
// object's dynamic attributes so they know what tag references to subscribe
|
||||
// to via the MxAccessGateway service.
|
||||
service GalaxyRepository {
|
||||
rpc TestConnection(TestConnectionRequest) returns (TestConnectionReply);
|
||||
rpc GetLastDeployTime(GetLastDeployTimeRequest) returns (GetLastDeployTimeReply);
|
||||
rpc DiscoverHierarchy(DiscoverHierarchyRequest) returns (DiscoverHierarchyReply);
|
||||
|
||||
// Server-stream of deploy events. The server emits the current state immediately
|
||||
// on subscribe (so clients can bootstrap their cache without waiting for the next
|
||||
// deploy), then emits one event each time the gateway's hierarchy cache observes
|
||||
// a new galaxy.time_of_last_deploy. The sequence field is monotonically
|
||||
// increasing per server start; gaps indicate the per-subscriber buffer dropped
|
||||
// older events because the client was too slow.
|
||||
rpc WatchDeployEvents(WatchDeployEventsRequest) returns (stream DeployEvent);
|
||||
|
||||
// Returns the direct children of a parent object (or the root objects when
|
||||
// `parent` is unset). Designed for OPC UA-style lazy expand: clients walk
|
||||
// one level at a time instead of paging the full hierarchy. Filters mirror
|
||||
// DiscoverHierarchy exactly. Backed by the same shared hierarchy cache.
|
||||
rpc BrowseChildren(BrowseChildrenRequest) returns (BrowseChildrenReply);
|
||||
}
|
||||
|
||||
message TestConnectionRequest {}
|
||||
|
||||
message TestConnectionReply {
|
||||
bool ok = 1;
|
||||
}
|
||||
|
||||
message GetLastDeployTimeRequest {}
|
||||
|
||||
message GetLastDeployTimeReply {
|
||||
bool present = 1;
|
||||
google.protobuf.Timestamp time_of_last_deploy = 2;
|
||||
}
|
||||
|
||||
message DiscoverHierarchyRequest {
|
||||
// Maximum number of objects to return. The server applies its default when
|
||||
// unset and rejects non-positive values.
|
||||
int32 page_size = 1;
|
||||
// Opaque token returned by a previous DiscoverHierarchy response.
|
||||
string page_token = 2;
|
||||
// Optional. When set, return only this object and its descendants.
|
||||
// Empty = full hierarchy.
|
||||
oneof root {
|
||||
int32 root_gobject_id = 3;
|
||||
string root_tag_name = 4;
|
||||
string root_contained_path = 5;
|
||||
}
|
||||
// Optional. Cap on descendant depth from root. Zero returns only the root.
|
||||
// Unset means unlimited depth.
|
||||
google.protobuf.Int32Value max_depth = 6;
|
||||
// Optional object category id filters.
|
||||
repeated int32 category_ids = 7;
|
||||
// Optional case-insensitive substring filters against template names.
|
||||
repeated string template_chain_contains = 8;
|
||||
// Optional anchored, case-insensitive glob over object tag_name.
|
||||
string tag_name_glob = 9;
|
||||
// Optional. Unset or true includes attributes. False returns object skeletons.
|
||||
optional bool include_attributes = 10;
|
||||
// Optional. Return only objects with at least one alarm-bearing attribute.
|
||||
bool alarm_bearing_only = 11;
|
||||
// Optional. Return only objects with at least one historized attribute.
|
||||
bool historized_only = 12;
|
||||
}
|
||||
|
||||
message DiscoverHierarchyReply {
|
||||
repeated GalaxyObject objects = 1;
|
||||
// Non-empty when another page is available.
|
||||
string next_page_token = 2;
|
||||
// Total number of objects in the cached hierarchy at the time of the call.
|
||||
int32 total_object_count = 3;
|
||||
}
|
||||
|
||||
message WatchDeployEventsRequest {
|
||||
// Optional. When set, the bootstrap event is suppressed if the cached deploy
|
||||
// time matches this value. Future events are still emitted normally.
|
||||
google.protobuf.Timestamp last_seen_deploy_time = 1;
|
||||
}
|
||||
|
||||
message DeployEvent {
|
||||
// Monotonically increasing per server start. Gaps indicate dropped events.
|
||||
uint64 sequence = 1;
|
||||
// Server wall-clock when the cache observed the deploy.
|
||||
google.protobuf.Timestamp observed_at = 2;
|
||||
// Galaxy.time_of_last_deploy. Absent only when the Galaxy table reports null.
|
||||
google.protobuf.Timestamp time_of_last_deploy = 3;
|
||||
bool time_of_last_deploy_present = 4;
|
||||
int32 object_count = 5;
|
||||
int32 attribute_count = 6;
|
||||
}
|
||||
|
||||
message GalaxyObject {
|
||||
int32 gobject_id = 1;
|
||||
string tag_name = 2;
|
||||
string contained_name = 3;
|
||||
string browse_name = 4;
|
||||
int32 parent_gobject_id = 5;
|
||||
bool is_area = 6;
|
||||
int32 category_id = 7;
|
||||
int32 hosted_by_gobject_id = 8;
|
||||
repeated string template_chain = 9;
|
||||
repeated GalaxyAttribute attributes = 10;
|
||||
}
|
||||
|
||||
message GalaxyAttribute {
|
||||
string attribute_name = 1;
|
||||
string full_tag_reference = 2;
|
||||
// Raw Galaxy SQL `dbo.data_type` identifier, passed through unchanged.
|
||||
// This is NOT a member of `mxaccess_gateway.v1.MxDataType` — Galaxy's
|
||||
// type enumeration is distinct from MXAccess's wire data-type enum and
|
||||
// the two must not be cast or compared. The GalaxyRepository service is
|
||||
// metadata-only and deliberately does not share types with
|
||||
// mxaccess_gateway.proto. See docs/GalaxyRepository.md.
|
||||
int32 mx_data_type = 3;
|
||||
// Human-readable name from Galaxy's `dbo.data_type` table (e.g. "Float",
|
||||
// "Integer", "Boolean"). Free-form Galaxy text; not a stable enum.
|
||||
string data_type_name = 4;
|
||||
bool is_array = 5;
|
||||
int32 array_dimension = 6;
|
||||
bool array_dimension_present = 7;
|
||||
// Raw Galaxy SQL attribute-category identifier, passed through unchanged.
|
||||
// Galaxy-specific; not mapped to any gateway enum. See
|
||||
// docs/GalaxyRepository.md.
|
||||
int32 mx_attribute_category = 8;
|
||||
// Raw Galaxy SQL security-classification identifier, passed through
|
||||
// unchanged. Galaxy-specific; not mapped to any gateway enum. See
|
||||
// docs/GalaxyRepository.md.
|
||||
int32 security_classification = 9;
|
||||
bool is_historized = 10;
|
||||
bool is_alarm = 11;
|
||||
}
|
||||
|
||||
message BrowseChildrenRequest {
|
||||
// Parent selector. Empty oneof returns root objects (parent_gobject_id == 0).
|
||||
oneof parent {
|
||||
int32 parent_gobject_id = 1;
|
||||
string parent_tag_name = 2;
|
||||
string parent_contained_path = 3;
|
||||
}
|
||||
|
||||
// Maximum number of direct children to return. Server default 500; cap 5000.
|
||||
int32 page_size = 4;
|
||||
// Opaque token returned by a previous BrowseChildren response. Bound to the
|
||||
// cache sequence, parent selector, and the filter set; a mismatch returns
|
||||
// InvalidArgument.
|
||||
string page_token = 5;
|
||||
|
||||
// --- Filter parity with DiscoverHierarchy. AND-combined. ---
|
||||
repeated int32 category_ids = 6;
|
||||
repeated string template_chain_contains = 7;
|
||||
string tag_name_glob = 8;
|
||||
optional bool include_attributes = 9;
|
||||
bool alarm_bearing_only = 10;
|
||||
bool historized_only = 11;
|
||||
}
|
||||
|
||||
message BrowseChildrenReply {
|
||||
// Direct children matching the filter, sorted areas-first then by
|
||||
// case-insensitive display name (same order as the dashboard tree).
|
||||
repeated GalaxyObject children = 1;
|
||||
// Non-empty when another page of siblings is available.
|
||||
string next_page_token = 2;
|
||||
// Total matching direct children of the parent (post-filter).
|
||||
int32 total_child_count = 3;
|
||||
// Parallel array, indexed with `children`. True when the child has at least
|
||||
// one matching descendant under the same filter set. Lets a UI choose
|
||||
// whether to draw an expand triangle without an extra round trip.
|
||||
repeated bool child_has_children = 4;
|
||||
// Cache sequence this reply was projected from. Clients may pass it back as
|
||||
// part of the page_token contract. Mismatch on the next page -> InvalidArgument.
|
||||
uint64 cache_sequence = 5;
|
||||
}
|
||||
+30
@@ -0,0 +1,30 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
|
||||
<PropertyGroup>
|
||||
<IsPackable>true</IsPackable>
|
||||
<PackageId>ZB.MOM.WW.GalaxyRepository</PackageId>
|
||||
<Authors>ZB.MOM.WW</Authors>
|
||||
<Description>Read-only Galaxy object-hierarchy browse library for the ZB.MOM.WW SCADA family. Provides a SQL provider for the Galaxy Repository database and a canonical gRPC service for exposing the hierarchy to modern .NET 10 clients — extracted from MxAccessGateway so any consumer can browse the Galaxy without loading 32-bit COM.</Description>
|
||||
<PackageTags>galaxy;repository;browse;aveva;wonderware;system-platform;scada;grpc;sql;zb-mom-ww</PackageTags>
|
||||
<PackageProjectUrl>https://gitea.dohertylan.com/dohertj2/zb-mom-ww-galaxyrepository</PackageProjectUrl>
|
||||
<RepositoryUrl>https://gitea.dohertylan.com/dohertj2/zb-mom-ww-galaxyrepository</RepositoryUrl>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="Microsoft.Data.SqlClient" />
|
||||
<PackageReference Include="Grpc.AspNetCore" />
|
||||
<PackageReference Include="Google.Protobuf" />
|
||||
<PackageReference Include="Microsoft.Extensions.Hosting.Abstractions" />
|
||||
<PackageReference Include="Microsoft.Extensions.Options.ConfigurationExtensions" />
|
||||
<PackageReference Include="Grpc.Tools">
|
||||
<PrivateAssets>all</PrivateAssets>
|
||||
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
|
||||
</PackageReference>
|
||||
</ItemGroup>
|
||||
|
||||
<!-- Proto files are added in Task 2; the empty glob is intentional and builds cleanly. -->
|
||||
<ItemGroup>
|
||||
<Protobuf Include="Protos\*.proto" GrpcServices="Server" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
@@ -0,0 +1,134 @@
|
||||
using System.Runtime.CompilerServices;
|
||||
using ZB.MOM.WW.GalaxyRepository;
|
||||
|
||||
namespace ZB.MOM.WW.GalaxyRepository.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// In-memory <see cref="IGalaxyRepository"/> returning canned rowsets. Counts the heavy
|
||||
/// hierarchy/attribute reads so tests can assert deploy-gated skips, and can be flipped to
|
||||
/// throw so the failure path is exercisable.
|
||||
/// </summary>
|
||||
internal sealed class FakeGalaxyRepository : IGalaxyRepository
|
||||
{
|
||||
private readonly IReadOnlyList<GalaxyHierarchyRow> _hierarchy;
|
||||
private readonly IReadOnlyList<GalaxyAttributeRow> _attributes;
|
||||
|
||||
public FakeGalaxyRepository(
|
||||
IReadOnlyList<GalaxyHierarchyRow> hierarchy,
|
||||
IReadOnlyList<GalaxyAttributeRow> attributes,
|
||||
DateTime? deployTime)
|
||||
{
|
||||
_hierarchy = hierarchy;
|
||||
_attributes = attributes;
|
||||
DeployTime = deployTime;
|
||||
}
|
||||
|
||||
/// <summary>The deploy time returned by <see cref="GetLastDeployTimeAsync"/>; mutate to simulate a redeploy.</summary>
|
||||
public DateTime? DeployTime { get; set; }
|
||||
|
||||
/// <summary>When set, every query throws this exception (simulates an unreachable database).</summary>
|
||||
public Exception? ThrowOnQuery { get; set; }
|
||||
|
||||
public int HierarchyReadCount { get; private set; }
|
||||
|
||||
public int AttributeReadCount { get; private set; }
|
||||
|
||||
public Task<bool> TestConnectionAsync(CancellationToken ct = default) =>
|
||||
ThrowOnQuery is null ? Task.FromResult(true) : throw ThrowOnQuery;
|
||||
|
||||
public Task<DateTime?> GetLastDeployTimeAsync(CancellationToken ct = default)
|
||||
{
|
||||
if (ThrowOnQuery is not null)
|
||||
{
|
||||
throw ThrowOnQuery;
|
||||
}
|
||||
|
||||
return Task.FromResult(DeployTime);
|
||||
}
|
||||
|
||||
public Task<List<GalaxyHierarchyRow>> GetHierarchyAsync(CancellationToken ct = default)
|
||||
{
|
||||
if (ThrowOnQuery is not null)
|
||||
{
|
||||
throw ThrowOnQuery;
|
||||
}
|
||||
|
||||
HierarchyReadCount++;
|
||||
return Task.FromResult(_hierarchy.ToList());
|
||||
}
|
||||
|
||||
public Task<List<GalaxyAttributeRow>> GetAttributesAsync(CancellationToken ct = default)
|
||||
{
|
||||
if (ThrowOnQuery is not null)
|
||||
{
|
||||
throw ThrowOnQuery;
|
||||
}
|
||||
|
||||
AttributeReadCount++;
|
||||
return Task.FromResult(_attributes.ToList());
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Records published deploy events so tests can assert publication.</summary>
|
||||
internal sealed class RecordingDeployNotifier : IGalaxyDeployNotifier
|
||||
{
|
||||
public List<GalaxyDeployEventInfo> Published { get; } = [];
|
||||
|
||||
public GalaxyDeployEventInfo? Latest { get; private set; }
|
||||
|
||||
public void Publish(GalaxyDeployEventInfo info)
|
||||
{
|
||||
Published.Add(info);
|
||||
Latest = info;
|
||||
}
|
||||
|
||||
public async IAsyncEnumerable<GalaxyDeployEventInfo> SubscribeAsync(
|
||||
[EnumeratorCancellation] CancellationToken cancellationToken)
|
||||
{
|
||||
if (Latest is { } latest)
|
||||
{
|
||||
yield return latest;
|
||||
}
|
||||
|
||||
await Task.CompletedTask.ConfigureAwait(false);
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// In-memory <see cref="IGalaxyHierarchySnapshotStore"/>. Pre-seed <see cref="Snapshot"/>
|
||||
/// to exercise the restore path; reads <see cref="SaveAsync"/> back to assert persistence.
|
||||
/// </summary>
|
||||
internal sealed class FakeSnapshotStore : IGalaxyHierarchySnapshotStore
|
||||
{
|
||||
public GalaxyHierarchySnapshot? Snapshot { get; set; }
|
||||
|
||||
public int SaveCount { get; private set; }
|
||||
|
||||
public int LoadCount { get; private set; }
|
||||
|
||||
public Task SaveAsync(GalaxyHierarchySnapshot snapshot, CancellationToken cancellationToken)
|
||||
{
|
||||
SaveCount++;
|
||||
Snapshot = snapshot;
|
||||
return Task.CompletedTask;
|
||||
}
|
||||
|
||||
public Task<GalaxyHierarchySnapshot?> TryLoadAsync(CancellationToken cancellationToken)
|
||||
{
|
||||
LoadCount++;
|
||||
return Task.FromResult(Snapshot);
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// A <see cref="TimeProvider"/> whose UTC clock is fixed (and advanceable) so the cache's
|
||||
/// staleness projection (which fires after a 5-minute threshold) is deterministic.
|
||||
/// </summary>
|
||||
internal sealed class StubTimeProvider(DateTimeOffset start) : TimeProvider
|
||||
{
|
||||
private DateTimeOffset _now = start;
|
||||
|
||||
public override DateTimeOffset GetUtcNow() => _now;
|
||||
|
||||
public void Advance(TimeSpan delta) => _now += delta;
|
||||
}
|
||||
+236
@@ -0,0 +1,236 @@
|
||||
using ZB.MOM.WW.GalaxyRepository;
|
||||
|
||||
namespace ZB.MOM.WW.GalaxyRepository.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// Tests for <see cref="GalaxyHierarchyCache"/> first-load, deploy-gating, snapshot
|
||||
/// restore, persistence, and status-transition behavior. Uses an in-memory
|
||||
/// <see cref="IGalaxyRepository"/> and snapshot store plus a fixed
|
||||
/// <see cref="StubTimeProvider"/> so no SQL is touched and no asserts are time-sensitive.
|
||||
/// </summary>
|
||||
public sealed class GalaxyHierarchyCacheTests
|
||||
{
|
||||
private static readonly DateTimeOffset FixedNow = new(2026, 1, 1, 12, 0, 0, TimeSpan.Zero);
|
||||
private static readonly DateTime DeployTime = new(2026, 1, 1, 0, 0, 0, DateTimeKind.Utc);
|
||||
|
||||
private static List<GalaxyHierarchyRow> SampleHierarchy() =>
|
||||
[
|
||||
new() { GobjectId = 1, TagName = "Area1", ContainedName = "Area1", BrowseName = "Area1", IsArea = true },
|
||||
new() { GobjectId = 2, TagName = "Pump01", ContainedName = "Pump01", BrowseName = "Pump01", ParentGobjectId = 1 },
|
||||
];
|
||||
|
||||
private static List<GalaxyAttributeRow> SampleAttributes() =>
|
||||
[
|
||||
new() { GobjectId = 2, AttributeName = "PV", FullTagReference = "Pump01.PV", IsHistorized = true, IsAlarm = true },
|
||||
];
|
||||
|
||||
[Fact]
|
||||
public async Task RefreshAsync_FirstLoad_PopulatesCurrentWithDataAndUnblocksWaitForFirstLoad()
|
||||
{
|
||||
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime);
|
||||
RecordingDeployNotifier notifier = new();
|
||||
using GalaxyHierarchyCache cache = new(repository, notifier, new StubTimeProvider(FixedNow));
|
||||
|
||||
// Before refresh, the gate is unset and there is no data.
|
||||
Assert.False(cache.Current.HasData);
|
||||
Assert.Equal(GalaxyCacheStatus.Unknown, cache.Current.Status);
|
||||
|
||||
await cache.RefreshAsync(CancellationToken.None);
|
||||
|
||||
// First load completes (does not hang) and Current now holds usable data.
|
||||
await cache.WaitForFirstLoadAsync(new CancellationTokenSource(TimeSpan.FromSeconds(5)).Token);
|
||||
GalaxyHierarchyCacheEntry current = cache.Current;
|
||||
Assert.True(current.HasData);
|
||||
Assert.Equal(GalaxyCacheStatus.Healthy, current.Status);
|
||||
Assert.Equal(2, current.ObjectCount);
|
||||
Assert.Equal(1, current.AreaCount);
|
||||
Assert.Equal(1, current.AttributeCount);
|
||||
Assert.Equal(1, current.HistorizedAttributeCount);
|
||||
Assert.Equal(1, current.AlarmAttributeCount);
|
||||
|
||||
// The heavy queries ran exactly once and a deploy event was published.
|
||||
Assert.Equal(1, repository.HierarchyReadCount);
|
||||
Assert.Equal(1, repository.AttributeReadCount);
|
||||
GalaxyDeployEventInfo published = Assert.Single(notifier.Published);
|
||||
Assert.Equal(2, published.ObjectCount);
|
||||
Assert.Equal(1, published.AttributeCount);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task RefreshAsync_NoDeployChange_SkipsHeavyQueriesOnSecondRefresh()
|
||||
{
|
||||
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime);
|
||||
using GalaxyHierarchyCache cache = new(repository, new RecordingDeployNotifier(), new StubTimeProvider(FixedNow));
|
||||
|
||||
await cache.RefreshAsync(CancellationToken.None);
|
||||
await cache.RefreshAsync(CancellationToken.None);
|
||||
|
||||
// Deploy time unchanged => the heavy hierarchy/attribute reads happened only once.
|
||||
Assert.Equal(1, repository.HierarchyReadCount);
|
||||
Assert.Equal(1, repository.AttributeReadCount);
|
||||
Assert.True(cache.Current.HasData);
|
||||
Assert.Equal(GalaxyCacheStatus.Healthy, cache.Current.Status);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task RefreshAsync_DeployAdvances_RebuildsAndBumpsSequence()
|
||||
{
|
||||
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime);
|
||||
RecordingDeployNotifier notifier = new();
|
||||
using GalaxyHierarchyCache cache = new(repository, notifier, new StubTimeProvider(FixedNow));
|
||||
|
||||
await cache.RefreshAsync(CancellationToken.None);
|
||||
long firstSequence = cache.Current.Sequence;
|
||||
|
||||
repository.DeployTime = DeployTime.AddHours(1);
|
||||
await cache.RefreshAsync(CancellationToken.None);
|
||||
|
||||
Assert.Equal(2, repository.HierarchyReadCount);
|
||||
Assert.Equal(firstSequence + 1, cache.Current.Sequence);
|
||||
Assert.Equal(2, notifier.Published.Count);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task RefreshAsync_FirstQueryFailsNoPriorData_StatusUnavailableButFirstLoadStillCompletes()
|
||||
{
|
||||
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime)
|
||||
{
|
||||
ThrowOnQuery = new TimeoutException("galaxy db unreachable"),
|
||||
};
|
||||
using GalaxyHierarchyCache cache = new(repository, new RecordingDeployNotifier(), new StubTimeProvider(FixedNow));
|
||||
|
||||
await cache.RefreshAsync(CancellationToken.None);
|
||||
|
||||
// First load must complete so callers do not hang, even though the query failed.
|
||||
await cache.WaitForFirstLoadAsync(new CancellationTokenSource(TimeSpan.FromSeconds(5)).Token);
|
||||
Assert.False(cache.Current.HasData);
|
||||
Assert.Equal(GalaxyCacheStatus.Unavailable, cache.Current.Status);
|
||||
Assert.Contains("unreachable", cache.Current.LastError);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task RefreshAsync_QueryFailsAfterPriorData_DegradesToStaleAndKeepsData()
|
||||
{
|
||||
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime);
|
||||
using GalaxyHierarchyCache cache = new(repository, new RecordingDeployNotifier(), new StubTimeProvider(FixedNow));
|
||||
|
||||
await cache.RefreshAsync(CancellationToken.None);
|
||||
Assert.True(cache.Current.HasData);
|
||||
|
||||
// A later refresh fails: data is retained but flagged Stale.
|
||||
repository.DeployTime = DeployTime.AddHours(1);
|
||||
repository.ThrowOnQuery = new InvalidOperationException("transient");
|
||||
await cache.RefreshAsync(CancellationToken.None);
|
||||
|
||||
Assert.True(cache.Current.HasData);
|
||||
Assert.Equal(GalaxyCacheStatus.Stale, cache.Current.Status);
|
||||
Assert.Equal(2, cache.Current.ObjectCount);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Current_AfterStalenessThreshold_ProjectsHealthyToStale()
|
||||
{
|
||||
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime);
|
||||
StubTimeProvider clock = new(FixedNow);
|
||||
using GalaxyHierarchyCache cache = new(repository, new RecordingDeployNotifier(), clock);
|
||||
|
||||
await cache.RefreshAsync(CancellationToken.None);
|
||||
Assert.Equal(GalaxyCacheStatus.Healthy, cache.Current.Status);
|
||||
|
||||
// Advance past the 5-minute staleness threshold with no successful refresh.
|
||||
clock.Advance(TimeSpan.FromMinutes(6));
|
||||
|
||||
Assert.Equal(GalaxyCacheStatus.Stale, cache.Current.Status);
|
||||
// Data is still present — Stale means "old", not "gone".
|
||||
Assert.True(cache.Current.HasData);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task RefreshAsync_PersistsSnapshotAfterSuccessfulHeavyRefresh()
|
||||
{
|
||||
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime);
|
||||
FakeSnapshotStore store = new();
|
||||
using GalaxyHierarchyCache cache = new(
|
||||
repository, new RecordingDeployNotifier(), new StubTimeProvider(FixedNow), logger: null, snapshotStore: store);
|
||||
|
||||
await cache.RefreshAsync(CancellationToken.None);
|
||||
|
||||
Assert.Equal(1, store.SaveCount);
|
||||
Assert.NotNull(store.Snapshot);
|
||||
Assert.Equal(2, store.Snapshot!.Hierarchy.Count);
|
||||
Assert.Single(store.Snapshot.Attributes);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task RefreshAsync_SnapshotRestore_ServesLastKnownDataAsStaleWhenDatabaseUnreachable()
|
||||
{
|
||||
// The snapshot store already holds a persisted dataset (last-known browse data).
|
||||
FakeSnapshotStore store = new()
|
||||
{
|
||||
Snapshot = new GalaxyHierarchySnapshot(
|
||||
LastDeployTime: DeployTime,
|
||||
SavedAt: FixedNow.AddMinutes(-1),
|
||||
Hierarchy: SampleHierarchy(),
|
||||
Attributes: SampleAttributes()),
|
||||
};
|
||||
|
||||
// The Galaxy database is unreachable on this cold start.
|
||||
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime)
|
||||
{
|
||||
ThrowOnQuery = new TimeoutException("cold start, db down"),
|
||||
};
|
||||
RecordingDeployNotifier notifier = new();
|
||||
using GalaxyHierarchyCache cache = new(
|
||||
repository, notifier, new StubTimeProvider(FixedNow), logger: null, snapshotStore: store);
|
||||
|
||||
await cache.RefreshAsync(CancellationToken.None);
|
||||
|
||||
// First load is satisfied by the restored snapshot, not by SQL.
|
||||
await cache.WaitForFirstLoadAsync(new CancellationTokenSource(TimeSpan.FromSeconds(5)).Token);
|
||||
Assert.Equal(1, store.LoadCount);
|
||||
GalaxyHierarchyCacheEntry current = cache.Current;
|
||||
Assert.True(current.HasData);
|
||||
// Restored data is "last-known", surfaced as Stale until the live DB confirms.
|
||||
Assert.Equal(GalaxyCacheStatus.Stale, current.Status);
|
||||
Assert.Equal(2, current.ObjectCount);
|
||||
Assert.Equal(DeployTime, current.LastDeployTime!.Value.UtcDateTime);
|
||||
|
||||
// A deploy event was published for the restored data.
|
||||
Assert.Single(notifier.Published);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task RefreshAsync_SnapshotRestoreThenLiveQuery_PromotesRestoredDataToHealthy()
|
||||
{
|
||||
FakeSnapshotStore store = new()
|
||||
{
|
||||
Snapshot = new GalaxyHierarchySnapshot(
|
||||
LastDeployTime: DeployTime,
|
||||
SavedAt: FixedNow.AddMinutes(-1),
|
||||
Hierarchy: SampleHierarchy(),
|
||||
Attributes: SampleAttributes()),
|
||||
};
|
||||
// DB is reachable and reports the SAME deploy time the snapshot was pulled at.
|
||||
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime);
|
||||
using GalaxyHierarchyCache cache = new(
|
||||
repository, new RecordingDeployNotifier(), new StubTimeProvider(FixedNow), logger: null, snapshotStore: store);
|
||||
|
||||
await cache.RefreshAsync(CancellationToken.None);
|
||||
|
||||
// Restore seeds Stale data; the same-deploy live query promotes it to Healthy
|
||||
// without re-running the heavy hierarchy/attribute reads.
|
||||
Assert.Equal(GalaxyCacheStatus.Healthy, cache.Current.Status);
|
||||
Assert.Equal(0, repository.HierarchyReadCount);
|
||||
Assert.True(cache.Current.HasData);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Dispose_CanBeCalledWithoutHavingRefreshed()
|
||||
{
|
||||
FakeGalaxyRepository repository = new(SampleHierarchy(), SampleAttributes(), DeployTime);
|
||||
GalaxyHierarchyCache cache = new(repository, new RecordingDeployNotifier(), new StubTimeProvider(FixedNow));
|
||||
|
||||
// Dispose must be safe even when no refresh ever ran (semaphore never entered).
|
||||
cache.Dispose();
|
||||
}
|
||||
}
|
||||
+458
@@ -0,0 +1,458 @@
|
||||
using Grpc.Core;
|
||||
using ZB.MOM.WW.GalaxyRepository;
|
||||
using ZB.MOM.WW.GalaxyRepository.Grpc;
|
||||
|
||||
namespace ZB.MOM.WW.GalaxyRepository.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// Pure-logic tests for <see cref="GalaxyHierarchyProjector"/> and
|
||||
/// <see cref="GalaxyBrowseProjector"/>. No SQL: the cache entry under test is built
|
||||
/// from a small hand-made hierarchy through the same materialization the live cache
|
||||
/// uses (a fake <see cref="IGalaxyRepository"/> driven through
|
||||
/// <see cref="GalaxyHierarchyCache.RefreshAsync"/>), so the projectors are exercised
|
||||
/// against a real <see cref="GalaxyHierarchyIndex"/>.
|
||||
/// </summary>
|
||||
public sealed class GalaxyHierarchyProjectorTests
|
||||
{
|
||||
/// <summary>
|
||||
/// Builds a realistic cache entry by driving a fake repository through the cache's
|
||||
/// own refresh path. This goes through <c>BuildEntry</c> + <see cref="GalaxyHierarchyIndex.Build"/>
|
||||
/// exactly as production does, rather than reaching for an internal factory.
|
||||
/// </summary>
|
||||
private static GalaxyHierarchyCacheEntry BuildEntry(
|
||||
IReadOnlyList<GalaxyHierarchyRow> hierarchy,
|
||||
IReadOnlyList<GalaxyAttributeRow> attributes)
|
||||
{
|
||||
FakeGalaxyRepository repository = new(hierarchy, attributes, deployTime: new DateTime(2026, 1, 1, 0, 0, 0, DateTimeKind.Utc));
|
||||
using GalaxyHierarchyCache cache = new(repository, new RecordingDeployNotifier());
|
||||
cache.RefreshAsync(CancellationToken.None).GetAwaiter().GetResult();
|
||||
GalaxyHierarchyCacheEntry entry = cache.Current;
|
||||
Assert.True(entry.HasData);
|
||||
return entry;
|
||||
}
|
||||
|
||||
// A small but representative galaxy:
|
||||
// PlantArea (area, id 1)
|
||||
// ├─ LineA (area, id 2)
|
||||
// │ ├─ Pump01 (id 10, template "Pump", historized+alarm attr)
|
||||
// │ └─ Valve01 (id 11, template "Valve", plain attr)
|
||||
// └─ Mixer01 (id 12, template "Mixer", alarm attr only)
|
||||
// StandaloneTank (id 20, no parent — a root object)
|
||||
private static GalaxyHierarchyCacheEntry BuildSampleEntry()
|
||||
{
|
||||
List<GalaxyHierarchyRow> hierarchy =
|
||||
[
|
||||
Hierarchy(1, "PlantArea", parent: 0, isArea: true, category: 100),
|
||||
Hierarchy(2, "LineA", parent: 1, isArea: true, category: 100),
|
||||
Hierarchy(10, "Pump01", parent: 2, category: 200, templates: ["$Pump", "$UserDefined"]),
|
||||
Hierarchy(11, "Valve01", parent: 2, category: 201, templates: ["$Valve"]),
|
||||
Hierarchy(12, "Mixer01", parent: 1, category: 202, templates: ["$Mixer"]),
|
||||
Hierarchy(20, "StandaloneTank", parent: 0, category: 203, templates: ["$Tank"]),
|
||||
];
|
||||
|
||||
List<GalaxyAttributeRow> attributes =
|
||||
[
|
||||
// Pump01: historized AND alarm-bearing.
|
||||
Attribute(10, "Pump01.PV", historized: true, alarm: true),
|
||||
Attribute(10, "Pump01.SP", historized: false, alarm: false),
|
||||
// Valve01: plain.
|
||||
Attribute(11, "Valve01.Cmd", historized: false, alarm: false),
|
||||
// Mixer01: alarm only.
|
||||
Attribute(12, "Mixer01.Fault", historized: false, alarm: true),
|
||||
// StandaloneTank: historized only.
|
||||
Attribute(20, "StandaloneTank.Level", historized: true, alarm: false),
|
||||
];
|
||||
|
||||
return BuildEntry(hierarchy, attributes);
|
||||
}
|
||||
|
||||
private static GalaxyHierarchyRow Hierarchy(
|
||||
int id,
|
||||
string tagName,
|
||||
int parent,
|
||||
bool isArea = false,
|
||||
int category = 0,
|
||||
IReadOnlyList<string>? templates = null) => new()
|
||||
{
|
||||
GobjectId = id,
|
||||
TagName = tagName,
|
||||
ContainedName = tagName,
|
||||
BrowseName = tagName,
|
||||
ParentGobjectId = parent,
|
||||
IsArea = isArea,
|
||||
CategoryId = category,
|
||||
TemplateChain = templates ?? Array.Empty<string>(),
|
||||
};
|
||||
|
||||
private static GalaxyAttributeRow Attribute(
|
||||
int gobjectId,
|
||||
string fullTagReference,
|
||||
bool historized,
|
||||
bool alarm) => new()
|
||||
{
|
||||
GobjectId = gobjectId,
|
||||
AttributeName = fullTagReference.Split('.')[^1],
|
||||
FullTagReference = fullTagReference,
|
||||
IsHistorized = historized,
|
||||
IsAlarm = alarm,
|
||||
};
|
||||
|
||||
[Fact]
|
||||
public void Project_NoFilters_ReturnsEveryObject()
|
||||
{
|
||||
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
|
||||
|
||||
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, new DiscoverHierarchyRequest());
|
||||
|
||||
Assert.Equal(6, result.TotalObjectCount);
|
||||
Assert.Equal(6, result.Objects.Count);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Project_PageSizeAndOffset_SlicesTheOrderedResult()
|
||||
{
|
||||
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
|
||||
DiscoverHierarchyRequest request = new();
|
||||
|
||||
GalaxyHierarchyQueryResult full = GalaxyHierarchyProjector.Project(entry, request, browseSubtreeGlobs: null, offset: 0, pageSize: int.MaxValue);
|
||||
GalaxyHierarchyQueryResult page1 = GalaxyHierarchyProjector.Project(entry, request, browseSubtreeGlobs: null, offset: 0, pageSize: 2);
|
||||
GalaxyHierarchyQueryResult page2 = GalaxyHierarchyProjector.Project(entry, request, browseSubtreeGlobs: null, offset: 2, pageSize: 2);
|
||||
GalaxyHierarchyQueryResult page3 = GalaxyHierarchyProjector.Project(entry, request, browseSubtreeGlobs: null, offset: 4, pageSize: 2);
|
||||
|
||||
// Total is unaffected by paging.
|
||||
Assert.Equal(6, page1.TotalObjectCount);
|
||||
Assert.Equal(2, page1.Objects.Count);
|
||||
Assert.Equal(2, page2.Objects.Count);
|
||||
Assert.Equal(2, page3.Objects.Count);
|
||||
|
||||
// The three pages reconstruct the full ordered result with no gaps/dupes.
|
||||
List<int> paged =
|
||||
[
|
||||
.. page1.Objects.Select(o => o.GobjectId),
|
||||
.. page2.Objects.Select(o => o.GobjectId),
|
||||
.. page3.Objects.Select(o => o.GobjectId),
|
||||
];
|
||||
Assert.Equal(full.Objects.Select(o => o.GobjectId), paged);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Project_OffsetPastEnd_ReturnsEmptyPageButRealTotal()
|
||||
{
|
||||
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
|
||||
|
||||
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(
|
||||
entry, new DiscoverHierarchyRequest(), browseSubtreeGlobs: null, offset: 999, pageSize: 10);
|
||||
|
||||
Assert.Empty(result.Objects);
|
||||
Assert.Equal(6, result.TotalObjectCount);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Project_PageSignature_IsStableAcrossPagesAndMatchesComputeFilterSignature()
|
||||
{
|
||||
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
|
||||
DiscoverHierarchyRequest request = new() { TagNameGlob = "Pump*" };
|
||||
|
||||
string expected = GalaxyHierarchyProjector.ComputeFilterSignature(request, browseSubtreeGlobs: null);
|
||||
GalaxyHierarchyQueryResult page1 = GalaxyHierarchyProjector.Project(entry, request, browseSubtreeGlobs: null, offset: 0, pageSize: 1);
|
||||
GalaxyHierarchyQueryResult page2 = GalaxyHierarchyProjector.Project(entry, request, browseSubtreeGlobs: null, offset: 1, pageSize: 1);
|
||||
|
||||
// The signature a caller computes to mint a page token round-trips: the projector
|
||||
// reports the same signature on every page of the same filter set.
|
||||
Assert.Equal(expected, page1.FilterSignature);
|
||||
Assert.Equal(expected, page2.FilterSignature);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ComputeFilterSignature_DiffersWhenAnyFilterChanges()
|
||||
{
|
||||
DiscoverHierarchyRequest baseRequest = new() { TagNameGlob = "Pump*" };
|
||||
DiscoverHierarchyRequest differentGlob = new() { TagNameGlob = "Valve*" };
|
||||
DiscoverHierarchyRequest differentAlarm = new() { TagNameGlob = "Pump*", AlarmBearingOnly = true };
|
||||
|
||||
string baseSig = GalaxyHierarchyProjector.ComputeFilterSignature(baseRequest, null);
|
||||
|
||||
Assert.NotEqual(baseSig, GalaxyHierarchyProjector.ComputeFilterSignature(differentGlob, null));
|
||||
Assert.NotEqual(baseSig, GalaxyHierarchyProjector.ComputeFilterSignature(differentAlarm, null));
|
||||
Assert.NotEqual(baseSig, GalaxyHierarchyProjector.ComputeFilterSignature(baseRequest, browseSubtreeGlobs: ["PlantArea/*"]));
|
||||
// Same inputs => same signature (deterministic).
|
||||
Assert.Equal(baseSig, GalaxyHierarchyProjector.ComputeFilterSignature(new DiscoverHierarchyRequest { TagNameGlob = "Pump*" }, null));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Project_MaxDepthZero_FromRoot_ReturnsOnlyTheRoot()
|
||||
{
|
||||
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
|
||||
DiscoverHierarchyRequest request = new() { RootGobjectId = 1, MaxDepth = 0 };
|
||||
|
||||
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, request);
|
||||
|
||||
GalaxyObject only = Assert.Single(result.Objects);
|
||||
Assert.Equal(1, only.GobjectId);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Project_MaxDepthOne_FromRoot_ReturnsRootAndDirectChildrenOnly()
|
||||
{
|
||||
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
|
||||
// PlantArea(1) depth 0; LineA(2) and Mixer01(12) depth 1; Pump01/Valve01 depth 2.
|
||||
DiscoverHierarchyRequest request = new() { RootGobjectId = 1, MaxDepth = 1 };
|
||||
|
||||
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, request);
|
||||
|
||||
Assert.Equal([1, 2, 12], result.Objects.Select(o => o.GobjectId).OrderBy(id => id));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Project_NegativeMaxDepth_ThrowsInvalidArgument()
|
||||
{
|
||||
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
|
||||
DiscoverHierarchyRequest request = new() { MaxDepth = -1 };
|
||||
|
||||
RpcException ex = Assert.Throws<RpcException>(() => GalaxyHierarchyProjector.Project(entry, request));
|
||||
Assert.Equal(StatusCode.InvalidArgument, ex.StatusCode);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Project_UnknownRoot_ThrowsNotFound()
|
||||
{
|
||||
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
|
||||
DiscoverHierarchyRequest request = new() { RootGobjectId = 99999 };
|
||||
|
||||
RpcException ex = Assert.Throws<RpcException>(() => GalaxyHierarchyProjector.Project(entry, request));
|
||||
Assert.Equal(StatusCode.NotFound, ex.StatusCode);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Project_HistorizedOnly_ReturnsOnlyObjectsWithAHistorizedAttribute()
|
||||
{
|
||||
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
|
||||
DiscoverHierarchyRequest request = new() { HistorizedOnly = true };
|
||||
|
||||
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, request);
|
||||
|
||||
// Pump01(10) and StandaloneTank(20) carry historized attributes.
|
||||
Assert.Equal([10, 20], result.Objects.Select(o => o.GobjectId).OrderBy(id => id));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Project_AlarmBearingOnly_ReturnsOnlyObjectsWithAnAlarmAttribute()
|
||||
{
|
||||
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
|
||||
DiscoverHierarchyRequest request = new() { AlarmBearingOnly = true };
|
||||
|
||||
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, request);
|
||||
|
||||
// Pump01(10) and Mixer01(12) carry alarm attributes.
|
||||
Assert.Equal([10, 12], result.Objects.Select(o => o.GobjectId).OrderBy(id => id));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Project_AlarmAndHistorizedTogether_RequiresBoth()
|
||||
{
|
||||
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
|
||||
DiscoverHierarchyRequest request = new() { AlarmBearingOnly = true, HistorizedOnly = true };
|
||||
|
||||
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, request);
|
||||
|
||||
// Only Pump01(10) carries an attribute set that is both historized and alarm-bearing.
|
||||
GalaxyObject only = Assert.Single(result.Objects);
|
||||
Assert.Equal(10, only.GobjectId);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Project_TagNameGlob_MatchesAnchoredCaseInsensitive()
|
||||
{
|
||||
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
|
||||
|
||||
GalaxyHierarchyQueryResult prefix = GalaxyHierarchyProjector.Project(entry, new DiscoverHierarchyRequest { TagNameGlob = "Pump*" });
|
||||
Assert.Equal([10], prefix.Objects.Select(o => o.GobjectId));
|
||||
|
||||
// Case-insensitive.
|
||||
GalaxyHierarchyQueryResult lower = GalaxyHierarchyProjector.Project(entry, new DiscoverHierarchyRequest { TagNameGlob = "pump01" });
|
||||
Assert.Equal([10], lower.Objects.Select(o => o.GobjectId));
|
||||
|
||||
// '?' single-char wildcard: "Pump0?" matches "Pump01".
|
||||
GalaxyHierarchyQueryResult single = GalaxyHierarchyProjector.Project(entry, new DiscoverHierarchyRequest { TagNameGlob = "Pump0?" });
|
||||
Assert.Equal([10], single.Objects.Select(o => o.GobjectId));
|
||||
|
||||
// Anchored: a bare substring that is not a prefix matches nothing.
|
||||
GalaxyHierarchyQueryResult anchored = GalaxyHierarchyProjector.Project(entry, new DiscoverHierarchyRequest { TagNameGlob = "ump01" });
|
||||
Assert.Empty(anchored.Objects);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Project_CategoryIds_FilterByObjectCategory()
|
||||
{
|
||||
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
|
||||
DiscoverHierarchyRequest request = new() { CategoryIds = { 200, 201 } };
|
||||
|
||||
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, request);
|
||||
|
||||
// category 200 = Pump01(10), category 201 = Valve01(11).
|
||||
Assert.Equal([10, 11], result.Objects.Select(o => o.GobjectId).OrderBy(id => id));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Project_TemplateChainContains_IsSubstringAndCaseInsensitive()
|
||||
{
|
||||
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
|
||||
DiscoverHierarchyRequest request = new() { TemplateChainContains = { "pump" } };
|
||||
|
||||
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, request);
|
||||
|
||||
GalaxyObject only = Assert.Single(result.Objects);
|
||||
Assert.Equal(10, only.GobjectId);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Project_IncludeAttributesDefault_CarriesAttributes()
|
||||
{
|
||||
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
|
||||
DiscoverHierarchyRequest request = new() { TagNameGlob = "Pump*" };
|
||||
|
||||
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, request);
|
||||
|
||||
GalaxyObject pump = Assert.Single(result.Objects);
|
||||
Assert.Equal(2, pump.Attributes.Count);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Project_IncludeAttributesFalse_ReturnsSkeletons()
|
||||
{
|
||||
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
|
||||
DiscoverHierarchyRequest request = new() { TagNameGlob = "Pump*", IncludeAttributes = false };
|
||||
|
||||
GalaxyHierarchyQueryResult result = GalaxyHierarchyProjector.Project(entry, request);
|
||||
|
||||
GalaxyObject pump = Assert.Single(result.Objects);
|
||||
Assert.Empty(pump.Attributes);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Project_IncludeAttributesFalse_DoesNotMutateTheCachedEntry()
|
||||
{
|
||||
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
|
||||
|
||||
// Project with attributes stripped, then again with attributes included.
|
||||
GalaxyHierarchyProjector.Project(entry, new DiscoverHierarchyRequest { TagNameGlob = "Pump*", IncludeAttributes = false });
|
||||
GalaxyHierarchyQueryResult included = GalaxyHierarchyProjector.Project(entry, new DiscoverHierarchyRequest { TagNameGlob = "Pump*" });
|
||||
|
||||
// The earlier strip cloned the object — the cached entry still holds the attributes.
|
||||
GalaxyObject pump = Assert.Single(included.Objects);
|
||||
Assert.Equal(2, pump.Attributes.Count);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Project_InvalidOffsetOrPageSize_Throws()
|
||||
{
|
||||
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
|
||||
|
||||
Assert.Throws<ArgumentOutOfRangeException>(() =>
|
||||
GalaxyHierarchyProjector.Project(entry, new DiscoverHierarchyRequest(), null, offset: -1, pageSize: 10));
|
||||
Assert.Throws<ArgumentOutOfRangeException>(() =>
|
||||
GalaxyHierarchyProjector.Project(entry, new DiscoverHierarchyRequest(), null, offset: 0, pageSize: 0));
|
||||
}
|
||||
|
||||
// ---- GalaxyBrowseProjector ----
|
||||
|
||||
[Fact]
|
||||
public void ProjectChildren_OfPlantArea_ReturnsDirectChildrenAreasFirst()
|
||||
{
|
||||
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
|
||||
BrowseChildrenRequest request = new() { ParentGobjectId = 1 };
|
||||
|
||||
GalaxyBrowseChildrenResult result = GalaxyBrowseProjector.ProjectChildren(entry, request, browseSubtreeGlobs: null, offset: 0, pageSize: 100);
|
||||
|
||||
// Direct children of PlantArea(1) are LineA(2, area) and Mixer01(12, non-area);
|
||||
// areas sort first.
|
||||
Assert.Equal([2, 12], result.Children.Select(c => c.GobjectId));
|
||||
Assert.Equal(2, result.TotalChildCount);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ProjectChildren_ChildHasChildrenFlag_ReflectsPresenceOfChildren()
|
||||
{
|
||||
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
|
||||
BrowseChildrenRequest request = new() { ParentGobjectId = 1 };
|
||||
|
||||
GalaxyBrowseChildrenResult result = GalaxyBrowseProjector.ProjectChildren(entry, request, browseSubtreeGlobs: null, offset: 0, pageSize: 100);
|
||||
|
||||
Dictionary<int, bool> hasChildren = result.Children
|
||||
.Select((child, index) => (child.GobjectId, result.ChildHasChildren[index]))
|
||||
.ToDictionary(t => t.GobjectId, t => t.Item2);
|
||||
|
||||
// LineA(2) contains Pump01/Valve01 -> true; Mixer01(12) is a leaf -> false.
|
||||
Assert.True(hasChildren[2]);
|
||||
Assert.False(hasChildren[12]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ProjectChildren_OfRoot_ReturnsTopLevelObjects()
|
||||
{
|
||||
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
|
||||
// Empty parent oneof => roots (parent id 0).
|
||||
BrowseChildrenRequest request = new();
|
||||
|
||||
GalaxyBrowseChildrenResult result = GalaxyBrowseProjector.ProjectChildren(entry, request, browseSubtreeGlobs: null, offset: 0, pageSize: 100);
|
||||
|
||||
// Roots: PlantArea(1, area) and StandaloneTank(20, non-area); areas first.
|
||||
Assert.Equal([1, 20], result.Children.Select(c => c.GobjectId));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ProjectChildren_FilterMatchingDescendant_SurfacesNonMatchingAncestor()
|
||||
{
|
||||
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
|
||||
// Pump01 lives two levels under PlantArea. Browsing PlantArea's children with a
|
||||
// Pump glob should still surface LineA (which itself does not match) because it
|
||||
// contains a matching descendant.
|
||||
BrowseChildrenRequest request = new() { ParentGobjectId = 1, TagNameGlob = "Pump*" };
|
||||
|
||||
GalaxyBrowseChildrenResult result = GalaxyBrowseProjector.ProjectChildren(entry, request, browseSubtreeGlobs: null, offset: 0, pageSize: 100);
|
||||
|
||||
GalaxyObject surfaced = Assert.Single(result.Children);
|
||||
Assert.Equal(2, surfaced.GobjectId);
|
||||
Assert.True(result.ChildHasChildren[0]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ProjectChildren_UnknownParent_ThrowsNotFound()
|
||||
{
|
||||
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
|
||||
BrowseChildrenRequest request = new() { ParentGobjectId = 99999 };
|
||||
|
||||
RpcException ex = Assert.Throws<RpcException>(() =>
|
||||
GalaxyBrowseProjector.ProjectChildren(entry, request, null, 0, 100));
|
||||
Assert.Equal(StatusCode.NotFound, ex.StatusCode);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ProjectChildren_Paging_SlicesAndPreservesTotal()
|
||||
{
|
||||
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
|
||||
// LineA(2) has two direct children: Pump01, Valve01.
|
||||
BrowseChildrenRequest request = new() { ParentGobjectId = 2 };
|
||||
|
||||
GalaxyBrowseChildrenResult page1 = GalaxyBrowseProjector.ProjectChildren(entry, request, null, offset: 0, pageSize: 1);
|
||||
GalaxyBrowseChildrenResult page2 = GalaxyBrowseProjector.ProjectChildren(entry, request, null, offset: 1, pageSize: 1);
|
||||
|
||||
Assert.Equal(2, page1.TotalChildCount);
|
||||
Assert.Single(page1.Children);
|
||||
Assert.Single(page2.Children);
|
||||
Assert.NotEqual(page1.Children[0].GobjectId, page2.Children[0].GobjectId);
|
||||
// Same filter+parent => same signature on both pages.
|
||||
Assert.Equal(page1.FilterSignature, page2.FilterSignature);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ResolveParentId_ByTagName_ResolvesToGobjectId()
|
||||
{
|
||||
GalaxyHierarchyCacheEntry entry = BuildSampleEntry();
|
||||
BrowseChildrenRequest request = new() { ParentTagName = "LineA" };
|
||||
|
||||
int id = GalaxyBrowseProjector.ResolveParentId(entry, request);
|
||||
|
||||
Assert.Equal(2, id);
|
||||
}
|
||||
}
|
||||
+84
@@ -0,0 +1,84 @@
|
||||
using Microsoft.Extensions.Options;
|
||||
using ZB.MOM.WW.GalaxyRepository;
|
||||
|
||||
namespace ZB.MOM.WW.GalaxyRepository.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// Round-trip tests for the real <see cref="GalaxyHierarchySnapshotStore"/> over a temp
|
||||
/// file path: save then load, no-op when persistence is disabled, and clean disposal.
|
||||
/// </summary>
|
||||
public sealed class GalaxyHierarchySnapshotStoreTests : IDisposable
|
||||
{
|
||||
private readonly string _path = Path.Combine(
|
||||
Path.GetTempPath(),
|
||||
$"galaxyrepo-snap-{Guid.NewGuid():N}.json");
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
if (File.Exists(_path))
|
||||
{
|
||||
File.Delete(_path);
|
||||
}
|
||||
}
|
||||
|
||||
private static GalaxyHierarchySnapshot SampleSnapshot() => new(
|
||||
LastDeployTime: new DateTimeOffset(2026, 1, 1, 0, 0, 0, TimeSpan.Zero),
|
||||
SavedAt: new DateTimeOffset(2026, 1, 1, 12, 0, 0, TimeSpan.Zero),
|
||||
Hierarchy:
|
||||
[
|
||||
new GalaxyHierarchyRow { GobjectId = 1, TagName = "Area1", IsArea = true },
|
||||
new GalaxyHierarchyRow { GobjectId = 2, TagName = "Pump01", ParentGobjectId = 1 },
|
||||
],
|
||||
Attributes:
|
||||
[
|
||||
new GalaxyAttributeRow { GobjectId = 2, AttributeName = "PV", FullTagReference = "Pump01.PV", IsHistorized = true },
|
||||
]);
|
||||
|
||||
[Fact]
|
||||
public async Task SaveThenLoad_RoundTripsTheSnapshot()
|
||||
{
|
||||
using GalaxyHierarchySnapshotStore store = new(
|
||||
Options.Create(new GalaxyRepositoryOptions { PersistSnapshot = true, SnapshotCachePath = _path }));
|
||||
|
||||
await store.SaveAsync(SampleSnapshot(), CancellationToken.None);
|
||||
GalaxyHierarchySnapshot? loaded = await store.TryLoadAsync(CancellationToken.None);
|
||||
|
||||
Assert.NotNull(loaded);
|
||||
Assert.Equal(2, loaded!.Hierarchy.Count);
|
||||
Assert.Single(loaded.Attributes);
|
||||
Assert.Equal("Pump01.PV", loaded.Attributes[0].FullTagReference);
|
||||
Assert.True(loaded.Attributes[0].IsHistorized);
|
||||
Assert.Equal(SampleSnapshot().LastDeployTime, loaded.LastDeployTime);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task SaveAndLoad_AreNoOps_WhenPersistenceDisabled()
|
||||
{
|
||||
using GalaxyHierarchySnapshotStore store = new(
|
||||
Options.Create(new GalaxyRepositoryOptions { PersistSnapshot = false, SnapshotCachePath = _path }));
|
||||
|
||||
await store.SaveAsync(SampleSnapshot(), CancellationToken.None);
|
||||
|
||||
Assert.False(File.Exists(_path));
|
||||
Assert.Null(await store.TryLoadAsync(CancellationToken.None));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task TryLoad_ReturnsNull_WhenNoFileExists()
|
||||
{
|
||||
using GalaxyHierarchySnapshotStore store = new(
|
||||
Options.Create(new GalaxyRepositoryOptions { PersistSnapshot = true, SnapshotCachePath = _path }));
|
||||
|
||||
Assert.Null(await store.TryLoadAsync(CancellationToken.None));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task TryLoad_ReturnsNull_WhenFileIsNotValidJson()
|
||||
{
|
||||
await File.WriteAllTextAsync(_path, "{ this is not valid json");
|
||||
using GalaxyHierarchySnapshotStore store = new(
|
||||
Options.Create(new GalaxyRepositoryOptions { PersistSnapshot = true, SnapshotCachePath = _path }));
|
||||
|
||||
Assert.Null(await store.TryLoadAsync(CancellationToken.None));
|
||||
}
|
||||
}
|
||||
+25
@@ -0,0 +1,25 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
|
||||
<PropertyGroup>
|
||||
<IsPackable>false</IsPackable>
|
||||
<!-- Test project does not ship; no XML docs required (overrides Directory.Build.props). -->
|
||||
<GenerateDocumentationFile>false</GenerateDocumentationFile>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="coverlet.collector" />
|
||||
<PackageReference Include="Microsoft.NET.Test.Sdk" />
|
||||
<PackageReference Include="xunit" />
|
||||
<PackageReference Include="xunit.runner.visualstudio" />
|
||||
<PackageReference Include="Microsoft.Data.SqlClient" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<Using Include="Xunit" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\..\src\ZB.MOM.WW.GalaxyRepository\ZB.MOM.WW.GalaxyRepository.csproj" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
@@ -0,0 +1,122 @@
|
||||
# ZB.MOM.WW.SPHistorianClient — Design
|
||||
|
||||
**Date:** 2026-06-19
|
||||
**Status:** Approved — proceeding to implementation plan.
|
||||
|
||||
## Goal
|
||||
|
||||
Repackage the proven, pure-managed .NET 10 `AVEVA.Historian.Client` SDK (delivered in
|
||||
`HistorianSDK_2023R2/histsdk-migration.zip` from `10.100.0.48`) as the family-branded shared
|
||||
library **`ZB.MOM.WW.SPHistorianClient`** (System Platform Historian Client), following the same
|
||||
conventions as the other `ZB.MOM.WW.*` shared libraries in this repo.
|
||||
|
||||
## Context — what the source bundle contains
|
||||
|
||||
`histsdk-migration.zip` → `histsdk-migration/`:
|
||||
|
||||
- `histsdk/` — the SDK git repo. `src/AVEVA.Historian.Client/` is a **pure-managed .NET 10** client
|
||||
for AVEVA Historian (no `aahClientManaged.dll` / `aahClient.dll` / native AVEVA runtime — the wire
|
||||
protocol is reverse-engineered and re-implemented in C#). ~165–188 unit + gated-live tests pass.
|
||||
- `analysis-2023r2/` — reverse-engineering analysis (recovered protos, decompiled stock contract,
|
||||
transport writeup). **Kept separate from the repo on purpose.**
|
||||
|
||||
Two transport families exist in the SDK:
|
||||
|
||||
| Transport | Protocol | Platform | Verification |
|
||||
|---|---|---|---|
|
||||
| `LocalPipe`, `RemoteTcpIntegrated`, `RemoteTcpCertificate` | WCF/MDAS (2020) | **Windows-only** | **live-verified**: raw/aggregate(16 modes)/at-time/event reads, browse, metadata, status, `EnsureTag`/`DeleteTag` |
|
||||
| `RemoteGrpc` | gRPC (2023 R2) | cross-platform (Grpc.Net.Client/.Web) | unit-tested; **not yet live-verified** against a real 2023 R2 server (`ExchangeKey` auth step unproven) |
|
||||
|
||||
## Decisions (locked)
|
||||
|
||||
1. **Approach: port + rebrand.** Copy the SDK source into `ZB.MOM.WW.SPHistorianClient`, rename the
|
||||
root namespace, adopt ZB conventions, bring the unit tests, drop non-shippable artifacts. One
|
||||
coherent shared library — a published package should not ship a third-party (AVEVA) namespace or
|
||||
non-redistributable reverse-engineering artifacts.
|
||||
2. **Transports: both WCF + gRPC.** Ship everything that works. WCF members keep
|
||||
`[SupportedOSPlatform("windows")]`; the gRPC path runs anywhere. No working code discarded.
|
||||
3. **Not a "component normalization."** There is no duplicated historian code across the three apps
|
||||
to converge — this is a net-new shared library that simply follows ZB packaging conventions.
|
||||
|
||||
## Repository layout
|
||||
|
||||
Plain files committed into this repo (NOT a nested git repo — see the
|
||||
`shared-libs-are-plain-files-not-nested-repos` convention):
|
||||
|
||||
```
|
||||
ZB.MOM.WW.SPHistorianClient/
|
||||
Directory.Build.props # net10.0, Nullable, ImplicitUsings, LangVersion latest, Version 0.1.0, central pkg mgmt
|
||||
Directory.Packages.props # central PackageVersion entries
|
||||
ZB.MOM.WW.SPHistorianClient.slnx
|
||||
CLAUDE.md README.md .gitignore
|
||||
src/ZB.MOM.WW.SPHistorianClient/ # the single package
|
||||
HistorianClient.cs, HistorianClientOptions.cs, HistorianTransport.cs
|
||||
Models/ Protocol/ Transport/ Wcf/ Wcf/Contracts/ Grpc/ Grpc/Protos/*.proto
|
||||
DependencyInjection/AddZbSpHistorianClient (ZB-idiomatic DI extension)
|
||||
tests/ZB.MOM.WW.SPHistorianClient.Tests/ # offline unit/golden-byte + gated-live integration
|
||||
artifacts/ # dotnet pack output
|
||||
```
|
||||
|
||||
## Port mechanics
|
||||
|
||||
- Copy `src/AVEVA.Historian.Client/` and `tests/AVEVA.Historian.Client.Tests/` from the bundle.
|
||||
- Rename the C# root namespace `AVEVA.Historian.Client` → `ZB.MOM.WW.SPHistorianClient` across all
|
||||
files: 74 `namespace` declarations spanning the root + 6 sub-namespaces
|
||||
(`.Models`, `.Wcf`, `.Wcf.Contracts`, `.Protocol`, `.Transport`, `.Grpc`), all `using` directives,
|
||||
and the `InternalsVisibleTo` to the test assembly. Drop the `InternalsVisibleTo` to
|
||||
`AVEVA.Historian.ReverseEngineering` (tool not shipped).
|
||||
- **Leave the proto wire contracts untouched:** the 6 `Grpc/Protos/*.proto` keep
|
||||
`option csharp_namespace = "ArchestrA.Grpc.Contract.*"` — that is AVEVA's wire contract, not ours.
|
||||
`Grpc.Tools` keeps generating the client stubs at build.
|
||||
- Convert inline `PackageReference` versions to central management in `Directory.Packages.props`,
|
||||
matching the `ZB.MOM.WW.Telemetry` template.
|
||||
|
||||
## Dependencies
|
||||
|
||||
- **Library:** `Google.Protobuf`, `Grpc.Net.Client`, `Grpc.Net.Client.Web`, `Grpc.Tools` (build-only,
|
||||
`PrivateAssets=all`), `System.ServiceModel.NetNamedPipe`, `System.ServiceModel.NetTcp`,
|
||||
`System.Security.Cryptography.Xml`. Add `Microsoft.Extensions.DependencyInjection.Abstractions` +
|
||||
`Microsoft.Extensions.Options` for the DI extension.
|
||||
- **Tests:** `xunit`, `xunit.runner.visualstudio`, `Microsoft.NET.Test.Sdk`, `coverlet.collector`,
|
||||
`Microsoft.Data.SqlClient` (SQL post-check tests).
|
||||
|
||||
## Excluded (safety / non-redistributable / Windows-native)
|
||||
|
||||
- `tools/` reverse-engineering harnesses (.NET Framework, reference native AVEVA binaries).
|
||||
- `analysis-2023r2/decompiled/` — proprietary AVEVA decompilations (not redistributable).
|
||||
- `scripts/` — Frida / PowerShell / Python capture tooling.
|
||||
- `docs/reverse-engineering/` — identity-bearing `.ndjson` / capture evidence.
|
||||
|
||||
**Kept:** the recovered `.proto` files (needed to build), the offline unit tests, and a sanitized
|
||||
architecture/surface summary folded into `CLAUDE.md` / `README.md`. `.gitignore` blocks the
|
||||
identity-bearing patterns (`*.ndjson`, `current/`, `aveva-install-*/`, `artifacts/`-raw, etc.).
|
||||
|
||||
## Public surface (preserved 1:1)
|
||||
|
||||
`HistorianClient` + `HistorianClientOptions` façade; `Models/*`; `HistorianTransport` enum
|
||||
(`LocalPipe` / `RemoteTcpIntegrated` / `RemoteTcpCertificate` / `RemoteGrpc`); operations:
|
||||
`ProbeAsync`, `ReadRawAsync` / `ReadAggregateAsync` / `ReadAtTimeAsync`, `ReadEventsAsync`,
|
||||
`BrowseTagNamesAsync`, `GetTagMetadataAsync`, status calls, `EnsureTagAsync` / `DeleteTagAsync`.
|
||||
|
||||
**One ZB-idiomatic addition:** `AddZbSpHistorianClient(...)` DI extension mirroring `AddZbTelemetry`
|
||||
— thin: binds `HistorianClientOptions` and registers `HistorianClient`. Optional to consumers.
|
||||
|
||||
## Cross-platform & testing posture
|
||||
|
||||
- WCF members already carry `[SupportedOSPlatform("windows")]`; the library builds and unit-tests on
|
||||
macOS/Linux. gRPC path is portable.
|
||||
- Offline unit/golden-byte tests run anywhere. Live integration tests stay gated by `HISTORIAN_*`
|
||||
env vars and skip cleanly when unset.
|
||||
- Verify `dotnet build` + `dotnet test` pass locally (macOS) before finishing.
|
||||
|
||||
## Packaging
|
||||
|
||||
`dotnet pack -c Release -o ./artifacts` → `ZB.MOM.WW.SPHistorianClient.0.1.0.nupkg`. Gitea URLs in
|
||||
package metadata. **Not pushed/published** to any feed unless explicitly requested.
|
||||
|
||||
## Out of scope (this pass)
|
||||
|
||||
- Wiring `ZB.MOM.WW.SPHistorianClient` into any consumer (e.g. OtOpcUa Phase C HistoryRead) — a
|
||||
separate follow-on.
|
||||
- Live-verifying the gRPC `RemoteGrpc` path against a real 2023 R2 server.
|
||||
- Writing samples (`AddS2`) — architecturally blocked in the source SDK; remains out of scope.
|
||||
@@ -0,0 +1,569 @@
|
||||
# ZB.MOM.WW.SPHistorianClient Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans (or subagent-driven-development) to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Repackage the proven, pure-managed .NET 10 `AVEVA.Historian.Client` SDK from the migration bundle as the family-branded shared library `ZB.MOM.WW.SPHistorianClient`, following the same conventions as the other `ZB.MOM.WW.*` libraries in this repo.
|
||||
|
||||
**Architecture:** This is a **port + rebrand**, not a rewrite. Copy the SDK `src/` and `tests/` into a new `ZB.MOM.WW.SPHistorianClient/` directory, rewrite the C# root namespace `AVEVA.Historian.Client` → `ZB.MOM.WW.SPHistorianClient` (leaving the proto-generated `ArchestrA.Grpc.Contract.*` wire contracts untouched), adopt ZB conventions (`Directory.Build.props` / `Directory.Packages.props` central package management, `.slnx`, `CLAUDE.md`/`README.md`), drop the non-shippable reverse-engineering tooling and proprietary decompilations, add one ZB-idiomatic DI extension, then build/test/pack.
|
||||
|
||||
**Tech Stack:** .NET 10, C# (net10.0), WCF/MDAS (`System.ServiceModel.*`, Windows-only transports), gRPC (`Grpc.Net.Client` + `Grpc.Tools`, cross-platform 2023 R2 transport), xUnit. Central package management.
|
||||
|
||||
**Design doc:** `docs/plans/2026-06-19-sphistorianclient-design.md`
|
||||
|
||||
**Branch:** `feat/sphistorianclient` (already created; design doc already committed at `bbb7942`).
|
||||
|
||||
---
|
||||
|
||||
## Source bundle location (read-only inputs)
|
||||
|
||||
The SDK source lives in an extracted bundle under `/tmp`:
|
||||
|
||||
- Extracted root: `/tmp/histsdk/extracted/histsdk-migration/histsdk/`
|
||||
- SDK source: `…/histsdk/src/AVEVA.Historian.Client/` — **74 `.cs` + 6 `.proto`**
|
||||
- SDK tests: `…/histsdk/tests/AVEVA.Historian.Client.Tests/` — **25 `.cs`**
|
||||
- Re-extract fallback (if `/tmp` was cleaned): `cd /tmp/histsdk && unzip -o -q histsdk-migration.zip -d extracted`
|
||||
|
||||
**Never copy:** `tools/` (RE harnesses, .NET Framework + native AVEVA refs), `analysis-2023r2/decompiled/` (proprietary, non-redistributable), `scripts/`, `docs/reverse-engineering/` (identity-bearing captures), `bin/`/`obj/`, the bundle's `.git/`, and the bundle's original `.csproj` files (we author fresh ZB ones).
|
||||
|
||||
**Gotchas baked into this plan (from prior repo experience):**
|
||||
- Do **not** set `TreatWarningsAsErrors` — the WCF/SSPI code carries `[SupportedOSPlatform("windows")]` and will emit CA platform warnings on macOS that must stay warnings.
|
||||
- Central package management means **no inline `Version=` on any `PackageReference`** (that is `NU1008`). All versions live in `Directory.Packages.props`.
|
||||
- `Microsoft.Data.SqlClient` may surface an `NU1903` advisory on restore. Without `TreatWarningsAsErrors` it is a warning. If a restore ever hard-fails on it, add `-p:NuGetAudit=false` to the build/test command.
|
||||
- macOS `sed -i` requires an explicit empty backup arg: `sed -i '' 's/…/…/g'`.
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Scaffold the library skeleton
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~4 min
|
||||
**Parallelizable with:** none (every later task depends on this)
|
||||
|
||||
**Files:**
|
||||
- Create: `ZB.MOM.WW.SPHistorianClient/Directory.Build.props`
|
||||
- Create: `ZB.MOM.WW.SPHistorianClient/Directory.Packages.props`
|
||||
- Create: `ZB.MOM.WW.SPHistorianClient/.gitignore`
|
||||
- Create: `ZB.MOM.WW.SPHistorianClient/ZB.MOM.WW.SPHistorianClient.slnx`
|
||||
- Create: `ZB.MOM.WW.SPHistorianClient/src/ZB.MOM.WW.SPHistorianClient/ZB.MOM.WW.SPHistorianClient.csproj`
|
||||
- Create: `ZB.MOM.WW.SPHistorianClient/tests/ZB.MOM.WW.SPHistorianClient.Tests/ZB.MOM.WW.SPHistorianClient.Tests.csproj`
|
||||
|
||||
**Step 1: `Directory.Build.props`** (mirrors `ZB.MOM.WW.Telemetry/Directory.Build.props`)
|
||||
|
||||
```xml
|
||||
<Project>
|
||||
|
||||
<PropertyGroup>
|
||||
<TargetFramework>net10.0</TargetFramework>
|
||||
<Nullable>enable</Nullable>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<LangVersion>latest</LangVersion>
|
||||
<Version>0.1.0</Version>
|
||||
<ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
|
||||
</PropertyGroup>
|
||||
|
||||
</Project>
|
||||
```
|
||||
|
||||
**Step 2: `Directory.Packages.props`** (versions lifted verbatim from the bundle's two `.csproj` files)
|
||||
|
||||
```xml
|
||||
<Project>
|
||||
|
||||
<PropertyGroup>
|
||||
<ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<!-- Historian SDK runtime deps (WCF/MDAS transports — Windows-only at runtime) -->
|
||||
<PackageVersion Include="System.Security.Cryptography.Xml" Version="10.0.7" />
|
||||
<PackageVersion Include="System.ServiceModel.NetNamedPipe" Version="10.0.652802" />
|
||||
<PackageVersion Include="System.ServiceModel.NetTcp" Version="10.0.652802" />
|
||||
|
||||
<!-- 2023 R2 gRPC transport (cross-platform) -->
|
||||
<PackageVersion Include="Google.Protobuf" Version="3.24.4" />
|
||||
<PackageVersion Include="Grpc.Net.Client" Version="2.58.0" />
|
||||
<PackageVersion Include="Grpc.Net.Client.Web" Version="2.58.0" />
|
||||
<PackageVersion Include="Grpc.Tools" Version="2.59.0" />
|
||||
|
||||
<!-- ZB-idiomatic DI extension (only non-BCL lib dependency) -->
|
||||
<PackageVersion Include="Microsoft.Extensions.DependencyInjection.Abstractions" Version="10.0.7" />
|
||||
|
||||
<!-- Test -->
|
||||
<PackageVersion Include="Microsoft.NET.Test.Sdk" Version="17.14.1" />
|
||||
<PackageVersion Include="xunit" Version="2.9.3" />
|
||||
<PackageVersion Include="xunit.runner.visualstudio" Version="3.1.4" />
|
||||
<PackageVersion Include="coverlet.collector" Version="6.0.4" />
|
||||
<PackageVersion Include="Microsoft.Data.SqlClient" Version="6.0.2" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
```
|
||||
|
||||
**Step 3: `.gitignore`**
|
||||
|
||||
```gitignore
|
||||
bin/
|
||||
obj/
|
||||
# identity-bearing / non-redistributable — never commit
|
||||
*.ndjson
|
||||
current/
|
||||
aveva-install-*/
|
||||
```
|
||||
|
||||
**Step 4: `ZB.MOM.WW.SPHistorianClient.slnx`**
|
||||
|
||||
```xml
|
||||
<Solution>
|
||||
<Folder Name="/src/">
|
||||
<Project Path="src/ZB.MOM.WW.SPHistorianClient/ZB.MOM.WW.SPHistorianClient.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/tests/">
|
||||
<Project Path="tests/ZB.MOM.WW.SPHistorianClient.Tests/ZB.MOM.WW.SPHistorianClient.Tests.csproj" />
|
||||
</Folder>
|
||||
</Solution>
|
||||
```
|
||||
|
||||
**Step 5: `src/ZB.MOM.WW.SPHistorianClient/ZB.MOM.WW.SPHistorianClient.csproj`**
|
||||
|
||||
(Derived from the bundle's `AVEVA.Historian.Client.csproj`: inline versions removed for central
|
||||
management; ZB package metadata added; `InternalsVisibleTo` retargeted to the ZB test assembly and
|
||||
the `…ReverseEngineering` one dropped; proto glob uses forward slashes for cross-platform MSBuild.)
|
||||
|
||||
```xml
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
|
||||
<PropertyGroup>
|
||||
<IsPackable>true</IsPackable>
|
||||
<PackageId>ZB.MOM.WW.SPHistorianClient</PackageId>
|
||||
<Authors>ZB.MOM.WW</Authors>
|
||||
<Description>Pure-managed .NET 10 client for AVEVA System Platform Historian (Wonderware) for the ZB.MOM.WW SCADA family. The wire protocol is reverse-engineered and re-implemented in C# — no native AVEVA runtime dependency. Surfaces history reads (raw / aggregate / at-time / event), tag browse + metadata, status, and tag create/delete over the WCF/MDAS transports (Windows) plus a cross-platform gRPC transport for 2023 R2.</Description>
|
||||
<PackageTags>aveva;wonderware;historian;system-platform;scada;timeseries;grpc;wcf;zb-mom-ww</PackageTags>
|
||||
<PackageProjectUrl>https://gitea.dohertylan.com/dohertj2/zb-mom-ww-sphistorianclient</PackageProjectUrl>
|
||||
<RepositoryUrl>https://gitea.dohertylan.com/dohertj2/zb-mom-ww-sphistorianclient</RepositoryUrl>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="System.Security.Cryptography.Xml" />
|
||||
<PackageReference Include="System.ServiceModel.NetNamedPipe" />
|
||||
<PackageReference Include="System.ServiceModel.NetTcp" />
|
||||
<PackageReference Include="Microsoft.Extensions.DependencyInjection.Abstractions" />
|
||||
</ItemGroup>
|
||||
|
||||
<!-- 2023 R2 gRPC transport (RemoteGrpc). Pure-managed: Grpc.Net.Client + Google.Protobuf.
|
||||
Grpc.Tools is build-only (PrivateAssets=all) and generates the client stubs from the
|
||||
recovered contract under Grpc/Protos at build. -->
|
||||
<ItemGroup>
|
||||
<PackageReference Include="Google.Protobuf" />
|
||||
<PackageReference Include="Grpc.Net.Client" />
|
||||
<PackageReference Include="Grpc.Net.Client.Web" />
|
||||
<PackageReference Include="Grpc.Tools">
|
||||
<PrivateAssets>all</PrivateAssets>
|
||||
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
|
||||
</PackageReference>
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<Protobuf Include="Grpc/Protos/*.proto" GrpcServices="Client" ProtoRoot="Grpc/Protos" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<AssemblyAttribute Include="System.Runtime.CompilerServices.InternalsVisibleToAttribute">
|
||||
<_Parameter1>ZB.MOM.WW.SPHistorianClient.Tests</_Parameter1>
|
||||
</AssemblyAttribute>
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
```
|
||||
|
||||
**Step 6: `tests/ZB.MOM.WW.SPHistorianClient.Tests/ZB.MOM.WW.SPHistorianClient.Tests.csproj`**
|
||||
|
||||
```xml
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
|
||||
<PropertyGroup>
|
||||
<IsPackable>false</IsPackable>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="coverlet.collector" />
|
||||
<PackageReference Include="Microsoft.NET.Test.Sdk" />
|
||||
<PackageReference Include="Microsoft.Data.SqlClient" />
|
||||
<PackageReference Include="xunit" />
|
||||
<PackageReference Include="xunit.runner.visualstudio" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<Using Include="Xunit" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\..\src\ZB.MOM.WW.SPHistorianClient\ZB.MOM.WW.SPHistorianClient.csproj" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
```
|
||||
|
||||
**Step 7: Verify the skeleton is well-formed (build will fail — no sources yet — that is expected)**
|
||||
|
||||
Run: `cd ZB.MOM.WW.SPHistorianClient && dotnet restore ZB.MOM.WW.SPHistorianClient.slnx`
|
||||
Expected: restore **succeeds** (proves the props/csproj XML and central package versions resolve). A
|
||||
follow-up `dotnet build` would fail only because no `.cs` exist yet — do not build here.
|
||||
|
||||
**Step 8: Commit**
|
||||
|
||||
```bash
|
||||
cd /Users/dohertj2/Desktop/scadaproj
|
||||
git add ZB.MOM.WW.SPHistorianClient/
|
||||
git commit -m "feat(sphistorianclient): scaffold shared library skeleton (props, csprojs, slnx)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Port source + tests with namespace rewrite
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~4 min
|
||||
**Parallelizable with:** none
|
||||
**Blocked by:** Task 1
|
||||
|
||||
**Files:**
|
||||
- Create (scripted copy): `ZB.MOM.WW.SPHistorianClient/src/ZB.MOM.WW.SPHistorianClient/**/*.{cs,proto}` (74 `.cs` + 6 `.proto`)
|
||||
- Create (scripted copy): `ZB.MOM.WW.SPHistorianClient/tests/ZB.MOM.WW.SPHistorianClient.Tests/**/*.cs` (25 `.cs`)
|
||||
|
||||
This task is a deterministic copy + namespace rewrite — run the script, then verify counts.
|
||||
|
||||
**Step 1: Copy + rewrite (single script)**
|
||||
|
||||
```bash
|
||||
set -euo pipefail
|
||||
BUNDLE=/tmp/histsdk/extracted/histsdk-migration/histsdk
|
||||
DEST=/Users/dohertj2/Desktop/scadaproj/ZB.MOM.WW.SPHistorianClient
|
||||
|
||||
# Guard: re-extract if /tmp was cleaned
|
||||
if [ ! -d "$BUNDLE/src/AVEVA.Historian.Client" ]; then
|
||||
cd /tmp/histsdk && unzip -o -q histsdk-migration.zip -d extracted
|
||||
fi
|
||||
|
||||
# --- src: copy .cs + .proto, preserving subdirs (NOT the old .csproj) ---
|
||||
SRC="$DEST/src/ZB.MOM.WW.SPHistorianClient"
|
||||
cd "$BUNDLE/src/AVEVA.Historian.Client"
|
||||
find . \( -name '*.cs' -o -name '*.proto' \) | while read -r f; do
|
||||
mkdir -p "$SRC/$(dirname "$f")"
|
||||
cp "$f" "$SRC/$f"
|
||||
done
|
||||
|
||||
# --- tests: copy .cs only (NOT the old .csproj) ---
|
||||
TST="$DEST/tests/ZB.MOM.WW.SPHistorianClient.Tests"
|
||||
cd "$BUNDLE/tests/AVEVA.Historian.Client.Tests"
|
||||
find . -name '*.cs' | while read -r f; do
|
||||
mkdir -p "$TST/$(dirname "$f")"
|
||||
cp "$f" "$TST/$f"
|
||||
done
|
||||
|
||||
# --- namespace rewrite in .cs ONLY (proto wire contracts stay ArchestrA.Grpc.Contract.*) ---
|
||||
find "$SRC" "$TST" -name '*.cs' -print0 \
|
||||
| xargs -0 sed -i '' 's/AVEVA\.Historian\.Client/ZB.MOM.WW.SPHistorianClient/g'
|
||||
```
|
||||
|
||||
**Step 2: Verify counts and that the rename is total**
|
||||
|
||||
```bash
|
||||
DEST=/Users/dohertj2/Desktop/scadaproj/ZB.MOM.WW.SPHistorianClient
|
||||
echo "src cs: $(find "$DEST/src" -name '*.cs' | wc -l) (expect 74)"
|
||||
echo "src proto: $(find "$DEST/src" -name '*.proto' | wc -l) (expect 6)"
|
||||
echo "test cs: $(find "$DEST/tests" -name '*.cs' | wc -l) (expect 25)"
|
||||
echo "leftover AVEVA.Historian.Client in .cs: $(grep -rl 'AVEVA\.Historian\.Client' "$DEST" --include='*.cs' | wc -l) (expect 0)"
|
||||
echo "proto namespace preserved: $(grep -l 'ArchestrA.Grpc.Contract' "$DEST"/src/ZB.MOM.WW.SPHistorianClient/Grpc/Protos/*.proto | wc -l) (expect 6)"
|
||||
```
|
||||
|
||||
Expected: `74`, `6`, `25`, `0`, `6`. If "leftover" is non-zero, inspect those files — the only legitimate
|
||||
remaining mentions would be inside comments/strings that happen to differ in casing/spacing; a clean
|
||||
port should show `0`.
|
||||
|
||||
**Step 3: Commit**
|
||||
|
||||
```bash
|
||||
cd /Users/dohertj2/Desktop/scadaproj
|
||||
git add ZB.MOM.WW.SPHistorianClient/src ZB.MOM.WW.SPHistorianClient/tests
|
||||
git commit -m "feat(sphistorianclient): port SDK source + tests, rebrand namespace to ZB.MOM.WW.SPHistorianClient"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Build + test green
|
||||
|
||||
**Classification:** high-risk
|
||||
**Estimated implement time:** ~5 min (plus restore/build wall-time)
|
||||
**Parallelizable with:** none
|
||||
**Blocked by:** Task 2
|
||||
|
||||
This is the integration gate. The port must compile and the offline test suite must pass on this macOS host.
|
||||
|
||||
**Files:**
|
||||
- Modify (only if the build surfaces a defect): any ported file under `ZB.MOM.WW.SPHistorianClient/src` or `…/tests`, or the two `.csproj`.
|
||||
|
||||
**Step 1: Build**
|
||||
|
||||
Run: `cd ZB.MOM.WW.SPHistorianClient && dotnet build ZB.MOM.WW.SPHistorianClient.slnx`
|
||||
Expected: **Build succeeded.** Platform-compatibility (CAxxxx `[SupportedOSPlatform("windows")]`) warnings
|
||||
are acceptable and must remain warnings. If restore hard-fails on `NU1903`, re-run with
|
||||
`-p:NuGetAudit=false`.
|
||||
|
||||
**Step 2: Test**
|
||||
|
||||
Run: `dotnet test ZB.MOM.WW.SPHistorianClient.slnx`
|
||||
Expected: all tests pass; the live integration tests (`HistorianClientIntegrationTests`,
|
||||
`HistorianGrpcIntegrationTests`, `RemoteTcpIntegrationTests`) **skip cleanly** because no `HISTORIAN_*`
|
||||
env vars are set. The bundle's `MIGRATION-README.md` documents ~188 tests passing on macOS with the
|
||||
live ones skipped — treat a comparable count with **zero failures** as success.
|
||||
|
||||
**Step 3: Triage rules (if not green)**
|
||||
- Compile error referencing `AVEVA.Historian.Client` → a file was missed by the rewrite; re-run the
|
||||
Task 2 sed on that file.
|
||||
- `NU1008` (version on PackageReference) → an inline `Version=` slipped into a `.csproj`; remove it
|
||||
(version belongs in `Directory.Packages.props`).
|
||||
- Missing generated gRPC type (e.g. `ArchestrA.Grpc.Contract.*` not found) → confirm the `<Protobuf>`
|
||||
glob in the src `.csproj` resolves the 6 `Grpc/Protos/*.proto` and that `Grpc.Tools` restored.
|
||||
- A genuine test failure (not a skip) → this is a real port defect; fix the ported code, do **not**
|
||||
delete/weaken the test.
|
||||
|
||||
**Step 4: Commit (only if Step 3 required edits)**
|
||||
|
||||
```bash
|
||||
git add -A ZB.MOM.WW.SPHistorianClient/
|
||||
git commit -m "fix(sphistorianclient): resolve port build/test fallout"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Add the `AddZbSpHistorianClient` DI extension (TDD)
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~4 min
|
||||
**Parallelizable with:** Task 5
|
||||
**Blocked by:** Task 3
|
||||
|
||||
`HistorianClientOptions` uses `required` + `init`-only properties, so the extension takes a fully-built
|
||||
options instance (not an `Action<T>` configurator). It depends only on
|
||||
`Microsoft.Extensions.DependencyInjection.Abstractions`.
|
||||
|
||||
**Files:**
|
||||
- Create: `ZB.MOM.WW.SPHistorianClient/src/ZB.MOM.WW.SPHistorianClient/DependencyInjection/ZbSpHistorianClientServiceCollectionExtensions.cs`
|
||||
- Test: `ZB.MOM.WW.SPHistorianClient/tests/ZB.MOM.WW.SPHistorianClient.Tests/DependencyInjectionTests.cs`
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
```csharp
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using ZB.MOM.WW.SPHistorianClient;
|
||||
|
||||
namespace ZB.MOM.WW.SPHistorianClient.Tests;
|
||||
|
||||
public class DependencyInjectionTests
|
||||
{
|
||||
[Fact]
|
||||
public void AddZbSpHistorianClient_resolves_client_and_options()
|
||||
{
|
||||
var services = new ServiceCollection();
|
||||
var options = new HistorianClientOptions { Host = "localhost" };
|
||||
|
||||
services.AddZbSpHistorianClient(options);
|
||||
|
||||
using var sp = services.BuildServiceProvider();
|
||||
Assert.Same(options, sp.GetRequiredService<HistorianClientOptions>());
|
||||
Assert.NotNull(sp.GetRequiredService<HistorianClient>());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void AddZbSpHistorianClient_throws_when_host_missing()
|
||||
{
|
||||
var services = new ServiceCollection();
|
||||
var options = new HistorianClientOptions { Host = "" };
|
||||
|
||||
Assert.Throws<ArgumentException>(() => services.AddZbSpHistorianClient(options));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void AddZbSpHistorianClient_throws_on_null_options()
|
||||
{
|
||||
var services = new ServiceCollection();
|
||||
Assert.Throws<ArgumentNullException>(() => services.AddZbSpHistorianClient(null!));
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Step 2: Run — verify it fails to compile** (`AddZbSpHistorianClient` not defined)
|
||||
|
||||
Run: `dotnet test ZB.MOM.WW.SPHistorianClient.slnx --filter "FullyQualifiedName~DependencyInjectionTests"`
|
||||
Expected: FAIL (does not compile / method missing).
|
||||
|
||||
**Step 3: Implement**
|
||||
|
||||
```csharp
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
|
||||
namespace ZB.MOM.WW.SPHistorianClient;
|
||||
|
||||
/// <summary>
|
||||
/// ZB.MOM.WW DI registration for <see cref="HistorianClient"/>. Mirrors the family's
|
||||
/// <c>AddZb*</c> convention. Because <see cref="HistorianClientOptions"/> is <c>required</c>/
|
||||
/// <c>init</c>-only, callers pass a fully-built options instance (bind it from configuration in the
|
||||
/// consuming app, e.g. <c>config.GetSection("Historian").Get<HistorianClientOptions>()</c>).
|
||||
/// </summary>
|
||||
public static class ZbSpHistorianClientServiceCollectionExtensions
|
||||
{
|
||||
public static IServiceCollection AddZbSpHistorianClient(
|
||||
this IServiceCollection services,
|
||||
HistorianClientOptions options)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(services);
|
||||
ArgumentNullException.ThrowIfNull(options);
|
||||
if (string.IsNullOrWhiteSpace(options.Host))
|
||||
{
|
||||
throw new ArgumentException(
|
||||
"HistorianClientOptions.Host must be set.", nameof(options));
|
||||
}
|
||||
|
||||
services.AddSingleton(options);
|
||||
// HistorianClient opens a fresh channel per operation and has a no-op DisposeAsync,
|
||||
// so transient is safe and avoids assuming the shared dialect is concurrency-safe.
|
||||
services.AddTransient<HistorianClient>();
|
||||
return services;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Step 4: Run — verify pass**
|
||||
|
||||
Run: `dotnet test ZB.MOM.WW.SPHistorianClient.slnx --filter "FullyQualifiedName~DependencyInjectionTests"`
|
||||
Expected: PASS (3/3).
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add ZB.MOM.WW.SPHistorianClient/src/ZB.MOM.WW.SPHistorianClient/DependencyInjection \
|
||||
ZB.MOM.WW.SPHistorianClient/tests/ZB.MOM.WW.SPHistorianClient.Tests/DependencyInjectionTests.cs
|
||||
git commit -m "feat(sphistorianclient): add AddZbSpHistorianClient DI extension"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Author `CLAUDE.md` + `README.md`
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~4 min
|
||||
**Parallelizable with:** Task 4
|
||||
**Blocked by:** Task 3
|
||||
|
||||
Sanitized docs only — **no hostnames, credentials, customer tag names, or capture data.** Model the
|
||||
structure on `ZB.MOM.WW.Telemetry/CLAUDE.md` (overview, package table, build/test/pack commands,
|
||||
status) but adapt to a single-package library.
|
||||
|
||||
**Files:**
|
||||
- Create: `ZB.MOM.WW.SPHistorianClient/CLAUDE.md`
|
||||
- Create: `ZB.MOM.WW.SPHistorianClient/README.md`
|
||||
|
||||
**`CLAUDE.md` must cover:**
|
||||
- One-paragraph overview: pure-managed .NET 10 AVEVA System Platform Historian client, no native AVEVA
|
||||
dependency, reverse-engineered wire protocol. Ported from the `histsdk` migration bundle.
|
||||
- The supported operation surface table (copy the README table from the bundle:
|
||||
`ProbeAsync`, `ReadRawAsync`, `ReadAggregateAsync` (16 modes), `ReadAtTimeAsync`, `ReadEventsAsync`,
|
||||
`BrowseTagNamesAsync`, `GetTagMetadataAsync`, `GetConnectionStatusAsync`,
|
||||
`GetStoreForwardStatusAsync`, `GetSystemParameterAsync`, `EnsureTagAsync`, `DeleteTagAsync`).
|
||||
- Transport matrix: `LocalPipe` / `RemoteTcpIntegrated` / `RemoteTcpCertificate` (WCF, Windows-only,
|
||||
live-verified) vs `RemoteGrpc` (2023 R2, cross-platform, **not yet live-verified**).
|
||||
- Out of scope: writing samples (`AddS2` architecturally blocked), discrete/string tag creation.
|
||||
- DI: the `AddZbSpHistorianClient(options)` extension + the bind-from-config note.
|
||||
- Build/test/pack commands (from this dir):
|
||||
`dotnet build ZB.MOM.WW.SPHistorianClient.slnx` / `dotnet test …` /
|
||||
`dotnet pack ZB.MOM.WW.SPHistorianClient.slnx -c Release -o ./artifacts`.
|
||||
- Live integration tests gated by `HISTORIAN_*` env vars (skip cleanly when unset). List the env vars.
|
||||
|
||||
**`README.md`:** a trimmed public-facing version — overview, quick-start snippet (the bundle's
|
||||
`HistorianClient` usage example, namespace updated to `ZB.MOM.WW.SPHistorianClient`), supported surface
|
||||
table, build/test commands.
|
||||
|
||||
**Commit:**
|
||||
|
||||
```bash
|
||||
git add ZB.MOM.WW.SPHistorianClient/CLAUDE.md ZB.MOM.WW.SPHistorianClient/README.md
|
||||
git commit -m "docs(sphistorianclient): add CLAUDE.md + README.md"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Pack verification
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~3 min
|
||||
**Parallelizable with:** none
|
||||
**Blocked by:** Task 4, Task 5
|
||||
|
||||
**Files:**
|
||||
- Create (build output): `ZB.MOM.WW.SPHistorianClient/artifacts/ZB.MOM.WW.SPHistorianClient.0.1.0.nupkg`
|
||||
|
||||
**Step 1: Full green build + test once more, then pack**
|
||||
|
||||
```bash
|
||||
cd /Users/dohertj2/Desktop/scadaproj/ZB.MOM.WW.SPHistorianClient
|
||||
dotnet test ZB.MOM.WW.SPHistorianClient.slnx
|
||||
dotnet pack ZB.MOM.WW.SPHistorianClient.slnx -c Release -o ./artifacts
|
||||
```
|
||||
|
||||
Expected: tests pass (live ones skip); pack produces `artifacts/ZB.MOM.WW.SPHistorianClient.0.1.0.nupkg`.
|
||||
|
||||
**Step 2: Sanity-check the package contents**
|
||||
|
||||
```bash
|
||||
unzip -l artifacts/ZB.MOM.WW.SPHistorianClient.0.1.0.nupkg | grep -E 'ZB.MOM.WW.SPHistorianClient.dll|.nuspec'
|
||||
```
|
||||
Expected: the lib DLL and nuspec are present.
|
||||
|
||||
**Step 3: Commit the nupkg** (matches the family convention — `ZB.MOM.WW.Telemetry` commits its `artifacts/*.nupkg`)
|
||||
|
||||
```bash
|
||||
cd /Users/dohertj2/Desktop/scadaproj
|
||||
git add -f ZB.MOM.WW.SPHistorianClient/artifacts/ZB.MOM.WW.SPHistorianClient.0.1.0.nupkg
|
||||
git commit -m "build(sphistorianclient): pack 0.1.0 nupkg"
|
||||
```
|
||||
|
||||
> **Do NOT push or publish** to the Gitea feed. Per repo experience, "published/adopted" claims must
|
||||
> not be made without explicit user direction + feed verification.
|
||||
|
||||
---
|
||||
|
||||
## Task 7: Index the new library in the umbrella `CLAUDE.md` (optional)
|
||||
|
||||
**Classification:** trivial
|
||||
**Estimated implement time:** ~2 min
|
||||
**Parallelizable with:** none
|
||||
**Blocked by:** Task 6
|
||||
|
||||
**Files:**
|
||||
- Modify: `CLAUDE.md` (repo root umbrella index)
|
||||
|
||||
Add a short reference so the umbrella index reflects the newly-hosted library (the intro paragraph
|
||||
that enumerates the hosted `ZB.MOM.WW.*` sources, and/or a one-line pointer near the component table
|
||||
noting `ZB.MOM.WW.SPHistorianClient` is a net-new shared library — **not** a component normalization).
|
||||
|
||||
> **Caveat:** repo-root `CLAUDE.md` already has **pre-existing uncommitted edits** (unrelated to this
|
||||
> work). Before editing, run `git diff CLAUDE.md` and make sure your commit message reflects that it
|
||||
> may bundle those edits — or stage only the hunks you add. If this risks entangling unrelated changes,
|
||||
> skip this task and leave it for the user.
|
||||
|
||||
**Commit:**
|
||||
|
||||
```bash
|
||||
git add CLAUDE.md
|
||||
git commit -m "docs: index ZB.MOM.WW.SPHistorianClient in umbrella CLAUDE.md"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Done criteria
|
||||
|
||||
- `ZB.MOM.WW.SPHistorianClient/` exists with `src/`, `tests/`, props, `.slnx`, `CLAUDE.md`, `README.md`.
|
||||
- `dotnet build` + `dotnet test` are green on macOS (live integration tests skip cleanly).
|
||||
- `AddZbSpHistorianClient` DI extension present + tested.
|
||||
- `artifacts/ZB.MOM.WW.SPHistorianClient.0.1.0.nupkg` produced.
|
||||
- All work committed on `feat/sphistorianclient`. Not pushed/published.
|
||||
@@ -0,0 +1,13 @@
|
||||
{
|
||||
"planPath": "docs/plans/2026-06-19-sphistorianclient.md",
|
||||
"tasks": [
|
||||
{"id": 1, "subject": "Task 1: Scaffold library skeleton", "status": "completed"},
|
||||
{"id": 2, "subject": "Task 2: Port source + tests with namespace rewrite", "status": "completed", "blockedBy": [1]},
|
||||
{"id": 3, "subject": "Task 3: Build + test green", "status": "completed", "blockedBy": [2]},
|
||||
{"id": 4, "subject": "Task 4: Add AddZbSpHistorianClient DI extension (TDD)", "status": "completed", "blockedBy": [3]},
|
||||
{"id": 5, "subject": "Task 5: Author CLAUDE.md + README.md", "status": "completed", "blockedBy": [3]},
|
||||
{"id": 6, "subject": "Task 6: Pack verification", "status": "completed", "blockedBy": [4, 5]},
|
||||
{"id": 7, "subject": "Task 7: Index new lib in umbrella CLAUDE.md (optional)", "status": "completed", "blockedBy": [6]}
|
||||
],
|
||||
"lastUpdated": "2026-06-19"
|
||||
}
|
||||
@@ -0,0 +1,216 @@
|
||||
# ZB.MOM.WW.HistorianGateway — Design
|
||||
|
||||
**Date:** 2026-06-23
|
||||
**Status:** Design approved (brainstorming complete) — ready for implementation planning
|
||||
**Author:** brainstorming session (Joseph Doherty + Claude)
|
||||
|
||||
## 1. Summary
|
||||
|
||||
A new **full-feature sidecar** in the SCADA/OT sister-project family, modelled on
|
||||
`MxAccessGateway` (`mxaccessgw`). It does two things:
|
||||
|
||||
1. **Read-only Galaxy metadata server** — exposes the AVEVA System Platform
|
||||
("Wonderware") **Galaxy object hierarchy** (areas / objects / templates /
|
||||
instances / attributes), sourced from the **Galaxy Repository SQL DB**, the same
|
||||
data `mxaccessgw`'s `galaxy_repository` feature serves today.
|
||||
2. **Full read/write gRPC API to the AVEVA (Wonderware) Historian** — reads (raw,
|
||||
aggregate with all 15 retrieval modes, at-time, blocks, events, browse, metadata,
|
||||
status) **and writes** (historical/backfill values, event send, tag-config
|
||||
create/delete/rename/extended-properties, plus resilience helpers).
|
||||
|
||||
It reuses the family's **common shared packages and styles**: `ZB.MOM.WW.Auth`,
|
||||
`ZB.MOM.WW.Theme`, `ZB.MOM.WW.Telemetry`(+`.Serilog`), `ZB.MOM.WW.Health`,
|
||||
`ZB.MOM.WW.Configuration`, `ZB.MOM.WW.Audit`.
|
||||
|
||||
### Key reframing discovered during brainstorming
|
||||
|
||||
- The historian write surface lives in the **`histsdk`** repo
|
||||
(`gitea.dohertylan.com/dohertj2/histsdk`, namespace `AVEVA.Historian.Client`),
|
||||
which is **far ahead** of the stale `scadaproj/ZB.MOM.WW.SPHistorianClient` port
|
||||
(2026-06-19 snapshot, reads + tag create/delete only, value-writes marked
|
||||
"blocked"). `histsdk` has since added a **live-validated gRPC write surface**:
|
||||
`AddHistoricalValuesAsync` (historical/backfill, gRPC-only), `SendEventAsync`
|
||||
(events, both transports), `EnsureTags`/`DeleteTags`/`RenameTags`/
|
||||
`AddTagExtendedProperties` (config writes, gRPC), plus higher-level
|
||||
`HistorianStoreForwardWriter` (durable outbox) and a redundant-write cluster.
|
||||
- **Hard server-side limit no client can lift:** `AddS2` *streaming live
|
||||
process-sample* writes are GATED — the historian runtime cache only ingests from
|
||||
configured IOServer / Application Server pipelines. "Live current value" writes are
|
||||
therefore done via the **SQL path** (`aaAnalogTagInsert` → `INSERT INTO History`),
|
||||
not gRPC.
|
||||
- **No COM anywhere.** The historian SDK is pure-managed and Galaxy browse is plain
|
||||
SQL, so — unlike `mxaccessgw` — this sidecar needs **no x86 worker and no
|
||||
two-process split**. It is a single .NET 10 x64 process.
|
||||
|
||||
## 2. Decisions (locked during brainstorming)
|
||||
|
||||
| Decision | Choice |
|
||||
|---|---|
|
||||
| Purpose | **General-purpose gateway** — reusable modern façade over Historian + Galaxy metadata for any gRPC client (the way `mxaccessgw` is the façade for MXAccess). |
|
||||
| Historian client code | **Vendor `histsdk`** (`AVEVA.Historian.Client`) into the sidecar repo as self-contained vendored source; namespace kept as-is to ease future re-sync. |
|
||||
| Galaxy browse code | **New shared lib `ZB.MOM.WW.GalaxyRepository`** in scadaproj, extracted from `mxaccessgw`'s `galaxy_repository` browse; consumed by both `mxaccessgw` and this sidecar. |
|
||||
| Dashboard | **Full Blazor dashboard** on `ZB.MOM.WW.Theme` (login + Galaxy browser + Historian console + API-key admin + status/health). |
|
||||
| Historian write scope | **All:** historical/backfill writes, event send, tag-config writes, **and** resilience extras (store-and-forward outbox, redundant write fan-out, SQL live-value path). |
|
||||
| Connection model | **Approach A — stateless gateway over pooled service-identity connections.** Clients authenticate to the gateway (ZB API key); the gateway owns historian credentials and reuses pooled, pre-authenticated connections. |
|
||||
| Repo location | **New standalone sibling repo** `~/Desktop/HistorianGateway`, gitea remote `historiangw`. |
|
||||
| Name / namespace | **`ZB.MOM.WW.HistorianGateway`**. |
|
||||
|
||||
## 3. Architecture & solution structure
|
||||
|
||||
Single .NET 10 x64 ASP.NET Core process.
|
||||
|
||||
```
|
||||
~/Desktop/HistorianGateway/
|
||||
src/
|
||||
ZB.MOM.WW.HistorianGateway.Server/ ASP.NET Core host: gRPC services + Blazor dashboard + /healthz + /metrics
|
||||
ZB.MOM.WW.HistorianGateway.Contracts/ the gateway's own .proto + generated types (for client codegen distribution)
|
||||
vendor/AVEVA.Historian.Client/ VENDORED from histsdk; ArchestrA.Grpc.Contract.* protos + reads/writes/store-forward/redundancy
|
||||
tests/
|
||||
ZB.MOM.WW.HistorianGateway.Tests/ unit + env-gated live integration + bUnit dashboard
|
||||
docs/plans/, CLAUDE.md, README.md
|
||||
ZB.MOM.WW.HistorianGateway.slnx
|
||||
```
|
||||
|
||||
**Cross-repo pieces:**
|
||||
|
||||
- **`scadaproj/ZB.MOM.WW.GalaxyRepository`** (new shared lib, plain files — NOT a
|
||||
nested git repo): carries the **canonical `galaxy_repository.proto`** (adopted from
|
||||
`mxaccessgw`'s existing contract so OtOpcUa's wire shape is not broken), the SQL
|
||||
browse provider (connect to Galaxy Repository SQL → hierarchy model), and a
|
||||
reusable gRPC service implementation both hosts can `MapGrpcService<>()`.
|
||||
`mxaccessgw` adopting it is a **tracked follow-on** (same "built → adopted" pattern
|
||||
as the other normalized components); this sidecar consumes it from the start.
|
||||
- **Shared ZB packages consumed:** `ZB.MOM.WW.Auth`
|
||||
(Abstractions+Ldap+ApiKeys+AspNetCore), `ZB.MOM.WW.Theme`, `ZB.MOM.WW.Telemetry`
|
||||
(+`.Serilog`), `ZB.MOM.WW.Health`, `ZB.MOM.WW.Configuration`, `ZB.MOM.WW.Audit`.
|
||||
|
||||
## 4. gRPC API surface
|
||||
|
||||
Gateway's own curated contract (`ZB.MOM.WW.HistorianGateway.Grpc.V1`), grouped by
|
||||
concern — not a 1:1 SDK dump. The vendored `ArchestrA.Grpc.Contract.*` protos stay
|
||||
internal; clients see only the gateway contract.
|
||||
|
||||
| Service | RPCs | Notes |
|
||||
|---|---|---|
|
||||
| `HistorianRead` | `ReadRaw`, `ReadAggregate`, `ReadBlocks`, `ReadEvents` *(server-streaming)*; `ReadAtTime` | `ReadAggregate` exposes all 15 retrieval modes |
|
||||
| `HistorianWrite` | `AddHistoricalValues`, `SendEvent`, `WriteLiveValues` | `WriteLiveValues` = SQL path (gRPC streaming is gated) |
|
||||
| `HistorianTags` | `BrowseTagNames` *(streaming)*, `GetTagMetadata`, `EnsureTags`, `DeleteTags`, `RenameTags`, `AddTagExtendedProperties` | |
|
||||
| `HistorianStatus` | `Probe`, `GetConnectionStatus`, `GetStoreForwardStatus`, `GetSystemParameter` | |
|
||||
| `GalaxyRepository` | Browse areas / objects / templates / instances / attributes *(read-only)* | canonical proto from the shared lib |
|
||||
|
||||
**Authorization** is via **API-key scopes at the gateway** (Approach A trust
|
||||
boundary): `historian:read`, `historian:write`, `historian:tags:write`,
|
||||
`galaxy:read`.
|
||||
|
||||
## 5. Connection & data flow
|
||||
|
||||
```
|
||||
gRPC client ──(ZB API key)──► HistorianGateway ──┬─ pooled, pre-authed gRPC conn ──► AVEVA Historian (RemoteGrpc 2023R2)
|
||||
├─ store-forward outbox (SQLite) ─ replays writes on reconnect
|
||||
├─ redundant-write fan-out ──────► historian members (All/Any ack)
|
||||
├─ SqlConnection ──► Runtime DB (live-value writes via aaAnalogTagInsert/History)
|
||||
└─ ZB.MOM.WW.GalaxyRepository ──► Galaxy Repository SQL (read-only browse)
|
||||
```
|
||||
|
||||
- **Pooled connections:** the expensive auth handshake (`ValidateClientCredential` /
|
||||
ECDH `ExchangeKey`) runs **once per connection on open**, then is reused across
|
||||
requests; connections are health-checked with auto-reconnect. Write operations use
|
||||
the write-enabled session mode (`0x401`).
|
||||
- **Store-forward:** writes flow through the SDK's `HistorianStoreForwardWriter` — on
|
||||
an unreachable historian, enqueue to durable SQLite; a background drain replays on
|
||||
reconnect.
|
||||
- **Redundancy:** `HistorianRedundantWriteResult` fan-out to configured members under
|
||||
an All/Any ack policy; per-member result surfaced to the caller.
|
||||
- **SQL live-write** and **Galaxy browse** are independent SQL paths, each with its
|
||||
own validated connection config.
|
||||
|
||||
**Configuration** (all `ZB.MOM.WW.Configuration`-validated, aggregated by
|
||||
`ConfigPreflight` at startup): historian (host, gRPC port 32565, transport=RemoteGrpc,
|
||||
TLS, service identity/credentials), redundant members, store-forward path, Galaxy
|
||||
Repository SQL connection string, Runtime DB connection string (SQL live-write),
|
||||
Auth (LDAP + API-key pepper). Secrets live in the operator environment, never in repo.
|
||||
|
||||
## 6. Cross-cutting infrastructure + dashboard
|
||||
|
||||
- **Auth (`ZB.MOM.WW.Auth`):** gRPC clients use peppered-HMAC API keys (keyId/Bearer),
|
||||
validated by a gRPC interceptor enforcing per-service scopes. Dashboard uses LDAP
|
||||
login (`.Ldap`+`.AspNetCore`), cookie auth, `IGroupRoleMapper<TRole>`, canonical
|
||||
`ZbClaimTypes`/`ZbCookieDefaults`, canonical-six roles, dev against the shared
|
||||
GLAuth (`10.100.0.35:3893`, `dc=zb,dc=local`). `DisableLogin` dev/deploy switch.
|
||||
- **Telemetry (`.Telemetry`+`.Serilog`):** `AddZbTelemetry` (Resource
|
||||
`service.name=historian-gateway` + standard instrumentation + always-on Prometheus
|
||||
`/metrics`, OTLP opt-in) + `AddZbSerilog`. App Meters: read/write counts + latency,
|
||||
store-forward queue depth, pool connection state, redundancy ack outcomes.
|
||||
- **Health (`.Health`):** three-tier ready/active/healthz + canonical JSON writer.
|
||||
Probes: historian gRPC (`GrpcDependencyHealthCheck`), Galaxy Repository SQL +
|
||||
Runtime DB (`DatabaseHealthCheck`), store-forward drain status.
|
||||
- **Configuration (`.Configuration`):** `OptionsValidatorBase` / `ValidationBuilder` /
|
||||
`AddValidatedOptions` / `ConfigPreflight` (§5).
|
||||
- **Audit (`.Audit`, DEEP-adopt):** canonical `AuditEvent` + SQLite `IAuditWriter`
|
||||
(MxGateway-style). Audited: tag-config writes, historical/event writes, API-key
|
||||
admin, login/logout. `Actor` wired from the Auth principal via `IAuditActorAccessor`.
|
||||
- **Dashboard (Blazor, `.Theme`):** Technical-Light side-rail shell + `LoginCard`
|
||||
`/login`. Pages: **Status** (pool / store-forward / redundancy / version),
|
||||
**Galaxy browser** (read-only hierarchy tree), **Historian console** (query with
|
||||
raw/aggregate + mode picker + time range; role-gated write test for value insert /
|
||||
event send), **API-key admin** (list/create/revoke keys + scopes), **Health**.
|
||||
|
||||
## 7. Error handling
|
||||
|
||||
- **gRPC status mapping:** `ProtocolEvidenceMissingException` (unsupported op/type —
|
||||
e.g. non-analog tag, non-string event property) → `Unimplemented`/`FailedPrecondition`
|
||||
with a clear "not in reverse-engineered surface" message; auth →
|
||||
`Unauthenticated`/`PermissionDenied`; historian down → `Unavailable`; bad range /
|
||||
unknown tag → `InvalidArgument`/`NotFound`.
|
||||
- **Gated ops:** live streaming-sample writes (`AddS2`) are **not exposed** (no RPC);
|
||||
live-value writes route through SQL `WriteLiveValues`.
|
||||
- **Write resilience:** with store-forward enabled, an unreachable historian returns
|
||||
*accepted + queued* (not an error); otherwise `Unavailable`. Redundancy surfaces a
|
||||
per-member result; All-policy fails if any member fails, Any-policy succeeds on ≥1 ack.
|
||||
- **Pool:** transient failures → reconnect + bounded retry; auth-handshake failure →
|
||||
fail fast with diagnostic. No secrets/real hostnames in errors or logs (histsdk
|
||||
safety rule).
|
||||
|
||||
## 8. Testing
|
||||
|
||||
- **Unit:** gRPC services against a faked historian-client seam + faked Galaxy
|
||||
provider; scope/auth interceptor; config validators; SDK-model ↔ proto mapping.
|
||||
- **Golden/protocol:** carry over `histsdk`'s golden byte tests for the vendored
|
||||
client (historical "ON" buffer, event "OS" buffer, registration buffers) so the
|
||||
vendored copy stays faithful.
|
||||
- **Integration (env-gated, live, CI/macOS-safe):** real 2023 R2 historian + Galaxy
|
||||
Repository SQL — read/write round-trips and browse via the self-cleaning
|
||||
sandbox-tag lifecycle (`HISTORIAN_GRPC_WRITE_SANDBOX_TAG`); skipped when env vars
|
||||
absent.
|
||||
- **Dashboard:** bUnit component tests. **Smoke:** `/healthz`, `/metrics`, gRPC
|
||||
`Probe`.
|
||||
|
||||
## 9. Out of scope / non-goals
|
||||
|
||||
- `AddS2` live streaming process-sample writes (GATED server-side; SQL path covers
|
||||
live values instead).
|
||||
- Non-analog tag creation, revision/edit writes, bit-faithful store-forward framing
|
||||
(per `histsdk` capability matrix — `BOUNDED`/`HARD`/`GATED` items not selected).
|
||||
- A two-process / x86 worker split (not needed — no COM).
|
||||
- Re-syncing or replacing the existing stale `scadaproj/ZB.MOM.WW.SPHistorianClient`
|
||||
port (we vendor `histsdk` instead; the stale port is left as-is).
|
||||
|
||||
## 10. Implementation components (high level)
|
||||
|
||||
1. **`ZB.MOM.WW.GalaxyRepository` shared lib** (scadaproj) — extract from
|
||||
`mxaccessgw`, canonical proto + SQL browse provider + reusable gRPC service.
|
||||
2. **Vendor `histsdk`** `AVEVA.Historian.Client` into the new repo + carry its golden
|
||||
tests.
|
||||
3. **Repo scaffold + host + shared-package wiring** (Auth/Telemetry/Health/
|
||||
Configuration/Audit) + validated options + `ConfigPreflight`.
|
||||
4. **gRPC contract + services** (Read / Write / Tags / Status / GalaxyRepository).
|
||||
5. **Connection layer** — pooled pre-authed connections, store-forward, redundancy,
|
||||
SQL live-write path.
|
||||
6. **Auth** — API-key scope interceptor + LDAP dashboard auth + Audit wiring.
|
||||
7. **Blazor dashboard** pages (Theme).
|
||||
8. **Telemetry + Health** probes/meters.
|
||||
9. **Tests** — unit / golden / env-gated integration / bUnit.
|
||||
10. **Docs + repo/gitea setup** — `CLAUDE.md`, `README.md`, gitea remote.
|
||||
|
||||
> `mxaccessgw` adoption of `ZB.MOM.WW.GalaxyRepository` is a separate tracked
|
||||
> follow-on, not part of the initial sidecar delivery.
|
||||
@@ -0,0 +1,523 @@
|
||||
# ZB.MOM.WW.HistorianGateway Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Build a single .NET 10 x64 sidecar that exposes (1) a read-only Galaxy object-hierarchy metadata gRPC server and (2) a full read/write gRPC API to the AVEVA Historian, with a Blazor dashboard, reusing the family's shared `ZB.MOM.WW.*` packages.
|
||||
|
||||
**Architecture:** One ASP.NET Core process hosting gRPC services + Blazor (no COM, no x86 worker). The historian write/read surface comes from the **vendored `histsdk` client** (`AVEVA.Historian.Client`). The Galaxy browse comes from a **new shared lib `ZB.MOM.WW.GalaxyRepository`** in scadaproj (extracted from mxaccessgw, wire-compatible `galaxy_repository.v1`). Connection model: stateless gateway over a **pooled, pre-authenticated service-identity connection**; clients authenticate to the gateway via peppered-HMAC API keys with per-service scopes.
|
||||
|
||||
**Tech Stack:** .NET 10, ASP.NET Core, Grpc.AspNetCore 2.76, Grpc.Net.Client 2.58 (vendored), Google.Protobuf, Microsoft.Data.SqlClient, Microsoft.Data.Sqlite, Blazor InteractiveServer, `ZB.MOM.WW.Theme` 0.3.1, `ZB.MOM.WW.Auth` 0.1.2, `ZB.MOM.WW.Telemetry`/`.Serilog` 0.1.0, `ZB.MOM.WW.Health` 0.1.0, `ZB.MOM.WW.Audit` 0.1.0, `ZB.MOM.WW.Configuration` 0.1.0, xUnit, bUnit.
|
||||
|
||||
**Reference sources (read these for exact patterns — do NOT re-discover):**
|
||||
- Design doc: `docs/plans/2026-06-23-historian-gateway-design.md`
|
||||
- mxaccessgw (the model): `~/Desktop/MxAccessGateway/src/ZB.MOM.WW.MxGateway.Server/` — `GatewayApplication.cs` (host wiring), `Security/Authorization/*` (gRPC API-key interceptor + scope resolver), `Galaxy/GalaxyRepository.cs` (the SQL to extract), `Galaxy/GalaxyRepositoryOptions.cs`, `Galaxy/GalaxyHierarchyCache.cs`, `Galaxy/GalaxyRepositoryServiceCollectionExtensions.cs`, `Contracts/Protos/galaxy_repository.proto`, `Dashboard/Components/*` (Blazor + Theme).
|
||||
- histsdk clone (to vendor): `/tmp/histsdk-explore/src/AVEVA.Historian.Client/` + `/tmp/histsdk-explore/tests/AVEVA.Historian.Client.Tests/`.
|
||||
- Shared package signatures: captured in the design session; key paths under `~/Desktop/scadaproj/ZB.MOM.WW.{Telemetry,Health,Configuration,Audit,Auth,Theme}/`.
|
||||
|
||||
**Conventions for every task:** TDD where a seam exists (write the failing test first). Exact file paths in the `Files:` block ARE the implementer's contract. Commit after each task. Tests must stay green on macOS with no live historian/SQL (live tests are env-gated and skip when env vars are absent).
|
||||
|
||||
---
|
||||
|
||||
## Phase 0 — Shared `ZB.MOM.WW.GalaxyRepository` lib (in scadaproj)
|
||||
|
||||
> Built in `~/Desktop/scadaproj/ZB.MOM.WW.GalaxyRepository/` as plain files (NOT a nested git repo — see memory `shared-libs-are-plain-files-not-nested-repos`). Wire-compatible: keep proto `package galaxy_repository.v1` and all field numbers identical to mxaccessgw's so OtOpcUa is unaffected; only the C# `csharp_namespace` becomes neutral. mxaccessgw adoption of this lib is a separate follow-on, NOT in this plan.
|
||||
|
||||
### Task 1: Scaffold the GalaxyRepository lib project
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~3 min
|
||||
**Parallelizable with:** Task 7 (vendoring), Task 9 (repo scaffold)
|
||||
|
||||
**Files:**
|
||||
- Create: `~/Desktop/scadaproj/ZB.MOM.WW.GalaxyRepository/ZB.MOM.WW.GalaxyRepository.slnx`
|
||||
- Create: `~/Desktop/scadaproj/ZB.MOM.WW.GalaxyRepository/src/ZB.MOM.WW.GalaxyRepository/ZB.MOM.WW.GalaxyRepository.csproj`
|
||||
- Create: `~/Desktop/scadaproj/ZB.MOM.WW.GalaxyRepository/tests/ZB.MOM.WW.GalaxyRepository.Tests/ZB.MOM.WW.GalaxyRepository.Tests.csproj`
|
||||
|
||||
**Steps:**
|
||||
1. Create the `.csproj` (net10.0, `Nullable`/`ImplicitUsings` enabled, packable, `PackageId=ZB.MOM.WW.GalaxyRepository`, `Version=0.1.0`). PackageReferences: `Microsoft.Data.SqlClient` 6.0.2, `Grpc.AspNetCore` 2.76.0, `Google.Protobuf`, `Microsoft.Extensions.Hosting.Abstractions`, `Microsoft.Extensions.Options.ConfigurationExtensions`. Add `<Protobuf Include="Protos\*.proto" GrpcServices="Server" />`.
|
||||
2. Create the test `.csproj` (net10.0, `IsPackable=false`, xUnit 2.9.3 + `Microsoft.NET.Test.Sdk` 17.14.1 + `Microsoft.Data.SqlClient`), ProjectReference to the lib.
|
||||
3. Create the `.slnx` listing both projects.
|
||||
4. Run: `dotnet build ~/Desktop/scadaproj/ZB.MOM.WW.GalaxyRepository/ZB.MOM.WW.GalaxyRepository.slnx` — Expected: builds (no sources yet, 0 warnings).
|
||||
5. Commit: `git -C ~/Desktop/scadaproj add ZB.MOM.WW.GalaxyRepository && git -C ~/Desktop/scadaproj commit -m "feat(galaxyrepo): scaffold ZB.MOM.WW.GalaxyRepository shared lib"`
|
||||
|
||||
### Task 2: Port the canonical galaxy_repository.proto (neutral namespace)
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~4 min
|
||||
**Parallelizable with:** none (Task 3+ depend on generated types)
|
||||
|
||||
**Files:**
|
||||
- Create: `~/Desktop/scadaproj/ZB.MOM.WW.GalaxyRepository/src/ZB.MOM.WW.GalaxyRepository/Protos/galaxy_repository.proto`
|
||||
|
||||
**Steps:**
|
||||
1. Copy mxaccessgw's `Contracts/Protos/galaxy_repository.proto` verbatim, changing ONLY `option csharp_namespace` to `"ZB.MOM.WW.GalaxyRepository.Grpc"`. Keep `package galaxy_repository.v1`, all services (`TestConnection`, `GetLastDeployTime`, `DiscoverHierarchy`, `WatchDeployEvents`, `BrowseChildren`), and every message/field number identical (wire compatibility).
|
||||
2. Run: `dotnet build .../ZB.MOM.WW.GalaxyRepository.slnx` — Expected: PASS; generated `GalaxyRepository.GalaxyRepositoryBase`, `GalaxyObject`, `GalaxyAttribute`, etc. appear under namespace `ZB.MOM.WW.GalaxyRepository.Grpc`.
|
||||
3. Commit: `feat(galaxyrepo): canonical galaxy_repository.v1 proto (neutral namespace)`
|
||||
|
||||
### Task 3: Port the SQL browse provider (`GalaxyRepository` + rows + options)
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** none
|
||||
|
||||
**Files:**
|
||||
- Create: `.../src/ZB.MOM.WW.GalaxyRepository/GalaxyRepositoryOptions.cs`
|
||||
- Create: `.../src/ZB.MOM.WW.GalaxyRepository/GalaxyHierarchyRow.cs`
|
||||
- Create: `.../src/ZB.MOM.WW.GalaxyRepository/GalaxyAttributeRow.cs`
|
||||
- Create: `.../src/ZB.MOM.WW.GalaxyRepository/IGalaxyRepository.cs`
|
||||
- Create: `.../src/ZB.MOM.WW.GalaxyRepository/GalaxyRepository.cs`
|
||||
|
||||
**Steps:**
|
||||
1. Port `GalaxyRepositoryOptions` from mxaccessgw `Galaxy/GalaxyRepositoryOptions.cs` — rename section const to `ZB.MOM.WW.GalaxyRepository` (the consuming app picks its own section path at registration), drop MxGateway-specific defaults. Keep `ConnectionString`, `CommandTimeoutSeconds`, `DashboardRefreshIntervalSeconds`, `PersistSnapshot`, `SnapshotCachePath`.
|
||||
2. Port `GalaxyHierarchyRow` / `GalaxyAttributeRow` DTOs and the `IGalaxyRepository` interface (`TestConnectionAsync`, `GetLastDeployTimeAsync`, `GetHierarchyAsync`, `GetAttributesAsync`).
|
||||
3. Port `GalaxyRepository.cs` **verbatim** including the two SQL blocks (`HierarchySql`, `AttributesSql`) and the `SqlConnection`/`SqlDataReader` mapping loops — these are validated reverse-engineered queries; do NOT modify the SQL.
|
||||
4. Run: `dotnet build` — Expected: PASS.
|
||||
5. Commit: `feat(galaxyrepo): SQL browse provider (hierarchy + attributes)`
|
||||
|
||||
### Task 4: Port the in-memory hierarchy cache + snapshot + deploy notifier + refresh service
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** none
|
||||
|
||||
**Files:**
|
||||
- Create: `.../GalaxyHierarchyCacheEntry.cs`, `.../IGalaxyHierarchyCache.cs`, `.../GalaxyHierarchyCache.cs`
|
||||
- Create: `.../IGalaxyDeployNotifier.cs`, `.../GalaxyDeployNotifier.cs`
|
||||
- Create: `.../IGalaxyHierarchySnapshotStore.cs`, `.../GalaxyHierarchySnapshotStore.cs`
|
||||
- Create: `.../GalaxyHierarchyRefreshService.cs` (`BackgroundService`)
|
||||
- Create: `.../GalaxyHierarchyProjector.cs` (paging/filter projection used by the gRPC service)
|
||||
|
||||
**Steps:**
|
||||
1. Port these from mxaccessgw's `Galaxy/` folder, adjusting namespaces to `ZB.MOM.WW.GalaxyRepository`. Keep the cache's first-load gate, refresh semaphore, snapshot restore, and deploy-poll refresh trigger.
|
||||
2. Port `GalaxyHierarchyProjector` (the `Project(...)` + `ComputeFilterSignature(...)` used by `DiscoverHierarchy`/`BrowseChildren` paging).
|
||||
3. Run: `dotnet build` — Expected: PASS.
|
||||
4. Commit: `feat(galaxyrepo): hierarchy cache + snapshot + refresh service`
|
||||
|
||||
### Task 5: Port the reusable gRPC service + DI extension
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** none
|
||||
|
||||
**Files:**
|
||||
- Create: `.../Grpc/GalaxyRepositoryGrpcService.cs`
|
||||
- Create: `.../DependencyInjection/GalaxyRepositoryServiceCollectionExtensions.cs`
|
||||
|
||||
**Steps:**
|
||||
1. Port `GalaxyRepositoryGrpcService` from mxaccessgw's `Grpc/GalaxyRepositoryGrpcService.cs`, but REMOVE the mxaccessgw-specific `IGatewayRequestIdentityAccessor`/`ApiKeyConstraints` browse-subtree filtering (the gateway will apply its own auth at the interceptor layer). Keep `DiscoverHierarchy`, `BrowseChildren`, `TestConnection`, `GetLastDeployTime`, `WatchDeployEvents`. Base class: `ZB.MOM.WW.GalaxyRepository.Grpc.GalaxyRepository.GalaxyRepositoryBase`.
|
||||
2. Write `AddZbGalaxyRepository(this IServiceCollection, IConfiguration, string sectionPath)` modeled on mxaccessgw's `AddGalaxyRepository` — bind options from `sectionPath`, register `GalaxyRepository`/`IGalaxyRepository`, notifier, snapshot store, cache, and the refresh `HostedService`. Add a companion `MapZbGalaxyRepository(this IEndpointRouteBuilder)` that `MapGrpcService<GalaxyRepositoryGrpcService>()`.
|
||||
3. Run: `dotnet build` — Expected: PASS.
|
||||
4. Commit: `feat(galaxyrepo): reusable gRPC service + AddZbGalaxyRepository DI`
|
||||
|
||||
### Task 6: Unit tests for the projector + DI smoke; pack
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** none
|
||||
|
||||
**Files:**
|
||||
- Create: `.../tests/ZB.MOM.WW.GalaxyRepository.Tests/GalaxyHierarchyProjectorTests.cs`
|
||||
- Create: `.../tests/ZB.MOM.WW.GalaxyRepository.Tests/GalaxyHierarchyCacheTests.cs`
|
||||
|
||||
**Steps:**
|
||||
1. **Write failing tests first:** projector paging (page_token round-trip, max_depth, `historized_only`/`alarm_bearing_only` filters, attribute include toggle) against a hand-built `GalaxyHierarchyCacheEntry` fixture; cache first-load gate + snapshot restore using a fake `IGalaxyRepository`. (SQL provider itself is exercised by env-gated integration later — no live DB in unit tests.)
|
||||
2. Run: `dotnet test .../ZB.MOM.WW.GalaxyRepository.slnx` — Expected: FAIL (types/asserts).
|
||||
3. Implement any small helper gaps surfaced; re-run — Expected: PASS.
|
||||
4. Run: `dotnet pack .../src/ZB.MOM.WW.GalaxyRepository/ZB.MOM.WW.GalaxyRepository.csproj -c Release` — Expected: `ZB.MOM.WW.GalaxyRepository.0.1.0.nupkg` produced.
|
||||
5. Commit: `test(galaxyrepo): projector + cache tests; pack 0.1.0`
|
||||
|
||||
---
|
||||
|
||||
## Phase 1 — Sidecar repo scaffold + vendor histsdk
|
||||
|
||||
### Task 7: Vendor the histsdk client + its golden tests
|
||||
|
||||
**Classification:** high-risk
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** Task 1
|
||||
|
||||
**Files:**
|
||||
- Create: `~/Desktop/HistorianGateway/src/vendor/AVEVA.Historian.Client/**` (copied)
|
||||
- Create: `~/Desktop/HistorianGateway/tests/AVEVA.Historian.Client.Tests/**` (copied)
|
||||
- Create: `~/Desktop/HistorianGateway/src/vendor/AVEVA.Historian.Client/VENDORING.md`
|
||||
|
||||
**Steps:**
|
||||
1. `mkdir -p ~/Desktop/HistorianGateway/src/vendor ~/Desktop/HistorianGateway/tests`. Copy `/tmp/histsdk-explore/src/AVEVA.Historian.Client/` and `/tmp/histsdk-explore/tests/AVEVA.Historian.Client.Tests/` into those locations.
|
||||
2. In the vendored test `.csproj`, REMOVE the `ProjectReference` to `tools/AVEVA.Historian.ReverseEngineering` (not vendored) and delete any test classes that depend on that tooling namespace (the RE-sanitizer tests). KEEP the protocol/golden tests: `HistorianTagWriteProtocolTests`, `HistorianEventRowProtocolTests`, `GrpcEventSendProtocolTests`, `WcfDataQueryProtocolTests`, `StoreForwardOutboxTests`, `RedundancyTests`, version-gate tests. Fix the surviving test `.csproj` ProjectReference path to the new vendored client location.
|
||||
3. Keep namespace `AVEVA.Historian.Client` as-is (eases re-sync). Write `VENDORING.md` recording: source repo `gitea.dohertylan.com/dohertj2/histsdk`, the commit/date of the snapshot, and "do not hand-edit; re-vendor from upstream."
|
||||
4. Run: `dotnet build ~/Desktop/HistorianGateway/src/vendor/AVEVA.Historian.Client/AVEVA.Historian.Client.csproj` then `dotnet test ~/Desktop/HistorianGateway/tests/AVEVA.Historian.Client.Tests/` — Expected: build PASS; golden/offline tests PASS (live env-gated tests skip).
|
||||
5. Commit (in the new repo, after Task 8 inits it — if running before Task 8, defer the commit): `chore(vendor): vendor histsdk AVEVA.Historian.Client + golden tests`
|
||||
|
||||
### Task 8: Initialize the sidecar repo + solution + Directory.Build.props
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~3 min
|
||||
**Parallelizable with:** none (Task 7 output is added here)
|
||||
|
||||
**Files:**
|
||||
- Create: `~/Desktop/HistorianGateway/.gitignore`
|
||||
- Create: `~/Desktop/HistorianGateway/Directory.Build.props`
|
||||
- Create: `~/Desktop/HistorianGateway/ZB.MOM.WW.HistorianGateway.slnx`
|
||||
|
||||
**Steps:**
|
||||
1. `git -C ~/Desktop/HistorianGateway init` (this IS its own app repo — unlike shared libs). Add a .NET `.gitignore`.
|
||||
2. `Directory.Build.props`: `net10.0`, `Nullable`/`ImplicitUsings` enable, `<Platforms>x64</Platforms>`, `<PlatformTarget>x64</PlatformTarget>`, common `LangVersion`.
|
||||
3. Create `.slnx` referencing: `src/vendor/AVEVA.Historian.Client`, `tests/AVEVA.Historian.Client.Tests` (and the projects added in later phases — add them as created).
|
||||
4. Run: `dotnet build ~/Desktop/HistorianGateway/ZB.MOM.WW.HistorianGateway.slnx` — Expected: PASS.
|
||||
5. Commit: `chore: init repo + solution + Directory.Build.props` (then re-commit Task 7's vendored tree if it was deferred).
|
||||
|
||||
---
|
||||
|
||||
## Phase 2 — Host + configuration + shared-package wiring
|
||||
|
||||
### Task 9: Create the Contracts project + historian_gateway.proto skeleton
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** Task 1
|
||||
|
||||
**Files:**
|
||||
- Create: `~/Desktop/HistorianGateway/src/ZB.MOM.WW.HistorianGateway.Contracts/ZB.MOM.WW.HistorianGateway.Contracts.csproj`
|
||||
- Create: `.../Contracts/Protos/historian_gateway.proto`
|
||||
|
||||
**Steps:**
|
||||
1. `.csproj` net10.0, `Grpc.AspNetCore` 2.76.0, `<Protobuf Include="Protos\*.proto" GrpcServices="Both" />`.
|
||||
2. Author `historian_gateway.proto` (`package historian_gateway.v1; option csharp_namespace = "ZB.MOM.WW.HistorianGateway.Contracts.Grpc";`) with the **service stubs and message shells** for the 4 historian services: `HistorianRead` (ReadRaw/ReadAggregate/ReadBlocks/ReadEvents server-streaming, ReadAtTime unary), `HistorianWrite` (AddHistoricalValues, SendEvent, WriteLiveValues), `HistorianTags` (BrowseTagNames streaming, GetTagMetadata, EnsureTags, DeleteTags, RenameTags, AddTagExtendedProperties), `HistorianStatus` (Probe, GetConnectionStatus, GetStoreForwardStatus, GetSystemParameter). Map the message fields to the vendored `HistorianSample`/`HistorianAggregateSample`/`HistorianEvent`/`HistorianTagMetadata`/`HistorianHistoricalValue` shapes (timestamps as `google.protobuf.Timestamp`, `RetrievalMode` as an enum mirroring the SDK's 15 modes).
|
||||
3. Run: `dotnet build` — Expected: PASS; gateway gRPC base classes generated. Add project to `.slnx`.
|
||||
4. Commit: `feat(contracts): historian_gateway.v1 proto + generated types`
|
||||
|
||||
### Task 10: Create the Server project + minimal boot
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** none
|
||||
|
||||
**Files:**
|
||||
- Create: `.../src/ZB.MOM.WW.HistorianGateway.Server/ZB.MOM.WW.HistorianGateway.Server.csproj`
|
||||
- Create: `.../Server/Program.cs`
|
||||
- Create: `.../Server/appsettings.json`, `.../Server/appsettings.Development.json`
|
||||
|
||||
**Steps:**
|
||||
1. `.csproj` (Sdk `Microsoft.NET.Sdk.Web`): PackageReferences exactly mirroring mxaccessgw's Server csproj versions — `Grpc.AspNetCore` 2.76.0, `ZB.MOM.WW.Auth.{Abstractions,Ldap,ApiKeys,AspNetCore}` 0.1.2, `ZB.MOM.WW.Audit` 0.1.0, `ZB.MOM.WW.Theme` 0.3.1, `ZB.MOM.WW.Configuration` 0.1.0, `ZB.MOM.WW.Health` 0.1.0, `ZB.MOM.WW.Telemetry`+`.Serilog` 0.1.0, `Serilog.AspNetCore`/`.Sinks.Console`/`.Sinks.File`, `Microsoft.Data.Sqlite` 10.0.7, `Microsoft.Data.SqlClient` 6.0.2, `Polly.Core` 8.6.6. ProjectReferences: Contracts + vendored `AVEVA.Historian.Client` + `ZB.MOM.WW.GalaxyRepository` (project ref to the scadaproj lib, or pkg ref to its 0.1.0 nupkg).
|
||||
2. `Program.cs`: minimal `WebApplication` that calls `AddZbSerilog`/`AddZbTelemetry` (ServiceName `historian-gateway`), `builder.Services.AddGrpc()`, maps `/healthz` + `/metrics` via `MapZbHealth`/`MapZbMetrics`, boots. (Subsystems wired in later tasks.)
|
||||
3. Run: `dotnet build` then `dotnet run --project .../Server` and `curl -s localhost:<port>/healthz` — Expected: 200; `curl /metrics` returns Prometheus text. Add project to `.slnx`.
|
||||
4. Commit: `feat(server): host scaffold + telemetry/serilog/health boot`
|
||||
|
||||
### Task 11: Configuration options + validators + ConfigPreflight
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** none
|
||||
|
||||
**Files:**
|
||||
- Create: `.../Server/Configuration/HistorianOptions.cs` + `HistorianOptionsValidator.cs`
|
||||
- Create: `.../Server/Configuration/GalaxyOptions.cs` (thin wrapper / reuse `GalaxyRepositoryOptions`)
|
||||
- Create: `.../Server/Configuration/RuntimeDbOptions.cs` + validator (SQL live-write)
|
||||
- Create: `.../Server/Configuration/RedundancyOptions.cs` + validator
|
||||
- Create: `.../Server/Configuration/StoreForwardOptions.cs` + validator
|
||||
- Modify: `.../Server/Program.cs` (register `AddValidatedOptions<,>` + run `ConfigPreflight`)
|
||||
- Test: `.../tests/ZB.MOM.WW.HistorianGateway.Tests/Configuration/ValidatorTests.cs`
|
||||
|
||||
**Steps:**
|
||||
1. **Write failing validator tests first** using `OptionsValidatorBase`/`ValidationBuilder` semantics (e.g., missing `Historian:Host` → failure; bad port → failure; `Transport` one-of; redundancy `MinCount(members,1)` when enabled). Run — Expected: FAIL.
|
||||
2. Implement options records + validators (subclass `OptionsValidatorBase<T>`, use `ValidationBuilder.Required/Port/HostPort/OneOf/PositiveTimeSpan/MinCount`). Map `HistorianOptions` → vendored `HistorianClientOptions` (Host, Port default 32565, `Transport=RemoteGrpc`, `GrpcUseTls`, credentials, `AllowUntrustedServerCertificate`).
|
||||
3. In `Program.cs`, `AddValidatedOptions<,>` each, and run a `ConfigPreflight` (RequireValue host, RequirePort) before host build.
|
||||
4. Run: `dotnet test` — Expected: PASS.
|
||||
5. Commit: `feat(server): validated options + ConfigPreflight`
|
||||
|
||||
---
|
||||
|
||||
## Phase 3 — Connection layer (vendored client → gateway)
|
||||
|
||||
### Task 12: `IHistorianClient` seam over the vendored client
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** none
|
||||
|
||||
**Files:**
|
||||
- Create: `.../Server/Historian/IHistorianClient.cs` (interface mirroring the read/write methods the services need)
|
||||
- Create: `.../Server/Historian/VendoredHistorianClient.cs` (adapts `AVEVA.Historian.Client.HistorianClient`)
|
||||
- Test: `.../tests/.../Historian/HistorianClientSeamTests.cs`
|
||||
|
||||
**Steps:**
|
||||
1. **Write failing test** that a `FakeHistorianClient : IHistorianClient` can be substituted and returns canned samples (this seam is what makes the gRPC services unit-testable without a live historian). Run — Expected: FAIL.
|
||||
2. Define `IHistorianClient` with the methods the services call (ReadRaw/ReadAggregate/ReadAtTime/ReadBlocks/ReadEvents/BrowseTagNames/GetTagMetadata/Probe/GetConnectionStatus/GetStoreForwardStatus/GetSystemParameter/AddHistoricalValues/SendEvent/EnsureTag/DeleteTag/RenameTags/AddTagExtendedProperties). Implement `VendoredHistorianClient` delegating to the real `HistorianClient`.
|
||||
3. Run: `dotnet test` — Expected: PASS.
|
||||
4. Commit: `feat(historian): IHistorianClient seam + vendored adapter`
|
||||
|
||||
### Task 13: Connection pool (pre-authenticated, reused, health-checked)
|
||||
|
||||
**Classification:** high-risk
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** none
|
||||
|
||||
**Files:**
|
||||
- Create: `.../Server/Historian/HistorianConnectionPool.cs` (+ `IHistorianConnectionPool`)
|
||||
- Modify: `.../Server/Program.cs` (DI singleton)
|
||||
- Test: `.../tests/.../Historian/HistorianConnectionPoolTests.cs`
|
||||
|
||||
**Steps:**
|
||||
1. **Write failing test** asserting the pool opens/authenticates a connection once and reuses it across N borrow calls (count handshakes via a fake transport/lease factory), and that a faulted connection is evicted + re-created. Run — Expected: FAIL.
|
||||
2. Implement a lease-based pool keyed by target; lazy-open with the auth handshake once; reuse; `SemaphoreSlim`-guarded reconnect on fault; expose `Lease()` returning a pooled `IHistorianClient`. (The vendored client is `IAsyncDisposable`; the pool owns lifecycle.)
|
||||
3. Run: `dotnet test` — Expected: PASS.
|
||||
4. Commit: `feat(historian): pooled pre-authenticated connection pool`
|
||||
|
||||
### Task 14: Store-forward + redundancy + SQL live-write wiring
|
||||
|
||||
**Classification:** high-risk
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** none
|
||||
|
||||
**Files:**
|
||||
- Create: `.../Server/Historian/HistorianWriteCoordinator.cs` (routes writes → pool, store-forward, or redundancy per config)
|
||||
- Create: `.../Server/Historian/SqlLiveValueWriter.cs` (`WriteLiveValues` via `aaAnalogTagInsert` + `INSERT INTO History`)
|
||||
- Modify: `.../Server/Program.cs`
|
||||
- Test: `.../tests/.../Historian/HistorianWriteCoordinatorTests.cs`, `.../SqlLiveValueWriterTests.cs`
|
||||
|
||||
**Steps:**
|
||||
1. **Write failing tests:** (a) when store-forward enabled + historian unreachable, the coordinator enqueues (uses vendored `HistorianStoreForwardWriter` over a fake sink) and reports `Queued`; (b) when redundancy configured, it fans out via `HistorianRedundantClient` and returns per-member results under All/Any; (c) `SqlLiveValueWriter` builds the correct parameterized command sequence (assert against a fake `IDbCommand` recorder — no live SQL). Run — Expected: FAIL.
|
||||
2. Implement the coordinator (compose vendored `HistorianStoreForwardWriter` + `HistorianRedundantClient` from config) and `SqlLiveValueWriter` (omit the server-managed `Quality` column; honor the storage-activation note from the SQL reference memory).
|
||||
3. Run: `dotnet test` — Expected: PASS.
|
||||
4. Commit: `feat(historian): write coordinator (store-forward + redundancy) + SQL live-write`
|
||||
|
||||
---
|
||||
|
||||
## Phase 4 — gRPC services + auth interceptor
|
||||
|
||||
### Task 15: `HistorianRead` service (representative TDD task; sets the pattern)
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** Task 17 after the mapper exists
|
||||
|
||||
**Files:**
|
||||
- Create: `.../Server/Grpc/HistorianReadService.cs`
|
||||
- Create: `.../Server/Grpc/HistorianProtoMapper.cs` (SDK model ↔ proto)
|
||||
- Modify: `.../Server/Program.cs` (`MapGrpcService`)
|
||||
- Test: `.../tests/.../Grpc/HistorianReadServiceTests.cs`
|
||||
|
||||
**Steps:**
|
||||
1. **Write failing test:** with a `FakeHistorianClient` yielding 3 `HistorianSample`s, calling `ReadRaw` streams 3 mapped proto rows; `ReadAggregate` passes the right `RetrievalMode`+interval; an unknown tag → `RpcException(NotFound)`; bad time range → `InvalidArgument`. Use an in-memory `IServerStreamWriter<T>` capture. Run — Expected: FAIL.
|
||||
2. Implement `HistorianReadService : HistorianRead.HistorianReadBase` consuming `IHistorianConnectionPool.Lease()`; implement `HistorianProtoMapper` (Timestamp conversions, RetrievalMode enum map). Map exceptions per design §7.
|
||||
3. Run: `dotnet test` — Expected: PASS.
|
||||
4. Commit: `feat(grpc): HistorianRead service + proto mapper`
|
||||
|
||||
### Task 16: `HistorianWrite` service
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** Task 17, Task 18 (no file overlap)
|
||||
|
||||
**Files:** Create `.../Server/Grpc/HistorianWriteService.cs`; Modify `Program.cs`; Test `.../Grpc/HistorianWriteServiceTests.cs`
|
||||
|
||||
**Steps:** TDD per the Task 15 pattern. `AddHistoricalValues`/`SendEvent` route through `HistorianWriteCoordinator`; `WriteLiveValues` through `SqlLiveValueWriter`. Map `ProtocolEvidenceMissingException` → `Unimplemented`, unreachable+store-forward → `OK` with `Queued` status, redundancy per-member results into the reply. Commit: `feat(grpc): HistorianWrite service`.
|
||||
|
||||
### Task 17: `HistorianTags` service
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** Task 16, Task 18
|
||||
|
||||
**Files:** Create `.../Server/Grpc/HistorianTagsService.cs`; Modify `Program.cs`; Test `.../Grpc/HistorianTagsServiceTests.cs`
|
||||
|
||||
**Steps:** TDD. `BrowseTagNames` (streaming), `GetTagMetadata`, `EnsureTags`/`DeleteTags`/`RenameTags`/`AddTagExtendedProperties` via the seam/pool. Map unsupported tag types (`ProtocolEvidenceMissingException`) → `FailedPrecondition`. Commit: `feat(grpc): HistorianTags service`.
|
||||
|
||||
### Task 18: `HistorianStatus` service
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~4 min
|
||||
**Parallelizable with:** Task 16, Task 17
|
||||
|
||||
**Files:** Create `.../Server/Grpc/HistorianStatusService.cs`; Modify `Program.cs`; Test `.../Grpc/HistorianStatusServiceTests.cs`
|
||||
|
||||
**Steps:** TDD. `Probe`/`GetConnectionStatus`/`GetStoreForwardStatus`/`GetSystemParameter`. Commit: `feat(grpc): HistorianStatus service`.
|
||||
|
||||
### Task 19: Galaxy gRPC wiring (consume the shared lib)
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~3 min
|
||||
**Parallelizable with:** Task 16–18
|
||||
|
||||
**Files:** Modify `.../Server/Program.cs` (`AddZbGalaxyRepository(config, "Galaxy")` + `MapZbGalaxyRepository()`); Modify `appsettings.json`
|
||||
|
||||
**Steps:** Register the shared lib's service + refresh hosted service; add `Galaxy:ConnectionString` config. Run: `dotnet run` + grpcurl `DiscoverHierarchy` against a fake/empty config returns `Unavailable` until cache loads (no live DB needed to prove wiring). Commit: `feat(server): wire shared GalaxyRepository gRPC service`.
|
||||
|
||||
### Task 20: API-key auth interceptor + scope resolver
|
||||
|
||||
**Classification:** high-risk
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** none
|
||||
|
||||
**Files:**
|
||||
- Create: `.../Server/Security/GatewayGrpcScopeResolver.cs` (maps request type → scope)
|
||||
- Create: `.../Server/Security/GatewayGrpcAuthorizationInterceptor.cs`
|
||||
- Create: `.../Server/Security/GatewayScopes.cs` (`historian:read|write`, `historian:tags:write`, `galaxy:read`)
|
||||
- Modify: `.../Server/Program.cs` (`AddZbApiKeyAuth` + `AddGrpc(o => o.Interceptors.Add<...>())`)
|
||||
- Test: `.../tests/.../Security/GrpcAuthorizationTests.cs`
|
||||
|
||||
**Steps:**
|
||||
1. **Write failing tests:** missing/invalid key → `Unauthenticated`; valid key without the required scope → `PermissionDenied`; valid key with scope → continuation runs. Fake `IApiKeyVerifier`. Run — Expected: FAIL.
|
||||
2. Implement modeled on mxaccessgw's `GatewayGrpcAuthorizationInterceptor` + `GatewayGrpcScopeResolver` (switch on request type → scope), using shared `IApiKeyVerifier.VerifyAsync`. Respect a `Disabled` auth mode for dev.
|
||||
3. Run: `dotnet test` — Expected: PASS.
|
||||
4. Commit: `feat(security): gRPC API-key interceptor + scope enforcement`
|
||||
|
||||
---
|
||||
|
||||
## Phase 5 — Audit
|
||||
|
||||
### Task 21: Canonical SQLite audit writer + actor accessor + wiring
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** Task 22 (dashboard auth) after interfaces exist
|
||||
|
||||
**Files:**
|
||||
- Create: `.../Server/Audit/SqliteAuditWriter.cs` (`IAuditWriter`), `.../Server/Audit/HttpAuditActorAccessor.cs` (`IAuditActorAccessor`)
|
||||
- Modify: write services (Tasks 16,17) + interceptor (Task 20) to emit `AuditEvent`s
|
||||
- Modify: `.../Server/Program.cs` (`AddZbAudit` + register writer/actor)
|
||||
- Test: `.../tests/.../Audit/SqliteAuditWriterTests.cs`
|
||||
|
||||
**Steps:**
|
||||
1. **Write failing test:** writing an `AuditEvent` persists a row with the canonical 9 fields (`EventId`/`OccurredAtUtc`/`Actor`/`Action`/`Outcome`/`Category`/`Target`/`SourceNode`/`DetailsJson`), domain fields in `DetailsJson`; writer swallows internal errors. Use an in-memory SQLite. Run — Expected: FAIL.
|
||||
2. Implement the SQLite writer (table create-if-missing) modeled on MxGateway's audit store; `HttpAuditActorAccessor` reads the Auth principal. Emit audit at tag/value/event writes, API-key admin, login/logout, with `Actor` from the accessor.
|
||||
3. Run: `dotnet test` — Expected: PASS.
|
||||
4. Commit: `feat(audit): canonical SQLite audit writer + actor wiring`
|
||||
|
||||
---
|
||||
|
||||
## Phase 6 — Blazor dashboard
|
||||
|
||||
### Task 22: Dashboard shell, LDAP cookie auth, login/logout
|
||||
|
||||
**Classification:** high-risk
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** none
|
||||
|
||||
**Files:**
|
||||
- Create: `.../Server/Dashboard/Components/{App,Routes,_Imports}.razor`, `Layout/{MainLayout,LoginLayout}.razor`, `Pages/Login.razor`
|
||||
- Create: `.../Server/Dashboard/DashboardServiceCollectionExtensions.cs`, `.../Dashboard/DashboardEndpointRouteBuilderExtensions.cs`, `.../Dashboard/DashboardAuthenticator.cs`, `.../Dashboard/DashboardGroupRoleMapper.cs`
|
||||
- Modify: `Program.cs` (`AddGatewayDashboard` + `MapRazorComponents<App>` + auth/antiforgery middleware)
|
||||
- Test: `.../tests/ZB.MOM.WW.HistorianGateway.Tests/bUnit/LayoutRenderTests.cs`
|
||||
|
||||
**Steps:**
|
||||
1. **Write failing bUnit test** that `MainLayout` renders `<ThemeShell>` with the nav rail and `LoginCard` renders on the login page. Run — Expected: FAIL.
|
||||
2. Port the dashboard shell from mxaccessgw (`App.razor` with `ThemeHead`/`ThemeScripts`, `MainLayout` with `ThemeShell`+`NavRailSection`/`NavRailItem`, `Login.razor` using `LoginCard` posting to `/auth/login`). Wire `AddZbLdapAuth(config,"Ldap")`, cookie auth via `ZbCookieDefaults.Apply`, `IGroupRoleMapper<CanonicalRole>`, `DisableLogin` switch, `IAuditActorAccessor`.
|
||||
3. Run: `dotnet test` (bUnit) then `dotnet run` and load `/login` in a browser/curl — Expected: tests PASS; login page renders themed.
|
||||
4. Commit: `feat(dashboard): Theme shell + LDAP cookie auth + login`
|
||||
|
||||
### Task 23: Status + Health pages
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~4 min
|
||||
**Parallelizable with:** Task 24, Task 25
|
||||
|
||||
**Files:** Create `.../Dashboard/Components/Pages/{StatusPage,HealthPage}.razor` (+ a `DashboardStatusService`); Test bUnit render.
|
||||
|
||||
**Steps:** TDD bUnit render. Status shows pool state, store-forward queue depth, redundancy members, version (from a status service reading the pool/coordinator). Commit: `feat(dashboard): status + health pages`.
|
||||
|
||||
### Task 24: Galaxy browser page
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** Task 23, Task 25
|
||||
|
||||
**Files:** Create `.../Dashboard/Components/Pages/GalaxyBrowserPage.razor` + tree node view (port mxaccessgw `BrowsePage`/`BrowseTreeNodeView`, read-only, no add-tag); Test bUnit.
|
||||
|
||||
**Steps:** TDD bUnit render against the shared lib's cache. Commit: `feat(dashboard): read-only Galaxy browser`.
|
||||
|
||||
### Task 25: Historian console page (query + role-gated write test)
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** Task 23, Task 24
|
||||
|
||||
**Files:** Create `.../Dashboard/Components/Pages/HistorianConsolePage.razor` (+ `DashboardHistorianService` calling the seam/pool); Test bUnit.
|
||||
|
||||
**Steps:** TDD bUnit. Query form (tag, time range, raw/aggregate + mode picker) renders results; write-test panel (historical value insert / event send) visible only to Engineer+ roles via `AuthorizeView`. Commit: `feat(dashboard): historian query + role-gated write console`.
|
||||
|
||||
### Task 26: API-key admin page
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** Task 23–25
|
||||
|
||||
**Files:** Create `.../Dashboard/Components/Pages/ApiKeysPage.razor` (+ `DashboardApiKeyManagementService` over the shared ApiKeys store); Test bUnit.
|
||||
|
||||
**Steps:** TDD bUnit. List/create (show secret once)/revoke keys with scope selection. Commit: `feat(dashboard): API-key admin`.
|
||||
|
||||
---
|
||||
|
||||
## Phase 7 — Telemetry meters + Health probes
|
||||
|
||||
### Task 27: App meters
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~4 min
|
||||
**Parallelizable with:** Task 28
|
||||
|
||||
**Files:** Create `.../Server/Observability/GatewayMetrics.cs`; Modify services/coordinator/pool to record; Modify `Program.cs` (`o.Meters=[GatewayMetrics.MeterName]`); Test `.../Observability/GatewayMetricsTests.cs`.
|
||||
|
||||
**Steps:** TDD with `MeterListener`. Counters/histograms: read/write counts + latency, store-forward queue depth (observable gauge), pool connection state, redundancy ack outcomes. Commit: `feat(obs): gateway meters`.
|
||||
|
||||
### Task 28: Health probes
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~4 min
|
||||
**Parallelizable with:** Task 27
|
||||
|
||||
**Files:** Create `.../Server/Health/{HistorianConnectionHealthCheck,StoreForwardDrainHealthCheck}.cs`; Modify `Program.cs` (`AddHealthChecks` with `GrpcDependencyHealthCheck` for historian, SQL checks for Galaxy + Runtime DB, custom checks, tagged `ZbHealthTags.Ready`); Test health-check unit tests.
|
||||
|
||||
**Steps:** TDD. Probes flip Unhealthy when a dependency is down (fake deps). Commit: `feat(health): historian/galaxy/runtime-db/store-forward probes`.
|
||||
|
||||
---
|
||||
|
||||
## Phase 8 — Integration, docs, repo
|
||||
|
||||
### Task 29: Env-gated live integration tests
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** none
|
||||
|
||||
**Files:** Create `.../tests/.../Integration/{HistorianRoundTripTests,GalaxyBrowseTests}.cs`
|
||||
|
||||
**Steps:** Gated on `HISTORIAN_GRPC_HOST`/`HISTORIAN_GRPC_WRITE_SANDBOX_TAG` and a Galaxy SQL connection env var; `Skip` when absent. Cover read→write→read-back via the self-cleaning sandbox-tag lifecycle and a Galaxy `DiscoverHierarchy`. Run `dotnet test` (skips locally). Commit: `test: env-gated live integration`.
|
||||
|
||||
### Task 30: Full-suite green gate + smoke
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~3 min
|
||||
**Parallelizable with:** none
|
||||
|
||||
**Steps:** Run `dotnet build ZB.MOM.WW.HistorianGateway.slnx` + `dotnet test` (whole solution) on macOS with no live env — Expected: ALL green, live tests skipped. `dotnet run` + curl `/healthz` (200), `/metrics` (text), grpcurl `HistorianStatus/Probe`. Fix any gaps. Commit: `chore: green gate + smoke`.
|
||||
|
||||
### Task 31: CLAUDE.md + README + gitea remote + scadaproj index
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** none
|
||||
|
||||
**Files:** Create `~/Desktop/HistorianGateway/{CLAUDE.md,README.md}`; copy the two design/plan docs into its `docs/plans/`; Modify `~/Desktop/scadaproj/CLAUDE.md` (index the new sidecar + note the GalaxyRepository follow-on for mxaccessgw).
|
||||
|
||||
**Steps:**
|
||||
1. Write `CLAUDE.md` (overview, build/run/test commands, the no-COM single-process note, the vendored-histsdk + shared-GalaxyRepository dependencies, config sections, env vars) and `README.md`.
|
||||
2. Create the gitea repo `historiangw` and push: `git -C ~/Desktop/HistorianGateway remote add origin https://gitea.dohertylan.com/dohertj2/historiangw.git && git push -u origin main` (confirm remote name/visibility with the user first).
|
||||
3. Update scadaproj's umbrella `CLAUDE.md` runtime/implementation table with the new project row; commit scadaproj separately.
|
||||
4. Commit: `docs: CLAUDE.md + README; index in scadaproj`.
|
||||
|
||||
---
|
||||
|
||||
## Dependency summary (for parallel dispatch)
|
||||
|
||||
- **Foundational, no blockers:** Task 1 (galaxy lib scaffold), Task 7 (vendor histsdk), Task 8 (repo init) — Task 8 consumes Task 7's tree.
|
||||
- **Galaxy lib chain:** 2→3→4→5→6 (sequential; share files).
|
||||
- **Sidecar chain:** 8→9→10→11→12→13→14, then gRPC services 15→(16,17,18 parallel),19, then 20, then 21.
|
||||
- **Dashboard:** 22→(23,24,25,26 parallel) after Task 20 (auth) + Task 13/14 (data) + Task 5/19 (galaxy).
|
||||
- **Obs:** 27,28 parallel after Task 14.
|
||||
- **Close-out:** 29→30→31 after everything.
|
||||
|
||||
## Notes / non-goals (from design §9)
|
||||
- No `AddS2` live streaming-sample writes (GATED) — live values only via SQL `WriteLiveValues`.
|
||||
- No two-process/x86 worker (no COM).
|
||||
- mxaccessgw adopting `ZB.MOM.WW.GalaxyRepository` is a tracked follow-on, NOT in this plan.
|
||||
@@ -0,0 +1,37 @@
|
||||
{
|
||||
"planPath": "docs/plans/2026-06-23-historian-gateway-implementation.md",
|
||||
"tasks": [
|
||||
{"id": 1, "subject": "Task 1: Scaffold the GalaxyRepository lib project", "status": "pending"},
|
||||
{"id": 2, "subject": "Task 2: Port the canonical galaxy_repository.proto (neutral namespace)", "status": "pending", "blockedBy": [1]},
|
||||
{"id": 3, "subject": "Task 3: Port the SQL browse provider", "status": "pending", "blockedBy": [2]},
|
||||
{"id": 4, "subject": "Task 4: Port the in-memory hierarchy cache + snapshot + refresh service", "status": "pending", "blockedBy": [3]},
|
||||
{"id": 5, "subject": "Task 5: Port the reusable gRPC service + DI extension", "status": "pending", "blockedBy": [4]},
|
||||
{"id": 6, "subject": "Task 6: Unit tests for projector/cache; pack 0.1.0", "status": "pending", "blockedBy": [5]},
|
||||
{"id": 7, "subject": "Task 7: Vendor histsdk AVEVA.Historian.Client + golden tests", "status": "pending"},
|
||||
{"id": 8, "subject": "Task 8: Init sidecar repo + solution + Directory.Build.props", "status": "pending", "blockedBy": [7]},
|
||||
{"id": 9, "subject": "Task 9: Contracts project + historian_gateway.proto skeleton", "status": "pending", "blockedBy": [8]},
|
||||
{"id": 10, "subject": "Task 10: Server project + minimal boot (telemetry/serilog/health)", "status": "pending", "blockedBy": [9, 7, 6]},
|
||||
{"id": 11, "subject": "Task 11: Configuration options + validators + ConfigPreflight", "status": "pending", "blockedBy": [10]},
|
||||
{"id": 12, "subject": "Task 12: IHistorianClient seam over vendored client", "status": "pending", "blockedBy": [10]},
|
||||
{"id": 13, "subject": "Task 13: Connection pool (pre-authenticated, reused)", "status": "pending", "blockedBy": [12]},
|
||||
{"id": 14, "subject": "Task 14: Write coordinator (store-forward + redundancy) + SQL live-write", "status": "pending", "blockedBy": [13]},
|
||||
{"id": 15, "subject": "Task 15: HistorianRead service + proto mapper", "status": "pending", "blockedBy": [13]},
|
||||
{"id": 16, "subject": "Task 16: HistorianWrite service", "status": "pending", "blockedBy": [14, 15]},
|
||||
{"id": 17, "subject": "Task 17: HistorianTags service", "status": "pending", "blockedBy": [13, 15]},
|
||||
{"id": 18, "subject": "Task 18: HistorianStatus service", "status": "pending", "blockedBy": [13, 15]},
|
||||
{"id": 19, "subject": "Task 19: Galaxy gRPC wiring (consume shared lib)", "status": "pending", "blockedBy": [10, 5]},
|
||||
{"id": 20, "subject": "Task 20: API-key auth interceptor + scope resolver", "status": "pending", "blockedBy": [15]},
|
||||
{"id": 21, "subject": "Task 21: SQLite audit writer + actor accessor + wiring", "status": "pending", "blockedBy": [16, 17, 20]},
|
||||
{"id": 22, "subject": "Task 22: Dashboard shell + LDAP cookie auth + login", "status": "pending", "blockedBy": [20]},
|
||||
{"id": 23, "subject": "Task 23: Status + Health pages", "status": "pending", "blockedBy": [22, 14]},
|
||||
{"id": 24, "subject": "Task 24: Galaxy browser page", "status": "pending", "blockedBy": [22, 19]},
|
||||
{"id": 25, "subject": "Task 25: Historian console page (role-gated write)", "status": "pending", "blockedBy": [22, 15]},
|
||||
{"id": 26, "subject": "Task 26: API-key admin page", "status": "pending", "blockedBy": [22]},
|
||||
{"id": 27, "subject": "Task 27: App meters", "status": "pending", "blockedBy": [14]},
|
||||
{"id": 28, "subject": "Task 28: Health probes", "status": "pending", "blockedBy": [14, 19]},
|
||||
{"id": 29, "subject": "Task 29: Env-gated live integration tests", "status": "pending", "blockedBy": [16, 17, 18, 19]},
|
||||
{"id": 30, "subject": "Task 30: Full-suite green gate + smoke", "status": "pending", "blockedBy": [21, 26, 27, 28, 29]},
|
||||
{"id": 31, "subject": "Task 31: CLAUDE.md + README + gitea remote + scadaproj index", "status": "pending", "blockedBy": [30]}
|
||||
],
|
||||
"lastUpdated": "2026-06-23"
|
||||
}
|
||||
@@ -0,0 +1,377 @@
|
||||
# MES and Delmia-DNC integrations — API & MXAccess write specification
|
||||
|
||||
Documents two existing Wonderware integrations hosted on the `ZimmerBiomet` Gitea org
|
||||
(`http://wonder-app-vd03.zmr.zimmer.com:3000`):
|
||||
|
||||
| Integration | Repo | What it does | Who does the MXAccess write |
|
||||
|---|---|---|---|
|
||||
| **MES** | [`ZimmerBiomet/MESAPI`](http://wonder-app-vd03.zmr.zimmer.com:3000/ZimmerBiomet/MESAPI) (solution `WWSupport`) | REST API the Camstar MES calls to move-in / move-out work orders against a machine, and to read machine alarm status | **The service itself** (`MesNotifier`, in-repo) |
|
||||
| **Delmia DNC** | [`ZimmerBiomet/DelmiaIntegration`](http://wonder-app-vd03.zmr.zimmer.com:3000/ZimmerBiomet/DelmiaIntegration) | Pull an NC/recipe document from the DELMIA/Apriso (Intercim) DNC server and push the resulting recipe-download notification into Wonderware | **An external receiver service** at `wonder-app-vd01:9001/notify`; the actual flag handshake is implemented in the Galaxy `$DelmiaReceiver` object (`ProcessRecipe`/`Reset` scripts) — *not* in this repo |
|
||||
|
||||
Both integrations talk to AVEVA System Platform ("Galaxy") through **MXAccess COM**
|
||||
(`ArchestrA.MxAccess.LMXProxyServerClass`) and use the same general pattern:
|
||||
|
||||
> **Handshake pattern** — read a *ready* flag → write the data tags → set a *trigger* flag
|
||||
> → wait (bounded by a timeout) for a *complete* flag → read a *success* flag + *error text*
|
||||
> → return the result and unsubscribe.
|
||||
|
||||
All facts below are taken verbatim from source at the repo `master` branch (cloned
|
||||
2026-06-17) unless explicitly marked **(inferred)**.
|
||||
|
||||
---
|
||||
|
||||
## 1. MES integration — `MESAPI` / `WWSupport`
|
||||
|
||||
### 1.1 Topology & hosting
|
||||
|
||||
```
|
||||
Camstar MES ──HTTPS/JSON──▶ WWSupport API (ServiceStack self-host)
|
||||
│ ├─ SQL Server "BT" (machine lookup by SAPID, alarm catalog)
|
||||
│ └─ MXAccess COM (LMXProxyServerClass, client "MesNotifier")
|
||||
▼
|
||||
Galaxy object {MachineCode}.MesReceiver.* (move-in / move-out tags)
|
||||
Galaxy object {MachineCode}.{AlarmName}.* (alarm attributes)
|
||||
```
|
||||
|
||||
- **Framework:** ServiceStack, self-hosted via `AppSelfHostBase` (`AppHost : base("APIServer", typeof(MesServices).Assembly)`), .NET Framework, run as a Windows service.
|
||||
- **Listen URL (per environment, `App.config` → `HttpListener`):**
|
||||
- DEV `http://*:9501/` · QA `http://*:9500/` · PROD `http://*:9500/`
|
||||
- **Database (`App.config` → connection `BatchDB`, DB `BT`):**
|
||||
- DEV `wonder-sql-vd01.zmr.zimmer.com` · QA `wondersqlqa.zmr.zimmer.com` · PROD (same form). User `wonderapp`.
|
||||
- **Auth:** every operation is decorated `[Authenticate]` + `[RequiredRole("MESAPI")]` (`MesServices.cs`).
|
||||
`AppHost` registers an `AuthFeature` with two providers: `ApiKeyAuthProvider` and `LdapAuthProvider`.
|
||||
- Unauthenticated → **401**; authenticated without the `MESAPI` role → **403**.
|
||||
- **Serialization:** `JsConfig.IncludeNullValues = true` (null fields ARE emitted in JSON). `PostmanFeature` + `OpenApiFeature` (Swagger) are enabled.
|
||||
- **MES counterpart object:** the live Galaxy attribute listing for the receiver object is in
|
||||
[`mesrec.md`](mesrec.md) (`$MESReceiver` template). Note the API binds tags under the contained
|
||||
name `MesReceiver` (i.e. `{MachineCode}.MesReceiver.<tag>`).
|
||||
|
||||
### 1.2 Endpoints (inputs / outputs)
|
||||
|
||||
Routes come from `[Route(...)]` on the request DTOs; all are **POST**, JSON in / JSON out, handled by `MesServices.Any(...)` which resolves a per-request `MesNotifier`.
|
||||
|
||||
#### `POST /mes/movein` → `MoveInResponse`
|
||||
|
||||
Request `MoveInRequest`:
|
||||
|
||||
| Field | Type | Notes |
|
||||
|---|---|---|
|
||||
| `SAPID` | string | machine key; looked up in `BT.Machine` to get `Machine.Code` |
|
||||
| `OperatorName` | string | |
|
||||
| `JobSequenceNumber` | string | |
|
||||
| `WorkOrders` | `List<WorkOrderInfo>` | each = `{ WorkOrderNumber: string, PartNumber: string }` |
|
||||
|
||||
Response `MoveInResponse`: `WasSuccessful` (bool), `ErrorText` (string), `BatchID` (int?, only set if machine returns non-zero).
|
||||
|
||||
#### `POST /mes/moveout` → `MoveOutResponse`
|
||||
|
||||
Request `MoveOutRequest`: `SAPID` (string), `OperatorName` (string), `WorkOrders` (`List<WorkOrderInfo>`).
|
||||
*(Move-out has no `JobSequenceNumber`.)*
|
||||
Response `MoveOutResponse`: identical shape to `MoveInResponse` (`WasSuccessful`, `ErrorText`, `BatchID`).
|
||||
|
||||
#### `POST /mes/alarmstatus` → `AlarmStatusResponse`
|
||||
|
||||
Request `AlarmStatusRequest`:
|
||||
- `MachineFilter` = `{ MachineID: int?, SAPID: string, ZTag: string, Code: string }` (any one identifies the machine)
|
||||
- `AlarmFilter` = `{ NameFilter: string, MinSeverity: int?, MaxSeverity: int?, IncludeTriggered: bool=true, IncludeAcked: bool=true, FlaggedOnly: bool=false }`
|
||||
|
||||
#### `POST /mes/simplealarmstatus` → `AlarmStatusResponse`
|
||||
|
||||
Request `SimpleAlarmStatusRequest`: `SAPID` (string). Internally loads only alarms with `FlaggedForMES == true` for that machine.
|
||||
|
||||
Response `AlarmStatusResponse` (both alarm endpoints): `WasSuccessful` (bool), `ErrorText` (string), `Alarms` (`List<AlarmInfo>`).
|
||||
|
||||
`AlarmInfo`: `Name` (string), `HierarchicalName` (string, `{Code}.{AlarmName}`), `Description` (string), `IsFlaggedForMES` (bool), `Severity` (int), `StatusCode` (string — `"Triggered"` or `"Triggered.Acked"`), `TriggeredDT` (DateTime), `AckDT` (DateTime?), `AckComment` (string).
|
||||
|
||||
### 1.3 MXAccess connection model
|
||||
|
||||
`MesNotifier` owns one MXAccess proxy for the request:
|
||||
|
||||
```csharp
|
||||
_lmxProxy = new ArchestrA.MxAccess.LMXProxyServerClass();
|
||||
_lmxHandle = _lmxProxy.Register("MesNotifier");
|
||||
_lmxProxy.OnDataChange += ...; // value updates resolve pending read/OnValue tasks
|
||||
_lmxProxy.OnWriteComplete += ...; // write acks resolve pending write tasks
|
||||
```
|
||||
|
||||
Tags are added with `AddItem` + `AdviseSupervisory` (subscribe), updated via `OnDataChange`,
|
||||
and removed with `UnAdvise` + `RemoveItem` on cleanup. Reads/writes are wrapped as `Task<bool>`
|
||||
that complete on the corresponding callback or **fail (`false`) on cancellation/timeout**.
|
||||
A read is considered valid only if MXAccess **quality == 192** ("good").
|
||||
|
||||
**Target selection:** request `SAPID` → `db.Single<Machine>(x => x.SAPID == SAPID)` → `Machine.Code`
|
||||
becomes the tag prefix. Move tags live under `{Code}.MesReceiver.*`; alarm tags under
|
||||
`{Code}.{MachineAlarm.Name}.*` (alarm catalog from `db.Select<MachineAlarm>(...)`).
|
||||
|
||||
### 1.4 Tag mappings
|
||||
|
||||
**Move-in** (`MesMoveInTagset`, all `{Code}.MesReceiver.<tag>`):
|
||||
|
||||
| Tag | Type | Dir | Role | Source field |
|
||||
|---|---|---|---|---|
|
||||
| `MoveInReadyFlag` | bool | read | gate — must be `true` before writing | — |
|
||||
| `MoveInFlag` | bool | **write** | **trigger** — set `true` last | — |
|
||||
| `MoveInCompleteFlag` | bool | read | completion — handshake waits on this | — |
|
||||
| `MoveInSuccessfulFlag` | bool | read | result | → `response.WasSuccessful` |
|
||||
| `MoveInErrorText` | string | read | result | → `response.ErrorText` |
|
||||
| `MoveInBatchID` | int | read | result | → `response.BatchID` (if ≠ 0) |
|
||||
| `MoveInOperatorName` | string | write | data | `request.OperatorName` |
|
||||
| `MoveInJobSequenceNumber` | string | write | data | `request.JobSequenceNumber` |
|
||||
| `MoveInNumberWorkOrders` | int | write | data | `request.WorkOrders.Count` |
|
||||
| `MoveInWorkOrderNumbers[]` | string[] | write | data (fixed length 50) | `WorkOrders.Select(w => w.WorkOrderNumber)` |
|
||||
| `MoveInPartNumbers[]` | string[] | write | data (fixed length 50) | `WorkOrders.Select(w => w.PartNumber)` |
|
||||
|
||||
**Move-out** (`MesMoveOutTagset`): identical set with `MoveOut` prefix, **minus** `JobSequenceNumber`
|
||||
(`MoveOutReadyFlag`, `MoveOutFlag`, `MoveOutCompleteFlag`, `MoveOutSuccessfulFlag`, `MoveOutErrorText`,
|
||||
`MoveOutBatchID`, `MoveOutOperatorName`, `MoveOutNumberWorkOrders`, `MoveOutWorkOrderNumbers[]`, `MoveOutPartNumbers[]`).
|
||||
|
||||
**Alarms** (`AlarmTagset`, all `{Code}.{AlarmName}.<attr>`): `Quality` (int), `InAlarm` (bool),
|
||||
`TimeAlarmOn` (DateTime), `DescAttrName` (string), `Acked` (bool), `TimeAlarmAcked` (DateTime?), `AckMsg` (string).
|
||||
|
||||
### 1.5 The handshake — `MesNotifier.MoveIn` (move-out is identical with `MoveOut*` tags)
|
||||
|
||||
Whole operation is bounded by **`new CancellationTokenSource(30000)` = 30 s**.
|
||||
|
||||
1. **Look up machine** by `SAPID`. Not found → `WasSuccessful=false`, `ErrorText="Failed to find machine with SAPID '{SAPID}'"`, return.
|
||||
2. **Subscribe** to every move-in tag (`Advise(t, cts)`), `await Task.WhenAll(...)`. Any subscription that fails / quality ≠ 192 → `ErrorText="Failed to connect to machine"`.
|
||||
3. **Check ready flag:** `if (!MoveInReadyFlag.Value)` → `ErrorText="Machine move in ready flag not set to true"`, stop.
|
||||
4. **Arm completion watch:** `Task<bool> flagTask = MoveInCompleteFlag.OnValue(true, cts);` (completes when the flag goes `true`, or `false` on the 30 s timeout).
|
||||
5. **Write data + trigger (in parallel, trigger last):** `MoveInOperatorName`, `MoveInJobSequenceNumber`, `MoveInNumberWorkOrders`, `MoveInPartNumbers[]` (padded to 50), `MoveInWorkOrderNumbers[]` (padded to 50), then `MoveInFlag = true`. `await Task.WhenAll(writeTasks)`; any write `!= true` → `ErrorText="Failed to write move in information to machine"`.
|
||||
6. **Wait for completion:** `await Task.WhenAll(flagTask)`.
|
||||
- `flagTask.Result == true` → read results: `WasSuccessful = MoveInSuccessfulFlag.Value`, `ErrorText = MoveInErrorText.Value`, `BatchID = MoveInBatchID.Value` (if ≠ 0).
|
||||
- `flagTask.Result == false` (timed out) → `WasSuccessful=false`, `ErrorText="Timeout waiting for move in information to be processed"`.
|
||||
7. **Cleanup:** `Tags.ForEach(Unadvise)` and return.
|
||||
|
||||
```csharp
|
||||
using (var cts = new CancellationTokenSource(30000)) { // 30 s budget
|
||||
...
|
||||
if (!moveInTagset.MoveInReadyFlag.Value) { /* not-ready error */ }
|
||||
var flagTask = moveInTagset.MoveInCompleteFlag.OnValue(true, cts); // arm completion watch
|
||||
var writeTasks = new List<Task<bool>> {
|
||||
Write(moveInTagset.MoveInOperatorName.Handle, request.OperatorName, cts),
|
||||
Write(moveInTagset.MoveInJobSequenceNumber.Handle, request.JobSequenceNumber, cts),
|
||||
Write(moveInTagset.MoveInNumberWorkOrders.Handle, request.WorkOrders.Count, cts),
|
||||
Write(moveInTagset.MoveInPartNumbers.Handle, request.WorkOrders.Select(wo => wo.PartNumber).ToFixedLength(50), cts),
|
||||
Write(moveInTagset.MoveInWorkOrderNumbers.Handle, request.WorkOrders.Select(wo => wo.WorkOrderNumber).ToFixedLength(50), cts),
|
||||
Write(moveInTagset.MoveInFlag.Handle, true, cts) // TRIGGER — set last
|
||||
};
|
||||
await Task.WhenAll(writeTasks);
|
||||
await Task.WhenAll(flagTask);
|
||||
if (flagTask.Result) {
|
||||
response.WasSuccessful = moveInTagset.MoveInSuccessfulFlag.Value;
|
||||
response.ErrorText = moveInTagset.MoveInErrorText.Value;
|
||||
if (moveInTagset.MoveInBatchID.Value != 0) response.BatchID = moveInTagset.MoveInBatchID.Value;
|
||||
} else {
|
||||
response.WasSuccessful = false;
|
||||
response.ErrorText = "Timeout waiting for move in information to be processed";
|
||||
}
|
||||
moveInTagset.Tags.ForEach(Unadvise);
|
||||
}
|
||||
```
|
||||
|
||||
> There is **no busy-poll loop**: completion is event-driven via the MXAccess `OnDataChange`
|
||||
> callback; the 30 s `CancellationTokenSource` is the only timeout.
|
||||
|
||||
### 1.6 Alarm-status path
|
||||
|
||||
1. Resolve machine from `MachineFilter` (`SAPID` / `Code` / `ZTag` / `MachineID`) — wrong/missing → error.
|
||||
2. Load `MachineAlarm` rows for the machine; apply filters (`FlaggedOnly`, `MinSeverity`, `MaxSeverity`, case-insensitive `NameFilter.Contains`). *(`IncludeTriggered` is read but not used in the filter.)*
|
||||
3. Subscribe + read each alarm's `Quality` and `InAlarm` (30 s budget). Bad quality / read failure → `ErrorText="Failed to read machine alarm status"`.
|
||||
4. For alarms where `InAlarm == true`, additionally read `TimeAlarmOn`, `DescAttrName`, `Acked`, `TimeAlarmAcked`, `AckMsg`.
|
||||
5. Build `AlarmInfo` per triggered alarm; `StatusCode = "Triggered.Acked"` if acked else `"Triggered"`. If `AlarmFilter.IncludeAcked == false`, acked alarms are skipped.
|
||||
6. Unsubscribe; on failure `Alarms` is cleared.
|
||||
|
||||
### 1.7 Outputs / error handling (MES)
|
||||
|
||||
- **Transport status is always 200** for handled responses — success/failure is carried by the body's `WasSuccessful` flag + `ErrorText`. (401/403 only from the auth layer.)
|
||||
- Success: `{ "WasSuccessful": true, "ErrorText": null, "BatchID": <int|null> }`.
|
||||
- Failure/timeout: `{ "WasSuccessful": false, "ErrorText": "<message>", "BatchID": null }`.
|
||||
- Distinct `ErrorText` values: machine-not-found, "Failed to connect to machine", "…ready flag not set to true", "Failed to write … to machine", "Timeout waiting for … to be processed", "Failed to read machine alarm status".
|
||||
|
||||
---
|
||||
|
||||
## 2. Delmia-DNC integration — `DelmiaIntegration` (+ Galaxy `$DelmiaReceiver`)
|
||||
|
||||
### 2.1 Topology — three hops
|
||||
|
||||
```
|
||||
Operator (DelmiaIntegration.exe WinForms)
|
||||
│ ① DelmiaClient ──HTTP POST (form-url-encoded)──▶ DELMIA/Apriso DNC "Downloader.asmx"
|
||||
│ (e.g. http://dnc-app-vd01.zmr.zimmer.com/IntercimService/Downloader.asmx)
|
||||
│ ◀── XML (SearchResults / DownloadResult, ns http://intercim.com/ruleset) ──
|
||||
│ ② recipe file written to disk; WWNotifier.exe launched with CLI args
|
||||
▼
|
||||
WWNotifier.exe ──HTTP POST (JSON RecipeDownload)──▶ WW receiver service (http://wonder-app-vd01:9001/notify)
|
||||
◀── JSON RecipeDownloadResult ── │ ③ MXAccess COM write
|
||||
▼
|
||||
Galaxy object {machine}.$DelmiaReceiver.* (recipe tags + flags)
|
||||
+ ArchestrA scripts ProcessRecipe / Reset
|
||||
```
|
||||
|
||||
**Assemblies in the repo:** `DelmiaContracts` (XML DTO library), `DelmiaIntegration` (`DelmiaClient` + WinForms UI),
|
||||
`WWNotifier.exe` (console notifier), plus test harnesses (`AdminTestUtil`, `DownloadTestUtil`, `TestUI`).
|
||||
|
||||
> **Scope note.** Hops ① and ② are fully in this repo. Hop ③ — the service at `:9001/notify`
|
||||
> that actually performs the MXAccess write — is **not** in the `ZimmerBiomet` Gitea org; only the
|
||||
> JSON contract (below) and the Galaxy-side `$DelmiaReceiver` object (scripts + attributes, exported
|
||||
> under `AA_EXPORT/`) are available. `WWNotificationSystem` *also* uses MXAccess but is an unrelated
|
||||
> tag→email alerting service (port `:9876`, client name `WWNotifierMonitor`) — **not** the recipe receiver.
|
||||
|
||||
### 2.2 DNC server interface (`DelmiaClient`, hop ①)
|
||||
|
||||
- **Transport:** `HttpClient.PostAsync` with `FormUrlEncodedContent`; response is XML deserialized
|
||||
with `XmlSerializer`. Base URL is `DelmiaClient.URL`; per-call `Timeout` default **30 s**.
|
||||
Action is appended to the base URL (`URL.TrimEnd('/') + "/<Action>"`).
|
||||
- **Base URL (from `AdminTestUtil` `DefaultURL`):** `http://dnc-app-vd01.zmr.zimmer.com/IntercimService/Downloader.asmx`.
|
||||
- On any exception the client returns a result object with the error in `ErrorMessage` /
|
||||
`TransferSuccessful=false` (it does not throw to the caller).
|
||||
|
||||
| Method (sync + `…Async`) | POST action | Form fields | Returns |
|
||||
|---|---|---|---|
|
||||
| `Search` | `/Search` | `username, machineID, partNumber, operationNumber` | `SearchResults` |
|
||||
| `RequestProvenDocument` | `/RequestProvenDocument` | `username, machineID, partNumber, operationNumber, workOrderNumber` | `DownloadResult` |
|
||||
| `RequestDocument` | `/RequestDocument` | `username, machineID, partNumber, operationNumber, workOrderNumber, documentKey` | `DownloadResult` |
|
||||
|
||||
DTO field lists (`DelmiaContracts`, XML namespace `http://intercim.com/ruleset`):
|
||||
|
||||
- **`SearchResults`**: `Results` (`List<SearchResult>`), `ErrorMessage` (string).
|
||||
- **`SearchResult`**: `ShopOrderKey` (int), `ShopOrderID` (string), `ShopOrderStatus` (string),
|
||||
`ShopOrderOperKey` (int), `ShopOrderOperID` (string), `ShopOrderOperStatus` (string),
|
||||
`DocumentKey` (int), `DocumentObjectID` (int, with `…Specified` flag), `DocumentName` (string),
|
||||
`DocumentRev` (string), `DocumentStatus` (string), `DocumentURL` (string), `PartID` (string), `PartRev` (string).
|
||||
- **`DownloadResult`**: `UserKey` (int), `UserName` (string), `UserSite` (string), `MachineKey` (int),
|
||||
`MachineID` (string), `MachineSite` (string), `WorkOrderNumber` (string), `ShopOrderKey` (int),
|
||||
`ShopOrderID` (string), `ShopOrderStatus` (string), `ShopOrderOperKey` (int), `ShopOrderOperID` (string),
|
||||
`ShopOrderOperStatus` (string), `DocumentKey` (int), `DocumentName` (string), `DocumentRev` (string),
|
||||
`DocumentStatus` (string), `PartID` (string), `PartRev` (string), **`TransferSuccessful` (bool)**,
|
||||
**`ErrorMessage` (string)**.
|
||||
- **`MachineInfo`** (contract present; not called by current code): `MachineKey` (int), `MachineID` (string),
|
||||
`MachineName` (string), `DownloadPath` (string), `MachineDescription` (string), `MachineSite` (string), `MachineStatus` (string).
|
||||
- **`UserInfo`**: `UserKey` (int), `UserName` (string), `UserSite` (string), `IsActive` (bool).
|
||||
|
||||
**Recipe file:** the downloaded document is a key/value recipe file parsed by `DelmiaIntegration/Models/RecipeSet.cs`
|
||||
(`KEY,VALUE` lines; typed accessors `GetString/GetInt/GetBool/GetFloat/GetDouble/...`). It is written to disk;
|
||||
its path is what gets handed to `WWNotifier` (`--downloadpath`).
|
||||
|
||||
### 2.3 WWNotifier (hop ②) — invocation & handoff contract
|
||||
|
||||
`WWNotifier.exe` (uses `CommandLineParser`); CLI options (`CommandLineOptions`):
|
||||
|
||||
| Short | Long | Required | Field |
|
||||
|---|---|---|---|
|
||||
| `-d` | `--downloadpath` | yes | `DownloadPath` (recipe file path) |
|
||||
| `-m` | `--machine` | yes | `MachineCode` |
|
||||
| `-w` | `--workorder` | yes | `WorkOrderNumber` |
|
||||
| `-p` | `--partnumber` | yes | `PartNumber` |
|
||||
| `-s` | `--seqop` | no | `JobStepNumber` |
|
||||
| `-u` | `--username` | no | `Username` |
|
||||
|
||||
Config (`WWNotifier/App.config`): `NotifyURL = http://wonder-app-vd01.zmr.zimmer.com:9001/notify`
|
||||
(comma-separated list allowed — tried in order until one succeeds), `NotifyTimeout = 30` (seconds,
|
||||
applied as the global Flurl HTTP timeout).
|
||||
|
||||
Handoff (Flurl): `url.PostJsonAsync(recipeDownload).ReceiveJson<RecipeDownloadResult>()`.
|
||||
|
||||
- **Request body `RecipeDownload`** (JSON): `MachineCode`, `DownloadPath`, `WorkOrderNumber`, `PartNumber`, `JobStepNumber`, `Username` (all string).
|
||||
- **Response body `RecipeDownloadResult`** (JSON): `Result` (bool), `ResultText` (string).
|
||||
- **Outputs:** prints `YES` and exit code `0` on success; prints `NO` + a message and sets exit code `-1`
|
||||
on failure (parse error, missing `NotifyURL`/`NotifyTimeout`, `Result==false`, or HTTP exception).
|
||||
*(Caveat: on a caught exception it logs `error.InnerException.Message`, which throws a NRE when there is no inner exception — so bare transport errors surface only as a generic failure.)*
|
||||
|
||||
### 2.4 MXAccess write — Galaxy `$DelmiaReceiver` object (hop ③)
|
||||
|
||||
The receiver service maps `RecipeDownload` fields onto the `$DelmiaReceiver` object instance selected by
|
||||
`MachineCode`. Object attributes (from `AA_EXPORT/.../$DelmiaReceiver` export) and their roles:
|
||||
|
||||
| Attribute | Type | Role | Maps from |
|
||||
|---|---|---|---|
|
||||
| `ReadyFlag` | Boolean | gate — receiver expects `true` before writing | — |
|
||||
| `DownloadPath` | String | data | `RecipeDownload.DownloadPath` |
|
||||
| `WorkOrderNumber` | String | data | `RecipeDownload.WorkOrderNumber` |
|
||||
| `PartNumber` | String | data | `RecipeDownload.PartNumber` |
|
||||
| `JobStepNumber` | String | data | `RecipeDownload.JobStepNumber` |
|
||||
| `Username` | String | data | `RecipeDownload.Username` |
|
||||
| `RecipeDownloadFlag` | Boolean | **trigger** — set `true` to start processing | — |
|
||||
| `RecipeProcessedFlag` | Boolean | completion — handshake waits on this | — |
|
||||
| `RecipeProcessResult` | Boolean | result | → `RecipeDownloadResult.Result` |
|
||||
| `RecipeProcessResultText` | String | result | → `RecipeDownloadResult.ResultText` |
|
||||
|
||||
*(`MachineCode` selects which receiver instance; it is not itself a written attribute.)*
|
||||
|
||||
**Galaxy-side handshake (authoritative — ArchestrA scripts on `$DelmiaReceiver`):**
|
||||
|
||||
`ProcessRecipe` (runs when `RecipeDownloadFlag` is set):
|
||||
```
|
||||
Me.RecipeDownloadFlag = false; ' clear trigger
|
||||
Me.ReadyFlag = false; ' clear ready
|
||||
try
|
||||
Me.RecipeProcessResult = true;
|
||||
Me.RecipeProcessResultText = "Success";
|
||||
catch
|
||||
Me.RecipeProcessResult = false;
|
||||
Me.RecipeProcessResultText = "Failed to read recipe file";
|
||||
endtry;
|
||||
Me.RecipeProcessedFlag = true; ' signal completion
|
||||
```
|
||||
|
||||
`Reset` (clears the slot for the next download):
|
||||
```
|
||||
Me.RecipeDownloadFlag = false; Me.RecipeProcessedFlag = false;
|
||||
Me.RecipeProcessResult = false; Me.RecipeProcessResultText = "";
|
||||
Me.DownloadPath = ""; Me.WorkOrderNumber = ""; Me.PartNumber = "";
|
||||
Me.JobStepNumber = ""; Me.Username = "";
|
||||
```
|
||||
|
||||
**Receiver-side sequence (inferred** — mirrors the MES handshake and is driven by the flags above; the C# source is not in the repo**):**
|
||||
1. resolve the `$DelmiaReceiver` instance from `MachineCode`;
|
||||
2. (optionally) verify `ReadyFlag == true`;
|
||||
3. write `DownloadPath`, `WorkOrderNumber`, `PartNumber`, `JobStepNumber`, `Username`;
|
||||
4. set `RecipeDownloadFlag = true` (trigger) → Galaxy `ProcessRecipe` fires;
|
||||
5. wait for `RecipeProcessedFlag == true`, bounded by the request timeout;
|
||||
6. read `RecipeProcessResult` → `Result`, `RecipeProcessResultText` → `ResultText`; return the `RecipeDownloadResult` JSON;
|
||||
7. `Reset` the object.
|
||||
|
||||
### 2.5 Outputs / error handling (Delmia)
|
||||
|
||||
- **DNC server call:** failures are swallowed into the returned DTO — `SearchResults.ErrorMessage`, or
|
||||
`DownloadResult.TransferSuccessful=false` + `ErrorMessage="Failed to call Delmia web service at '<URL>'."`.
|
||||
- **Notify handoff:** `RecipeDownloadResult.Result` (bool) + `ResultText` (string); `WWNotifier` exit code
|
||||
`0` (`YES`) / `-1` (`NO`).
|
||||
- **Galaxy script:** `RecipeProcessResultText` is `"Success"` or `"Failed to read recipe file"`.
|
||||
|
||||
---
|
||||
|
||||
## 3. Side-by-side summary
|
||||
|
||||
| | MES (`MESAPI`) | Delmia DNC (`DelmiaIntegration`) |
|
||||
|---|---|---|
|
||||
| Caller | Camstar MES (HTTP/JSON) | Operator UI → DELMIA DNC server, then WWNotifier |
|
||||
| API style | ServiceStack REST, `POST /mes/*` | DNC = form-url-encoded → XML; notify = JSON POST |
|
||||
| Who writes MXAccess | the service (`MesNotifier`, in-repo) | external `:9001/notify` receiver (source not in repo) |
|
||||
| MXAccess client | `LMXProxyServerClass`, register `"MesNotifier"` | `LMXProxyServerClass` (receiver), Galaxy `$DelmiaReceiver` scripts |
|
||||
| Target object | `{MachineCode}.MesReceiver.*` | `{MachineCode}.$DelmiaReceiver` instance |
|
||||
| Ready / trigger / complete | `MoveInReadyFlag` / `MoveInFlag` / `MoveInCompleteFlag` | `ReadyFlag` / `RecipeDownloadFlag` / `RecipeProcessedFlag` |
|
||||
| Result / error | `MoveInSuccessfulFlag` / `MoveInErrorText` (+`MoveInBatchID`) | `RecipeProcessResult` / `RecipeProcessResultText` |
|
||||
| Timeout | 30 s (`CancellationTokenSource(30000)`), event-driven | 30 s HTTP (`NotifyTimeout`); Galaxy wait at receiver |
|
||||
|
||||
---
|
||||
|
||||
## 4. Sources & open gaps
|
||||
|
||||
**Repos (Gitea `ZimmerBiomet`, `master`):**
|
||||
- MES: `MESAPI` — `APIServer.ServiceInterface/MesServices.cs`, `MesNotifier.cs`,
|
||||
`Mes{MoveIn,MoveOut}Tagset.cs`, `AlarmTagset.cs`, `Tag.cs`/`OnValueTask.cs`;
|
||||
`APIServer.ServiceModel/Types/*`; `APIServer/AppHost.cs`, `App.config`.
|
||||
- Delmia: `DelmiaIntegration` — `DelmiaIntegration/DelmiaClient.cs`, `Models/RecipeSet.cs`;
|
||||
`WWNotifier/Program.cs`, `CommandLineOptions.cs`, `Models/RecipeDownload(Result).cs`, `App.config`;
|
||||
`DelmiaContracts/*`.
|
||||
|
||||
**Galaxy export (`~/Desktop/AA_EXPORT/EXTRACTED/$DelmiaReceiver`):** `scripts/ProcessRecipe.txt`,
|
||||
`scripts/Reset.txt`, `$DelmiaReceiver.top_level_attributes.csv`. MES receiver object attributes: [`mesrec.md`](mesrec.md).
|
||||
|
||||
**Open gaps / to verify against the box:**
|
||||
1. The Delmia recipe **`/notify` receiver service** source (the actual MXAccess writer at `wonder-app-vd01:9001`)
|
||||
was not found in the Gitea org — §2.4 receiver steps are inferred from the contract + Galaxy scripts.
|
||||
2. MES tag prefix in code is `{Code}.MesReceiver.*`, while the live probe in `mesrec.md` shows a top-level
|
||||
`MESReceiver_002` instance — confirm the exact contained-name/instance convention on the live Galaxy.
|
||||
3. PROD `HttpListener`/DB host values should be read from the deployed `App.config`, not assumed.
|
||||
@@ -0,0 +1,77 @@
|
||||
# MESReceiver object — attributes (wonder-app-vd03 Galaxy)
|
||||
|
||||
Source: live AVEVA Galaxy DB **`ZB`** on **wonder-app-vd03** (the MxAccessGateway box),
|
||||
read via the gateway's own `AttributesSql` (recursive `deployed_package_chain` over
|
||||
`dynamic_attribute` / `attribute_definition`) run with `sqlcmd` over ssh — `2026-06-16`.
|
||||
|
||||
## Instance probed
|
||||
- **TagName:** `MESReceiver_002` (GobjectId `5909`, ParentGobjectId `5908`, HostedBy `5049`)
|
||||
- **TemplateChain:** `$MESDemo.MESReceiver` → `$MESReceiver` → `$gUserDefined` → `$UserDefined`
|
||||
- **CategoryId:** 10, `IsArea = false`
|
||||
- There are many MESReceiver instances on this Galaxy (1,253 `MESReceiver` references in the
|
||||
`galaxy-snapshot.json` hierarchy cache). `MESReceiver_002` is representative of the template.
|
||||
|
||||
> Note: the cached `galaxy-snapshot.json` (gateway Server dir) holds the **object hierarchy only**
|
||||
> — it carries **no attributes**, so attribute discovery requires the DB query (or gRPC `galaxy-discover`).
|
||||
|
||||
## Container number
|
||||
There is **no attribute literally named `ContainerNumber`**. The MES container number is carried by:
|
||||
- **`MoveInMesContainerNum`** — String (move-in interface)
|
||||
- **`MoveOutMesContainerNum`** — String (move-out interface)
|
||||
|
||||
Full tag references: `MESReceiver_002.MoveInMesContainerNum`, `MESReceiver_002.MoveOutMesContainerNum`.
|
||||
|
||||
## MES interface UDAs (attribute-category 10, security-classification 1 = secured/operate)
|
||||
|
||||
### Move-In
|
||||
| Attribute | Type | Full tag reference |
|
||||
|---|---|---|
|
||||
| **MoveInMesContainerNum** | String | `MESReceiver_002.MoveInMesContainerNum` |
|
||||
| MoveInBatchID | Integer | `MESReceiver_002.MoveInBatchID` |
|
||||
| MoveInJobSequenceNumber | String | `MESReceiver_002.MoveInJobSequenceNumber` |
|
||||
| MoveInNumberWorkOrders | Integer | `MESReceiver_002.MoveInNumberWorkOrders` |
|
||||
| MoveInWorkOrderNumbers | String[] | `MESReceiver_002.MoveInWorkOrderNumbers[]` |
|
||||
| MoveInPartNumbers | String[] | `MESReceiver_002.MoveInPartNumbers[]` |
|
||||
| MoveInOperatorName | String | `MESReceiver_002.MoveInOperatorName` |
|
||||
| MoveInFlag | Boolean | `MESReceiver_002.MoveInFlag` |
|
||||
| MoveInReadyFlag | Boolean | `MESReceiver_002.MoveInReadyFlag` |
|
||||
| MoveInCompleteFlag | Boolean | `MESReceiver_002.MoveInCompleteFlag` |
|
||||
| MoveInSuccessfulFlag | Boolean | `MESReceiver_002.MoveInSuccessfulFlag` |
|
||||
| MoveInErrorText | String | `MESReceiver_002.MoveInErrorText` |
|
||||
|
||||
### Move-Out (symmetric; no JobSequenceNumber)
|
||||
| Attribute | Type | Full tag reference |
|
||||
|---|---|---|
|
||||
| **MoveOutMesContainerNum** | String | `MESReceiver_002.MoveOutMesContainerNum` |
|
||||
| MoveOutBatchID | Integer | `MESReceiver_002.MoveOutBatchID` |
|
||||
| MoveOutNumberWorkOrders | Integer | `MESReceiver_002.MoveOutNumberWorkOrders` |
|
||||
| MoveOutWorkOrderNumbers | String[] | `MESReceiver_002.MoveOutWorkOrderNumbers[]` |
|
||||
| MoveOutPartNumbers | String[] | `MESReceiver_002.MoveOutPartNumbers[]` |
|
||||
| MoveOutOperatorName | String | `MESReceiver_002.MoveOutOperatorName` |
|
||||
| MoveOutFlag | Boolean | `MESReceiver_002.MoveOutFlag` |
|
||||
| MoveOutReadyFlag | Boolean | `MESReceiver_002.MoveOutReadyFlag` |
|
||||
| MoveOutCompleteFlag | Boolean | `MESReceiver_002.MoveOutCompleteFlag` |
|
||||
| MoveOutSuccessfulFlag | Boolean | `MESReceiver_002.MoveOutSuccessfulFlag` |
|
||||
| MoveOutErrorText | String | `MESReceiver_002.MoveOutErrorText` |
|
||||
|
||||
## Standard ArchestrA `$UserDefined` / system attributes (also present)
|
||||
`AlarmCntsBySeverity[]`, `AlarmCntsBySeverityEnableShelved[]`, `AlarmInhibit`, `AlarmMode`,
|
||||
`AlarmModeCmd`, `AlarmMostUrgentAcked`, `AlarmMostUrgentInAlarm`, `AlarmMostUrgentMode`,
|
||||
`AlarmMostUrgentSeverity`, `AlarmMostUrgentShelved`, `AliasName`, `Area`, `CmdData`, `CodeBase`,
|
||||
`ConfigVersion`, `ContainedName`, `Container`, `Errors[]`, `ExecutionRelatedObject`,
|
||||
`ExecutionRelativeOrder`, `Extensions`, `HierarchicalName`, `Host`, `InAlarm`, `MinorVersion`,
|
||||
`PropagatedAlarmInhibit`, `ScanState`, `ScanStateCmd`, `SecurityGroup`, `ShortDesc`, `Tagname`,
|
||||
`UDAs`, `UserAttrData`, `PropagatedAlarmInhibit`.
|
||||
|
||||
## How to reproduce
|
||||
1. `MESReceiver` instances are in the gateway hierarchy cache
|
||||
`E:\ApiInstall\MxGateway\Server\galaxy-snapshot.json` (objects only, no attrs).
|
||||
2. Attribute list: run the gateway's `AttributesSql`
|
||||
(`src/.../Server/Galaxy/GalaxyRepository.cs`) scoped to one instance via
|
||||
`... WHERE g.is_template = 0 AND g.deployed_package_id <> 0 AND g.tag_name = 'MESReceiver_002'`
|
||||
in the `deployed_package_chain` anchor, then `sqlcmd -S (local) -d ZB -U wwadmin -P <pwd>`
|
||||
on the box (Galaxy creds from `MxGateway.Galaxy.ConnectionString` in the gateway appsettings).
|
||||
3. Live values would need an MXAccess read through the gateway (gRPC), not the repo SQL.
|
||||
|
||||
In OtOpcUa (Galaxy-as-standard-driver model) these bind as ordinary equipment tags:
|
||||
`Tag{ DriverInstanceId = GalaxyMxGateway, TagConfig = {"FullName":"MESReceiver_002.MoveInMesContainerNum"} }`.
|
||||
@@ -0,0 +1,212 @@
|
||||
# New Jersey — DARS reactors Z28061 / Z28062 (wonder Galaxy)
|
||||
|
||||
Source: live AVEVA Galaxy DB **`ZB`** on **wonder-app-vd03** (MxAccessGateway box), via the gateway's `AttributesSql` over `sqlcmd`/ssh — `2026-06-16`. Lists the **meaningful (user-defined, cat-10) attributes**; ArchestrA system attributes (alarm framework, identity, security) and per-field config sub-attributes (`.EngUnits`, `.TrendHi`, `.Dev.*`, history/alarm settings) and scripts are omitted.
|
||||
|
||||
## Hierarchy
|
||||
```
|
||||
NewJersey (area)
|
||||
└─ CVDAisle_1 ($CVDAisle, gobject 7243)
|
||||
├─ Z28061 ($DARSReactor, gobject 7171)
|
||||
│ ├─ Left (Left_002, $DARSReactor.Left, gobject 7172)
|
||||
│ └─ Right (Right_002, $DARSReactor.Right, gobject 7173)
|
||||
└─ Z28062 ($DARSReactor, gobject 7202)
|
||||
├─ Left (Left_003, $DARSReactor.Left, gobject 7203)
|
||||
└─ Right (Right_003, $DARSReactor.Right, gobject 7204)
|
||||
```
|
||||
(`Z28061Sim`, gobject 7146, is a simulator sibling of Z28061.)
|
||||
|
||||
## Objects
|
||||
| Object | GobjectId | Template | Compound path | UDA attrs | Total attrs |
|
||||
|---|---|---|---|---|---|
|
||||
| Left_002 | 7172 | `$DARSReactor.Left` | Z28061.Left | 72 | 1907 |
|
||||
| Left_003 | 7203 | `$DARSReactor.Left` | Z28062.Left | 72 | 1907 |
|
||||
| Right_002 | 7173 | `$DARSReactor.Right` | Z28061.Right | 72 | 1907 |
|
||||
| Right_003 | 7204 | `$DARSReactor.Right` | Z28062.Right | 72 | 1907 |
|
||||
| Z28061 | 7171 | `$DARSReactor` | Z28061 | 16 | 195 |
|
||||
| Z28062 | 7202 | `$DARSReactor` | Z28062 | 16 | 195 |
|
||||
|
||||
> Attribute sets are template-defined: `Z28061`≡`Z28062`, `Left_002`≡`Left_003`, `Right_002`≡`Right_003`. Listed once per template.
|
||||
|
||||
## $DARSReactor — Z28061 / Z28062 (reactor body)
|
||||
|
||||
**Instances:** `Z28061` (Z28061, gobject 7171), `Z28062` (Z28062, gobject 7202)
|
||||
**Template:** `$DARSReactor` · **16 meaningful (UDA) attributes** · 195 total incl. system + config sub-attributes.
|
||||
|
||||
| Attribute | Type | Hist | Alarm |
|
||||
|---|---|---|---|
|
||||
| ChlorineFlow | Integer | ✓ | |
|
||||
| ConfirmTimeoutSec | Integer | | |
|
||||
| Deadband | Double | | |
|
||||
| HeartbeatInterval | Integer | | |
|
||||
| HeartbeatRequest | Boolean | | |
|
||||
| HeartbeatTimeout | Integer | | |
|
||||
| HeartbeatTimeoutAlarm | Boolean | | ✓ |
|
||||
| HydrogenFlow | Integer | ✓ | |
|
||||
| LeakTestMaxDelta | Integer | | |
|
||||
| LeakTestMinDuration | Integer | | |
|
||||
| LeakTestTimeout | Integer | | |
|
||||
| MachineCode | String | | |
|
||||
| MachineDescription | String | | |
|
||||
| MachineID | String | | |
|
||||
| Runtime_Alert | Boolean | | |
|
||||
| TrunkPressure | Float | ✓ | |
|
||||
|
||||
## $DARSReactor.Left — Left_002 / Left_003 (left chamber)
|
||||
|
||||
**Instances:** `Left_002` (Z28061.Left, gobject 7172), `Left_003` (Z28062.Left, gobject 7203)
|
||||
**Template:** `$DARSReactor.Left` · **72 meaningful (UDA) attributes** · 1907 total incl. system + config sub-attributes.
|
||||
|
||||
| Attribute | Type | Hist | Alarm |
|
||||
|---|---|---|---|
|
||||
| AIR_SVC | String | | |
|
||||
| AirAvg | Integer | ✓ | |
|
||||
| AirFlow | Integer | ✓ | |
|
||||
| AirFlowDevAlertPcnt | Float | ✓ | |
|
||||
| AR_SVC | String | | |
|
||||
| ArAvg | Integer | ✓ | |
|
||||
| ArgonFlow | Integer | ✓ | |
|
||||
| ChlorineFlowR | Integer | | |
|
||||
| CL2_SVC | String | | |
|
||||
| Cl2Avg | Integer | ✓ | |
|
||||
| ClFlowDevAlertPcnt | Float | ✓ | |
|
||||
| CoilDiameter | Boolean | ✓ | |
|
||||
| CoilHeight | Integer | ✓ | |
|
||||
| ContainerID | String | ✓ | |
|
||||
| ContainerLoaded | Boolean | ✓ | |
|
||||
| CurrentStep | Integer | | |
|
||||
| CycleEndConfirm | Boolean | | |
|
||||
| CycleEndNotify | Boolean | | |
|
||||
| CycleEndTimeoutAlarm | Boolean | | |
|
||||
| CycleRunning | Boolean | ✓ | |
|
||||
| CycleStartConfirm | Boolean | | |
|
||||
| CycleStartNotify | Boolean | | |
|
||||
| CycleStartTimeoutAlarm | Boolean | | |
|
||||
| Deadband | Double | | |
|
||||
| DelmiaJobStep | Integer | | |
|
||||
| FRR_Reset | Boolean | ✓ | |
|
||||
| FRR_Runtime | Integer | ✓ | |
|
||||
| FRR_Warning | Boolean | ✓ | ✓ |
|
||||
| FurnaceTemp | Integer | ✓ | |
|
||||
| FurnaceTempOverAlarmDev | Integer | ✓ | |
|
||||
| FurnaceTempOverAlertDev | Integer | ✓ | |
|
||||
| FurnaceTempUnderAlarmDev | Integer | ✓ | |
|
||||
| FurnaceTempUnderAlertDev | Integer | ✓ | |
|
||||
| H2_SVC | String | | |
|
||||
| H2Avg | Integer | ✓ | |
|
||||
| HydrogenFlowR | Integer | | |
|
||||
| HyFlowDevAlertPcnt | Float | ✓ | |
|
||||
| MachineBatchID | Integer | | |
|
||||
| MachineBatchWOID | Integer | | |
|
||||
| MachineCycleID | Integer | | |
|
||||
| MantleTemp | Integer | ✓ | |
|
||||
| MoveInReady | Boolean | | |
|
||||
| MoveOutReady | Boolean | | |
|
||||
| MTA_Lockout | Boolean | ✓ | ✓ |
|
||||
| MTA_Reset | Boolean | ✓ | |
|
||||
| MTA_Runtime | Integer | ✓ | |
|
||||
| MTA_Warning | Boolean | ✓ | ✓ |
|
||||
| NumOfParts | Integer | ✓ | |
|
||||
| NumOfTurns | Integer | ✓ | |
|
||||
| OperatorID | String | ✓ | |
|
||||
| PartNumber | String | ✓ | |
|
||||
| PID_Alarm | Boolean | | ✓ |
|
||||
| PID_AlarmString | String | | |
|
||||
| PID_CV | Integer | ✓ | |
|
||||
| PotNumber | String | ✓ | |
|
||||
| PotTempAvg | Integer | ✓ | |
|
||||
| PreviousStep | Integer | | |
|
||||
| ReactorTempAvg | Integer | ✓ | |
|
||||
| RunDuration | Integer | | |
|
||||
| RunEndTime | Time | | |
|
||||
| RunStartTime | Time | | |
|
||||
| SampleCount | Integer | | |
|
||||
| StartingWeight | Integer | ✓ | |
|
||||
| TableSide | Boolean | | |
|
||||
| TableStatus | Integer | | |
|
||||
| TC_Furnace_ID | String | | |
|
||||
| TC_Mantle_ID | String | | |
|
||||
| TC_Spare_ID | String | | |
|
||||
| Vacuum | Float | ✓ | |
|
||||
| VacuumAvg | Float | ✓ | |
|
||||
| VacuumMode | Boolean | ✓ | |
|
||||
| WorkOrder | String | | |
|
||||
|
||||
## $DARSReactor.Right — Right_002 / Right_003 (right chamber)
|
||||
|
||||
**Instances:** `Right_002` (Z28061.Right, gobject 7173), `Right_003` (Z28062.Right, gobject 7204)
|
||||
**Template:** `$DARSReactor.Right` · **72 meaningful (UDA) attributes** · 1907 total incl. system + config sub-attributes.
|
||||
|
||||
| Attribute | Type | Hist | Alarm |
|
||||
|---|---|---|---|
|
||||
| AIR_SVC | String | | |
|
||||
| AirAvg | Integer | ✓ | |
|
||||
| AirFlow | Integer | ✓ | |
|
||||
| AirFlowDevAlertPcnt | Float | ✓ | |
|
||||
| AR_SVC | String | | |
|
||||
| ArAvg | Integer | ✓ | |
|
||||
| ArgonFlow | Integer | ✓ | |
|
||||
| ChlorineFlowR | Integer | | |
|
||||
| CL2_SVC | String | | |
|
||||
| Cl2Avg | Integer | ✓ | |
|
||||
| ClFlowDevAlertPcnt | Float | ✓ | |
|
||||
| CoilDiameter | Boolean | ✓ | |
|
||||
| CoilHeight | Integer | ✓ | |
|
||||
| ContainerID | String | ✓ | |
|
||||
| ContainerLoaded | Boolean | ✓ | |
|
||||
| CurrentStep | Integer | | |
|
||||
| CycleEndConfirm | Boolean | | |
|
||||
| CycleEndNotify | Boolean | | |
|
||||
| CycleEndTimeoutAlarm | Boolean | | |
|
||||
| CycleRunning | Boolean | ✓ | |
|
||||
| CycleStartConfirm | Boolean | | |
|
||||
| CycleStartNotify | Boolean | | |
|
||||
| CycleStartTimeoutAlarm | Boolean | | |
|
||||
| Deadband | Double | | |
|
||||
| DelmiaJobStep | Integer | | |
|
||||
| FRR_Reset | Boolean | ✓ | |
|
||||
| FRR_Runtime | Integer | ✓ | |
|
||||
| FRR_Warning | Boolean | ✓ | ✓ |
|
||||
| FurnaceTemp | Integer | ✓ | |
|
||||
| FurnaceTempOverAlarmDev | Integer | ✓ | |
|
||||
| FurnaceTempOverAlertDev | Integer | ✓ | |
|
||||
| FurnaceTempUnderAlarmDev | Integer | ✓ | |
|
||||
| FurnaceTempUnderAlertDev | Integer | ✓ | |
|
||||
| H2_SVC | String | | |
|
||||
| H2Avg | Integer | ✓ | |
|
||||
| HydrogenFlowR | Integer | | |
|
||||
| HyFlowDevAlertPcnt | Float | ✓ | |
|
||||
| MachineBatchID | Integer | | |
|
||||
| MachineBatchWOID | Integer | | |
|
||||
| MachineCycleID | Integer | | |
|
||||
| MantleTemp | Integer | ✓ | |
|
||||
| MoveInReady | Boolean | | |
|
||||
| MoveOutReady | Boolean | | |
|
||||
| MTA_Lockout | Boolean | ✓ | ✓ |
|
||||
| MTA_Reset | Boolean | ✓ | |
|
||||
| MTA_Runtime | Integer | ✓ | |
|
||||
| MTA_Warning | Boolean | ✓ | ✓ |
|
||||
| NumOfParts | Integer | ✓ | |
|
||||
| NumOfTurns | Integer | ✓ | |
|
||||
| OperatorID | String | ✓ | |
|
||||
| PartNumber | String | ✓ | |
|
||||
| PID_Alarm | Boolean | | ✓ |
|
||||
| PID_AlarmString | String | | |
|
||||
| PID_CV | Integer | ✓ | |
|
||||
| PotNumber | String | ✓ | |
|
||||
| PotTempAvg | Integer | ✓ | |
|
||||
| PreviousStep | Integer | | |
|
||||
| ReactorTempAvg | Integer | ✓ | |
|
||||
| RunDuration | Integer | | |
|
||||
| RunEndTime | Time | | |
|
||||
| RunStartTime | Time | | |
|
||||
| SampleCount | Integer | | |
|
||||
| StartingWeight | Integer | ✓ | |
|
||||
| TableSide | Boolean | | |
|
||||
| TableStatus | Integer | | |
|
||||
| TC_Furnace_ID | String | | |
|
||||
| TC_Mantle_ID | String | | |
|
||||
| TC_Spare_ID | String | | |
|
||||
| Vacuum | Float | ✓ | |
|
||||
| VacuumAvg | Float | ✓ | |
|
||||
| VacuumMode | Boolean | ✓ | |
|
||||
| WorkOrder | String | | |
|
||||
@@ -109,13 +109,49 @@ Defaults target docker-dev; override via flags or env:
|
||||
- `../galaxy-hierarchy.json` — the source of truth, pulled live from the gateway
|
||||
- `requirements.txt`, `.venv/`
|
||||
|
||||
## Scope note — company-UNS shape
|
||||
## Company-shape overlay (`populate-equipment`)
|
||||
|
||||
This tool loads the galaxy in its **native hierarchy**
|
||||
(`OtOpcUa/TestMachine_NNN/<signal>`), which is the only shape that can carry live
|
||||
Galaxy values: OtOpcUa forbids the `GalaxyMxGateway` driver in an `Equipment`
|
||||
namespace, so a custom `Enterprise/Site/Area/Line/Equipment` UNS (e.g. the
|
||||
Northwind model in `../company-uns.json`) must be a separate **Equipment**
|
||||
namespace fed by an `OpcUaClient` driver + `UnsMappingTable` that remaps this
|
||||
mirror. That overlay is the designed next layer; `../company-uns.json` already
|
||||
carries the area/line/equipment → galaxy-ref mapping it needs.
|
||||
Besides the galaxy-native mirror, the tool can load the **Northwind company
|
||||
shape** (`filling / line-1 / rinser-01 / speed-rpm`) as a second, **Equipment**-kind
|
||||
namespace (`nw-uns`, in cluster `MAIN`) from `../company-uns.json`. Each company
|
||||
signal is a **VirtualTag** (+ a `Script`) whose script simply mirrors the live
|
||||
galaxy-mirror tag for that signal:
|
||||
|
||||
```csharp
|
||||
return ctx.GetTag("TestMachine_001.TestDouble").Value;
|
||||
```
|
||||
|
||||
so the company shape carries live **VALUES** driven off the same Galaxy source — no
|
||||
driver, no `BadWaitingForInitialData` once the galaxy mirror is up. The `ctx.GetTag`
|
||||
literal is the signal's `source.fullTagReference`; the engine's `DependencyExtractor`
|
||||
harvests it and subscribes the VirtualTag to that galaxy-mirror tag. This needs
|
||||
OtOpcUa `master` ≥ the Equipment-namespace VirtualTag materialisation milestone (WS-3),
|
||||
which materialises `VirtualTag`/`Script` rows on deploy and added the **headless
|
||||
deploy** endpoint.
|
||||
|
||||
```bash
|
||||
./.venv/bin/python otopcua_uns.py populate-equipment # 3 areas / 8 lines / 40 equipment / 1036 VirtualTags
|
||||
curl -s -X POST http://localhost:9200/api/deployments -H 'X-Api-Key: docker-dev-deploy-key' # headless deploy
|
||||
./.venv/bin/python otopcua_uns.py verify-equipment --expect 1036 --require-good 396 --wait --wait-seconds 300 # structure + live values
|
||||
```
|
||||
|
||||
> **Verified live 2026-06-07** (OtOpcUa `feat/equipment-namespace-live-values`): galaxy mirror
|
||||
> **396/396 Good**, company overlay **396 Good** on `opc.tcp://localhost:4840`, `VERIFY-EQUIPMENT: PASS`.
|
||||
> Why 396 of 1036? The shipped `company-uns.json` invents **1036 distinct** `ctx.GetTag` refs, but only
|
||||
> **396** of them match a real galaxy-mirror tag — so 396 signals are backed by a live source (and all 396
|
||||
> go Good); the other 640 cite synthetic refs with no galaxy tag (`BadNodeIdUnknown`). That ratio is a
|
||||
> property of the company model, not the streaming path — **every signal with a resolvable live source
|
||||
> streams Good.** So `--require-good 396` is the meaningful gate for the current model. Survives a node
|
||||
> restart with no re-deploy (the bootstrap-restore path re-materialises + re-applies the VirtualTags).
|
||||
|
||||
UNS folders carry the friendly **DisplayName** (`filling`); the BrowseName/NodeId
|
||||
stay the stable logical Id (`nw-area-filling`) — standard OPC UA. **No driver:** the
|
||||
company signals are VirtualTags (which link to Equipment + a Script, not a driver); a
|
||||
placeholder `nw-uns-modbus` driver is kept only because an Equipment namespace is
|
||||
expected to have one, but no `Tag` binds to it. `verify-equipment --require-good N`
|
||||
reads each leaf's value and asserts at least N are Good (default `0` = structure-only,
|
||||
back-compat); `--wait` polls until the deploy + change-triggered evaluations land.
|
||||
Tracked in `OtOpcUa/docs/plans/2026-06-06-equipment-namespace-materialization-scope.md` (WS-3).
|
||||
|
||||
`clean` removes both the mirror tags and the company overlay (the `VirtualTag` +
|
||||
`Script` rows, in FK-safe order, plus the namespace/driver/equipment/areas/lines).
|
||||
|
||||
@@ -25,12 +25,21 @@ the `nw-` id prefix so `clean` can remove them without touching other config.
|
||||
|
||||
Idempotent: populate upserts by TagId; re-running is a no-op when unchanged.
|
||||
|
||||
There are TWO overlays:
|
||||
• the galaxy-native mirror (`populate`) — SystemPlatform driver Tags, 396 tags;
|
||||
• the Northwind company shape (`populate-equipment`) — an Equipment-kind namespace
|
||||
whose 1036 signals are VirtualTags. Each VirtualTag's Script simply mirrors the
|
||||
live galaxy-mirror tag (`return ctx.GetTag("<fullTagReference>").Value;`), so the
|
||||
company shape carries live VALUES driven off the same Galaxy source.
|
||||
|
||||
Subcommands:
|
||||
generate Build the load plan from galaxy-hierarchy.json (writes load-plan.json)
|
||||
populate Upsert the SystemPlatform mirror Tag rows into the config DB
|
||||
verify Check DB rows present + live OPC UA values are Good on :4840
|
||||
status Show config-DB + address-space state
|
||||
clean Delete all nw-* mirror Tag rows
|
||||
generate Build the load plan from galaxy-hierarchy.json (writes load-plan.json)
|
||||
populate Upsert the SystemPlatform mirror Tag rows into the config DB
|
||||
populate-equipment Load the company shape as VirtualTag+Script rows (mirror the galaxy tags)
|
||||
verify Check DB rows present + live OPC UA values are Good on :4840
|
||||
verify-equipment Browse the company tree; --require-good asserts live values
|
||||
status Show config-DB + address-space state
|
||||
clean Delete all nw-* mirror Tags + the company VirtualTag/Script overlay
|
||||
|
||||
Deploy is a human-gated AdminUI action (no SQL/REST trigger exists); populate
|
||||
and clean print the reminder and `verify --wait` polls until it lands.
|
||||
@@ -38,11 +47,13 @@ and clean print the reminder and `verify --wait` polls until it lands.
|
||||
Deps: pymssql, asyncua (see requirements.txt; use the bundled .venv).
|
||||
"""
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import time
|
||||
import uuid
|
||||
|
||||
# ── config (overridable via env / flags) ───────────────────────────────────
|
||||
DEF_MSSQL = dict(
|
||||
@@ -58,9 +69,26 @@ DEF_GALAXY_JSON = os.environ.get(
|
||||
"OTOPCUA_GALAXY_JSON",
|
||||
os.path.join(os.path.dirname(__file__), "..", "galaxy-hierarchy.json"),
|
||||
)
|
||||
ID_PREFIX = "nw-mirror-" # all rows we own carry this TagId prefix
|
||||
DEF_COMPANY_JSON = os.environ.get(
|
||||
"OTOPCUA_COMPANY_JSON",
|
||||
os.path.join(os.path.dirname(__file__), "..", "company-uns.json"),
|
||||
)
|
||||
ID_PREFIX = "nw-mirror-" # SystemPlatform galaxy-mirror TagId prefix
|
||||
LOAD_PLAN = os.path.join(os.path.dirname(__file__), "load-plan.json")
|
||||
|
||||
# Equipment-overlay (company-shape) object ids — all carry the nw- prefix so
|
||||
# `clean` can remove them. The Equipment namespace is a SECOND namespace loaded
|
||||
# alongside the galaxy mirror. Each company signal is a VirtualTag (+ Script) whose
|
||||
# script mirrors the live SystemPlatform galaxy-mirror tag for that signal — so the
|
||||
# overlay carries live VALUES (scope doc WS-3), not just structure.
|
||||
EQ_CLUSTER = os.environ.get("OTOPCUA_EQ_CLUSTER", "MAIN")
|
||||
EQ_NS = "nw-uns"
|
||||
EQ_ID_PREFIX = "nweq-" # VirtualTag/Script logical-id prefix (cleanup by prefix scan)
|
||||
|
||||
# galaxy dataTypeName / gen_uns dtype → valid OtOpcUa DriverDataType
|
||||
_DTYPE_FIX = {"Double": "Float64", "Float": "Float32"}
|
||||
_ACCESS = {"ReadOnly": "0", "Read": "0", "ReadWrite": "1"}
|
||||
|
||||
# ── the value signals we mirror, per $TestMachine instance ──────────────────
|
||||
# (galaxy attribute name, OtOpcUa DriverDataType, access '0'=Read/'1'=ReadWrite)
|
||||
SIGNALS = [
|
||||
@@ -194,13 +222,123 @@ def cmd_populate(args):
|
||||
return 0
|
||||
|
||||
|
||||
def _eq_signal_ids(equipment_id, folder, name):
|
||||
"""Deterministic (VirtualTagId, ScriptId) for a company signal. Both carry the
|
||||
EQ_ID_PREFIX so `clean` removes exactly what was created. The two ids share the
|
||||
same per-signal hash but differ by a kind token so they never collide across the
|
||||
global UX_VirtualTag_LogicalId / UX_Script_LogicalId unique indexes. Capped at the
|
||||
64-char id column width."""
|
||||
base = hashlib.sha1(f"{equipment_id}|{folder}|{name}".encode()).hexdigest()[:20]
|
||||
return EQ_ID_PREFIX + "vt-" + base, EQ_ID_PREFIX + "sc-" + base
|
||||
|
||||
|
||||
def cmd_populate_equipment(args):
|
||||
"""Load the company-shape Equipment namespace from company-uns.json: a second
|
||||
(Equipment-kind) namespace alongside the galaxy mirror, with the Northwind
|
||||
Area/Line/Equipment/Signal tree. Each signal is a VirtualTag whose Script mirrors
|
||||
the live galaxy-mirror tag for that signal — `return ctx.GetTag("<ref>").Value;` —
|
||||
so the company shape streams live VALUES off the same Galaxy source (no driver,
|
||||
no BadWaitingForInitialData once the galaxy mirror is up). Idempotent:
|
||||
drop-and-recreate of the nw- overlay rows."""
|
||||
with open(args.company_json) as f:
|
||||
doc = json.load(f)
|
||||
u = doc["uns"]
|
||||
conn, cur = connect(args.mssql)
|
||||
|
||||
# Drop any prior overlay (child rows first), then recreate. VirtualTag/Script go
|
||||
# before Equipment (VirtualTag.EquipmentId logical-FKs Equipment). Equipment is
|
||||
# scoped by its overlay UnsLine ('nw-line-%') — NOT by the EquipmentId, which is now the
|
||||
# canonical 'EQ-'+uuid form (see DraftValidator) and no longer carries an 'nw-' prefix.
|
||||
cur.execute("DELETE FROM dbo.VirtualTag WHERE VirtualTagId LIKE %s", (EQ_ID_PREFIX + "%",))
|
||||
cur.execute("DELETE FROM dbo.Script WHERE ScriptId LIKE %s", (EQ_ID_PREFIX + "%",))
|
||||
cur.execute("DELETE FROM dbo.Equipment WHERE UnsLineId LIKE 'nw-line-%'")
|
||||
cur.execute("DELETE FROM dbo.UnsLine WHERE UnsLineId LIKE 'nw-line-%'")
|
||||
cur.execute("DELETE FROM dbo.UnsArea WHERE UnsAreaId LIKE 'nw-area-%'")
|
||||
# Equipment is now driver-less, but purge any driver still bound to the overlay namespace —
|
||||
# self-heals environments that ran an older loader which created the 'nw-uns-modbus' placeholder.
|
||||
cur.execute("DELETE FROM dbo.DriverInstance WHERE NamespaceId=%s", (EQ_NS,))
|
||||
cur.execute("DELETE FROM dbo.Namespace WHERE NamespaceId=%s", (EQ_NS,))
|
||||
|
||||
cur.execute(
|
||||
"INSERT INTO dbo.Namespace (NamespaceRowId, NamespaceId, ClusterId, Kind, NamespaceUri, Enabled) "
|
||||
"VALUES (NEWID(), %s, %s, 'Equipment', %s, 1)",
|
||||
(EQ_NS, EQ_CLUSTER, doc.get("namespace", {}).get("namespaceUri", "urn:northwind:birmingham:uns")))
|
||||
|
||||
for a in u["unsAreas"]:
|
||||
cur.execute("INSERT INTO dbo.UnsArea (UnsAreaRowId, UnsAreaId, ClusterId, Name) VALUES (NEWID(), %s, %s, %s)",
|
||||
("nw-" + a["unsAreaId"], EQ_CLUSTER, a["name"]))
|
||||
for l in u["unsLines"]:
|
||||
cur.execute("INSERT INTO dbo.UnsLine (UnsLineRowId, UnsLineId, UnsAreaId, Name) VALUES (NEWID(), %s, %s, %s)",
|
||||
("nw-" + l["unsLineId"], "nw-" + l["unsAreaId"], l["name"]))
|
||||
|
||||
eq_n = vt_n = 0
|
||||
for e in u["equipment"]:
|
||||
eq_uuid = uuid.uuid5(uuid.NAMESPACE_URL, "otopcua-nw-eq/" + e["equipmentId"])
|
||||
# Canonical EquipmentId: matches OtOpcUa DraftValidator.DeriveEquipmentId
|
||||
# ("EQ-" + EquipmentUuid.ToString("N")[..12].ToLowerInvariant()). uuid.UUID.hex is
|
||||
# already lowercase, 32 hex chars, no dashes — .hex[:12] is the first 12.
|
||||
eq_id = "EQ-" + eq_uuid.hex[:12].lower()
|
||||
eq_uuid = str(eq_uuid)
|
||||
cur.execute(
|
||||
"INSERT INTO dbo.Equipment (EquipmentRowId, EquipmentId, EquipmentUuid, UnsLineId, "
|
||||
"Name, MachineCode, Manufacturer, Model, Enabled) VALUES (NEWID(), %s, %s, %s, %s, %s, %s, %s, 1)",
|
||||
(eq_id, eq_uuid, "nw-" + e["unsLineId"], e["name"], e["machineCode"],
|
||||
e.get("manufacturer"), e.get("model")))
|
||||
eq_n += 1
|
||||
for t in e["tags"]:
|
||||
dtype = _DTYPE_FIX.get(t["dataType"], t["dataType"])
|
||||
folder = t.get("folderPath")
|
||||
# The galaxy-mirror MXAccess ref (e.g. TestMachine_001.TestDouble) is the upstream
|
||||
# the VirtualTag mirrors. DependencyExtractor harvests the literal in ctx.GetTag(),
|
||||
# so the engine subscribes to exactly this path on the galaxy-mirror driver.
|
||||
full = t["source"]["fullTagReference"]
|
||||
vt_id, sc_id = _eq_signal_ids(e["equipmentId"], folder, t["name"])
|
||||
source_code = f'return ctx.GetTag("{full}").Value;'
|
||||
source_hash = hashlib.sha256(source_code.encode()).hexdigest()
|
||||
cur.execute(
|
||||
"INSERT INTO dbo.Script (ScriptRowId, ScriptId, Name, SourceCode, SourceHash, Language) "
|
||||
"VALUES (NEWID(), %s, %s, %s, %s, 'CSharp')",
|
||||
(sc_id, t["name"], source_code, source_hash))
|
||||
cur.execute(
|
||||
"INSERT INTO dbo.VirtualTag (VirtualTagRowId, VirtualTagId, EquipmentId, Name, DataType, "
|
||||
"ScriptId, ChangeTriggered, TimerIntervalMs, Historize, Enabled) "
|
||||
"VALUES (NEWID(), %s, %s, %s, %s, %s, 1, NULL, %s, 1)",
|
||||
(vt_id, eq_id, t["name"], dtype, sc_id, 1 if t.get("historize") else 0))
|
||||
vt_n += 1
|
||||
|
||||
conn.commit()
|
||||
conn.close()
|
||||
print(f"populated equipment overlay: namespace {EQ_NS} ({EQ_CLUSTER}), "
|
||||
f"{len(u['unsAreas'])} areas, {len(u['unsLines'])} lines, {eq_n} equipment, "
|
||||
f"{vt_n} VirtualTags (+ {vt_n} mirror Scripts)")
|
||||
print()
|
||||
print(f">>> NEXT: deploy (headless) — curl -s -X POST {args.deploy_url.replace('/deployments','')}/api/deployments "
|
||||
f"-H 'X-Api-Key: {args.deploy_key}'")
|
||||
print(">>> then run: otopcua_uns.py verify-equipment --require-good 1036 --wait")
|
||||
return 0
|
||||
|
||||
|
||||
def cmd_clean(args):
|
||||
conn, cur = connect(args.mssql)
|
||||
cur.execute("DELETE FROM dbo.Tag WHERE TagId LIKE %s", (ID_PREFIX + "%",))
|
||||
n = cur.rowcount
|
||||
# Also drop the company-shape Equipment overlay (child rows first): VirtualTag and
|
||||
# Script (both nweq-*) before Equipment, then the rest.
|
||||
# Equipment is scoped by its overlay UnsLine ('nw-line-%') — NOT by the EquipmentId,
|
||||
# which is now the canonical 'EQ-'+uuid form (see DraftValidator) with no 'nw-' prefix.
|
||||
cur.execute("DELETE FROM dbo.VirtualTag WHERE VirtualTagId LIKE %s", (EQ_ID_PREFIX + "%",))
|
||||
cur.execute("DELETE FROM dbo.Script WHERE ScriptId LIKE %s", (EQ_ID_PREFIX + "%",))
|
||||
cur.execute("DELETE FROM dbo.Equipment WHERE UnsLineId LIKE 'nw-line-%'")
|
||||
cur.execute("DELETE FROM dbo.UnsLine WHERE UnsLineId LIKE 'nw-line-%'")
|
||||
cur.execute("DELETE FROM dbo.UnsArea WHERE UnsAreaId LIKE 'nw-area-%'")
|
||||
# Purge any driver still bound to the overlay namespace (e.g. the legacy 'nw-uns-modbus'
|
||||
# placeholder created by an older loader) so 'clean' fully removes the overlay.
|
||||
cur.execute("DELETE FROM dbo.DriverInstance WHERE NamespaceId=%s", (EQ_NS,))
|
||||
cur.execute("DELETE FROM dbo.Namespace WHERE NamespaceId=%s", (EQ_NS,))
|
||||
conn.commit()
|
||||
conn.close()
|
||||
print(f"removed {n} nw-* mirror tag(s). Deploy again at {args.deploy_url} to drop them from the address space.")
|
||||
print(f"removed {n} nw-* mirror tag(s) + the {EQ_NS} equipment overlay. "
|
||||
f"Deploy again at {args.deploy_url} to drop them from the address space.")
|
||||
return 0
|
||||
|
||||
|
||||
@@ -298,6 +436,119 @@ def sample_values(endpoint, n):
|
||||
return [("<browse error>", str(e), "?")]
|
||||
|
||||
|
||||
def browse_tree(endpoint, max_depth=8, top_prefix=None, read_values=False):
|
||||
"""Recursively descend the OtOpcUa address space and count leaf variables, returning
|
||||
(folder_count, leaf_count, leaf_paths, good_count). A node with no children is a leaf
|
||||
signal — this correctly handles the DEEP Equipment UNS tree
|
||||
(Area/Line/Equipment/[FolderPath]/Signal), unlike browse_summary which assumes the flat
|
||||
2-level Galaxy hierarchy. When top_prefix is set, only top-level OtOpcUa folders whose
|
||||
browse name starts with it are counted (e.g. 'nw-area-' scopes to the company Equipment
|
||||
overlay, excluding the Galaxy mirror folders). When read_values is True, each leaf's value
|
||||
is read and good_count tallies the Good-quality ones (else good_count is 0)."""
|
||||
import asyncio
|
||||
from asyncua import Client
|
||||
|
||||
async def maybe_good(node, acc):
|
||||
if not read_values:
|
||||
return
|
||||
try:
|
||||
dv = await node.read_data_value()
|
||||
if dv.StatusCode and dv.StatusCode.is_good():
|
||||
acc["good"] += 1
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
async def walk(node, path, depth, acc):
|
||||
if depth >= max_depth:
|
||||
return
|
||||
for ch in await node.get_children():
|
||||
try:
|
||||
name = (await ch.read_browse_name()).Name
|
||||
except Exception:
|
||||
continue
|
||||
child_path = path + "/" + name
|
||||
grandkids = await ch.get_children()
|
||||
if grandkids:
|
||||
acc["folders"] += 1
|
||||
await walk(ch, child_path, depth + 1, acc)
|
||||
else:
|
||||
acc["leaves"] += 1
|
||||
acc["paths"].append(child_path)
|
||||
await maybe_good(ch, acc)
|
||||
|
||||
async def run():
|
||||
acc = {"folders": 0, "leaves": 0, "paths": [], "good": 0}
|
||||
async with Client(endpoint) as c:
|
||||
for k in await c.nodes.objects.get_children():
|
||||
if (await k.read_browse_name()).Name != "OtOpcUa":
|
||||
continue
|
||||
for top in await k.get_children():
|
||||
tn = (await top.read_browse_name()).Name
|
||||
if top_prefix and not tn.startswith(top_prefix):
|
||||
continue
|
||||
if await top.get_children():
|
||||
acc["folders"] += 1
|
||||
await walk(top, "OtOpcUa/" + tn, 1, acc)
|
||||
else:
|
||||
acc["leaves"] += 1
|
||||
acc["paths"].append("OtOpcUa/" + tn)
|
||||
await maybe_good(top, acc)
|
||||
return acc["folders"], acc["leaves"], acc["paths"], acc["good"]
|
||||
try:
|
||||
return asyncio.run(run())
|
||||
except Exception as e:
|
||||
return (f"<{type(e).__name__}: {e}>", 0, [], 0)
|
||||
|
||||
|
||||
def cmd_verify_equipment(args):
|
||||
"""Browse the full UNS tree by friendly Area/Line/Equipment/Signal names and report the leaf
|
||||
signal count. With --expect N, exit non-zero unless exactly N leaf signals are present (the
|
||||
equipment-namespace structure-materialisation check). With --require-good N (>0), also read
|
||||
each leaf's value and require at least N Good ones (the live-VALUE check for the VirtualTag
|
||||
overlay) — back-compat default 0 = structure-only. --wait polls so it can wait for the deploy
|
||||
+ change-triggered VirtualTag evaluations to land."""
|
||||
top_prefix = None if args.all else "nw-area-"
|
||||
scope = "whole address space" if args.all else "company overlay (nw-area-*)"
|
||||
read_values = args.require_good > 0
|
||||
deadline = time.time() + (args.wait_seconds if args.wait else 0)
|
||||
while True:
|
||||
folders, leaves, paths, good = browse_tree(
|
||||
args.opcua_endpoint, top_prefix=top_prefix, read_values=read_values)
|
||||
struct_ok = args.expect is None or leaves == args.expect
|
||||
good_ok = good >= args.require_good
|
||||
if (struct_ok and good_ok) or time.time() >= deadline:
|
||||
break
|
||||
print(f" waiting for deploy/values… ({leaves} leaves"
|
||||
+ (f", {good} Good" if read_values else "") + ")")
|
||||
time.sleep(5)
|
||||
|
||||
suffix = f", {good} Good value(s)" if read_values else ""
|
||||
print(f"equipment tree : {folders} folder(s), {leaves} leaf signal(s){suffix} "
|
||||
f"on {args.opcua_endpoint} [{scope}]")
|
||||
for p in sorted(paths)[:args.show]:
|
||||
print(f" {p}")
|
||||
if len(paths) > args.show:
|
||||
print(f" … and {len(paths) - args.show} more")
|
||||
|
||||
passed = True
|
||||
if args.expect is not None:
|
||||
struct_ok = leaves == args.expect
|
||||
passed = passed and struct_ok
|
||||
print(" structure :",
|
||||
f"PASS ({leaves} == {args.expect})" if struct_ok
|
||||
else f"FAIL (expected {args.expect}, found {leaves})")
|
||||
if args.require_good > 0:
|
||||
good_ok = good >= args.require_good
|
||||
passed = passed and good_ok
|
||||
print(" live good :",
|
||||
f"PASS ({good} >= {args.require_good})" if good_ok
|
||||
else f"FAIL (expected >= {args.require_good} Good, found {good})")
|
||||
if args.expect is None and args.require_good == 0:
|
||||
return 0
|
||||
print("VERIFY-EQUIPMENT:", "PASS" if passed else "FAIL")
|
||||
return 0 if passed else 1
|
||||
|
||||
|
||||
# ── arg parsing ─────────────────────────────────────────────────────────────
|
||||
def main(argv):
|
||||
p = argparse.ArgumentParser(description="Reloadable populate + verify for the OtOpcUa galaxy UNS.")
|
||||
@@ -305,6 +556,9 @@ def main(argv):
|
||||
p.add_argument("--driver", default=DEF_DRIVER, help="SystemPlatform GalaxyMxGateway driver instance id")
|
||||
p.add_argument("--opcua-endpoint", default=DEF_OPCUA)
|
||||
p.add_argument("--deploy-url", default="http://localhost:9200/deployments")
|
||||
p.add_argument("--deploy-key", default=os.environ.get("OTOPCUA_DEPLOY_KEY", "docker-dev-deploy-key"),
|
||||
help="X-Api-Key for the headless POST /api/deployments endpoint")
|
||||
p.add_argument("--company-json", default=DEF_COMPANY_JSON)
|
||||
p.add_argument("--sql-host", default=DEF_MSSQL["host"])
|
||||
p.add_argument("--sql-port", type=int, default=DEF_MSSQL["port"])
|
||||
p.add_argument("--sql-user", default=DEF_MSSQL["user"])
|
||||
@@ -313,18 +567,33 @@ def main(argv):
|
||||
sub = p.add_subparsers(dest="cmd", required=True)
|
||||
sub.add_parser("generate")
|
||||
sub.add_parser("populate")
|
||||
sub.add_parser("populate-equipment",
|
||||
help="load the company-shape Equipment namespace from company-uns.json (structure-only)")
|
||||
sub.add_parser("clean")
|
||||
sub.add_parser("status")
|
||||
vp = sub.add_parser("verify")
|
||||
vp.add_argument("--wait", action="store_true", help="poll until the deploy lands")
|
||||
vp.add_argument("--wait-seconds", type=int, default=120)
|
||||
ep = sub.add_parser("verify-equipment",
|
||||
help="recursively browse the Equipment UNS tree + count leaf signals "
|
||||
"(+ optionally assert live Good values)")
|
||||
ep.add_argument("--expect", type=int, default=None, help="assert exactly N leaf signals")
|
||||
ep.add_argument("--require-good", type=int, default=0,
|
||||
help="read each leaf's value and require >= N Good ones (0 = structure-only, default)")
|
||||
ep.add_argument("--show", type=int, default=20, help="how many leaf paths to print")
|
||||
ep.add_argument("--all", action="store_true",
|
||||
help="count the whole address space (default: only the nw-area-* company overlay)")
|
||||
ep.add_argument("--wait", action="store_true",
|
||||
help="poll until the deploy lands + (with --require-good) values go Good")
|
||||
ep.add_argument("--wait-seconds", type=int, default=120)
|
||||
|
||||
a = p.parse_args(argv)
|
||||
a.mssql = dict(host=a.sql_host, port=a.sql_port, user=a.sql_user,
|
||||
password=a.sql_password, database=a.sql_db)
|
||||
return {
|
||||
"generate": cmd_generate, "populate": cmd_populate, "clean": cmd_clean,
|
||||
"status": cmd_status, "verify": cmd_verify,
|
||||
"generate": cmd_generate, "populate": cmd_populate,
|
||||
"populate-equipment": cmd_populate_equipment, "clean": cmd_clean,
|
||||
"status": cmd_status, "verify": cmd_verify, "verify-equipment": cmd_verify_equipment,
|
||||
}[a.cmd](a)
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user