scadaproj/docs/plans/2026-06-23-historian-gateway-design.md

# ZB.MOM.WW.HistorianGateway — Design

**Date:** 2026-06-23
**Status:** Design approved (brainstorming complete) — ready for implementation planning
**Author:** brainstorming session (Joseph Doherty + Claude)

## 1. Summary

A new **full-feature sidecar** in the SCADA/OT sister-project family, modelled on
`MxAccessGateway` (`mxaccessgw`). It does two things:

1. **Read-only Galaxy metadata server** — exposes the AVEVA System Platform
   ("Wonderware") **Galaxy object hierarchy** (areas / objects / templates /
   instances / attributes), sourced from the **Galaxy Repository SQL DB**, the same
   data `mxaccessgw`'s `galaxy_repository` feature serves today.
2. **Full read/write gRPC API to the AVEVA (Wonderware) Historian** — reads (raw,
   aggregate with all 15 retrieval modes, at-time, blocks, events, browse, metadata,
   status) **and writes** (historical/backfill values, event send, tag-config
   create/delete/rename/extended-properties, plus resilience helpers).

It reuses the family's **common shared packages and styles**: `ZB.MOM.WW.Auth`,
`ZB.MOM.WW.Theme`, `ZB.MOM.WW.Telemetry`(+`.Serilog`), `ZB.MOM.WW.Health`,
`ZB.MOM.WW.Configuration`, `ZB.MOM.WW.Audit`.

### Key reframing discovered during brainstorming

- The historian write surface lives in the **`histsdk`** repo
  (`gitea.dohertylan.com/dohertj2/histsdk`, namespace `AVEVA.Historian.Client`),
  which is **far ahead** of the stale `scadaproj/ZB.MOM.WW.SPHistorianClient` port
  (2026-06-19 snapshot, reads + tag create/delete only, value-writes marked
  "blocked"). `histsdk` has since added a **live-validated gRPC write surface**:
  `AddHistoricalValuesAsync` (historical/backfill, gRPC-only), `SendEventAsync`
  (events, both transports), `EnsureTags`/`DeleteTags`/`RenameTags`/
  `AddTagExtendedProperties` (config writes, gRPC), plus higher-level
  `HistorianStoreForwardWriter` (durable outbox) and a redundant-write cluster.
- **Hard server-side limit no client can lift:** `AddS2` *streaming live
  process-sample* writes are GATED — the historian runtime cache only ingests from
  configured IOServer / Application Server pipelines. "Live current value" writes are
  therefore done via the **SQL path** (`aaAnalogTagInsert` → `INSERT INTO History`),
  not gRPC.
- **No COM anywhere.** The historian SDK is pure-managed and Galaxy browse is plain
  SQL, so — unlike `mxaccessgw` — this sidecar needs **no x86 worker and no
  two-process split**. It is a single .NET 10 x64 process.

## 2. Decisions (locked during brainstorming)

| Decision | Choice |
|---|---|
| Purpose | **General-purpose gateway** — reusable modern façade over Historian + Galaxy metadata for any gRPC client (the way `mxaccessgw` is the façade for MXAccess). |
| Historian client code | **Vendor `histsdk`** (`AVEVA.Historian.Client`) into the sidecar repo as self-contained vendored source; namespace kept as-is to ease future re-sync. |
| Galaxy browse code | **New shared lib `ZB.MOM.WW.GalaxyRepository`** in scadaproj, extracted from `mxaccessgw`'s `galaxy_repository` browse; consumed by both `mxaccessgw` and this sidecar. |
| Dashboard | **Full Blazor dashboard** on `ZB.MOM.WW.Theme` (login + Galaxy browser + Historian console + API-key admin + status/health). |
| Historian write scope | **All:** historical/backfill writes, event send, tag-config writes, **and** resilience extras (store-and-forward outbox, redundant write fan-out, SQL live-value path). |
| Connection model | **Approach A — stateless gateway over pooled service-identity connections.** Clients authenticate to the gateway (ZB API key); the gateway owns historian credentials and reuses pooled, pre-authenticated connections. |
| Repo location | **New standalone sibling repo** `~/Desktop/HistorianGateway`, gitea remote `historiangw`. |
| Name / namespace | **`ZB.MOM.WW.HistorianGateway`**. |

## 3. Architecture & solution structure

Single .NET 10 x64 ASP.NET Core process.

```
~/Desktop/HistorianGateway/
  src/
    ZB.MOM.WW.HistorianGateway.Server/      ASP.NET Core host: gRPC services + Blazor dashboard + /healthz + /metrics
    ZB.MOM.WW.HistorianGateway.Contracts/   the gateway's own .proto + generated types (for client codegen distribution)
    vendor/AVEVA.Historian.Client/          VENDORED from histsdk; ArchestrA.Grpc.Contract.* protos + reads/writes/store-forward/redundancy
  tests/
    ZB.MOM.WW.HistorianGateway.Tests/       unit + env-gated live integration + bUnit dashboard
  docs/plans/, CLAUDE.md, README.md
  ZB.MOM.WW.HistorianGateway.slnx
```

**Cross-repo pieces:**

- **`scadaproj/ZB.MOM.WW.GalaxyRepository`** (new shared lib, plain files — NOT a
  nested git repo): carries the **canonical `galaxy_repository.proto`** (adopted from
  `mxaccessgw`'s existing contract so OtOpcUa's wire shape is not broken), the SQL
  browse provider (connect to Galaxy Repository SQL → hierarchy model), and a
  reusable gRPC service implementation both hosts can `MapGrpcService<>()`.
  `mxaccessgw` adopting it is a **tracked follow-on** (same "built → adopted" pattern
  as the other normalized components); this sidecar consumes it from the start.
- **Shared ZB packages consumed:** `ZB.MOM.WW.Auth`
  (Abstractions+Ldap+ApiKeys+AspNetCore), `ZB.MOM.WW.Theme`, `ZB.MOM.WW.Telemetry`
  (+`.Serilog`), `ZB.MOM.WW.Health`, `ZB.MOM.WW.Configuration`, `ZB.MOM.WW.Audit`.

## 4. gRPC API surface

Gateway's own curated contract (`ZB.MOM.WW.HistorianGateway.Grpc.V1`), grouped by
concern — not a 1:1 SDK dump. The vendored `ArchestrA.Grpc.Contract.*` protos stay
internal; clients see only the gateway contract.

| Service | RPCs | Notes |
|---|---|---|
| `HistorianRead` | `ReadRaw`, `ReadAggregate`, `ReadBlocks`, `ReadEvents` *(server-streaming)*; `ReadAtTime` | `ReadAggregate` exposes all 15 retrieval modes |
| `HistorianWrite` | `AddHistoricalValues`, `SendEvent`, `WriteLiveValues` | `WriteLiveValues` = SQL path (gRPC streaming is gated) |
| `HistorianTags` | `BrowseTagNames` *(streaming)*, `GetTagMetadata`, `EnsureTags`, `DeleteTags`, `RenameTags`, `AddTagExtendedProperties` | |
| `HistorianStatus` | `Probe`, `GetConnectionStatus`, `GetStoreForwardStatus`, `GetSystemParameter` | |
| `GalaxyRepository` | Browse areas / objects / templates / instances / attributes *(read-only)* | canonical proto from the shared lib |

**Authorization** is via **API-key scopes at the gateway** (Approach A trust
boundary): `historian:read`, `historian:write`, `historian:tags:write`,
`galaxy:read`.

## 5. Connection & data flow

```
gRPC client ──(ZB API key)──► HistorianGateway ──┬─ pooled, pre-authed gRPC conn ──► AVEVA Historian (RemoteGrpc 2023R2)
                                                  ├─ store-forward outbox (SQLite) ─ replays writes on reconnect
                                                  ├─ redundant-write fan-out ──────► historian members (All/Any ack)
                                                  ├─ SqlConnection ──► Runtime DB (live-value writes via aaAnalogTagInsert/History)
                                                  └─ ZB.MOM.WW.GalaxyRepository ──► Galaxy Repository SQL (read-only browse)
```

- **Pooled connections:** the expensive auth handshake (`ValidateClientCredential` /
  ECDH `ExchangeKey`) runs **once per connection on open**, then is reused across
  requests; connections are health-checked with auto-reconnect. Write operations use
  the write-enabled session mode (`0x401`).
- **Store-forward:** writes flow through the SDK's `HistorianStoreForwardWriter` — on
  an unreachable historian, enqueue to durable SQLite; a background drain replays on
  reconnect.
- **Redundancy:** `HistorianRedundantWriteResult` fan-out to configured members under
  an All/Any ack policy; per-member result surfaced to the caller.
- **SQL live-write** and **Galaxy browse** are independent SQL paths, each with its
  own validated connection config.

**Configuration** (all `ZB.MOM.WW.Configuration`-validated, aggregated by
`ConfigPreflight` at startup): historian (host, gRPC port 32565, transport=RemoteGrpc,
TLS, service identity/credentials), redundant members, store-forward path, Galaxy
Repository SQL connection string, Runtime DB connection string (SQL live-write),
Auth (LDAP + API-key pepper). Secrets live in the operator environment, never in repo.

## 6. Cross-cutting infrastructure + dashboard

- **Auth (`ZB.MOM.WW.Auth`):** gRPC clients use peppered-HMAC API keys (keyId/Bearer),
  validated by a gRPC interceptor enforcing per-service scopes. Dashboard uses LDAP
  login (`.Ldap`+`.AspNetCore`), cookie auth, `IGroupRoleMapper<TRole>`, canonical
  `ZbClaimTypes`/`ZbCookieDefaults`, canonical-six roles, dev against the shared
  GLAuth (`10.100.0.35:3893`, `dc=zb,dc=local`). `DisableLogin` dev/deploy switch.
- **Telemetry (`.Telemetry`+`.Serilog`):** `AddZbTelemetry` (Resource
  `service.name=historian-gateway` + standard instrumentation + always-on Prometheus
  `/metrics`, OTLP opt-in) + `AddZbSerilog`. App Meters: read/write counts + latency,
  store-forward queue depth, pool connection state, redundancy ack outcomes.
- **Health (`.Health`):** three-tier ready/active/healthz + canonical JSON writer.
  Probes: historian gRPC (`GrpcDependencyHealthCheck`), Galaxy Repository SQL +
  Runtime DB (`DatabaseHealthCheck`), store-forward drain status.
- **Configuration (`.Configuration`):** `OptionsValidatorBase` / `ValidationBuilder` /
  `AddValidatedOptions` / `ConfigPreflight` (§5).
- **Audit (`.Audit`, DEEP-adopt):** canonical `AuditEvent` + SQLite `IAuditWriter`
  (MxGateway-style). Audited: tag-config writes, historical/event writes, API-key
  admin, login/logout. `Actor` wired from the Auth principal via `IAuditActorAccessor`.
- **Dashboard (Blazor, `.Theme`):** Technical-Light side-rail shell + `LoginCard`
  `/login`. Pages: **Status** (pool / store-forward / redundancy / version),
  **Galaxy browser** (read-only hierarchy tree), **Historian console** (query with
  raw/aggregate + mode picker + time range; role-gated write test for value insert /
  event send), **API-key admin** (list/create/revoke keys + scopes), **Health**.

## 7. Error handling

- **gRPC status mapping:** `ProtocolEvidenceMissingException` (unsupported op/type —
  e.g. non-analog tag, non-string event property) → `Unimplemented`/`FailedPrecondition`
  with a clear "not in reverse-engineered surface" message; auth →
  `Unauthenticated`/`PermissionDenied`; historian down → `Unavailable`; bad range /
  unknown tag → `InvalidArgument`/`NotFound`.
- **Gated ops:** live streaming-sample writes (`AddS2`) are **not exposed** (no RPC);
  live-value writes route through SQL `WriteLiveValues`.
- **Write resilience:** with store-forward enabled, an unreachable historian returns
  *accepted + queued* (not an error); otherwise `Unavailable`. Redundancy surfaces a
  per-member result; All-policy fails if any member fails, Any-policy succeeds on ≥1 ack.
- **Pool:** transient failures → reconnect + bounded retry; auth-handshake failure →
  fail fast with diagnostic. No secrets/real hostnames in errors or logs (histsdk
  safety rule).

## 8. Testing

- **Unit:** gRPC services against a faked historian-client seam + faked Galaxy
  provider; scope/auth interceptor; config validators; SDK-model ↔ proto mapping.
- **Golden/protocol:** carry over `histsdk`'s golden byte tests for the vendored
  client (historical "ON" buffer, event "OS" buffer, registration buffers) so the
  vendored copy stays faithful.
- **Integration (env-gated, live, CI/macOS-safe):** real 2023 R2 historian + Galaxy
  Repository SQL — read/write round-trips and browse via the self-cleaning
  sandbox-tag lifecycle (`HISTORIAN_GRPC_WRITE_SANDBOX_TAG`); skipped when env vars
  absent.
- **Dashboard:** bUnit component tests. **Smoke:** `/healthz`, `/metrics`, gRPC
  `Probe`.

## 9. Out of scope / non-goals

- `AddS2` live streaming process-sample writes (GATED server-side; SQL path covers
  live values instead).
- Non-analog tag creation, revision/edit writes, bit-faithful store-forward framing
  (per `histsdk` capability matrix — `BOUNDED`/`HARD`/`GATED` items not selected).
- A two-process / x86 worker split (not needed — no COM).
- Re-syncing or replacing the existing stale `scadaproj/ZB.MOM.WW.SPHistorianClient`
  port (we vendor `histsdk` instead; the stale port is left as-is).

## 10. Implementation components (high level)

1. **`ZB.MOM.WW.GalaxyRepository` shared lib** (scadaproj) — extract from
   `mxaccessgw`, canonical proto + SQL browse provider + reusable gRPC service.
2. **Vendor `histsdk`** `AVEVA.Historian.Client` into the new repo + carry its golden
   tests.
3. **Repo scaffold + host + shared-package wiring** (Auth/Telemetry/Health/
   Configuration/Audit) + validated options + `ConfigPreflight`.
4. **gRPC contract + services** (Read / Write / Tags / Status / GalaxyRepository).
5. **Connection layer** — pooled pre-authed connections, store-forward, redundancy,
   SQL live-write path.
6. **Auth** — API-key scope interceptor + LDAP dashboard auth + Audit wiring.
7. **Blazor dashboard** pages (Theme).
8. **Telemetry + Health** probes/meters.
9. **Tests** — unit / golden / env-gated integration / bUnit.
10. **Docs + repo/gitea setup** — `CLAUDE.md`, `README.md`, gitea remote.

> `mxaccessgw` adoption of `ZB.MOM.WW.GalaxyRepository` is a separate tracked
> follow-on, not part of the initial sidecar delivery.