mxaccessgw/docs/GalaxyRepository.md

# Galaxy Repository Browse

The gateway exposes a read-only browse surface over the AVEVA System Platform
Galaxy Repository (the SQL Server database named `ZB`). Clients use it to
enumerate the deployed object hierarchy and each object's attributes
before subscribing to runtime values via the existing `MxAccessGateway` RPCs.

This is a metadata layer: it never reads or writes runtime tag values, never
goes through MXAccess COM, and never needs the x86 worker. It runs entirely
inside the .NET 10 gateway process and talks to the Galaxy Repository over
plain SQL.

## Why It Exists

Without browse, a client must already know `tag_name.AttributeName` strings to
issue `AddItem`. The Galaxy Repository is the authoritative source for those
names — the same database that System Platform itself reads when the
ArchestrA IDE renders the deployment tree. Surfacing that data over gRPC lets
remote clients build a navigable address space without any coupling to the
COM layer or the host platform.

`HierarchySql` is the object-hierarchy query originally ported from the
equivalent OPC UA server in the OtOpcUa project. `AttributesSql` has since
diverged from OtOpcUa — see [Built-in vs configured attributes](#built-in-vs-configured-attributes)
— and is no longer kept in sync with it.

## RPC Surface

The service is defined in
`src/MxGateway.Contracts/Protos/galaxy_repository.proto` under package
`galaxy_repository.v1`.

| RPC | Purpose |
|-----|---------|
| `TestConnection` | Connectivity probe. Returns `{ ok: bool }` after a `SELECT 1`. Does not throw on SQL failure — returns `ok = false`. Always hits SQL directly so it remains a true health check. |
| `GetLastDeployTime` | Returns the cached `galaxy.time_of_last_deploy`. Served from the shared hierarchy cache; refreshed in the background. |
| `DiscoverHierarchy` | Returns one page of the deployed hierarchy plus each returned object's attributes (configured and built-in — see [Built-in vs configured attributes](#built-in-vs-configured-attributes)). **Served from cache** — see [Hierarchy Cache](#hierarchy-cache). |
| `WatchDeployEvents` | **Server-streaming.** The server emits the current state immediately on subscribe (so clients can bootstrap without waiting), then emits one event per detected deploy change. See [Deploy Notifications](#deploy-notifications). |

`DiscoverHierarchy` is a paged unary RPC. The raw request accepts `page_size`
and `page_token`; the server defaults omitted page size to 1000 objects and
caps every page at 5000 objects. Page tokens bind to the cache sequence and the
active filter set, so changing filters between pages returns `InvalidArgument`
instead of mixing snapshots. Official high-level clients preserve the older
"return the full hierarchy" behavior by looping pages internally.

The request can also slice the cached hierarchy without running new SQL. A
caller may choose one root (`root_gobject_id`, `root_tag_name`, or
`root_contained_path`) and may combine that with `max_depth`, category ids,
template-chain substring filters, an anchored case-insensitive tag-name glob,
alarm-only, historized-only, and `include_attributes = false` for a skeleton
tree. All filters are applied with AND semantics, and `total_object_count`
reports the post-filter count.

## Hierarchy Cache

The gateway holds a single shared `IGalaxyHierarchyCache`
(`src/MxGateway.Server/Galaxy/GalaxyHierarchyCache.cs`) — every
`DiscoverHierarchy` and `GetLastDeployTime` request reads from this cache
rather than hitting SQL. Many clients can browse concurrently with at most
one SQL query in flight.

Refresh strategy is **deploy-time gated**:

1. The hosted `GalaxyHierarchyRefreshService` ticks every
   `MxGateway:Galaxy:DashboardRefreshIntervalSeconds` seconds (default 30).
2. Each tick queries the cheap `SELECT time_of_last_deploy FROM galaxy` first.
3. If the deploy timestamp is unchanged, the heavy hierarchy + attributes
   queries are **skipped**. The cache simply marks `LastSuccessAt`.
4. If the deploy timestamp changed (or no data has loaded yet), the cache
   pulls hierarchy + attributes, materializes a Galaxy object list plus a
   dashboard summary once, replaces the entry atomically, and publishes a
   deploy event.

Materializing objects and dashboard summaries at refresh time means subsequent
`DiscoverHierarchy` calls page over an immutable object list. The dashboard
uses the precomputed summary and does not rescan raw SQL rowsets on each
snapshot.

When SQL is unreachable, the cache retains the previous data and flips
`Status` to `Stale` (or `Unavailable` if no data was ever loaded). A
`SqlException` never bubbles out as the client-facing error.

### First-load behavior

If a client calls `DiscoverHierarchy` before the background service has
populated the cache, the gRPC handler waits up to 5 seconds for the first
load to complete before returning. If the first load fails or times out,
the client gets `Unavailable` with a short reason. Once any load completes
(success or failure), this wait is skipped on subsequent calls.

## Deploy Notifications

`WatchDeployEvents` is a server-streaming RPC backed by
`IGalaxyDeployNotifier` (`src/MxGateway.Server/Galaxy/GalaxyDeployNotifier.cs`).
The notifier maintains a private bounded channel per subscriber so a slow
client cannot back-pressure other subscribers or the publisher.

Subscriber lifecycle:

1. On subscribe, the notifier emits the **current** event (the state of the
   most recent successful refresh) so the subscriber can sync its local cache
   without waiting for the next deploy. Clients that already know about that
   deploy can pass `last_seen_deploy_time` in the request to suppress the
   bootstrap event.
2. As the cache observes new deploy timestamps, it publishes one event per
   change. Each event carries:
   - `sequence` — monotonic per server start; gaps signal a dropped event.
   - `observed_at` — server wall-clock when the cache saw the deploy.
   - `time_of_last_deploy` (+ `time_of_last_deploy_present`) — the Galaxy
     timestamp; absent only when the source row reports null.
   - `object_count`, `attribute_count` — counts on the new deploy, useful for
     dashboards and "did anything important change" gates without re-pulling.
3. If the subscriber's per-subscriber buffer fills (bound = 16 events with
   `DropOldest`), older events are dropped. Clients use the `sequence` field
   to detect this.
4. Cancellation (or transport disconnect) removes the subscriber.

Typical client pattern:

```text
1. Open WatchDeployEvents stream (with last_seen_deploy_time if you have one).
2. On each event, decide whether to call DiscoverHierarchy to refresh local cache.
3. If sequence skipped a number, treat it as a dropped event and refresh.
```

### Reply shape

```proto
message GalaxyObject {
  int32 gobject_id = 1;
  string tag_name = 2;
  string contained_name = 3;
  string browse_name = 4;        // contained_name when present, else tag_name
  int32 parent_gobject_id = 5;
  bool is_area = 6;
  int32 category_id = 7;
  int32 hosted_by_gobject_id = 8;
  repeated string template_chain = 9;
  repeated GalaxyAttribute attributes = 10;
}

message GalaxyAttribute {
  string attribute_name = 1;
  string full_tag_reference = 2;   // e.g. "DelmiaReceiver_001.DownloadPath"
  int32 mx_data_type = 3;          // raw Galaxy mx_data_type integer
  string data_type_name = 4;
  bool is_array = 5;
  int32 array_dimension = 6;
  bool array_dimension_present = 7; // distinguishes "no dimension" from 0
  int32 mx_attribute_category = 8;
  int32 security_classification = 9;
  bool is_historized = 10;
  bool is_alarm = 11;
}

message DiscoverHierarchyRequest {
  int32 page_size = 1;    // omitted/0 uses the server default of 1000
  string page_token = 2;  // opaque token returned by the previous page
  oneof root {
    int32 root_gobject_id = 3;
    string root_tag_name = 4;
    string root_contained_path = 5;
  }
  google.protobuf.Int32Value max_depth = 6;
  repeated int32 category_ids = 7;
  repeated string template_chain_contains = 8;
  string tag_name_glob = 9;
  optional bool include_attributes = 10;
  bool alarm_bearing_only = 11;
  bool historized_only = 12;
}

message DiscoverHierarchyReply {
  repeated GalaxyObject objects = 1;
  string next_page_token = 2;
  int32 total_object_count = 3;
}
```

### Built-in vs configured attributes

Each `GalaxyObject` carries two kinds of attribute, both surfaced the same way
in the `attributes` list:

- **Configured (dynamic) attributes** — attributes added in the ArchestrA IDE
  attribute editor. Stored in the Galaxy `dynamic_attribute` table.
- **Built-in attributes** — attributes every object inherits from its
  primitives: the object framework, the engine/platform primitives, and the
  per-attribute extensions (Alarm, History, Boolean, …). Stored in
  `attribute_definition` and reached through `primitive_instance`.

Built-in attributes are why an `AppEngine` or `WinPlatform` object reports its
`Engine.*` and `Alarm*` attributes, and why an alarmed attribute such as
`TestAlarm001` reports its extension leaves `TestAlarm001.Acked`,
`TestAlarm001.AckMsg`, `TestAlarm001.ActiveAlarmState`, and so on. An earlier
version of the browse query returned only configured attributes, so those
objects came back empty or partial; including built-ins makes the browse
surface match what System Platform's own Object Viewer shows. Expect roughly
seven times as many attributes as configured-only — the dashboard attribute
count reflects this.

Two rules govern the built-in rows:

- **No category filter.** `attribute_definition` uses a different
  `mx_attribute_category` numbering than `dynamic_attribute`, so only the
  `_`-prefixed-name and `.Description` exclusions apply to built-ins. (The
  configured-attribute category allow-list is unchanged.)
- **`is_historized` / `is_alarm` are always `false` for built-in rows.** Those
  flags identify a configured attribute that *anchors* a history or alarm
  extension (e.g. `TestAlarm001`), not the extension's machinery leaves
  (`TestAlarm001.Acked`). `alarm_bearing_only` and `historized_only` therefore
  still select the anchor attributes, not their built-in children.

When a configured attribute and a built-in attribute resolve to the same
reference, the configured attribute wins.

### Contained name vs tag name

Galaxy objects carry two names. `tag_name` is globally unique and is what
MXAccess expects in `AddItem`. `contained_name` is the human-readable name
used in the IDE browse tree, scoped to the parent. The browse RPC exposes
both: clients display `browse_name` to users and pass `tag_name` (or
`full_tag_reference`) into MXAccess subscriptions. When `contained_name` is
empty (top-level objects), `browse_name` falls back to `tag_name`.

### Data types

`mx_data_type` is returned as the raw Galaxy integer rather than mapped to a
language-neutral enum. The gateway makes no assumption about the client's
target type system — clients map to OPC UA, JSON, .NET CLR types, or
something else as appropriate. The Galaxy `data_type` table description is
also passed through as `data_type_name`.

`array_dimension_present` is a separate boolean because protobuf scalar
fields cannot express null. Use it to distinguish "no dimension reported" from
"dimension is zero."

## Architecture

```text
gRPC client(s)
  -> GalaxyRepositoryGrpcService (src/MxGateway.Server/Grpc/)
    DiscoverHierarchy, GetLastDeployTime  -> IGalaxyHierarchyCache.Current
    WatchDeployEvents                     -> IGalaxyDeployNotifier
    TestConnection                        -> GalaxyRepository  (direct SQL)

GalaxyHierarchyRefreshService (BackgroundService)
  -> IGalaxyHierarchyCache.RefreshAsync
    -> GalaxyRepository.GetLastDeployTimeAsync   (cheap, every tick)
    -> GalaxyRepository.GetHierarchyAsync         (only on deploy change)
    -> GalaxyRepository.GetAttributesAsync        (only on deploy change)
    -> GalaxyProtoMapper.MapObject (materialize GalaxyObject list once)
    -> DashboardGalaxySummary (precompute dashboard counts once)
    -> IGalaxyDeployNotifier.Publish (only on deploy change)
```

Component breakdown:

- `GalaxyRepository` (`src/MxGateway.Server/Galaxy/GalaxyRepository.cs`) holds
  the SQL. Both `HierarchySql` and `AttributesSql` walk template-derivation and
  package-derivation chains via recursive CTEs and pick the most-derived
  override per object. `HierarchySql` still matches the OtOpcUa original;
  `AttributesSql` does not — it additionally enumerates built-in primitive
  attributes (see [Built-in vs configured attributes](#built-in-vs-configured-attributes)).
- `GalaxyHierarchyCache`
  (`src/MxGateway.Server/Galaxy/GalaxyHierarchyCache.cs`) holds the most
  recent immutable `GalaxyHierarchyCacheEntry` (materialized objects +
  precomputed dashboard summary + counts + status). All gRPC clients share the
  same entry.
- `GalaxyHierarchyRefreshService`
  (`src/MxGateway.Server/Galaxy/GalaxyHierarchyRefreshService.cs`) is a
  hosted `BackgroundService` that drives `RefreshAsync` on the configured
  interval, with deploy-time gating to avoid unnecessary heavy queries.
- `GalaxyDeployNotifier`
  (`src/MxGateway.Server/Galaxy/GalaxyDeployNotifier.cs`) is a thin
  per-subscriber-channel fan-out for streaming clients.
- `GalaxyProtoMapper`
  (`src/MxGateway.Server/Grpc/GalaxyProtoMapper.cs`) converts row models to
  proto messages. Used by the cache during refresh to materialize the reply
  once.
- `GalaxyRepositoryGrpcService`
  (`src/MxGateway.Server/Grpc/GalaxyRepositoryGrpcService.cs`) implements
  the four RPCs.

## Configuration

Bound to `MxGateway:Galaxy` via `GalaxyRepositoryOptions`.

| Option | Default | Description |
|--------|---------|-------------|
| `MxGateway:Galaxy:ConnectionString` | `Server=localhost;Database=ZB;Integrated Security=True;TrustServerCertificate=True;Encrypt=False;` | SQL Server connection string for the Galaxy Repository. Integrated Security against `localhost` is the dev default; production deployments should override this through the standard double-underscore environment variable form, e.g. `MxGateway__Galaxy__ConnectionString`. |
| `MxGateway:Galaxy:CommandTimeoutSeconds` | `60` | Per-command SQL timeout. Applies to all three RPCs. |

The connection string is not treated as a secret in dev (`Integrated
Security`), but production deployments that use SQL authentication should set
the override via environment variable rather than committing credentials to
`appsettings.json`.

The dashboard parses this connection string and displays only non-secret
fields: server, database, integrated security, encrypt, and trust-server-
certificate. It never displays user id, password, access token, or arbitrary
unparsed connection string text.

## Authorization

All four Galaxy RPCs (including `WatchDeployEvents`) require the
`metadata:read` API-key scope. Browse is read-only metadata, equivalent in
privilege to `MxCommandKind.GetSessionState` or `MxCommandKind.GetWorkerInfo`.
The mapping lives in `GatewayGrpcScopeResolver`; see
[Authorization](./Authorization.md) for the full scope catalog.

API keys can also carry `browse_subtrees` constraints. `DiscoverHierarchy`
intersects those contained-path globs with the caller's request filters.
`WatchDeployEvents` still emits deploy notifications, but its object and
attribute counts are scoped to the caller's browsable subtrees.

A request without an API key returns `Unauthenticated`. A request with a key
that lacks `metadata:read` returns `PermissionDenied` with the missing scope
embedded in the status detail.

## Dashboard Surface

The gateway's Blazor dashboard surfaces a Galaxy summary in two places:

- An overview card on `/dashboard` showing connectivity status, last deploy
  timestamp, object count (with area count), attribute total, historized and
  alarm counts, and last successful refresh.
- A dedicated `/dashboard/galaxy` page with object-category and top-template
  breakdowns plus a Sync Info table covering last successful refresh, last
  attempt, refresh interval, redacted connection string, and command timeout.

Both views are projected from the same `IGalaxyHierarchyCache` that backs the
gRPC service. The dashboard does not run its own refresh — when the
background `GalaxyHierarchyRefreshService` updates the cache, both the
overview card and the `/dashboard/galaxy` page pick up the new state on the
next dashboard tick. When SQL is unreachable, the cache retains the previous
data and flips `Status` to `Stale` or `Unavailable`; the dashboard surfaces
that as a yellow or red status badge plus the truncated error.

## Operational Notes

- The service is registered alongside `MxAccessGatewayService` in
  `GatewayApplication.MapGatewayEndpoints`. Both services share the same
  authorization interceptor and authentication policy.
- Failures to reach the Galaxy database surface as `Unavailable`. Detailed
  SQL exceptions are logged at `Warning` and never returned to clients.
- Integration tests live in
  `src/MxGateway.IntegrationTests/Galaxy/GalaxyRepositoryLiveTests.cs`. Set
  `MXGATEWAY_RUN_LIVE_GALAXY_TESTS=1` (and optionally
  `MXGATEWAY_LIVE_GALAXY_CONN`) to run them; otherwise they skip.

## Related Documentation

- [Contracts](./Contracts.md)
- [Grpc](./Grpc.md)
- [Authorization](./Authorization.md)
- [Gateway Configuration](./GatewayConfiguration.md)