mxaccessgw/docs/plans/2026-05-28-lazy-browse-design.md

# Lazy Browse for Galaxy Repository

Date: 2026-05-28
Status: approved, ready for implementation plan

## Problem

`GalaxyRepository.DiscoverHierarchy` returns the whole hierarchy (paged, but
clients still walk every object). The dashboard `BrowsePage` then materializes
the full tree into a Blazor component graph on every render. For large
deployments this means oversized gRPC replies for external clients and a
heavy DOM for the dashboard, even when an operator only wants to expand a
single Area.

External integrations (OPC UA bridges in particular) need OPC UA-style "browse
one level on demand" semantics instead of bulk download.

## Scope

Lazy browse lands at three layers:

1. **gRPC** — a new `BrowseChildren` RPC that returns the direct children of a
   parent.
2. **Dashboard** — `BrowsePage` loads roots only and fetches each level on
   expand, in-process (no gRPC self-hop).
3. **Server projection** — a parent→children index added to the existing
   `IGalaxyHierarchyCache` entry; reused by both the RPC and the dashboard
   service.

**Out of scope:** changing how the gateway loads Galaxy SQL. The hierarchy
cache still pulls the full deployment on each deploy change; persistence and
the dashboard summary are unchanged. The lazy story is wire-side and
UI-side only.

## Architecture

```text
gRPC client                           Dashboard (Blazor circuit)
   |                                       |
   v                                       v
GalaxyRepositoryGrpcService           IDashboardBrowseService (in-process)
   .BrowseChildren                        .GetRoots / .GetChildren
        \                                /
         \                              /
          v                            v
        GalaxyBrowseProjector.ProjectChildren
                       |
                       v
        IGalaxyHierarchyCache.Current  (GalaxyHierarchyCacheEntry)
                       |
                       +-- ObjectViews         (existing)
                       +-- ChildrenByParent    (NEW, built once per refresh)
```

`GalaxyBrowseProjector` is the single source of truth for "direct children of
a parent, filtered, ordered, paged." Both the gRPC handler and the dashboard
service call it.

## Proto contract

Added to `src/ZB.MOM.WW.MxGateway.Contracts/Protos/galaxy_repository.proto`,
additive only per the existing wire-compatibility policy.

```proto
service GalaxyRepository {
  // ... existing RPCs ...
  rpc BrowseChildren(BrowseChildrenRequest) returns (BrowseChildrenReply);
}

message BrowseChildrenRequest {
  oneof parent {
    int32 parent_gobject_id = 1;
    string parent_tag_name = 2;
    string parent_contained_path = 3;
  }
  int32 page_size = 4;                  // default 500, max 5000
  string page_token = 5;                // opaque

  // Filter parity with DiscoverHierarchy. AND-combined.
  repeated int32 category_ids = 6;
  repeated string template_chain_contains = 7;
  string tag_name_glob = 8;
  optional bool include_attributes = 9; // default true
  bool alarm_bearing_only = 10;
  bool historized_only = 11;
}

message BrowseChildrenReply {
  repeated GalaxyObject children = 1;
  string next_page_token = 2;
  int32 total_child_count = 3;          // matching direct children (post-filter)
  repeated bool child_has_children = 4; // parallel-indexed with `children`
  uint64 cache_sequence = 5;
}
```

`GalaxyObject` and `GalaxyAttribute` are unchanged. Root browse = empty
`parent` oneof.

### Error mapping

| Condition | Status |
|---|---|
| Unknown `parent_gobject_id` / `parent_tag_name` / `parent_contained_path` | `NotFound` |
| Stale `page_token` (cache deployed forward) | `InvalidArgument`; current `cache_sequence` in trailers |
| Filter set differs between pages of the same token | `InvalidArgument` |
| First load not complete within 5s | `Unavailable` |
| API key missing `metadata:read` scope | `PermissionDenied` |
| No API key | `Unauthenticated` |

Stale and filter-changed page tokens both surface as `InvalidArgument` — same contract as `DiscoverHierarchy`, since `BrowseChildren` reuses the same token encoding (`sequence:filter-signature:offset`).

`browse_subtrees` API-key constraints intersect with the request as today.

## Server projection

### New index on `GalaxyHierarchyCacheEntry`

Built once per refresh, in `GalaxyHierarchyIndex`:

```csharp
// gobject_id -> direct children, sorted areas-first then by display name.
// Roots (parent_gobject_id == 0 or absent) live under sentinel key 0.
IReadOnlyDictionary<int, IReadOnlyList<GalaxyObjectView>> ChildrenByParent;
```

Cost: one O(n) pass over `ObjectViews` during refresh. Memory: O(n) — one int
plus a list reference per object, negligible next to the existing object list.

### `GalaxyBrowseProjector.ProjectChildren`

1. Resolve `parent` selector to `parent_gobject_id` (root sentinel for empty).
   Unknown parent → `NotFound`.
2. Look up direct children from `ChildrenByParent`.
3. Apply the same filter pipeline as `DiscoverHierarchy`. Extract the filter
   logic from `GalaxyHierarchyProjector.ApplyFilters` into a shared helper
   used by both projectors so filter semantics cannot drift.
4. Intersect with API-key `browse_subtrees` globs.
5. Memoize the filtered, ordered list keyed by
   `(parent_id, filter_signature)` in a
   `ConditionalWeakTable<GalaxyHierarchyCacheEntry, ...>` — mirrors the
   existing `GalaxyHierarchyProjector.FilteredViewCache` pattern; GC'd when
   the entry is.
6. Compute `child_has_children` for each returned child against the filtered
   descendant set (recurse down `ChildrenByParent`, short-circuit on first
   match; memoize the per-child boolean alongside the filtered list).
7. Slice page; encode `next_page_token` =
   `{cache_sequence, parent_id, filter_signature, offset}` base64 protobuf.

Sort order: areas first, then `OrdinalIgnoreCase` by display name —
identical to `DashboardBrowseTreeBuilder.CompareNodes` so the dashboard and
external clients see the same ordering.

## gRPC handler

`GalaxyRepositoryGrpcService.BrowseChildren`:

1. Wait up to 5s for first cache load (same `WaitForFirstLoadAsync` pattern
   as `DiscoverHierarchy`).
2. Read `Current` entry.
3. Call `GalaxyBrowseProjector.ProjectChildren`.
4. Map `GalaxyObjectView` → `GalaxyObject` proto via existing
   `GalaxyProtoMapper.MapObject`. Skeleton branch (`include_attributes=false`)
   omits the attribute list — already supported by the mapper for
   `DiscoverHierarchy`.

Authorization: `GatewayGrpcScopeResolver.BrowseChildren` requires
`metadata:read` — same scope as `DiscoverHierarchy`.

## Dashboard

New in-process service:

```csharp
public interface IDashboardBrowseService
{
    BrowseLevelResult GetRoots(BrowseFilterArgs filter);
    BrowseLevelResult GetChildren(int parentGobjectId, BrowseFilterArgs filter);
    GalaxyObject? FindByContainedPath(string path); // for deep-link expansion
}

public sealed record BrowseLevelResult(
    IReadOnlyList<DashboardBrowseNode> Nodes,
    int TotalCount,
    ulong CacheSequence);
```

Backed by the same `GalaxyBrowseProjector` the gRPC service uses — dashboard
and external clients render identical results.

`BrowsePage.razor` changes:

- Initial render fetches roots only.
- Each `DashboardBrowseNode` tracks `LoadState`
  (`NotLoaded` / `Loading` / `Loaded` / `Error`).
- `BrowseTreeNodeView.razor` shows the expand triangle from
  `HasChildrenHint` (the projector's `child_has_children`) regardless of
  whether children have been loaded yet. First expand triggers an async load
  and re-render.
- Loaded children are kept for the lifetime of the Blazor circuit OR until
  the cache sequence advances (subscribed via the existing
  `IGalaxyDeployNotifier`). On invalidation: collapse all expansions, show a
  one-line "Galaxy redeployed, click to re-expand" hint.
- Attributes render inline under their object (unchanged) since
  `include_attributes=true` is the default.
- Search box: when non-empty, fall back to the existing bulk
  `DiscoverHierarchy` path + `DashboardBrowseTreeBuilder`. Lazy browse is for
  unfiltered exploration; building a streaming search is out of scope.

`DashboardBrowseTreeBuilder.Build` stays, used by the search fallback and
the existing tests.

## Testing

Unit tests (no live MXAccess / Galaxy required):

- `GalaxyHierarchyIndexTests` — `ChildrenByParent` correctness: roots under
  sentinel, deep nesting, self-parented objects, duplicate ids.
- `GalaxyBrowseProjectorTests` —
  - Direct-children selection by `gobject_id`, `tag_name`,
    `contained_path`.
  - Filter parity with `DiscoverHierarchy`: same fixtures, same filter
    permutations, identical filtered-set output.
  - `child_has_children` correctness under filters that eliminate
    descendants.
  - Ordering matches `DashboardBrowseTreeBuilder` byte-for-byte.
  - Sibling pagination across multiple pages.
  - Page-token round trip (serialize → deserialize → same offset).
  - Stale `page_token` → `InvalidArgument`.
  - Unknown parent → `NotFound`.
  - Filter change between pages of the same token → `InvalidArgument`.
- `GalaxyRepositoryGrpcServiceTests` — new `BrowseChildren` happy path,
  first-load wait, `browse_subtrees` intersection, `metadata:read` scope
  enforcement.
- `DashboardBrowseServiceTests` — in-process service matches projector
  output, deploy-sequence invalidation, error states.
- `ProtobufContractRoundTripTests` — extended for
  `BrowseChildrenRequest` / `BrowseChildrenReply`.

Manual verification on a live MXAccess host: expand a deep Area in the
dashboard, force a redeploy mid-expand, observe invalidation banner.

No new opt-in live-only tests — the projector is pure over the cache, and
the cache itself is already exercised by existing live tests.

## Documentation updates (same change)

Per the repo rule that doc updates ride with the code change:

- `docs/GalaxyRepository.md` — add `BrowseChildren` to the RPC Surface
  table; new subsection covering semantics, filter parity, page-token
  contract, and error mapping. Update the architecture diagram to show the
  shared projector.
- `gateway.md` — one-line entry for `BrowseChildren` in the Galaxy RPC
  table.
- `clients/{dotnet,python,rust,go,java}/README.md` — short "Browsing
  lazily" snippet showing a `BrowseChildren` walk with `child_has_children`
  driving expand-triangle UI.
- `docs/DesignDecisions.md` — entry noting the explicit choice to keep the
  full hierarchy in the server cache; lazy is wire-only.

## Non-goals

- SQL-side incremental loading. The gateway still pulls the full Galaxy
  hierarchy on each deploy change.
- Multi-snapshot retention. A deploy invalidates outstanding page tokens;
  clients re-walk.
- Streaming search. Search keeps the bulk `DiscoverHierarchy` + tree-build
  path.
- Reconciling a partially-expanded dashboard tree across a deploy — the
  invalidation banner forces a re-expand.