Claude-Session: https://claude.ai/code/session_01Y5ef34iUUMjmwnMpL9xgmB
79 KiB
Component: Central UI
Purpose
The Central UI is a web-based management interface hosted on the central cluster. It provides all configuration, deployment, monitoring, and troubleshooting workflows for the SCADA system. There is no live machine data visualization — the UI is focused on system management, with the exception of on-demand debug views.
Location
Central cluster only. Sites have no user interface.
Technology
- Framework: Blazor Server (ASP.NET Core). UI logic executes on the server, updates pushed to the browser via SignalR.
- Keeps the entire stack in C#/.NET, consistent with the rest of the system (Akka.NET, EF Core).
- SignalR provides built-in support for real-time UI updates.
- Dialog service confirm labels. Every confirmation goes through
IDialogService.ConfirmAsync(title, message, danger, confirmLabel), rendered by the singleDialogHostinMainLayout.dangerselects styling only (btn-danger);confirmLabelnames the action. When no label is supplied the button falls back to "Delete" for a danger confirm and "Confirm" otherwise — a convenience for the ~20 genuine delete call sites, and the reason the Health dashboard's Trigger failover dialog used to offer a red Delete button (deferred-work register row 28). Any destructive action that is not literally a delete must passconfirmLabel: failover → Fail over, secured-write approve/reject → Approve / Reject, instance disable → Disable, and every parked-message / notification / cached-call discard → Discard. The two most-cited verbs are pinned by test:Monitoring/FailoverConfirmLabelTests(failover, plus that an unlabelled danger confirm still defaults to "Delete") andDeployment/TopologyDisableConfirmLabelTests(Topology's instance Disable reads "Disable" and stays red, while its Delete still reads "Delete"). Both drive a realDialogServicebehind a realDialogHost, so they pin what the operator reads, not the argument a substitute received. - Dialog confirm severity — three rungs, not two (deferred-work register row 75).
DialogSeveritydrives the confirm button's colour:Normal→btn-primary,Caution→btn-warning,Destructive→btn-danger. OnlyDestructiveinherits the "Delete" label fallback above; an unlabelledCautionreads Confirm, because a caution action is by definition not a delete.- Why a middle rung exists. The boolean was a two-state flag for a three-state concept, so a consequential-but-not-destructive action had to choose between looking like a delete and looking like the safe default. The Secured Writes page is where that bit: Approve and Reject both rendered as identical red buttons, so at the moment of a two-person-control decision the colour carried no discriminating information at all — and the affirmative half of the control wore the delete affordance.
- The decision. Approve is
Caution(amber): approving executes a real write to live plant equipment, so it is not a benign primary action, but it is not destroying anything either. Reject keepsDestructive(red): it is terminal and irreversible for the request — the Operator must resubmit — and holding it red is what keeps the two opposite decisions visually distinct. Dropping Reject to a neutral button was considered and rejected: it would simply invert the bias, making declining look like the safe default. - Colour is never the only signal — the button always carries its verb (see the label contract above), so the mapping is an accessibility affordance, not the meaning.
DialogSeverityis an additive overload, not a replacement: the boolean form remains the signature for the ~34 existing call sites, which are genuinely two-state. Adding a fifth optional parameter to the boolean form instead would have silently un-matched every existing four-argument NSubstitute stub — the row-28 gotcha, where a stub that stops matching returns a nullTaskand the caller NREs with no compile error. Reach for the overload only whenCautionis what you mean.
Review hardening (2026-09-09)
- Cookie login validates an antiforgery token before reading credentials or calling LDAP; the login form supplies the token. The JSON token endpoint retains its non-cookie client contract.
- Manual failover goes through a scoped
AuthorizedManualFailoverServicethat refreshes the circuit's Administrator grant before calling the host singleton. The component's visibility policy is an additional presentation guard. - Flow Debugger retry/discard validates the current Deployer grant and resolves site scope inside its service. Scoped callers cannot relay an unknown site. Refresh merges preserve Replay's event ordering and append only rows known exclusively to the live run.
- Health and Alarm Summary observe the whole posted refresh task and release their poll gate even when renderer dispatch fails during disposal.
- The Topology initially loads permitted sites. Expand a site and select Load instances to load that site's area metadata and first root-instance page; each expanded area has its own loader. Previous and Next replace that branch's 50-instance page. Queries are untracked and omit member collections; templates are not loaded. Live refresh and staleness checks cover loaded pages. Search and counts explicitly describe loaded data.
TemplateFolderTreecompares immutable value snapshots of folder/template hierarchy and extra children before rebuilding. In-place renames, moves, derived-state changes, and composition edits remain visible; filter and reveal changes are applied independently.- Navigation grants Flow Debugger its own policy and hides Notifications unless the principal holds a role that admits a child. Route-policy coverage freezes every CentralUI route and the external Secrets route.
Failover Behavior
- A load balancer sits in front of the central cluster and routes to the active node.
- On central failover, the Blazor Server SignalR circuit is interrupted. The browser automatically attempts to reconnect via SignalR's built-in reconnection logic.
- Since sessions use authentication cookies carrying an embedded JWT (not server-side state), the user's authentication survives failover — the new active node validates the same cookie-embedded JWT. No re-login required if the token is still valid.
- Active debug view streams and in-progress deployment status subscriptions are lost on failover and must be re-opened by the user.
- Both central nodes share the same ASP.NET Data Protection keys (stored in the configuration database or shared configuration) so that tokens and anti-forgery tokens remain valid across failover.
Real-Time Updates
- Debug view: Real-time display of attribute values and alarm states via gRPC streaming. When the user opens a debug view, a
DebugStreamBridgeActoron the central side opens a gRPC server-streaming subscription to the site'sSiteStreamGrpcServerfor the selected instance, then requests an initialDebugViewSnapshotover the central→site gRPC command channel (SiteCommandService,QueryReply.DebugViewSnapshot). OngoingAttributeValueChangedandAlarmStateChangedevents flow via the gRPC data stream to the bridge actor, which delivers them to the Blazor component via callbacks that callInvokeAsync(StateHasChanged)to push UI updates through the built-in SignalR circuit. - Health dashboard: Site status, connection health, error rates, and buffer depths update via a 10-second auto-refresh timer. Since health reports arrive from sites every 30 seconds, a 10s poll interval catches updates within one reporting cycle without unnecessary overhead.
- Deployment status: Pending/in-progress/success/failed transitions push to the UI immediately via SignalR (built into Blazor Server). No polling required for deployment tracking.
Responsibilities
- Provide authenticated access to all management workflows.
- Enforce role-based access control in the UI (Admin, Design, Deployment with site scoping).
- Present data from the configuration database, and from site clusters via remote queries.
Workflows / Pages
Template Authoring (Design Role)
- The
/design/templatespage uses a split-pane layout: a folder/template tree sidebar on the left and the editor on the right. - The tree shows nested
TemplateFolderentities with their templates underneath; composition children render inline beneath their owning template (right-click "Open composed template" reveals and selects the target). Compositions are shown by effective set (own + inherited): a derived/composed member surfaces the slots it inherits from its base — e.g.LeakTestcomposed onto baseReactorSideappears under each derived…ReactorSidemember — badged "inherited" and read-only (Rename/Delete are offered on the base's own slot, not the inherited copy). - Per-kind context menus on folder, template, and composition nodes expose the relevant operations (new folder, new template, rename, move, delete, move to folder). A Template node also offers "New Instance…", which navigates to
/deployment/instances/create?templateId={id}with the picker field prefilled; it is gated on theDeployerrole claim (exactly whatRequireDeploymentrequires) so the item never leads a Designer-only user to a 403. Root-level folders also carry a context menu. Folder sibling reorder is done via Move up / Move down menu items (M9/T23,ReorderTemplateFolderCommand); drag-drop is not implemented (permanently deferred). Tree expansion state persists insessionStorage, and deep links (/design/templates/{id}) reveal and select the target node. - A search box above the tree (M9/T22) filters visible nodes by substring match; it is wired to
TemplateFolderTree.Filter. - The
TemplateEditpage shows a read-only "Inherited members" panel (M9/T26) listing the full multi-level effective inherited member set (origin, locked state, merged HiLo config) resolved byGetResolvedTemplateMembersCommand/TemplateInheritanceResolver. A "Base changed" banner appears when the resolver's staleness summary indicates the parent template has changed since the child was last edited. This is read-only — no "update-derived" mutation is exposed; the child is redeployed through the normal flow to pick up base changes. - Create, edit, and delete templates.
- Template deletion is blocked if any instances or child templates reference the template. The UI displays the references preventing deletion.
- Manage template hierarchy (inheritance) — visual tree of parent/child relationships.
- Manage composition — add/remove feature module instances within templates. Naming collision detection provides immediate feedback if composed modules introduce duplicate attribute, alarm, or script names.
- Define and edit attributes, alarms, and scripts on templates.
TemplateAlarmDialogcarries an "MES reportable" checkbox bound toTemplateAlarm.IsMesReportable(default unchecked, 2026-09-11) alongside the existing Name/Trigger Type/Priority/Trigger Definition/Description/Lock fields.- Member authoring dialogs are hosted, not page-embedded (M10 residual, 2026-08-01): the template editor's Attribute, Alarm, Native Alarm Source, and Script forms all open through
IDialogService.ShowAsync, so the singleDialogHostinMainLayoutowns the backdrop, focus trap, Escape, and focus restoration. Each form body is its own component beside the page (TemplateAttributeDialog,TemplateAlarmDialog,TemplateNativeAlarmSourceDialog,TemplateScriptDialog), following theMoveDataConnectionDialogpattern — the body renders validation and server errors inline and stays open, closing only on a successful save; persistence stays on the page (which ownsTemplateService) behind anOnSaveAsyncdelegate. - Native Alarms tab (
TemplateEdit): a tab alongside Attributes / Alarms / Scripts / Compositions that lists the template's native alarm source bindings — the OPC UA Alarms & Conditions / MxAccess Gateway sources whose alarm state the instance mirrors. Each binding carries Name, Connection, Source Reference, optional Condition Filter, Description, and a Lock flag. Add / edit / delete go through a hosted dialog (TemplateNativeAlarmSourceDialog):- Name — unique within the template (lock/inherit bookkeeping mirrors
TemplateAlarm). - Connection — a dropdown filtered to alarm-capable connections only (OPC UA and MxGateway protocols).
- Source Reference — the native key (OPC UA SourceNode / notifier nodeId, or MxAccess object/area).
- Condition Filter (optional) — blank mirrors all conditions under the source.
- Description (optional) and Lock (prevents instance-level override, like locked alarms/attributes).
- CRUD is repository-direct (Blazor Server runs in-process against
ICentralUiRepository); no Akka round-trip is needed for design-time authoring.
- Name — unique within the template (lock/inherit bookkeeping mirrors
- Set lock flags on attributes, alarms, and scripts.
- Visual indicator showing inherited vs. locally defined vs. overridden members.
- On-demand validation: A "Validate" action allows Design users to run comprehensive pre-deployment validation (flattening, naming collisions, script compilation, trigger references) without triggering a deployment. Provides early feedback during authoring.
- Last-write-wins editing — no pessimistic locks or conflict detection on templates.
Shared Script Management (Design Role)
- Create, edit, and delete shared (global) scripts.
- Shared scripts are not associated with any template.
- On-demand validation (compilation check) available.
External System Management (Design Role)
- Define external system contracts: connection details, API method definitions (parameters, return types).
- Define retry settings per external system (max retry count, fixed time between retries).
- The external system detail page includes a "Recent activity" link that opens the Audit Log page pre-filtered to
Channel = ApiOutboundandTargetstarts-with the system name — surfacing the system's recent outbound API audit history.
Database Connection Management (Design Role)
- Define named database connections: server, database, credentials.
- Define retry settings per connection (max retry count, fixed time between retries).
Notification List Management (Design Role)
- Create, edit, and delete notification lists.
- Each notification list has a
Type—EmailorSms(SMS via Twilio shipped 2026-06-19; a Teams adapter was evaluated and dropped). The type determines the type-specific targets a list carries. - Manage recipients (name + email) within each
Emaillist. - Configure SMTP settings.
Site & Data Connection Management (Admin Role)
- Create, edit, and delete site definitions, including gRPC node addresses (GrpcNodeA/GrpcNodeB). The legacy Akka node addresses (NodeA/NodeB) are still stored and editable but have had no runtime consumer since the ClusterClient→gRPC migration's Phase 4 — every central→site dial resolves the gRPC pair. The form labels them as legacy so an operator does not mistake them for a live setting.
- Define data connections and assign them to sites (name, protocol type, connection details).
- Data connection form: "Primary Endpoint Configuration" (required JSON text area) and optional "Backup Endpoint Configuration" (collapsible section, hidden by default, revealed via "Add Backup Endpoint" button; "Remove Backup" button when editing an existing backup). "Failover Retry Count" numeric input (default 3, min 1, max 20) is visible only when a backup endpoint is configured.
- Verify endpoint (OPC UA): the OPC UA endpoint editor (in the data connection form) carries a "Verify endpoint" button that asks the target site to probe the configured endpoint — a temporary, short-lived connect against the live (or edited-but-unsaved) config. The result reports success or a typed failure kind (e.g. unreachable, untrusted certificate, server error). When the failure is an untrusted server certificate, the probe captures the cert (Subject / Issuer / Thumbprint / validity / DER) and the editor shows a detail panel with a "Trust certificate" button. The probe itself never trusts the cert — trusting is an explicit, Admin-gated action (see Server certificate management). After a Trust, Verify re-runs automatically and should then succeed.
- Data connection list page: Shows Primary Config and Backup Config columns. Active Endpoint column populated from health reports.
- Connection live-status indicators (M9/T25): the design DataConnections page polls
ConnectionHealthQueryService(~10 s interval) which readsSiteHealthReport.DataConnectionStatusesalready delivered by Health Monitoring; no new site-side code. Each connection row displays its current status badge (Connected / Disconnected / Unknown). - Move data connection between sites (M9/T24): a Move… action on a data connection opens
MoveDataConnectionDialog. The underlyingMoveDataConnectionCommandguards against: target site does not exist, name collision at the target, anyInstanceConnectionBindingreferencing the connection, and anyInstanceNativeAlarmSourceOverride.ConnectionNameOverridereference at the source site's templates. Blocked connections display the blocking reason. - Server certificate management (
/design/connections/{id}/certificates, Admin role): a per-connection page that lists the contents of the site's OPC UA trusted-peer and rejected certificate stores (Subject / Issuer / Thumbprint / validity / Trusted-or-Rejected status) with a Remove action. The page makes clear the store is node-wide for the site (shared by every site node), not per data connection — trusting or removing a certificate affects all OPC UA connections at that site. Trust and Remove are central commands relayed to both site nodes so the node-local PKI stores stay consistent across failover (see Component-SiteRuntime.md, Component-DataConnectionLayer.md). - The site detail page exposes a new "Audit feed" tab that hosts the Audit Log page pre-filtered to
Site = <site>— an in-context view of every operational audit event for that site.
Schema Library (Design Role)
- A Schema Library page (M9/T32, under the Design nav group) provides CRUD for
SharedSchemaentries — named, reusable JSON Schema definitions. Each entry has a name (globally unique) and a JSON Schema body. - Schemas are referenced from parameter/return definitions using
{"$ref":"lib:Name"}. The resolver expands refs at deploy-time, design-time validation, and inbound-API runtime (cycle and depth guards enforced; a dangling ref blocks deployment). - The Central UI also uses the
ParameterValueForm(M9/T30) to renderobjectandlistparameters as typed nested inputs when a JSON Schema (or$ref) is attached to the parameter definition. Monaco-based JSON fields gain hover and completion (M9/T31) from the attached schema.
Inbound API Management (Admin Role for keys, Design Role for methods)
- Manage inbound API keys (create, enable / disable, delete) and define API methods (name, parameters, return values, approved keys, implementation script).
- The API key detail page includes a "Recent calls" link that opens the Audit Log page pre-filtered to
Actor = <key name>andChannel = ApiInbound— surfacing the key's recent inbound-call audit history.
Bundle Export / Import (Design Role for export, Admin Role for import)
Export Bundle(/design/transport/export) — multi-step wizard for exporting a.scadabundleartifact containing templates, shared scripts, external systems, and central-only configuration. Visible to users withRequireDesign.Import Bundle(/design/transport/import) — multi-step wizard for uploading and applying a.scadabundleinto the current environment, with per-artifact diff review and conflict resolution. Visible to users withRequireAdmin.
Area Management (Designer or Deployer Role)
- Areas are authored inside the Deployment Topology workflow (
Topology.razor,RequireDeployment); managing them is gated any-of [Designer, Deployer], not Administrator (arch-review C6). - Define hierarchical area structures per site.
- Parent-child area relationships.
- Assign areas when managing instances.
Instance Management (Deployment Role)
- Create instances from templates at a specific site.
- The template is chosen from the Templates folder tree, not a flat dropdown (
InstanceCreate, designdocs/plans/2026-08-21-template-picker.md). The Template field is a read-only, folder-qualified path box ("Controllers / Conveyor / MES Line Controller", rendered byTemplatePathFormatter) with a Choose… button; both openTemplatePickerDialog, aIDialogService.ShowAsyncbody wrapping the same sharedTemplateFolderTreethe Templates page renders — its own search box, expand/collapse all, templates-only selection (folders just toggle), double-click to pick, and a footer showing the pending path. It opens expanded to and highlighting the current pick via the tree's additiveRevealKeysparameter (ancestors of the namedf:/t:keys start expanded when no filter is active; an active filter's own reveal set still wins, and callers that pass nothing are unaffected). A flat<select>was unusable once the catalogue grew past a screenful and showed no folder context at all.?templateId=preselects. The page takes aTemplateIdquery parameter, which is what the Templates tree's "New Instance…" navigates with; combined with the existing?siteId=, the Connection Bindings card below is populated on first render. An id that no longer exists leaves the field blank and shows "Template N no longer exists — choose another." rather than failing the page. With no templates at all, Choose… is disabled and the field points at the Templates page.
- Connection Bindings at creation (
InstanceCreate, designdocs/plans/2026-08-21-create-instance-bindings-design.md). Once a template AND a site are chosen, the create form grows a Connection Bindings card showing the SAME module groups the Configure page shows, so one Create can yield a deployable instance instead of always landing on Configure with 100% of the rows unbound. The groups come fromIFlatteningPipeline.PreviewAsync(templateId, siteId)— a flatten of an unsaved instance with no overrides and no bindings — fed through the sameBindingGroupBuilderConfigure uses. Group headers only: per-member editing stays on Configure.- Bind all to — a connection + root object, then Prefill groups sets the connection on every group and fills blank Objects only: the
(instance)group gets the root verbatim; a module group whose template defaults already carry a shared prefix keeps it (pill: from template); any other module group is suggested{root}.{groupKey}(pill: suggested). An Object the operator typed is never overwritten. Provenance pills exist because a suggestion is a guess about the address space — a sibling-object layout needs correcting before Create. - Browse… per group (and for the root) opens the shared
NodeBrowserDialogwithSelectableNodeClasses = {Object, Variable}— the create page binds a module to its container node, which the Variable-only default could not select. Picking a leaf under the module still works:CreateBindingPlanner.ObjectFromPickstrips a trailing.{memberTail}matching any member of that group. Enabled only for browsable protocols (OPC UA, MxGateway), the same gate Configure applies. - Partial binding is legal. A group with both a connection and a non-blank Object contributes
{object}.{memberTail}for every member; a group missing either is simply skipped and stays unbound for Configure. The footer states the count that will be saved and how many groups stay unbound (and that Deploy is rejected until they are bound). A Skip bindings checkbox restores the pre-2026-08-21 behaviour exactly. - Create is two calls, and the instance survives the second failing.
CreateInstanceAsyncthenSetConnectionBindingsAsync(an upsert — the same command the Configure page's Save uses; no new server-side surface). The page then ALWAYS navigates to/deployment/instances/{id}/configure?created=1plus either&bound=Nor&bindError=<message>, and Configure renders a one-shot notice from those query params. Leaving the operator on a stale create form after the row already exists would invite a duplicate-name retry. - A preview flatten that fails (or throws) degrades to a warning inside the card and suppresses the binding save — the instance is still creatable, and Configure remains the authority.
- Bind all to — a connection + root object, then Prefill groups sets the connection on every group and fills blank Objects only: the
- The template is chosen from the Templates folder tree, not a flat dropdown (
- Assign instances to areas.
- Bind data connections — per-attribute binding where each attribute with a data source reference individually selects its data connection from the site's available connections. The binding rows come from the instance's flattened configuration (
IFlatteningPipeline.FlattenAndValidateAsync(id, validateScripts:false)), so every data-sourced attribute the deploy gate will check appears — composed feature-module members (LeftMESReceiver.MoveInFlag,LeftReactorSide.LeakTest.DeltaVac, …) included. (Until 2026-08-20 the panel was built from the template's direct attributes only, which left 264 of a composed QA instance's 280 bindings invisible and made the instance undeployable from the UI; designdocs/plans/2026-08-20-grouped-connection-bindings-design.md.) Only a genuine flatten failure falls back to the direct-attribute list, with a visible warning banner; a validation failure (e.g. unbound attributes — the very thing the panel exists to fix) is not a reason to fall back.- Grouped by module. Rows are grouped by the first segment of the canonical name —
(instance)for direct attributes first, then one collapsible group per composed module in name order; nested modules fold into their first segment (LeftReactorSideshows incl. LeakTest) and members keep their nested tail (LeakTest.DeltaVac). The pure grouping/inference/apply logic isBindingGroupBuilder(CentralUIServices/), unit-tested without rendering the page. - Group header = connection + Object + Apply. Each group header carries a connection select and an Object input; Apply to n rewrites every member of the group to
<object>.<memberTail>on that connection (a blank Object clears the overrides so the template default applies; surrounding whitespace and a trailing.are tolerated). A header connection left at— none —/— mixed —keeps each member's current connection — Apply never unbinds a group (the Save path is an upsert, so an unbinding could not persist anyway). The Object input is pre-filled with the inferred shared prefix over every member's effective reference (override if set, else the default):MESReceiver_022.MoveInFlag+MESReceiver_022.ScanStateCmd⇒MESReceiver_022; nested object paths such asLeft_002.DelmiaReceiverinfer too; a direct-attribute group whose defaults already carry a shared prefix infers it with no overrides at all. Once any member has an override, the group shows a mixed pill — and the input's placeholder invites an object to unify it — when members disagree, an override does not end in its member tail, or a member still sits on a bare default the deploy gate would reject; a group with no overrides and bare defaults is merely "unset", not mixed. A binding to a connection that no longer exists at the site counts as unbound (it renders as— none —and the gate rejects it). Apply edits the in-memory maps only — Save Bindings persists every row (hidden rows included) through the existingSetConnectionBindingsupsert; until then the header shows ann changed · unsavedpill counting members whose (connection, override) differs from the last loaded/saved state, so per-row edits and group Applies both surface. - Pills and filters. The card header totals unbound rows and mixed groups; each group shows
n unbound(red) orbound, plusmixed; unbound rows render highlighted. A Show only unbound filter hides bound rows and fully-bound groups; Expand all opens every group. The legacy all-groups connection select (Assign all to…) is retained. A composed member whose template default is the bare member name (MoveInFlag) carries a ⚠ marker — such a default cannot be a real address and the deploy gate will reject the instance until an override (or a group Object) is set. - Template default column shows
ResolvedAttribute.DefaultDataSourceReference— the reference as resolved from the template chain before the per-instance override — so the operator always sees both the default and the effective override. Test Bindings (button label carries the testable-row count) reads every bound row with an effective path, composed rows included. - Override — optional per-attribute OPC UA node id (or other protocol address). When set, replaces the template's
DataSourceReferenceat flattening time; when blank, the template default is used. The greyed placeholder shows the template default for context. - Browse… — opens the OPC UA Tag Browser dialog, populated live from the site's OPC UA server via
BrowseOpcUaNodeCommand. Visible only when the row's connection uses the OPC UA protocol; disabled until a connection is picked on that row. The dialog lazy-loads the address space, supports manual node-id entry as a fallback, and remains usable when the site or its OPC UA session is offline (the manual-paste field stays active even on error). The dialog adds:- Load more — when a browse level is truncated, a "Load more" affordance fetches the next page using the server's continuation point (
BrowseNext); an expired continuation point falls back to a fresh browse. - Search — a search box runs a bounded recursive address-space search (depth + result caps) at the site, matching a case-insensitive substring against node DisplayName/path; clicking a result selects it. The dialog surfaces a "showing first N — refine" note when a result cap is hit.
- Type column — Variable rows display best-effort type info (data type friendly name, scalar/array value rank, writable flag) read from the server during browse.
- Load more — when a browse level is truncated, a "Load more" affordance fetches the next page using the server's continuation point (
- Grouped by module. Rows are grouped by the first segment of the canonical name —
- Set instance-level attribute overrides (non-locked attributes only).
- Bulk override CSV import (
InstanceConfigure): a BlazorInputFileupload accepts a CSV ofAttributeName, Value, ElementType?rows (ElementTypeonly forListattributes). Each row is validated against the instance's flattened attribute schema (name exists + value type-compatible, reusing the existing override validation); the import is all-or-nothing — any per-row error aborts the whole upload with a per-row error summary and nothing is applied. On success the rows are upserted through the same ManagementActor add/update-override handlers used by the inline editor. The same import is available from the CLI (instance import-overrides --file, see Component-CLI.md).
- Bulk override CSV import (
- Alarm Overrides card (
InstanceConfigure) gained an "MES reportable override" tri-state select (2026-09-11) in the per-alarm override modal (not on the list row), bound toInstanceAlarmOverride.IsMesReportableOverride— the options are "Inherited (reportable | not reportable)" (the parenthetical reflects the template alarm's own flag) →null, "Reportable" →true, and "Not reportable" →false, which is a real override pinning the alarm off for this instance rather than a clear. The list row shows the inherited flag as a green MES badge and any override in its override summary. Locked alarms are filtered out of the override list entirely (the card only lists non-locked alarms), so there is no disabled-row state for them.instance alarm-override set --mes-reportable true|false(omit the flag to inherit) is the CLI equivalent. - Native Alarm Source Overrides card (
InstanceConfigure): a card placed after the Alarm Overrides card, listing the template's native alarm sources for per-instance binding. Each row offers inline override of the three fields that typically vary per physical instance:- Connection — a dropdown (same alarm-capable filtering as the template editor).
- Source Reference — the concrete native key for this instance.
- Filter — the per-instance condition filter.
- A blank field inherits the template default (the greyed placeholder shows the inherited value for context, mirroring the per-attribute Override field). Save and Clear act per row — Save persists the row's overrides, Clear reverts the row to the template-inherited binding. Locked template sources are not overridable.
- Bulk retarget CSV import: an
InputFileupload on the card header accepts a CSV ofSourceName, Connection, SourceReference, Filterrows — the same file format and batch rules as the CLIinstance native-alarm-source import --file(see Component-CLI.md), parsed with the sharedNativeAlarmSourceOverrideCsvParser. Each row is validated against the template's source bindings (name resolves, source not template-locked, no duplicate source in the file); the import is all-or-nothing — any error aborts the whole upload with a per-line error summary and nothing is applied. Semantics are merge, not full replace: sources absent from the file keep their existing override, a blank field keeps the inherited value, and a row with all three fields blank clears that source's override. On success the rows are upserted through the same repository path the inline Save uses.
- Filter/search instances by site, area, template, or status.
- Disable instances — stops data collection, script triggers, and alarm evaluation at the site while retaining the deployed configuration.
- Enable instances — re-activates a disabled instance.
- Delete instances — removes the running configuration from the site. Blocked if the site is unreachable. Store-and-forward messages are not cleared.
- The instance detail page exposes a new "Audit feed" tab that hosts the Audit Log page pre-filtered to the instance (
Site = <site>and theInstance / Scriptfilter set to the instance unique name) — an in-context view of every operational audit event involving that instance.
Deployment (Deployment Role)
- View list of instances with staleness indicators (deployed config differs from template-derived config).
- Filter by site, area, template.
- View diff between deployed and current template-derived configuration.
- Deploy updated configuration to individual instances. Pre-deployment validation runs automatically before any deployment is sent — validation errors are displayed and block deployment.
- Track deployment status (pending, in-progress, success, failed).
- Server-side paging + server-computed status counts (residual R3). The page is a database-paged list, not a client-materialized one. It used to read every
DeploymentRecord(an insert-only table — one row per deploy attempt for the whole retention window) plus everyInstance, then site-scope, sort, count the four status tiles and slice a 25-row page in the Blazor circuit's memory, on first render and on every deployment-status push. All four jobs are now SQL:IDeploymentManagerRepository.QueryDeploymentListPageAsync(filter, pageNumber, pageSize)returns one page ofDeploymentListRow—DeploymentRecordinner-joined toInstance, so the instance's display name and site travel with the rows that need them (the join is exact: the FK isRestrictand instance deletion removes the records first) — plus the total count of the filtered set.GetDeploymentStatusCountsAsync(filter)returns the tile counts from one grouped aggregation. It deliberately ignores the filter'sStatus— the tiles are the status breakdown of the otherwise-identically-filtered set, so each keeps its own total while it is the selected one.- Site scoping runs in the query. The permitted-site grant is pushed in as
SiteIdScopeand resolved through the record's instance; an empty grant is a real filter matching nothing, never "unconstrained".DeploymentRecordhas noSiteIdof its own — the join is what makes this expressible. - This is now the ONLY paged read of
DeploymentRecords. The CLI/HTTPQueryDeploymentscommand had a second one (QueryDeploymentSummariesAsync: offset instead of page number, no total, no instance join, instance-id-list scoping); it is deleted, and both surfaces share this query. SeeComponent-ManagementService.md→ QueryDeployments for the three behavioral drifts the duplication had accumulated (deferred-work register rows 37, 86–88). - Offset paging, not the Audit Log's keyset cursor — deliberately. This page's pager is numbered and jump-to-any-page, so it needs a page count, which only a total can give it; a keyset cursor can express neither. The total is required for the tiles regardless. The deep-offset cost that pushes high-volume tables to keyset is bounded here by the terminal-record retention purge, unlike the 365-day central
AuditLog. This mirrors the Notification Outbox, which is offset-paged for the same reason. Ordering isDeployedAt DESC, Id DESC; theIdtie-break is load-bearing, becauseDeployedAtties on rapid redeploys and an unstable sort key makes offset paging repeat or drop rows between pages. - The whole-table
GetAllDeploymentRecordsAsyncread was deleted with its last caller.
- Filtering. Status filter (the tiles double as the control — clicking one applies it, clicking it again clears) plus a free-text search matched DB-side against instance unique name, deployment id, revision hash and initiating user. A filter change resets to page 1. Search input is trailing-edge debounced at 500 ms, so a typed term is one query rather than one per keystroke.
- Push-reload coalescing (arch-review WP2.4). The notifier fires per status write, so a site-wide bulk deploy of N instances drove 2N+ back-to-back reloads on the same circuit. Pushes are leading-edge debounced (500 ms): the first push after an idle gap reloads immediately (a single deployment stays as responsive as before), and every push inside the window collapses into one trailing reload. Server-side paging shrank what a reload costs but not how many arrive, so the coalescing still holds — it now bounds round-trips to the database rather than table scans.
System-Wide Artifact Deployment (Deployment Role)
- Explicitly deploy shared scripts, external system definitions, database connection definitions, and data connection definitions to all sites or to an individual site. (Notification lists and SMTP configuration are central-only and are not deployed.)
- Per-site deployment: A "Deploy Artifacts" button on the Sites admin page allows deploying all artifacts to an individual site.
- Deploy all: A bulk action deploys artifacts to all sites at once.
- This is a separate action from instance deployment — system-wide artifacts are not automatically pushed when definitions change.
- Track per-site deployment status.
Debug View (Deployment Role)
- Select a deployed instance and open a live debug view — the instance picked through the same shared
InstancePickercombobox as the Flow Debugger, with anItemLabelsuffix of(State)on each row. - Real-time streaming of all attribute values (with quality and timestamp) and alarm states for that instance.
- The
DebugStreamServicecreates aDebugStreamBridgeActoron the central side. The bridge actor opens a gRPC server-streaming subscription to the site'sSiteStreamGrpcServerfor the selected instance, then requests an initialDebugViewSnapshotover the central→site gRPC command channel (SiteCommandService). - Ongoing events (
AttributeValueChanged,AlarmStateChanged) flow via the gRPC data stream directly to the bridge actor — they do not travel on the command channel. - Events are delivered to the Blazor component via callbacks, which call
InvokeAsync(StateHasChanged)to push UI updates through the built-in SignalR circuit. - Render coalescing (arch-review WP2.4). Streamed events no longer trigger an individual dispatcher marshal +
StateHasChanged()each — a chatty instance used to drive one full render (and two full tree rebuilds) per value change. Events now land in a thread-safe pending map keyed by attribute/alarm name (repeated updates to the same tag inside one window collapse to the latest), and exactly one dispatcher marshal per 250ms coalesce window drains the map, bumps a version stamp, and renders once. This also fixes a latent thread-safety issue: the render dictionaries were plain (non-concurrent)Dictionarys enumerated by the render thread while written from the Akka/gRPC callback thread. - A pulsing "Live" indicator replaces the static "Connected" badge when streaming is active.
- Subscribe-on-demand — stream starts when opened, stops when closed.
- Read-only per-instance view (one instance per connection); no alarm acknowledgement is available from Debug View.
Tabbed Layout
The Debug View page uses a two-tab layout — an Attributes tab and an Alarms tab — replacing the earlier side-by-side flat tables. Each tab renders its data as a collapsible hierarchy tree using the existing generic TreeView<TItem> component. As of 2026-08-01 that component implements the full WAI-ARIA tree keyboard pattern — roving tabindex (one Tab stop per tree), Arrow/Home/End movement, Enter/Space activation, plus aria-level/aria-posinset/aria-setsize — so every tree surface in the Central UI (Debug View, Data Connections, Topology, the template folder browser) is keyboard-navigable. See docs/components/TreeView.md.
Tree hierarchy — the hierarchy is derived from the path-qualified canonical names already present in the debug snapshot (e.g. Motor1.Compressor.Pump). The instance is the root node; composed modules are collapsible branch nodes; individual attributes (Attributes tab) or alarms/native-source bindings (Alarms tab) are leaf nodes.
Branch-level status roll-up — a collapsed branch shows a summary badge so the operator can assess health without expanding:
- Alarms tab: worst alarm state among descendants + count of active conditions.
- Attributes tab: a bad/uncertain-quality indicator when any descendant attribute has non-Good quality.
Attributes Tab
Displays all attribute values for the instance in the collapsible tree. Each leaf shows the attribute value, quality, and timestamp. The existing [InstanceUniqueName].[AttributePath].[AttributeName] canonical path is the basis for the tree structure.
Alarms Tab (Computed + Native)
The Alarms tab is the only runtime surface for native OPC UA Alarms & Conditions and MxAccess Gateway alarms (no dedicated operator/alarm-summary page). All configured alarms are shown with current status, even when quiet/Normal — no alarm is hidden simply because it has not fired.
Both enriched AlarmStateChanged events (live, via the gRPC stream) and the initial DebugViewSnapshot (via the gRPC command channel) carry the unified alarm shape, so all alarms appear on the first paint and update in place. Native alarms are a read-only mirror — the source system owns the alarm lifecycle (ack / shelve / suppress); the Debug View never offers ack-back or any command action.
Native source binding nodes — a configured native alarm source binding is itself a tree node, placed by its canonical name in the hierarchy. Its live mirrored conditions nest as child rows beneath it. A quiet binding (no currently active conditions) renders a "no active conditions" placeholder row — it is never hidden, so the operator can see every configured binding regardless of alarm state. This requires the backend to emit a placeholder AlarmStateChanged with IsConfiguredPlaceholder = true for each idle binding (see Component-SiteRuntime.md — Instance Actor Wiring). The NativeSourceCanonicalName field on AlarmStateChanged events identifies which binding node a live condition belongs to.
Per-leaf alarm rendering (leaf nodes are individual conditions for native alarms, and the alarm itself for computed alarms):
- Kind badge — distinguishes Computed alarms from native ones (OPC UA or MxAccess), driven by the event's
AlarmKinddiscriminator. - Sev — the unified 0–1000 severity (
AlarmConditionState.Severity). Computed rows surface their integer priority on the same scale. - Source reference subtitle — for native rows, the
SourceReference(e.g.Tank01.Level.HiHi) renders as a monospace subtitle. Computed rows have no subtitle. - State badges — orthogonal condition sub-states: Unacked, Shelved, and Suppressed appear only when the corresponding
AlarmConditionStateflag is set. Computed alarms are auto-acked and never shelved/suppressed. - Row tooltip — surfaces native metadata not warranting its own column:
AlarmTypeName, category, operator user and comment, original raise time, current/limit value. - Computed alarms render unchanged from the prior flat-table style; the enrichment is purely additive for native rows.
Flow Debugger (Flow Debugger policy; Live run = Deployment Role)
/deployment/flow-debugger(nav: Deployment → Flow Debugger; designdocs/plans/2026-09-05-flow-debugger-design.md, plandocs/plans/2026-09-05-flow-debugger.md). Runs one entry point with operator-supplied inputs and renders every step the run passes through as a four-lane swimlane (Central / Site / Tags / External) — request, route call, site script run, attribute writes, attribute waits, outbound API / database / notification calls, response — each node carrying its input, output, duration and outcome; a tabbed detail panel (Input · Output · Writes or Lifecycle · Timing, plus a Where this ran column) opens on the selected node withOpen in Debug View(script entries, where the instance is known) andAudit rows (n)(→/audit/log?executionId=) drill-ins.- Two modes. Live is a real run — the method script executes on the active central node with tracing on, or a real attribute write goes through the routed-write path and the site's triggered run is polled back — audited like any other call. Replay executes nothing: it rebuilds the flow from an existing execution's
ScriptTraceaudit rows plus itsApiOutbound/DbOutbound/Notificationrows (?executionId=drill-in from the Audit Log drawer's and the Execution Chain page's Open in Flow Debugger actions). - External lane (v1.1, plan
docs/plans/2026-09-06-flow-debugger-outbound-calls.md). EveryExternalSystem.Call/CachedCall,Database.*andNotify.Senda traced run makes is a node on the fourth lane — kindsApiCall/DbCall/Notify(eyebrowsAPI CALL/DATABASE/NOTIFY), title = the system.method / connection / list, a status pill (Ok/Pending/Parked/Failed) plussync|cached|notify. One node per operation: a cached call's or notification's later lifecycle rows (Forwarded → Attempted → Delivered / Parked, written minutes later by the S&F forwarder or the Notification Outbox) fold into the node (FlowStep.Lifecycle, a Lifecycle tab in the detail panel with Delivered by) instead of drawing four boxes. A sync call sits in the execution chain; a cached call or notification hangs off it on a dashed async connector, and a trigger cascade's spawn edge carries a bolt badge on the connector's riser — the edge itself keeps an ordinary arrowhead (the bolt was the arrowhead until 2026-09-08, and on the upward Tags→Site hop it read as a broken arrow). The spawn edge starts at the parent's last write of the attribute the child'striggerAttributenames (FlowGraphBuilder); only when the child names no attribute, or the parent never wrote it, does the timestamp rule (last non-async parent step at or before the child) decide — a burst of writes landing in one millisecond used to draw the cascade from the last of them rather than the one that fired it. A Live run returns only the rows the run itself emitted, so a graph left withPendingnodes refreshes itself through the Replay command every 5 s for up to 2 min (page constants), and a Refresh outcomes button re-reads on demand in either mode; the state chip stays Completed — pending outbound work is not a failed run. A refresh merges the re-read rows into the run's known rows byEventId(fresh copy wins) and rebuilds the graph from the union, so it only ever adds nodes: the site's outbound rows reach central's audit store by telemetry seconds after the run returned them inline, and a Replay read taken in between would otherwise erase an already-drawn node (observed on the rig 2026-09-06 with a resolved cached DB write). The union is exact because the inline rows are the veryAuditEventobjects the site writer persists. v1.2 (follow-ups plandocs/plans/2026-09-06-flow-debugger-followups.md, register row 111 closed): a folded node carries itsCorrelationId(FlowStep.CorrelationId— theTrackedOperationId/NotificationId), so the detail panel deep-links to/site-calls/report?correlationId=<id>//notifications/report?correlationId=<id>(Open in Site Calls / Open in Notification report; both pages accept that query, pre-filter to the one row through an additive per-id query filter —SiteCallQueryRequest.TrackedOperationIdFilter/NotificationOutboxQueryRequest.NotificationIdFilter— show a removable Operation / Notification pill and auto-open the row's detail modal; noteNotificationIdis minted as a Guid inNform — no dashes — so the Notify link passesNand the report page normalises any Guid-shaped value toN, whileTrackedOperationIdis avarchar(36)Dstring and the Site Calls filter binds avarcharparameter for the PK seek), and aParkednode offers Retry / Discard in place: the same site-scope re-check, confirm dialog (Discard = danger) and central→site / Notification Outbox relay the report pages use (IFlowDebugService.RetryOutboundAsync/DiscardOutboundAsync→CommunicationService), then the page re-polls Replay (5 s / 2 min) until the node leavesParked. Prediction is async-safe:FlowDebugService.PredictMethodAsyncresolves route targets by a resolve-and-rewalk loop — its original blocking resolver deadlocked the whole Blazor circuit on the renderer sync context the moment a method routing to a site script was chosen (found 2026-09-06; pinned by a single-thread-pump unit test and the PlaywrightFlowDebuggerTests, which log in through the fixture and run a Live run of an ephemeral method in a real browser). - Step logs and exceptions (v1.4, 2026-09-09, design
docs/plans/2026-09-09-flow-debugger-step-logs-design.md). A traced run shows, on the node that produced them, the exception a failing step threw and every line the script (or the runtime on its behalf) logged. A log line is an ordinary audit row —ScriptTrace/ScriptLog, one per line — thatFlowGraphBuilderfolds into its owning step'sFlowStep.Logsand never draws as a node; attribution isExtra.stepEventIdwhen the emitting helper pre-minted its step id, else the execution's anchor row (ScriptRunon a site,InboundRequest/ the debugger's synthesizedRouteCallroot centrally), else that execution's first step, elseFlowGraph.UnattachedLogLines. Rendering: a node footer badge (✕ nerrors,! nwarnings,≡ ninfo, highest severity first,data-test="flow-node-logs", hidden with no lines); a failed node's subtitle becomesExceptionType: message(namespace stripped, innermost exception, generic-safe) fed by the newFlowStep.ErrorDetail; a Log (n) tab in the detail panel (flow-step-tab-log) carrying the full exception block first, then the lines under anAll | Warn +filter with a copy button and a "n more on <step>" cross-step hint that selects the named step; and aLog: n errors · n warnings · n infocount in the status strip (flow-log-counts) that counts LINES, not steps — aLog.Errorline is not a failed step and never moves the state chip. Two truncation notes sit beside it:flow-logs-truncated(inline log lines capped — refresh outcomes pulls the rest, fromFlowDebugRunResult.LogsTruncated; it clears once a refresh merges in more log rows, because Replay reads the complete set) andflow-unattached-logs. Live and Replay read the same rows, so the two modes render identically once telemetry has landed. - Authorization (arch-review F3). The page is
RequireFlowDebugger—AuthorizationPolicies.FlowDebuggerAccessRoles= Deployer, Administrator, Viewer — so an auditor can open it and Replay a past execution without the ability to run anything, andFlowDebugReplayCommandaccepts the same set. Everything that executes stays Deployer-only: the three run commands (FlowDebugRunMethodCommand/FlowDebugRunScriptCommand/FlowDebugRunValueChangeCommand), the Live entry-point controls, the parked-node Retry/Discard actions and the?methodId=/?instanceId=/?script=drill-ins; for a non-Deployer the page pins itself to Replay mode. - Two entry points (script entry redesigned 2026-09-08,
docs/plans/2026-09-08-flow-debugger-script-entry-design.md). An inbound API method (the sameParameterValueFormthe script pane uses, generated from the method'sParameterDefinitions— typed inputs, nested object groups, add/remove list item editors,$ref:lib:resolution, inline validation — plus a Form | Raw JSON toggle whose raw mode is a Monaco JSON editor for the whole request body, parsed and validated against the method's schema on every edit; Run is disabled while either surface is invalid, and a failed parse is never sent — Gitea #37,docs/plans/2026-09-10-flow-debugger-method-params-design.md; the pre-#37 pane was one text box per top-level field with page-side coercion, which could not take an array-of-object parameter at all) or a site script (Site → Instance → Script pickers, site-scope filtered): a script whose trigger is ValueChange or Conditional shows one New value for<attribute>field, with the attribute derived from the script's own trigger configuration rather than picked; every other script kind (Call, Interval, Expression, or no trigger) shows aParameterValueFormgenerated from the script'sParameterDefinitions, or a "this script takes no parameters" note when there are none. A method-entry Live run is additionally site-scoped at route time on the server (RouteHelper.WithSiteScope→SITE_SCOPE_DENIED, arch-review F2), so an operator cannot reach a site outside their scope by choosing a method that routes there. Both entry points pick the target instance through the sharedInstancePickercombobox (2026-09-08,docs/plans/2026-09-08-instance-picker-design.md): a text field whose dropdown is the site's area tree (InstanceTreeBuilder— all areas collapsed on open, un-areaed instances at the root, orphaned rows surface at the root), typed text filters by instance name and expands to the matches, Enter picks the first visible match, Escape restores the selection. - Predicted graph first. Choosing an entry point renders the statically predicted shape (
FlowShapePredictor, Script Analysis REQ-SA-7) before anything runs — nodes whose target depends on inputs render dashed; after a run, predicted nodes the run never produced render greyed. - Execution path.
IFlowDebugServicedispatchesFlowDebugRunMethodCommand/FlowDebugRunScriptCommand/FlowDebugRunValueChangeCommand/FlowDebugReplayCommandto the per-nodeManagementActorthroughManagementActorHolder(the same seam the HTTP/managementendpoint uses), so the server enforces the Deployer gate and the active-node refusal. A standby node answers the three run commands withSTANDBY_NODE, which the page renders as an inline alert with the server message verbatim; Replay is served from any node. The same four commands back the CLI'sflow-debug run-method | run-script | run-value-change | replay. The client wait is derived from the server's own bound, not a fixed duration (arch-review F4):FlowDebugService.RunAskTimeoutFor(timeoutSeconds)= the method'sTimeoutSeconds+ the server'sFlowDebugRunner.RunGrace(10 s) + 15 s client slack, floored at 30 s, andValueChangeAskTimeoutFor(waitSeconds)= the trigger-wait window + 15 s (Replay stays a flat 30 s); the CLI mirrors it withrun-method --timeout-seconds(default 120, set it to the method's timeout + 25 for long methods) and arun-value-changewait of--wait-seconds + 25. A long-running method used to be cut off client-side before the server's own grace expired. The script entry point (2026-09-08) reuses the same two timeout functions by kind: a call-kind script run waitsRunAskTimeoutFor(script.ExecutionTimeoutSeconds ?? 30), a write-kind script run waitsValueChangeAskTimeoutFor(45); the page's?instanceId=+?script=drill-in replaces the former?attribute=query parameter. - State machine
Idle → Predicted → Running → Completed | Failed | TimedOut; the status strip shows run number, start, total duration, step/boundary/write counts and the rootExecutionId(linking to the Execution Chain page). A run whose route call came back with no child execution and no Site-lane steps shows a site not traced note (older site build, or an untraced call). Last entry point and per-method inputs persist inlocalStorage. - The renderer is the custom-SVG
FlowGraphView(no third-party charting) — pure, unit-tested layout; culture-invariant coordinates; each node a keyboard-activatablerole="button". Node text is HTML inside an SVGforeignObjectso the browser measures it: the title wraps to two lines and ellipsizes by width (never by character count), the subtitle by width on one line, and the hover tooltip carries the full text. A lane whose real titles share a dotted head (Tags:DelmiaReceiver.RecipeDownloadFlag,DelmiaReceiver.RecipeProcessedFlag, …) shows that head once in the lane gutter (DelmiaReceiver.*) and each node shows only the tail that tells it apart (FlowGraphView.CommonDottedPrefix).
Alarm Summary (Deployment Role)
- A dedicated operator Alarm Summary page (
/monitoring/alarms,RequireDeployment) gives a cross-instance, read-only roll-up of live alarm state at a site — the operator-facing complement to the per-instance Debug View. - Data path — no new site-side code and no central alarm store. The page selects a site, queries its deployed instances, then fans out the existing per-instance
DebugViewSnapshotAsk concurrently (capped with aSemaphoreSlim) and aggregates the returnedAlarmStatesclient-side. The fan-out is partial-results tolerant: instances that time out are listed as "not reporting" while the rest still render. This snapshot fan-out now doubles as the seed for a near-real-time live feed (see Live updates below) rather than the sole refresh mechanism. - Live updates — the page is driven by a transient, per-site central live alarm cache (
ISiteAlarmLiveCache, owned by the Communication component; see Component-Communication). On site select the page subscribes to the cache; the cache runs one shared, reference-counted per-site aggregator that seeds from the snapshot fan-out and then stays warm on a single site-wide, alarm-onlySubscribeSitegRPC stream (seed-then-stream, dedup by(InstanceUniqueName, AlarmName, SourceReference)). Applied deltas raise an in-process change event (mirroringIDeploymentStatusNotifier) that the Blazor circuit pushes to the browser viaStateHasChanged()— no new SignalR hub.AlarmSummaryService.BuildFromLiveAlarmsrebuilds the roll-up + rows from the cache's current alarm set. The cache is purely in-memory on the active central node — there is still no persisted central alarm store; on a NodeA↔NodeB failover the new active node re-seeds from scratch. - View — roll-up tiles (total active, worst severity, unacked count, per-
AlarmKindcounts) plus a flat, sortable, filterable table. Filters cover instance,AlarmKind(Computed / NativeOpcUa / NativeMxAccess), state, acked/unacked, severity threshold, and name search. - Row virtualization (arch-review WP2.4). Above
VirtualizeThreshold(150) visible rows the table switches from a plainforeachto BlazorVirtualize(ItemSize=37,<tr>spacers), so a large-site alarm burst renders only the on-screen rows instead of every row in the DOM. Below the threshold the plainforeachstays — cheaper than the virtualization machinery and needs no JS interop. Both paths share one row-template, so the switch is invisible to styling/behavior. - Read-only — there are no ack / shelve / suppress controls (native alarms remain read-only by design).
- Refresh — manual refresh button plus the 15s poll timer (mirroring the Health dashboard), now retained as a fallback +
NotReportingauthority behind the live cache: when the cache reportsIsLive, the page renders live-cache state; when a stream is unhealthy, the aggregator has died (deathwatch resetsIsLive), or a site has not yet seeded, the poll keeps the page fresh so a stream failure never blanks it. Since WP2.3IsLivealso tracks the stream itself: a site-wide stream that faults or ends gracefully dropsIsLiveon the spot, so the page falls back to polling for the reopen window instead of rendering a snapshot that has quietly stopped updating. When live, the poll updates only theNotReportinglist and leaves the row set to the delta path, so a slow fan-out can never momentarily revert a fresher live delta (R2 N5). (Aggregated live stream delivered 2026-07-10 — seedocs/plans/2026-07-10-aggregated-live-alarm-stream-plan.md.)- Poll fan-out is deduplicated too (arch-review WP2.4). The cold-cache/fallback poll runs through
SharedAlarmSummaryService, a process-level memoizing façade overAlarmSummaryService: one memo slot per site, single-flight, just-under-15s window, so N operators watching the same site's fallback poll cost the node ONE per-instance debug-snapshot fan-out per window, not N. While the live cache is serving a site there is no poll fan-out at all — the façade instead answers straight fromISiteAlarmLiveCache(sameBuildFromLiveAlarmsCorepath the live subscription uses), so the aggregator's own seed/reconcile fan-out is the only one running.
- Poll fan-out is deduplicated too (arch-review WP2.4). The cold-cache/fallback poll runs through
- Reuse — the alarm badge/formatter markup is factored out of Debug View into a shared
AlarmStateBadgescomponent consumed by both Debug View and this page.
Parked Message Management (Deployment Role)
- Query sites for parked messages (external system calls, cached DB writes). (Parked notifications are managed centrally on the Notification Outbox page, not here.)
- View message details (target, payload, retry count, timestamps).
- Retry or discard individual parked messages.
Notification Outbox (Deployment Role)
- Monitor and manage centrally-delivered notifications. The Notification Outbox dispatches every notification store-and-forwarded from sites and logs each one to the central
Notificationstable. - KPI tiles at the top of the page: queue depth (
Pending+Retrying), stuck count, parked count, delivered in the last interval, and oldest pending age. The KPIs are central-computed on demand from theNotificationstable. - A queryable notification list filterable by status, type, source site, notification list, and time range, with a stuck-only toggle and keyword search on subject. Each row shows the notification's status, retry count, last error, and key timestamps.
- Retry and Discard actions are available on parked notifications: Retry returns the notification to
Pendingand resetsRetryCount/NextAttemptAt; Discard moves it toDiscarded. The row is retained either way so the table stays a complete audit record. - Each row exposes a "View audit history" action that opens the Audit Log page pre-filtered to
CorrelationId = NotificationId, surfacing every operational audit event recorded for that notification. - Stuck rows are visually badged — a notification is stuck if it is
PendingorRetryingand older than the configurable stuck-age threshold. Stuck detection is display-only; there is no automated escalation or alerting. - All queries are served from the central
Notificationstable — no remote per-site queries are needed, unlike the Parked Message Management page.
Secured Writes (Operator / Verifier / Administrator Roles)
- A Secured Writes page (
/operations/secured-writes) drives the two-person authorization workflow for writes through the MxAccess Gateway: an Operator initiates the write, a separate Verifier approves it, and only an approved write reaches the site. - The page itself is gated by
RequireSecuredWriteAccess— satisfied by any ofOperator/Verifier/Administrator. Although the pending/history lists are read-only, they expose process-sensitive tag values, so the page is not open to every authenticated user; this aligns with the ManagementActorListSecuredWritesCommandany-of gate (arch-review UA1). The submit and approve/reject sub-actions are further gated byRequireOperator/RequireVerifierrespectively (below). - Operator (submit) — a submit form gated by
RequireOperator: pick the site → an MxGateway connection on that site → the tag path → a typed value → an optional comment. Submission inserts aPendingPendingSecuredWriterow centrally; it does not write anything yet. - Verifier (approve / reject) — a pending queue gated by
RequireVerifierwith Approve / Reject (+comment) actions. Approve shows a confirmation of the exact site / connection / tag / value before firing, and surfaces the submission age (Submitted <age> ago (<UTC timestamp>)) so a verifier can catch a stale setpoint before it reaches the device. The two confirmations are deliberately styled apart: Approve is amber (DialogSeverity.Caution) and Reject stays red — see the confirm-severity contract under Technology for why (register row 75). The verifier's own submissions are disabled in the UI and rejected server-side (no self-approval). On approve, central marks the rowApprovedand relays the write to the site MxGateway (recordsExecuted/Failed); reject moves it toRejectedwith a reason. The pending queue is unpaged — it is bounded by the submission TTL: aPendingrow that is neither approved nor rejected before the TTL elapses expires to the terminalExpiredstatus and drops out of the queue into History. - History — terminal rows (Executed / Failed / Rejected / Expired, where
Expiredis a real TTL-driven terminal status) with the full who/when/outcome trail (operator, verifier, comments, timestamps, any execution error). Terminal rows accumulate without bound, so the History table is offset-paged (page-size 50, Prev/Next) driven by the server's unpagedTotalCount(ListSecuredWritesCommandSkip/Take); the pager buttons disable at the real bounds. - Every lifecycle event (submit / approve / reject / execute) is written to the central Audit Log; the rows share the
PendingSecuredWrite.IdasCorrelationIdso they join into one operation (see Component-ManagementService.md, Component-AuditLog.md). - Dev caveat: with
DisableLoginon, the auto-login identity holds all roles, so the two-person flow cannot be exercised end-to-end by a single user via the dev UI — no-self-approval is covered by handler tests; real two-person use requires two real identities.
Site Calls (Deployment Role)
- Monitor cached calls store-and-forwarded from sites —
ExternalSystem.CachedCall()andDatabase.CachedWrite()operations. Scoped to theExternalCallandDatabaseWritekinds only; notifications keep their separate Notification Outbox page and are not merged here. - A queryable cached-call list filterable by site, kind, status, and time range. Each row shows the call's timestamp, site, kind, target summary, status badge, retry count, and last error.
- Retry and Discard actions are available on
Parkedrows only —Failedrows are not actionable, since a permanent failure would simply fail again and its error was already returned synchronously to the calling script. The actions issue central→site commands to the owning site; if the site is offline the UI surfaces a "site unreachable" message. - Each row exposes a "View audit history" action that opens the Audit Log page pre-filtered to
CorrelationId = TrackedOperationId, showing every operational audit event recorded for that cached call. - Data is served from the central Site Call Audit component's
SiteCallstable. The page is read-mostly — an eventually-consistent mirror of site state; the site remains the source of truth.
Health Monitoring Dashboard (All Roles)
- Overview of all sites with online/offline status.
- Per-site detail: active/standby node status, data connection health, script error rates, alarm evaluation error rates, store-and-forward buffer depths.
- Headline Notification Outbox KPI tiles — queue depth, stuck count, and parked count. These are central-computed by the Notification Outbox from the central
Notificationstable (not part of any site health report). The full outbox view is on the dedicated Notification Outbox page. - Headline Site Call Audit KPI tiles — buffered count, parked count, and failed-last-interval. These are central-computed by the Site Call Audit component from the central
SiteCallstable (not part of any site health report). The full cached-call view is on the dedicated Site Calls page. - Headline Audit KPI tiles — three tiles in a new "Audit" KPI group: Audit volume, Audit error rate, and Audit backlog. These are sourced from the Audit Log component (#23) and Health Monitoring per the metric definitions in Component-HealthMonitoring.md; the dashboard simply surfaces them. The full audit query view is on the dedicated Audit Log page.
- Trigger failover (
CentralFailoverControl, Administrator only) — on the central card and on each site card. Disabled with an explanatory tooltip when there is no standby to take over. It always confirms first, and the confirmation's red button reads Fail over (see the dialog-label contract under Technology); the central variant's copy states that this page is served by the node being restarted and will disconnect and reconnect, while the site variant deliberately does not, since a site is a different cluster. - Shared KPI cache (arch-review WP2.4). All four KPI families above — Notification Outbox, Site Call Audit, Audit, and their per-site/per-node breakdowns — are read through a process-level
IKpiSnapshotCache, not queried per page load. Each accessor is independently memoized (8s TTL) with single-flight production, so N Blazor circuits polling the same KPI inside one window (e.g. ten operators with the Health dashboard open) cost the node ONE aggregate SQL round trip per KPI per window, not N. A failed query is never memoized — the next caller re-attempts it, and the calling page's existing per-tile "unavailable" degradation is unchanged. Every KPI-tile page (Health, Notification Outbox, Site Calls, Audit Log) shares the same cache instance.
Site Event Log Viewer (Deployment Role)
- Query site event logs remotely.
- Filter by event type, time range, instance.
- View script executions, alarm events (activations, clears, evaluation errors), deployment events (including script compilation results), connection status changes, store-and-forward activity, instance lifecycle events (enable, disable, delete).
Audit Log (Admin / Audit Role)
- Lives under a new top-level "Audit" nav group (sibling to Notifications). In v1 the Audit nav group contains this single Audit Log page; the pre-existing Configuration Audit Log Viewer remains its own page below.
- Global query / filter / drilldown over the central
AuditLogtable maintained by the Audit Log component (#23). Read-only — the table is append-only, so there are no edit actions on rows. - Read access to the page requires the
OperationalAuditpermission (Security & Auth #10). Per-site row scoping reuses the existing site-permission model: a user sees only rows for sites they are authorized to operate. Bulk export (see below) additionally requiresAuditExport. The split mirrors the CLI's permission model (see Component-CLI.md). - Filter bar (top of page, collapses to a single row when not focused):
- Time range — relative (15m / 1h / 24h / 7d) or custom.
- Channel — multi-select:
ApiOutbound,DbOutbound,Notification,ApiInbound. - Kind — multi-select; the available options are filtered by the selected Channels.
- Status — multi-select.
- Site — multi-select, scoped to the user's authorized sites.
- Instance / Script — text search with autocomplete.
- Target — text search (system + method, DB connection, list name).
- Actor — text search (inbound API key name).
- CorrelationId — paste a
TrackedOperationId/NotificationId/ request-id to see the full event sequence for one operation. - "Errors only" toggle — shorthand for
Status NOT IN (Success, Delivered, Enqueued).
- Results grid (custom Blazor + Bootstrap component, consistent with the rest of the UI — no third-party grid):
- Columns, all resizable and reorderable, persisted per user:
OccurredAtUtc,Site,Channel,Kind,Status,Target,Actor,DurationMs,HttpStatus,ErrorMessage. - Keyset pagination ordered by
(OccurredAtUtc desc, EventId desc). Default page size 100. - Clicking a row opens the drilldown drawer.
- Columns, all resizable and reorderable, persisted per user:
- Drilldown drawer:
- Pretty-prints
RequestSummary/ResponseSummary— JSON is auto-detected and syntax-highlighted; SQL is syntax-highlighted. - Surfaces redaction indicators wherever headers or fields were stripped at write time, per the Audit Log component's "Payload Capture Policy".
- "Copy as cURL" action on
ApiOutboundandApiInboundrows. - "Show all events for this operation" link — re-applies the current view filtered by the row's
CorrelationId.
- Pretty-prints
- Export button on the page header streams a server-side CSV of the current filter (default cap 100k rows; larger exports go through the CLI). Requires the
AuditExportpermission.
Configuration Audit Log Viewer (Admin Role)
- Pre-existing viewer for the
IAuditServiceconfiguration-change log (template / instance / site / etc. before-after edits). Lives under the same Audit nav group as the operational Audit Log above. - Query the central configuration audit log.
- Filter by user, entity type, action type, time range.
- View before/after state for each change.
LDAP Group Mapping (Admin Role)
- Map LDAP groups to system roles (Admin, Design, Deployment).
- Configure site-scoping for Deployment role groups.
Dependencies
- Template Engine: Provides template and instance data models, flattening, diff calculation, and validation.
- Deployment Manager: Triggers deployments, system-wide artifact deployments, and instance lifecycle commands. Provides deployment status.
- Communication Layer: Routes debug view subscriptions, remote queries to sites.
- Security & Auth: Authenticates users and enforces role-based access.
- Configuration Database: All central data, including audit log data for the audit log viewer. Accessed via
ICentralUiRepository. - Health Monitoring: Provides site health data for the dashboard.
- Notification Outbox: Provides notification delivery KPIs and serves the
Notificationstable queries and Retry/Discard actions for the Notification Outbox page. - Site Call Audit: Serves the
SiteCallstable queries and relays Retry/Discard actions to sites for the Site Calls page. - Audit Log (#23): Serves all
AuditLogtable queries (filter / grid / drilldown / CSV export) for the new Audit Log page and the drill-in surfaces on Notifications, Site Calls, External Systems, Inbound API keys, Sites, and Instances. Payload capture, redaction, and per-site authorization follow the Audit Log component's "Payload Capture Policy" and "Security & Tamper-Evidence" sections. - KPI History (#26): The Central UI hosts the
KpiHistoryQueryService(scoped-repository read overIKpiHistoryRepository.GetRawSeriesAsync+KpiSeriesBucketer, dual-ctor test seam) and renders the reusable custom-SVGKpiTrendChartfed by it. Trend sections appear on the Notification Outbox, Site Calls, and Audit Log pages and in a per-site panel on the Health Monitoring dashboard; a query failure degrades to an unavailable-chart placeholder rather than breaking the page. See Component-KpiHistory.md.
Configuration mutations and live circuit authority (2026-09-09)
Page configuration handlers use ConfigurationCommandService and the guarded management command boundary. Database changes and their scrubbed audit entries commit in one transaction, including generated entity IDs. Existing instance service operations execute through the same transaction seam and retain their domain validation and audit ownership. Circuit read tracking is cleared after a separate transaction scope changes configuration.
Cookie circuits retain their initial principal without retaining HttpContext; authoritative reads re-map stored directory groups against current grants. Mutation handlers recheck role and persisted target site immediately before dispatch. Passive session pings do not count as user activity. Trusted browser input posts an antiforgery-protected activity signal; the server activity tracker also makes it visible to the existing circuit.
The explicit server-configured development auto-login mode preserves its synthetic all-role principal. It uses the Cookies scheme but has no LDAP group claims; it must not enter directory re-mapping. This exception follows validated AuthDisableLoginOptions, never a caller-supplied claim.
Isolated Test Run workers
Test Run executes in a disposable CentralUI.SandboxWorker process. A process-wide admission gate permits at most four workers. Queueing, compilation, execution and gateway calls consume the requested deadline (1–10 seconds); cancellation kills the process tree and capacity is released only after confirmed exit. Console capture retains at most 32,000 characters with an explicit truncation marker; return JSON is limited to 32,000 bytes. Private IPC frames are limited to 1 MiB and stderr is drained without accumulation. Each worker has a 512 MiB managed-heap ceiling. The worker is a resource/lifetime boundary; the semantic script trust policy still gates compilation.
External calls, shared-script lookup and bound-instance operations use a parent broker. Configured SQL connections are recreated inside the worker using credentials passed only over its private pipe, preserving the existing SQL permissions and API. Termination prevents further calls and closes worker-owned sockets; it cannot undo SQL or remote work already accepted by another server. Timeout results do not retain a terminated worker's partial console buffer.
The deadline terminates the child independently of gateway cancellation. If a broker provider ignores cancellation, the response waits for that operation to finish and its admission slot remains charged; this preserves the owning request scope and prevents unbounded abandoned operations.