Second slice of the 2026-09-09 whole-system review remediation
(docs/plans/2026-09-09-arch-review.md §10), on top of the High slice 146a1323.
Runtime/devices: SR-2 alarm on-trigger integration scope; SR-3 correlated NativeAlarmActor
subscribe retry; SR-4 immutable audited DB target; SR-5 ordered InstanceActor stop/flush;
SR-12 rehydration fence; SR-13 S&F gate recheck per delivery; DC-4 no OPC UA security
downgrade; DC-5 alarm snapshot for every new subscriber; ORCH-1 sync-dispose joins the pump.
Management/UI: MG-2 site scope on audit tree + Flow Debugger Replay; MG-3 stale-probe memo
removed; MG-4 server-owned bulk-deploy deadline; UI-5 sandbox runs in a killable child
process (new CentralUI.SandboxWorker project); UI-14 bounded sandbox output/IPC.
Edge/audit: EI-1 relay audit ownership; EI-2 SkipBodyCapture suppresses error text; EI-3/EI-11
error-field redaction + caps; EI-4 single relay deadline, pre-execution-only retry; EI-5
authenticated relay provenance; EI-10 body cap before audit read.
Persistence/contracts: DL-1 global Instance.UniqueName unique index + conflict translation;
DL-2 DataConnection protocol config encrypted at rest (migration
20260909203046_EnforceGlobalInstanceNamesAndProtectProtocolConfiguration — coordinated
central upgrade, old binaries cannot read protected rows); DL-3 bounded Replay paging;
DL-6 cross-site Overwrite of a deployed instance rejected; XC-4 proto descriptor contract test.
Config/hygiene: XC-1 central listener isolation by local port; XC-2/XC-3 inert SMTP + cron
keys removed from rig settings; CM-2 env2 bootstrap guard enabled; DC-6 deploy site exit code;
DC-7 notifier retries only explicit standby rejection; XC-5 ClusterClient purged from docs.
Gates recorded in the report: build 0/0, 9 397 passed / 0 failed, rig redeployed on all
eight nodes with the migration applied. 46 Low findings remain open.
The trigger-cascade (Spawn) edge used a lightning-bolt marker IN PLACE of the arrowhead. On the upward
Tags→Site hop the bolt sat on top of the riser and read as a broken arrow (rig screenshot 2026-09-08).
Every connector now ends in an ordinary arrowhead; the bolt is a badge (circle + bolt) centred on the
connector's vertical run (FlowGraphView.SpawnBadgePoint, kept in step with OrthogonalPath/AsyncPath).
Same screenshot, second defect: the cascade spawned by "RecipeDownloadFlag = false" was drawn from the
LAST write of the burst (RecipeProcessedFlag) because all four writes share a millisecond and the
spawner rule was time-only. FlowGraphBuilder now prefers the parent's last AttributeWrite of the attribute
the child's triggerAttribute names (at or before the child, else the last such write); the time/position
rules remain the fallback when the child names no attribute or the parent never wrote it.
Verified live on the rebuilt rig (DelmiaSim01 / DelmiaReceiverSimulator): both spawn edges show an
arrowhead + badge, the second one originates from the RecipeDownloadFlag write. bUnit + builder tests
updated/added; Playwright FlowDebuggerTests 4/4.
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
Code review of the merged arch-review remediation (Tasks 1-13) found four items.
1. FlowDebugRunner returned its OWN synthesized rows un-redacted. RunMethodAsync's
inbound row and RunValueChangeAsync's root row were handed to WriteCentralAsync
(whose writer redacts a private copy) and then put RAW into the returned Events —
so a Live run's caller saw the unfiltered row while Replay of the same execution
showed the redacted one, defeating F1's "Live shows what Replay reads back". Both
methods now resolve IAuditRedactor once per run and place the filtered copy into
the result (including the early-return path on a failed attribute write); the raw
row still reaches the writer, which redacts itself.
2. FlowGraphBuilder's spawner search degenerated to "the last step of the parent".
Since executions are ordered as contiguous blocks, position[p] < childPosition
holds for EVERY parent step, so a run shaped ScriptRun -> Write(A) -> Wait ->
Write(B) whose Write(A) fired the cascade drew the Spawn edge from Write(B). New
rule, in order: (a) the last non-async, non-Response parent step occurring at or
before the child's first step — parent and child of a trigger cascade share the
site clock, so this picks the write that actually fired it; (b) the position rule
as fallback, which is the mixed-clock case F5 was written for (a child clock
behind its parent's, where no parent step satisfies (a)); (c) parentChain[0].
The class remark's stale "Replay order is unchanged" claim is replaced with what
actually happens to a central Database.* call made after Route.To().Call.
3. FlowTraceCollector's byte budget counts UTF-16 chars of DetailsJson only, so
"well under the 4 MB gRPC default" overstated it. Documented honestly: ~1 MB of
UTF-8 for ASCII payloads, ~3 MB worst case for non-ASCII, both under 4 MB before
framing — and raise it only against a real serialized-size measurement.
4. CLI --timeout-seconds help + README row now say the management endpoint caps
flow-debug run-method at ManagementService:LongRunningCommandTimeout (default
5 min), so a larger client value still 504s at the server's ceiling.
Deferred-work register rows 112 (site-side inline trace truncation is invisible to
the operator — needs a proto field) and 113 (the F2 method-entry site scope gates
routed traffic only; ApiMethods have no site affinity to scope against) record what
this review deliberately did not fix.
Tests: FlowGraphBuilderTests gains the same-clock Write(A)/Wait/Write(B) cascade
(red before the rule change, and the clock-behind test still passes on rule (b));
FlowDebugRunnerTests extends the redactor test to the inbound row and adds two
value-change cases (settled poll, failed write). Commons 793, ManagementService 615,
CentralUI 1264, CLI 462 — all green.
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
Central rows carry central's clock, site rows the site's, so the global
OccurredAtUtc sort let a 1-2 s skew float a whole site execution ahead of the
inbound row / RouteCall that started it, and pushed the trigger-cascade spawner
search onto its parentChain[0] fallback.
FlowGraphBuilder.Build now orders executions by the input index of their
earliest-given row and orders steps by timestamp only INSIDE one execution
(one clock), breaking ties by kind rank then input index. A folded outbound
node keeps its earliest-by-time row's identity, which need not be the row given
first, so its index is the minimum over the rows folded into it. The spawner
search uses position in the ordered list rather than a raw timestamp compare.
FlowDebugRunner.RunMethodAsync keeps the collector's emission order instead of
re-sorting; the value-change path keeps its OrderBy (all rows share one site
clock). Replay input arrives timestamp-sorted from the repository, so its order
is unchanged.
The DelmiaShaped test fixture is still shuffled within each execution (the
inside-one-execution timestamp rule is unchanged) but now lists the executions
in emission order, since that is what fixes their relative order.
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
Every emit site hands FlowTraceCollector the same AuditEvent it hands the
writer, but redaction and payload capping lived inside the writers, which
return a NEW filtered record — so the inline copy that rides
ScriptCallResult.TraceEvents / RouteToCallResponse.TraceEvents was the RAW
event: no per-target SQL-parameter redaction, no header scrubbing, no 8 KB /
64 KB truncation. Live and Replay of the same execution could therefore show
different payloads, and the 500-event-only cap left the reply free to breach
the 4 MB gRPC receive limit — which would fail a real call, since the same
reply carries the script's return value.
The collector now applies the host IAuditRedactor itself (outside the lock;
the writers keep their own idempotent Apply) and enforces a byte budget
(DefaultMaxBytes = 1 MB over the redacted DetailsJson) alongside the event
cap, flagging Truncated when either trips. It moves to Commons
(Types/Debugging) so the central FlowDebugRunner can share it.
The redactor is a singleton, so ScriptRunLauncher resolves it from the ROOT
provider and keeps constructing the collector before the try — the traced
ScriptRun row is emitted from the catch blocks too, including the
shed-at-dequeue path that throws before the DI scope exists.
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
Review of Task 2 (2026-09-06 outbound-calls plan): (1) an outbound step that
precedes the first chain step of its execution — the default Replay shape when
a run's ScriptTrace rows are absent — was orphaned or given a solid spawner
edge; leading async steps now hang off the spawner on Async edges and the
first non-async step gets the normal link. (2) IsAsync keyed on the first row
being CachedSubmit; it now mirrors the projector's sync rule (first row not
ApiCall/DbWrite) so a group missing its submit row still branches. (3) Same-
tick rows tie-broke by GUID; new LifecycleRank (submit < attempt < resolve,
non-terminal < terminal) makes first/latest attribution order-independent.
(4) DbSubtitle used GetInt32 (FormatException on out-of-range/fractional) —
now TryGetInt32, never throws. Also: External-lane edges never count as
central↔site crossings; DurationMs clamped; FlowStep doc updated.
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
Fold rule: outbound rows sharing (kind, CorrelationId) become one FlowStep with
a Lifecycle list — first row's identity/time/input, latest row's outcome/
subtitle/output/error, DurationMs = sync row's own or submit→terminal wall time
(null while pending). Chain skips async steps (Notify, or first row
CachedSubmit) and links them with Async edges from the preceding chain step.
Spawner edges from a non-RouteCall are Spawn: true. The Response links from the
last non-async step of the inbound execution (the RouteCall), not the globally
last cascaded step (D9). FlowGraphView places a Response at max column + 1 so
the D9 origin does not pull it left (pulled forward from Task 7 to keep the
layout test green).
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
ResolveKind maps ApiOutbound/DbOutbound/Notification to ApiCall/DbCall/Notify;
ResolveLane puts the three on FlowLane.External; outbound outcomes map
Delivered→Ok, Failed→Failed, Parked/Discarded→Parked, else Pending; subtitles
'{status} · sync|cached', '{op} · n rows' from a DbWrite Extra, '{status}' for
Notify. New FlowStepProjector.IsOutbound helper.
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
Flow Debugger (design 2026-09-05 §5.1): trace steps are audit rows. New
AuditChannel.ScriptTrace appended after Cluster and four kinds appended after
ManualFailover so stored ordinals never move; lock-in tests + alog.md §4 updated.
Claude-Session: https://claude.ai/code/session_01GSLkkHKz9JrAtmJjmQK3cP
Register row 98 — the site-scope authorization gate was FAIL-OPEN on an
unresolvable target. The resolve-then-check shape was written as
`if (target != null) EnforceSiteScope(...)`, so a null resolve skipped the
check entirely and a site-scoped caller passed the gate for any id in the
system. Authorization now fails CLOSED: a target that cannot be PROVEN to sit
inside the caller's scope is treated as outside it.
The defect turned out to have THREE more instances than the register recorded.
Besides `EnforceSiteScopeForInstance` and `EnforceSiteScopeForIdentifier`, the
same inline shape sat in `HandleGetInstance`, `HandleGetSite` and
`HandleGetDataConnection` — read handlers that resolve their own target and so
never call the helpers. Those were the sharper half of the bug: they answered
success-with-null for a nonexistent id and Unauthorized for an out-of-scope
one, making the pair an id-enumeration oracle over the whole fleet. All five
now route through one shared `DenyUnresolvedTargetIfScoped` arm.
Two properties are load-bearing and pinned by tests:
- System-wide callers (Administrator, or a Deployer with no site grants) are
untouched — they return at the guards above the lookup and keep the
handler's ordinary "not found" / null answer.
- The denial is indistinguishable from an out-of-scope denial: same
ManagementUnauthorized shape, and a message that never states whether the
target exists.
Register row 99 — CLI `audit query --status delivered` failed client-side while
the server would have accepted it: `AcceptOnlyFromAmong` matches ordinally
against the capitalised literals. Same treatment row 88 gave `deploy status` —
the client-side gate is removed, the valid set moves into the option help text,
and the server becomes the single authoritative check.
That required ADDING the server check, not just relying on one: the shared
audit filter parser was LAX, silently dropping an unrecognised value and
leaving the dimension unconstrained — so removing the CLI gate alone would have
turned a loud client-side failure into a plausible, complete-looking page that
did not answer the question asked. New `ParseEnumListStrict` (case-insensitive,
rejects ordinals — `Enum.TryParse` accepts "3", and a filter is a name-based
contract) backs the ManagementService /api/audit/query + /api/audit/export
endpoints, which now return 400 BAD_REQUEST naming the valid set.
Other AcceptOnlyFromAmong sites surveyed in the audit group: `--channel`,
`--kind`, `--status` on BOTH `query` and `export` (6 uses) plus `export
--format` (rejected `--format CSV` though the server lowercases it) — all seven
fixed here. Deliberately left alone: `template folder reorder --direction`
(pure client-side literal, no server-side parse to defer to) and the global
`--format json|table` (CLI-local output selection, never sent to a server).
The CentralUI AuditLogPage drill-in parser and the CentralUI export endpoint it
feeds deliberately KEEP the lax parse — they read a query string a human may
have bookmarked, where a stale fragment must still render the page. Both
contracts now live side by side in AuditQueryParamParsers and are documented as
non-reconcilable.
Docs: Component-ManagementService.md gains a "Site-scope gate: fail-closed on
an unresolvable target" section and a strict-filter note on the audit
endpoints; Component-Security.md records the fail-closed shape under Deployer
site scoping; the CLI README marks the audit filter values case-insensitive and
states the no-silent-drop / no-ordinals contract.
Tests: 5 new ManagementActorTests fail before the row-98 fix (verified by
reverting ManagementActor.cs) and cover scoped+nonexistent -> denied,
system-wide+nonexistent -> unchanged (both Administrator and no-grant Deployer),
scoped+in-scope -> allowed, and the indistinguishability property. 21 CLI tests
fail before the row-99 fix. Four pre-existing tests that pinned the removed
client-side gate were rewritten to pin the new contract.
Deferred-work register rows 37, 86, 87, 88 — the structural duplication and the
three behavioral divergences it had accumulated, fixed together.
Row 37 (structural). `IDeploymentManagerRepository.QueryDeploymentSummariesAsync`
is DELETED. It and `QueryDeploymentListPageAsync` were two paged reads of the same
insert-only table with different contracts — raw offset vs page number, no total vs
total, no instance join vs joined, instance-id-list scoping vs site-id scoping. The
CLI/HTTP `QueryDeployments` handler now goes through `QueryDeploymentListPageAsync`
with a `DeploymentListFilter`, exactly like the Central UI page. The
`DeploymentRecordSummary` projection went with it (`DeploymentListRow` is the single
row shape); its `MaxPageSize` const moved to `DeploymentListPage`, which also gains
`DefaultPageSize`.
Row 86 (Medium, authorization — the headline). `SiteScopeService.ResolveAsync` was
claims-only, so an Administrator carrying `SiteId` claims saw the whole fleet through
the CLI (`ManagementActor.EnforceSiteScope` has always exempted Administrator) and
only their granted sites on the Deployments page. One user, one dataset, two answers.
Administrator = system-wide is the intended semantic on both sides; the UI was the
defect. The role claim is read explicitly with an OrdinalIgnoreCase comparison rather
than via `IsInRole`, which compares claim VALUES ordinally and only resolves when the
identity carries the matching roleType — the two sides must not disagree on a
differently-cased role name. An Administrator's site claims are not collected at all,
so the invariant "system-wide => empty permitted-id set" holds for every caller.
Designer is NOT exempt: the class doc claimed "any Admin/Design user" is system-wide,
which was true of neither side, so the doc was corrected rather than the bypass
widened.
Row 87 (Low). The scoped branch used to resolve the site grant into an instance-id
list by loading EVERY instance in the system, non-transactionally, on every scoped
query; the grant now rides as `SiteIdScope` and is resolved by the query's own join.
An unparseable permitted-site id is DROPPED, never ignored — dropping can only narrow
a grant, discarding it would widen it to fleet-wide. And `PageSize <= 0` no longer
means UNLIMITED down at the repository (`--page-size 0` materialized the whole
retention window): it means "unspecified" and takes the CLI's own default of 50,
capped at 1000. Bounding without telling would be silent truncation, so the handler
now returns a `DeploymentQueryResult` envelope carrying `totalCount` plus the page
coordinates actually applied — with a CLI table projection (mirroring row 57's
`AuditConfigTableProjection`) that unwraps it and appends a "showing N of M" note.
Row 88 (Low, UX). An unrecognised `--status` fell back to "no filter", so a typo like
`--status Sucess` returned every status — a complete-looking answer to a question the
caller never asked. It is now a `ManagementError` naming the valid values. Parsing
stays case-insensitive; a numeric string is rejected too, since `Enum.TryParse`
accepts "7" and yields an undefined value (guarded with `Enum.IsDefined`). Loud
failure over silent full results, per row 46.
Tests: 7 new `SiteScopeServiceTests` pinning Administrator = system-wide (verified
failing 6/7 against the pre-fix service, the Designer pin passing both ways); the
ManagementActor deployment suite migrated onto the consolidated query with new pins
for the status error, case-insensitive parse, numeric rejection, page-size default/
cap/floor, empty-scope-on-garbage-grant, and "reads no instances at all"; a repository
test for site-scope+status composing over a bounded page; and 6 CLI tests for the new
table projection.
LocalDbOplogBacklog was reported and Prometheus-exported with nothing consuming
it: not sampled into KPI history, so a purge-burst spike could not be reviewed
after the fact, and no threshold treatment anywhere. Both halves shipped.
KPI SAMPLING. SiteHealthKpiSampleSource now emits localDbOplogBacklog and
localDbReplicationConnected (0/1), and the backlog is charted on the Health
dashboard's per-site trend panel. Charting is the point: a point-in-time tile
can never answer 'did that spike drain?', which is the only thing separating an
expected purge burst from a stalled peer.
Both are CONDITIONALLY emitted rather than coerced with ?? 0 like the
auditBacklogPending metric beside them. Both health-report fields are nullable
and null means 'no reading' — replication not configured (rig site-b/site-c) or
a failed poll — never zero. A fabricated 0 would render a pair that cannot read
its own oplog, or one deliberately unreplicated, as perfectly healthy once a
minute forever, burying the real series. So a site emits 12 metrics when
unreplicated and 14 when replicated, and a GAP in these two series truthfully
means 'not replicated / not reporting'. Note false and null differ for the
connected flag: a configured link that is DOWN samples as 0; no link samples as
nothing. Both fold as Gauge (summing an oplog depth, or a 0/1 flag, per hour is
meaningless).
THRESHOLD SEMANTICS — sustained, not instantaneous. This is the design, not a
refinement. The site_events retention purge is CDC-captured like any other
write, so the backlog jumps by the deleted batch and drains as the peer acks
it; a bare 'backlog > threshold' badge would fire on every replicated site
every day and be ignored within a week. Healthy is defined by the backlog
RETURNING to ~0, which only a dwell can express. CentralHealthAggregator holds
LocalDbBacklogAboveSince — the instant the backlog first rose above
LocalDbBacklogThreshold (default 1000) — and the badge appears only after
LocalDbBacklogSustainedFor (default 15 min). A report at or below the threshold
clears the anchor; one above leaves the ORIGINAL instant intact, so the window
is sustained rather than sliding.
The null case is deliberately a third branch: a null backlog leaves the anchor
UNCHANGED rather than clearing it. Null is evidence neither of a drain nor of a
backlog, so a node going blind mid-climb must not silently reset its own dwell,
and an unreplicated site (null forever) never accumulates an anchor and can
never be badged.
Validation rejects a non-positive threshold (0 would badge every replicated site
permanently, since an idle backlog already sits at it) and a dwell shorter than
one ReportInterval (satisfiable by a single sample, which collapses the check
back into the bare threshold it replaces).
DOC CORRECTION worth flagging: this row's premise was 'mirror the SiteAuditBacklog
configurable threshold badge'. That badge does not exist — Component-HealthMonitoring.md
described it in two places but no option, no comparison and no markup were ever
implemented. The two stale claims are corrected here rather than left to send the
next reader looking for it; the LocalDb badge is built fresh on the Metrics-stale
badge's pattern, which is a real configurable-threshold badge.
Tests: HealthMonitoring 115 (9 new aggregator tests incl. purge-burst-drains,
unreplicated-never-anchors, null-mid-climb, exact-threshold, stale-report; 5 new
validator tests; 4 new sampler tests incl. the null-semantics and false-vs-null
cases); CentralUI 1042 (5 new badge/chart tests incl. the purge-burst negative
control); Commons Kpi 93; KpiHistory 54.
ManagementService references Commons but not Transport, so MapFault could not
see BundleLockedException and the per-bundle 3-strike lockout reached the CLI
as "An internal error occurred (CorrelationId=...)" — the operator got a
correlation id instead of the reason, and no hint that further attempts on
that bundle would be refused.
Applies the pattern register row 44 established for BundleUnlockThrottledException:
- Relocate BundleLockedException from Transport/Import/ to
Commons/Types/Transport/, beside BundleUnlockThrottledException. Both
consumers (BundleImporter, CentralUI TransportImport) already carried the
Commons.Types.Transport using, so no call-site edits were needed.
- Add curated catch arms in ManagementActor.HandlePreviewBundle and
HandleImportBundle that rethrow as ManagementCommandException, so the
message survives MapFault's disclosure filter.
- Mirror the row-44 throttle fault-mapping test for both handlers.
- Document the relocation in docs/components/Transport.md and
docs/requirements/Component-Transport.md.
Tests: ManagementService 499 passed (+2 new), Transport 167 passed.
BundleUnlockRateLimiter was registered (ServiceCollectionExtensions), injected
into BundleImporter, unit-tested, documented on
TransportOptions.MaxUnlockAttemptsPerIpPerHour = 10 and pinned in
appsettings.Central.json — but _unlockRateLimiter was never invoked. A
brute-force passphrase attacker against a stolen bundle was bounded only by the
per-bundle 3-strike lockout, which a re-export side-steps (new ContentHash).
LoadAsync now registers every unlock attempt against the limiter. Ordering is
load-bearing: AFTER the per-bundle lockout check (hammering a locked bundle
spends no hourly budget) and BEFORE the decrypt (so a denial burns no per-bundle
strike — otherwise the throttle would be a way to lock a bundle someone is
legitimately importing). Over budget throws the new
BundleUnlockThrottledException, fail-closed with a clear message on both
surfaces; it lives in Commons because the management surface (Commons-only
references) must catch it to return a curated error rather than MapFault's
generic internal-error text.
Client key plumbing:
- Central UI: new scoped CircuitClientAddressProvider snapshots
Connection.RemoteIpAddress at circuit establishment — the same constraint and
the same pattern as CookieAuthenticationStateProvider, since HttpContext is
null by the time a circuit submits a passphrase.
- CLI/management: additive ManagementEnvelope.ClientIp, stamped by
ManagementEndpoints and threaded through DispatchCommand to the preview/import
bundle handlers.
- No attributable address => a single SHARED "<unattributed>" bucket, not a
bypass: unknown clients get a stricter budget, never none. Per-client
isolation is only real behind a ForwardedHeaders-trusted proxy, the same
caveat the login throttle carries (Component-Security.md).
Docs: Component-Transport.md + docs/components/Transport.md record the two-level
ordering, the client-key sourcing, the shared-bucket fallback and a
troubleshooting entry for the new error.
Tests: 6 importer tests (cap enforced; denial != passphrase failure, no strike
burned; per-client isolation; shared unattributed bucket; unencrypted bundles
consume no budget; a locked bundle spends no hourly budget) — 4 of the 6 fail
with the limiter call disabled, i.e. against the dead-code state. Plus
ManagementActor tests pinning the ClientIp forwarding and the curated
throttle error.
The Deployment Status page client-materialized the whole deployment list. It read
EVERY DeploymentRecord — an insert-only table, one row per deploy attempt for the
retention window — plus EVERY Instance, then site-scoped, sorted, counted the four
status tiles and sliced a 25-row page in the Blazor circuit's memory. That ran on
first render AND on every IDeploymentStatusNotifier push, so the cost scaled with
the age of the system rather than the size of the page.
All four jobs move into SQL:
- `IDeploymentManagerRepository.QueryDeploymentListPageAsync(filter, page, size)`
returns one page of `DeploymentListRow` — DeploymentRecord INNER JOINed to
Instance, so the instance display name and site travel with the rows that need
them — plus the total count of the filtered set. The join is exact: the FK is
Restrict and DeleteInstanceAsync removes the records first, so no orphan exists.
- `GetDeploymentStatusCountsAsync(filter)` returns the tile counts from ONE grouped
aggregation, deliberately ignoring the filter's Status: the tiles are the status
BREAKDOWN of the filtered set, so honouring it would zero three of four tiles the
moment an operator clicked one.
- Site scoping runs in the query as `SiteIdScope` resolved through the record's
instance (DeploymentRecord has no SiteId of its own). An EMPTY grant stays a real
filter matching nothing, never "unconstrained".
- The now-callerless whole-table `GetAllDeploymentRecordsAsync` is deleted.
OFFSET paging, not the Audit Log's keyset cursor, and deliberately so: this page's
pager is numbered and jump-to-any-page, so it needs a page count, which only a total
can give it — a keyset cursor can express neither, and the total is required for the
tiles regardless. The deep-offset cost that pushes high-volume tables to keyset is
bounded here by the terminal-record retention purge, unlike the 365-day AuditLog.
This mirrors the Notification Outbox, offset-paged for the same reason. Ordering is
DeployedAt DESC, Id DESC — the Id tie-break is load-bearing, because DeployedAt ties
on rapid redeploys and an unstable sort key makes offset paging repeat or drop rows.
UI: the four status tiles become the status filter (click to apply, click again to
clear, aria-pressed, phrasing-only content so a <button> stays valid), plus a
free-text search matched DB-side against instance name, deployment id, revision hash
and initiating user. Search is TRAILING-edge debounced at 500ms — the same Timer +
lock + _disposed idiom as the existing leading-edge push coalescer, minus the leading
edge, because a search box must not query on the first keystroke. A filter change
resets to page 1; a page past the end falls back to the last real page. Bootstrap
only, existing PagerWindow pager retained.
The WP2.4 push coalescing is unchanged and still earns its keep: server paging shrank
what a reload costs, not how many arrive — it now bounds database round-trips rather
than table scans.
Tests: 19 new SQLite repository tests (paging slice + total, tie-break stability
across pages, page/size clamping, past-the-end, the joined projection, every filter
dimension incl. the empty-scope security case, and the grouped counts' status-blind
contract); 15 new bUnit page tests (page-1 request, server total drives the pager,
Next re-queries, tiles show server counts not page counts, tile filter + toggle,
system-wide vs site-scoped scope push, debounce collapses a keystroke burst to one
query, clear-filters, dispose with an armed timer). The two existing Deployments
suites re-point their reload assertions at the new query.
Doc: Component-CentralUI.md Deployment section — the "no server-side paging" known
residual is replaced by the shipped design.
MES alarm-status API §5.2 (docs/plans/2026-06-30-mes-alarm-status-api.md,
Phase 1 tasks 2-4). Site `Call` scripts had NO way to read alarm condition
state: the `Alarm` global exists only inside an on-trigger handler and
describes the one alarm that fired, and native mirrored conditions were
reachable only from the Debug View. That gap blocked the CvdReactor
SimpleAlarmStatus/AlarmStatus scripts entirely -- they cannot be written
without it. `Alarms.CurrentAsync()` closes it.
The data was already local: the script runs inside its own Instance Actor's
context, so this is a LOCAL Ask -- the same mechanism attribute reads use, no
cross-cluster hop. A dedicated GetAlarmSnapshotRequest/Response is used rather
than reusing DebugSnapshotRequest, which would materialise every attribute
value on every alarm poll; both are served from the same
BuildAlarmStatesSnapshot(), so the script view and the operator's Debug View
can never disagree.
Deliberate shape decisions:
- NOT scope-prefixed, unlike Attributes. Alarm identity is not a
scope-relative attribute name (computed alarms are keyed by configured
name, native conditions by a source-supplied reference), so prefixing
would hand a composed script a silently truncated list.
- Read-only. Native alarms are a read-only mirror of the source (no
ack-back), so no acknowledge/shelve operation is exposed.
- Placeholder rows are NOT pre-filtered: a caller must be able to tell
"binding configured and quiet" from "binding unknown". The documented
filter is `Active && !IsConfiguredPlaceholder`.
- ScriptAlarm lives in Commons so the runtime accessor and the compile-only
surface project to the SAME type -- a script binding at the design-time
gate binds identically at the site. Condition is the authority for
active/acked/severity, so one filter expression works across computed and
native alarms.
Mirrored on BOTH design-time surfaces. ScriptCompileSurface is covered by the
reflection parity guard (AlarmsAccessor added to its mirror pairs). The Central
UI Test-Run SandboxScriptHost is the third, hand-maintained mirror that the
parity test cannot reach (Central UI does not reference Site Runtime); without
it the design page would false-flag CS1061 on scripts the deploy gate accepts.
It throws a labelled ScriptSandboxException at run time -- there is no central
route to per-instance alarm state, and returning an empty list would read as
"nothing is in alarm", which is worse than an error.
ScriptTrustPolicy needs NO change, and the reason is structural rather than
incidental: the trust boundary is a deny-list over API roots, not an allow-list
of context members. A test pins that no ForbiddenScopes entry prefixes the
Commons script-surface namespace, so a future deny-list entry cannot silently
make ScriptAlarm untouchable.
Tests: 6 accessor cases (Ask contract, full native projection incl. AckTime,
unacked, computed-alarm derivation, placeholder visibility, scope-independence),
2 InstanceActor snapshot cases incl. equality with the Debug View row set, the
full MES script shape compiling against ScriptCompileSurface, 2 trust cases,
and a sandbox diagnose-clean case reading every projected ScriptAlarm field.
MES alarm-status API §6.4 (docs/plans/2026-06-30-mes-alarm-status-api.md,
Phase 1 task 1). MES needs a real AckDT for a triggered alarm, and the mirror
carried acked-vs-unacked but never WHEN. AckTime now rides the whole path:
DCL transition -> AlarmStateChanged -> gRPC AlarmStateUpdate -> site SQLite.
Stamping rule, identical on both protocols: non-null ONLY while the condition
is active AND acknowledged. That single predicate yields all three required
behaviours -- null while unacked, cleared on re-raise (a re-raise arrives
unacknowledged), and no phantom ack on a return-to-normal. The last one is
load-bearing for MxGateway, which maps INACTIVE to Acknowledged = true; without
the active check every clear would claim an ack the system never observed.
Provenance is honest, never fabricated:
- OPC UA A&C supplies a TRUE ack instant, so we now select it:
AcknowledgeableConditionType/AckedState/TransitionTime at SelectClause
index 18, APPENDED so the positional reads at 0-17 keep their meaning.
Servers that omit the field fall back to the event's own Time.
- MxAccess Gateway supplies none, so the ack transition's own timestamp is
used -- accurate to when the system SAW the ack. An ACTIVE_ACKED
re-subscribe snapshot restores one from LastTransitionTimestamp rather
than dropping it.
The decision lives in pure mappers (Opc/Mx AlarmMapper.DeriveAckTime), so it is
unit-tested with no live server or gateway.
Additive-only throughout: init-only property on AlarmStateChanged, trailing
optional positional on NativeAlarmTransition (all 14-arg call sites untouched),
proto field 24 (never reusing a number) regenerated via docker/regen-proto.sh
sitestream with the csproj diff verified empty.
Persistence rides native_alarm_state's existing metadata_json blob rather than
a new column -- deliberately. That table is RegisterReplicated in
SiteLocalDbSetup and LocalDb builds its CDC triggers from the column list at
registration time, so an additive JSON property changes no schema, no triggers
and no replication contract; metadata_json is exactly the extension point UA4
introduced for this. Rows written before the field deserialize it as null.
Tests: 4 OPC UA + 6 MxGateway mapper cases, 3 NativeAlarmActor (emit,
failover rehydrate, pre-AckTime row), 1 proto round-trip incl. the null case,
4 Commons additive/back-compat. The OPC UA SelectClause count lock-in moves
18 -> 19 with an index-18 assertion -- intended, the clause is appended, which
is precisely what that guard exists to make visible.
Admin-triggered failover of the central pair. IManualFailoverService is declared in
CentralUI (plain strings, so that project stays Akka-free); AkkaManualFailoverService
implements it in the Host and is registered only in the Central branch.
- Leave, never Down: singletons hand over instead of being killed.
- Target = oldest Up member with the Central role, mirroring ActiveNodeEvaluator, so
the node acted on is exactly the one hosting the singletons (never the leader,
whose address-ordered definition diverges from singleton placement after a restart).
- Peer guard: returns null when fewer than 2 Up Central members — failing over a lone
node is an outage, not a failover.
- Audited BEFORE the Leave is issued via ICentralAuditWriter: the acting node can be
the one that goes away, and an audit written after could be lost to the shutdown it
describes. Best-effort — audit failure never blocks the failover.
New audit taxonomy: AuditChannel.Cluster + AuditKind.ManualFailover (operator-initiated
topology actions are not script trust-boundary crossings, but are exactly what an audit
log exists to attribute). Lock-in tests updated 5->6 channels, 16->17 kinds.
alog.md §4 updated per the lock-in tests' contract. Both tables were already stale --
the Channel row omitted SecuredWrite and the Kind table claimed 10 while the code had
16 -- so they are completed here, not merely appended to.
ManualFailoverTests: 3 real-cluster tests (oldest leaves + survivor takes over, peer
guard refuses with a positive still-running assert, dry-run probe does not perturb).
Closes the operator parity gap the deferred-work register tracked as #12: native
alarm source overrides could only be set one-at-a-time, while attribute overrides
had a CSV bulk path. Adds an all-or-nothing CSV import for native sources.
- Commons: extract the RFC-4180 line splitter into a shared CsvLineSplitter
(refactor OverrideCsvParser onto it — no behavior change, pinned by its tests);
new NativeAlarmSourceOverrideCsvParser (header SourceName,Connection,
SourceReference,Filter; blank = inherited).
- Commons: NativeAlarmSourceOverrideEntry + bulk SetInstanceNativeAlarmSource-
OverridesCommand (auto-registered via the reflection command registry).
- ManagementService: Deployer-gated handler — flattens once, validates every
source resolves + is unlocked + no duplicates up front, then upserts the whole
batch under a single SaveChanges (true all-or-nothing txn). Added to the frozen
authorization matrix; dispatch-coverage guard passes.
- CLI: `instance native-alarm-source import --instance-id --file` (parity with
`instance import-overrides`) + README.
- Tests: native parser (Commons), CLI parse/entry mapping, 3 bulk-handler tests
(happy path single-commit, locked-source reject, unresolved-source reject).
Also corrects a stale XML-doc line in ScriptRuntimeContext (WaitForAttribute
quality-gated mode is shipped, not "planned") and updates the deferred-work
register: marks #7/#13/#15/#16/#20 verified-resolved, #12 as CLI/API-shipped with
only the Central UI upload affordance still pending.
Claude-Session: https://claude.ai/code/session_01MtdgwpEeCUn6cUA5f1LMPj
#05-T19. Import name-resolution findings are now split by origin: template-script
(and attribute-default-expression) references become non-blocking advisory
warnings (ConflictKind.Warning + ImportResult.Warnings) since the design-time
deploy gate re-validates authoritatively; ApiMethod-script references stay hard
errors (SemanticValidationException / ConflictKind.Blocker) as they have no
downstream gate. Names declared locally in a body (Roslyn-parsed local functions
/ methods) are excluded so a script's own helper isn't flagged; KnownNonReferenceNames
extended with common BCL/LINQ/SQL surface. DetectBlockersAsync and
RunSemanticValidationAsync Pass 1 apply the identical split so preview and apply
agree. Rollback/hard-fail tests retargeted to ApiMethods (the still-hard-failing
vehicle). CentralUI preview renders the new Warning badge/advisory.
Claude-Session: https://claude.ai/code/session_01MtdgwpEeCUn6cUA5f1LMPj
Add IsLocked to ResolvedNativeAlarmSource; FlatteningService skips overrides on a
locked source (mirrors attribute/alarm lock rules); ManagementActor SetInstanceNativeAlarmSourceOverride
flattens the instance and rejects an override on a locked source (and rejects a
canonical name that does not resolve, preventing dangling overrides).
Claude-Session: https://claude.ai/code/session_01MtdgwpEeCUn6cUA5f1LMPj
Joseph Doherty