Central and each site are SEPARATE Akka clusters, so central cannot act on a
site's membership -- it asks. New TriggerSiteFailover/SiteFailoverAck contract
travels the existing ClusterClient command/control channel (mirroring the
RetryParkedOperation relay); the site's own SiteCommunicationActor performs the
graceful Leave and acks the outcome.
- ClusterFailoverCoordinator moved out of Host into Communication/ClusterState,
beside ActiveNodeEvaluator. Both paths now share ONE oldest-Up implementation;
SiteCommunicationActor cannot reference Host, and the two definitions must not
drift or the node asked to leave stops being the singleton host.
- Site scope is the SITE-SPECIFIC role (site-{SiteId}), not the base Site role --
site singletons are placed on the former, so the base role would move the wrong
node. Pinned by a unit test asserting the role string and by a real-cluster test.
- Site-side guards: refuses a command addressed to another site (a misroute must
never fail over a site the operator did not select), refuses when there is no
peer, and reports a fault as an ack rather than throwing into supervision --
a restart there would drop central's Ask into a bare timeout and lose the reason.
- Ack is sent before the Leave takes effect so it still reaches central.
- UI: the same control now serves both scopes via a SiteId parameter. The site
confirmation deliberately does NOT claim the admin's page will disconnect --
it won't, and crying wolf there devalues the central warning that is real. A
site refusal and an unreachable site surface distinctly.
- Rolling upgrade: a site on an older binary has no handler, so the message
dead-letters and the Ask times out, reported as "site did not respond". That
is the honest outcome; documented on the contract.
Fallout fixed: HealthPageTests now renders the page inside
CascadingAuthenticationState with the real policy set and IAuthorizationService,
because the cards embed an AuthorizeView. That mirrors production, where the
layout supplies the cascading value.
CentralFailoverControl lives in Components/Health/ (matching AuditKpiTiles /
SiteCallKpiTiles) rather than inline in Health.razor, so it is testable without
standing up the whole dashboard's DI graph.
- Admin-gated via AuthorizeView + RequireAdmin. The Health page itself is
intentionally all-roles, so the gate belongs on the control, not the page.
- Disabled with an explanatory title when the pair has no online standby;
the authoritative guard remains server-side against live cluster membership.
- Confirmation dialog (IDialogService, the page idiom) warns that singletons
hand over, in-flight work on the active node is interrupted, and THIS PAGE
will disconnect and reconnect against the new active node -- Traefik routes
the UI to the node being restarted, so a working failover otherwise reads as
a crash the admin caused.
- A refused failover (service returns null) surfaces the refusal; the UI never
reports a failover that did not happen.
7 bUnit tests. Two harness requirements that bit first: AuthorizeView needs a
cascading AuthenticationState (the app supplies it from the layout), and
BunitContext pre-registers a placeholder IAuthorizationService that throws on
policy evaluation -- both handled the same way NavMenuTests documents.
Runbook paragraphs added to Component-ClusterInfrastructure.md (new Manual
Failover section) and docker/README.md.
Tasks 2 + 5a-writer + 5b of the LocalDb Phase 1 plan, landed together because
they are one indivisible change: the schema, the writer that fills it, and every
consumer that assumed the old id semantics.
WHY the id changes. Site pairs will replicate site_events with last-writer-wins on
the primary key. With INTEGER PRIMARY KEY AUTOINCREMENT both nodes independently
mint 1, 2, 3... for unrelated events, so sync would treat them as the same row and
silently overwrite. A GUID makes the event log a pure union across the pair.
Task 2 - schema extracted so the Host's AddZbLocalDb onReady can create the tables
before RegisterReplicated installs capture triggers (pre-registration rows are
never captured):
- new OperationTrackingSchema.Apply / SiteEventLogSchema.Apply, plain
Microsoft.Data.Sqlite, no LocalDb dependency
- both stores delegate their InitializeSchema to them (idempotent, so a
directly-constructed store still works)
- OperationTracking is unchanged - it already had a TEXT PK and replicates as-is
Task 5a - SiteEventLogger mints Guid.NewGuid("N") per event and inserts it.
Task 5b - three consumers assumed a monotonic integer id. All three move to
timestamp ordering; leaving any one behind would be a live bug:
- EventLogQueryService: "id > $afterId" would return an ARBITRARY subset of a
GUID-keyed table and SILENTLY DROP ROWS from page-through. Now a composite
(timestamp, id) keyset cursor with an opaque string token; timestamps are not
unique, so id is the tie-break that guarantees exactly-once paging.
- EventLogPurgeService: "ORDER BY id ASC LIMIT 1000" would delete a RANDOM batch
instead of the oldest. Now orders by timestamp.
- EventLogEntry.Id and both ContinuationTokens: long -> string.
WIRE COMPATIBILITY. Those DTOs cross the site<->central Akka boundary
(SiteCommunicationActor -> CommunicationService -> ManagementActor / CentralUI).
No rolling-upgrade shim is needed because both sides ship in the same deployable
and the rig redeploys as a unit. Checked: no Akka serializer binding pins these
types by name. A stale numeric token degrades to "start from the beginning"
(a visible repeat) rather than throwing or losing rows.
Tests: the two that encoded the old semantics were rewritten to guard the new
invariant rather than deleted - uniqueness instead of monotonicity, and
oldest-purged-first keyed on timestamp. That second test also exposed a latent
weakness: the bulk seed stamped every row with the same UtcNow, so "oldest" was
never actually well-defined; rows now get distinct increasing timestamps, kept
inside the retention window so the retention purge does not eat them first.
Verified:
dotnet build ZB.MOM.WW.ScadaBridge.slnx -> 0 Error(s), 0 Warning(s)
SiteEventLogging.Tests -> 70 passed (12 new schema tests, red-first)
CentralUI.Tests -> 925 passed
Commons.Tests -> 684 passed
SiteRuntime.Tests -> 529 passed, 1 pre-existing flaky failure
(InstanceActorChildAttributeRaceTests - passes 3/3 in
isolation with and without this change)
Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
Wire the operator Alarm Summary page to the transient per-site live alarm
cache (ISiteAlarmLiveCache, T4). Live-cache-first: on site select the page
subscribes and rebuilds rows/rollup from near-real-time onChanged deltas; the
15s poll is kept untouched as the authority for NotReporting and as the
safety net whenever the cache is not live (pre-seed or degraded stream). Both
paths mutate shared state only via the Blazor dispatcher, so they never race,
and each rebuild is an idempotent snapshot.
- IAlarmSummaryService.BuildFromLiveAlarms: flattens a live AlarmStateChanged
snapshot to AlarmSummaryRows with the same deterministic instance-then-name
sort as GetSiteAlarmsAsync; NotReporting always empty on the live path.
- AlarmSummary.razor: inject ISiteAlarmLiveCache; subscribe on select,
re-subscribe on site change, unsubscribe on leave + Dispose.
- Tests: service BuildFromLiveAlarms flatten/sort/empty; page subscribe,
poll-fallback render, onChanged rebuild, unsubscribe on leave/change/dispose.
Claude-Session: https://claude.ai/code/session_01MtdgwpEeCUn6cUA5f1LMPj
Adds 30d (720h) and 90d (2160h) toggle buttons alongside the existing
24h/7d controls on all four KPI trend surfaces. No fetch-logic changes:
the setters already recompute fromUtc and re-fetch, and the query service
(T5) transparently routes windows > RollupThresholdHours to hourly rollups
— so these windows are what actually exercise the rollup path. bUnit tests
extend the Audit and SiteCalls trend harnesses to cover the new buttons.
Claude-Session: https://claude.ai/code/session_01MtdgwpEeCUn6cUA5f1LMPj
Additive UpdateSmtpConfigCommand params (OAuth2Authority/OAuth2Scope),
preserve-on-null handler application, CLI --oauth2-authority/--oauth2-scope
on smtp update, Central UI text inputs shown only for the OAuth2 auth type,
and the Component-NotificationService doc paragraph with M365 defaults.
Claude-Session: https://claude.ai/code/session_01MtdgwpEeCUn6cUA5f1LMPj
#05-T19. Import name-resolution findings are now split by origin: template-script
(and attribute-default-expression) references become non-blocking advisory
warnings (ConflictKind.Warning + ImportResult.Warnings) since the design-time
deploy gate re-validates authoritatively; ApiMethod-script references stay hard
errors (SemanticValidationException / ConflictKind.Blocker) as they have no
downstream gate. Names declared locally in a body (Roslyn-parsed local functions
/ methods) are excluded so a script's own helper isn't flagged; KnownNonReferenceNames
extended with common BCL/LINQ/SQL surface. DetectBlockersAsync and
RunSemanticValidationAsync Pass 1 apply the identical split so preview and apply
agree. Rollback/hard-fail tests retargeted to ApiMethods (the still-hard-failing
vehicle). CentralUI preview renders the new Warning badge/advisory.
Claude-Session: https://claude.ai/code/session_01MtdgwpEeCUn6cUA5f1LMPj
Additive RequestedBy on Retry/DiscardSiteCallRequest, plumbed from the Central UI.
On an Applied relay, SiteCallAuditActor emits one best-effort CachedResolve central
direct-write row (Submitted/Discarded) with the operator as Actor. Gated on an
injected ICentralAuditWriter (null in existing tests → no mirror read, unchanged);
the site remains the source of truth for the state change.
Additive RequestedBy on Retry/DiscardNotificationRequest, plumbed from the Central
UI. RetryAsync emits a NotifyDeliver Submitted row (records who un-parked); Discard
stamps the operator on the Terminal row. Best-effort — audit failure never aborts
the action.
Remove project bookkeeping citations from shipped code comments across the
solution: hyphenated task IDs (WP-14, StoreAndForward-025), milestone/task/
issue refs (M3, Task 4, Audit Log #23, #21), Bundle X task-bundle labels,
and C/D/K/S/T phase labels.
Comment text only — no code logic, string/log literals, or XML-doc structure
changed. Genuine descriptions are preserved (only the citation is stripped),
and technical lookalikes are retained (UTF-8, SHA-256, T00:00:00, M365,
UTC-5, pre-C4/pre-C5 schema versions). Flagged by the new CommentChecker
TaskReferenceInComment / TrackingReferenceInComment checks plus targeted
grep passes; full solution builds clean, append-only guard tests pass.
Fully suppress search UI (box + results panel) when ShowSearch=false, and
reset search state on ShowAsync so a reused dialog never shows stale results.
The templates tree rendered a derived/composed member (e.g. LeftReactorSide,
derived from ReactorSide) as a flat leaf, omitting compositions it inherits
from its base (e.g. LeakTest composed onto ReactorSide). BuildCompositionLeavesFor
recursed only over a template's OWN composition rows; an inherited composition
row lives on the ancestor, and TemplateComposition has no IsInherited placeholder
(unlike attributes/alarms/scripts/native-sources), so the child's own Compositions
was empty. Same 'derived templates don't surface inherited members' family as
followups #1/#2, but for compositions. Deploy/flatten was always correct
(TemplateResolver.ResolveAllMembers walks the chain) — display-only.
Fix:
- BuildCompositionLeavesFor now renders the EFFECTIVE composition set (own +
inherited) via EffectiveCompositionsFor, which walks the inheritance chain
(leaf->root, child wins on InstanceName), mirroring the resolver.
- Inherited slots are flagged (TemplateTreeNode.IsInherited), badged 'inherited'
in the label, and their context menu offers only 'Open composed template'
(Rename/Delete edit the ancestor's slot, so suppressed on inherited nodes).
- The same inherited row can appear under several derived members (LeakTest under
both LeftReactorSide and RightReactorSide), so composition nodes use a
path-qualified KeyOverride to keep TreeView selection/expansion keys unique;
recursion is cycle-guarded.
Tests: +1 bUnit (TemplatesPageTests.Renders_InheritedComposition_UnderDerivedComposedMember);
CentralUI suite 867 green; full solution builds 0/0.
Docs: Component-CentralUI.md (effective composition set in tree); known-issues
tracker #9 recorded + resolved.
Note: CentralUI change — shows on wonder-app-vd03 only after that host is redeployed.
Derived templates store IsInherited placeholder rows mirroring inherited
members, but a base member added/changed/removed AFTER a child was derived
never reached the child — leaving the editor's editable tabs incomplete (#1)
and stored rows drifted from the resolved set (#2).
Fix (one order-independent reconcile, two entry points):
- Auto-propagation: every attribute/alarm/script add/update/delete now
reconciles the template's derived subtree (TemplateService.ReconcileDescendantsAsync),
hooked into all member-mutating paths incl. native-alarm-source CRUD in the
ManagementActor.
- Resync: ResyncInheritedMembersAsync repairs a template + its subtree on
demand — materialize missing placeholders, re-sync drifted ones, remove
orphans, across attributes/alarms/scripts/native sources. Exposed as
management ResyncInheritedMembersCommand (Designer-gated, audited) → CLI
`template resync-members` → a Resync button on the editor's staleness banner.
Reconcile drives off TemplateInheritanceResolver (same precedence + HiLo merge
as deploy), only ever touches IsInherited placeholders (never an authored
override), and matches the staleness comparison keys so the banner clears.
BuildDerivedTemplate now also materializes native-source placeholders at
compose time (previously omitted → any inherited native source was perpetually
stale).
Tests: +8 TemplateServiceTests (materialize / drift-update / orphan-remove /
override-untouched / base-cascade / multi-type / direct-propagate / end-to-end
add) + 1 ManagementService test fix (native-source add resolves TemplateService).
Affected suites green: TemplateEngine 446, ManagementService 230, CentralUI 866,
CLI 333, Transport 127, ConfigurationDatabase 307; full solution builds 0/0.
Docs: Component-TemplateEngine.md "Inherited-Member Propagation & Resync";
CLI README `template resync-members`; known-issues tracker #1/#2 resolved.
Code-review finding UI-Med-2: the design doc + delivery adapter treat FromNumber and
MessagingServiceSid as either-or, but the entity ctor, EF schema, UI and CLI all hard-
required FromNumber — so a Messaging-Service-only Twilio config (a normal production
setup) could not be created. Bring the implementation into line with the spec:
- Commons: SmsConfiguration.FromNumber -> string? (ctor fromNumber optional);
UpdateSmsConfigCommand.FromNumber -> string?.
- ConfigurationDatabase: FromNumber.IsRequired(false) + migration SmsFromNumberOptional
(ALTER COLUMN nullable, idempotent; Down backfills '' — harmless, MsgSid keeps it
deliverable) + regenerated model snapshot.
- Transport: SmsConfigDto.FromNumber -> string? (round-trips a Messaging-Service-only config).
- CentralUI: form validation requires AccountSid + at-least-one-of(FromNumber, MsgSid);
nullable create/edit paths; From-number help text.
- CLI: --from-number no longer Required; BuildUpdateSmsConfigCommand validates the either-or.
- Adapter: From branch null-forgiving (guarded by the existing incomplete-config check).
Tests: ManagementActor MsgSid-only persists null FromNumber; CLI MsgSid-only builds +
neither-throws + contract (--from-number not Required); CentralUI MsgSid-only save.
Findings from the per-module code review of the SMS feature (code-reviews/):
- ManagementService (High): UpdateSmsConfig + UpdateSmtpConfig were Designer-gated
while both /notifications/{sms,smtp} pages enforce RequireAdmin — a Designer
blocked in the UI could still rotate a production credential via CLI. Moved both
to the Administrator arm so the actor gate matches the UI.
- ManagementService (Medium): UpdateSmsConfig treated --auth-token "" as a value,
silently clearing the stored Twilio token. Guard on IsNullOrWhiteSpace so empty ==
omitted (SMTP Credentials keeps its null-only guard — empty is valid for no-auth).
- CentralUI (Medium): NotificationLists recipient badge rendered Name <Email>
unconditionally, showing "Name <>" for SMS lists. Now type-aware (phone for SMS).
- ConfigurationDatabase (Medium): AddSmsNotifications.Down() backfilled NULL emails
to '' — silent data loss for SMS-only recipients. Added a pre-drop guard that
refuses rollback while such rows exist.
- NotificationOutbox (Low): SMS body truncation could split a surrogate pair at the
cap boundary; back off one code unit to stay well-formed.
- Commons (Low): NotificationRecipient public ctor name-guard now matches the
ForEmail factory (IsNullOrWhiteSpace). Documented SmsConfiguration.MaxRetries/
RetryDelay as RESERVED (dispatcher reuses the shared SMTP-derived retry policy).
Pre-existing render-crash (introduced by eb4bce3e): a @* *@ comment between
attributes inside the site-health-trends <select> start tag rendered as an
invalid attribute name in a real browser (InvalidCharacterError on setAttribute),
tearing down the SignalR circuit whenever Site Health Trends renders. bUnit's
virtual DOM masked it; the live Playwright smoke (KpiTiles) caught it. Comment
moved outside the start tag.
- Root context menu now has tabindex/focus + Escape-key close (OnRootMenuKeyDown) mirroring the node menu
- Opening root menu calls _tree.DismissNodeContextMenu(); opening node menu fires OnNodeContextMenuOpened → DismissRootContextMenu so only one menu is ever visible
- Add FolderContextMenu_MoveUp_IsDisabled_OnFirstSibling and FolderContextMenu_MoveDown_IsDisabled_OnLastSibling bUnit tests
Expose TemplateScript.ExecutionTimeoutSeconds and MinTimeBetweenRuns (previously
settable only via Transport bundle import) on the CLI and Central UI authoring surfaces.
- Commons: add additive trailing optionals MinTimeBetweenRuns (TimeSpan?) and
ExecutionTimeoutSeconds (int?) to Add/UpdateTemplateScriptCommand.
- ManagementActor: thread both new fields into the built TemplateScript on add/update.
- CLI template script add/update: new --min-time-between-runs (duration: ms/s/min,
bare number = seconds, 0 = unset, mirroring DurationInput) and
--execution-timeout-seconds (int) flags, with client-side duration validation.
- Central UI TemplateEdit: add an Execution timeout input (seconds) on the script
trigger tab, mirroring the existing Min-time-between-runs control; null/0 = site default.
- Tests: TemplateScriptTimingTests pins the option surface + duration parsing; updated
the stale 'no UI control' comment on the TemplateService round-trip test.
Joseph Doherty