Within an equal NON-terminal rank the newest UpdatedAtUtc wins, unfreezing the
Attempted-phase RetryCount/LastError/HttpStatus. Terminal ranks (>=3) are
excluded from the tiebreaker so terminal immutability is preserved (Delivered
never overwrites Parked); equal stamps stay idempotent, lower rank stays a no-op.
SiteAuditRetentionService (site-only IHostedService) ticks PurgeExpiredAsync on
InitialDelay(5m)/PurgeInterval(24h, clamped >=1m), cutoff = UtcNow - RetentionDays.
Registered site-only in AddAuditLogHealthMetricsBridge; options bound from
AuditLog:SiteRetention. Per-tick failures swallowed. Added a test-only
PurgeIntervalOverride (mirrors SiteCallAuditOptions) so the ms-cadence tick test
can bypass the 1-min production clamp. Doc updated. (PLAN-04 Task 3, S1/U2)
Renames stale ActiveNodeHealthCheck references to OldestNodeActiveHealthCheck
(Task 7 rename) across the requirements + components doc sets, corrects
leader→oldest-member wording and the embedded IsActiveNode snippet, and adds an
explicit 'production TLS profile not yet implemented' roadmap note.
deploy/wonder-app-vd03 NodeName overlay edits deferred: that production deploy
artifact is not tracked in this repo (same as Tasks 16 & 20).
WhenTerminated watchdog calls IHostApplicationLifetime.StopApplication() when
the ActorSystem dies outside StopAsync (SBR self-down + run-coordinated-shutdown-when-down),
so the service supervisor restarts the node as a fresh incarnation. Optional
ctor param keeps every existing construction site compiling.
Plan's deploy/wonder-app-vd03/install.ps1 service-recovery-actions edit is
skipped: that production deploy artifact is not tracked in this repo (Task 16
skipped its appsettings.Central.json for the same reason).
CompileAndRegister only swaps the cached handler on a successful compile, so a
management (or DB) save of a non-compiling inbound script silently kept the
previously-registered handler serving — stale results, still HTTP 200 — while
UpdateApiMethod reported success. A broken save thus looked like a no-op, with
the real Roslyn error only in the central log.
Surface the diagnostics instead: a new CompileAndRegister(method, out errors)
overload reports the compile/trust-model errors (the bool overload is kept for
existing callers), and HandleCreate/UpdateApiMethod return them as a top-level
compileWarning on the management result (null when the script compiled). The
save still persists and the previously registered version keeps serving — the
warning just stops a non-compiling save from looking like a success.
- InboundScriptExecutor: Compile returns its diagnostics alongside the handler.
- ManagementActor: TryCompileAndDescribe + flat ApiMethodResult projection so
compileWarning rides at the top level without polluting the ApiMethod POCO.
- Component-InboundAPI.md: document the non-fatal-but-surfaced contract.
- Tests: executor diagnostics + non-compiling-update-keeps-previous-handler,
plus an end-to-end ManagementActor test asserting compileWarning is returned.
The templates tree rendered a derived/composed member (e.g. LeftReactorSide,
derived from ReactorSide) as a flat leaf, omitting compositions it inherits
from its base (e.g. LeakTest composed onto ReactorSide). BuildCompositionLeavesFor
recursed only over a template's OWN composition rows; an inherited composition
row lives on the ancestor, and TemplateComposition has no IsInherited placeholder
(unlike attributes/alarms/scripts/native-sources), so the child's own Compositions
was empty. Same 'derived templates don't surface inherited members' family as
followups #1/#2, but for compositions. Deploy/flatten was always correct
(TemplateResolver.ResolveAllMembers walks the chain) — display-only.
Fix:
- BuildCompositionLeavesFor now renders the EFFECTIVE composition set (own +
inherited) via EffectiveCompositionsFor, which walks the inheritance chain
(leaf->root, child wins on InstanceName), mirroring the resolver.
- Inherited slots are flagged (TemplateTreeNode.IsInherited), badged 'inherited'
in the label, and their context menu offers only 'Open composed template'
(Rename/Delete edit the ancestor's slot, so suppressed on inherited nodes).
- The same inherited row can appear under several derived members (LeakTest under
both LeftReactorSide and RightReactorSide), so composition nodes use a
path-qualified KeyOverride to keep TreeView selection/expansion keys unique;
recursion is cycle-guarded.
Tests: +1 bUnit (TemplatesPageTests.Renders_InheritedComposition_UnderDerivedComposedMember);
CentralUI suite 867 green; full solution builds 0/0.
Docs: Component-CentralUI.md (effective composition set in tree); known-issues
tracker #9 recorded + resolved.
Note: CentralUI change — shows on wonder-app-vd03 only after that host is redeployed.
Closes the four remaining items in the 2026-06-24 template-inheritance/CLI
follow-up tracker.
#4 — CLI `instance set-bindings` can now set DataSourceReferenceOverride.
`--bindings` accepts an optional 3rd element per entry:
[attributeName, dataConnectionId, dataSourceReferenceOverride]. A string
sets the override; a JSON null or an omitted 3rd element leaves it unset
(template default). TryParseBindings accepts 2- or 3-element entries and
rejects a non-string/non-null 3rd element or 4+ elements with a clean
error. Previously the CLI sent the override as null and silently wiped any
existing one (only a raw POST /management could set it).
#5 — `template update` is partial, not full-replace (fixed server-side so all
clients benefit). UpdateTemplateAsync now uses leave-unchanged semantics:
a null description keeps the stored value (pass "" to clear); a null
parentTemplateId keeps the existing parent. Parent stays immutable — a
non-null differing value is still rejected — but omitting --parent-id is
now a no-op instead of failing every derived-template update.
#6 — compact `template list`/`get` table output + `--detail`. Table output is
now id/name/description/parent/derived + member counts (#attrs/#alarms/
#scripts/#comps/#nativeAlarms) via TemplateTableProjection, fed through a
new optional tableProjector seam on CommandHelpers. `--detail` restores the
full dump. JSON output is left untouched (always full) so machine consumers
are unaffected — the projector only runs on the table path.
#8 — structured deploy-time validation error. New ValidationResult.SummarizeErrors()
(Commons) returns a grouped, capped summary: leading total count, one line
per ValidationCategory, and a per-module rollup (canonical name up to its
last dot) with counts + "... and N more module(s)" caps. DeploymentService
uses it for the "Pre-deployment validation failed" message and logs the full
per-entry list via LogWarning. Replaces the flat semicolon-joined dump that
became a wall of text for instances with 50-194 unbound attributes.
Tests: +8 Commons (SummarizeErrors), +8 CLI (4 binding 3-element / 4 table
projection), +2 net TemplateEngine (partial-update). Affected suites green:
Commons 587, CLI 341, TemplateEngine 447, DeploymentManager 101,
ManagementService 230, CentralUI 866; full solution builds 0/0.
Docs: Component-DeploymentManager.md "Validation Error Reporting"; CLI README
(set-bindings 3-element form, template update leave-unchanged, list/get
--detail); UpdateTemplateCommand doc; known-issues tracker #4/#5/#6/#8 resolved
(all 8 items now closed).
Derived templates store IsInherited placeholder rows mirroring inherited
members, but a base member added/changed/removed AFTER a child was derived
never reached the child — leaving the editor's editable tabs incomplete (#1)
and stored rows drifted from the resolved set (#2).
Fix (one order-independent reconcile, two entry points):
- Auto-propagation: every attribute/alarm/script add/update/delete now
reconciles the template's derived subtree (TemplateService.ReconcileDescendantsAsync),
hooked into all member-mutating paths incl. native-alarm-source CRUD in the
ManagementActor.
- Resync: ResyncInheritedMembersAsync repairs a template + its subtree on
demand — materialize missing placeholders, re-sync drifted ones, remove
orphans, across attributes/alarms/scripts/native sources. Exposed as
management ResyncInheritedMembersCommand (Designer-gated, audited) → CLI
`template resync-members` → a Resync button on the editor's staleness banner.
Reconcile drives off TemplateInheritanceResolver (same precedence + HiLo merge
as deploy), only ever touches IsInherited placeholders (never an authored
override), and matches the staleness comparison keys so the banner clears.
BuildDerivedTemplate now also materializes native-source placeholders at
compose time (previously omitted → any inherited native source was perpetually
stale).
Tests: +8 TemplateServiceTests (materialize / drift-update / orphan-remove /
override-untouched / base-cascade / multi-type / direct-propagate / end-to-end
add) + 1 ManagementService test fix (native-source add resolves TemplateService).
Affected suites green: TemplateEngine 446, ManagementService 230, CentralUI 866,
CLI 333, Transport 127, ConfigurationDatabase 307; full solution builds 0/0.
Docs: Component-TemplateEngine.md "Inherited-Member Propagation & Resync";
CLI README `template resync-members`; known-issues tracker #1/#2 resolved.
#3 — CollisionDetector counted a derived template's IsInherited placeholder
rows as a distinct origin from the parent members the inheritance walk
re-adds, reporting a spurious "Naming collision" for every inherited row and
blocking any attribute/composition add to a derived template. CollectDirectMembers
now skips IsInherited rows on the direct-template and inherited-parent walks;
it keeps them for the composed-module walk, where placeholders are the sole
representation of a derived module's inherited members (that walk does not
climb the composed template's parent chain).
#7 — SandboxAttributeAccessor (Central UI Test-Run host) omitted
WriteBatchAndWaitAsync / WaitAsync / WaitForAsync, so the editor false-flagged
valid instance scripts with CS1061 even though `template validate` and the
deploy gate accept them. Added the five overloads mirroring the runtime
AttributeAccessor; they throw a labelled ScriptSandboxException if run in
Test Run (the central sandbox has no device-batch / event-waiter transport).
Tests: +3 CollisionDetector unit + 1 end-to-end TemplateService (derived add
now succeeds); +2 ScriptAnalysisService diagnose-clean. Each new test verified
to fail without its fix with the exact user-facing symptom. Full suites green
(TemplateEngine.Tests 438, CentralUI.Tests 866).
Docs: Component-TemplateEngine.md (inherited-placeholder collision rule),
Component-ScriptAnalysis.md (third sandbox surface + its compile-clean guard),
known-issues tracker #3/#7 marked resolved and the minor note promoted to #8.
Fixes the 8 findings from the 2026-06-24 re-review (commit c42bb485), with a
regression test per Medium finding:
- DataConnectionLayer-029 (Med): HandleAlarmSubscribeCompleted now mirrors the
tag-path re-check — if a feed is already stored for the source, release the
redundant just-created subscription instead of overwriting + leaking the first
one (the double-subscribe window DCL-023 reopened). +regression test.
- InboundAPI-031 (Med): remove WaitForAttribute's local 5s grace backstop (tighter
than the CommunicationService Ask's timeout+IntegrationTimeout round-trip budget,
so a slow-but-valid timed-out 'false' got cancelled into a 500). Link only the
client-abort + explicit caller tokens; the lower layer owns the backstop. +test.
- SiteRuntime-032 (Med): derive the deployed count from an authoritative set of
deployed config names (HashSet) instead of a map-presence-gated int, so deleting
a DISABLED instance decrements correctly (SiteRuntime-029's gate leaked it).
+deploy->disable->delete regression test.
- StoreAndForward-028 (Med): reset _bufferedCount in StopAsync alongside the
register-guard so a same-instance Stop->Start re-seeds from a clean base (no ~2N
gauge double-count). +restart regression test.
- AuditLog-017 (Low): test the OnIngestAsync scope-resolution guard (actor survives,
replies empty, counts the failure) — no longer unpinned.
- CentralUI-037 / ScriptAnalysis-009 / SiteRuntime-033 (Low): doc-comment + spec
fixes (Database-throws in the inbound sandbox; baseReferences param wording;
native-alarm cap return-to-normal + per-condition NativeAlarmDropped eviction).
Targeted suites green: SiteRuntime 5, StoreAndForward 6, InboundAPI 31,
DataConnectionLayer 10, AuditLog 5, ScriptAnalysis 40, CentralUI ScriptAnalysis 52.
Resolves InboundAPI-026/027/028/029 (+ newly-surfaced -030).
- 026: authorize the scoped Database helper in the design doc; SQL-injection
protection is parameter binding (values never concatenated); allow writes via
ExecuteAsync; drop the false 'read-only' claim. Named connections only.
- 027: async ADO.NET end-to-end (no .GetAwaiter().GetResult()); honour the method
deadline token on ExecuteScalarAsync/ExecuteReaderAsync/ExecuteNonQueryAsync +
a CommandTimeout backstop derived from the method timeout.
- 028: negative-path tests (null-gateway, deadline cancellation, parameterization)
+ e2e Database + WaitForAttribute cases through the real endpoint.
- 029: WaitForAttribute is bounded by its WAIT timeout (per-wait CTS + client-abort
+ explicit token), NOT the method deadline (spec §6) — a long wait may outlive the
method timeout; WithRequestAborted threads the raw client-abort token separately.
- 030: Central UI compile-surface mirrors (InboundScriptHost / SandboxInboundScriptHost)
gained the Database member (drifted since the runtime helper was added) so the
authorized async API type-checks at the design-time gate.
Remediation from the full per-module code review at 4307c381 (findings recorded
separately in code-reviews/).
Highs fixed:
- DeploymentManager-025/SiteRuntime-031: stop broadcasting notification lists + SMTP
configs (incl. credentials) to sites; site purges already-persisted rows on apply
(enforces the central-only delivery design; clears plaintext SMTP creds at rest).
- DataConnectionLayer-023: guard the native-alarm subscribe path against the
mid-flight-unsubscribe adapter-feed leak (mirrors the DCL-021 tag-path fix).
- SiteEventLogging-024: normalize From/To query bounds to UTC (the -016 fix the
audit trail claimed but never committed).
- KpiHistory-001: add an in-flight guard to the recorder sample tick.
- ScriptAnalysis-001: harden the trust analyzer's TPA-absent fallback (resolve
forbidden anchors in the minimal reference set; warn on degraded mode) — anchors
added to validation references only, never the compile gate.
(InboundAPI-026 left to the feat/ipsen-movein effort per owner decision.)
Medium/Low: DM-026 deterministic deploy-status tiebreaker; SR-027/028/029/030
native-alarm leak/phantom-active/delete-during-redeploy fixes; AL-013/014/016;
TE-024 (folder-mutation audit rows now persisted)/025; SF-025 gauge-provider
clear-on-stop; ESG-025/026; SEC-023/024/025; SCA-007/008/009; plus doc/test
accuracy COM-023/024, HOST-025/026, HM-024/025, NS-027/028.
Full-solution build 0 warnings; ~3560 tests across 18 touched suites green.
Adds `cached-call retry` and `cached-call discard` subcommands that relay
to the existing Deployer-gated RetryParkedMessageCommand /
DiscardParkedMessageCommand via the central SiteCallAuditActor → site relay.
ManagementCommandRegistry already covered both types via reflection auto-discovery.
CommandTreeTests updated to include cached-call (group count 16 → 17).
- Component-CentralUI.md: replace flat-table Debug View section with tabbed
tree layout (Attributes + Alarms tabs, TreeView<TItem> reuse, hierarchy from
canonical names, branch roll-up, all-configured-alarms rule, native source
binding nodes with quiet-binding placeholder rows, per-leaf rendering detail)
- Component-SiteRuntime.md (Instance Actor Wiring): add idle-binding placeholder
emission via BuildAlarmStatesSnapshot(), _nativeAlarmKinds map, and
NativeSourceCanonicalName stamping on live native events
- Component-SiteRuntime.md (Enriched AlarmStateChanged): document two new
additive fields — NativeSourceCanonicalName (string?) and
IsConfiguredPlaceholder (bool) — plus their gRPC proto fields 22/23 and
StreamRelayActor/SiteStreamGrpcClient pack/unpack
- Component-Commons.md (Attribute Stream DTOs): extend AlarmStateChanged bullet
with the same two additive fields and proto field numbers
NJ-6: full solution builds 0/0; feature-targeted tests green (Commons codec 38,
TemplateEngine InstanceService 17, ConfigDB normalizer 8, Transport serializer 12,
SiteRuntime InstanceActor 47). Component-Commons now describes the native-typed
List encoding + read-both decode + the three normalization paths. #93/M3 folded in.
Adds a "Dev Disable-Login Flag" subsection to Component-Security.md covering
ScadaBridge:Security:Auth:DisableLogin / User, the AutoLoginAuthenticationHandler
mechanism, and the no-environment-guard / startup-warning production risk.
Ships DisableLogin: false under ScadaBridge → Security → Auth in:
- src/.../Host/appsettings.json (canonical default)
- docker/central-node-a/appsettings.Central.json
- docker/central-node-b/appsettings.Central.json
Also records DL-3 commit SHAs in the plan tasks file.
Spike outcome: the shared ILdapAuthService (ZB.MOM.WW.Auth.Abstractions, an external
NuGet package) exposes ONLY AuthenticateAsync(username, password, ct) — no passwordless
service-account group-search. A live LDAP group re-query for an active session therefore
requires a new lib method and is OUT OF SCOPE (cannot modify the external package).
Implemented the always-achievable layers (cookie-only; no embedded JWT for cookie principals):
- /auth/login now stores the user's raw LDAP groups (one zb:group claim each) plus a
zb:lastrolerefresh anchor (login time, UTC), seeding the LastActivity idle anchor too.
- SessionClaimBuilder: single shared DRY claim-builder used by BOTH /auth/login AND the
refresh path, so the two claim shapes cannot drift (canonical identity/role/scope claims
with nameType/roleType pinned, plus the M2.19 group + refresh-anchor additions).
- CookieSessionValidator (TimeProvider-injected, unit-testable) + a thin
CookieAuthenticationEvents.OnValidatePrincipal adapter:
* idle-timeout: a session past IdleTimeoutMinutes (default 30) is RejectPrincipal+SignOut;
consistent with the cookie ExpireTimeSpan+SlidingExpiration window (same value).
* role refresh WITHOUT LDAP: when older than RoleRefreshThresholdMinutes (new option,
default 15) the DB-backed RoleMapper re-runs on the STORED groups, claims are rebuilt
via the shared builder, the anchor advances, principal is replaced + cookie renewed.
Revoked DB mappings drop the user's roles mid-session.
* fail-soft: any refresh error KEEPS the existing principal (no sign-out, never throws)
— mirrors the documented "LDAP failure: active sessions continue with current roles".
- Documented residual limitation in Component-Security.md: central role-mapping/scope
changes apply within ~15 min without LDAP; live directory group-membership changes are
picked up only at next login (needs a passwordless group-search on the external
ZB.MOM.WW.Auth.Ldap lib — tracked follow-up).
Tests (Security.Tests, all green): CookieSessionValidatorTests + SessionClaimBuilderParityTests
— idle reject/keep, LDAP-free remap-from-stored-groups, revoked-roles loss, sub-threshold
no-refresh, refresh-throws-keeps-session, and login/refresh claim-parity.
git blame shows commit 1d5465f3 deliberately added NotDeployed to CanDelete so an
undeployed instance can have its orphan record fully removed. Code + tests already
permit it; the spec matrix said 'No'. Per M2.17, reconcile doc→code (not the reverse):
matrix now reads 'Delete from Not deployed = Yes (removes the orphan record)' with a
note, and CanDelete carries a remark citing the rationale + origin commit.
REQ-HOST-4a lists "required cluster singletons running (if applicable)" as a
readiness criterion, but /health/ready only checked database + akka-cluster.
Add a third Ready-tagged check, RequiredSingletonsHealthCheck, registered in the
Central-role AddHealthChecks() chain (so it is naturally role-scoped — site nodes
never run it).
Probe: for each required central singleton, Ask its local ClusterSingletonProxy
an Identify with a short bounded per-singleton timeout (~2s, probes run
concurrently via Task.WhenAll). A non-null ActorIdentity.Subject within the
timeout means the singleton is running and reachable through the proxy; a null
subject or a timeout means unreachable → Unhealthy, naming the unreachable
singleton(s). The check never throws (catch-all → Unhealthy) and resolves
ActorSystem lazily from DI per probe (Unhealthy if Akka not yet up).
Required-always set = the five singleton proxies created unconditionally in
AkkaHostedService.RegisterCentralActors: notification-outbox, audit-log-ingest,
site-call-audit, audit-log-purge, site-audit-reconciliation. There are no
feature/config-gated central singletons today; any future gated singleton is the
"if applicable" case and must NOT be added to the required set.
Leadership-agnostic: the proxy reaches the singleton from either central node, so
a ready standby still reports ready (readiness must not require cluster
leadership — that is the Active tier's job). During a brief singleton handover the
probe may time out and the node flaps to not-ready, which is correct (a node
mid-handover is legitimately not fully ready); no retries, to keep the probe fast.
Tests (TDD): RequiredSingletonsHealthCheckTests exercises the probe against a
TestKit ActorSystem — all proxies present+reachable → Healthy; one missing →
Unhealthy naming it; ActorSystem absent → Unhealthy, no throw. HealthCheckTests
regression-guards the Ready tag + absence of the Active tag on the new check.
Object/List parameters and return values were shape-validated only (object vs
array), with no field-level/nested type checks — type-wrong nested data passed
inbound validation and failed only at script runtime. Add recursive type
validation (declared Object field types, List element type, scalars at any depth)
with path-qualified errors, symmetric across ParameterValidator and ReturnValueValidator.
Both validators now parse the canonical JSON Schema definition format (the
Central UI / MigrateParametersToJsonSchema output) via a shared recursive engine,
Commons.Types.InboundApi.InboundApiSchema, instead of the legacy flat
[{name,type}] array which they could not even deserialize from migrated rows.
The legacy flat-array form is still accepted on read for transition safety.
Undeclared fields are rejected at every level (consistent with the existing
top-level unexpected-parameter rejection); a present-but-null value satisfies
any type, only absence of a required field is an error.
Gitea renders mermaid inline, so the flow/state/hierarchy/DAG diagrams
move to text-in-markdown: auto-layout (removes the manual overlap-prone
draw.io step), diffable source, no committed binaries, and a dark-text
theme so labels stay legible. Keep draw.io PNGs only for the two complex
bespoke diagrams (logical architecture, env2 topology) where pixel
control still wins. All 24 mermaid blocks validated by rendering.
Joseph Doherty