Measured first: a scratch scaling probe (seed -> export -> load/preview/apply
into a fresh target and over itself, in-memory provider, every repository
call counted, DetectChanges counted via ChangeTracker.DetectingAllChanges)
showed load, preview and the instance pass scale linearly (500 / 1000 / 2000
instances apply in 0.13 / 0.27 / 0.35 s), but the post-flush template
rewire passes were quadratic: 100 / 400 / 800 composing, derived templates
applied in 0.5 / 8.1 / 26.5 s, with DetectChanges running 5N+10 times and
GetTemplateWithChildrenAsync 2N+2 times.
Cause: ResolveAlarmScriptLinksAsync, ResolveCompositionEdgesAsync and
ResolveInheritanceEdgesAsync found each template with
_dbContext.Templates.Local.FirstOrDefault(...) - per template, and per
composition edge / base lookup. Every DbSet.Local access runs a full
DetectChanges over everything the import has staged. The first two passes
also re-read every template's children one query at a time (a split query
per call on SQL Server).
Fix (no bundle-format or semantic change):
- each pass indexes the tracked templates by name once
(SnapshotLocalTemplatesByName; ordinal, first wins like FirstOrDefault).
These passes never stage a template or rename one, so the index stays
exact for the pass;
- the alarm and composition passes read their templates' children in one
batched GetTemplatesWithChildrenAsync, matched back by the stub's id;
- the alarm pass only loads templates whose bundled alarms name an
on-trigger script (the only ones it can rewire).
Result on the same probe: 100 / 400 / 800 templates apply in
0.13 / 0.62 / 0.64 s; DetectChanges is a constant 9; per-template children
reads 0, batched reads 1 (add) / 2 (overwrite).
Evidence: ImportApplyScalingTests (Transport.IntegrationTests) exports 5 and
40 composing, derived templates with alarm -> script links, imports each
into an empty target (Add) and again (Overwrite), and asserts the imported
inheritance / composition / alarm-script links, zero per-template children
reads, at most 3 batched reads, and an identical DetectChanges count for 5
and 40 templates. Fails on the old importer (12 per-template reads), and
with a per-DTO Local snapshot reintroduced in one pass (DetectChanges 51
vs 16). Transport unit (198) and integration (138) suites pass.
Not changed: preview's per-name lookups for database connections,
notification lists and API methods stay per-name - moving them to an
in-memory ordinal match could change name matching under a case-insensitive
SQL collation.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
DeploymentService.TryPurgeTerminalDeploymentRecordsAsync rate-limited its
opportunistic retention sweep with a process-wide static tick field, so only
one test in the assembly could drive the sweep and its swallow-on-failure path
was untested.
The limit now lives in TerminalPurgeRateLimiter: a TimeProvider-driven
compare-and-swap claim (MinInterval 6 h) registered by AddDeploymentManager as
a DI singleton, so every scoped DeploymentService still shares one limit per
node. DeploymentService takes it as an optional trailing constructor parameter;
a hand-built instance gets its own limiter. The sweep's cutoff now comes from
the same clock. Behaviour is otherwise unchanged.
Evidence (TerminalPurgeRateLimiterTests):
- claim once per interval on an injected clock; 16 concurrent claimants ->
exactly one wins;
- two services with their own limiters both sweep, two sharing one sweep once;
- cutoff = injected now - retention;
- a failing artifact batch is logged as a warning with the exception and
swallowed, the already-purged terminal rows are still reported, the next
call inside the interval does not retry and the first call after it does;
a first-batch failure returns 0 and skips the artifact purge;
- AddDeploymentManager registers the limiter as a singleton shared across scopes.
Mutation-checked: a process-wide static limiter fails 5 tests (incl. the
existing artifact-purge test); narrowing the catch fails both failure tests.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
The last two whole-library template reads are gone. Both fed the
inheritance-only TemplateInheritanceResolver, which walks one template's
parent chain and never follows compositions.
- TemplateService.ResyncInheritedMembersAsync (operator repair, CLI + editor)
now reads #86's GetTemplateReconcileClosureAsync (derived subtree + the
template's chain, tracked). When nothing derives from the template the
closure is empty, so it falls back to the new tracked
GetTemplateChainAsync - resyncing a leaf reads only its own chain.
- ManagementActor.HandleGetResolvedTemplateMembers now reads
GetTemplateChainAsync(id, trackChanges: false): the chain alone, untracked,
with script bodies (the resolver projects script code, so #66's body-free
analysis closure would not be equivalent).
The chain walk is factored out of TemplateReconcileClosure into
CollectChainAsync (one load per level, stops on parent cycles and dangling
parents) and shared by both.
Evidence:
- TemplateChainReadTests (#86 fixture): for every template plus an unknown
id, the closure-backed resync issues exactly the writes, audit row and
result of the whole-graph resync with GetAllTemplatesAsync wired to throw;
the resolver answers identically from the chain and the whole graph.
17 of these fail against the old service; 13 fail with the leaf chain
fallback removed (mutation-checked).
- TemplateChainRepositoryTests (SQLite, recorded SQL): a 4-deep chain loads
4 templates / 8 attributes instead of 46 / 92, one split-query set per
level, untracked on request and tracked for the resync.
- ResolvedTemplateMembersReadTests: the actor answers from the untracked
chain read and never calls GetAllTemplatesAsync (fails on the old actor).
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Health page gains PendingUpdatesHealthTiles (stale instances / sites with stale
artifacts, Deployer+Administrator only, site-scope narrowed, shared cache). New
GetPendingUpdates management command (Deployer) and 'health pending-updates' CLI
leaf. Flatten/cache computation untouched; the command reads
IPendingUpdatesEvaluator directly because the UI cache is not reachable from
ManagementService.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
New SiteWriteFailureBadges component renders warning badges for
SiteEventLogWriteFailures, SiteAuditWriteFailures and AuditRedactionFailure
when > 0 on online sites. The CLI 'health summary' already serialises the full
latest report, so these counters are visible there unchanged.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
TemplateService.ReconcileDescendantsAsync - run after every base member
add/update/delete, template and native-alarm-source alike - loaded the whole
tracked template graph (every template, member and script body) on each call,
which after #66 was the dominant cost of those paths. It now calls the new
ITemplateEngineRepository.GetTemplateReconcileClosureAsync: the same tracked,
fully-included shape as GetAllTemplatesAsync, restricted to the edited
template's derived subtree plus its own inheritance chain. The reconcile is
inheritance-only (TemplateInheritanceResolver never follows compositions) and
every descendant's chain runs through the edited template, so that set is
everything it reads.
The walk (Commons TemplateReconcileClosure, the descendant-direction twin of
#66's TemplateAnalysisClosure) loads one children batch per subtree depth and
one batch per ancestor level, terminates on parent cycles and stops at
dangling parents. A template with no children - the common case - costs one
empty probe and loads nothing.
Evidence:
- TemplateReconcileClosureTests: for every template of a graph with a deep
subtree, an ancestor above the edited template, a composed module,
look-alike unrelated trees, a parent cycle and a dangling parent, the
closure-backed reconcile issues exactly the writes (add/update/delete,
audit, save) of the old whole-graph reconcile, and the resolver answers
identically for every descendant. Fails with the old service (whole-graph
load wired to throw) and with the ancestor walk removed (mutation-checked).
- TemplateReconcileClosureRepositoryTests (SQLite, recorded SQL): 5 templates
/ 10 attributes instead of 47 / 94, tracked with script bodies; a leaf costs
one probe query and tracks nothing.
- TemplateServiceTests: attribute add + update propagate two levels down with
GetAllTemplatesAsync wired to throw; existing propagation tests now run
against the closure via ServeReconcileClosureFromFullGraph.
ResyncInheritedMembersAsync (operator action) still reads the whole graph.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
SiteAuditBacklogSnapshot gains an optional trailing RejectedCount (gauge of
Rejected forward-state rows, read from the site store), carried on the wire as
SiteAuditBacklogSnapshotDto.rejected_count = 4 (additive; old sites read as 0,
old central ignores it). Health page shows an 'Audit rows rejected' badge.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Each cached client records when it was last handed out; a periodic sweep (half of the new
CommunicationOptions.GrpcClientIdleTtl, default 30 min, 0 disables) disposes and removes entries
unused for the TTL that have no live subscription (SiteStreamGrpcClient.ActiveSubscriptionCount).
Lookup and the sweep's check-and-remove share one lock so a handed-out client is never disposed
under its caller; an evicted client is re-created on next GetOrCreate.
Codec-encoded scalars ("True", "42.5") reached RealOpcUaClient.WriteValuesAsync
as strings and were sent as String Variants, so a Boolean/Double node answered
BadTypeMismatch (0x80740000).
RealOpcUaClient now reads each written node's DataType attribute once per
session (batched, cached, cleared on connect/disconnect; derived types and
enumerations resolved via the supertype chain) and converts non-null scalar
values with the new OpcUaWriteCoercion table: Boolean, SByte..UInt64, Float,
Double, String, DateTime, Guid, all culture-invariant. Overflow, fractional
values to integer types and unparseable strings are refused client-side with
BadTypeMismatch and a message naming the value and target type - never a
rounded value. Arrays, null, unmapped types and nodes whose DataType read
fails pass through unchanged. WriteValueAsync now surfaces the outcome's own
status code instead of always BadNodeIdUnknown. MxGateway path untouched.
Tests: OpcUaWriteCoercionTests (coercion table incl. overflow, invalid and
culture-sensitive strings) and OpcUaWriteCoercionLiveTests (RequiresOpcUa:
String writes to Double/Boolean/UInt32 opc-plc nodes read back typed; reverting
the coercion reproduces 0x80740000). Component-DataConnectionLayer.md updated.
DeploySiteAsync's body moves into a private core taking the site plus an
optional area-id filter; site deploy behaviour is unchanged. The subtree is
one in-memory ParentAreaId walk (cycle-safe) shared by the deploy and the
preview. DeploymentService gains an ITemplateEngineRepository dependency for
the area lookups.
L1: an artifact apply holding the slot past 10 min (ctor-overridable) logs an
Error site event naming it without releasing the slot; PostStop answers
queued applies Failed. L2: a boot-batch instance whose actor cannot be created
emits an Error site event and leaves the deployed set. L3: a failed enable
reverts the committed is_enabled (generation-bumped both ends). L4: the routed
SetAttributes batch Ask is the 30 s per-write bound plus 10 s mailbox slack.
ConnectionActorNames.Sanitize (shared by the DCL manager and the site deploy
gate) keeps ASCII letters/digits and -_.*+:@&=,!~';() only. The manager fails a
connection whose sanitised name is invalid or already used (Error + site
event) instead of restarting; DeployCompileValidator rejects a config whose
connection names collide after sanitising.
HandleOverridesLoaded now sends children a site-local AttributeValueSeed
(stream still gets the AttributeValueChanged). ScriptActor updates its
snapshot, Conditional truth value and Expression last result (seed-only eval,
never fires); AlarmActor evaluates it as a normal change.
DeployCompileValidator.WarmCompileCache dedupes script bodies/trigger
expressions across a batch and compiles them with Parallel.ForEach (half the
cores). CreateInstanceActorBatch starts the next batch's warm as it arms the
stagger timer; creation order and delay are unchanged. The warm's parsed
configs are reused at creation.
The deploy compile warm parses once off-thread; the parsed FlattenedConfiguration
rides DeployCompileWarmed to the gate (new Validate(FlattenedConfiguration)
overload), the redeploy buffer, CreateInstanceActor's DCL ensure and a new
InstanceActor constructor overload. Stored-row paths parse once inside
CreateInstanceActor; an unparseable stored row still fails init via S6.
Adds SiteStorageService.GetAllDeployedConfigMetadataAsync. The sweep decides
from the projection, then fetches single rows for its starts/restarts off-
thread and re-validates them (generation guard, revision match, pipeline/
running guards), releasing the per-revision claim of any repair that does
not act. Boot load unchanged.
One Ask per attribute became one site-local InstanceGet/SetAttributesRequest
answered in a single pass, keeping the per-attribute GetAttributeResponse /
SetStaticAttributeResponse outcomes. The single-write path is refactored to a
Task<SetStaticAttributeResponse> outcome shared by both paths; static writes
and rejections still reply synchronously.
Each apply upserts its full set then deletes everything else, off the actor
thread; two overlapping commands interleaved and deleted artifacts central
never removed. The Deployment Manager now runs one apply at a time, queuing
the rest in arrival order; the slot is released by a self-message that lands
behind the apply's own DCL push.
SR-U1: children are created before the async override load lands, so their
trigger snapshots kept the deploy default forever after a restart/failover/
enable/sweep start. HandleOverridesLoaded now republishes every changed value
through PublishAndNotifyChildren(evaluateWaiters:false), and skips attributes
a static write touched before the load (newer than the stored row).
SR-M3: PostStop cancels each WaitForAttribute timeout and replies
'instance stopped' instead of leaving scripts to time out and timeouts to
dead-letter.
Joseph Doherty