Commit Graph

2144 Commits

Author SHA1 Message Date
Joseph Doherty 1dd89d523b fix(config-db): EnableRetryOnFailure fleet-wide + execution-strategy wrap of the combined-telemetry dual-write
Installs SqlServerRetryingExecutionStrategy (5 retries, 30s cap) so transient SQL
faults retry transparently on every read-side path. Manual transactions aren't
auto-retried, so AuditLogIngestActor's idempotent dual-write is wrapped in an
explicit CreateExecutionStrategy().ExecuteAsync to become retriable. Verified
empirically that existing manual transactions (BundleImporter) are unaffected —
EF skips the strategy while a transaction is already active.
2026-07-09 08:03:30 -04:00
Joseph Doherty 6b06d1efcb fix(site-call-audit): same-rank freshness tiebreaker — retrying calls no longer freeze RetryCount/LastError at first write
Within an equal NON-terminal rank the newest UpdatedAtUtc wins, unfreezing the
Attempted-phase RetryCount/LastError/HttpStatus. Terminal ranks (>=3) are
excluded from the tiebreaker so terminal immutability is preserved (Delivered
never overwrites Parked); equal stamps stay idempotent, lower rank stays a no-op.
2026-07-09 07:56:39 -04:00
Joseph Doherty 1b53de1933 fix(kpi-history): backlogTotal trend records the real site-backlog aggregate instead of a hardwired zero 2026-07-09 07:51:55 -04:00
Joseph Doherty 1dd993ec2f docs(archreview): sync PLAN-04 tasks.json — Tasks 7-9 completed (Batch 3, append-only-honesty docs + SiteCalls index + exec-tree window bound) 2026-07-09 07:11:26 -04:00
Joseph Doherty 5d0fcd4e66 perf(audit-log): bound GetExecutionTreeAsync edge scan to a root-anchored time window (partition elimination) 2026-07-09 07:10:35 -04:00
Joseph Doherty f1957606ff perf(site-call-audit): filtered IX_SiteCalls_NonTerminal index — KPI predicates seek the live queue instead of scanning 365 days 2026-07-09 07:06:50 -04:00
Joseph Doherty eb69b93947 docs(audit-log): document append-only enforcement honestly (CI guard default; DB roles are optional DBA hardening; purger now has CREATE TABLE + scoped DELETE) 2026-07-09 07:03:10 -04:00
Joseph Doherty 2a080a3e12 docs(archreview): sync PLAN-04 tasks.json — Tasks 4-6 completed (Batch 2, maintenance-timeout + purge-failure health + purger grants) 2026-07-09 06:53:46 -04:00
Joseph Doherty 7555e65746 fix(config-db): grant CREATE TABLE + scoped DELETE to scadabridge_audit_purger so a segregated maintenance principal can actually run the purge 2026-07-09 06:52:44 -04:00
Joseph Doherty 14c4df54f9 feat(audit-log): surface partition-purge failure as health event + counter 2026-07-09 06:49:28 -04:00
Joseph Doherty b59aa2d717 fix(audit-log): explicit long CommandTimeout on partition-switch and per-channel purge maintenance paths 2026-07-09 06:43:23 -04:00
Joseph Doherty 1b6b0aab2a docs(archreview): sync PLAN-04 tasks.json — Tasks 1-3 completed (Batch 1, site retention chain) 2026-07-09 06:22:42 -04:00
Joseph Doherty 74aa872c07 feat(audit-log): daily site SQLite retention purge job (closes unbounded site DB growth)
SiteAuditRetentionService (site-only IHostedService) ticks PurgeExpiredAsync on
InitialDelay(5m)/PurgeInterval(24h, clamped >=1m), cutoff = UtcNow - RetentionDays.
Registered site-only in AddAuditLogHealthMetricsBridge; options bound from
AuditLog:SiteRetention. Per-tick failures swallowed. Added a test-only
PurgeIntervalOverride (mirrors SiteCallAuditOptions) so the ms-cadence tick test
can bypass the 1-min production clamp. Doc updated. (PLAN-04 Task 3, S1/U2)
2026-07-09 06:22:19 -04:00
Joseph Doherty b5271da5c8 feat(audit-log): SiteAuditRetentionOptions (7-day default, clamped 1-90)
RetentionDays clamped [1,90]; PurgeInterval clamped to >= 1 min (Timer spin
footgun); 5-min InitialDelay so a daily-recycled node still purges. (PLAN-04 Task 2)
2026-07-09 06:13:05 -04:00
Joseph Doherty 731ee69797 feat(audit-log): add site SQLite PurgeExpiredAsync honoring the ForwardState invariant
Retention purge deletes Forwarded/Reconciled rows older than the cutoff and
reclaims pages via incremental_vacuum; Pending rows are never purged on age
alone (would lose audit data that never reached central). Temp-table purge set
avoids the SQLite 999-param limit; runs under _writeLock. (PLAN-04 Task 1, S1/U2)
2026-07-09 06:11:54 -04:00
Joseph Doherty 1586e791f7 docs(archreview): sync stale Status Board — PLAN-01 & PLAN-03 now Complete/merged (78/192)
The Status Board still showed PLAN-01 as 'branch not yet merged' and PLAN-03
as 'In progress, 0 done'; both are 100% complete and on main (PLAN-01
ec47cb56, PLAN-03 c7a0c922). Update the two rows, correct the stale
'NOT yet merged' PLAN-01 narrative, and add a PLAN-03 completion line with
the corrected initiative total (78 of 192).
2026-07-09 06:00:37 -04:00
Joseph Doherty 9049690bdb fix(centralui-tests): register stub IBrowseService in SecuredWritesTests harness (PLAN-07 known-issue resolved)
The Secured Writes page embeds NodeBrowserDialog (@inject IBrowseService) for
the OPC UA tag picker; bUnit resolves the injected service at render time, so
three SecuredWritesTests failed with 'no registered service of type
IBrowseService'. Register a Substitute.For<IBrowseService>() singleton in the
fixture (mirrors NodeBrowserDialogSearchTests). Suite now 7/7 green.

Marks the PLAN-07 'Post-plan known issues' entry and the 00-MASTER-TRACKER
registry cross-reference RESOLVED.
2026-07-09 05:55:53 -04:00
Joseph Doherty 765ff131d8 docs(archreview): log SecuredWrites IBrowseService bUnit-harness failures against PLAN-07 (pre-existing, surfaced during PLAN-03 full-suite run) 2026-07-09 05:46:29 -04:00
Joseph Doherty c7a0c92244 docs(archreview): sync PLAN-03 tasks.json — Tasks 25-26 completed (Batch 9, plan complete 26/26) 2026-07-09 02:13:13 -04:00
Joseph Doherty 1db0c32115 docs(requirements): sync SiteRuntime + DCL specs with hardening changes (C3/C4/C5 + S1-S6, P2/P4/P6, UA1/UA4 behavior notes) 2026-07-09 02:13:13 -04:00
Joseph Doherty c9ac075e39 fix(site-runtime,dcl): low-severity cleanups — observed adapter dispose (S9), alarm-filter overwrite warning (S10), bounded-channel comment (C2), invariant-culture condition fallback (C6) 2026-07-09 02:09:06 -04:00
Joseph Doherty 30886c9426 docs(archreview): sync PLAN-03 tasks.json — Tasks 22-24 completed (Batch 8) 2026-07-09 01:27:21 -04:00
Joseph Doherty 29ee9176a7 feat(site-runtime): wire ConfigFetchRetryCount into the standby replicated-config fetch (UA2) 2026-07-09 01:26:58 -04:00
Joseph Doherty c457e8f464 feat(site-runtime): cert-trust reconcile-on-join — singleton pushes trusted certs to (re)joining site nodes (UA1) 2026-07-09 01:22:13 -04:00
Joseph Doherty 39976772d9 feat(site-runtime): CertStoreActor exports trusted certs (thumbprint+DER) for node reconciliation (UA1 groundwork) 2026-07-09 01:15:51 -04:00
Joseph Doherty 667d863e58 docs(archreview): mark PLAN-03 Tasks 19-21 (Batch 7) completed in tasks.json 2026-07-09 01:10:18 -04:00
Joseph Doherty 2efe1a4202 feat(site-runtime): persist native-alarm display metadata so rehydrated conditions render fully (UA4) 2026-07-09 01:09:13 -04:00
Joseph Doherty 326d945d85 perf(site-runtime): coalesce native-alarm mirror persistence into batched flushes (P4) 2026-07-09 01:05:20 -04:00
Joseph Doherty d2feb92bfd fix(site-runtime): site SQLite runs WAL with an explicit busy-timeout floor (S8) — concurrent writers stop serializing on the rollback journal 2026-07-09 01:00:12 -04:00
Joseph Doherty dd16cd52b2 docs(archreview): mark PLAN-03 Tasks 16-18 (Batch 6) completed in tasks.json 2026-07-09 00:55:43 -04:00
Joseph Doherty d124025cc7 perf(site-runtime): per-attribute child routing in InstanceActor — attribute changes reach only interested children (P2) 2026-07-09 00:55:26 -04:00
Joseph Doherty 1673268aee perf(dcl): tag->instances reverse index replaces per-update all-instances scan (P3) 2026-07-09 00:48:22 -04:00
Joseph Doherty f294b46263 fix(dcl): capture adapter into locals before background tasks — no off-thread reads of the mutable _adapter field (S7) 2026-07-09 00:44:09 -04:00
Joseph Doherty ef69fab6f6 docs(archreview): register PLAN-03 Task 15 readiness-handshake deviation as a follow-up (owner: Task 26 doc capture) 2026-07-09 00:36:20 -04:00
Joseph Doherty faa9d34cd1 docs(archreview): PLAN-03 tasks.json — mark T13-T15 complete (Batch 5) 2026-07-09 00:33:53 -04:00
Joseph Doherty 957db62df1 fix(site-runtime): deployment whose Instance Actor dies during init reports Failed, exactly-once reply (S6)
Success now requires BOTH persistence commit AND an InstanceActorInitialized
readiness signal from the actor's PreStart — persistence can commit before the
actor's async init has run or failed, so persistence alone must not report
Success. An actor that dies during init never signals readiness; the Terminated
fallback fails that deployment and rolls back the optimistic state. The
persisted-row rollback is deferred until the store commits so it cannot race the
optimistic write.

Deviation from plan Task 15: the plan's swallow-only guard did not handle the
persistence-first ordering (empirically the store commits before the Terminated
signal, so Success was reported for a dead actor). Added the readiness handshake
to make the join deterministic.
2026-07-09 00:33:40 -04:00
Joseph Doherty db5ad02cc9 test(site-runtime): harden scheduler OldestBusyAge gauge assertion — assert non-null, not strictly-positive (TickCount64 ~15ms granularity flakes under load) 2026-07-09 00:33:40 -04:00
Joseph Doherty 9343b4f30f fix(site-runtime): watch every Instance Actor; unexpected termination evicts the dead ref from routing (S6) 2026-07-09 00:20:12 -04:00
Joseph Doherty 2dabbbdde0 fix(site-runtime): retry the startup deployed-config load on SQLite failure (S5) — no more silent zero-instance node 2026-07-09 00:17:36 -04:00
Joseph Doherty b126fda22e chore(archreview): PLAN-03 tasks.json — mark T10-T12 complete 2026-07-09 00:05:24 -04:00
Joseph Doherty fa556b4f8d fix(site-runtime): NativeAlarmActor retries a lost subscribe response, not only a failed one (S4) 2026-07-09 00:05:09 -04:00
Joseph Doherty feb6f34e90 fix(site-runtime): InstanceActor retries failed/lost tag subscriptions per connection (S4/UA6) — closes the unknown-connection ordering race 2026-07-09 00:03:44 -04:00
Joseph Doherty bd89c1474e perf(site-runtime): AlarmActor trigger expressions evaluate on the script scheduler (P1) 2026-07-08 23:59:54 -04:00
Joseph Doherty 6baf5a1d45 chore(archreview): PLAN-03 tasks.json — mark T7-T9 complete 2026-07-08 23:50:35 -04:00
Joseph Doherty 5aee90d155 perf(site-runtime): ScriptActor trigger expressions evaluate on the script scheduler, coalesced, edge state actor-confined (P1) 2026-07-08 23:50:21 -04:00
Joseph Doherty 467edd74ac feat(site-runtime): stuck-script watchdog names the script holding a scheduler thread past timeout (S2) 2026-07-08 23:46:48 -04:00
Joseph Doherty dddafc025e feat(site-runtime): periodic scheduler-stats reporter feeding site health (S2/UA5) 2026-07-08 23:43:09 -04:00
Joseph Doherty d44b6a0e52 chore(archreview): PLAN-03 tasks.json — mark T4-T6 complete 2026-07-08 23:38:07 -04:00
Joseph Doherty 7d50555fd3 feat(health): additive script-scheduler stats on ISiteHealthCollector + SiteHealthReport (S2/UA5) 2026-07-08 23:37:50 -04:00
Joseph Doherty 474ec8a221 feat(site-runtime): script scheduler gauges — queue depth, busy threads, oldest-busy age (S2/UA5) 2026-07-08 23:35:09 -04:00