Joseph Doherty
fa556b4f8d
fix(site-runtime): NativeAlarmActor retries a lost subscribe response, not only a failed one (S4)
2026-07-09 00:05:09 -04:00
Joseph Doherty
feb6f34e90
fix(site-runtime): InstanceActor retries failed/lost tag subscriptions per connection (S4/UA6) — closes the unknown-connection ordering race
2026-07-09 00:03:44 -04:00
Joseph Doherty
bd89c1474e
perf(site-runtime): AlarmActor trigger expressions evaluate on the script scheduler (P1)
2026-07-08 23:59:54 -04:00
Joseph Doherty
5aee90d155
perf(site-runtime): ScriptActor trigger expressions evaluate on the script scheduler, coalesced, edge state actor-confined (P1)
2026-07-08 23:50:21 -04:00
Joseph Doherty
467edd74ac
feat(site-runtime): stuck-script watchdog names the script holding a scheduler thread past timeout (S2)
2026-07-08 23:46:48 -04:00
Joseph Doherty
dddafc025e
feat(site-runtime): periodic scheduler-stats reporter feeding site health (S2/UA5)
2026-07-08 23:43:09 -04:00
Joseph Doherty
7d50555fd3
feat(health): additive script-scheduler stats on ISiteHealthCollector + SiteHealthReport (S2/UA5)
2026-07-08 23:37:50 -04:00
Joseph Doherty
474ec8a221
feat(site-runtime): script scheduler gauges — queue depth, busy threads, oldest-busy age (S2/UA5)
2026-07-08 23:35:09 -04:00
Joseph Doherty
b83d1ed19c
fix(site-runtime): site-side compile failure now rejects the deployment per spec (S3) — synchronous validation gate before actor creation
...
Validation runs on the actor thread (not off-thread as the plan sketched): piping
the verdict back to self reorders concurrent deploys relative to each other and to
delete/disable, breaking the redeploy-supersede FIFO ordering (SR020/SR029). A deploy
is an infrequent admin command, so a brief synchronous Roslyn compile is acceptable.
2026-07-08 23:33:48 -04:00
Joseph Doherty
e20c329cad
feat(site-runtime): pure deploy-time compile validator for flattened configs (S3 groundwork)
2026-07-08 23:21:42 -04:00
Joseph Doherty
a8a01f026b
fix(dcl): cancel old event loop + dispose old MxGateway client on reconnect (S1)
2026-07-08 23:19:11 -04:00
Joseph Doherty
11eb5157e6
fix(dcl): dispose+detach previous OPC UA client on reconnect (S1) — stops session leak and stale ConnectionLost flapping
2026-07-08 23:16:27 -04:00
Joseph Doherty
ae94b22d7d
test(reconciliation): pin inclusive (>=) cursor reads at the site (audit + site-calls); EventId/upsert dedup absorbs boundary duplicates
2026-07-08 21:58:57 -04:00
Joseph Doherty
846279f6a8
chore(communication): low-severity cleanup — eviction counting, shared ingest timeout, split app heartbeat interval, doc notes
2026-07-08 21:54:42 -04:00
Joseph Doherty
11c72ecf0f
feat(health): oldest-parked-message age on the site health report (parked-retention aging signal)
2026-07-08 21:49:29 -04:00
Joseph Doherty
0321ad0c20
feat(store-and-forward): peer-join full-buffer resync — standby anti-entropy for the S&F buffer
2026-07-08 21:43:11 -04:00
Joseph Doherty
3eef9d1077
feat(store-and-forward): GetAllMessagesAsync/ReplaceAllAsync storage primitives for standby buffer resync
2026-07-08 21:22:25 -04:00
Joseph Doherty
c6ea332c94
perf(store-and-forward): post audit-observer notifications to an ordered single-reader channel, latency-isolating the sweep
2026-07-08 21:19:59 -04:00
Joseph Doherty
5fe1b9747b
perf(store-and-forward): epoch-ms due-check with additive column, backfill, and (status, due) index
2026-07-08 21:14:26 -04:00
Joseph Doherty
499a37ac78
perf(store-and-forward): enable WAL + per-connection busy_timeout on the S&F SQLite store
2026-07-08 21:01:53 -04:00
Joseph Doherty
7626d5ba34
test(communication): pin wire round-trip + type identity of cross-cluster message contracts
2026-07-08 21:00:02 -04:00
Joseph Doherty
4bdd5f0e4a
fix(notifications): Notify.Send enqueues unbounded (maxRetries 0) and defers to sweep — no 30s script-thread block, no stranding
2026-07-08 20:56:09 -04:00
Joseph Doherty
c3a8576863
fix(notifications): park corrupt buffered notification payloads instead of silently discarding as delivered (supersedes StoreAndForward-018)
2026-07-08 20:48:16 -04:00
Joseph Doherty
05a71a0260
feat(store-and-forward): deferToSweep enqueue mode with immediate sweep kick for latency-sensitive callers
2026-07-08 20:46:08 -04:00
Joseph Doherty
283ce0647c
fix(store-and-forward): standby applies Add/Park/Requeue as upserts so a lost Add self-heals
2026-07-08 20:42:39 -04:00
Joseph Doherty
66cb0febad
fix(communication): replicate heartbeat marks to the peer central node (closes post-failover offline blind window)
2026-07-08 20:11:54 -04:00
Joseph Doherty
59d9cba621
perf(store-and-forward): parallel per-target sweep lanes (cap 4), sequential within lane
2026-07-08 20:07:30 -04:00
Joseph Doherty
1459f9407f
perf(store-and-forward): short-circuit a (category,target) lane after its first transient failure per sweep
2026-07-08 20:06:06 -04:00
Joseph Doherty
1a53b8082a
perf(store-and-forward): bound retry sweep with SweepBatchLimit (default 500) on the due-rows query
2026-07-08 17:45:11 -04:00
Joseph Doherty
e75cd6f3d8
fix(communication): debug-stream teardown/failover unsubscribes via TryGet, never creates or disposes shared channels
2026-07-08 17:43:40 -04:00
Joseph Doherty
70e7dfd7b9
fix(store-and-forward): inline ordered replication dispatch; Warning + counter on replication failures
2026-07-08 17:35:29 -04:00
Joseph Doherty
ca5e79e922
fix(communication): key gRPC client cache by (site, endpoint) so per-session failover cannot dispose shared channels
2026-07-08 17:33:03 -04:00
Joseph Doherty
76c10c5de6
fix(store-and-forward): gate the retry sweep behind an active-node delivery gate (standby must be passive)
2026-07-08 17:30:21 -04:00
Joseph Doherty
48e97fee01
feat(host): exit process on unexpected ActorSystem termination — completes the down-if-alone recovery loop
...
WhenTerminated watchdog calls IHostApplicationLifetime.StopApplication() when
the ActorSystem dies outside StopAsync (SBR self-down + run-coordinated-shutdown-when-down),
so the service supervisor restarts the node as a fresh incarnation. Optional
ctor param keeps every existing construction site compiling.
Plan's deploy/wonder-app-vd03/install.ps1 service-recovery-actions edit is
skipped: that production deploy artifact is not tracked in this repo (Task 16
skipped its appsettings.Central.json for the same reason).
2026-07-08 16:56:24 -04:00
Joseph Doherty
dea69842d5
fix(host): rate-limit dead-letter warnings (10/min + suppression summary); metric counting unchanged
2026-07-08 16:52:25 -04:00
Joseph Doherty
ce66e194a8
feat(cluster): AllowSingleNodeCluster flag — single-seed installs drop the phantom seed
...
Validator lowers the seed-count floor to 1 when AllowSingleNodeCluster is set,
otherwise still requires 2 and points operators at the flag. The plan's
deploy/wonder-app-vd03/appsettings.Central.json edit is skipped: that production
deploy artifact is not tracked in this repo (only its deployment record doc is).
2026-07-08 16:39:12 -04:00
Joseph Doherty
0e3c1df1a7
fix(host): dev site seed no longer targets the metrics port; validator rejects seed-vs-MetricsPort
2026-07-08 16:37:43 -04:00
Joseph Doherty
d962c77bb7
fix(health): evict deleted sites from the aggregator on the periodic site refresh
2026-07-08 16:27:38 -04:00
Joseph Doherty
87c7255912
feat(ui): health dashboard shows metrics-stale badge and offline-since timestamp
2026-07-08 16:23:43 -04:00
Joseph Doherty
c73b7faa11
feat(health): metrics-stale signal + status-transition timestamps; spec now matches heartbeat-liveness code
2026-07-08 16:21:30 -04:00
Joseph Doherty
6300d6d399
fix(health): acked SendAsync transport — report-loss counter restore is live, not dead code
2026-07-08 16:12:48 -04:00
Joseph Doherty
17af376a8e
feat(comm): ack site health reports end-to-end (SiteHealthReportAck, additive contract)
2026-07-08 16:09:08 -04:00
Joseph Doherty
f7b9d342e4
refactor(host): central singletons via CentralSingletonRegistrar — outbox/audit-ingest gain drain tasks
2026-07-08 16:05:52 -04:00
Joseph Doherty
c255ec31c9
feat(host): CentralSingletonRegistrar — canonical singleton manager+proxy+drain helper
2026-07-08 15:59:01 -04:00
Joseph Doherty
7138d47630
test(cluster): restarted original node stays standby — oldest-member active semantics proven (graceful-leave variant per 2-node keep-oldest gap)
2026-07-08 15:57:45 -04:00
Joseph Doherty
f164f62a8b
fix(cluster): give cluster-leave phase a 15s timeout so 10s singleton drains can complete
2026-07-08 15:55:06 -04:00
Joseph Doherty
ff89e3145c
fix(host): /health/active = oldest member + DB reachable — partition-safe Traefik routing
2026-07-08 15:51:56 -04:00
Joseph Doherty
da6d5ae12c
fix(host): unify active-node on oldest-member semantics — purge/self-report/labels now track the singleton host
2026-07-08 15:48:07 -04:00
Joseph Doherty
6320be9954
feat(host): ClusterActivityEvaluator — oldest-member (singleton-host) active-node definition
2026-07-08 15:43:13 -04:00
Joseph Doherty
099728f05a
fix(transport): wire template inheritance edges on bundle import (C3) — derived templates no longer land as roots
2026-07-08 15:17:47 -04:00