Task 1 of the Phase 2 plan. The gate stops the plan, but not for the reason it
anticipated: oplog sizing is fine and D6 is resolved.
BLOCKER: the Phase 1 consolidated LocalDb (site-localdb.db) throws SQLite
Error 10 'disk I/O error' on essentially every write on the ACTIVE node under
sustained load — site_events, OperationTracking and the audit telemetry paths
all fail, and site event logging silently drops events. Isolated to the load
(not the node, not host-side observation) by failing over between nodes, and to
LocalDb specifically (legacy store-and-forward.db / scadabridge.db in the same
bind-mounted directory take zero errors under identical load).
Phase 2 would register 8 more tables into that database — including the two
highest-volume ones — while deleting the bespoke mechanisms that currently
carry them. Must be root-caused first.
Also recorded: D6's premise corrected (largest known production config_json is
~60-70 KB, not >128 KB — but MaxBatchSize 500 is still unsafe, use 16); D4's
premise corrected (alarm writes are bounded by per-SourceReference coalescing at
a 100 ms flush); sf_messages has a hard 50 rows/sec structural ceiling; and
exceeding the oplog caps is a graceful snapshot-resync, not a failure.
Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
Three verification sweeps over SiteRuntime, StoreAndForward, and Host/rig/docs
plus the LocalDb library source. Load-bearing corrections: D1 (the guarded write
has a second surviving caller, SiteReconciliationActor), D3 (the active node
already purges — Task 12 becomes a pin), D6 (new: the 4 MB gRPC cap vs
row-count-only batching), Task 1 (rewritten method — the Phase 2 tables are not
in the Phase 1 oplog), and Task 14 (do not register the notification/SMTP tables).
Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
Moves scadabridge.db's 9 config tables + sf_messages into the consolidated
LocalDb file and deletes SiteReplicationActor + StoreAndForward's
ReplicationService.
Resolves the phase 2 gate's five open questions from code recon (D1-D5):
D1 notify-and-fetch is DELETED, not preserved. It exists only because the
config blob exceeds Akka's 128KB frame; LocalDb sync is gRPC. The
deployed_at version guard protected against a stale fetch racing, so it
dies with the fetch. Do not reproduce it on LWW - different clocks.
D2 ReplaceAllAsync is deleted and the N1 directional guard becomes
unnecessary: LocalDb's snapshot resync merges per-row LWW and never
deletes (SnapshotApplier has no DELETE; LwwApplier.cs:69-78 discards a
lower-HLC incoming row). Semantic change - the standby is convergent,
no longer byte-identical.
D3 The SMTP purge (plaintext passwords) rides the replication path and is
re-homed to the active node BEFORE any deletion.
D4 native_alarm_state volume is measured by a rig soak, not assumed. Task 1
gates the plan and stops it if the oplog cannot absorb the churn.
D5 No dual-mechanism period forecloses rolling site upgrades - both nodes
must stop and start together.
Recon also found the gate doc's "two test files are the spec" undercounts:
the real specification is five files, including the N1 Critical regression
test and the only Requeue coverage.
Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
Task 14. A GATE document, not an implementation plan - the plan explicitly wants
Phase 2 planned against post-Phase-1 reality, which means after this record
exists rather than from the original design alone.
Records what Phase 1 actually established (ordering constraints, where
registration lives and why, the fail-closed interceptor, the silent meter
allowlist), what must be ported before anything is deleted (both bespoke
replicator test files, treated as specifications, plus the accepted N5
bounded-duplicate race), and why Phase 2 is a harder risk class than Phase 1:
it REPLACES a working mechanism rather than adding one where none existed,
sf_messages has external side effects, config tables drive deployment, there is
no dual-mechanism period, and the migration is real this time rather than the
usual no-op.
The most important line is an open question, not an answer: choosing
MaxOplogRows / MaxOplogAge / snapshot-resync thresholds for sf_messages write
rates needs a Phase-1 rig SOAK that has not been run. The live gate proved
correctness, not steady-state behaviour over time. Phase 2 should not be planned
without it.
tasks.json finalized: all 15 tasks completed, every deviation from the plan
recorded with its reason.
Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
Execution review against the code found the integer site_events.id is
load-bearing in three places the original Task 5 did not list:
- keyset cursor EventLogQueryService.cs:70,141,153 (id > $afterId)
- storage-cap purge EventLogPurgeService.cs:164 (ORDER BY id ASC)
- wire contracts EventLogEntry.Id, both ContinuationTokens (long)
cross the site<->central Akka boundary
A GUID PK against the existing cursor silently drops rows, and against the
purge deletes arbitrary rows instead of the oldest. Task 5 is therefore split
into 5a (writer, standard) and 5b (read path + DTOs, high-risk), with 5b's
full file set spelled out and Task 10 gated on it.
Also recorded: Task 4's real registration site, and that both legacy DB paths
are CWD-relative and unset in docker today (so they live outside the data
volume and are already lost on container recreate - Phase 1 fixes that).
Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
Gitea #18: the process-wide ScriptExecutionScheduler was reached via a static
singleton (Shared) directly inside ScriptActor, ScriptExecutionActor, AlarmActor,
AlarmExecutionActor, and ScriptSchedulerStatsReporter, so it could not be
substituted — a shared mutable global for anything running more than one logical
site in a process, most visibly the test assembly. Add an optional scheduler
injection seam to all five: the Host injects nothing and gets the shared pool
(byte-for-byte unchanged), while a test (or a future multi-site host) hands an
actor its own instance. Spawning actors thread their scheduler down to the
execution children they create. Shared() now also recreates a disposed cached
instance rather than returning it (new IsDisposed guard), so a disposed scheduler
can never silently poison every later script/alarm execution.
Gitea #16: ScriptActorTests now runs on its OWN scheduler (disposed at class
teardown), so the faulted-task test can no longer strand a worker on the
process-wide pool and starve unrelated test classes (one prior run failed 22
tests). EvalGate's wait is bounded to 30 s (was unbounded — the actual leak
mechanism) and reset per test; the teardown releases one permit per worker
instead of Release(Entries), which under-released in exactly the failure case.
Full SiteRuntime suite: 6/6 runs green (524/524); was 3/6 failing on clean main.
Fixes: #18Fixes: #16
A native OPC UA alarm source's SourceReference has to be two things at once:
it is parsed as a NodeId to open the monitored item, and matched as a plain-name
prefix against the event's SourceName to route the transition to an instance. No
string is both, so a NodeId-form binding subscribed correctly and then silently
dropped every transition — "pymodbus/plc/HR200".StartsWith("nsu=...;s=pymodbus/plc/HR200")
is false. Only the empty (Server-object) binding worked, because StartsWith("")
matches everything, which is why the sole live smoke test never caught it.
Each OPC UA alarm feed is opened for exactly one binding, so every transition on
it belongs to that binding. The adapter now tags each transition's routing
identity (SourceObjectReference) with the binding string verbatim via the pure
OpcUaAlarmMapper.BuildIdentity, making DataConnectionActor's routing key an exact
match regardless of whether the binding is stored as ns=<index> or the durable
nsu=<uri> form. The Server-object aggregate feed keeps an empty routing identity,
so it reaches only "mirror everything" subscribers and never leaks into a
specific-node binding. The per-condition SourceReference key stays the readable
SourceName.ConditionName, so persistence and display are unchanged, and MxGateway
is untouched — its bindings are names and its mapper already emits matching names.
Unblocked by lmxopcua#473 (OtOpcUa now populates SourceNode/SourceName/EventType
on conditions); SourceName is the RawPath, so the per-condition key is unique.
Live end-to-end verification against native alarms still needs a v3 rig.
Fixes: Gitea #17
A stored ns=<index> reference is only meaningful against one server's namespace
table. A server that adds, removes or reorders a namespace silently re-points
every binding: best case BadNodeIdUnknown, worst case the index now names a
different namespace holding a colliding identifier and the binding resolves to
the wrong node with Good quality. Nothing could detect that, because ScadaBridge
stored no namespace URI anywhere.
OtOpcUa v3.0 makes this concrete: v2's sole custom namespace and v3's raw
namespace both sit at index 2, so v2-era bindings resolve against v3 without
error while meaning something else entirely.
Adds OpcUaNodeReference as the single translation seam between stored references
and the wire. Resolve() accepts both ns=<index>;s=<id> (existing bindings, which
keep working unchanged) and the durable nsu=<uri>;s=<id>, mapping the URI to the
live index at use time; it is wired into every parse site — subscribe, read,
write, alarm-subscribe and browse. An unpublished URI now throws naming the URI
rather than binding to whatever occupies that index, and a svr= reference to
another server is rejected instead of being resolved against the wrong address
space.
The browser emits ToDurable(), so what the picker shows is what gets stored and
newly-authored bindings are index-proof from the start. That also closes a
round-trip gap: browse previously emitted ExpandedNodeId.ToString(), which for a
URI- or server-index-carrying reference produced a string NodeId.Parse could not
read back — the same method already resolved it correctly 57 lines later.
Bindings stored before this change keep their ns= form and keep working; they are
only as durable as the server's namespace order. Re-authoring against the picker
is what makes them durable, and that re-bind still needs a live v3 rig.
Refs: Gitea #14
Ports the legacy WWSupport/APIServer MES alarm-status endpoints onto the
Inbound API, mirroring the MesMoveIn/MesMoveOut pattern; draft for review,
not yet executed. Committed ahead of PLAN-R2-08 T13's hygiene pass.
Alarm Summary is now live-cache-driven via a transient per-site central
live alarm cache (ISiteAlarmLiveCache) fed by a site-wide, alarm-only
SubscribeSite gRPC stream (seed-then-stream), with the 15s poll retained
as fallback + NotReporting authority. No persisted central alarm store
([PERM]). Updated Component-CentralUI, Component-Communication (new §6.1
+ SubscribeSite RPC), M7 design (T13 delivered note), native-alarms
[PERM] note, stillpending follow-up note, CLAUDE.md, and moved register
row #10 to Resolved (left #22 in place for the sibling branch).
Claude-Session: https://claude.ai/code/session_01MtdgwpEeCUn6cUA5f1LMPj
Update Component-KpiHistory.md with the KpiRollupHourly schema, the third
recorder rollup tick + one-shot backfill, dual raw/rollup purge retention,
raw-vs-rollup query routing by RollupThresholdHours, per-metric gauge-vs-rate
aggregation (KpiMetricAggregationCatalog), the four new options + bounds, and
30d/90d trend windows. Append a Delivered-2026-07-10 note to the m6 design
plan, move register row #22 from Deferred to Resolved, and note rollups in the
CLAUDE.md KPI History bullet. Docs-only; no code changed.
Claude-Session: https://claude.ai/code/session_01MtdgwpEeCUn6cUA5f1LMPj
Two draft task-by-task plans for the two Scale/YAGNI deferrals whose revisit
triggers are now in view:
- #10 aggregated live alarm stream: transient per-site in-memory central live
cache seeded by the snapshot fan-out + additive SubscribeSite alarm-only gRPC
stream, pushed via the IDeploymentStatusNotifier pattern; honors the [PERM]
no-central-alarm-store rule. 8 tasks.
- #22 KPI hourly rollups: separate KpiRollupHourly table, recorder hourly fold
with failover-safe lookback re-fold, per-metric gauge-vs-rate aggregation,
range-threshold routing in the query service, longer rollup retention, one-shot
backfill, plus 30d/90d window buttons so rollups have a caller. 8 tasks.
Claude-Session: https://claude.ai/code/session_01MtdgwpEeCUn6cUA5f1LMPj
Consolidates the reporter's poll cadence onto SqliteAuditWriterOptions instead of
the hard-coded 30 s constant, closing deferred-work register item #21 (the config-
shape cleanup its own XML doc flagged as a follow-up).
- SqliteAuditWriterOptions.BacklogPollIntervalSeconds (default 30).
- Reporter reads it via IOptions; precedence explicit-override (tests) > configured
> 30 s default; a non-positive value falls back to the default. Corrected the
stale "hard-code / tunable in a follow-up" class-doc.
- Cadence tests + register updated (row 21 -> Resolved).
Claude-Session: https://claude.ai/code/session_01MtdgwpEeCUn6cUA5f1LMPj
Closes the operator parity gap the deferred-work register tracked as #12: native
alarm source overrides could only be set one-at-a-time, while attribute overrides
had a CSV bulk path. Adds an all-or-nothing CSV import for native sources.
- Commons: extract the RFC-4180 line splitter into a shared CsvLineSplitter
(refactor OverrideCsvParser onto it — no behavior change, pinned by its tests);
new NativeAlarmSourceOverrideCsvParser (header SourceName,Connection,
SourceReference,Filter; blank = inherited).
- Commons: NativeAlarmSourceOverrideEntry + bulk SetInstanceNativeAlarmSource-
OverridesCommand (auto-registered via the reflection command registry).
- ManagementService: Deployer-gated handler — flattens once, validates every
source resolves + is unlocked + no duplicates up front, then upserts the whole
batch under a single SaveChanges (true all-or-nothing txn). Added to the frozen
authorization matrix; dispatch-coverage guard passes.
- CLI: `instance native-alarm-source import --instance-id --file` (parity with
`instance import-overrides`) + README.
- Tests: native parser (Commons), CLI parse/entry mapping, 3 bulk-handler tests
(happy path single-commit, locked-source reject, unresolved-source reject).
Also corrects a stale XML-doc line in ScriptRuntimeContext (WaitForAttribute
quality-gated mode is shipped, not "planned") and updates the deferred-work
register: marks #7/#13/#15/#16/#20 verified-resolved, #12 as CLI/API-shipped with
only the Central UI upload affordance still pending.
Claude-Session: https://claude.ai/code/session_01MtdgwpEeCUn6cUA5f1LMPj
Joseph Doherty