HistorianAdapterActor fired `_ = EnqueueAsync(...)` per alarm transition
with no in-flight cap, so an alarm storm (a native source re-raising
hundreds of conditions on reconnect) spawned hundreds of concurrent SQLite
inserts that serialised on the file lock and held pool threads. It also kept
a Receive<AlarmHistorianEvent> with no production sender: a transition told
directly AND published on `alerts` would have been historized twice.
Transitions now go to a bounded channel (10000) drained by one loop, so the
sink sees one enqueue at a time in arrival order; a full queue holds the
mailbox (backpressure, nothing dropped); PostStop flushes the queue within
the shared 5 s stop budget. The direct receive is deleted. The gate tests
drive transitions instead; a directly-told AlarmHistorianEvent is shown to
be unhandled against a positive control.
Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
The continuous-historization recorder awaited every outbox append (an fsync
under PerEntry) on its mailbox, so ingestion was capped at 1/fsync_latency
bursts per second while the mux kept telling into an unbounded, unmetered
mailbox. GetStatus also suspended the mailbox on an unbounded outbox count.
- Gated ingest bursts go to a bounded channel (64 bursts) drained by one
ordered append loop, which commits every burst waiting behind a slow
append in a single AppendBatchAsync (FIFO kept). Only a full queue holds
the mailbox (backpressure, bounded memory). Counters are Interlocked; the
loop nudges the drain. PostStop flushes queued bursts within the shared
5 s stop budget.
- GetStatus reads counters on the actor thread and pipes a count bounded by
a 10 s timeout (-1 when it does not answer); it no longer suspends the
mailbox. RecorderStatus gains PendingAppends, MailboxDepth and
AppendBackpressureWaits; two gauges publish pending appends and mailbox
depth.
- ContinuousHistorization:CommitMode defaults to Periodic (100 ms). The
Host's appsettings.json, its unparseable-value fallback and docker-dev
still say PerEntry: filed as a Host follow-up (outside this lane).
Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
The QuestDB live writer waited for WAL visibility after every tag group, and
the wait polled wal_tables() every 10 ms. With DrainConcurrency = 8 that was
up to eight pollers at ~100 queries/s each against a database whose WAL
apply was already behind.
New Runtime seam IHistorianPassWriter (WriteLiveValuesInPassAsync +
FlushAsync). The recorder writes every group of a pass through it and calls
FlushAsync once after all groups report, before the ack, whenever any group
wrote (an un-acked pass is redelivered and its insert-if-absent check must
see this pass's rows). A flush failure does not fail the pass. The QuestDB
writer implements it: in-pass writes only mark a flush as owed; FlushAsync
waits once. The poll now backs off 10 -> 20 -> ... -> 200 ms, never past the
timeout. Direct WriteLiveValuesAsync callers still wait per call.
Also updates the QuestDB cancelled-write integration test to the rethrow
contract from the previous commit.
Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Akka TestKit's AwaitCondition without an interval polls every max/10 of the
DILATED budget. Measured here: a condition that turned true at 2 s under a
15 s budget returned after 3003 ms at timefactor 1 (3 polls) and 4502 ms at
timefactor 3 (2 polls). AwaitAssert / AwaitAssertAsync are unaffected (a
fixed 100 ms: 21 polls in ~2 s at both factors).
RuntimeActorTestBase and ControlPlaneActorTestBase now declare
AwaitCondition overloads that hide TestKit's and default the interval to
PollInterval (20 ms), so every subclass call without an interval binds to
them (measured after: returns 5-22 ms after the condition holds, at both
factors). PresenceBudget's doc no longer claims the helpers return "the
instant the condition holds" - it states the per-helper poll behaviour.
Bare-TestKit classes with AwaitCondition calls get an explicit interval:
TelemetryDialDeadlineTests / TelemetryDialSupervisorTests (next to their own
undilated 5 s / 8 s WaitForCount deadlines and a 1 s dial deadline) and
OpcUaEndpointStepDownActorTests (RuntimeActorTestBase.PollInterval, now
public like PresenceBudget). ContinuousHistorizationRecorderTypedValueTests'
AwaitConditionAsync was left alone (historian code is being changed in
parallel).
Suite results (Release, 0 warnings):
- Runtime.Tests 946/3 skipped/0 failed at timefactor 3 (1 m 45 s, was
2 m 57 s) and 1 (1 m 18 s, was 2 m 57 s);
- ControlPlane.Tests 319/0 failed at 3 (1 m 51 s) and 1 (1 m 11 s);
- AdminUI.Tests Telemetry 15/0 failed at 3 and 1.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
DriverInstanceActor.RunWriteAsync cancelled every write with a hard-coded,
undilated 5 s CancellationTokenSource, so writeTimeout / DefaultWriteTimeout
(then 10 s) only ever served as the pessimistic abandon bound: a configured
bound above 5 s never took effect.
The write's token is now cancelled at the configured write timeout; a write
that ignores its token is abandoned WriteAbandonGrace later (half the
timeout, at most 2 s) so a token-honouring driver still gets to answer the
cancellation itself. Both bounds run on an injectable TimeProvider
(Props writeClock, System by default).
DefaultWriteTimeout is now 5 s (cancel 5 s, abandon 7 s), not 10 s: the
host's write Ask is 8 s, and a write allowed to run past it would already
have been reported as a timeout and its node reverted while the device could
still take the value - the exact failure the off-mailbox write work fixed.
Production behaviour is therefore unchanged for the cancel (5 s) and tighter
for a token-ignoring driver (abandoned at 7 s instead of 10 s, i.e. inside
the Ask). DriverHostActor's Ask comment updated to match.
Tests (DriverInstanceActorConcurrentWriteTests, manual clock copied from
Ab.Transport.Tests into Runtime.Tests/Harness/ManualTimeProvider):
- a write held past 6 s under a 10 s write timeout succeeds;
- positive control: at 10 s the token is cancelled ("write timeout");
- a token-ignoring write is still pending at 10 s and abandoned (BadTimeout)
after the grace.
Mutation checks: a fixed 5 s token on the same clock fails the first test;
dropping the grace fails the third (and the second, where the abandon then
races the cancellation).
342a825d re-check: its held-write tests were written around the undilated
5 s cap. Connect() now defaults the write timeout to the DILATED presence
budget, so a held, explicitly-released write no longer races any undilated
deadline; tests about the bound pass their own. AwaitWritesEntered keeps its
20 ms interval (doc updated).
Runtime.Tests: 946 passed / 3 skipped / 0 failed at AKKA_TEST_TIMEFACTOR 3
and 1. Solution Release build: 0 warnings.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
A_queued_write_older_than_the_budget_is_failed_BadTimeout_instead_of_written_late
failed hosted runs 2215 (7ec5b95d) and 2260 (8ef5adc6) at
"(await inFlight).Success should be True but was False", after 7 s each time.
It is not load: it fails on an idle machine with AKKA_TEST_TIMEFACTOR=3, the
value v2-ci sets, and passes with 1.
Akka's AwaitCondition with no interval polls every max/10 of the dilated
budget: PresenceBudget 15 s x 3 = 45 s, so a 4.5 s sleep after a first poll
that runs before the pool has entered the write. The held write is bounded by
DriverInstanceActor.RunWriteAsync's fixed, undilated 5 s
CancellationTokenSource, so 4.5 s of polling plus the test's 600 ms settle
cancelled it ("write timeout") before ReleaseWrites(). The sibling tests that
hold a write took 7 s for the same reason and passed only by margin.
All eight AwaitCondition calls in the class now go through
AwaitWritesEntered, which polls every 20 ms. The budget (PresenceBudget)
and every assertion are unchanged.
Local, AKKA_TEST_TIMEFACTOR=3: before 6/7 (this test failing, 7 s); after
7/7 in three runs, test 3 s.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Benchmark wall-clock budget 1 s -> 5 s (measured 5-60 ms; still catches O(N^2)). Host-status wait is already a TCS signal; its 30 s ceiling is raised to 120 s so a starved box cannot time out before the keep-alive reports the loss.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
CSV: IsArray with blank/0 ArrayLength is the variable-length shape browse-commit and the composer produce, so its own export now re-imports.
Plain MQTT: MqttTagDefinitionFactory.Inspect (the driver's parse, consulted by the authoring inspector) now advises when isArray is set on a Plain-shaped tag; Sparkplug-shaped blobs stay quiet.
Live Sparkplug array test and SDK reflection items remain open.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
The #733 miss budget (MissBurst / MissRefillPerSecond) was process-wide, because
the impersonation seam did not carry the client's address, so one noisy client
could drain it for everybody while the known-id set was unavailable.
- IOpcUaUserAuthenticator gains AuthenticateUserNameAsync(username, password,
IPEndPoint? remoteEndpoint, ct) (default: ignore it and delegate); the server
calls it. IApiKeyStore gains the matching FindAsync(..., IPAddress?, ...).
- The SDK (1.5.378) exposes no client address publicly. SecureChannelRemoteEndpoint
reads it from the TCP listener's private channel map by reflection (through the
protected TransportListeners, exposed by OtOpcUaSdkServer), keyed by the
session's secure channel id. Fail-soft: null => process-wide accounting only.
- BoundedApiKeyStore charges a miss to the client's own bucket (keyed by IP
address) and then to the process-wide one, which stays as the backstop. A
refusal by either takes from neither; a hit refunds both. New options
Security:ApiKeys:MissBurstPerEndpoint (10) / MissRefillPerSecondPerEndpoint (2),
0 disables; telemetry outcome endpoint_miss_budget. At most 4096 addresses are
tracked; idle buckets are pruned and an address beyond the cap is charged
process-wide only.
Tests: BoundedApiKeyStorePerEndpointTests (noisy client vs quiet one, process-wide
backstop, refund + refill, off switch, authenticator forwards the address) and
over the wire ApiKeyMissBudgetPerEndpointTests (activation hands the authenticator
the loopback endpoint; real authenticator + bounded store refuse a client's 3rd
and 4th unknown key without a store read). Each new behaviour test fails with its
fix mutated out. Host.Tests 224/0, OpcUaServer.Tests 1019/1 (the one failure is the
wall-clock NodeManagerLockTimeBenchmarkTests under suite load; passes alone).
Docs: security.md.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Residuals of #760:
1. An ack routed to a disabled / not-yet-spawned driver, or to a stubbed child
(driver does not run on this node, or its factory rejected the config), was
dropped and left the condition Acknowledged. DriverHostActor now reverts it
through the same NativeAlarmProjector.TryRevertAcknowledge path as a refused
ack.
2. A native ack arriving while the host is Stale was dropped before the projector
recorded it, so it could never be reverted. Stale now marks the whole
NativeAlarmAckPreflightView unavailable (SetHostUnavailable), so the node
manager refuses every native ack synchronously; BecomeSteady clears it.
3. (keyed sessions without a Galaxy identity) is not addressed.
Tests: DriverHostActorNativeAlarmTests' client-ack test and the FOCAS respawn
theory now start the host's own driver (AckingAlarmDriver; a factory building a
real FocasDriver per spawn over per-instance fake CNCs, with a real config-change
respawn) - both fail against the fix in their old shape. New: ack to a disabled
driver reverts, ack to a stubbed driver reverts, a Stale host refuses through the
pre-check view - all three fail with the fix mutated out. Runtime.Tests 943/0/3.
Docs: AlarmTracking.md, .claude/rules/alarms.md.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
R06:M-5 made Sparkplug *Array leaves unbindable because the ingestor refused
them. The ingestor now decodes the packed bytes_value of all 13 array types
(new SparkplugArrayCodec: LE fixed widths, int64-ms DateTime, uint32-count +
MSB-first BooleanArray, NUL-terminated UTF-8 StringArray; golden vectors are
the spec's examples, reproduced byte-for-byte by SparkplugNet 1.3.10) and
publishes a typed 1-D array through the scalar coercion, all-or-nothing.
Shape uses the fleet's one array model: the tag's platform isArray/arrayLength
keys (ValueRank=1, ArrayDimensions=[arrayLength] or [0] = variable length).
The driver reads them with TagConfigIntent's exact semantics and never rejects
on them, so composer, driver, authoring model and deploy inspector agree
(MqttArrayShapeParityTests). Shape mismatch (array metric on a scalar tag, or
the reverse) is BadTypeMismatch - a String tag no longer gets the packed
buffer as base64; a malformed body or a length differing from a declared
arrayLength is BadDecodingError (MTConnect #627 verdict). Template/DataSet
members of an array type take the same path. Arrays stay read-only (encoder
refuses them; MQTT is not in DriverArrayWriteSupport).
Browser: array leaves are bindable again and state isArray in AddressFields;
the commit mapper writes "isArray": true (no arrayLength). Discovery reports
IsArray/ArrayDim. Docs: Mqtt.md "Array metrics", design table, drivers rule.
Tests: codec (golden + malformed), ingest through the real codec + Dispatch
(the project's MiniBroker never forwards a PUBLISH, so there is no in-process
broker harness), browse, discovery, commit mapper, parity corpus. Mutating the
fix back out fails 18 MQTT + 9 AdminUI tests.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
AbLegacyDriver stamped LastSuccessfulRead (init, read, write), the read's sample
timestamps and host-state change times from raw DateTime.UtcNow; only the backoff
path read the injectable Clock. All of them now read Clock(), so
NativePcccBridgeStatusHealthTests drives a manual clock (init at T0, read at T1)
and asserts LastSuccessfulRead == T1 exactly instead of spin-waiting for the wall
clock to pass the init stamp (#767). Without the driver change the 6 tests fail
(init stamp is wall-clock, not T0).
Directory.Build.props sets TreatWarningsAsErrors for every project since #766, so
the 75 per-csproj copies (64 src + 11 tests) are removed and the props comment
says it is now the only place. tools/mem-probe keeps its explicit opt-out.
Release build: 0 warnings, 0 errors; a planted CS0168 still fails the build in a
src project and in a test project that used to carry its own copy.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
The native-ack path is fire-and-forget after the Part 9 Acknowledge returns, so
a Galaxy ack refused for lack of a credential (#749), or rejected by MXAccess,
left the client with Good and the condition showing Acknowledged while Galaxy
was never acked. Both options from the issue, layered:
1. Synchronous refusal for what the driver can decide without I/O. New optional
IAlarmAcknowledgePreflight (Core.Abstractions); GalaxyDriver refuses an
anonymous session when the driver has no AlarmAck fallback account.
DriverHostActor publishes a condition NodeId -> owning-driver snapshot into
NativeAlarmAckPreflightView (lock-free, volatile); the node manager's new
NativeAlarmAckPreflight seam, consulted after the authz and Primary gates,
returns BadUserAccessDenied with the reason - nothing routed, applied or
reported. A keyed session is not pre-checked (its identity lives in the
Secrets store, which must not be read under the SDK lock).
2. Revert for everything else. RouteAlarmAck carries the condition NodeId;
DriverInstanceActor replies AlarmAckFailed on a refusal, a backend failure,
a driver without an alarm source, or an ack dropped while connecting /
reconnecting. DriverHostActor re-projects the condition unacknowledged via
NativeAlarmProjector.TryRevertAcknowledge, but only while the local ack is
still pending - any driver transition projected since then wins. A disabled
or unspawned driver is not reverted (unchanged).
GatewayGalaxyAlarmAcknowledger now throws DriverOperationRefusedException on a
non-zero native hresult (wrong password, no ack permission) instead of only
warning, so that case reverts too; as a refusal it neither retries nor trips
the breaker.
Tests (each fails with its fix mutated out): projector revert/supersede,
DriverHostActor refused + failed ack -> unacked AlarmStateUpdate (successful
ack control), preflight view snapshot, node-manager preflight unit test,
over-the-wire ConditionMethodEventTests (BadUserAccessDenied, no route, no
event; positive control), Galaxy preflight + hresult refusal.
Docs: Galaxy.md, AlarmTracking.md, ScriptedAlarms.md, .claude/rules/alarms.md.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
HandlePublishError treated any anonymous BadNoSubscription (subscription id 0)
as loss of a tracked subscription whenever anything was tracked. A publish the
SDK queued while the session briefly held no subscription (the driver had just
deleted its last one) is answered BadNoSubscription and can be handled after the
replacement is registered, so a routine unsubscribe->resubscribe declared the new
subscription lost: Degraded health, a BadNoCommunication quality-only annotation
on every alarm condition and ConnectionLost (host re-initialise). That annotation
is the extra Unspecified event the #728 ConditionRefresh wire test saw.
- A BadNoSubscription that names a subscription id now counts only if the id is
tracked (same rule as BadSubscriptionIdInvalid).
- An anonymous BadNoSubscription within StaleNoSubscriptionGrace (5 s) of a
delete that left the driver tracking nothing is ignored (Debug log). The window
is stamped on removal and restarted when the delete completes; failed-subscribe
removes stamp it too; it resets with the session. A delete that leaves another
subscription tracked opens no window. A genuine loss inside the window is still
caught by the PublishingStopped heuristic; only the immediate ConnectionLost is
forgone.
- OpcUaClientStaleNoSubscriptionTests: stale error after alarm and data
resubscribe (fail without the fix), id-attributed stale error, plus genuine-loss
controls (after the grace, no recent delete, non-last delete, session release),
each mutation-checked.
- The ConditionRefresh wire test now records Kind, AlarmType, message, severity,
quality, restatement and timestamp of every post-resubscribe event on failure.
Wire test locally: 1/10 passing before (all 9 failures the 0x80310000
ConditionType quality-only annotation), 12/12 after.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
IOpcUaEndpointState (review-remediation lane C3) recorded whether the SDK started, but only the
health check read it: a fused node whose endpoint failed (port 4840 taken, ...) still advertised
250/240 and stayed driver Primary, so the Primary-gated data plane ran on a node no client could
reach while the reachable peer refused writes as Secondary.
- Move IOpcUaEndpointState/OpcUaEndpointState/OpcUaEndpointStatus from Host to Commons.OpcUa so
the Runtime can consume it.
- ServiceLevelCalculator: new NodeHealthInputs.EndpointListening (default true). false -> 0,
ahead of every other input including the +10 Primary bonus.
- OpcUaPublishActor: takes the endpoint state, feeds EndpointListening (Failed only; NotStarted /
Stopped are boot/shutdown windows) on the DB-backed and DB-less paths, and the legacy role-only
seam returns 0 for a failed endpoint too.
- New OpcUaEndpointStepDownActor (one per driver node): a Primary (ClusterActiveNode.SelfIsActive,
the election's own primitive) with a Failed endpoint gracefully Leaves its mesh - the manual
failover mechanism - so the peer becomes the oldest Up driver member, hosts the singletons and is
Primary. The election itself is not bent. Guarded on a peer having recently reported its own
endpoint listening (EndpointReport on the new opcua-endpoint-state DPS topic), so a pair with
both endpoints failed does not bounce the role and restart nodes forever, and a lone node never
shuts itself down.
- Runtime now references ZB.MOM.WW.Health.Akka (ClusterActiveNode).
- Docs: Redundancy.md tier table + new "Endpoint start failure: Primary step-down" section;
cluster-topology rule; HealthEndpoints / OpcUaEndpointHealthCheck comments (F-C1 landed).
Tests: ServiceLevelCalculatorTests (+5), OpcUaPublishActorEndpointStateTests (7: DB-less,
DB-backed, legacy seam, NotStarted/Stopped do not demote, a later failure demotes on the next
tick), OpcUaEndpointStepDownActorTests (3, real two-node mesh: holds while the peer is not
listening then leaves and the peer becomes Primary; failed Secondary and healthy Primary never
leave). Each was mutation-checked: removing the calculator/legacy guard, the Leave, the
peer-listening guard or the Primary guard fails the corresponding tests.
mxaccessgw 488649c+ acknowledges by AuthenticateUser as the operator and
writing <alarm>.AckMsg, so AcknowledgeAlarmRequest now needs a real Galaxy
operator_user + operator_domain + operator_password. The OPC UA data plane is
API-key/anonymous and carries no Galaxy password, so the driver resolves the
account per ack:
1. the session's API key's Galaxy ack identity, set on AdminUI /apikeys and
held ONLY in the encrypted Secrets store
(otopcua/apikeys/<keyId>/galaxy-ack-{user,password}); a key with a user but
no password secret is refused, never silently downgraded;
2. else the driver instance's AlarmAck fallback account (DriverConfig
FallbackUser/FallbackDomain/FallbackPasswordSecretRef; secret:/${secret:}/
env:/file:/dev: only — a literal is refused);
3. else DriverOperationRefusedException: nothing is sent, never an empty
operator_user. Refusals are excluded from the resilience retry/breaker.
- Pins ZB.MOM.WW.MxGateway.Client/Contracts 0.2.1 -> 0.3.0 (vendored into
nuget-packages/ from the Gitea feed, per the local-mxgw source mapping).
- The session ApiKeyId (not a secret) rides NativeAlarmAck ->
RouteNativeAlarmAck -> RouteAlarmAck -> AlarmAcknowledgeRequest; the password
is resolved in the driver and never travels through actor messages.
- The OPC UA principal + key id are appended to the AckMsg comment
("[OPC UA: <principal>, key <id>]") and logged, so a shared fallback
account does not lose who acknowledged.
- Password never logged: GalaxyAckCredential.ToString omits it, the
acknowledger scrubs it from every gateway message it echoes and withholds
any exception that echoes it; audit rows carry only the key id.
- AdminUI: /apikeys "Ack identity" editor (set/remove; delete-key clears it),
Galaxy driver form gains the fallback section.
- Synchronous OPC UA status for a refusal is follow-up #760 (the native ack
path is fire-and-forget after the Part 9 method returns).
Tests: request mapping (password populated), password absent from logs and
exception text (mutation-checked), key-mapped / fallback / refused paths at
resolver and driver level, breaker non-trip, key-id propagation through node
manager and DriverHostActor, AdminUI service + form round-trip.
Closes#749
Staleness is not a liveness proof: DriverInstanceActor publishes on a 30 s
tick and dedups on a health fingerprint, so a running Faulted, idle or
tagless driver deleted in /raw (undeployed) went stale and was evicted.
DeployedDriverSet reads, per node, the latest and latest-Applied
NodeDeploymentState deployments plus the latest Sealed one, parses each
artifact's DriverInstances once (cached by deployment id, pruned to the live
set, only the blob column of an uncached row is read), and /hosts unions it
with the authored set. The staleness clause stays as a second guard. An
unreadable artifact or a failed read means 'unknown' and skips eviction.
The Hosts.razor comment claiming '~12 health publishes' is corrected.
N-new-2: the CSV download's Content-Disposition carries an RFC 5987
filename* via ContentDispositionHeaderValue.SetHttpFileName.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
I1: /hosts eviction keyed off the AUTHORED DriverInstances, so a driver
deleted in /raw but not yet deployed - still running, still publishing -
vanished every tick. The stores' sweep is now EvictUnconfiguredStale(set,
olderThanUtc): an entry goes only when unconfigured AND its PublishedUtc is
older than 60 s. Tests cover unconfigured+fresh kept (positive control),
unconfigured+stale evicted, configured+stale kept, and the deploy hand-off.
M1: Test Connect falls back to the driver's documented ProbeTimeoutSeconds
default (Galaxy 30, OpcUaClient 15, ...) when the key is absent.
M2: the budget never undercuts the config's root timeoutMs (+1 s).
M5: RawBrowseModal compares the open generation captured before the await,
so a close-then-reopen mid-open closes the stale session.
M6: /alarms-historian keeps a 30 s re-poll on 'not registered' (a driver
node may still be starting) instead of stopping for good.
M7: /scripts source fetch failure shows in the dialog, not the error boundary.
M8: the export modal maps only its preview rows; the endpoint writes via
Results.Text (no second byte[] copy).
M9: /hosts config queries are ordered, so SequenceEqual change detection holds.
N1 tooltip, N2 CRLF-safe regex, N3 throttled Warning for subscriber faults,
N5 equipment-import help text now describes RFC 4180 quoting.
H3: MonacoEditor documents that themes are page-global.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
RequestRebirthAsync now returns only once the requested births have arrived
and gone quiet, but the modal still told the operator to re-expand the tree in
a few seconds. ConfirmRebirthAsync re-keys the tree against the same session
(as the Refresh button does) when the session and modal are still current,
drops the now-stale armed scope, and keeps the outcome line.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Since R06:M-5 the Sparkplug browser states no address for a metric the
ingestor will not read and labels its descriptor 'not bindable', but the commit
refusal still said the attribute lookup returned nothing. RawBrowseModal keeps
the attribute's descriptor on the selection, and DescribeUncommittableLeaf
reports a 'cannot be bound in this version' message echoing it. A lookup that
really returned nothing keeps the old text. A test pins the marker to the
browser's own string.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
/fleet/diagnostics was reachable only by URL: each driver-role row on /fleet now
links to it (FleetDiagnosticsLinks.For escapes the host:port node id). The
diagnostics table showed the string instance id because the wire carries no
display name; the page now joins the authored DriverInstances.Name from the
ConfigDb (enrichment only - a failed lookup falls back to the id) and keeps the
id beneath the name.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The dev-only demo page shipped routable in the production assembly. Its route
moves from @page to a code-behind [Route] under #if DEBUG, so a Release build
keeps the component type (nothing referencing it breaks) but has no /dev/*
route. The page-authorization guard classifies it under #if DEBUG, and
DevRouteGatingTests asserts each configuration's half (Debug: exactly the demo
route; Release: none) with a positive control. Both configurations pass.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
ImportEquipmentModal split each line on commas, so a quoted Manufacturer with
a comma silently shifted every later column. EquipmentCsvParser reads through
the shared Commons CsvParser (the raw-tag importer's authority): quoted
fields, doubled quotes, CRLF and blank lines parse; malformed quoting is an
error, not a guess. Header/shape messages are unchanged.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
v3 retired the equipment-Tag binding; IUnsTreeService.LoadTagsForEquipmentAsync
always returned empty, had no production caller, and EquipmentTagRow plus the
always-zero tagCounts / EquipmentRow.TagCount plumbing existed only to feed it.
All removed; the equipment badge count (virtual tags) is unchanged.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The equipment page's Remove reference / Delete virtual tag / Delete alarm row
buttons and the ScriptEdit Delete ran on the first click. They now go through
RawConfirmDialog, the same confirm /raw uses. The ScriptEdit change is limited
to the delete button and its dialog.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
I-1 - a non-C# paste produced thousands of Roslyn markers. The Problems-list report sent
them all as one JS->.NET message, which could exceed the 1 MB hub limit and kill the
circuit. The JS now sends at most MaxReportedMarkers (200) markers, errors first, each
message cut to 300 chars, plus the total. ScriptProblemReport.FromEditor clamps again
on arrival, and the Problems list shows "showing N of Total" / "+N more". The diagnostics
endpoint also bounds its response: Distinct, errors first, MaxDiagnosticMarkers (1000),
messages cut to 1000 chars. An error is always kept, so editor-accepts == publish-accepts.
M-1 - drop the per-instance compilation lineage; analyses derive from BaseCompilation.
M-2 - the VirtualTag modal clears its problems (and editor ref) on close and on a script
reload.
M-5 - an over-limit text is announced at blur (OnSyncRefused), not only at Save.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Every picker and RawBrowseModal stored the OpenAsync token after the await; if
the host was disposed (or the modal closed) meanwhile, its teardown had already
run with no token, so the plant-side session lived until the reaper's idle TTL.
BrowseOpenGuard.CloseIfOrphaned closes it at once; each host sets _disposed
first thing in DisposeAsync (RawBrowseModal also treats !Visible as gone).
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
A bare catch {} turned every failure - including an admin-only node with no
historian actor - into 'Loading...' forever, and a tick queued after a sync
timer dispose still issued an Ask. HistorianSinkStatusReader classifies a poll
as Available / NotOnThisRole (permanent: shown once, polling stops) /
Unavailable (shown with the last good status and its read time). The page
follows Fleet.razor: IAsyncDisposable, _disposed, overlap guard, page CTS.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
DriverTestConnectButton always sent a fixed 10 s, and the Galaxy/OpcUaClient
probes run under min(caller, configured), so ProbeTimeoutSeconds - documented
on every driver as the Test Connect timeout - could shorten a probe but never
lengthen it (Galaxy's 30 s default was silently 10 s). TestConnectBudget reads
the root ProbeTimeoutSeconds from the merged config at click time, clamped to
the actor's [1, 60] s, falling back to TimeoutSeconds. DeviceModal (the only
host) needs no change. The Galaxy/OpcUaClient form hints are reworded.
The 30 s result-clear timer moves from System.Timers.Timer + an async void
Elapsed lambda to System.Threading.Timer, drained in DisposeAsync, with a
generation so a superseded fire cannot clear a newer result.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
DriverConfigModal, DeviceModal and RawTagModal built a new parameter dictionary
and new EventCallbacks every render. ParameterDictionaryCache reuses the
dictionary while the values it carries are unchanged and rebuilds it with the
current value when one changes (a build-once cache would strand edits); the
callbacks are created once. Blazor still re-sets an EventCallback-bearing
child's parameters per parent render, so the forms' _lastParsed guards stay.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The import review grid rendered every parsed row and re-scanned all rows three
times per render for its counts; RawCsvReviewView recomputes rows + counts once
per change and the grid renders through Virtualize. The export modal embedded the
whole CSV twice (textarea + base64 data: URI); it now shows a 50-line preview
and links to GET /api/raw/tags.csv (ConfigEditor, like /raw), which runs the same
RawTagCsvExport builder and sends it as an attachment.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
R04:P1 - the editor no longer ships its full text JS->.NET on every keystroke. It reports
at blur, and ScriptEdit / the VirtualTag modal read the authoritative value with
MonacoEditor.GetValueAsync at Save (validation runs after the pull). The Blazor hub's
MaximumReceiveMessageSize is raised from 32 KB to 1 MB. The JS refuses a value that
would not fit instead of sending an oversize frame, and Save surfaces that refusal
rather than persisting stale source.
R04:P9 - diagnostics carry a generation counter and an AbortController. Every edit aborts
the in-flight request, and only the response for the current text is applied.
R04:S2 - the always-empty inlay-hints provider, endpoint, service method and contracts
are removed.
R04:S3 - MarkersChanged is typed (DiagnosticMarker) and feeds a new ScriptProblemsList.
Clicking a problem calls RevealLineAsync. Markers cross to .NET only when a list
subscribes.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The deploy API's 202 pointed at a route that did not exist. The GET reads the
deployment (projected, no artifact blob) and its per-node apply states, and
carries the POST's own gate: API key in X-Api-Key, AllowAnonymous against the
cookie fallback, 503 while Security:DeployApiKey is unset.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The recursive RenderNode rendered every row of an expanded device, and built a
fresh context-menu list + closures per node per render. RawTreeRows flattens the
expanded tree (explicit stack) into rows rendered by one Virtualize; each level
shows 500 children, then a 'Show more' row. RawNodeMenuCache keys menus weakly
on the node instance and rebuilds on a picker-mode or Enabled change.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
R04:P2 - every analysis derives from one process-wide bound base compilation via
ReplaceSyntaxTree (the scoped service keeps its current compilation); every entry point
takes the request token, the endpoints bind RequestAborted and answer 499 on abort, and
the Monaco providers abort their fetch when Monaco cancels the request.
R04:P3 - the catalog snapshot carries a sorted, de-duplicated string[] (binary-search
prefix completion) and an Ordinal path->info dictionary, built once per snapshot.
R04:S1 - Diagnose emits an OTSCRIPT_SETVTAG_INERT warning for each ctx.SetVirtualTag
call. It is a warning, so editor-accepts == publish-accepts still holds.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The list loaded every script entity including SourceCode to render names and
hashes. ScriptListQueries projects {ScriptId, Name, Language, SourceHash} for
the list and fetches one script's source when its Source dialog opens, with a
generation guard against a superseded open.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Each received event used to Insert(0) into a List and render the full page.
LiveTailBuffer queues events from the bridge thread into a bounded newest-first
ring; CoalescingRenderScheduler (TimeProvider-driven, drained on dispose)
flushes and renders at most once per 200 ms window. ScriptLog's filtered rows
are recomputed on flush/filter change instead of three times per render.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
OnSnapshotChanged used to re-project every cluster group and re-render the
whole page for each driver publish, per open tab. It now only marks a
CoalescingDirtyFlag; the 5 s tick rebuilds when the flag is set or the config
load changed (or evicted) anything.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The driver-status and resilience stores had no removal path, so a deleted
driver stayed on /hosts as a frozen row until restart. Both stores gain
RetainOnly, which /hosts calls after each successful config load (never on a
failed one).
The stores and InProcessBroadcaster now raise their events one subscriber at a
time, so a throwing handler cannot starve later subscribers or escape onto the
bridge actor. The alert/script-log bridges publish inside their try, and the
resilience bridge catches a store failure instead of restarting.
Refs #745
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The Modbus lane (#741 U3) gave the driver String[] and BitInRegister[] holding-register array encoders, and
moved address resolution to ModbusTagAddressResolution (keys first, a bit region coerced to Bool, strict key
reads), but DriverArrayWriteSupport still listed both array shapes read-only (follow-up F1), so a client
write was BadNotWritable before it reached the driver.
Modbus row now mirrors the driver end to end:
- Keys shape only (region + address both present). A string-only addressString tag is resolved by the
driver with its own family through a parser Core.Abstractions cannot reference, so it stays read-only
(pinned: the conservative direction).
- Strict reads as the resolver: address an integer 0-65535; a present region / dataType / byteOrder /
stringByteOrder string must name a member.
- Coils: any dataType (coerced to Bool), <= 1968 coils (FC15).
- HoldingRegisters (FC16, never Bool): numeric, String ((stringLength+1)/2 registers, stringLength >= 1),
BitInRegister (1 register, bitIndex 0-15, span ending <= 65535), all within maxRegistersPerWrite.
FC23 is opt-in and never changes the answer.
- Elements convert (ScadaBridge int[] -> Int16 stays writable); only a String / DateTime node over a
numeric or bit element is left out, since such an element never converts.
Tests: the parity test's PendingF1 exemption and its guard are gone; the parity matrix is now region x
element type x node type against the real WriteAsync with a spec-conformant device (FC15 > 1968 refused),
plus shape edges at every cap boundary (numeric, String, BitInRegister, coil), strict key reads, FC23
on/off, string-only, the ScadaBridge int[] shape and U3 wire bytes. Core.Abstractions row tests and a
composer test (String[] and BitInRegister[] writable; over-cap, no-stringLength and string-only read-only).
Docs: table XML doc, Uns.md, modbus-addressing.md, drivers/Modbus.md, data-plane rules.
Refs #741
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
I-1: a node whose in-memory trail is one short of the stored trail (it missed
a command the peer applied, whose entry then replicated in) silently dropped
its own next entry — in failover, the survivor's newest ack — because the
positional "past MAX(seq)" rule treated it as already stored.
SaveAsync now reconciles the last 16 in-memory entries against the stored
rows from that position on, by content (Kind/User/Text, timestamps within a
5-minute skew window; matched rows are consumed so repeats count). The peer's
copy of the same command is recognised and not re-appended; an unmatched
entry is inserted at its own ordinal when free, otherwise appended at
MAX(seq)+1 with a Warning — kept, never silent. Bounded read, constant write.
m1: MigrateInlineComments copies only — it no longer UPDATEs the condition
row (a fresh-HLC re-stamp could beat a newer peer ack under LWW); the next
save clears comments_json. m2: it returns unparseable alarm ids and OnReady
logs a Warning per id (+ an Information tally). m3/m4: documented in the
store remarks (Remove vs lagging peer; in-memory index == durable ordinal).
Tests: store-level failover sequence (missed command -> survivor's ack and
next entry persisted once, re-save writes 0 rows, one Warning) with the
dual-write positive control (peer copy present -> 0 rows, no Warning) and a
repeated-identical-comment case; a loopback pair failover test (link down,
survivor acks, rejoin -> both entries on both peers, held); migration
no-row-rewrite, partial fill and unparseable reporting; OnReady Warning.
Both failover tests fail against the round-0 store.
Refs #748
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
N3: a keyless blob carrying the addressResolved marker was served from its
string by the driver but saved as HR[0] by the editor model / CSV path. The
precedence rule now lives in ONE pure resolver in Driver.Modbus.Contracts
(ModbusTagAddressResolution.Classify/Resolve): keys when region+address are
both present, else a string-only addressString resolved with the driver's
family (refused when it cannot parse), else refused. The marker never decides.
Consumers that now ask it instead of restating it: the driver's tag reader
(ModbusTagDefinitionFactory.FromTagConfig), the deploy gate (Inspect /
DescribeRejection), the editor model (AddressShape / IsStringOnly /
HasKeyAddress / ResolveWith, ToJson stamping, Validate), the CSV export cells
and the editor hint. The model writes keys only when it loaded them or a field
was authored - never defaults it did not resolve (also closes two divergences
the new matrix found: malformed-string+marker and non-string addressString were
saved as HR[0]).
One table-driven test (ModbusAddressAuthorityAgreementTests) runs 15 persisted
shapes through driver, deploy gate, model load/save/Validate and the real CSV
export->import under all four families; breaking the CSV export or reverting
the model to the marker rule fails it.
Minors: m1 CSV address cells keep the string+marker while it still describes
them (lossless unchanged round trip), drop it when contradicted, and the review
grid reports the drop; m2 the pre-lane test uses the real export residual; m3
unmarked string-only + typed-cell case added; m4 stale docs/summary and the
unused factory parameter removed.
Refs #741
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Every Part 9 transition used to re-serialize the alarm's whole comment trail
into alarm_condition_state.comments_json and replicate it to the pair peer:
O(n) per transition, O(n^2) over an alarm's life (measured: one ack at 10k
comments shipped 1,080,553 oplog bytes; now ~1.6 KB, the same as at 10).
- New replicated table alarm_condition_comment, PK (scripted_alarm_id, seq),
seq = the entry's ordinal in the trail. Ordinal (not content) keying makes
the two pair nodes' copies of one logical entry (each stamped by its own
clock, since both nodes apply every command) collapse to one row under LWW.
- LocalDbAlarmConditionStateStore.SaveAsync appends only entries past the
stored MAX(seq) (INSERT OR IGNORE, append-only) and upserts a fixed-size
condition row with comments_json = "[]", in one transaction. Remove drops
the trail with the row. Load merges the table with any inline trail.
- comments_json column kept (always "[]") so the condition table's shape and
replication digest are unchanged and old-shape rows still load exactly.
- AlarmConditionStateSchema.MigrateInlineComments: idempotent per-row move of
inline trails, run by LocalDbSetup.OnReady after RegisterReplicated so the
moved entries replicate. Comment table joins ReplicatedTables (now five).
Tests: constant transition payload at 10 vs 10k comments (+ instrument
positive control), old-shape back-compat load, migration idempotency /
partial re-run / lazy move, OnReady boot migration captured in the oplog,
and two loopback pair tests (A->B and B->A trail convergence; the shared
entry converges to one copy, with a distinct-alarm positive control).
Refs #748
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Joseph Doherty