PageAuthorizationGuardTests.EveryRoutablePage_isClassifiedAndMatchesExpectedPolicy
was red on master: the FleetDiagnostics page (lane A, 953bd275) was
routable but unclassified. It declares AuthenticatedRead, the same
policy as /fleet, and is classified as such. Count comment 16 -> 17.
Refs #731#734
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review m-1. The bind moves from Program.cs into
Host.Configuration.ScriptingOptionsBinding.Bind, which binds the
section and clamps it. Program.cs calls it, and
ScriptingOptionsBindingTests drives it with in-memory configuration.
Every documented key takes a non-default value, so a misspelled
property or key goes red instead of silently falling back to the
defaults. The tests also cover a clamped bind (1e300 / 1024 / 0, three
adjustments), an absent section, and the sibling LogFilePath /
LogTopicMinLevel keys leaving the bind untouched.
Refs #731#734
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review I-2. Under factory Props a supervision restart of
ScriptedAlarmHostActor builds an empty engine, and nothing re-sent
ApplyScriptedAlarms. Until the next deploy the host had no mux interest
(inputs stopped), and its ownership filter silently dropped every
operator ack and shelve.
The host now tells its parent ScriptedAlarmHostRestarted from
PostRestart. DriverHostActor keeps the last plans it applied and
re-sends ApplyScriptedAlarms when the signal comes from the current
host, in Steady, Applying and Stale. This mirrors how a driver child's
ReprovisionRequested is answered. The OnAlarmsLoaded comment, which
assumed a re-apply that never happened, now says where it comes from.
DriverHostActorScriptedAlarmRestartTests deploys one scripted alarm
through a real host, crashes the alarm host, does NOT re-apply by hand,
and asserts that interest is re-registered, a dependency push activates
the alarm, and an operator ack reaches the engine. It is red without
the signal (times out waiting for the re-registration).
Refs #731#734
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
N1: a full alarm-state snapshot also sets the condition's Quality, so it now
supersedes an earlier quality-only write for the same id in the deferred
buffer; a quality arriving after the state is kept and replays after it.
N2 / M1: the replay is not inert on a node the apply KEPT - an SDK-applied
Acknowledge/Confirm/Shelve or an optimistic client Write in the window would
be rolled back (with spurious Part 9 events). The replay is now scoped to
nodes the apply created: variables the applier ensured during this apply
(AddressSpaceApplier.BeginRecordingEnsuredVariables) or every variable after a
full rebuild; conditions in the applied composition but not the previous one
(also on a rebuild, which carries a kept condition's state over). Option (b),
a sink that reports whether a live write landed, needs the Commons
DeferredAddressSpaceSink seam outside this lane (follow-up F7).
M2: the benchmark remark now matches its Category=Benchmark trait.
M3: TestHooks and the seam-carrying constructor are internal; PropsForTests
produces the actor through an IIndirectActorProducer.
M4: pinned replay-nothing-and-drop-the-buffer after a failed plan.
M5: PreRestart answers every queued rebuild requester (degraded).
Refs #732
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review I-1. Context.Stop is asynchronous, so a removed child kept
publishing through the rest of the apply handler and its own stop.
Those bursts were forwarded after SourceRemoved and re-retained the
removed driver's last Good in the mux, where nothing would ever clear
it while a replay subscriber held the ref (a removed driver never
publishes again, and its PostStop Bad fan resolves no refs once the
index is rebuilt without it).
StopChild and HandleRestartDriver now retire the child (_retiredChildren
+ Context.Watch) before stopping it. ForwardBatchToMux drops a burst
whose Sender is retired, and Terminated (handled in Steady, Applying and
Stale) removes the entry. Akka delivers Terminated after the watched
child's last message, so the set is bounded and a live incarnation is
never filtered. ConnectivityChanged from a retired child is
deliberately not filtered, because the respawn path relies on its Bad
fan.
Tests (review m-3 folded in): a stale burst spoofed as the removed child,
held alive in PostStop by a new RecordingSubscribableDriver.ShutdownGate,
no longer re-retains (red with the guard disabled). New cases cover a
config-change respawn and an operator Restart, each with the new
driver's value replaying as the positive control. The arrange step now
waits for the real subscription set and fishes for both drivers'
values at once (two sequential fishes could discard the other burst).
Refs #731#734
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
It had no production sender: every driver data change is already a
batch, and the comms-loss fan was converted earlier. DependencyMuxActor
drops its per-value receive, DriverInstanceActor drops the record, and
DependencyMuxActorTests' 13 sends move to the ValuePublishes.Single
one-item batch (same semantics through the batch path). XML docs and
the living docs (DriverLifecycle, VirtualTags, ScriptEditor) are
reworded; historical plan docs are left as written.
Refs #731#734
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
DriverHostActor now tells the mux SourceRemoved(driverInstanceId)
wherever a driver object ends: StopChild (deploy removal and the stop
half of a config-change respawn) and an operator Restart. It is not
sent on a supervision restart, which re-uses the same driver. Before
this nothing sent it, so a removed driver's last Good stayed in the
mux and replayed to the next replay-opting subscriber as current.
DriverHostActorSourceRemovedTests drives a real mux with two real
driver children: after a deploy drops DI-1 only DI-2's value replays
(positive control: both replay before the removal), and a supervision
restart of DI-1 leaves its value replayable.
Residual (documented on EvictRetainedValues): a burst the old child
enqueued just before it stopped can still be forwarded after the
eviction; the next publish, a comms-loss fan or the grace sweep
bounds it.
Refs #731#734
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
ScriptContext.CancellationToken is backed by an AsyncLocal that
ScriptEvaluator.RunAsync scopes around the compiled delegate, so it is
exactly the token ScriptRunGuard cancels on timeout. A polling script
can now end its own abandoned run and free the slot. AsyncLocal rather
than a field because the scripted-alarm engine reuses one context per
alarm: an orphan keeps seeing its own cancelled token. The scoping
entry point is internal (IVT to Core.Scripting) so a script cannot
replace its own token.
Editor parity: ScriptAnalysisService dot-completion now walks the
receiver's base chain (instance members only, stopping before
System.Object). Before this, a member declared only on ScriptContext
compiled and published but never completed. DiagnoseTests pins the
member as allow-listed; CompletionTests pins the completion.
Refs #731#734
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
C1 (critical): B6 computes the plan off the pinned thread while B1 made writes
non-creating, so a value/status/alarm write for a node the in-flight plan adds
was skipped, and change-driven drivers never republish it (BadWaitingForInitialData
indefinitely; on a full rebuild every re-subscribe value landed on discarded
nodes). OpcUaPublishActor now applies writes immediately AND remembers the
latest per (realm, id) while a plan is in flight or queued (value+status
compose; alarm state/quality coalesce to the latest, which the node's delta
gate makes inert on an existing condition), replays it after each apply before
answering, and drops it once the pipeline is idle. Bounded (250k keys by
default; beyond it new keys are not held, logged once).
I2: the data-plane-authority parse runs in its own try inside the plan task; a
throw is logged and never degrades the address-space apply.
I3: the SDK's DeleteMonitoredItems releases an item's component-cache
reference on a Good delete; a tombstoned item's reference was already
released, so the delete took one from the re-created node's entry. Released
items are remembered and the orphan delete takes one back first. Event items
never held a reference (EventManager + SubscribeToEvents), so the three
event-item releases are gone.
I4: direct tests for the HistoryUpdate override's per-request updater check
and the HistoryReadEvents override's source guard (both reached by restoring
the bit after a reset - the B7 race window); fixed the false remark.
Minors: the MaterialiseAlarmCondition kind-swap tombstones the dropped
instance; a bytes-less rebuild arriving mid-plan queues behind it (replies in
order); a PlanReady/PlanFailed from a previous actor incarnation is ignored and
its requester answered degraded; the tie-cluster end bound of an uncapped read
fits against the backend ceiling (defence in depth - guarded by a sweep test);
the lock-time benchmark stays in the default run, headroom noted.
Refs #732
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
FocasDataType.String was authorable on any address, but no area read returns text: PMC
fell through to a byte read, PARAM to Int32, MACRO to a double, and the String node
received a boxed number at Good.
One rule, FocasTagDefinitionFactory.DescribeDataTypeMismatch, now backs every path:
- FromTagConfig rejects the pair, so the driver skips + reports it (BadNodeIdUnknown,
never a Good number) and Inspect names the reason at deploy;
- FocasTagConfigModel.Validate refuses it on the typed fields;
- the AdminUI editor only offers String where it is readable and shows the rule's text
inline (kept visible for a loaded blob that already carries it);
- the CLI read/subscribe/probe/write commands refuse it at the boundary.
String stays valid on FIXED: paths (Identity/Program name/mode text) and is ignored on
ALARM: tags; an unparseable address keeps its own diagnosis.
Razor change (FocasTagConfigEditor.razor) needs the Wave 4 live check.
Refs #740
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Re-review 1: HandleDispatch's terminal guard and its AwaitingApplyAcks flip read the row separately, so a
lost-dispatch closure committed between them was silently reopened. The flip now carries the RowVersion the
guard read; a change in between is a concurrency conflict (nothing seeded, nothing published, coordinator
reset). Pinned on real SQL Server with a closure injected between the two reads (fails without the fix).
AdminOps' apply deadline and the coordinator's now both read DeployTimings.ApplyDeadline, passed explicitly
to both singletons. Microsoft.IdentityModel.Protocols(.OpenIdConnect) pinned to 8.11.0 and referenced
directly by the Security project, so the whole IdentityModel family resolves 8.11.0 (fresh restore clean
of NU190x/NU1605).
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
BoundedApiKeyStore kept the set (volatile) and its DateTimeOffset stamp (non-volatile, 16 bytes) in separate
fields, so a reader could pair a new set with an old stamp or read a torn stamp; they are now one immutable
snapshot behind a volatile reference. An inner store that cannot list ids now switches the set off with a
per-instance flag instead of writing KnownIdSet = false into the shared ApiKeyStoreGuardOptions.
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
RefreshAsync now tells "the store answered: no such secret" (revocation - drop the refreshed key AND the
lazily cached one) apart from "the read failed" (keep the previous key). Store reads fan out through
Parallel.ForEachAsync capped at MaxConcurrentRefreshReads = 8 instead of one read per cluster at once.
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The one-snapshot refactor re-sorted every collection in memory with an ordinal comparer, while the old
composer's SQL ORDER BY used the database's case-insensitive collation; mixed-case keys therefore re-hashed
an unchanged configuration (spurious drift fleet-wide, a "new" revision on a no-op redeploy). The snapshot
factory now reads every composed table ORDER BY its natural key server-side and Compose serialises in the
order given. Pinned byte-for-byte against a verbatim replica of the old composer on the real SQL Server.
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
I7: a Dispatching row committed after the coordinator's PreStart whose Tell never arrived had no owner and
refused every later deploy. The in-flight check (now fully off the admin singleton's thread, serialised by the
actor-local compose flag alone) closes a Dispatching row with zero NodeDeploymentState rows that is past the
apply deadline as TimedOut (optimistic on RowVersion) and proceeds.
HandleDispatch ignores a dispatch for a Sealed/PartiallyFailed/TimedOut row, so a late or duplicated dispatch
cannot flip it back. Coordinator status/reason writes attach a key+RowVersion stub instead of loading the
multi-MB ArtifactBlob; proven on the real SQL Server (blob untouched, seal lands).
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
SQL Server's DbUpdateException for one duplicate names every entry in the statement batch, so detaching
ex.Entries dropped the whole flush. On a row-level rejection the writer now asks the database which of the
batch's EventIds it already holds, drops exactly those, re-saves the rest, and falls back to row-by-row only
if a non-duplicate row is still refused. Fresh context + entities per attempt. Pinned against the real
central SQL Server (new SqlServerTestDatabase harness) and a faithful whole-batch interceptor.
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
- ScriptLogRateLimiter.LimitPerWindow. The ScriptLogTopicSink and
VirtualTagActor summaries now print the configured limit, not
DefaultLimitPerWindow. Tests assert "limit N/window" for a non-default
N.
- CalculationDriver.Dispose cancels, then takes _evalGate (bounded 2 s)
and disposes the gate and _evalCts. The pump and timer ticks enter the
gate through TryEnterEvalGateAsync, which stops cleanly once the
driver is disposed. Test: Dispose with a 6 s runaway run in flight
returns promptly, disposes both, and a late trigger is harmless.
- ScriptedAlarmEngine captures _shutdownToken once and disposes
_shutdownCts in Dispose when every drained task has completed. A task
stuck past the grace leaves the CTS to the GC. Lifecycle tests assert
the disposal.
- DraftValidator: CalculationNoEffectiveTrigger is skipped for a
change-triggered tag whose scriptId does not resolve. That case is
already CalculationScriptMissing or CalculationScriptNotFound, and
there are no reads to count. Both existing tests now assert the
absence (red before the change).
- docs/VirtualTags.md: System.Threading is denied as a whole namespace,
with an allow-list; the type list is now accurate (Unsafe,
RuntimeHelpers, and the ForbiddenMembers on System.Type); the resource
limits section covers the run guard and the allocation budget.
docs/ScriptEditor.md: accurate deny-list and a new "Runtime safety"
section covering timeout, refusal, quarantine after 3 timeouts,
allocation budget, lifting, and rate-limit summaries that appear only
on the source's next entry after a burst then quiet.
Refs #734
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The window used to count every terminal row, so 50+ failed deploys pushed the last good artifact out of it.
Now the newest KeepSealed (>= 1) Sealed rows keep their blob, failed rows are prunable unless referenced in
the ReferencedDays window. Pinned: 60 TimedOut + 1 old Sealed -> Sealed retained, 60 TimedOut emptied.
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The fixture wedged EVERY write on one shared token-ignoring release gate, so
the later same-tag write was also hung, and the 300 ms pessimistic bound
abandoned it while the test was still polling before ReleaseWrites(). Its
reason was "write did not complete within" (the bound), not "queued for" (the
queue budget), and it had already entered the driver, so the tag had been
released. The release path is correct. Only the first write now hangs, and
later writes complete at once.
The abandon reason formatted the bound with TotalSeconds:F0, which printed a
300 ms bound as "0s". It now uses milliseconds, and the test asserts it.
Refs #731
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Coordinator ruling (I6): until ServiceLevel consumes IOpcUaEndpointState (F-C1), the probe must not sit on
the Active tier of a fused node, because Traefik routes the AdminUI on /health/active. Driver-only nodes keep
it on Ready + Active; admin-only nodes do not register it. Pinned per role.
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
ServerAuthenticationStateProvider throws InvalidOperationException from GetAuthenticationStateAsync in any
scope where no circuit set its state (minimal-API requests, hosted-service scopes). The accessor now treats
that as "no circuit identity" and falls back to the HTTP principal. Pinned with the real provider on both
sides (unset -> HTTP fallback; set -> circuit principal wins).
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
The hand-rolled bearer handler is replaced by AddJwtBearer (Microsoft.AspNetCore.Authentication.JwtBearer
10.0.7, pinned on the same servicing line as the other AspNetCore packages). Validation parameters come
from JwtTokenService.BuildValidationParameters, which now pins ValidAlgorithms = HS256 for both the
bearer scheme and TryValidate; MapInboundClaims is off so the minted Role/zb:username keys resolve.
/auth/token requires the cookie scheme, so a bearer can no longer mint a fresh token for itself.
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
DependencyMuxActor retained the last value of every ref ever registered,
and never removed the entries. The continuous-historization recorder
registers every historized ref, so on a historizing node the map grew
with the historized address space and never shrank. The class remark
claimed a bound that did not hold.
- Retain only refs that at least one subscriber registered with
ReplayRetained: true names (VT actors, the Calculation mux adapter, the
scripted-alarm host). A non-opting subscriber is still delivered to but
retains nothing.
- When a ref's last replay-opting subscriber leaves, its value is kept
for a grace (default 2 min, Props(retentionGrace)) so a restart
(old adapter gone, new one not yet registered) still replays. A
periodic sweep then drops it along with the entry. Entries with no
subscriber and nothing retained are removed.
- New SourceRemoved(DriverInstanceId) evicts the values a removed driver
produced, so its last Good is never replayed. The mux records the
publishing driver per retained ref. No production sender exists yet:
DriverHostActor is outside this lane (follow-up).
- Remark rewritten. Tests: retention bounded to replay-opting refs,
release after grace (with an in-grace replay as the positive control),
SourceRemoved eviction, and the restart-replay scenario still green.
Refs #734
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
B6 moved the address-space-rebuild span into ApplyPlan, so the two rebuilds
that complete synchronously on the actor thread (no applier -> raw-sink
rebuild; no artifact bytes -> held) stopped emitting it
(OtOpcUaTelemetryHookTests.RebuildAddressSpace_starts_an_address_space_rebuild_span).
Both now open their own span.
B7 (late HistorianDataSource / HistorianDataUpdater re-evaluate the
capability bits) reversed three tests that pinned the old "fixed at
materialisation" contract. Re-expressed against the new one, end to end:
- events: a folder promoted without a source is unsupported, then served
once a source is wired (source reached);
- in-process HistoryUpdate: clearing the updater retracts HistoryWrite and
the old updater is never reached;
- over the wire: a late updater lights HistoryWrite and an insert reaches
it; a reset retracts the bit and the insert is BadHistoryOperationUnsupported.
Refs #732
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
- I1 ScriptRunGuard: a run refused because the previous one is still
executing is a Bad publish plus otopcua.script.run_refused, and no longer
a quarantine strike. Only a timeout (and an allocation breach) counts;
a refusal neither adds to nor resets the timeout streak; a completed run
(value or throw) resets it. On a 1 Hz trigger with a 2 s budget a single
4 s overrun is one timeout plus two refusals and must not quarantine.
Tests updated across the guard, the alarm engine, the Calculation
evaluator and the VT evaluator, plus
Refusals_neither_count_toward_nor_reset_the_timeout_streak.
- I3 ScriptedAlarmEngine: the quality bucket is tracked only after the
predicate completes (or in the inputs-not-ready path), so the tracked
bucket is the last one annotated. A refused or quarantined predicate
used to re-track the input bucket Good before the attempt and then flip
it Bad, emitting QualityChanged on every push. The run-guard tests now
count emissions (exactly 1 across N pushes, refused and quarantined
cases), with a healthy sibling's dispatch count as the positive control
that every push was processed. Both were red before the fix.
Refs #734
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Lane H1 follow-through: SyncAdvertisedAggregates now also calls
HistoryPaging.AdvertiseMaxReturnDataValues(Server, source
.MaxRawValuesPerRead) so HistoryServerCapabilities/MaxReturnDataValues
(i=11273) follows the historian source on every assignment (the Null
default advertises 0 = no limit); a false return logs a Warning (the cap
is enforced by HistoryRead regardless).
Test: NodeManagerMaxReturnDataValuesSyncTests reads i=11273 back through
the SDK master node manager after setting a capped source (7000, 250)
and after resetting to the Null source (0).
Refs #732#736
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
H1 fuzz finding (routed to Lane B because AppendEndBoundAsync lives in
OtOpcUaNodeManager.cs): a bounded HistoryRead-Raw page sized BELOW
NumValuesPerNode by the backend's row ceiling (RawPageLimit.Plan) took
the end bound whenever its data fitted, because the fit was judged
against NumValuesPerNode instead of the page's effective cap — a full
ceiling-sized page came back with cap + 1 values. The bound is now
fitted against the page allowance (the ceiling-adjusted data cap plus
the start bound a first page already carries); when the page is full
the bound goes to the next page through the continuation point.
Test: NodeManagerHistoryReadBoundsTests
.A_full_ceiling_sized_page_carries_the_end_bound_on_the_next_page
(ceiling 4 ⇒ 3 data rows; first page = start bound + 3 with a CP, second
page = the end bound alone; control: no ceiling ⇒ one page of 5).
Refs #732#736
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review 01 U3. EnsureFolderIsEventNotifier granted the EventNotifiers
.HistoryRead bit (and registered the event-history source) only when a
real HistorianDataSource was wired at promotion time, and EnsureVariable
composed the AccessLevels.HistoryWrite bit from whether a
HistorianDataUpdater existed at materialise time — so an assignment
after the first apply left existing folders without event history and
existing nodes refused by the SDK's HistoryUpdate admission until a full
rebuild, and a source reset to Null never took the bit back.
The two setters now re-evaluate: HistorianDataSource walks every promoted
folder (bit + source registration, both directions), HistorianDataUpdater
recomposes the AccessLevel of every historized variable (Writable read
back off the node's own CurrentWrite bit). The Host's boot ordering is no
longer load-bearing.
Tests: NodeManagerCapabilityRefreshTests (late source lights the bit on
an already-promoted folder and a reset takes it back, live-event
subscription untouched; late updater grants HistoryWrite to historized
nodes only and a reset takes it back, authored Writable preserved).
Refs #732
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review 02 P5. OpcUaPublishActor ran the whole apply — DeploymentArtifact
.Open, ParseComposition, AddressSpacePlanner.Compute and the sink
materialisation — on the opcua-synchronized-dispatcher, so a large
artifact serialised every live value batch (and, because the node
manager lock is the SDK's, every client Publish) behind the parse.
StartRebuild now runs Open + ParseDataPlaneAuthority + ParseComposition
+ Compute in Task.Run and pipes a PlanReady (or PlanFailed) back to Self;
only ApplyPlan — the sink passes — runs on the actor thread. Rebuilds are
applied strictly in arrival order (a plan is a diff against _lastApplied,
which only the apply advances): one in flight, later ones queue in
_pendingRebuilds while value/status batches keep flowing. The apply is
closed (CompleteAddressSpaceApply) and the RebuildCompleted reply sent on
every path, PlanFailed included. The bytes-less ConfigDb fallback
(LoadArtifact / LoadLatestArtifact) is deleted: it had no production
producer and would have blocked the pinned thread on central SQL; the
dbFactory Props/ctor parameter is kept for wiring compatibility (F4).
Tests: OpcUaPublishActorOffThreadPlanTests — a value batch sent right
after a 50k-tag rebuild lands before that rebuild's first materialise
call (positive control: all 50k tags still materialise); back-to-back
rebuilds apply in arrival order (the second diffs against the first);
unreadable bytes answer degraded with nothing torn down. The rebuild /
apply-failure suites now hand the actor the artifact bytes every
production producer carries (SealArtifact records them; Rebuild(dep)).
Refs #732
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review 01 U2. The composer never emits EquipmentTags in v3 (device values
live in the Raw subtree and reach equipment as UnsReferenceVariables), so
the equipment-tag plan diff sets (Added/Removed/ChangedEquipmentTags,
EquipmentTagDelta), the planner's diff over them, the classifier's
TagDeltaIsSurgicalEligible and its Rebuild clause, the applier's
MaterialiseEquipmentTags pass, surgical ChangedEquipmentTags loop,
removed-tag teardown loop and add-announcement term, and the publish
actor's equipment-tag pass were unreachable, maintained dead weight.
IsEmpty keeps its semantics (an otherwise-empty plan is still Empty).
Kept, deliberately: AddressSpaceComposition.EquipmentTags and the
EquipmentTagPlan record — Runtime's DeploymentArtifact (outside this
lane) still assigns the empty set and the artifact parity tests assert
it; follow-up F2 removes both together.
Tests: the equipment-tag vehicle tests for PureRemove / AddRemoveMix /
the rebuild ratchet are re-expressed over raw tags (same applier paths);
the equipment-tag surgical and classifier tests are deleted (their raw /
UNS-reference equivalents already exist); SubscriptionSurvivalTests now
materialises its tags through MaterialiseRawSubtree.
Refs #732
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review 01 L3/H5/H4/P6/P4.
- L3/H5 OnEquipmentTagWrite's dispatch is factored into DispatchDeviceWrite:
a gateway that throws synchronously is refused inline with
BadDeviceFailure (it used to escape the handler under the SDK Lock), and
a faulted/cancelled dispatch task has its exception observed and logged
and carries the real reason into the revert/audit event instead of
surfacing as UnobservedTaskException at finalization.
- H4 every timestamp the manager stamps (condition quality, timed-shelve
expiry, method-transition events, carry-over, revert, revalidation)
reads the injected Clock; RemainingShelveMs takes the clock reading.
- P6 OpcUa:Quotas:* (OpcUaQuotaOptions on OpcUaApplicationHostOptions):
Min/MaxRequestThreadCount, MaxQueuedRequestCount with the former
hard-coded 5/100/200 as defaults, and the TransportQuotas members as
nullable overrides over the SDK defaults.
- P4 the SDK line exposes IHistoryReadAsyncNodeManager but CustomNodeManager2
offers no async virtual to override (adopting it means re-implementing
the base dispatcher), so the lever is the admission cap:
MaxConcurrentHistoryReadBatches defaults to clamp(2*ProcessorCount, 4, 16).
The Host still overrides it from ServerHistorianOptions (default 16) —
follow-up F3.
Tests: NodeManagerWriteDispatchTests (sync throw refused; faulted task
reverts; success leaves the value; clock-stamped revert),
OpcUaApplicationHostQuotaTests.
Refs #732
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review 01 P1/P2/P3/P5 — work done under the manager-wide SDK Lock on the
pinned publish thread, where every Read/Browse/Publish waits.
- P1 RebuildAddressSpace detaches the shared root's top-level children
once instead of every leaf from its parent (NodeState.RemoveChild is a
linear scan + RemoveAt, ~k²/2 compares per k-child folder); root
notifiers are cleared as a list instead of one O(F) RemoveRootNotifier
each; ForgetPredefinedSubtree / TombstoneMonitoredSubtree are iterative
with one child buffer.
- P2 RemoveEquipmentSubtree walks a parent→children index maintained by
EnsureFolder/EnsureVariable/MaterialiseAlarmCondition (keyed by the
parent id the caller named) instead of prefix-scanning six maps per
removed equipment; a surviving condition's wiring into a removed folder
is found through the folder's own notifier list; descendants are not
detached from a folder that is itself being discarded.
- P3 LowerAllEventsMonitoring(condition) replaces the full O(C) reconcile
on RemoveAlarmConditionNode / RemoveEquipmentSubtree / the kind-swap.
- P5 AlarmNotifierWiring.Folders is a reference-identity HashSet and an
already-wired pair skips the SDK's linear AddNotifier dedup.
Benchmark (NodeManagerLockTimeBenchmarkTests, M-series Mac under load,
three runs each): rebuild of a 10k-tag folder 75–150 ms → 5–6 ms;
removing 200 equipment (250 signals each) from a 50k-node space
471–681 ms → 58–64 ms; all-events bookkeeping visits for 10 condition
removals over 50 conditions 900 → 10. The two timing legs assert < 1 s;
the P3 leg asserts the visit count (machine-independent).
Refs #732
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review 01 L2. The SDK keys its fan-out lists by NodeId and its component
cache hands back the CACHED NodeState for a NodeId it already holds, so a
rebuild or a surgical remove+re-add left a NEW item on the same NodeId —
from any session — bound to the DEAD node: created Good, fed its stale
initial value, never again, until every session had deleted its items.
Every removal path (RebuildAddressSpace, RemoveVariableNode,
RemoveAlarmConditionNode incl. the condition's instance-scoped children,
RemoveEquipmentSubtree) now queues BadNodeIdUnknown to the node's data
items, drops the MonitoredNodes entry, releases each item's component
cache reference and forgets the _liveItems/_eventItems entries. Items
the ACL pass stood down (on no fan-out list) are swept by NodeId. A later
client delete of an orphaned item is completed by the DeleteMonitoredItem
override / the SubscribeToEvents unsubscribe arm instead of answering
BadMonitoredItemIdInvalid or NRE-ing in MonitoredNode2.Remove. The
revalidation restore re-resolves the node by NodeId and no longer
re-attaches to Handle.Node blindly. Timestamps now come from the injected
Clock (UtcNow).
Tests: StaleMonitoredNodeWireTests (real SDK, three legs: rebuild,
surgical remove+re-add, second session; each keeps the old item alive,
proves the new item receives a value written after its creation, the old
one gets BadNodeIdUnknown, and its delete succeeds — all three fail with
the tombstone neutered), NodeManagerStaleMonitoredNodeTests (stood-down
item swept on surgical remove and rebuild; sibling removal leaves it
restorable).
Refs #732
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Review 01 L1/L4/H2. WriteValue/WriteValues are non-creating (an unknown
id is skipped, counted and Debug-logged, as WriteStatuses always was), so
a driver publish racing a surgical remove can no longer resurrect the id
as a flat BaseDataType phantom under the shared root — nor hand a later
re-add of the same tag the phantom's parent/DataType/writability through
EnsureVariable's early return. WriteAlarmCondition for an unmaterialised
id is a logged no-op instead of minting the legacy bool[2] placeholder
(AddPredefinedNode is AddOrUpdate, so it replaced the SDK lookup entry of
any folder sharing the id). The applier no longer writes a condition
state against every removed equipment id (a folder, never a condition).
EnsureFolder/EnsureVariable/MaterialiseAlarmCondition refuse an id already
held by another node kind in the same realm. CreateVariable is deleted.
Tests: NodeManagerPhantomNodeTests (publish racing a remove leaves no
node; the re-add gets the authored shape and is live; unknown alarm id
is a no-op; the guard in all three directions, realm-qualified); the
sink / batch-write / write-statuses tests now declare their variables.
Refs #732
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
- RunWriteAsync wraps the invoker call in WaitAsync(writeTimeout) (Props,
default 10 s = 2x the 5 s write cap): a token-ignoring hung write now
answers BadTimeout, releases its slot and tag, and its orphan outcome
is observed and discarded - later writes to that tag reach the device.
- WriteCompleted carries the launching incarnation (from the process-wide
generation source); a late completion from a previous incarnation after
a supervision restart is ignored instead of freeing the new
incarnation's slot/tag.
- Test: a hung token-ignoring tag-a write is abandoned and a later tag-a
write reaches the device while the abandoned one is still inside it.
Refs #731
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
- GatewayHistorianDataSource.ReadRawWindowAsync keeps the cap + 1 probe budget
(budgetCap) separate from the client's cap: a strict-ceiling or crowding
refusal now reports Limit = the server cap that applied and advises setting
NumValuesPerNode at or below it (it said "lower NumValuesPerNode" with
Limit = cap + 1 to a client that sent 0).
- Fuzz: the oracle-source over-cap assertion was vacuous (cap 0 never counts);
it now checks no reverse window reached the forward-only shape. Class docs
name the post-floor cap set (3 ... 1000) and distinguish NumValuesPerNode
1 / 2 from the smallest server cap.
- HistoryPaging.IsFullPage doc says it judges the REQUEST cap (page + ties +
lookahead), which is finite under any limit; AdvertiseMaxReturnDataValues
swallows any exception (returns false for the caller to log), as its doc
promised.
Refs #736
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
BoundedApiKeyStore's negative cache was keyed by key id, so a flood of DISTINCT well-formed
otua_<random>_x credentials missed it every time and cost one SQL query per attempt under the
SDK's SessionManager-wide lock, no credential needed (R03:3.3).
- IApiKeyStore.ListKeyIdsAsync (default null = unsupported; ConfigDbApiKeyStore lists every
ApiKeyId): the bounded store keeps a known-id set refreshed every StoreCacheSeconds
(single-flight, failed refresh keeps the previous set) and refuses an unknown id without a store
read ({outcome=unknown_id}).
- A process-wide miss budget (token bucket, Security:ApiKeys:MissBurst=100 /
MissRefillPerSecond=20, ApiKeyStoreGuardOptions) bounds lookups that reach the store and miss
while no set is available; a hit refunds its token. Per-endpoint accounting needs the
OpcUaServer impersonation seam to carry the client address — follow-up.
- docs/security.md: the new knobs, and the known-cluster gate's documented fail-open window at
central boot (R03:4.2) plus the async PSK refresh (R03:3.4).
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
- Writes: a second write to a tag already inside the driver waits in the
queue behind it (_tagsInFlight) while other tags still run under the
cap of 4; queued writes are stamped and failed BadTimeout at dequeue
once older than the 8 s host Ask budget (Props queuedWriteBudget)
instead of being written late.
- Initialise attempts are bounded pessimistically (initializeTimeout,
default 30 s = 2x Tier C) on top of the invoker's token-only timeout:
an overrun is logged as abandoned, reported InitializeFailed so the
retry loop continues, and its late result is dropped by generation.
Generations come from a process-wide Interlocked counter so a prior
incarnation's late result cannot collide (F6).
- PostStop shares one shutdownTimeout budget across shutdown + dispose.
- TelemetryDialSupervisor cancels the establishment deadline on
StreamStopped; FleetDiagnosticsClient builds the ActorSelection inside
the try (a malformed node degrades to "unavailable"); the diagnostics
page labels the column "Last health publish".
- Tests: same-tag serialisation with a distinct-tag positive control;
stale queued write BadTimeout; token-ignoring hung init -> later
attempt connects; stopwatch assertion demoted to a ceiling.
Refs #731
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
- MeshGrpcCommandDialer.RetainOnly (IMeshCommandDialer, default no-op): CentralCommunicationActor
hands the dialer the current endpoint set on every contact refresh, so a renamed host or moved
port releases its cached GrpcChannel and idle HTTP/2 connection instead of living for the
process (R03:1.3).
- IMeshPskProvider.RefreshAsync (default = the old InvalidateAll): MeshPskProvider pre-resolves
every known cluster's key concurrently, bounded like the sync path, and swaps an immutable
snapshot in atomically; a failed read keeps that cluster's previous key. The actor calls it from
ContactsLoaded (after SetKnownClusters, off-actor via PipeTo) instead of InvalidateAll, so the
first ack per cluster per ContactRefreshSeconds no longer blocks a gRPC thread on a 3 s-bounded
store read or gets denied past the bound (R03:3.4).
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
AuditWriterActor documented per-row dedup on the UX_ConfigAuditLog_EventId index but flushed one
statement batch: a single duplicate EventId (any at-least-once producer across a singleton
hand-over) threw for all 500 rows (R03:2.6). SaveChanges now detaches the entries a
DbUpdateException names (2601/2627 logged at Debug as the expected duplicate, anything else at
Warning) and re-saves, bounded by the batch size; a failure naming no entry stays a whole-batch
failure.
Refs #733
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
Joseph Doherty