Bundle E left a gap in ExternalSystem.CachedCall: when the underlying HTTP
call succeeds immediately (WasBuffered=false), the store-and-forward retry
loop is never engaged and the ICachedCallLifecycleObserver hook never
fires. As a result Tracking.Status(id) would stay in Submitted forever and
the audit log would be missing the Attempted + CachedResolve pair the M3
contract requires.
Fix: capture the ExternalCallResult returned by IExternalSystemClient.
CachedCallAsync. When WasBuffered=false, emit the two missing telemetry
packets from the helper itself:
- ApiCallCached / Attempted (per-attempt mechanics row, HttpStatus +
ErrorMessage extracted via the same regex
the synchronous Call() audit row uses)
- CachedResolve / Delivered on Success, or
- CachedResolve / Failed on Success=false (immediate permanent
failure or transient failure without S&F).
The terminal CachedResolve row carries TerminalAtUtc so SiteCallAudit can
recognise the row as eligible for purge.
The WasBuffered=true path is unaffected — the S&F retry loop owns the
Attempted + Resolve emissions there via the CachedCallLifecycleBridge.
Database.CachedWrite is unaffected too because IDatabaseGateway.
CachedWriteAsync always enqueues into S&F (no immediate-success path).
Both new emissions are best-effort: a throwing forwarder is logged and
swallowed (alog.md §7) and each row is independently try/catch-wrapped so
a single fault cannot drop both halves of the terminal pair.
Tests in ExternalSystemCachedCallEmissionTests:
- CachedCall_ImmediateSuccess_EmitsAttemptedAndCachedResolve
- CachedCall_ImmediateFailure_EmitsAttemptedAndCachedResolveFailed
- CachedCall_BufferedPath_DoesNotEmitTerminalTelemetryFromHelper
Full suite: 244 SiteRuntime tests (3 new), 200 Host tests, all green.
Wire the M3 cached-call audit pipeline end-to-end for the database
channel and close the loop between the S&F lifecycle observer and the
site-side dual emitter.
* DatabaseCachedWriteEmissionTests covers Database.CachedWrite (set up
in Bundle E3): mints a TrackedOperationId, emits one CachedSubmit
packet on DbOutbound, threads the id into IDatabaseGateway, and is
best-effort on a thrown forwarder. Mirrors ExternalSystem.CachedCall
coverage from E3.
* CachedCallLifecycleBridge (new) implements ICachedCallLifecycleObserver
and lives alongside CachedCallTelemetryForwarder. The bridge ingests
per-attempt notifications from the S&F retry loop and fans them out
to the forwarder:
- TransientFailure -> 1 Attempted row
- Delivered -> Attempted + CachedResolve(Delivered)
- PermanentFailure -> Attempted + CachedResolve(Parked)
- ParkedMaxRetries -> Attempted + CachedResolve(Parked)
Channel string -> AuditKind mapping (ApiOutbound->ApiCallCached,
DbOutbound->DbWriteCached). Best-effort top-level catch swallows any
unexpected throw so the S&F retry bookkeeping is never disturbed.
* Bridge tests (7) cover all four outcomes, channel mapping, provenance
propagation, and the no-throw-on-forwarder-failure contract.
Bundle F (Host registration) will instantiate the bridge and inject it
into StoreAndForwardService.cachedCallObserver, closing the wiring path
end-to-end.
Bundle E task E6.
Rework ScriptRuntimeContext.ExternalSystem.CachedCall to fit the M3
combined-telemetry model:
* Mints a fresh TrackedOperationId and emits one CachedSubmit packet
via ICachedCallTelemetryForwarder BEFORE handing the call off — the
SiteCalls row is materialised before the first delivery attempt so
Tracking.Status(id) can observe a Submitted row even if immediate
delivery resolves before the helper returns.
* Threads the TrackedOperationId into IExternalSystemClient.CachedCallAsync
as a new optional parameter (and into IDatabaseGateway.CachedWriteAsync
for the Database mirror set up here for E6). The gateway uses the id
as the StoreAndForward messageId so the retry loop (Tasks E4/E5) can
recover it from StoreAndForwardMessage.Id.
* Returns the TrackedOperationId rather than ExternalCallResult — the
script's contract is now "get a tracking handle, observe outcome via
Tracking.Status". Best-effort emission: a thrown forwarder is logged
+ swallowed; the original call still runs and the id is still returned.
DatabaseHelper gets the matching siteId / sourceScript / forwarder
fields and a parallel CachedSubmit emitter (Channel=DbOutbound) so Task
E6's Database.CachedWrite mirror plugs in without further runtime
wiring.
New ICachedCallTelemetryForwarder seam in Commons.Interfaces.Services
so SiteRuntime depends on Commons (existing arrow) rather than
ScadaLink.AuditLog (would have introduced a new dependency).
Bundle E task E3 (and helper-shape work for E6).
Bundle G of Audit Log #23 M2. Bridges the FallbackAuditWriter primary-
failure counter into the Site Health Monitoring report payload so a
sustained audit-write outage surfaces on /monitoring/health instead of
disappearing into a NoOp sink.
- SiteHealthReport: add SiteAuditWriteFailures (defaulted, additive).
- ISiteHealthCollector + SiteHealthCollector: new
IncrementSiteAuditWriteFailures() counter, per-interval reset
semantics matching ScriptErrorCount / DeadLetterCount.
- HealthMetricsAuditWriteFailureCounter: adapter forwarding
IAuditWriteFailureCounter.Increment() to the collector.
- AddAuditLogHealthMetricsBridge(): swaps the NoOp default
registration for the real bridge; called from
SiteServiceRegistration after AddSiteHealthMonitoring + AddAuditLog.
- Existing host-wiring test updated: site composition now resolves
HealthMetricsAuditWriteFailureCounter (not NoOp).
Tests: HealthMonitoring 60 -> 63 (3 new), AuditLog 56 -> 59 (3 new),
full solution green.
Wraps IExternalSystemClient.CallAsync inside ScriptRuntimeContext's
ExternalSystemHelper so every script-initiated ExternalSystem.Call
produces exactly one ApiOutbound/ApiCall AuditEvent via IAuditWriter.
- Captures duration with Stopwatch.GetTimestamp() around the call.
- Builds the audit event with full provenance (SiteId, InstanceId,
SourceScript) and a fresh EventId; ForwardState=Pending.
- Maps Success → AuditStatus.Delivered, Failure (or thrown) → Failed;
parses HTTP {code} out of the ExternalSystemClient's error message
to populate HttpStatus.
- Audit emission is fully best-effort: event-build failures, sync
WriteAsync throws, AND async WriteAsync faults are all logged at
Warning and swallowed so the script's call path is never aborted
by an audit-write failure (alog.md §7).
- Original ExternalCallResult or original exception flows back to the
caller unchanged.
ScriptExecutionActor resolves IAuditWriter from DI and threads it
into ScriptRuntimeContext alongside the existing site identity.
Adds ExternalSystemCallAuditEmissionTests covering: success →
Delivered, HTTP 500 → Failed+httpStatus, HTTP 400 → Failed+httpStatus,
client-thrown network exception → Failed with original exception
re-thrown, audit-writer throw → original result returned, provenance
populated from context, DurationMs recorded.
Refs Audit Log #23 M2 Bundle F.
FU3: thread the executing script identifier from the script-execution
context down to the Notify outbox API so NotifyTarget.Send stamps
NotificationSubmit.SourceScript instead of leaving it null.
- ScriptRuntimeContext / NotifyHelper / NotifyTarget take an optional
sourceScript value, carried through to NotificationSubmit.SourceScript.
- ScriptExecutionActor supplies "ScriptActor:<scriptName>", matching the
Site Event Logging "Source" convention used for script error events.
- AlarmExecutionActor builds the context without the S&F engine, so its
Notify API is inert; sourceScript defaults to null there.
Notify.To(list).Send(subject,body) now generates a NotificationId GUID,
enqueues a Notification-category message into the site Store-and-Forward
Engine, and returns the NotificationId immediately (Task<string>). The
NotificationId is the single idempotency key end-to-end: it is the S&F
message Id, it is carried inside the buffered NotificationSubmit payload,
and it is the id the forwarder submits to central.
NotificationForwarder now deserializes the buffered payload as a
NotificationSubmit and reads NotificationId from it (re-stamping only the
site-owned SourceSiteId / SourceInstanceId), instead of deriving the id
from StoreAndForwardMessage.Id.
Adds NotifyHelper.Status(id): queries central via the site communication
actor; reports the site-local Forwarding state while the notification is
still buffered at the site, maps central's response when found, and
Unknown otherwise. Adds a NotificationDeliveryStatus record.
SiteCommunicationActor gains a NotificationStatusQuery forwarding handler
mirroring NotificationSubmit. StoreAndForwardService.EnqueueAsync gains an
optional messageId parameter and exposes GetMessageByIdAsync.
Conditional and Expression script triggers gain an optional `mode` field
in their TriggerConfiguration JSON:
- OnTrue (default): unchanged edge/per-change firing. An absent mode field
parses as OnTrue, so every existing trigger config behaves identically.
- WhileTrue: fires on the false->true edge, then re-fires on a periodic
timer while the condition holds; stops on the true->false edge. The
re-fire cadence is the script's MinTimeBetweenRuns; with none configured
the trigger degrades to a single edge fire and logs a warning.
ScriptActor tracks condition truth state and manages a dedicated
"whiletrue-trigger" timer. ScriptTriggerConfigCodec and ScriptTriggerEditor
round-trip the mode and expose an OnTrue/WhileTrue selector for the two
trigger kinds. Design: docs/plans/2026-05-18-whiletrue-trigger-mode-design.md
Tests: 7 ScriptActor runtime tests (edge fire, timer re-fire, stop,
re-arm, no-MinTimeBetweenRuns degrade, OnTrue regressions) + 14 codec /
editor tests. SiteRuntime suite 206 green, CentralUI suite 295 green.
InstanceActor._tagPathToAttribute was a Dictionary<string,string> — one tag
path mapped to a single attribute. When two attributes reference the same PLC
node (e.g. two composed cooling-tank modules both reading ns=3;s=Tank.Level,
or a pump's TempSensor and AlarmSensor both reading ns=3;s=Sensor.Reading),
SubscribeToDcl's map assignment overwrote, so only the last-registered
attribute ever received values — the rest stayed permanently Uncertain.
The map is now Dictionary<string,List<string>>; HandleTagValueUpdate fans each
update out to every attribute referencing the tag path, and each distinct tag
path is still subscribed only once per connection.
Adds DeploymentStateQuery request/response contracts (Commons), a site-side
handler (SiteRuntime), a CommunicationService query method (Communication), and
reconciliation in DeploymentService: when a prior record is InProgress or
Failed-on-timeout, query the site; if it already holds the target revision hash
mark the record Success without re-sending; on query failure fall through to a
normal deploy (site-side stale-rejection is the safety net).
Move all package versions into Directory.Packages.props so every project
resolves a single consistent version. Consolidates the Roslyn packages
(Microsoft.CodeAnalysis.CSharp.Scripting/Workspaces) onto 5.0.0, which
resolves the pre-existing NU1608 version-skew error in the test projects.
HandleSetStaticAttribute was made fire-and-forget (commit 2951507) — it no
longer replies with SetStaticAttributeResponse — but three InstanceActor
tests still ExpectMsg<SetStaticAttributeResponse> and timed out. Verify the
mutation via the GetAttributeRequest round-trip instead, which the FIFO
mailbox makes a sound sync point. Test intent (in-memory update, SQLite
persistence, serialized ordering) is unchanged.
Adds a new HiLo alarm trigger type with four configurable setpoints
(LoLo / Lo / Hi / HiHi). Each setpoint carries an optional priority,
deadband (for hysteresis), and operator message. The site runtime emits
AlarmStateChanged with an AlarmLevel field so consumers can differentiate
warning vs critical bands.
Plumbing:
- new AlarmLevel enum + AlarmStateChanged.Level/Message init properties
- AlarmTriggerEditor (Blazor) gets a HiLo render with severity tinting
- AlarmTriggerConfigCodec extracted from the editor for testability
- sitestream.proto carries level + message over gRPC
- SemanticValidator enforces numeric attribute, setpoint ordering,
non-negative deadband
- on-trigger scripts get an Alarm global (Name/Level/Priority/Message)
so notification routing can branch by severity
- per-instance InstanceAlarmOverride entity + EF migration + flattening
step + CLI commands; HiLo overrides merge setpoint-by-setpoint, binary
types whole-replace
- DebugView shows a Level badge + per-band message tooltip
- App.razor auto-reloads on permanent Blazor circuit failure
- docker/regen-proto.sh automates the proto regen workflow (the linux/arm64
protoc segfault means generated files are checked in for now)
Phases 1+2 of the design at
docs/plans/2026-05-12-script-scope-access-design.md.
Adds ergonomic scope-aware accessors to compiled scripts. A script
on a composed TempSensor reads its own attribute via
Attributes["Temperature"]; reaches up to the parent via
Parent.Attributes["SpeedRPM"]; invokes a child script via
Children["TempSensor"].CallScript("Sample"). All resolve to the
existing flat Instance.GetAttribute / SetAttribute / CallScript
delegates by prepending the script's canonical path prefix.
Runtime types (SiteRuntime.Scripts.ScopeAccessors):
AttributeAccessor sync indexer + GetAsync / SetAsync
CompositionAccessor Attributes + CallScript
ChildrenAccessor Children["name"] => CompositionAccessor
ScriptGlobals gains Scope, Attributes, Children, Parent properties.
Sync indexer blocks on the Instance Actor Ask; explicit GetAsync /
SetAsync are also available for callers that want to await.
Plumbing:
- Commons.Types.Scripts.ScriptScope record (SelfPath / ParentPath).
- ResolvedScript.Scope (defaults to ScriptScope.Root for back-compat).
- FlatteningService emits new ScriptScope(prefix, "") for each
composed script so a script defined on TempSensor composed under
a parent gets SelfPath = "TempSensor".
- ScriptActor reads the Scope from its ResolvedScript and forwards
it through ScriptExecutionActor into ScriptGlobals on each call.
RevisionHashService not touched: the per-script canonical name
already encodes the composition path, so any structural change
already flips the hash.
10 new unit tests on the path arithmetic. Site/Template engine
suites stay green (129 + 199).
Editor surface (Phase 3: metadata fetch, Phase 4: completion +
SCADA006 / SCADA007 diagnostics) follows in the next commits.
Replace raw dictionary casting with ScriptParameters wrapper that provides
Get<T>, Get<T?>, Get<T[]>, and Get<List<T>> with clear error messages,
numeric conversion, and JsonElement support for Inbound API parameters.
Switch site host to WebApplicationBuilder with Kestrel HTTP/2 gRPC server,
add GrpcPort/keepalive config, wire SiteStreamManager as ISiteStreamSubscriber,
expose gRPC ports in docker-compose, add site seed script, update all 10
requirement docs + CLAUDE.md + README.md for the new dual-transport architecture.
ClusterClient Sender refs are temporary proxies — valid for immediate reply
but not durable for future Tells. Events now flow as DebugStreamEvent through
SiteCommunicationActor → ClusterClient → CentralCommunicationActor → bridge
actor (same pattern as health reports). Also fix DebugStreamHub to use
IHubContext for long-lived callbacks instead of transient hub instance.
Wired ISiteHealthCollector calls for script errors (ScriptExecutionActor),
alarm eval errors (AlarmActor), dead letters (DeadLetterMonitorActor), and
S&F buffer depth placeholder. Added instance count tracking (deployed/
enabled/disabled) to SiteHealthReport via DeploymentManagerActor. Updated
Health Dashboard UI to show instance counts per site. All metrics flow
through the existing health report pipeline via ClusterClient.
Attributes bound to data connections now initialize with "Uncertain" quality,
distinguishing "never received a value" from "known good" or "connection lost."
Quality is tracked per attribute and included in GetAttributeResponse.
17 source projects (Commons + Host + 15 components) and 17 xUnit test projects.
SLNX format, net10.0, nullable enabled, warnings as errors. All components
reference Commons; Host references all components. Builds and tests clean.
