79 Commits

Author	SHA1	Message	Date
Joseph Doherty	81f09a7054	feat(commons): add IDriverBrowser/IBrowseSession/BrowseNode abstractions	2026-05-28 15:32:01 -04:00
Joseph Doherty	662f3f9f5c	refactor(driver-pages): address Phase 6/8 deep-review findings ... - Topic-name drift fix: DriverHealthChanged.TopicName and DriverControlTopic.Name now live on the message contracts in Commons. AkkaDriverHealthPublisher, DriverStatusSignalRBridge, DriverHostActor, and AdminOperationsActor all delegate to the single constant so a rename can't silently desynchronise publisher and subscriber. - DriverStatusPanel._opResultClearTimer switched from System.Timers.Timer to System.Threading.Timer + awaited DisposeAsync. Prevents an in-flight 8s clear-callback from invoking StateHasChanged on a component whose hub has already been released. - PublishHealthSnapshot deduplicates against the last published (state, lastSuccess, lastError, errorCount) fingerprint. The 30s heartbeat no longer floods the SignalR layer with identical Healthy snapshots — newly-joined clients still warm up via the snapshot store on JoinDriver.	2026-05-28 11:52:20 -04:00
Joseph Doherty	ffcc8d1065	feat(adminui): Reconnect/Restart on DriverStatusPanel (DriverOperator-gated) ... - RestartDriver / ReconnectDriver messages + AdminOperationsActor handlers (broadcast via driver-control DPS topic; audited via ConfigEdits). - DriverHostActor subscribes to driver-control; locates the matching child DriverInstanceActor and stops+respawns it (Restart) or sends it a ForceReconnect internal message (Reconnect — re-enters Reconnecting state without full stop). DriverInstanceSpec constructor call uses named args to handle the full 6-parameter signature. - New DriverOperator authorization policy mapped to DriverOperator or FleetAdmin role; documented in docs/security.md. Map LDAP group via GroupToRole (e.g. "ot-driver-operator": "DriverOperator"). - DriverStatusPanel renders Reconnect + Restart buttons when the user holds the DriverOperator policy (hidden otherwise). Restart requires an in-page Razor confirm block (no JS confirm, keeps SignalR event loop unblocked). Both buttons show a spinner and are disabled during in-flight; result chip auto-clears after 8s. Username sourced from AuthenticationStateProvider. Reconnect resolves to "ForceReconnect" (re-enter Reconnecting, not full stop+respawn) — transport drops and retries while actor and in-memory state are preserved. All DriverInstanceActor states handle ForceReconnect safely (no-op when already in transition).	2026-05-28 11:14:04 -04:00
Joseph Doherty	4b374fd177	feat(adminui): Test Connect button on every typed driver page ... - AdminProbeService routes TestDriverConnect through IAdminOperationsClient with a 65s outer guard (actor side already clamps to [1,60]). - Added generic AskAsync<T> to IAdminOperationsClient interface and AdminOperationsClient impl, delegating straight to the Akka proxy. - DriverTestConnectButton renders the button + inline result chip, auto-clears after 30s, disables during in-flight. - Wired into all 9 typed driver pages directly under the identity section. Sources timeout from the form's ProbeTimeoutSeconds; sources config JSON from the form's current Options (operator can test BEFORE saving).	2026-05-28 11:02:49 -04:00
Joseph Doherty	f3f328c25c	feat(adminops): IDriverProbe + TestDriverConnect actor handler ... - IDriverProbe abstraction in Core.Abstractions; one impl per driver type, resolved by DriverType string. Phase 7.3 + 7.4 add concrete probes for the 9 supported driver types. - TestDriverConnect / TestDriverConnectResult messages. - AdminOperationsActor.HandleTestDriverConnectAsync looks up the probe by DriverType, runs it with a [1,60]s clamped timeout, and returns success/latency or failure/message. Probes that throw or time out surface as soft failures.	2026-05-28 10:44:00 -04:00
Joseph Doherty	4203b84d51	feat(runtime): publish DriverHealthChanged via DriverInstanceActor ... - IDriverHealthPublisher in Core.Abstractions + NullDriverHealthPublisher no-op for tests/dev-stub paths. - AkkaDriverHealthPublisher in Runtime forwards to the cluster-wide `driver-health` DPS topic. - DriverInstanceActor instrumented to publish snapshots on every observable state change + a periodic 30s heartbeat so the AdminUI snapshot store warms up for newly-joined SignalR clients. - Sliding 5-minute Faulted-count tracked per actor via Queue<DateTime>. - DriverHostActor.SpawnChild threads clusterId (_localNode.Value) and the health publisher down to every DriverInstanceActor child. - ServiceCollectionExtensions.AddOtOpcUaRuntime registers AkkaDriverHealthPublisher as IDriverHealthPublisher singleton.	2026-05-28 10:22:44 -04:00
Joseph Doherty	4d5c6ac892	feat(messages): add DriverHealthChanged DPS contract	2026-05-28 10:10:16 -04:00
Joseph Doherty	64e3fbe035	docs: backfill XML documentation across 756 files ... Adds <summary>, <param>, <typeparam>, and <inheritdoc/> tags to public members surfaced by commentchecker — resolves 5,847 of 5,869 issues (99.6%) across three /fixdocs passes.	2026-05-28 08:10:17 -04:00
Joseph Doherty	7dfbca6469	feat(opcua): materialise SystemPlatform tags (Galaxy) as OPC UA variables ... Closes the gap where Tag rows with EquipmentId=NULL + Namespace.Kind=SystemPlatform (Galaxy hierarchy) existed in ConfigDb but were never surfaced in the OPC UA address space. Now they materialise as Variable nodes under a folder named for their FolderPath, browseable through any OPC UA client. Layers touched: - IOpcUaAddressSpaceSink: new EnsureVariable(nodeId, parentFolderId, displayName, dataType) signature on the sink interface, NullSink, DeferredSink, SdkSink. - OtOpcUaNodeManager.EnsureVariable: creates a BaseDataVariableState parented under the named folder (or root), initial Value=null + StatusCode=BadWaitingForInitialData; resolves Tag.DataType strings to the matching OPC UA built-in NodeId. Idempotent. - Phase7CompositionResult: new GalaxyTags collection of GalaxyTagPlan records carrying (TagId, DriverInstanceId, FolderPath, DisplayName, DataType, MxAccessRef). Constructor overloads keep existing call sites compiling. - Phase7Composer.Compose: now takes Tag + Namespace inputs, filters for SystemPlatform-namespace tags with EquipmentId=NULL, emits GalaxyTagPlan rows with MXAccess ref "FolderPath.Name". - Phase7Plan: new AddedGalaxyTags / RemovedGalaxyTags / ChangedGalaxyTags collections + GalaxyTagDelta record; IsEmpty + needsRebuild updated. - Phase7Planner.Compute: diffs GalaxyTags by TagId via existing DiffById helper. - DeploymentArtifact.ParseComposition: reads the Tags + Namespaces + DriverInstances arrays the ConfigComposer already emits, applies the same SystemPlatform filter, returns the same GalaxyTagPlan list as the composer so artifact-side and compose-side plans agree. - Phase7Applier: new MaterialiseGalaxyTags pass that ensures one folder per distinct FolderPath then one Variable per tag. NodeId for the variable is "<FolderPath>.<Name>" matching the MXAccess ref so the future Galaxy SubscribeBulk wiring can address them directly. - OpcUaPublishActor.RebuildAddressSpace: invokes MaterialiseGalaxyTags after MaterialiseHierarchy. _lastApplied initialiser updated for the new ctor. - seed-clusters.sql: pre-existing TestMachine_001.TestAlarm001..003 rows needed no change — the composer/applier now picks them up automatically. Verified end-to-end via docker-dev: deploy click → driver-a logs "Phase7Applier: Galaxy tags materialised (tags=3, folders=1)" → OPC UA Client CLI browses the three Variable nodes under TestMachine_001 folder. Reads return BadWaitingForInitialData status (expected — Galaxy driver's SubscribeBulk wiring to push values into the nodes is the remaining follow-up).	2026-05-26 15:43:22 -04:00
Joseph Doherty	607dc51dec	feat(opcua): #85 UNS Area/Line/Equipment folder hierarchy in SDK ... Phase7Composer now carries UnsAreaProjection + UnsLineProjection lists so the applier can materialise the full UNS topology in the OPC UA address space. New IOpcUaAddressSpaceSink.EnsureFolder(folderNodeId, parentNodeId, displayName) seam (no-op default, recorded in tests, forwarded by DeferredAddressSpaceSink, implemented by SdkAddressSpaceSink). The SDK- side OtOpcUaNodeManager gains an EnsureFolder API that creates FolderState nodes with proper parent linkage; RebuildAddressSpace now clears folders too so re-applies don't accumulate stale topology. Phase7Applier.MaterialiseHierarchy walks composition.UnsAreas → composition.UnsLines → composition.EquipmentNodes, calling EnsureFolder with the correct parent at each level. Idempotent — calling twice with the same composition is a no-op. OpcUaPublishActor.HandleRebuild invokes it after Phase7Applier.Apply so OPC UA clients browsing the server now see Area/Line/Equipment as proper folders rather than flat tag ids. DeploymentArtifact.ParseComposition reads UnsAreas + UnsLines from the JSON snapshot the ControlPlane emits, populating the new fields when present. Phase7Composer.Compose now accepts UnsAreas + UnsLines; a 3-arg overload preserves the old signature for legacy callers + existing tests. The Phase7CompositionResult convenience ctor likewise keeps the planner tests working without UNS data. 3 new hierarchy tests (pure unit + boot-verify against a real OtOpcUaSdkServer); OpcUaServer suite is 48/48 green (was 45, +3), Runtime 74/74 unchanged. Closes #85.	2026-05-26 10:48:56 -04:00
Joseph Doherty	2697af31d1	feat(opcua,host): #81 ServiceLevel SDK publisher ... SdkServiceLevelPublisher writes Server.ServiceLevel through the SDK's ServerObjectState — the standard OPC UA non-transparent-redundancy signal clients use to pick a primary. Writes are guarded by DiagnosticsLock so concurrent SDK diagnostics scans don't fight with our updates. DeferredServiceLevelPublisher mirrors the DeferredAddressSpaceSink late- binding pattern: Akka actors resolve IServiceLevelPublisher at construction, hosted service swaps the SDK publisher in after StandardServer.Start. Host Program.cs registers DeferredServiceLevelPublisher as the singleton bound to IServiceLevelPublisher; OtOpcUaServerHostedService gets it injected and fills it once IServerInternal is available. Tests boot a real StandardServer on a free port (cross-platform), call Publish, then verify ServerObject.ServiceLevel.Value reflects the write. 5 new tests; OpcUaServer suite now 45/45 green (was 40, +5). Closes #81 residual. Unblocks Task 60 (OPC UA dual-endpoint + ServiceLevel tests).	2026-05-26 10:37:42 -04:00
Joseph Doherty	52997ee164	feat(observability): F13d Prometheus + OpenTelemetry instrumentation ... OtOpcUaTelemetry (Commons/Observability) centralizes the project's Meter + ActivitySource so all instrumentation points emit through a single named surface. Counters cover the hot paths: otopcua.deploy.applied (outcome=ack|reject) otopcua.deploy.apply.duration (s, histogram) otopcua.driver.lifecycle (event=spawn|spawn_stub|stop|fault) otopcua.virtualtag.eval (outcome=ok|fail|skip) otopcua.scriptedalarm.transition (state=activated|acknowledged|cleared) otopcua.opcua.sink.write (kind=value|alarm|rebuild) otopcua.redundancy.service_level_change (level=byte) Plus two ActivitySource spans: otopcua.deploy.apply wraps DriverHostActor.ApplyAndAck otopcua.opcua.address_space_rebuild wraps OpcUaPublishActor.HandleRebuild Instruments are no-op until a listener attaches, so tests + dev hosts pay nothing for unread telemetry. Host Program.cs gains AddOtOpcUaObservability() (binds the OtOpcUa Meter + ActivitySource to OpenTelemetry, attaches a Prometheus exporter) and MapOtOpcUaMetrics() (mounts /metrics scrape endpoint). Driver-side internals + ASP.NET request metrics deliberately stay off — the scrape payload is scoped to OtOpcUa signals only. Tests use MeterListener + ActivityListener to verify VirtualTagActor.eval, OpcUaPublishActor.AttributeValueUpdate, and RebuildAddressSpace actually emit on the central instruments. Runtime suite is 72 / 72 green (+3). Closes #105. Path A (F13b/c/d) complete; next batch options: #85 UNS folder hierarchy in SDK, or F8b/F9b production engine bindings.	2026-05-26 10:29:40 -04:00
Joseph Doherty	50787823d3	feat(host,runtime): #108 Host DI bindings — OPC UA server + deferred sink ... Wires the OPC UA SDK into the fused Host's lifecycle on driver-role nodes + spawns OpcUaPublishActor with the proper sink/publisher/dbFactory/ applier resolution. The full read+write data path is now live in production: Deploy → DriverHost → OpcUaPublish → SDK NodeManager → subscribed OPC UA clients. DeferredAddressSpaceSink (Commons.OpcUa): - Thread-safe wrapper IOpcUaAddressSpaceSink that delegates to an inner sink swapped in at runtime. Needed because Akka actors resolve the sink at construction time, but the production sink (SdkAddressSpaceSink wrapping OtOpcUaNodeManager) only exists after the SDK StandardServer has started. - Defaults to NullOpcUaAddressSpaceSink so calls before swap are safe; SetSink(null) reverts (for graceful shutdown). OtOpcUaServerHostedService (Host.OpcUa): - IHostedService that owns the OPC UA SDK lifecycle. Reads OpcUaApplicationHostOptions from the 'OpcUa' config section, creates an OtOpcUaSdkServer, boots it through OpcUaApplicationHost, then swaps a real SdkAddressSpaceSink into the DeferredAddressSpaceSink singleton. - SDK boot failure is logged + non-fatal — the rest of the host (admin UI, driver actors) keeps running. Stop reverts to null sink. WithOtOpcUaRuntimeActors (Runtime): - Now spawns OpcUaPublishActor (new actor) + threads its ActorRef into DriverHostActor's Props so successful applies trigger the address-space rebuild pipeline. - Phase7Applier is constructed here from the resolved sink + a logger; OpcUaPublishActor takes both. - Prepends the opcua-synchronized-dispatcher HOCON so the extension is self-contained — consumers (Host, tests) don't need to redeclare the dispatcher block. - New OpcUaPublishActorKey + OpcUaPublishActorName for actor-registry resolution. - AddOtOpcUaRuntime now also TryAddSingleton's NullOpcUaAddressSpaceSink + NullServiceLevelPublisher so admin-only nodes (or tests that don't bind the Deferred sink) stay safe. Host.Program.cs (driver-role only): - Binds DeferredAddressSpaceSink as singleton + as IOpcUaAddressSpaceSink - AddHostedService<OtOpcUaServerHostedService>() Tests: OpcUaServer 24 -> 28 (+4 DeferredAddressSpaceSink unit tests), Runtime 69 -> 69 (existing ServiceCollectionExtensionsTests extended to verify the new mux + publish actor registration). All 6 v2 test suites green: 177 tests passing. Closes #108. Engine-wiring is now production-bound end-to-end on driver-role nodes — Deploy reaches real OPC UA Variable nodes that subscribed clients see.	2026-05-26 10:02:15 -04:00
Joseph Doherty	f427dc4f26	feat(runtime): #112 ScriptedAlarmActor state persistence via IAlarmActorStateStore ... ScriptedAlarmActor now survives actor restart: PreStart loads from the configured store + restores in-memory state; every Transition() fires a fire-and-forget save. ActiveState still re-derives from the evaluator on first tick (Phase 7 decision #14), but Acked state + lastAckUser persist verbatim so operators don't re-ack across an outage. Three pieces: - IAlarmActorStateStore seam in Commons.Engines, with the AlarmActorStateSnapshot record (alarmId / state / lastTransitionUtc / lastAckUser) and NullAlarmActorStateStore default. - EfAlarmActorStateStore in Runtime.ScriptedAlarms — production adapter over the existing ScriptedAlarmState table in ConfigDb. Maps the actor's 3-state enum to the table's AckedState column (Active⇒Unacknowledged, Acknowledged⇒Acknowledged, Inactive⇒ Acknowledged). Concurrency conflicts are logged + dropped — the next transition writes again. - ScriptedAlarmActor PreStart load (async, piped back as StateRestored) + Transition save. New Props overload takes the store; default is NullAlarmActorStateStore so tests stay quiet. Tests: Runtime 52 -> 57 (+5): - Transition writes Active then Acknowledged snapshots with lastAckUser populated - PreStart with persisted Active state restores so a subsequent AcknowledgeAlarm fires (not ignored as it would be from Inactive) - Empty store boots Inactive (AcknowledgeAlarm correctly ignored) - EfAlarmActorStateStore Save + Load round-trips via in-memory EF - Load for unknown alarmId returns null All 6 v2 test suites green: 157 tests passing. Closes #112. F9 (#80) remaining residual is predicate binding to Core.ScriptedAlarms.ScriptedAlarmEngine — split as F9b in tasks JSON.	2026-05-26 09:34:37 -04:00
Joseph Doherty	a1325299ce	feat(runtime): F10 OpcUaPublishActor sink seams + redundancy-driven ServiceLevel ... OpcUaPublishActor now routes through pluggable seams instead of just incrementing a counter: - IOpcUaAddressSpaceSink (Commons.OpcUa) — WriteValue / WriteAlarmState / RebuildAddressSpace. OpcUaQuality enum moved here from the actor's nested type so producers don't have to reference the actor itself. - IServiceLevelPublisher — Publish(byte). NullServiceLevelPublisher retains the last level for inspection. - The actor subscribes to the redundancy-state DPS topic in PreStart and maps the local node's NodeRedundancyState to a coarse ServiceLevel (Primary+leader=240, Primary=200, Secondary=100, Detached=0). This keeps the local SDK's ServiceLevel node honest without round-tripping back through the admin-singleton calculator. - ServiceLevelChanged dedupes identical levels so the SDK doesn't see redundant writes. - Sink + publisher exceptions are caught and logged; the actor never crashes its own dispatcher. - PropsForTests gets optional sink/publisher/localNode params and skips the DPS subscribe so unit tests stay on a vanilla TestKit cluster. Production binding to a real SDK NodeManager + Variable nodes is the remaining residual — split as F10b. Task 60 still blocked on F10b. Tests: Runtime 40 -> 46 (+6): - AttributeValueUpdate routes to sink - AlarmStateUpdate routes to sink - RebuildAddressSpace calls sink.Rebuild - ServiceLevelChanged dedupes - RedundancyStateChanged for primary-leader publishes 240 - RedundancyStateChanged for secondary publishes 100 All 6 v2 test suites green: 132 tests passing.	2026-05-26 09:10:55 -04:00
Joseph Doherty	14fb2b05ed	feat(runtime): F8/F9 engine evaluator seams + DPS fan-out ... VirtualTagActor and ScriptedAlarmActor now route through pluggable evaluator interfaces and fan out to the cluster's live-tail topics shipped in F15.3: - IVirtualTagEvaluator + NullVirtualTagEvaluator in Commons.Engines. VirtualTagActor calls evaluator on every DependencyValueChanged, dedupes unchanged values, forwards EvaluationResult to its parent, and publishes ScriptLogEntry Warning to the script-logs DPS topic whenever the evaluator fails. - IScriptedAlarmEvaluator + NullScriptedAlarmEvaluator. ScriptedAlarmActor takes an AlarmConfig (id/name/equipment-path/severity/predicate) and publishes both an AlarmTransitionEvent (alerts topic) and a ScriptLogEntry (script-logs topic) at every transition. Manual ConditionMet/Acknowledge/Cleared still flow through the same Transition() so callers without engine bindings still drive the state machine; the legacy single-string Props() overload routes through a default AlarmConfig. The Null* defaults keep the actors safe when no engine is bound — unconfigured nodes never spuriously alarm. Production binding to Core.VirtualTags.VirtualTagEngine and Core.ScriptedAlarms is the remaining residual (F8b/F9b — split in tasks JSON). Tests: Runtime 34 -> 40 (+6): - VirtualTagActorTests x3 (evaluator drives EvaluationResult, unchanged-value dedup, failure publishes Warning ScriptLogEntry) - ScriptedAlarmActorTests x3 (engine threshold drives Activated + Cleared on alerts topic, manual Acknowledge attribution). All 6 v2 test suites green: 126 tests passing.	2026-05-26 09:05:04 -04:00
Joseph Doherty	da141497f8	feat(runtime): F7 spawn lifecycle + F20 ShouldStub gate ... DriverHostActor.ApplyAndAck now reads the deployment artifact and reconciles its set of DriverInstanceActor children — spawn the missing, ApplyDelta to those with changed config, stop the removed/disabled. The diff lives in pure DriverSpawnPlanner so it can be unit-tested without an ActorSystem. Adds IDriverFactory in Core.Abstractions (consumed by Runtime) + DriverFactoryRegistryAdapter in Core.Hosting that wraps the existing v1 DriverFactoryRegistry — Runtime stays decoupled from Polly/Serilog, the Host wires the adapter once driver assemblies have registered. ShouldStub(type, roles) is now actually called on every spawn — Galaxy + Wonderware-Historian boot stubbed on macOS/Linux or whenever the host carries the dev role. Missing factory ⇒ stub fallback, never a crash. Tests: 24 → 34 in Runtime (+10): - DriverSpawnPlannerTests x7 (diff cases, type change ⇒ stop+respawn) - DeploymentArtifactTests x5 (empty/malformed/missing fields tolerant) - DriverHostActorReconcileTests x4 (spawn count, stub fallback, ShouldStub gate, second-apply stops the removed) All 6 v2 test suites green: 120 tests passing. Closes F20 (ShouldStub wired). F7 marked partial — subscription publishing + write path still stubbed in DriverInstanceActor itself.	2026-05-26 08:57:16 -04:00
Joseph Doherty	59858129cb	feat(adminui): F15.3 closes F15 — live alerts/script-log, CSV import, Monaco editor ... Final F15 batch wires up the SignalR-backed live pages, ports the bulk equipment importer, and progressively enhances the Script source editor with Monaco. Message contracts: - Commons.Messages.Alerts.AlarmTransitionEvent — fires on every alarm state transition; published on the `alerts` DPS topic by future ScriptedAlarmActor (F9) emits. - Commons.Messages.Logging.ScriptLogEntry — one log line emitted by a hosted script; published on the `script-logs` DPS topic by future VirtualTagActor (F8) + ScriptedAlarmActor (F9) emits. (Folder named "Logging" to dodge .gitignore's "logs/" rule.) SignalR plumbing: - AlertHub gains MethodName + bridge actor (AlertSignalRBridge) - ScriptLogHub introduced; ScriptLogSignalRBridge follows the same DPS-subscribe → IHubContext fan-out pattern as FleetStatusSignalRBridge - WithOtOpcUaSignalRBridges now spawns all three bridges - MapOtOpcUaHubs maps /hubs/script-log alongside the existing hubs Pages: - /alerts live alarm tail, 200-row capacity - /script-log live script-log tail with level + script filter, 500-row capacity - /clusters/{id}/equipment/import — CSV bulk Equipment add with preview (Name/MachineCode/UnsLineId/Driver + optional ZTag/SAPID/Manufacturer/Model; skips rows whose MachineCode already exists in the fleet) - ScriptEdit progressively enhanced with Monaco editor via JSInterop — the textarea remains Blazor's source of truth and Monaco syncs into it on every keystroke so @bind keeps working; falls back gracefully if the CDN is unreachable. MainLayout nav gains a "Live" section (Deployments, Alerts, Alarms historian) and a "Scripts" link under Scripting. ClusterEquipment surfaces the new Import CSV button. Tally: F15 ships ~42 razor pages + 3 SignalR hubs + 3 bridge actors. Microsoft.AspNetCore.SignalR.Client added (was already in central PM). All 104 v2 tests remain green.	2026-05-26 08:39:17 -04:00
Joseph Doherty	8f32b89fb9	feat(adminui): FleetDiagnosticsClient real Akka ActorSelection round-trip (F17) ... - New Commons.Messages.Fleet.GetDiagnostics request record. - DriverHostActor handles GetDiagnostics in all three states (Steady, Applying, Stale); replies with a NodeDiagnosticsSnapshot built from _currentRevision + the local NodeId. Drivers list is empty until F7 wires the per-instance children. - FleetDiagnosticsClient now resolves the target via ActorSelection at akka.tcp://{system}@{nodeId}/user/driver-host and Asks with a 3s timeout. On timeout/peer-down it returns an empty snapshot so the UI degrades gracefully rather than throwing. Two new integration tests in Host.IntegrationTests: - GetDiagnostics_returns_snapshot_with_target_NodeId verifies the cross-node Ask/Reply works. - GetDiagnostics_after_deploy_reports_current_revision exercises the end-to-end path: AdminOps starts a deployment, both DriverHostActors apply, then diagnostics reports the new revision on both nodes. All 98 v2 tests pass (was 96 + 2 new).	2026-05-26 06:58:11 -04:00
Joseph Doherty	f57f61deac	feat(audit): EventId + CorrelationId columns + filtered unique index (F3 + F4) ... ConfigAuditLog gains two nullable columns (EventId, CorrelationId) + a filtered unique index UX_ConfigAuditLog_EventId. EF migration 20260526105027_AddConfigAuditLogEventIdColumns is additive (nullable + filtered index = legacy rows backfill cleanly). AuditWriterActor now writes EventId + CorrelationId into the dedicated columns instead of synthesising a JSON wrapper into DetailsJson. Cross-restart dedup is now real: a retry of an already-flushed batch hits the unique index and SaveChanges throws; the existing catch drops the duplicate without losing the rest of the batch. WrapDetails helper deleted — F4 (its JSON hardening) becomes moot. AuditWriterActorTests.Details_wrapper_embeds_eventId_and_correlationId renamed + rewritten to assert against the columns. All 29 ControlPlane tests pass, all 95 v2 tests green.	2026-05-26 06:52:53 -04:00
Joseph Doherty	5cfbe8b5dd	test(host): deploy happy-path + idempotency integration tests (Task 59) ... DeployHappyPathTests exercises the full deploy pipeline on the 2-node harness: AdminOperationsActor → ConfigPublishCoordinator → DistributedPubSub → DriverHostActor on both nodes → ApplyAck → coordinator seals. Verifies both NodeDeploymentState rows reach Applied and Deployment.Status reaches Sealed. Exposed + fixed two production bugs along the way: 1. Coordinator was publishing DispatchDeployment on the "deployments" topic but never subscribed to anything — DriverHostActor ACKs published on the same topic could not reach it. Added dedicated "deployment-acks" topic with coordinator subscription in PreStart, and DriverHostActor publishes ACKs there. 2. NodeId derivation used member.Address.Host only — two cluster members on a shared loopback host (test harness, dev VMs) collided to one identity. The coordinator's expected-ack set became {1} and the system sealed after only half the nodes acked. Switched to host:port everywhere (ClusterRoleInfo + coordinator) so loopback nodes stay distinct and production identities are harmlessly more specific. Tests: 95 v2 tests pass (was 93 + 2 deploy tests), 0 skipped. Failover scenarios (design §8 cases 3-7: node-kill-mid-apply, split-brain, restart-during-deploy) deferred — they need controlled node-down primitives on the harness. Tracked as F22 (failover scenario test cases).	2026-05-26 06:34:36 -04:00
Joseph Doherty	d6fac2d81d	test(host): 2-node integration test harness + consolidate to one ActorSystem (Task 58) ... Builds TwoNodeClusterHarness: two in-process Host-equivalent nodes sharing an in-memory ConfigDb. Forms a 2-member Akka cluster. ClusterFormationTests proves both nodes see each other as admin+driver role members. Fixes a real production bug uncovered while wiring the harness — Program.cs ran two separate ActorSystems (one from AddOtOpcUaCluster.AkkaHostedService with cluster HOCON, one from Akka.Hosting.AddAkka with bare HOCON). Cluster singletons landed on the bare ActorSystem and could not actually form a cluster ("Configuration does not contain `akka.cluster` node"). Consolidation: - AddOtOpcUaCluster now only binds AkkaClusterOptions + registers IClusterRoleInfo - New WithOtOpcUaClusterBootstrap pushes embedded HOCON + Remote/Cluster options into Akka.Hosting's AkkaConfigurationBuilder - AkkaHostedService.cs deleted — Akka.Hosting now owns the lifecycle - Program.cs + harness call WithOtOpcUaClusterBootstrap inside AddAkka Why not WebApplicationFactory<Program>? Program.cs reads OTOPCUA_ROLES from process env (shared across in-process WAFs); the harness replays Program.cs's DI graph from a clean WebApplicationBuilder per node with per-node config overrides. Same production extensions, isolated config + Kestrel + Akka ports. Tests: 93 v2 tests pass (was 91 + 2 new cluster formation), 0 skipped.	2026-05-26 06:27:04 -04:00
Joseph Doherty	e2b357f89a	feat(host): role-gated Program.cs composes all v2 components	2026-05-26 05:22:59 -04:00
Joseph Doherty	c217c49f69	feat(cluster): ClusterRoleInfo wraps Akka.Cluster for app-facing role queries	2026-05-26 04:31:07 -04:00
Joseph Doherty	dfb06368cd	feat(cluster): parse OTOPCUA_ROLES env var with validation	2026-05-26 04:31:06 -04:00
Joseph Doherty	f184f8ed1b	feat(cluster): AkkaHostedService and DI extension	2026-05-26 04:31:05 -04:00
Joseph Doherty	3d0f4dc168	feat(cluster): embed Akka HOCON config matching ScadaLink tuning	2026-05-26 04:31:03 -04:00
Joseph Doherty	136234e7f2	feat(commons): add cluster/admin/diagnostics client interfaces	2026-05-26 04:27:19 -04:00
Joseph Doherty	5d3a5a40d7	feat(commons): add deploy/admin/audit/redundancy/fleet message contracts	2026-05-26 04:27:18 -04:00
Joseph Doherty	fee4a8c008	feat(commons): add correlation/execution/node/deployment/revisionhash types	2026-05-26 04:26:01 -04:00
Joseph Doherty	605dbf3dcc	feat(configdb): V2HostingAlignment migration consolidating Phase 1a-1e ... Phase 1f — the consolidator migration. Closes out the v2 entity-model rewrite by emitting a single EF migration that captures the cumulative schema delta from 14a (RowVersion) through 14e (drop generation entities). Generated: src/Core/ZB.MOM.WW.OtOpcUa.Configuration/Migrations/ 20260526081556_V2HostingAlignment.cs (1562 lines) 20260526081556_V2HostingAlignment.Designer.cs Migration shape (per `grep -nE migrationBuilder.\(...)`): Drop 12 ForeignKey constraints (one per live-edit entity's GenerationId FK) Drop 2 Tables (ConfigGeneration, ClusterNodeGenerationState) Drop 45 Indexes (every UX_*_Generation_* and IX_*_Generation_* across the 13 live-edit tables — 1 also dropped the unique-Primary filtered index UX_ClusterNode_Primary_Per_Cluster) Drop 13 Columns (12 GenerationId + 1 RedundancyRole) Add 12 RowVersion columns (one per live-edit entity) Create 4 Tables (Deployment, NodeDeploymentState, ConfigEdit, DataProtectionKeys) Create ~45 Indexes (recreated under the new naming pattern UX_<Table>_LogicalId / UX_<Table>_<X> with the GenerationId column stripped from composite keys) Notable EF quirks accepted: Unique-on-required-column indexes (UX_VirtualTag_LogicalId etc.) ship a `filter: "[VirtualTagId] IS NOT NULL"` clause that EF auto-inserts for SQL Server. Harmless — the column is C#-side `required` so NULL never appears. Verification: dotnet build src/Core/ZB.MOM.WW.OtOpcUa.Configuration -> 0 errors dotnet ef migrations script --idempotent (against placeholder DSN) -> 3259-line .sql produced OK tests/Core/ZB.MOM.WW.OtOpcUa.Configuration.Tests -> 0 errors Live `dotnet ef database update` against a scratch SQL Server deferred to Task 15 (Migrate-To-V2.ps1) — SSH to the docker host needs a key/password I don't have, and the always-on SQL at 10.100.0.35,14330 uses Integrated Security (Windows auth, unreachable from this macOS dev). The migration itself is structurally correct by construction (EF tooling generated it against the live DbContext model); the live-DB confidence step is the PowerShell wrapper's job. SchemaComplianceTests updates: - All_expected_tables_exist: removed ConfigGeneration + ClusterNodeGenerationState; added Deployment, NodeDeploymentState, ConfigEdit, DataProtectionKeys. - Filtered_unique_indexes_match_schema_spec: removed entries for UX_ClusterNode_Primary_Per_Cluster (Task 14d) and UX_ConfigGeneration_Draft_Per_Cluster (Task 14e). Two filtered uniques remain (UX_ClusterNodeCredential_Value, UX_ExternalIdReservation_KindValue_Active). - Check_constraints_match_schema_spec: added CK_ConfigEdit_FieldsJson_IsJson. StoredProceduresTests update: - Removed RedundancyRole + 'Primary' from the raw INSERT into ClusterNode so the DB-backed test runs against the new schema.	2026-05-26 04:18:50 -04:00
Joseph Doherty	e00f46d723	refactor(configdb): delete ConfigGeneration + ClusterNodeGenerationState ... Phase 1e of the v2 entity-model rewrite. With the FKs gone (Task 14b) and the apply pipeline replaced (Task 14c), the v1 draft/publish entities have no remaining v2 consumers. Deleted entity classes: src/Core/ZB.MOM.WW.OtOpcUa.Configuration/Entities/ConfigGeneration.cs src/Core/ZB.MOM.WW.OtOpcUa.Configuration/Entities/ClusterNodeGenerationState.cs Deleted enum classes (no v2 consumers): src/Core/ZB.MOM.WW.OtOpcUa.Configuration/Enums/GenerationStatus.cs src/Core/ZB.MOM.WW.OtOpcUa.Configuration/Enums/NodeApplyStatus.cs OtOpcUaConfigDbContext changes: - Removed DbSet<ConfigGeneration> ConfigGenerations - Removed DbSet<ClusterNodeGenerationState> ClusterNodeGenerationStates - Removed ConfigureConfigGeneration(modelBuilder) call + method body - Removed ConfigureClusterNodeGenerationState(modelBuilder) call + body - Tidied the "v2 deploy-model tables" header comment Navigation property cleanup: - ServerCluster.Generations collection -> removed - ClusterNode.GenerationState navigation -> removed doc-comment cref cleanup (replaced <see cref="X"/> with <c>X</c> for the deleted types so the C# XML comment compiler doesn't fail with CS1574): - Deployment.cs (cref to ConfigGeneration) - NodeDeploymentState.cs (cref to ClusterNodeGenerationState) - Core/OpcUa/EquipmentNodeWalker.cs (cref to ConfigGeneration in the EquipmentNamespaceContent record's doc-comment; while there, removed "All four collections are scoped to the same ConfigGeneration" since that's no longer true in v2) Verification: src/Core/ZB.MOM.WW.OtOpcUa.Configuration -> 0 errors src/Core/ZB.MOM.WW.OtOpcUa.Core -> 0 errors tests/Core/ZB.MOM.WW.OtOpcUa.Configuration.Tests -> 0 errors tests/Core/ZB.MOM.WW.OtOpcUa.Core.Tests -> 0 errors whole solution -> 15 errors (all in Server/Admin; transitive Server.Tests/Admin.Tests skip per the parent's failure, so the per-project count dropped vs Task 14d's 71)	2026-05-26 04:14:55 -04:00
Joseph Doherty	3c915e652e	refactor(configdb): drop ClusterNode.RedundancyRole (replaced by Akka leader) ... Phase 1d of the v2 entity-model rewrite. The static RedundancyRole column is replaced by Akka cluster's role-leader-of-"driver" election at runtime (see RedundancyStateActor + ServiceLevelCalculator in Task 35). Changes: - Removed `public required RedundancyRole RedundancyRole` from ClusterNode entity. - Removed `e.Property(x => x.RedundancyRole).HasConversion<string>()...` mapping from OtOpcUaConfigDbContext.ConfigureClusterNode. - Removed the `UX_ClusterNode_Primary_Per_Cluster` filtered unique index (filter referenced [RedundancyRole]='Primary'). - Dropped `using ZB.MOM.WW.OtOpcUa.Configuration.Enums` from ClusterNode.cs (no longer needed). - Deleted `Enums/RedundancyRole.cs` — the enum is unused in v2-kept code. - DraftValidator: dropped the "exactly one Primary per cluster" validation block. Comment in place explaining v2 picks primary at runtime via Akka. - DraftValidatorTests: dropped ValidateClusterTopology_flags_multiple_Primary test; reworked BuildNode helper to no longer take a `role` argument. Untouched (Server + Admin still reference RedundancyRole; accepted broken per Task 56 policy): src/Server/ZB.MOM.WW.OtOpcUa.Server/Redundancy/{ClusterTopologyLoader, RedundancyStatePublisher, RedundancyTopology, ServiceLevelCalculator}.cs src/Server/ZB.MOM.WW.OtOpcUa.Admin/Services/RedundancyMetrics.cs DB-runtime tests will fail against the new schema (Task 14f's migration drops the column) — to be updated in Task 14f's SchemaComplianceTests update: - SchemaComplianceTests.cs:55 (expected filtered index list) - StoredProceduresTests.cs:263 (raw INSERT names the column) Verification: src/Core/ZB.MOM.WW.OtOpcUa.Configuration -> 0 errors tests/Core/ZB.MOM.WW.OtOpcUa.Configuration.Tests -> 0 errors whole solution -> 71 errors (70 from Task 14b in Server/Admin, +1 new Server/Redundancy reference)	2026-05-26 04:11:57 -04:00
Joseph Doherty	1ddf8bb50e	refactor(configdb): delete v1 Apply pipeline (replaced by AdminOperationsActor) ... Phase 1c of the v2 entity-model rewrite. Deletes the draft/publish lifecycle machinery that v2 replaces with AdminOperationsActor + ConfigComposer + DriverInstanceActor.ApplyDelta. Deleted (6 files): src/Core/ZB.MOM.WW.OtOpcUa.Configuration/Apply/ IGenerationApplier.cs — interface for the apply pipeline GenerationApplier.cs — the v1 applier coordinating per-driver hook-back GenerationDiff.cs — typed wrapper over the sp_ComputeGenerationDiff SQL output ApplyCallbacks.cs — per-driver hook surface invoked by the applier ChangeKind.cs — enum {Added, Modified, Removed, Unchanged} tests/Core/ZB.MOM.WW.OtOpcUa.Configuration.Tests/GenerationApplierTests.cs The empty Apply/ directory is removed. Kept (repurposed in Task 39 for stale-config fallback): src/Core/ZB.MOM.WW.OtOpcUa.Configuration/LocalCache/GenerationSealedCache.cs src/Core/ZB.MOM.WW.OtOpcUa.Configuration/LocalCache/ResilientConfigReader.cs tests/Core/ZB.MOM.WW.OtOpcUa.Configuration.Tests/GenerationSealedCacheTests.cs tests/Core/ZB.MOM.WW.OtOpcUa.Configuration.Tests/ResilientConfigReaderTests.cs Naming rename (GenerationSealedCache -> DeploymentArtifactCache) deferred to Task 39 (DriverHostActor stale-config fallback) where the consumer is written. The type stays available under its v1 name until then. IDriver.cs doc-comment: replaced the "Used by IGenerationApplier..." sentence with "Invoked by the v2 DriverInstanceActor when ApplyDelta reports that only this driver's config changed in the new deployment." Server/Admin breakage from Task 14b unchanged (70 errors). Configuration + Core.Tests + Configuration.Tests stay green. src/Core/ZB.MOM.WW.OtOpcUa.Configuration -> 0 errors tests/Core/ZB.MOM.WW.OtOpcUa.Configuration.Tests -> 0 errors whole solution -> 70 errors (all in Server/Admin)	2026-05-26 04:09:17 -04:00
Joseph Doherty	13d3aeab09	refactor(configdb): drop GenerationId FK from live-edit entities ... Phase 1b of the v2 entity-model rewrite. The design's live-edit model means the 12 v2 live-edit entities no longer carry a generation scope — they're edited directly via AdminOperationsActor, with RowVersion (added in Task 14a) providing last-write-wins detection. Entity changes (12 files): Equipment, DriverInstance, Device, Tag, PollGroup, Namespace, UnsArea, UnsLine, NodeAcl, Script, VirtualTag, ScriptedAlarm - Removed: public long GenerationId - Removed: public ConfigGeneration? Generation (navigation) DbContext changes (OtOpcUaConfigDbContext.cs): - Removed 12 HasOne(x => x.Generation).WithMany().HasForeignKey... mappings - Rewrote ~36 indexes: dropped the GenerationId column from each composite key, renamed UX_<Table>_Generation_<X> -> UX_<Table>_<X> and IX_<Table>_Generation_<X> -> IX_<Table>_<X>. Logical IDs become globally unique (UX_<Table>_LogicalId on the LogicalId column alone). - Removed Namespace's redundant UX_Namespace_Generation_LogicalId_Cluster index (subsumed by the new UX_Namespace_LogicalId). Core.Tests fixtures (4 files): Removed "GenerationId = 1," lines from: - PermissionTrieBuilderTests.cs (NodeAcl Row factory) - PermissionTrieTests.cs (NodeAcl Row factory) - TriePermissionEvaluatorTests.cs (NodeAcl Row factory + 2 gen{1,5}Row mutations that test stale-generation evaluation; the trie itself still carries a generation tag via PermissionTrie.GenerationId, fed in via PermissionTrieBuilder.Build's generationId parameter, so the tests still exercise the production code path) - EquipmentNodeWalkerTests.cs (Area/Line/Eq/Tag/VirtualTag/ScriptedAlarm builders) Expected breakage (accepted per Task 56 policy): src/Server/ZB.MOM.WW.OtOpcUa.Server ~25 errors (DriverInstanceBootstrapper, AuthorizationBootstrap, EquipmentNamespaceContentLoader, Phase7Composer, ...) src/Server/ZB.MOM.WW.OtOpcUa.Admin ~45 errors (VirtualTags.razor, ScriptedAlarms.razor, DriverInstanceService, EquipmentService, EquipmentImportBatchService, UnsService, FocasDriverDetailService, ...) Server.Tests, Admin.Tests, Admin.E2ETests also break transitively (they project-reference Server/Admin). All deleted in Task 56. Verification: dotnet build src/Core/ZB.MOM.WW.OtOpcUa.Configuration -> 0 errors dotnet build tests/Core/ZB.MOM.WW.OtOpcUa.Core.Tests -> 0 errors dotnet build tests/Core/ZB.MOM.WW.OtOpcUa.Configuration.Tests -> 0 errors dotnet build (whole solution) -> 70 errors, all in Server/Admin	2026-05-26 04:06:25 -04:00
Joseph Doherty	4bb4ad8acb	feat(configdb): add RowVersion to live-edit entities ... Phase 1a of the v2 entity-model rewrite. Adds: public byte[] RowVersion { get; set; } = Array.Empty<byte>(); and the EF Core mapping e.Property(x => x.RowVersion).IsRowVersion(); to 12 live-edit entities: Equipment, DriverInstance, Device, Tag, PollGroup, Namespace, UnsArea, UnsLine, NodeAcl, Script, VirtualTag, ScriptedAlarm These are the entities that v2 admins will edit directly via AdminOperationsActor (no draft staging). RowVersion enables last-write-wins detection when two operators race on the same row. GenerationId FKs are still in place on these entities (removed in Task 14b); this commit only adds the rowversion column so the migration in Task 14f can emit ADD COLUMN before DROP FK as a single atomic step.	2026-05-26 03:58:58 -04:00
Joseph Doherty	8e2c4f2835	feat(configdb): add Deployment, NodeDeploymentState, ConfigEdit, DataProtectionKey entities ... Phase 1 entities for the v2 live-edit + snapshot-deploy model: Deployment — immutable artifact snapshot (replaces v1 ConfigGeneration row) Status enum {Dispatching, AwaitingApplyAcks, Sealed, PartiallyFailed, TimedOut}; carries the SHA256 RevisionHash and the SnapshotAndFlatten() ArtifactBlob; RowVersion for optimistic concurrency. NodeDeploymentState — per-(node, deployment) apply progress row owned by DriverHostActor (replaces single-row ClusterNodeGenerationState). Composite key (NodeId, DeploymentId) gives the ConfigPublishCoordinator the full history it needs to reconstruct in-flight state after a failover. ConfigEdit — append-only audit row written by AdminOperationsActor on every mutating op; optional ExecutionId correlates edits inside one admin transaction (e.g. an import batch). DataProtectionKey — ASP.NET DataProtection key ring storage via IDataProtectionKeyContext so every admin-role node decrypts the same cookies without sharing a filesystem. OtOpcUaConfigDbContext now implements IDataProtectionKeyContext and registers four new DbSets + four new ConfigureXxx mappings. Central package bumps (forced by Microsoft.AspNetCore.DataProtection.EntityFrameworkCore 10.0.7's transitive dep): Microsoft.EntityFrameworkCore.{,Design,InMemory,SqlServer} 10.0.0 -> 10.0.7 Microsoft.Extensions.{Configuration.Abstractions,Configuration.Json,Hosting,Hosting.WindowsServices,Http} 10.0.0 -> 10.0.7 EF migration generation + the ConfigGeneration drop + RedundancyRole column removal are deferred to Task 14 (high-risk, non-parallelizable).	2026-05-26 03:49:59 -04:00
Joseph Doherty	30a2104fa5	feat(scaffold): introduce 8 v2 component projects ... Adds the empty project skeletons that subsequent v2 tasks fill in: src/Core/ZB.MOM.WW.OtOpcUa.Commons (types, interfaces, message contracts) src/Core/ZB.MOM.WW.OtOpcUa.Cluster (Akka.Hosting + cluster wiring) src/Server/ZB.MOM.WW.OtOpcUa.Security (cookie+JWT auth, LDAP) src/Server/ZB.MOM.WW.OtOpcUa.ControlPlane (admin-role cluster singletons) src/Server/ZB.MOM.WW.OtOpcUa.Runtime (per-node driver actors) src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer (OPC UA SDK application host) src/Server/ZB.MOM.WW.OtOpcUa.AdminUI (Razor class library) src/Server/ZB.MOM.WW.OtOpcUa.Host (single fused web binary) Each project sets TreatWarningsAsErrors=true in its own csproj (per the Directory.Build.props deviation note in the previous commit). NuGetAuditSuppress entries cover transitive vulnerability advisories the new strictness surfaces: - GHSA-g94r-2vxg-569j (OpenTelemetry.Api 1.9.0 via Akka.Cluster.Hosting/Tools) - GHSA-h958-fxgg-g7w3 (Opc.Ua.Core 1.5.374.126 via OpcUaServer) - GHSA-37gx-xxp4-5rgx + GHSA-w3x6-4m5h-cxqf (legacy advisories already accepted) OpcUaServer pins OPCFoundation.NetStandard.Opc.Ua.Configuration to 1.5.374.126 via VersionOverride to match Opc.Ua.Server's transitive Opc.Ua.Core (same constraint as the legacy Server project). Runtime does NOT project-reference any concrete Driver.* assemblies; drivers load reflectively at runtime (Phase 6). Runtime gets the IDriver contract through Core.Abstractions instead. Host's Microsoft.Extensions.Hosting.WindowsServices is conditional on the Windows OS so the project builds on macOS dev machines. Build verification: dotnet build -> 438 warnings (all pre-existing xUnit1051 in legacy Server.Tests/Admin.Tests), 0 errors. Closes Task 9 (build green smoke check, no separate commit).	2026-05-26 03:44:56 -04:00
Joseph Doherty	2b811477d1	chore(build): introduce central package management for v2 ... Adds Directory.Packages.props (ManagePackageVersionsCentrally) and Directory.Build.props (net10.0/nullable/implicit usings/LangVersion latest). Strips Version attributes from every csproj PackageReference and consolidates versions into the central file. Side fixes (necessary to keep the build green on .NET SDK 10.0.105 on macOS): - Microsoft.CodeAnalysis.CSharp{,.Workspaces}: 5.3.0 -> 5.0.0. The 5.3.0 analyzer DLL references compiler 5.3.0.0 and the local SDK ships compiler 5.0.0.0, producing CS9057 on every project that loaded the Analyzers output. Master itself was broken on this machine pre-change. - Server + Server.Tests pin OPCFoundation.NetStandard.Opc.Ua.{Configuration, Client} to 1.5.374.126 via VersionOverride, matching Opc.Ua.Server's pin. Mixing 1.5.378.106 Opc.Ua.Core transitively with 1.5.374.126 Opc.Ua.Server breaks CustomNodeManager2 override signatures (CS0115 on LoadPredefinedNodes/Browse/HistoryRead*) and CS7069 in the tests. The pin disappears when the legacy Server project is deleted in Task 56. - Client.UI + Client.UI.Tests: NuGetAuditSuppress for GHSA-xrw6-gwf8-vvr9 (Tmds.DBus.Protocol 0.20.0 reaches both projects transitively from Avalonia.Desktop on Linux/macOS only). Deviation from the plan: TreatWarningsAsErrors=true is NOT set in Directory.Build.props because the pre-v2 Admin/Server test projects carry ~240 xUnit1051 analyzer warnings that would fail the build. New v2 projects opt in via their own csproj; the global flag can return once the legacy projects are deleted in Task 56.	2026-05-26 03:40:24 -04:00
Joseph Doherty	23d59d73f2	fix(scripting+alarms): close remaining re-review findings ... Single commit covering the four small/medium fixes from the updated code review. Core.Scripting-014 (Medium, Concurrency): CompiledScriptCache.Clear() used the key-only TryRemove(key, out var lazy) overload — same race shape Core.Scripting-006 closed in GetOrCompile's catch block. A concurrent re-add between snapshot and TryRemove was evicted + disposed while the new caller still held it. Replaced with the value-scoped TryRemove(KeyValuePair<,>) overload. Regression test Clear_uses_value_scoped_TryRemove_so_a_race_inserted_entry_survives added. Core.Scripting-013 (Medium, Security): Hand-rolled BuildWrapperSource pastes user source between literal braces; brace-balanced source could inject sibling methods/classes alongside CompiledScript.Run. Analyzer still walked the injected members so it wasn't a direct escape, but it relaxed the documented 'method body' authoring contract. Added EnforceSingleRunMember: after ParseText, the compilation unit must hold exactly one type (CompiledScript) and that type must hold exactly one member (the Run method). Any deviation throws CompilationErrorException with LMX001/ LMX002 diagnostic IDs and a Core.Scripting-013 reference in the message. Two regression tests added covering the sibling-method and sibling-class injection vectors. Core.Scripting-015 (Low, Correctness, latent): ToCSharpTypeName's generic branch truncated at the first backtick via IndexOf, silently dropping closed args of nested-generic shapes (Outer<T>.Inner<U>). No production caller exercises this shape today (all TContext/TResult are top-level non-nested), so the bug was latent. Rewrote the generic branch to walk the FullName segment-by- segment, consuming generic args per segment so nested shapes emit valid C# (global::Ns.Outer<T>.Inner<U> rather than the broken Outer<T,U>). Core.ScriptedAlarms-013 (Low, Documentation): The internal test accessors TryGetScratchReadCacheForTest / TryGetScratchContextForTest return live mutable scratch refilled in place under _evalGate. XML docs didn't warn future test authors about the synchronization contract. Added a <remarks> block to each documenting the only-safe-on-quiesced-engine + identity-or-single-key contract. Verification (suites green): Core.Scripting.Tests: 110/110 (was 107 — +3 new rejection/race tests) Core.ScriptedAlarms.Tests: 67/67 (unchanged — doc-only fix) Core.VirtualTags.Tests: 57/57 (unchanged) After this commit, all 12 findings from the updated re-review are closed (10 Resolved, 1 Won't Fix none, 1 Deferred — Driver.Galaxy-017). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-23 18:00:59 -04:00
Joseph Doherty	3a53d03d23	fix(scripting): block ThreadPool/Timer/AssemblyLoadContext in sandbox ... Core.Scripting-012 (High, Security) resolution. The Core.Scripting-008 rewrite broadened the BCL references list from a narrow allow-list to the full System.* + netstandard + Microsoft.Win32.Registry set, delegating the security gate entirely to ForbiddenTypeAnalyzer. Three categories of dangerous BCL types were reachable from script source without a deny-list entry: - System.Threading.ThreadPool — QueueUserWorkItem re-introduces the background-fanout threat Core.Scripting-003 closed against System.Threading.Tasks. - System.Threading.Timer — schedules unbounded callback work that outlives the per-evaluation timeout. - System.Runtime.Loader.AssemblyLoadContext — loads arbitrary DLLs. Defense-in-depth gap; invocation needs reflection (already denied) but the load itself was reachable. Fix: - Added 'System.Runtime.Loader' to ForbiddenNamespacePrefixes (preferred over type-granular per the recommendation so future BCL additions to that namespace are denied by default). - Added 'System.Threading.ThreadPool' and 'System.Threading.Timer' to ForbiddenFullTypeNames — both live in System.Threading shared with allowed primitives so they must be type-granular. Regression tests added to ScriptSandboxTests: Rejects_ThreadPool_QueueUserWorkItem_at_compile Rejects_Timer_new_at_compile Rejects_AssemblyLoadContext_at_compile Docs: docs/v2/implementation/phase-7-scripting-and-alarming.md decision #6 and the Sandbox-escape compliance-check row both updated to enumerate the new entries per the Core.Scripting-009 doc-sync convention. Two lower-impact suggestions from the finding's recommendation (System.Console, CultureInfo.DefaultThreadCurrentCulture) were intentionally not addressed and are recorded as accepted minor risks in the resolution. Verification: Core.Scripting.Tests 107/107 (was 104 + 3 new rejection tests). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-23 17:39:20 -04:00
Joseph Doherty	fb7c6c7046	fix(scripting): route engines through CompiledScriptCache (Core.Scripting-016) ... Both VirtualTagEngine.Load and ScriptedAlarmEngine.LoadAsync were calling ScriptEvaluator.Compile directly, bypassing CompiledScriptCache. The Core.Scripting-008 collectible-ALC fix wired Dispose only through the cache's Clear()/Dispose(), so the per-publish accretion the -008 fix was meant to eliminate was still in effect on the actual production path — the headline 'no more restarts needed' guarantee wasn't delivered. Resolution: - VirtualTagEngine + ScriptedAlarmEngine each gained a private CompiledScriptCache<TContext, TResult> instance. - Both Load methods now call _compileCache.GetOrCompile(source). - Publish-replace path: _compileCache.Clear() runs alongside the existing _tags / _alarms clears so the prior generation's ALCs are disposed before recompile. - Engine Dispose now calls _compileCache.Dispose() so shutdown actually releases the emitted assemblies. Side-fix in CompiledScriptCache: Dispose() set _disposed=true then called Clear(), but Clear() had a pre-existing 'if (_disposed) return' guard that aborted the drain unconditionally — making the Dispose-triggered cleanup a silent no-op. Removed the disposed-guard on Clear() (clearing an empty/ cleared cache is idempotent). Side-fix in ScriptedAlarmEngine.Dispose: cleared _alarms AFTER the Task.WhenAll drain. The drain guarantees no background callback is mid- flight, so clearing is safe. Previously _alarms was deliberately NOT cleared on Dispose (per Core.ScriptedAlarms-005), but that left the AlarmState records holding TimedScriptEvaluator → ScriptEvaluator → delegate references that rooted the emitted assemblies, defeating the cache's Dispose work on the engine side. Regression tests: - VirtualTagEngineTests.Dispose_unloads_compiled_script_assembly - ScriptedAlarmEngineTests.Dispose_unloads_compiled_predicate_assembly Both use WeakReference + bounded GC.Collect() to prove the emitted assembly is reclaimable after engine.Dispose(). The alarms test had to be synchronous (not 'async Task<WeakReference>') because async state machines capture locals as state-struct fields, keeping them alive past the method's apparent end and defeating GC. Verification: - Core.Scripting.Tests: 104/104 (unchanged). - VirtualTags.Tests: 57/57 (was 56 — +1 unload test). - ScriptedAlarms.Tests: 67/67 (was 66 — +1 unload test). - All other consumer suites still green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-23 17:33:34 -04:00
Joseph Doherty	0001cdd579	fix(scripted-alarms): reuse per-alarm evaluation scratch on the hot path ... Core.ScriptedAlarms-009 resolution: replace the per-call Dictionary + AlarmPredicateContext allocation with a per-alarm reusable AlarmScratch held in _scratchByAlarmId, refilled in place under _evalGate on each evaluation. The hot path no longer allocates per upstream tag change. Why this matters: On a busy line where many tags feeding many alarms change frequently, the old BuildReadCache allocated a fresh dictionary + context on every predicate evaluation — a steady stream of short-lived allocations the GC eventually has to reclaim. With the reuse, the dictionary and context are allocated once per alarm (on first evaluation) and refilled in place across every subsequent re-eval. Implementation: - New private AlarmScratch class holds the reusable Dictionary<string, DataValueSnapshot> read cache (pre-sized to the alarm's Inputs.Count) and the AlarmPredicateContext that wraps it by reference. The context observes refilled values without being re-created. - ConcurrentDictionary<string, AlarmScratch> _scratchByAlarmId on the engine, cleared in LoadAsync alongside _alarms so a config-publish drops the prior generation's scratch (Inputs / Logger may change). - EvaluatePredicateToStateAsync looks up scratch via GetOrAdd, calls the new RefillReadCache(Dictionary, IReadOnlySet) helper to clear + repopulate the dictionary in place, then runs the predicate against the reused context. - BuildReadCache removed. Safety: Reuse is serialised under _evalGate which guarantees no two threads ever observe the same scratch in a half-refilled state. The AlarmPredicateContext is bound to the scratch dictionary by reference, so the predicate's ctx.GetTag(path) sees the freshly-refilled values rather than a stale snapshot. Verification: - All 66 ScriptedAlarms tests pass (was 63 — three new regression tests locking the reuse contract). - All 56 VirtualTags tests still pass (unchanged). - All 104 Core.Scripting tests still pass (unchanged). New tests in ScriptedAlarmEngineTests: - Reevaluation_reuses_the_same_read_cache_dictionary — asserts ReferenceEquals(scratch_before, scratch_after) across two evaluations of the same alarm. - Reevaluation_reuses_the_same_predicate_context — same, for the context. - LoadAsync_drops_the_prior_generations_scratch — asserts a config publish wipes the prior scratch (so a stale Logger / Inputs can't leak into the new generation). Internal test hooks TryGetScratchReadCacheForTest / TryGetScratchContextForTest added via the existing InternalsVisibleTo for the tests project. Kept internal — not part of the public engine surface. Docs: - docs/v2/Galaxy.Performance.md "Scripted-alarm engine" section rewritten as "hot-path allocation reuse" documenting the new contract + reuse safety reasoning + the three regression tests. - code-reviews/Core.ScriptedAlarms/findings.md -009 flipped Won't Fix → Resolved. - code-reviews/README.md regenerated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-23 16:10:09 -04:00
Joseph Doherty	7b6ab2ec6f	fix(scripting): unload compiled-script assemblies via collectible ALC ... Core.Scripting-008 resolution: replace the legacy CSharpScript.CreateDelegate path with hand-rolled CSharpCompilation + Emit + collectible AssemblyLoadContext, so per-publish compile accretion no longer requires a server restart to reclaim. Why this was needed: Roslyn's CSharpScript path emits dynamically-compiled script assemblies into the default AssemblyLoadContext, which is non-collectible. Across config- publish generations each Clear() drops dictionary entries but the emitted assemblies stay loaded for process lifetime, so memory grows steadily on long-running servers with frequent publishes. The accepted-limitation note in docs/VirtualTags.md recommended scheduled restarts as the workaround; operator feedback was that restarts are difficult, so the underlying limitation was the right thing to fix. Implementation: - New ScriptAssemblyLoadContext(name, isCollectible: true) hosts one emitted script assembly per evaluator. - ScriptEvaluator.Compile synthesises a wrapper class around the user source (CompiledScript.Run(globals) — explicit return required per ordinary C# semantics, which every existing script already uses), builds a CSharpCompilation against the sandbox references, runs the ForbiddenTypeAnalyzer over the semantic model unchanged, emits to an in-memory PE stream, loads via ScriptAssemblyLoadContext.LoadFromStream, and binds a strongly-typed Func<ScriptGlobals<TContext>, TResult> delegate via reflection. - ScriptEvaluator now implements IDisposable — Dispose calls AssemblyLoadContext.Unload(), which makes the emitted assembly eligible for GC at the next collection cycle. - CompiledScriptCache.Clear() disposes every materialised evaluator before dropping its dictionary entry; CompiledScriptCache itself is now IDisposable for graceful server shutdown. - ScriptSandbox.Build returns a new SandboxConfig (References + Imports) instead of a Roslyn ScriptOptions; references now span BCL via the TRUSTED_PLATFORM_ASSEMBLIES set filtered to System.* + netstandard + Microsoft.Win32.Registry, so forbidden BCL types resolve at compile and ForbiddenTypeAnalyzer is the sole security gate (consistent with the Core.Scripting-001 / -002 model — references-list-only restriction is porous against type forwarding, so the analyzer must be the real gate). Verification: - All 104 Core.Scripting tests pass (was 101 — three new regression tests locking the unload contract). - All 56 VirtualTags tests pass (unchanged). - All 63 ScriptedAlarms tests pass (unchanged). - New CompiledScriptCacheTests: - Dispose_unloads_compiled_script_assembly_load_context — proves single- evaluator ALC unload via WeakReference + bounded GC.Collect() loop. - Clear_disposes_every_materialised_evaluator — proves publish-replace releases every prior generation's ALC. - GetOrCompile_after_Dispose_throws_ObjectDisposedException — locks the post-dispose contract. Docs: - docs/VirtualTags.md "Compile cache" section rewritten: the accepted- limitation note replaced with the unload contract + the new authoring convention (explicit return). - docs/ScriptedAlarms.md cross-reference updated to drop the obsolete restart guidance. - code-reviews/Core.Scripting/findings.md Core.Scripting-008 flipped Won't Fix → Resolved with the implementation summary. - code-reviews/README.md regenerated. Pre-existing breakage note: Driver.Galaxy fails the solution-wide build on master because its ProjectReference to the sibling mxaccessgw repo's MxGateway.Client targets a path that the sibling repo no longer has after a recent restructuring. This is unrelated to Core.Scripting-008 and was verified to exist on master before this branch was cut. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-23 15:55:04 -04:00
Joseph Doherty	3f01a24b45	fix(core-virtual-tags): resolve Low code-review findings (Core.VirtualTags-004,006,007,009,010,011,013) ... - Core.VirtualTags-004: CoerceResult now covers every scalar DriverDataType and throws on the default arm; Load rejects unsupported declared types. - Core.VirtualTags-006: Subscribe/Unsub prune empty observer-list entries from _observers under the same lock with a reconfirm-on-add race guard. - Core.VirtualTags-007: rewrote TimerTriggerScheduler so each TickGroup tracks an InFlight flag (Interlocked CAS); ticks that overlap a still- running tick for the same group are skipped + counted. - Core.VirtualTags-009: DirectDependencies / DirectDependents return a shared static empty set on miss instead of allocating per call. - Core.VirtualTags-010: corrected XML docs to reference the real engine symbols (OnUpstreamChange, CascadeAsync, etc.) instead of phantom types. - Core.VirtualTags-011: Load now rejects scripts whose declared Writes target a non-registered virtual-tag path. - Core.VirtualTags-013: DependencyCycleException renders SCC members as a set rather than a fabricated arrow-traversal edge path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-23 07:23:53 -04:00
Joseph Doherty	0a20de728d	fix(core-scripting): resolve Low code-review findings (Core.Scripting-005,006,008,009,011) ... - Core.Scripting-005: DependencyExtractor.HandleTagCall now recognises raw-string literal paths by checking the StringLiteralExpression node kind instead of the legacy StringLiteralToken kind. - Core.Scripting-006: scope CompiledScriptCache failed-compile eviction with TryRemove(KeyValuePair) so a racing retry entry is not evicted. - Core.Scripting-008: document the per-publish assembly accretion as an accepted limitation in docs/VirtualTags.md. - Core.Scripting-009: enumerate the authoritative deny-list (namespace prefixes + type-granular denies) in the Phase 7 decision-#6 entry to match ForbiddenTypeAnalyzer. - Core.Scripting-011: pin ScriptSandbox.Build, ScriptContext.Deadband boundary semantics, and end-to-end factory + companion-sink integration. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-23 07:23:42 -04:00
Joseph Doherty	99354bfaf2	fix(core-scripted-alarms): resolve Low code-review findings (Core.ScriptedAlarms-003,006,008,010,011; -009 documented) ... - Core.ScriptedAlarms-003: emit OnEvent OUTSIDE _evalGate by collecting pending emissions during the gate-held section and flushing them after release; eliminates re-entrancy deadlock the docs already promised. - Core.ScriptedAlarms-006: track every fire-and-forget Reevaluate / ShelvingCheck task in _inFlight; Dispose drains the set so the engine no longer races store writes against teardown. - Core.ScriptedAlarms-008: store comments as ImmutableList<AlarmComment> so AppendComment is O(log n) instead of O(n). - Core.ScriptedAlarms-010: document the deliberate input-quality asymmetry (Uncertain drives the predicate, renders {?} in the message) in docs/ScriptedAlarms.md and on MessageTemplate.Resolve remarks. - Core.ScriptedAlarms-011: propagate the no-op reason through TransitionResult.NoOp(state, reason) and log it from ScriptedAlarmEngine.ApplyAsync. - Core.ScriptedAlarms-009 (Won't Fix per recommendation): documented the per-evaluation dictionary allocation in docs/v2/Galaxy.Performance.md with a mitigation path if a future soak surfaces pressure. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-23 07:23:31 -04:00
Joseph Doherty	0da4f3b63a	fix(core-alarm-historian): resolve Low code-review findings (Core.AlarmHistorian-008,011) ... - Core.AlarmHistorian-008: cache queue depth in an Interlocked counter so EnqueueAsync no longer runs COUNT(*) on every alarm; consolidate DrainOnceAsync onto a single SqliteConnection per tick (purge, batch read, dead-letter, and outcome transaction all share it). - Core.AlarmHistorian-011: confirm the stale Galaxy.Host XML doc references were already fixed under earlier commits; flip to Resolved. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-23 05:38:26 -04:00
Joseph Doherty	b92fea15d4	fix(configuration): resolve Low code-review findings (Configuration-004,005,007,010,011) ... - Configuration-004: NodePermissions stored as int to match the EF HasConversion<int>() in OtOpcUaConfigDbContext.ConfigureNodeAcl. - Configuration-005: serialise LiteDbConfigCache.PutAsync so concurrent Put for the same (ClusterId, GenerationId) cannot duplicate rows. - Configuration-007: rethrow OperationCanceledException from GenerationApplier.ApplyPass when the caller's token is cancelled. - Configuration-010: scrub secrets and drop the full exception object from the ResilientConfigReader fallback warning log. - Configuration-011: pin the previously-uncovered GenerationApplier cancellation and path-length / publish-validation paths. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-23 05:38:18 -04:00
Joseph Doherty	8be6afbda4	fix(core): resolve Low code-review findings (Core-004,008,009,010,011,012) ... - Core-004: add ConfigureAwait(false) to DriverHost.RegisterAsync / UnregisterAsync / DisposeAsync. - Core-008: rewrite the BuildAddressSpaceAsync XML doc to correctly name the caller (OpcUaApplicationHost.PopulateAddressSpaces) that owns the per-driver isolation. - Core-009: snapshot DriverResilienceOptions once per non-idempotent write in CapabilityInvoker.ExecuteWriteAsync. - Core-010: switch DriverResilienceOptions.Resolve to TryGetValue with a diagnostic error message when a tier table is missing a capability. - Core-011: add an optional diagnostic callback to PermissionTrieBuilder so production callers can surface scope-path mismatches. - Core-012: correct the stale WedgeDetector ctor summary and add the Reconnecting row to DriverHealthReport's state matrix. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-23 05:38:09 -04:00