Commit Graph

1944 Commits

Author SHA1 Message Date
Joseph Doherty e5ad4c9f53 test(site-call-audit, deployment): cover error, retry and state-transition branches (ScadaBridge#73)
Backfills the branches arch-review 08 flagged as thin, chosen for what they
decide rather than for count. Every new test was mutation-checked: the
guarded line was broken and the test went red.

SiteCallAudit (SiteCallAuditFailurePathTests, 17 cases, in-memory - the
18 MSSQL-backed actor tests skip without a database, so the ingest and read
fault paths had no runnable coverage):
- a failed page write holds the reconciliation cursor; the next tick re-pulls
  the same window and advances only after the write lands
- the pinned latch publishes Pinned=true once across many pinned ticks and
  Pinned=false once when the site starts honouring after_id
- a site-enumeration fault skips one tick without wedging the single-flight
  guard
- an endless backlog drains at most 50 pages per tick, unpinned, and the next
  tick resumes from the advanced cursor
- purge single-flight: ticks during a blocked purge are dropped
- ingest fault replies Accepted=false and the next upsert is accepted
- detail / KPI / per-site / per-node / query faults map to Success=false with
  the root-cause message and an empty payload; the actor keeps serving
- query page size clamped to [1, 200]; an empty page carries no cursor
- Discard NotParked / OperationFailed (site message verbatim) with no audit
  row; Discard before transport registration is SiteUnreachable
- a throwing audit sink leaves an Applied relay Applied; a missing mirror row
  or unknown channel still audits the operator, falling back to ApiOutbound

DeploymentManager (DeploymentServiceFailurePathTests, 11 cases):
- a site that answers Failed commits a Failed record with the site's message
  (no timeout marker), leaves instance state and the deployed snapshot
  untouched, and audits Deploy (not DeployFailed)
- a prior non-timeout Failed record skips the query-before-redeploy
- a failing Failed-status write detaches pending changes and retries once
  (none when nothing was detached); the caller still gets the original error
- Disable / Enable / Delete answered with Success=false leave the instance
  unchanged, audit the attempt, and surface the site's message or the default

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 13:10:31 -05:00
Joseph Doherty c29b99c63b perf(templates): descendant reconcile loads only the subtree + chain (ScadaBridge#86)
TemplateService.ReconcileDescendantsAsync - run after every base member
add/update/delete, template and native-alarm-source alike - loaded the whole
tracked template graph (every template, member and script body) on each call,
which after #66 was the dominant cost of those paths. It now calls the new
ITemplateEngineRepository.GetTemplateReconcileClosureAsync: the same tracked,
fully-included shape as GetAllTemplatesAsync, restricted to the edited
template's derived subtree plus its own inheritance chain. The reconcile is
inheritance-only (TemplateInheritanceResolver never follows compositions) and
every descendant's chain runs through the edited template, so that set is
everything it reads.

The walk (Commons TemplateReconcileClosure, the descendant-direction twin of
#66's TemplateAnalysisClosure) loads one children batch per subtree depth and
one batch per ancestor level, terminates on parent cycles and stops at
dangling parents. A template with no children - the common case - costs one
empty probe and loads nothing.

Evidence:
- TemplateReconcileClosureTests: for every template of a graph with a deep
  subtree, an ancestor above the edited template, a composed module,
  look-alike unrelated trees, a parent cycle and a dangling parent, the
  closure-backed reconcile issues exactly the writes (add/update/delete,
  audit, save) of the old whole-graph reconcile, and the resolver answers
  identically for every descendant. Fails with the old service (whole-graph
  load wired to throw) and with the ancestor walk removed (mutation-checked).
- TemplateReconcileClosureRepositoryTests (SQLite, recorded SQL): 5 templates
  / 10 attributes instead of 47 / 94, tracked with script bodies; a leaf costs
  one probe query and tracks nothing.
- TemplateServiceTests: attribute add + update propagate two levels down with
  GetAllTemplatesAsync wired to throw; existing propagation tests now run
  against the closure via ServeReconcileClosureFromFullGraph.

ResyncInheritedMembersAsync (operator action) still reads the whole graph.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 13:10:31 -05:00
Joseph Doherty 0eac956d25 feat(health): report permanently rejected site audit rows on the site health report (ScadaBridge#69)
SiteAuditBacklogSnapshot gains an optional trailing RejectedCount (gauge of
Rejected forward-state rows, read from the site store), carried on the wire as
SiteAuditBacklogSnapshotDto.rejected_count = 4 (additive; old sites read as 0,
old central ignores it). Health page shows an 'Audit rows rejected' badge.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 13:05:48 -05:00
Joseph Doherty 3ae3f1bcef refactor(health): move ICentralHealthAggregator and SiteHealthState to Commons (ScadaBridge#71)
Interface goes to Commons.Interfaces.Services and the state record to
Commons.Messages.Health, so CentralCommunicationActor no longer needs the
HealthMonitoring namespace. Implementation stays in HealthMonitoring.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 13:05:48 -05:00
Joseph Doherty 4918210b78 build: adopt CentralPackageTransitivePinningEnabled, drop per-project SQLitePCLRaw pins (ScadaBridge#79)
Enables transitive pinning so the central SQLitePCLRaw.lib.e_sqlite3 2.1.12
pin applies everywhere (41 projects verified at 2.1.12). Required raising the
central floors: Newtonsoft.Json 13.0.4 and Microsoft.Extensions.* 10.0.13.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 13:05:48 -05:00
Joseph Doherty 792f16c2d5 perf(templates): member add/update loads only the chain + composition closure (ScadaBridge#66)
Attribute/alarm/script add (collision check) and update (parent-lock lookup)
used GetAllTemplatesForAnalysisAsync, materialising every template, attribute,
alarm and script in the library per edit. They now call the new
ITemplateEngineRepository.GetTemplateAnalysisClosureAsync: the same untracked,
body-free analysis projection, restricted to the template's inheritance chain
plus the composition closure of every chain member - exactly what
TemplateResolver and CollisionDetector walk (a composed module's own parent
chain is never climbed, so it is not loaded). Adds use the edited template's
closure, updates the parent's.

The walk (Commons TemplateAnalysisClosure) loads one batch per closure depth,
requests each id once, terminates on cycles and skips dangling edges.

Evidence:
- TemplateAnalysisClosureTests: for every template of a graph with chains,
  nested + diamond compositions, a derived module, cycles, dangling edges and
  colliding unrelated templates, ResolveAllMembers / DetectCollisions /
  FindMemberByCanonicalName answer identically from the closure and the whole
  graph (mutation-checked).
- TemplateAnalysisClosureRepositoryTests (SQLite, recorded SQL): 6 templates /
  12 attributes loaded instead of 47 / 94, in 4 queries (one per depth),
  untracked, script bodies never selected.
- TemplateServiceTests: two new tests with the whole-graph load wired to throw
  (fail on the old service); every existing add/update test now runs against
  the closure via AnalysisClosureMock.

ReconcileDescendantsAsync still loads the tracked full graph; not in scope.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 12:52:29 -05:00
Joseph Doherty d36825633b feat(streaming): carry AckTime on DebugAlarmStateDto (21) (ScadaBridge#84)
The live AlarmStateUpdate already carried ack_time (24) but the Debug View
snapshot DTO did not, so snapshot-seeded alarm rows showed a null ack time
until the next live update. Additive google.protobuf.Timestamp ack_time = 21
(absent => null, including from a pre-#84 site), vendored C# regenerated with
docker/regen-proto.sh sitecommand, mapped both ways in SiteCommandDtoMapper.
The native golden sample now sets AckTime (the computed one leaves it null),
so the round-trip golden pins both the present and the absent decode.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 12:45:21 -05:00
Joseph Doherty 03de3dcc14 fix(transport): drop the external-definition lookup cache after a committed bundle import (ScadaBridge#83)
The bundle importer writes external systems, methods and database connections
through the raw DbContext, bypassing the ManagementActor handlers that
invalidate ExternalDefinitionLookupCache (#65), so the importing node served
pre-import definitions for up to DefinitionCacheTtl. New Commons seam
IExternalDefinitionCacheInvalidator, implemented by the cache and registered as
the same singleton; BundleImporter takes it optionally and invalidates once
after every committed import (never on rollback; never throws).

Peer-central-node invalidation (the other half of #83) is NOT done here: the
issue leaves open whether the 10 s TTL bound is acceptable for the peer.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 12:40:43 -05:00
Joseph Doherty 6d5bb6189f feat(flow-debugger): carry site-side trace truncation on the route reply (ScadaBridge#54)
ScriptCallResult and RouteToCallResponse gain trailing Truncated/LogsTruncated,
filled from the site FlowTraceCollector after Complete(); RouteToCallResponseDto
fields 8/9 carry them (sitecommand regen). FlowDebugRunner ORs them into
FlowDebugRunResult on the script entry, and on the method entry through a new
optional truncation sink on RouteHelper.WithTraceSink (carried through every
With* clone). Previously a site run whose collector dropped rows came back as a
silently short list. Older sites send false.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 12:37:57 -05:00
Joseph Doherty 5310fd0e26 feat(streaming): carry IsMesReportable on AlarmStateUpdate (25) and DebugAlarmStateDto (20) (ScadaBridge#60)
Central's Debug View live stream and Alarm Summary cache always saw false
because neither wire message had the field. Additive proto fields, vendored
C# regenerated with docker/regen-proto.sh (sitestream + sitecommand), mapped
both ways in StreamRelayActor/SiteStreamGrpcClient and SiteCommandDtoMapper.
The live stream publishes Alarm Actor events, which never carried the flag,
so InstanceActor.HandleAlarmStateChanged now stamps it before retaining and
publishing. A pre-#60 site sends the proto3 default. No UI badge yet.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 12:31:37 -05:00
Joseph Doherty 6bd1fd3072 test(scripts): per-row exclusion seam in CompileSurfaceParityTests; guard ScriptLogger/CompileLog (ScadaBridge#57)
AccessorMirrorPairs rows now carry an ignoredRuntimeMethods list. An entry
must exist on the runtime accessor and must be ABSENT from the mirror, so a
stale or over-broad exclusion fails the theory. The (ScriptLogger, CompileLog)
pair joins the guarded set with FlushDropped excluded (run-end flush called
by ScriptRunLauncher, not script-reachable).

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 12:28:22 -05:00
Joseph Doherty 79957c4b96 feat(telemetry): export the site script compile-cache eviction counter as a Meter (ScadaBridge#68)
SiteScriptCompileCache now records each eviction sweep on scadabridge.site.script_compile_cache.evictions
(ScadaBridgeTelemetry). It rides the existing ZB.MOM.WW.ScadaBridge meter, which is already in
SiteServiceRegistration.ObservedMeters, so no new allowlist entry is needed; a Host test pins that
the meter stays allowlisted.
2026-11-09 12:08:25 -05:00
Joseph Doherty e2b15bd520 fix(communication): evict idle SiteStreamGrpcClientFactory (site, endpoint) clients (ScadaBridge#67)
Each cached client records when it was last handed out; a periodic sweep (half of the new
CommunicationOptions.GrpcClientIdleTtl, default 30 min, 0 disables) disposes and removes entries
unused for the TTL that have no live subscription (SiteStreamGrpcClient.ActiveSubscriptionCount).
Lookup and the sweep's check-and-remove share one lock so a handed-out client is never disposed
under its caller; an evicted client is re-created on next GetOrCreate.
2026-11-09 12:07:28 -05:00
Joseph Doherty e1734675e9 fix(management): invalidate ExternalDefinitionLookupCache on external-system / DB-connection edits (ScadaBridge#65)
ManagementActor now drops the central definition cache post-commit (via the existing
AfterConfigurationCommit seam) after every external system, external-system method and
database connection create/update/delete, so an edit takes effect on the editing node
immediately instead of after DefinitionCacheTtl. ManagementService gains a project reference
to ExternalSystemGateway. The other central node still converges within the TTL.
2026-11-09 12:05:20 -05:00
Joseph Doherty e0ec2a0507 fix(host): log a routine service stop as Information, not [FTL] (ScadaBridge#76)
An OperationCanceledException escaping RunAsync after ApplicationStopping has fired
(WindowsServiceLifetime.StopAsync gets a cancelled token after shutdown finished) is a
normal stop. Program.IsGracefulShutdown now routes it to an Information line; any other
exception, or a cancellation with no shutdown requested, still logs Fatal and rethrows.
2026-11-09 12:05:20 -05:00
Joseph Doherty a5c3de63e9 fix(dcl): coerce scalar OPC UA writes to the node DataType (#58)
Codec-encoded scalars ("True", "42.5") reached RealOpcUaClient.WriteValuesAsync
as strings and were sent as String Variants, so a Boolean/Double node answered
BadTypeMismatch (0x80740000).

RealOpcUaClient now reads each written node's DataType attribute once per
session (batched, cached, cleared on connect/disconnect; derived types and
enumerations resolved via the supertype chain) and converts non-null scalar
values with the new OpcUaWriteCoercion table: Boolean, SByte..UInt64, Float,
Double, String, DateTime, Guid, all culture-invariant. Overflow, fractional
values to integer types and unparseable strings are refused client-side with
BadTypeMismatch and a message naming the value and target type - never a
rounded value. Arrays, null, unmapped types and nodes whose DataType read
fails pass through unchanged. WriteValueAsync now surfaces the outcome's own
status code instead of always BadNodeIdUnknown. MxGateway path untouched.

Tests: OpcUaWriteCoercionTests (coercion table incl. overflow, invalid and
culture-sensitive strings) and OpcUaWriteCoercionLiveTests (RequiresOpcUa:
String writes to Double/Boolean/UInt32 opc-plc nodes read back typed; reverting
the coercion reproduces 0x80740000). Component-DataConnectionLayer.md updated.
2026-11-06 16:08:33 -05:00
Joseph Doherty 0e5402915b fix(centralui): pending-updates rail refreshes as soon as a deployment finishes — cache drops its snapshot on Success/Failed and signals the indicator, instead of serving the pre-deploy count for up to TTL + one poll 2026-10-22 17:11:18 -04:00
Joseph Doherty 6ed6800b84 fix(host): clear pooled SQLite handles before renaming the imported inbound-key file — Windows refused the rename, so the import re-ran every boot (QA vq03/vq04 2026-10-22) 2026-10-22 15:56:00 -04:00
Joseph Doherty e029dc1ccb fix(centralui): Deploy all runs in its own DI scope, live tick/Refresh/loaders pause during actions, dialog wording, failure/saved-state tests (Gitea #40) 2026-10-22 14:27:27 -04:00
Joseph Doherty bcd087efe8 feat(centralui): Topology 'Deploy all…' on site and area nodes with confirm dialog, Force and summary toast (Gitea #40) 2026-10-22 14:27:26 -04:00
Joseph Doherty d71f61e71b feat(centralui): Topology pre-loads and expands the first layer (sites -> top-level areas) (Gitea #39) 2026-10-22 14:27:26 -04:00
Joseph Doherty 1d38fb6c90 fix(cli): deploy area names a malformed summary as an invalid area deployment summary (Gitea #40 review) 2026-10-22 14:27:26 -04:00
Joseph Doherty 0956395d81 test(managementservice): MgmtDeployArea in-scope success path relays Force to a real DeploymentService and returns its summary (Gitea #40 review) 2026-10-22 14:27:26 -04:00
Joseph Doherty 74d5fba2b0 feat(cli): deploy area --id <areaId> [--force] bulk-deploys an area subtree with deploy site's deadline and exit-code contract (Gitea #40) 2026-10-22 14:27:26 -04:00
Joseph Doherty f72c72c52f feat(managementservice): MgmtDeployArea — Deployer-only, scoped by the area's owning site (fail-closed on an unknown area), long-running like MgmtDeploySite (Gitea #40) 2026-10-22 14:27:26 -04:00
Joseph Doherty eb1c9ea7f9 feat(deploymentmanager): DeployAreaAsync bulk-deploys an area subtree through the shared DeploySiteAsync core; PreviewSubtreeDeployAsync returns confirm counts (Gitea #40)
DeploySiteAsync's body moves into a private core taking the site plus an
optional area-id filter; site deploy behaviour is unchanged. The subtree is
one in-memory ParentAreaId walk (cycle-safe) shared by the deploy and the
preview. DeploymentService gains an ITemplateEngineRepository dependency for
the area lookups.
2026-10-22 14:27:26 -04:00
Joseph Doherty a04eb3a752 test(playwright): token-endpoint tests tolerate DisableLogin
Under DisableLogin /auth/token deliberately answers 404 (lane K). The tests
now probe it: on a login-disabled rig TokenEndpoint_ReturnsJwt skips and the
bad-credentials test asserts 404 (the exchange is closed); with login enabled
they keep the 200 + JWT and 401 assertions.
2026-10-22 14:27:26 -04:00
Joseph Doherty 4ae3ed3fdd test(playwright): DeploymentActionTests load the lazy Topology tree via a shared helper
The Topology page has loaded lazily (bounded instance pages behind 'Load
instances' loaders) since 9ee402bb (2026-09-09); TopologyAreaTests was
updated then but DeploymentActionTests still relied on one Expand-all click
revealing instance rows, so all four facts timed out locating the row. Move
OpenStableTopologyAsync into TopologyPageHelper and use it from both.
2026-10-22 14:27:26 -04:00
Joseph Doherty 560adefb0c fix(centralui): Topology inline area rename commits the typed name on Enter (Gitea #38)
The rename input bound _renameBuffer with the default onchange trigger. A
browser fires change only after the Enter keydown (or on blur, which cancels
by design), so CommitRename always saw the original name and silently
no-oped: a UI rename never persisted. Present since the page was written
(pre-dates 513fa1aa); the Playwright test masked it by dispatching a
synthetic change event before Enter. Bind on oninput instead.

Tests: bUnit InlineRename_TypeThenEnter_PersistsNewName (fails on the old
binding); the Playwright rename test now types + Enter like an operator.
Also fix the Playwright OpenStableTopologyAsync race: it counted 'Load
instances' loaders immediately after clicking Expand all, before the
circuit re-render landed, and returned an unexpanded tree; it now waits
for no collapsed branch first.
2026-10-22 14:27:26 -04:00
Joseph Doherty e7896494f1 test(site-runtime): SR5 cancel-queued-redeploy test observes which predecessor is queued instead of assuming Queued4
Deploys are compile-warmed off the actor thread and reach the four-slot
predecessor-stop queue in warm-completion order (cross-instance ordering is
not guaranteed by design). Since SR-P5 moved the config deserialization into
that warm task, the fifth-sent deploy often finishes warming before one of the
first four, takes a slot, and Queued4 terminated inside the 100 ms no-message
window (16/20 failures under parallel load). The test now waits for the four
in-flight predecessors to terminate, cancels the one still alive, and also
asserts the cancelled redeploy is answered Failed/superseded.
2026-10-22 10:35:37 -04:00
Joseph Doherty 7be22d2ac2 fix(auditlog): truncate over-width SiteCalls.LastError on the cached-telemetry dual-write and the reconciliation pull (review 2026-10-21 follow-up) 2026-10-22 09:58:46 -04:00
Joseph Doherty 89d8568fdf fix(site-runtime): a faulted seed trigger-expression eval adopts nothing instead of replaying as a real false (review 2026-10-21 follow-up L4) 2026-10-22 09:58:46 -04:00
Joseph Doherty d646f816a4 fix(site-runtime): refuse an enable while a deploy pipeline owns the instance; revert is_enabled only for an invalid name (review 2026-10-21 follow-up L3) 2026-10-22 09:58:46 -04:00
Joseph Doherty e60c316f30 fix(dcl): queue a create whose actor name belongs to a still-terminating child instead of refusing it (review 2026-10-21 follow-up L2) 2026-10-22 09:58:46 -04:00
Joseph Doherty 530a096111 fix(auditlog): advance the served-rowid mark under the write lock so a purge clamp is never overwritten stale-high (review 2026-10-21 follow-up L1) 2026-10-22 09:58:46 -04:00
Joseph Doherty 1f87942ce3 fix(auditlog): reconciliation abandons only row-intrinsic failures and only when a sibling row inserted (review 2026-10-21 follow-up M1) 2026-10-22 09:58:46 -04:00
Joseph Doherty 46ab5594ec fix(site-call-audit): an unknown-status row no longer aborts the rest of a batched page (review 2026-10-21 06/PERF-2 follow-up) 2026-10-22 09:50:51 -04:00
Joseph Doherty 5f7f889a96 fix(notification-outbox): keyset Pending back-fill + distinct ids so a mid-claim Retry cannot double-deliver (review 2026-10-21 06/PERF-5 follow-up) 2026-10-22 09:50:51 -04:00
Joseph Doherty 1627513fdb feat(configdb): ReviewRemediationIndexes migration — AuditLog (Kind, Actor) + AuditLogEntries composite indexes
AuditLog indexes are raw SQL ON ps_AuditLog_Month so they stay partition-aligned for the switch purge.
2026-10-22 09:42:02 -04:00
Joseph Doherty 88273693f9 fix(site-call-audit): reconciliation pull serves only un-acked tracking rows (node-local ack ledger), batched page upsert, prune removed sites' cursors (review 2026-10-21 06/PERF-2 + lane H follow-up) 2026-10-22 09:38:48 -04:00
Joseph Doherty e69be1654f fix(kpi-history): fold raw samples one hour at a time; only a unique-key violation is the benign fold race (review 2026-10-21 06/PERF-6, 08/P4, 08/C3) 2026-10-22 09:38:48 -04:00
Joseph Doherty e7fb71c2de fix(site-call-audit): KPI snapshots as single conditional-aggregate queries (review 2026-10-21 06/PERF-3, 08/P3) 2026-10-22 09:38:48 -04:00
Joseph Doherty 811a561885 fix(notification-outbox): per-attempt timestamps, Pending/Retrying split claim + sweep budget, SMTP MaxConcurrentConnections clamp (review 2026-10-21 06/COR-1, 06/PERF-5, 06/IMPL-2) 2026-10-22 09:38:48 -04:00
Joseph Doherty ebf4b96789 fix(notification-service): evict stale OAuth2 token cache entries after credential rotation (review 2026-10-21 06/MEM-3) 2026-10-22 09:38:48 -04:00
Joseph Doherty 7e08a1dbfb fix(transport): import reads the template library once per session, batches instance lookups, and expands stale instances to derived/composing templates over one flatten session (review 2026-10-21 04/P4, 08/P1, 04/C2) 2026-10-22 09:38:17 -04:00
Joseph Doherty 39025b28fa fix(transport): import rejects malformed site gRPC endpoints and non-actor-safe names as blockers, not mid-apply crashes (review 2026-10-21 03/R-1, lane-F ActorNameRules follow-up) 2026-10-22 09:38:17 -04:00
Joseph Doherty 04ac4c0491 fix(transport): export reads instance children once, site connections once per site, composed templates in batches (review 2026-10-21 04/P5) 2026-10-22 09:38:16 -04:00
Joseph Doherty 56aa386784 fix(transport): serialise the export content once and pack those bytes (review 2026-10-21 04/P8a) 2026-10-22 09:38:16 -04:00
Joseph Doherty 93fbd2e030 fix(transport): pin a compressed copy of the decrypted bundle on the import session, release the upload buffer early, document the footprint (review 2026-10-21 04/M2) 2026-10-22 09:38:16 -04:00
Joseph Doherty 55cebf2397 fix(transport): zero a bundle session's content when it expires, not only on apply/abandon (review 2026-10-21 04/C3) 2026-10-22 09:38:16 -04:00