Commit Graph

280 Commits

Author SHA1 Message Date
Joseph Doherty 811a561885 fix(notification-outbox): per-attempt timestamps, Pending/Retrying split claim + sweep budget, SMTP MaxConcurrentConnections clamp (review 2026-10-21 06/COR-1, 06/PERF-5, 06/IMPL-2) 2026-10-22 09:38:48 -04:00
Joseph Doherty 7e08a1dbfb fix(transport): import reads the template library once per session, batches instance lookups, and expands stale instances to derived/composing templates over one flatten session (review 2026-10-21 04/P4, 08/P1, 04/C2) 2026-10-22 09:38:17 -04:00
Joseph Doherty 93fbd2e030 fix(transport): pin a compressed copy of the decrypted bundle on the import session, release the upload buffer early, document the footprint (review 2026-10-21 04/M2) 2026-10-22 09:38:16 -04:00
Joseph Doherty d495494074 fix(management): purge terminal PendingSecuredWrites rows on the list sweep after a retention window (review 2026-10-21 08/G3) 2026-10-22 09:32:14 -04:00
Joseph Doherty b0a08bb63b fix(auditlog): explicit permanent rejections on the ingest ack (IngestAck.rejected_event_ids, field 2) with a dedicated counter; reconciler ignores transient faults for abandonment; persist the served high-water mark; prune only on a non-empty enumeration (review 2026-10-21 06/PERF-1, 06/IMPL-1 adversarial follow-up) 2026-10-22 09:00:48 -04:00
Joseph Doherty 004f3bd155 fix(configdb): delete dead repository members; completes 08/P8 for owned files (review 2026-10-21 08/P8; CentralUi members and tests went in the preceding P5 commit) 2026-10-22 08:33:09 -04:00
Joseph Doherty b811a2451b fix(dcl): remove unused poll-based WriteBatchAndWaitAsync from IDataConnection and both adapters (review 2026-10-21 02/U3) 2026-10-22 08:33:00 -04:00
Joseph Doherty 77b3a2c23f fix(commons): ActorNameRules accepts ( ) and " like Akka's IsValidPathElement (review 2026-10-21 01/SR-C1 parity) 2026-10-22 08:24:48 -04:00
Joseph Doherty 7aacc49043 fix(deployment): detach each instance's settled rows from the bulk deploy's shared unit of work after Finalize (review 2026-10-21 04/M3) 2026-10-22 08:22:42 -04:00
Joseph Doherty 32e63eaa96 fix(deployment): bulk-deploy up-to-date pre-filter uses batch hash/status reads and the operation's shared flatten session; completion log reports pre-filter cost (review 2026-10-21 04/P1) 2026-10-22 08:22:42 -04:00
Joseph Doherty df67fabe68 fix(deployment): pending-updates tick uses a slim instance projection, hash-only flatten, one untracked mismatch snapshot read and a per-tick secret lookup memo (review 2026-10-21 04/P2) 2026-10-22 08:22:42 -04:00
Joseph Doherty 669822772e fix(deployment): read every external system's methods in one query for the artifact snapshot (review 2026-10-21 04/P7) 2026-10-22 08:22:41 -04:00
Joseph Doherty 7c83a4aa94 fix(reconcile): batch startup reconcile into fixed round trips, bound central work by QueryTimeout, retry failed site passes with back-off (review 2026-10-21 03/P-1) 2026-10-22 08:22:41 -04:00
Joseph Doherty d6a9a79c27 fix(deployment): purge SystemArtifactDeploymentRecords in the terminal-record retention sweep (review 2026-10-21 08/G2) 2026-10-22 08:22:41 -04:00
Joseph Doherty bc1aca0ee6 feat(instances): GetInstancesPagedAsync SQL-side filter/order/page 2026-10-22 06:06:27 -04:00
Joseph Doherty 6bcca4b344 feat(templates): GetTemplateStubsAsync untracked identity/edge projection 2026-10-22 06:06:27 -04:00
Joseph Doherty 15c9de0f4a feat(commons): GrpcEndpointRules for site gRPC address validation 2026-10-22 06:05:06 -04:00
Joseph Doherty 3a53e7059b feat(commons): ActorNameRules mirroring Akka path-element validation 2026-10-22 06:05:06 -04:00
Joseph Doherty 0ffd4bdbea docs(src): fix CommentChecker findings across src/ — 1,032 → 5 (all false positives)
Comment/XML-doc changes only, no code logic:
- strip project-tracking references (WP*, P*-*, row-NN, #NN, Task N) from
  comments, keeping the explanatory rationale
- add missing summaries, <param>, <returns>, <typeparam> and <inheritdoc/>
- replace misused <inheritdoc/> with real docs; where <inheritdoc/> was
  combined with useful remarks, move the rationale into // comments
- drop stale replicationActor <param> on DeploymentManagerActor ctor

Remaining 5 are false positives (CWE-1236 x2, failures-1, year-0001, M365).
Build 0 errors / 0 warnings.
2026-10-07 14:12:25 -04:00
Joseph Doherty 55e1741075 fix(site-secrets): prune secret_mirror_status rows of decommissioned or renamed nodes
Each node now re-writes its mirror-status row every SecretMirrorStatusRefreshInterval
(10 min) as a heartbeat; SecretMirrorStatusReporter leaves off the health report and
deletes any row not refreshed for SecretMirrorStatusStaleAfter (1 h, validated >= 2x the
refresh). The delete is conditional on the checked_at it read, so a node that came back
keeps its fresh row; a returning node re-writes its row on its first pass.
2026-09-29 10:56:53 -04:00
Joseph Doherty 4e6409184c Merge branch 'fix/standby-secret-mirror-status' (M1: standby mirror failures on the health report)
# Conflicts:
#	CLAUDE.md
#	docs/plans/2026-09-28-site-secrets-artifact-deploy-design.md
2026-09-28 21:37:03 -04:00
Joseph Doherty 0289d51137 fix(site-secrets): report every site node's secret mirror failures on the health report (M1)
A DeployedSecretsMirror failure on the STANDBY node (e.g. a master-key mismatch
that stops a deployed secret decrypting there) was only logged on that node:
site health reports are sent by the active node only, so central never saw it.

- New 12th LocalDb-replicated table secret_mirror_status (node_name PK,
  failures_json of {SecretName, Message}, checked_at; no FKs). Each node's
  mirror writes only its own row, and only when its failure set changes plus
  once per start-up (every write replicates via CDC). Best-effort: a failed
  write is a Warning and is retried next pass. Node key = INodeIdentityProvider
  NodeName, falling back to the machine name.
- SecretMirrorStatusReporter (SiteRuntime, every node, 30 s) reads every row
  into ISiteHealthCollector.SetSecretMirrorStatuses -> additive nullable
  SiteHealthReport.SecretMirrorStatuses (null = no data).
- Wire: SiteHealthReportDto.secret_mirror_statuses = 33 (presence wrapper so
  null != empty); vendored CentralControl.cs regenerated; mapper both ways.
- Central UI Health card: 'Secret mirror failing' badge + compact per-node
  node / secret / message list.
- Tests: mirror status write-on-change/first-pass/failure isolation, storage
  round-trip, reporter/collector, DTO mapper round-trip incl. null/empty,
  bUnit, replicated-table set (twelve) + FK guard wording.
2026-09-28 21:33:10 -04:00
Joseph Doherty 29049c4ef6 fix(site-secrets): contain secret-store outages per site; audit removal-only deploys
Secret-store outage (final review M5): SiteSecretsCollector returns a store
read failure as SiteSecretsCollection.StoreError instead of throwing. A deploy
now fails only the sites that reference a secret (nothing sent, not stamped,
DeploySiteSecrets row with the error), and ComputeCurrentArtifactHashesAsync
omits only those sites (they read Out of date) instead of the exception
aborting the whole sweep and the fleet-wide pending-updates check.

Removal-only deploys: the site reply gains SecretsRemoved (proto field 7,
names only) - the deployed_secrets rows the deploy soft-removed. Central
carries it on SiteArtifactResult and writes the DeploySiteSecrets row with a
removed list whenever it is non-empty, so dropping a site's last secret:
reference is audited centrally rather than only in site logs.
2026-09-28 21:23:52 -04:00
Joseph Doherty 67a3950f74 fix(site-secrets): one shared secret: reference parser for scanner and MxGateway adapter
Final review M3: SecretReferenceScanner.TryParseSecretReference (trimmed, case-insensitive
prefix, trimmed name, empty = not a reference, ${secret:} never matches) is used by the scanner
and by MxGatewayDataConnection's ApiKey resolution, so a padded or case-variant reference
resolves the name central ships. A bare 'secret:' ApiKey now fails closed. M7: unit test pins
stamped hash == ComputeCurrentArtifactHashesAsync with a non-empty Secrets list.
2026-09-28 21:08:34 -04:00
Joseph Doherty ebc55c5f72 feat(artifacts): secrets and missing-secret markers in the artifact revision hash 2026-09-28 17:32:42 -04:00
Joseph Doherty 01cb1f5652 feat(secrets): SecretReferenceScanner derives a site's secret refs from its data-connection configs 2026-09-28 17:27:28 -04:00
Joseph Doherty bf7d2db0b1 feat(artifacts): additive Secrets list + SecretsApplied on the artifact deploy contract (proto field 9) 2026-09-28 17:22:57 -04:00
Joseph Doherty b3021e7c5f feat(scripts): read-only, allow-listed Files global for site scripts; Delmia test receiver parses recipe files
Script file access (design docs/plans/2026-09-28-script-file-access-design.md): Files.ReadTextAsync /
ReadLinesAsync / ExistsAsync on site ScriptGlobals (+ Instance.Files), backed by the site-singleton
ScriptFileReader. The one host-mediated exception to the System.IO ban - scripts still never name a
System.IO type. OFF by default: ScadaBridge:SiteRuntime:ScriptFileAccess:AllowedRoots (empty = every call
throws Disabled) + MaxFileBytes (1 MiB, validated 1..64 MiB, roots must be absolute). Guard: fully
qualified, no device paths / ADS, canonical path strictly inside a root (directory-boundary match), no
reparse point on the file or any directory below the root, size cap before and during the read.
Errors are one Commons type, ScriptFileAccessException(+ScriptFileAccessFailure); Commons.Types.Scripts
joins the default script imports on every surface so it binds by short name. Mirrored on
ScriptCompileSurface (CompileFiles, parity-pinned) and the Test Run sandbox (binds, refuses Disabled).
Inbound (central) scripts do not get it.

Rig: docker/recipes (sample RecipeValueName,RecipeValue files incl. an invalid one) bind-mounted
read-only at /app/recipes on both site-a nodes with AllowedRoots [/app/recipes]. The TestMesMachine
SimDelmiaRecipe simulator now reads + parses the downloaded file, converts each value to its recipe tag's
type and loads it all-or-nothing (14 new TestDelmiaReceiver attributes); verify-mes-test-machine.sh
group A covers accept / re-download / invalid lines / NotFound / OutsideAllowedRoots, VERIFY_GROUPS
selects groups (rig: AD).
2026-09-28 10:36:42 -04:00
Joseph Doherty 3b55d60ef9 fix(deploy): staleness batch logs one Warning per batch, not one per instance
Claude-Session: https://claude.ai/code/session_01Y5ef34iUUMjmwnMpL9xgmB
2026-09-14 13:48:06 -04:00
Joseph Doherty dd202e5dde feat(sites): record the artifact revision last deployed to a site
Site.ArtifactRevisionHash / ArtifactsDeployedAt (nullable, migration
AddSiteArtifactRevision) so the Sites page can tell whether a site's
system-wide artifacts are current. Not transported. Adds the
GetDeployedRevisionHashesAsync projection for the fleet-wide staleness batch.

Claude-Session: https://claude.ai/code/session_01Y5ef34iUUMjmwnMpL9xgmB
2026-09-14 13:33:59 -04:00
Joseph Doherty 17d9a4cd2c feat(scripts): ScriptAlarm.IsMesReportable + Alarms.MesReportableAsync() mirrored on both compile surfaces
Claude-Session: https://claude.ai/code/session_01Y5ef34iUUMjmwnMpL9xgmB
2026-09-11 10:18:05 -04:00
Joseph Doherty cbb5e83633 feat(site-runtime): stamp IsMesReportable on the Instance Actor alarm snapshot
Claude-Session: https://claude.ai/code/session_01Y5ef34iUUMjmwnMpL9xgmB
2026-09-11 10:08:39 -04:00
Joseph Doherty 63ec226582 feat(management): IsMesReportable on alarm commands, IsMesReportableOverride on SetInstanceAlarmOverride
Claude-Session: https://claude.ai/code/session_01Y5ef34iUUMjmwnMpL9xgmB
2026-09-11 10:07:42 -04:00
Joseph Doherty b07394447f feat(commons): IsMesReportable on TemplateAlarm, ResolvedAlarm; IsMesReportableOverride on InstanceAlarmOverride
Claude-Session: https://claude.ai/code/session_01Y5ef34iUUMjmwnMpL9xgmB
2026-09-11 09:52:34 -04:00
Joseph Doherty e5c536c6b7 feat(commons): HandshakePhase + HandshakeResult for the reset-aware handshake (#36)
Claude-Session: https://claude.ai/code/session_01XPsuy2ZZ5Gbyfpt8PrK5BV
2026-09-10 15:18:10 -04:00
Joseph Doherty 2750bd2809 Complete management API and CLI command parity 2026-09-10 00:51:45 -04:00
Joseph Doherty 9ee402bbcd Resolve all 46 low-severity architecture review findings 2026-09-09 22:56:36 -04:00
Joseph Doherty 9a87d7a328 Resolve all 33 medium-severity architecture review findings
Second slice of the 2026-09-09 whole-system review remediation
(docs/plans/2026-09-09-arch-review.md §10), on top of the High slice 146a1323.

Runtime/devices: SR-2 alarm on-trigger integration scope; SR-3 correlated NativeAlarmActor
subscribe retry; SR-4 immutable audited DB target; SR-5 ordered InstanceActor stop/flush;
SR-12 rehydration fence; SR-13 S&F gate recheck per delivery; DC-4 no OPC UA security
downgrade; DC-5 alarm snapshot for every new subscriber; ORCH-1 sync-dispose joins the pump.

Management/UI: MG-2 site scope on audit tree + Flow Debugger Replay; MG-3 stale-probe memo
removed; MG-4 server-owned bulk-deploy deadline; UI-5 sandbox runs in a killable child
process (new CentralUI.SandboxWorker project); UI-14 bounded sandbox output/IPC.

Edge/audit: EI-1 relay audit ownership; EI-2 SkipBodyCapture suppresses error text; EI-3/EI-11
error-field redaction + caps; EI-4 single relay deadline, pre-execution-only retry; EI-5
authenticated relay provenance; EI-10 body cap before audit read.

Persistence/contracts: DL-1 global Instance.UniqueName unique index + conflict translation;
DL-2 DataConnection protocol config encrypted at rest (migration
20260909203046_EnforceGlobalInstanceNamesAndProtectProtocolConfiguration — coordinated
central upgrade, old binaries cannot read protected rows); DL-3 bounded Replay paging;
DL-6 cross-site Overwrite of a deployed instance rejected; XC-4 proto descriptor contract test.

Config/hygiene: XC-1 central listener isolation by local port; XC-2/XC-3 inert SMTP + cron
keys removed from rig settings; CM-2 env2 bootstrap guard enabled; DC-6 deploy site exit code;
DC-7 notifier retries only explicit standby rejection; XC-5 ClusterClient purged from docs.

Gates recorded in the report: build 0/0, 9 397 passed / 0 failed, rig redeployed on all
eight nodes with the migration applied. 46 Low findings remain open.
2026-09-09 22:05:30 -04:00
Joseph Doherty 146a13238f Resolve all high-severity architecture review findings 2026-09-09 16:13:58 -04:00
Joseph Doherty 97f01ae98e fix(flow-debugger): final-review remediation — 2000-row trace pull, shared ExceptionHeader parser, surrogate-safe clip, wording, alarm source, register renumber (I1/I2/M1-M8) 2026-09-09 13:30:48 -04:00
Joseph Doherty ba7f8ee7a9 feat(management): Flow Debugger inbound row carries exception detail; LogsTruncated on the run result
FlowDebugRunner.RunMethodAsync now passes InboundScriptResult.ErrorDetail through to the
synthesized inbound audit row's errorDetail, so a SCRIPT_ERROR run's exception ToString() lands
on AuditDetails.ErrorDetail for the Flow Debugger's inbound node to read (never surfaced to the
HTTP caller). FlowDebugRunResult grows an additive LogsTruncated (default false, last positional
member) reporting the inline FlowTraceCollector's low-priority (ScriptLog) truncation separately
from Truncated (step-row fidelity); RunMethodAsync/RunScriptAsync thread collector.LogsTruncated
through on their two collector-backed results.
2026-09-09 12:23:38 -04:00
Joseph Doherty 16cdf0adf1 Merge wt/step-logs-t3: Task 3 FlowGraphBuilder log-line attachment 2026-09-09 11:40:05 -04:00
Joseph Doherty 395f38e019 feat(commons): FlowGraphBuilder folds ScriptLog rows onto their owning step (D2) 2026-09-09 11:33:54 -04:00
Joseph Doherty a45c0d47c5 Merge wt/step-logs-t2: Task 2 FlowTraceCollector low-priority lane 2026-09-09 11:33:15 -04:00
Joseph Doherty b73962831c refactor(commons): FlowTraceCollector.Add(lowPriority:) instead of a fourth public method (Commons method-count constraint) 2026-09-09 11:28:01 -04:00
Joseph Doherty befb0e3805 feat(commons): FlowTraceCollector low-priority lane for log lines (D7) 2026-09-09 11:26:05 -04:00
Joseph Doherty 8f88cab7f8 feat(commons): ScriptLog audit kind, log-line trace extra, FlowStep.ErrorDetail/Logs (flow-debugger step logs D1/D3) 2026-09-09 11:25:55 -04:00
Joseph Doherty 859922ba17 fix(central-ui): Flow Debugger spawn edges — real arrowhead + bolt badge; spawner picked by trigger attribute
The trigger-cascade (Spawn) edge used a lightning-bolt marker IN PLACE of the arrowhead. On the upward
Tags→Site hop the bolt sat on top of the riser and read as a broken arrow (rig screenshot 2026-09-08).
Every connector now ends in an ordinary arrowhead; the bolt is a badge (circle + bolt) centred on the
connector's vertical run (FlowGraphView.SpawnBadgePoint, kept in step with OrthogonalPath/AsyncPath).

Same screenshot, second defect: the cascade spawned by "RecipeDownloadFlag = false" was drawn from the
LAST write of the burst (RecipeProcessedFlag) because all four writes share a millisecond and the
spawner rule was time-only. FlowGraphBuilder now prefers the parent's last AttributeWrite of the attribute
the child's triggerAttribute names (at or before the child, else the last such write); the time/position
rules remain the fallback when the child names no attribute or the parent never wrote it.

Verified live on the rebuilt rig (DelmiaSim01 / DelmiaReceiverSimulator): both spawn edges show an
arrowhead + badge, the second one originates from the RecipeDownloadFlag write. bUnit + builder tests
updated/added; Playwright FlowDebuggerTests 4/4.

Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
2026-09-08 09:04:39 -04:00
Joseph Doherty 3d5759613f Merge branch 'worktree-agent-a02d51803adfe3011' into feat/flow-debugger-script-entry 2026-09-08 01:08:14 -04:00
Joseph Doherty 952b41f0f7 feat(commons): ScriptTriggerAttribute — monitored attribute of a ValueChange/Conditional trigger (flow debugger script entry)
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
2026-09-08 01:07:38 -04:00