Commit Graph

51 Commits

Author SHA1 Message Date
Joseph Doherty 2750bd2809 Complete management API and CLI command parity 2026-09-10 00:51:45 -04:00
Joseph Doherty 9ee402bbcd Resolve all 46 low-severity architecture review findings 2026-09-09 22:56:36 -04:00
Joseph Doherty 9a87d7a328 Resolve all 33 medium-severity architecture review findings
Second slice of the 2026-09-09 whole-system review remediation
(docs/plans/2026-09-09-arch-review.md §10), on top of the High slice 146a1323.

Runtime/devices: SR-2 alarm on-trigger integration scope; SR-3 correlated NativeAlarmActor
subscribe retry; SR-4 immutable audited DB target; SR-5 ordered InstanceActor stop/flush;
SR-12 rehydration fence; SR-13 S&F gate recheck per delivery; DC-4 no OPC UA security
downgrade; DC-5 alarm snapshot for every new subscriber; ORCH-1 sync-dispose joins the pump.

Management/UI: MG-2 site scope on audit tree + Flow Debugger Replay; MG-3 stale-probe memo
removed; MG-4 server-owned bulk-deploy deadline; UI-5 sandbox runs in a killable child
process (new CentralUI.SandboxWorker project); UI-14 bounded sandbox output/IPC.

Edge/audit: EI-1 relay audit ownership; EI-2 SkipBodyCapture suppresses error text; EI-3/EI-11
error-field redaction + caps; EI-4 single relay deadline, pre-execution-only retry; EI-5
authenticated relay provenance; EI-10 body cap before audit read.

Persistence/contracts: DL-1 global Instance.UniqueName unique index + conflict translation;
DL-2 DataConnection protocol config encrypted at rest (migration
20260909203046_EnforceGlobalInstanceNamesAndProtectProtocolConfiguration — coordinated
central upgrade, old binaries cannot read protected rows); DL-3 bounded Replay paging;
DL-6 cross-site Overwrite of a deployed instance rejected; XC-4 proto descriptor contract test.

Config/hygiene: XC-1 central listener isolation by local port; XC-2/XC-3 inert SMTP + cron
keys removed from rig settings; CM-2 env2 bootstrap guard enabled; DC-6 deploy site exit code;
DC-7 notifier retries only explicit standby rejection; XC-5 ClusterClient purged from docs.

Gates recorded in the report: build 0/0, 9 397 passed / 0 failed, rig redeployed on all
eight nodes with the migration applied. 46 Low findings remain open.
2026-09-09 22:05:30 -04:00
Joseph Doherty 97f01ae98e fix(flow-debugger): final-review remediation — 2000-row trace pull, shared ExceptionHeader parser, surrogate-safe clip, wording, alarm source, register renumber (I1/I2/M1-M8) 2026-09-09 13:30:48 -04:00
Joseph Doherty c1b1702397 fix(cli): single-key note rows render as raw indented lines under their step (Task 14 review) 2026-09-09 12:15:05 -04:00
Joseph Doherty c6529fa996 feat(cli): flow-debug logs column and --show-logs 2026-09-09 12:03:42 -04:00
Joseph Doherty 45c6b27d99 fix(cli): flow-debug help text names the script entry; replay role note; run-script guard tests
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
2026-09-08 01:17:32 -04:00
Joseph Doherty 067a858ce9 feat(cli): flow-debug run-script — run a site script by name through the flow debugger
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
2026-09-08 01:13:15 -04:00
Joseph Doherty 97b0a4ba16 fix(central-ui,cli): Flow Debugger client waits derive from the method timeout / wait window instead of a fixed 2 minutes (arch-review F4)
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
2026-09-06 14:13:02 -04:00
Joseph Doherty 7e3e809a70 feat(flow-debugger): FlowStep carries the audit CorrelationId; CLI table adds a correlation column
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
2026-09-06 10:58:50 -04:00
Joseph Doherty 3c9f9da818 feat(flow-debugger): outbound calls on the External lane — CLI projection, specs, rig demo + verification step I
CLI flow-debug table now projects through FlowGraphBuilder (one row per step,
outbound lifecycles folded, new 'lifecycle' column; per-event fallback kept).
Specs: Component-CentralUI (four lanes, node anatomy, pending refresh),
Component-AuditLog (central method-script Database.* now audited; replay reads
the three outbound channels), Component-InboundAPI (DbOutbound row per helper
call), Component-ScriptAnalysis (REQ-SA-7 kinds + Async edges), design doc
v1.1 status line, CLAUDE.md bullet, deferred-work register row 111 (Retry/
Discard + ?correlationId= deep links deferred). Rig: seed-flow-outbound-demo.sh
(RigRest/RigDb/FlowDemo + OutboundDemo script + FlowOutboundDemo method/key)
and verify-delmia-notifier.sh step I (live rows inline, replay until terminal).
Rig verification result follows in its own commit.

Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
2026-09-06 02:10:50 -04:00
Joseph Doherty 0fb7e16ced feat(cli): flow-debug run-method|run-value-change|replay
New top-level command group for the Flow Debugger (design 2026-09-05 §7.1). run-method /
run-value-change post the FlowDebugRun* management commands with a 2-minute client wait;
replay posts FlowDebugReplayCommand. A table projection unwraps the FlowDebugRunResult
envelope to one row per step (occurred | kind | lane | target | durationMs | status |
error) using the Commons FlowStepProjector's kind/lane resolution, with a trailing note
row for an incomplete run (error code verbatim, e.g. STANDBY_NODE) or a truncated event
list. --params must be a JSON object (INVALID_PARAMS otherwise). README section added;
command-tree test bumped to 18 groups.

Claude-Session: https://claude.ai/code/session_01GSLkkHKz9JrAtmJjmQK3cP
2026-09-06 00:11:42 -04:00
Joseph Doherty 02a374259b feat(cli): instance bind-module — rebind one composed module's members to a new object path via GetInstance → SetConnectionBindings upsert; --dry-run 2026-08-20 15:55:09 -04:00
Joseph Doherty b34ec10f8b fix(deploymentmanager): bulk site deploy skips up-to-date instances and un-poisons the shared DbContext (arch-review Low P7-5 + P7-6)
P7-6 (behaviour fix, plan decision 1): DeploySiteAsync redeployed EVERY
deployable instance unconditionally. A redeploy is not idempotent at the site —
it resets static attribute overrides and clears native alarm state by design — so
one site-wide push destroyed live operational state on every unchanged instance
in the batch, which is normally most of them. Bulk deploy now pre-filters with
IStaleInstanceProbe (the same seam the bundle importer and Deployments page use;
its memo timing was fixed by the Medium wave's P7-4): an instance whose freshly
computed revision hash still matches its DeployedConfigSnapshot is reported in the
existing additive SkippedInstances list with reason "up-to-date" and never
flattened, staged or sent. The filter is deliberately conservative — every
uncertainty deploys: only an Enabled instance qualifies, a missing snapshot
disqualifies, a prior record that ShouldQuerySiteBeforeRedeploy flags goes through
prepare so the reconcile that HEALS it still runs, and a null/faulting probe means
"indeterminate". New additive MgmtDeploySiteCommand.Force (default false = the new
filtered behaviour) plus CLI `deploy site --force` restores the unconditional push
for the case central cannot see (restored volume, rebuilt node). Composes with the
Medium wave's P7-2: Disabled wins — force suppresses the up-to-date filter only,
never the Disabled skip.

P7-5 (verified, then fixed): the cascade REPRODUCES. All three bulk phases share
one scoped DbContext, and a failed SaveChanges does not untrack the offending
entity — EF re-attempts it on the next save, including MarkDeploymentFailedAsync's
cleanup write. Measured on a real SQLite-backed ScadaBridgeDbContext: a permanent
save failure on instance 1's post-success side effects failed instances 2, 3 and 4
and left all three deployment records stranded InProgress. MarkDeploymentFailedAsync
now detaches the pending changes (new IDeploymentManagerRepository.DetachPendingChanges)
when the cleanup save fails and retries the Failed write once on the clean tracker,
so later instances save cleanly and the record reaches a terminal status. The
DeployFailed audit row moved outside the status-write try (best-effort, must not be
lost because the status write failed).

Tests: 5 new DeploySiteAsyncTests (skip + reason + no site round-trip; --force
deploys; Disabled skipped under both force values; the three not-enough-to-skip
guards), 2 new IntegrationTests over a real EF context (the poisoned-context
cascade, and the narrow EF tracking fact the fix rests on), CLI --force flag
parse test, and a MgmtDeploySite wire-default test. Proof of failure: both P7-6
behaviour tests and both P7-5 assertions fail against the unfixed code.
2026-08-19 16:41:04 -04:00
Joseph Doherty 9a3f006721 fix(management)+fix(cli): neutralize CSV formula injection + align export columns; deadline + Ctrl-C on audit export streaming (arch-review 2026-08-19 P6-7 + P6-8)
P6-7 (CWE-1236): AuditEndpoints CSV export quoted cells on comma/quote/CR/LF
but never neutralized a leading =, +, -, @, TAB, or CR — inbound-API-caller
text landing in Target/ErrorMessage/RequestSummary/ResponseSummary/Extra
could execute as a formula/DDE trigger when opened in Excel. AppendField now
prefixes a literal single quote on free-text columns only (never the
GUID/enum/numeric/date/bool columns the exporter formats itself), applied
before RFC 4180 quoting. The CSV was also missing ExecutionId/
ParentExecutionId/SourceNode that the JSONL export already carried; both are
now 24 columns in the same AuditExportRow order. A second, independently
maintained CSV writer (CentralUI's AuditLogExportService, feeding the Audit
Log page's bulk-export button) had the identical gap and got the identical
fix + column alignment.

P6-8: `audit export` was the one CLI HTTP path with no deadline at all — a
residual gap from the ManagementHttpClient timeout-honesty fix, which made
HttpClient.Timeout infinite and the per-call token the single deadline
everywhere except this streamed GET. RunExportAsync now takes a headers-
received deadline (CancelAfter, reusing ManagementHttpClient.
DefaultConnectTimeout) around the initial GetAsync, an inactivity deadline
around the streaming copy (rearmed on every successful read, so large exports
are bounded by silence, not total size), and an external CancellationToken
wired to Console.CancelKeyPress in AuditCommands so Ctrl-C cancels promptly.
A stalled/faulted stream now produces a friendly error noting the partial
output file instead of hanging or dumping a stack trace.
2026-08-19 15:01:45 -04:00
Joseph Doherty f33f25a10b fix(auth+cli): fail-closed site-scope gate (row 98); case-insensitive audit filter values with a strict server-side check (row 99)
Register row 98 — the site-scope authorization gate was FAIL-OPEN on an
unresolvable target. The resolve-then-check shape was written as
`if (target != null) EnforceSiteScope(...)`, so a null resolve skipped the
check entirely and a site-scoped caller passed the gate for any id in the
system. Authorization now fails CLOSED: a target that cannot be PROVEN to sit
inside the caller's scope is treated as outside it.

The defect turned out to have THREE more instances than the register recorded.
Besides `EnforceSiteScopeForInstance` and `EnforceSiteScopeForIdentifier`, the
same inline shape sat in `HandleGetInstance`, `HandleGetSite` and
`HandleGetDataConnection` — read handlers that resolve their own target and so
never call the helpers. Those were the sharper half of the bug: they answered
success-with-null for a nonexistent id and Unauthorized for an out-of-scope
one, making the pair an id-enumeration oracle over the whole fleet. All five
now route through one shared `DenyUnresolvedTargetIfScoped` arm.

Two properties are load-bearing and pinned by tests:
  - System-wide callers (Administrator, or a Deployer with no site grants) are
    untouched — they return at the guards above the lookup and keep the
    handler's ordinary "not found" / null answer.
  - The denial is indistinguishable from an out-of-scope denial: same
    ManagementUnauthorized shape, and a message that never states whether the
    target exists.

Register row 99 — CLI `audit query --status delivered` failed client-side while
the server would have accepted it: `AcceptOnlyFromAmong` matches ordinally
against the capitalised literals. Same treatment row 88 gave `deploy status` —
the client-side gate is removed, the valid set moves into the option help text,
and the server becomes the single authoritative check.

That required ADDING the server check, not just relying on one: the shared
audit filter parser was LAX, silently dropping an unrecognised value and
leaving the dimension unconstrained — so removing the CLI gate alone would have
turned a loud client-side failure into a plausible, complete-looking page that
did not answer the question asked. New `ParseEnumListStrict` (case-insensitive,
rejects ordinals — `Enum.TryParse` accepts "3", and a filter is a name-based
contract) backs the ManagementService /api/audit/query + /api/audit/export
endpoints, which now return 400 BAD_REQUEST naming the valid set.

Other AcceptOnlyFromAmong sites surveyed in the audit group: `--channel`,
`--kind`, `--status` on BOTH `query` and `export` (6 uses) plus `export
--format` (rejected `--format CSV` though the server lowercases it) — all seven
fixed here. Deliberately left alone: `template folder reorder --direction`
(pure client-side literal, no server-side parse to defer to) and the global
`--format json|table` (CLI-local output selection, never sent to a server).

The CentralUI AuditLogPage drill-in parser and the CentralUI export endpoint it
feeds deliberately KEEP the lax parse — they read a query string a human may
have bookmarked, where a stale fragment must still render the page. Both
contracts now live side by side in AuditQueryParamParsers and are documented as
non-reconcilable.

Docs: Component-ManagementService.md gains a "Site-scope gate: fail-closed on
an unresolvable target" section and a strict-filter note on the audit
endpoints; Component-Security.md records the fail-closed shape under Deployer
site scoping; the CLI README marks the audit filter values case-insensitive and
states the no-silent-drop / no-ordinals contract.

Tests: 5 new ManagementActorTests fail before the row-98 fix (verified by
reverting ManagementActor.cs) and cover scoped+nonexistent -> denied,
system-wide+nonexistent -> unchanged (both Administrator and no-grant Deployer),
scoped+in-scope -> allowed, and the indistinguishability property. 21 CLI tests
fail before the row-99 fix. Four pre-existing tests that pinned the removed
client-side gate were rewritten to pin the new contract.
2026-08-17 06:05:29 -04:00
Joseph Doherty 9f67147e7b refactor(deployments): one paged read of DeploymentRecords + close its three drifts
Deferred-work register rows 37, 86, 87, 88 — the structural duplication and the
three behavioral divergences it had accumulated, fixed together.

Row 37 (structural). `IDeploymentManagerRepository.QueryDeploymentSummariesAsync`
is DELETED. It and `QueryDeploymentListPageAsync` were two paged reads of the same
insert-only table with different contracts — raw offset vs page number, no total vs
total, no instance join vs joined, instance-id-list scoping vs site-id scoping. The
CLI/HTTP `QueryDeployments` handler now goes through `QueryDeploymentListPageAsync`
with a `DeploymentListFilter`, exactly like the Central UI page. The
`DeploymentRecordSummary` projection went with it (`DeploymentListRow` is the single
row shape); its `MaxPageSize` const moved to `DeploymentListPage`, which also gains
`DefaultPageSize`.

Row 86 (Medium, authorization — the headline). `SiteScopeService.ResolveAsync` was
claims-only, so an Administrator carrying `SiteId` claims saw the whole fleet through
the CLI (`ManagementActor.EnforceSiteScope` has always exempted Administrator) and
only their granted sites on the Deployments page. One user, one dataset, two answers.
Administrator = system-wide is the intended semantic on both sides; the UI was the
defect. The role claim is read explicitly with an OrdinalIgnoreCase comparison rather
than via `IsInRole`, which compares claim VALUES ordinally and only resolves when the
identity carries the matching roleType — the two sides must not disagree on a
differently-cased role name. An Administrator's site claims are not collected at all,
so the invariant "system-wide => empty permitted-id set" holds for every caller.
Designer is NOT exempt: the class doc claimed "any Admin/Design user" is system-wide,
which was true of neither side, so the doc was corrected rather than the bypass
widened.

Row 87 (Low). The scoped branch used to resolve the site grant into an instance-id
list by loading EVERY instance in the system, non-transactionally, on every scoped
query; the grant now rides as `SiteIdScope` and is resolved by the query's own join.
An unparseable permitted-site id is DROPPED, never ignored — dropping can only narrow
a grant, discarding it would widen it to fleet-wide. And `PageSize <= 0` no longer
means UNLIMITED down at the repository (`--page-size 0` materialized the whole
retention window): it means "unspecified" and takes the CLI's own default of 50,
capped at 1000. Bounding without telling would be silent truncation, so the handler
now returns a `DeploymentQueryResult` envelope carrying `totalCount` plus the page
coordinates actually applied — with a CLI table projection (mirroring row 57's
`AuditConfigTableProjection`) that unwraps it and appends a "showing N of M" note.

Row 88 (Low, UX). An unrecognised `--status` fell back to "no filter", so a typo like
`--status Sucess` returned every status — a complete-looking answer to a question the
caller never asked. It is now a `ManagementError` naming the valid values. Parsing
stays case-insensitive; a numeric string is rejected too, since `Enum.TryParse`
accepts "7" and yields an undefined value (guarded with `Enum.IsDefined`). Loud
failure over silent full results, per row 46.

Tests: 7 new `SiteScopeServiceTests` pinning Administrator = system-wide (verified
failing 6/7 against the pre-fix service, the Designer pin passing both ways); the
ManagementActor deployment suite migrated onto the consolidated query with new pins
for the status error, case-insensitive parse, numeric rejection, page-size default/
cap/floor, empty-scope-on-garbage-grant, and "reads no instances at all"; a repository
test for site-scope+status composing over a bounded page; and 6 CLI tests for the new
table projection.
2026-08-17 05:15:18 -04:00
Joseph Doherty e8a90f18dc merge: external-system retry policy end-to-end (register row 27) — MaxRetries/RetryDelay now travel artifact→gRPC→site schema→S&F enqueue, CLI --max-retries/--retry-delay-seconds, ArtifactDiff compares them 2026-08-15 17:56:21 -04:00
Joseph Doherty b2cab59cb9 feat(cli): compact table projection for audit-config query (register row 57)
The command passed no `tableProjector`, so `--format table` handed the renderer
an `AuditLogQueryResult` OBJECT. That takes the generic Property/Value branch,
which printed the entire `entries` array as raw JSON inside a single cell. JSON
output was and remains correct, and is the primary format for this command.

Adds `AuditConfigTableProjection`, mirroring `TemplateTableProjection`:
string->string over the raw body, applied only on the table path, returning the
input verbatim when it is not JSON or not a recognised shape so the renderer's
own fallbacks still apply. It unwraps the envelope to a row array — id,
timestamp, user, action, entityType, entityName, entityId — and drops
`afterStateJson`, the full post-change entity snapshot, which is the audit
analogue of template's fat child arrays: one per row fills a terminal.
`bundleImportId` is dropped as mostly-null noise.

`totalCount` would be lost by a plain unwrap, so a truncated page gains a single
trailing row with a `note` cell ("showing 50 of 137 — use --page / --page-size
for the rest"). When the page IS the whole result set the column is not emitted
at all, so a small query's table stays exactly as narrow as its data, and the
note row carries no `id` so it cannot be mistaken for an entry.

A `--detail` flag skips the projection and renders the raw payload, matching
`template list` / `template get` — that plus `--format json` is how to reach
`afterStateJson`. A bare array root is also accepted, so the projection does not
regress if the payload is ever unwrapped upstream or the deprecated `audit-log`
alias meets an older server.

Tests build their fixtures by serialising the REAL AuditLogQueryResult /
AuditLogEntry types, so renaming a server property fails the test instead of
silently blanking a column. CLI README + Component-CLI.md document the table
shape, the omitted fields, and --detail.
2026-08-15 13:22:41 -04:00
Joseph Doherty 41a5ecf14a feat(external-systems): deploy per-system retry policy to sites + CLI/management surface (register row 27)
An external system's MaxRetries/RetryDelay existed on the central entity but never
reached a site and had no management surface, so every ExternalSystem.CachedCall
buffered with the S&F engine default (DefaultMaxRetries 50 x DefaultRetryInterval
30s ~= 25 min to park) whatever central had configured. The sibling
DatabaseConnectionArtifact had always carried its equivalents.

The consumption end was already wired: ExternalSystemClient.CachedCallAsync already
read system.MaxRetries/RetryDelay and passed them to EnqueueAsync with the 0-as-unset
convention. Only the transport and persistence were missing.

Commons
- ExternalSystemArtifact gains MaxRetries/RetryDelay as optional positional params
  (0 / TimeSpan.Zero = unset), matching DatabaseConnectionArtifact rather than going
  nullable, so it maps 1:1 onto the entity and the proto3 scalars.
- Create/UpdateExternalSystemCommand gain nullable MaxRetries/RetryDelay, following
  the existing TimeoutSeconds "omit to preserve" convention. Both additive.

Central
- ManagementActor create maps both; update preserves-if-null.
- ArtifactDeploymentService projects them onto the artifact.

Wire
- site_command.proto ExternalSystemArtifactDto gains max_retries=7 /
  retry_delay=8 (google.protobuf.Duration), mirroring DatabaseConnectionArtifactDto.
  Additive: a pre-existing site decodes them as 0/absent, which IS the unset value,
  so old behaviour is preserved. Vendored C# regenerated via docker/regen-proto.sh
  sitecommand; csproj Protobuf item re-commented, no message types added or removed.

Site
- external_systems gains max_retries / retry_delay_ms via AddColumnIfMissing,
  DEFAULT 0 = unset. Storage upsert, repository SELECT/mapping (new ordinals 6/7
  appended so existing ones are untouched), apply path, and the legacy migrator's
  explicit column list all carry them.

Replicated-table decision: external_systems IS in SiteLocalDbSetup.ReplicatedTables,
so this was checked against the AckTime/native_alarm_state precedent. Real columns
are correct here, not a JSON bag: the table has no general-purpose config column
(auth_configuration and method_definitions are both semantically specific),
timeout_seconds already set the additive-column precedent on this exact table, and
sibling database_connections stores the same two values as plain columns. Verified in
LocalDb source that RegisterReplicated re-reads PRAGMA table_info every boot and its
install script is DROP-then-CREATE, so capture triggers rebuild over the new column
set automatically, and SiteStorageSchema.Apply runs before registration on the same
boot. The only cost is the ReplicatedTable.ComputeDigest change, which the sync
handshake compares fail-closed - a mixed-version pair refuses to sync loudly instead
of diverging silently. Both-nodes-together is already the standing site-pair rule, so
no new operational constraint; recorded in topology-guide.md.

Also fixed: ArtifactDiff.CompareExternalSystem never compared MaxRetries/RetryDelay,
so a bundle import that changed a system's retry budget previewed as "Identical" -
harmless while the values were inert, a live behaviour change now.

No EF migration: the central ExternalSystemDefinitions columns already exist
(InitialSchema). Central UI already had both form fields.

Tests: site storage round-trip + unset-default, site apply path, central artifact
projection, proto round-trip (via the shared sample), management create/preserve,
CLI option parse + alias, and diff reporting. Fail-before verified on the two core
tests (expected 7 got 0 on the read path; expected 9 got 0 on the apply path).

Docs: Component-ExternalSystemGateway, Component-StoreAndForward, Component-CLI,
components/StoreAndForward, components/SiteRuntime, deployment/topology-guide,
CLI README (which was also missing --timeout-seconds entirely).
2026-08-15 12:56:12 -04:00
Joseph Doherty e0e4b24679 fix(deploy+cli): review findings — honest CLI timeouts, watermark-complete staleness, phase-2 staging, lock-safe cancellation
Six adversarial-review findings, each verified against the code first.

F1 (HIGH) CLI HttpClient capped every call at min(30s, caller timeout),
silently truncating deploy site's 5-minute BulkDeployTimeout and the
5-minute bundle export/preview/import calls — which printed a fake
"504 Request timed out" while the server kept working. HttpClient.Timeout
is now Timeout.InfiniteTimeSpan (the per-call CTS is the single overall
deadline, connect included) with the connect phase bounded separately on
SocketsHttpHandler.ConnectTimeout. The env override is renamed to
SCADABRIDGE_HTTP_CONNECT_TIMEOUT_SECONDS to match its new meaning.

F2 (HIGH) StaleInstanceProbe's process-static memo served stale hashes
because nothing bumped the watermark on three paths:
  (a) BundleImporter commits through the raw DbContext, so no import ever
      moved the watermark — a second import overwriting the same template
      could be OMITTED from ImportResult.StaleInstanceIds. It now bumps
      once per apply ATTEMPT: after the commit, and after the rollback too
      (the probe runs pre-commit, so a rolled-back attempt leaves memos for
      state that never landed; bumping on both paths is the simplest
      correct shape, versus threading transaction awareness through a
      process-static cache).
  (b) CollectWatermarkBumps' default: arm silently no-op'd, contradicting
      its own doc. It now sets unattributed=true — an over-broad bump costs
      extra work, a missed one produces stale work.
  (c) DataConnection edits route through SiteRepository.SaveChangesAsync,
      which had no watermark at all, yet Protocol/Primary+Backup config/
      FailoverRetryCount are revision-hash inputs. It now bumps (BumpAll —
      a connection has no owning template) after a commit that touched one.

F3 (MED) CLI TemplateTableProjection read child ARRAYS, but ListTemplates
now returns database-projected TemplateSummary rows, so template list
printed all zeros. It now prefers the *Count scalars and falls back to
array length (template get still returns full entities). --detail help
text and README corrected: a listing cannot yield definitions, so --detail
renders the raw summary payload and template get --id is the full dump.

F4 (MED) DeploySiteAsync staged every PendingDeployment in phase 1 against
a 5-min TTL while phase 2 reached them one batch at a time, so tail
instances' fetch tokens could expire before their command was sent.
Staging moved into phase 2, immediately before each send; prepare keeps
its flatten/validate/record work. The staging write is the phase's only
repository touch and is serialised behind a 1-permit semaphore, so the
non-thread-safe DbContext constraint holds and the sends stay concurrent.

F5 (MED, latent) DeploySiteAsync leaked every held operation lock if
cancelled — a wedged per-instance semaphore is permanent for the process.
Phase 2 no longer throws (cancellation is recorded as a per-instance
outcome so phase 3 still runs), and an escape from phase 1 or 3 now
unwinds every unfinalised entry: Failed status + lock release.

F6 (LOW) ScriptCompileVerdictCache's promotion wrote hot directly,
bypassing SegmentCapacity (true ceiling 3x against a documented 2x).
Promotion now goes through Store, keeping generational semantics; _hot
and _cold are volatile.

Tests: CLI 396, DeploymentManager 133, ManagementService 494,
TemplateEngine 478, ScriptAnalysis 60, Transport 157, Transport
integration 106, ConfigurationDatabase 366 — all green, 0 build warnings.
The F2/F4/F5 regression tests were each confirmed to FAIL with their fix
reverted.
2026-08-14 23:51:04 -04:00
Joseph Doherty 4cd1441984 Merge branch 'worktree-agent-aae48b78656e5a4e0' into arch-review-remediation
# Conflicts:
#	src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Repositories/SiteExternalSystemRepository.cs
2026-08-14 21:16:11 -04:00
Joseph Doherty 48b3c40a7f perf(deploy): flatten-session caching, bulk DeploySiteAsync, paged management queries 2026-08-14 21:14:22 -04:00
Joseph Doherty a212283104 perf(misc): cached hot-path lookups, bounded observer queue, alarm-priority stream path
WP2.6 (arch-review remediation, cross-cutting misc):
- SiteExternalSystemRepository: name/ID-indexed ExternalSystemDefinitionCache replaces
  the fetch-all + reverse-map scan on every by-ID/method lookup; loaded once per
  redeploy, invalidated by DeploymentManagerActor after HandleDeployArtifacts applies
  external-system changes. Static JsonSerializerOptions for method-list parsing.
- Inbound API: short-TTL ApiMethodCache fronts the per-request ApiMethod repository
  fetch; invalidated by name via the existing ScriptArtifactChangeSubscriber/
  IScriptArtifactChangeBus pipeline, self-healing via TTL for changes the bus
  doesn't cover (e.g. Management API edits).
- StoreAndForward: the cached-call audit-observer queue — the one unbounded channel
  left in the system — is now bounded (ObserverQueueCapacity, default 10,000) with
  DropOldest overflow and a dropped-notification counter.
- SiteStreamManager: alarm state changes now travel a dedicated publish
  source/broadcast hub, isolated from the (far higher-volume) attribute path, so an
  attribute storm can no longer evict a pending alarm transition; the alarm hand-off
  queue is bounded with a drop counter surfaced on the site health report
  (SiteStreamAlarmDropCount via the new SiteStreamAlarmDropReporter), and publishing
  is skipped entirely at zero subscribers on either path.
- CLI ManagementHttpClient: explicit 30s HttpClient.Timeout on the shared
  construction (was the 100s framework default), overridable via
  SCADABRIDGE_HTTP_TIMEOUT_SECONDS.

Deviation: the failback-probe heartbeat item is NOT included — its only viable
surface (CentralChannelProvider.cs / heartbeat consumers) lives entirely in the
Communication project, explicitly off-limits to this work package this phase.

Tests: SiteRuntime.Tests (550), InboundAPI.Tests (278), StoreAndForward.Tests (133),
CLI.Tests (390), HealthMonitoring.Tests (97) — all green after full solution build.
2026-08-14 20:59:43 -04:00
Joseph Doherty 0fe1972960 feat(cli): notification smtp update --transport smtp|ews 2026-08-10 06:35:33 -04:00
Joseph Doherty 88638d774a feat(cli,management): close area-move and template-folder CLI parity gaps
Two verified-absent parity gaps between the service layer and the CLI /
ManagementActor command surface, both left as follow-ups by the 2026-05-11
design plans.

(1) area move. AreaService.MoveAreaAsync had existed since the deployment
topology page shipped but was reachable only from the Blazor UI. Adds
MoveAreaCommand(AreaId, NewParentAreaId?) to Commons, a ManagementActor
dispatch arm delegating straight to AreaService.MoveAreaAsync (not-found /
self-parent / descendant-cycle / cross-site / name-collision all surface as
the standard curated ManagementCommandException failure response; the service
writes its own "Move" audit row), and the CLI verb `site area move --id
[--parent-id]`. Omitting --parent-id moves the area to the site root, matching
the command's nullable NewParentAreaId. The command carries the SAME any-of
[Designer, Deployer] gate as CreateArea/UpdateArea/DeleteArea (arch-review C6):
re-parenting is the same structural authoring act, exposed on the same two
surfaces. Placed under the existing `site area` group rather than a new
top-level `area` group, alongside its create/update/delete siblings.

(2) template folder verbs. The five folder management commands have been
handled by ManagementActor since the folder-hierarchy plan, but the promised
CLI surface was never written. Adds `template folder
list|create|rename|move|reorder|delete` mapping 1:1 onto ListTemplateFolders /
CreateTemplateFolder / RenameTemplateFolder / MoveTemplateFolder /
ReorderTemplateFolder / DeleteTemplateFolder. --parent-id is omitted to target
the tree root; --direction takes the lowercase literals up/down, validated at
parse time by AcceptOnlyFromAmong (same case-sensitive contract as the audit
--channel/--kind/--status options).

Follow-on updates: the frozen authorization matrix gains its MoveArea entry
(reflection-driven, so a missing entry would have failed CI); CommandTreeTests
pins both new verb sets plus the omit-parent-id-means-root parse behaviour and
registry round-trips; ManagementActorTests covers the MoveArea role gate and
the delegate-to-service success/root/cycle/not-found paths; the CLI README and
Component-ManagementService.md document the new surface (the latter also gained
the previously-undocumented ReorderTemplateFolder); both plan docs' follow-up
lines are marked done.
2026-08-01 11:15:24 -04:00
Joseph Doherty f480a56737 feat(instance): native-alarm-source-override CSV bulk import (deferred #12) + doc fixes
Closes the operator parity gap the deferred-work register tracked as #12: native
alarm source overrides could only be set one-at-a-time, while attribute overrides
had a CSV bulk path. Adds an all-or-nothing CSV import for native sources.

- Commons: extract the RFC-4180 line splitter into a shared CsvLineSplitter
  (refactor OverrideCsvParser onto it — no behavior change, pinned by its tests);
  new NativeAlarmSourceOverrideCsvParser (header SourceName,Connection,
  SourceReference,Filter; blank = inherited).
- Commons: NativeAlarmSourceOverrideEntry + bulk SetInstanceNativeAlarmSource-
  OverridesCommand (auto-registered via the reflection command registry).
- ManagementService: Deployer-gated handler — flattens once, validates every
  source resolves + is unlocked + no duplicates up front, then upserts the whole
  batch under a single SaveChanges (true all-or-nothing txn). Added to the frozen
  authorization matrix; dispatch-coverage guard passes.
- CLI: `instance native-alarm-source import --instance-id --file` (parity with
  `instance import-overrides`) + README.
- Tests: native parser (Commons), CLI parse/entry mapping, 3 bulk-handler tests
  (happy path single-commit, locked-source reject, unresolved-source reject).

Also corrects a stale XML-doc line in ScriptRuntimeContext (WaitForAttribute
quality-gated mode is shipped, not "planned") and updates the deferred-work
register: marks #7/#13/#15/#16/#20 verified-resolved, #12 as CLI/API-shipped with
only the Central UI upload affordance still pending.

Claude-Session: https://claude.ai/code/session_01MtdgwpEeCUn6cUA5f1LMPj
2026-07-10 08:52:12 -04:00
Joseph Doherty 433c6db4ec feat(cli): browse/search/verify-endpoint/cert-trust commands — actor parity (arch-review C4)
Claude-Session: https://claude.ai/code/session_01MtdgwpEeCUn6cUA5f1LMPj
2026-07-10 05:57:58 -04:00
Joseph Doherty 655834f0b8 feat(management): additive Skip/Take paging on template/instance lists (arch-review P2)
Claude-Session: https://claude.ai/code/session_01MtdgwpEeCUn6cUA5f1LMPj
2026-07-10 05:57:14 -04:00
Joseph Doherty 8d4ffa7ef1 feat(management): secured-write list paging with TotalCount (arch-review P3)
Claude-Session: https://claude.ai/code/session_01MtdgwpEeCUn6cUA5f1LMPj
2026-07-10 05:42:35 -04:00
Joseph Doherty eabd2820eb feat(esg): --timeout-seconds on external-system create/update (management command + CLI, additive contract)
Claude-Session: https://claude.ai/code/session_01MtdgwpEeCUn6cUA5f1LMPj
2026-07-10 04:20:28 -04:00
Joseph Doherty cdd65beb6c feat(cli+templateengine+deploymanager): resolve follow-ups #4/#5/#6/#8 — CLI ergonomics + structured deploy validation error
Closes the four remaining items in the 2026-06-24 template-inheritance/CLI
follow-up tracker.

#4 — CLI `instance set-bindings` can now set DataSourceReferenceOverride.
  `--bindings` accepts an optional 3rd element per entry:
  [attributeName, dataConnectionId, dataSourceReferenceOverride]. A string
  sets the override; a JSON null or an omitted 3rd element leaves it unset
  (template default). TryParseBindings accepts 2- or 3-element entries and
  rejects a non-string/non-null 3rd element or 4+ elements with a clean
  error. Previously the CLI sent the override as null and silently wiped any
  existing one (only a raw POST /management could set it).

#5 — `template update` is partial, not full-replace (fixed server-side so all
  clients benefit). UpdateTemplateAsync now uses leave-unchanged semantics:
  a null description keeps the stored value (pass "" to clear); a null
  parentTemplateId keeps the existing parent. Parent stays immutable — a
  non-null differing value is still rejected — but omitting --parent-id is
  now a no-op instead of failing every derived-template update.

#6 — compact `template list`/`get` table output + `--detail`. Table output is
  now id/name/description/parent/derived + member counts (#attrs/#alarms/
  #scripts/#comps/#nativeAlarms) via TemplateTableProjection, fed through a
  new optional tableProjector seam on CommandHelpers. `--detail` restores the
  full dump. JSON output is left untouched (always full) so machine consumers
  are unaffected — the projector only runs on the table path.

#8 — structured deploy-time validation error. New ValidationResult.SummarizeErrors()
  (Commons) returns a grouped, capped summary: leading total count, one line
  per ValidationCategory, and a per-module rollup (canonical name up to its
  last dot) with counts + "... and N more module(s)" caps. DeploymentService
  uses it for the "Pre-deployment validation failed" message and logs the full
  per-entry list via LogWarning. Replaces the flat semicolon-joined dump that
  became a wall of text for instances with 50-194 unbound attributes.

Tests: +8 Commons (SummarizeErrors), +8 CLI (4 binding 3-element / 4 table
projection), +2 net TemplateEngine (partial-update). Affected suites green:
Commons 587, CLI 341, TemplateEngine 447, DeploymentManager 101,
ManagementService 230, CentralUI 866; full solution builds 0/0.

Docs: Component-DeploymentManager.md "Validation Error Reporting"; CLI README
(set-bindings 3-element form, template update leave-unchanged, list/get
--detail); UpdateTemplateCommand doc; known-issues tracker #4/#5/#6/#8 resolved
(all 8 items now closed).
2026-06-24 18:27:42 -04:00
Joseph Doherty 33e1802e6d feat(sms): make FromNumber optional — support Twilio Messaging-Service-only configs (UI-Med-2)
Code-review finding UI-Med-2: the design doc + delivery adapter treat FromNumber and
MessagingServiceSid as either-or, but the entity ctor, EF schema, UI and CLI all hard-
required FromNumber — so a Messaging-Service-only Twilio config (a normal production
setup) could not be created. Bring the implementation into line with the spec:

- Commons: SmsConfiguration.FromNumber -> string? (ctor fromNumber optional);
  UpdateSmsConfigCommand.FromNumber -> string?.
- ConfigurationDatabase: FromNumber.IsRequired(false) + migration SmsFromNumberOptional
  (ALTER COLUMN nullable, idempotent; Down backfills '' — harmless, MsgSid keeps it
  deliverable) + regenerated model snapshot.
- Transport: SmsConfigDto.FromNumber -> string? (round-trips a Messaging-Service-only config).
- CentralUI: form validation requires AccountSid + at-least-one-of(FromNumber, MsgSid);
  nullable create/edit paths; From-number help text.
- CLI: --from-number no longer Required; BuildUpdateSmsConfigCommand validates the either-or.
- Adapter: From branch null-forgiving (guarded by the existing incomplete-config check).

Tests: ManagementActor MsgSid-only persists null FromNumber; CLI MsgSid-only builds +
neither-throws + contract (--from-number not Required); CentralUI MsgSid-only save.
2026-06-19 15:19:40 -04:00
Joseph Doherty a9393c8913 test(sms): regression tests for code-review fixes
Lock the behaviors changed by the review-fix commit + the security invariants:

- ManagementActorTests: UpdateSms/SmtpConfig now require Administrator (updated the
  existing success cases from Designer); + UpdateSmsConfig_WithDesignerRole_Returns
  Unauthorized and _WithEmptyAuthToken_PreservesExistingToken regression tests.
- SecretEncryptionTests: SmsConfiguration.AuthToken stored-encrypted round-trip +
  null round-trip (AccountSid stays plaintext) — guards ApplySecretColumnEncryption.
- ArtifactDiffTests: CompareSmsConfiguration New/Identical/Modified + the secret
  presence-only invariant (value never echoed, presence-flip shows <present> only).
- UpdateCommandContractTests: notification sms update core fields Required, --auth-token optional.
- NotificationListsPageTests: SMS recipient badge shows phone, not "Name <>".
- NotificationOutboxActorDispatchTests: SMS-typed notification routes to the SMS
  adapter (StubAdapter.Type made configurable), not the Email adapter.
- NotificationRecipientTests (new): ForEmail/ForSms + public-ctor invariants.
2026-06-19 15:09:47 -04:00
Joseph Doherty 73df322a66 feat(sms): CLI list --type/--phones + notification sms group + channel-aware recipients (S6) 2026-06-19 10:40:09 -04:00
Joseph Doherty e3b83f8561 fix(cli): guard min-time overflow + normalize 0 exec-timeout to null + stale comment (#54 review) 2026-06-19 03:24:11 -04:00
Joseph Doherty 597d664a53 fix(cli): warn on --trigger-kind without --trigger-config (#257)
InjectAnalysisKind returns null on a null/empty trigger-config, so passing
--trigger-kind strict WITHOUT --trigger-config silently dropped the kind on
template script add/update and template alarm update. The CLI now detects that
combination (TriggerKindWillBeIgnored) and prints a clear warning to stderr,
then continues (warn-and-continue: the kind is advisory metadata, not a required
field, so the entity is still created — just without the requested analysis kind).
The --trigger-kind help text on all three commands now documents that it requires
--trigger-config, as does the CLI README.

This commit also carries the shared CLI command-builder file (Commands/TemplateCommands.cs)
and README, which the same builders edit for both #257 and the #54 flag additions —
the #54 message contracts/handler/UI/tests landed in the preceding commit.

- TriggerKindWillBeIgnored predicate + WarnIfTriggerKindIgnored stderr warning,
  wired into script add/update and alarm update SetActions.
- Shared option descriptions document the --trigger-config requirement.
- Adds the #54 CLI flags (--min-time-between-runs, --execution-timeout-seconds) and
  TryParseMinTimeBetweenRuns to the same builder file.
- Tests: TemplateTriggerKindIgnoredTests pins the warn predicate.
2026-06-19 03:14:24 -04:00
Joseph Doherty ae25b5a8d6 feat(cli): add --execution-timeout-seconds + --min-time-between-runs to template script (#54)
Expose TemplateScript.ExecutionTimeoutSeconds and MinTimeBetweenRuns (previously
settable only via Transport bundle import) on the CLI and Central UI authoring surfaces.

- Commons: add additive trailing optionals MinTimeBetweenRuns (TimeSpan?) and
  ExecutionTimeoutSeconds (int?) to Add/UpdateTemplateScriptCommand.
- ManagementActor: thread both new fields into the built TemplateScript on add/update.
- CLI template script add/update: new --min-time-between-runs (duration: ms/s/min,
  bare number = seconds, 0 = unset, mirroring DurationInput) and
  --execution-timeout-seconds (int) flags, with client-side duration validation.
- Central UI TemplateEdit: add an Execution timeout input (seconds) on the script
  trigger tab, mirroring the existing Min-time-between-runs control; null/0 = site default.
- Tests: TemplateScriptTimingTests pins the option surface + duration parsing; updated
  the stale 'no UI control' comment on the TemplateService round-trip test.
2026-06-19 03:14:10 -04:00
Joseph Doherty 0bd5e0986f fix(m9/T28b): forward --trigger-kind on alarm-update/script-add/script-update; fix default-selector test
Three dead-code bugs: --trigger-kind was registered but never read or forwarded on the
alarm-update, script-add, and script-update paths. Introduced TriggerConfigJson.InjectAnalysisKind
helper that rewrites any raw --trigger-config JSON blob, writing "analysisKind":"Strict" when
the flag is strict (case-insensitive) and stripping the key for any other value. Wired the
helper into all three handlers alongside the existing alarm-add path (which already used
AlarmTriggerConfigJson.Build). Added 6 unit tests for the new helper in TemplateTriggerKindTests.

Also fixed a false-positive bUnit test (AlarmTriggerEditor_Expression_NoAnalysisKindInConfig_
SelectorDefaultsAdvisory) that passed because "Advisory" appeared anywhere in the HTML; now
asserts select.GetAttribute("value") == "Advisory". Added the missing equivalent test for
ScriptTriggerEditor (ScriptTriggerEditor_Expression_NoAnalysisKindInConfig_SelectorDefaultsAdvisory).
2026-06-18 10:54:42 -04:00
Joseph Doherty dcc6f623e2 feat(m9/T28b): trigger analysis-kind selector (UI) + --trigger-kind (CLI)
Surfaces the T28a backend "analysisKind" discriminator in both authoring
surfaces: an Advisory|Strict <select> (id="alarm-trigger-kind" /
"script-trigger-kind") added to the Expression fragment of
AlarmTriggerEditor and ScriptTriggerEditor, and a --trigger-kind option
on template alarm/script add+update in the CLI.

Key/value contract: "analysisKind":"Strict" when strict; key omitted for
Advisory — exactly as ValidationService.IsStrictAnalysis reads it.
Selector only shown for Expression triggers; non-Expression triggers do
not emit the key even if IsStrictAnalysisKind is set on the model.

Both projects build 0 warnings; 101 CentralUI Trigger tests + 33 CLI
Template tests pass.
2026-06-18 10:44:57 -04:00
Joseph Doherty efcdd18794 feat(m9): CLI cached-call retry/discard command group
Adds `cached-call retry` and `cached-call discard` subcommands that relay
to the existing Deployer-gated RetryParkedMessageCommand /
DiscardParkedMessageCommand via the central SiteCallAuditActor → site relay.
ManagementCommandRegistry already covered both types via reflection auto-discovery.
CommandTreeTests updated to include cached-call (group count 16 → 17).
2026-06-18 10:13:56 -04:00
Joseph Doherty 565d53d0fe feat(transport): import name-map plumbing via CLI + ManagementActor (M8 D3) 2026-06-18 07:08:33 -04:00
Joseph Doherty bdbf5cdab0 feat(transport): export site/instance selection end-to-end via CLI + ManagementActor (M8 B4) 2026-06-18 06:14:39 -04:00
Joseph Doherty 586d54359c feat(cli): instance import-overrides --file (T16) 2026-06-18 02:27:40 -04:00
Joseph Doherty 55630b48b6 feat(audit): M5.6 SourceNode sentinel backfill (purge-role) + CLI + runbook note (T5) 2026-06-16 22:02:21 -04:00
Joseph Doherty 0569c5ff23 feat(audit): M5.1 audit tree endpoint + CLI audit tree (T8)
Add GET /api/audit/tree endpoint that accepts executionId query param,
authenticates via HTTP Basic + LDAP (OperationalAudit permission), calls
IAuditLogRepository.GetExecutionTreeAsync, and returns a JSON array of
ExecutionTreeNode. Returns 400 for missing/invalid GUID, 401/403 as normal.

Add `scadabridge audit tree --execution-id <guid> [--format table|json]`
CLI subcommand in AuditCommands.Build(). Adds AuditTreeHelpers with:
  - BuildUrl: constructs the relative URL + query string
  - RunTreeAsync: calls the endpoint, dispatches to table or JSON renderer
  - WriteTable: indented ASCII tree (root → children, [*] marks queried node)
  - WriteJson: pretty-printed JSON array pass-through

Tests: 7 new ManagementService endpoint tests (valid id, empty, 400, 401,
403, Viewer allowed, wrong role), 18 new CLI tests (parse, render, HTTP
error codes, JSON output, multi-level indentation, queried-node marker).
2026-06-16 21:20:54 -04:00
Joseph Doherty 85db4571b2 feat(cli): --element-type and JSON --value for List attributes 2026-06-16 16:18:08 -04:00
Joseph Doherty 8219b8ee18 fix(auth): C2 review — not-found throws (no spurious audit) on update/delete/set-methods, reject empty methods (unusable-key/stealth-disable), richer set-methods response, token advisory to stderr 2026-06-02 04:21:28 -04:00
Joseph Doherty c899cb162c refactor: scrub residual ScadaLink refs → ScadaBridge (env vars, config keys, assembly name, SQL login)
Renames the 13 SCADALINK_* runtime env vars → SCADABRIDGE_*, the ScadaLink__
.NET config keys → ScadaBridge__, the stale ScadaLink.Host.exe assembly name
→ ZB.MOM.WW.ScadaBridge.Host.exe, the scadalink_app SQL login → scadabridge_app,
and residual identifiers/comments/docs. Migration records (prior rename
tooling/design, DB-rename helper, this scrub script) carved out.

Adds tools/scrub-scadalink-refs.sh.
2026-05-31 21:50:38 -04:00
Joseph Doherty a6dcbf62cd feat(cli): native-alarm-source commands (template add/list/remove + instance set/clear) 2026-05-31 02:30:05 -04:00