Commit Graph

105 Commits

Author SHA1 Message Date
Joseph Doherty b95ee5a658 feat(audit): archive retention and superseded stale-archive cleanup (ScadaBridge#109)
Owner decisions recorded: the archive destination stays a directory/UNC
(no object-storage client; share + permissions named at enablement),
parquet-tools-only reading (no re-import), unsigned like the chain (#107).

AuditLog:Archive:RetentionDays (default 0 = never delete; else must exceed
AuditLog:RetentionDays, counted from the archived month's end) and
DeleteSupersededStaleArchives (default true) run after every purge tick.
A retried purge's stale Archived archive is deleted only when a later,
verifying, Purged archive of the same partition provably holds every row:
same prior anchor and the stale chain-end hash at its chain-end sequence
(row or tombstone), every unchained row present by EventId; otherwise it is
reported KeptStale. A sole copy left at Archived is never touched. Deletion
removes only the archive name's own three files in its folder, manifest
first, never a manifest-supplied path. `audit archive cleanup [--dry-run]`
= CleanupAuditArchives (Administrator).

AuditLogArchiveCleanupTests (MSSQL) cover retried purge, kept stale copy,
sole copy, retention and the path guard; each safety rule mutation-checked.
Follow-ups filed: #113, #114, #115.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 23:33:04 -05:00
Joseph Doherty c4d7ec9b96 feat(audit): off-box chain-head export, verify-heads, maintenance login (ScadaBridge#107)
Item 1 (owner decision: no signing). AuditLog:HashChain:HeadExport (default
off): both centrals periodically read every month's chain head (LastSeq,
LastHash, purge anchor, chained rows, tombstones; one snapshot under the
chain lock SHARED) and append the changed heads to a sink the DB login cannot
write - File (append-only JSONL per node per day, local or UNC) or Syslog
(RFC 5424 over UDP / TCP octet counting). `audit verify-heads --path` reads
those records back (file or syslog-collector lines) and the new
VerifyAuditChainHeads command checks each against the database: Matches /
Extended / Purged, or Mismatch / Regressed / MissingHead / MissingLink /
CountMismatch (exit 3). AuditChainHeadStoreTests forges a chain with the
production hasher: verify-chain says Intact, verify-heads says Mismatch.

Item 3. AuditLog:MaintenanceConnectionString (${secret:}-capable, unset =
today's single login) runs the chain job, chain verify / head export, the
partition switch-out + archive and the per-channel purge on
AuditMaintenanceContext (keyed audit-maintenance repository/archive, purge
actor prefers them). AuditMaintenanceLoginTests runs the hardened setup on
SQL Server: writer (datareader/datawriter/audit_writer) ingests and is
refused chaining and deleting; the chainer+purger login does the full cycle.
Two-login grants documented in Component-AuditLog.md.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 23:16:05 -05:00
Joseph Doherty 63ec45e2e3 fix(flow-debugger): refuse method-entry Live runs for site-scoped users (#55)
Owner decision (b), 2026-11-09. An inbound API method has no site
affinity, so the F2 route-time scope only bounded routed SITE traffic;
the method's central-side effects (Database.*, ExternalSystem,
notifications) still ran for a scoped Deployer. FlowDebugRunner now
refuses FlowDebugRunMethodCommand for a site-scoped caller (site ids,
not Administrator) before the script executes, audits the refusal as a
failed InboundRequest under the operator's name, and raises
SiteScopeViolationException -> ManagementUnauthorized, so the Central
UI and the CLI (same /management command) share the gate.

The page disables the "Inbound API method" entry for scoped Deployers
with the reason (tooltip + inline note), starts them on script entry,
and ignores ?methodId= / a stored method choice. Replay and the
instance-scoped script / value-change Live runs are unchanged. The
route-time scope is kept as defence in depth.

Tests: scoped refused (in- and out-of-scope routing; script never runs,
refusal audited), unscoped/Administrator allowed, scoped Replay allowed,
scoped in-scope value-change Live still runs; bUnit for the disabled
entry, the drill-in, and the unscoped control. Mutation-proven (gate
and UI flag reverted -> 4 red).

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 22:22:44 -05:00
Joseph Doherty 3ff463b6fd docs(audit): Parquet cold archive shipped, default off (ScadaBridge#42)
Component-AuditLog Retention & Purge describes the archive: directory and
layout, the three files, consistency with the hash chain (SHARED read,
guarded switch, anchor = archived chain end), fail-closed purge, retention
interplay (per-channel overrides are not archived, their tombstones are),
verify/read, and the AuditLog:Archive configuration. alog.md §13.6/§15.2/§16,
CLAUDE.md, HighLevelReqs AL-12, Component-CLI, Component-ManagementService,
docs/components/{CLI,ManagementService}.md and the CLI README drop the
"deferred / 501" wording and document `audit archive list|verify` and the
parquet export.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 17:12:12 -05:00
Joseph Doherty d1fc336e18 feat(management): parquet audit export and audit archive list/verify (ScadaBridge#42)
GET /api/audit/export?format=parquet returned 501. It now streams the same 24
columns as csv/jsonl (AuditExportRow, CsvHeader order) as one Parquet file,
one 10,000-row row group at a time. Each row group is encoded into a small
non-seekable staging buffer and copied to the response asynchronously, because
Kestrel rejects the synchronous writes the Parquet writer issues.

New management commands, with CLI parity:
- ListAuditArchives -> `audit archive list` (AuditArchiveListResult: enabled
  flag + every manifest).
- VerifyAuditArchive(Name) -> `audit archive verify --name` (files, chain,
  purge anchor); exit 0 verified, 3 not verified, 1 error, 2 unauthorized.
Both carry verify-chain's gate: OperationalAudit roles (Administrator, Viewer),
site-scoped non-Administrator refused because an archive spans every site.
VerifyAuditArchive uses the long-running Ask timeout. Error codes
ARCHIVE_NOT_CONFIGURED, INVALID_ARGUMENT, NOT_FOUND.

RequiredRoleMatrixTests freezes both gates; ManagementSurfaceParityTests now
probes `audit archive list|verify` alongside `audit verify-chain`.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 17:12:12 -05:00
Joseph Doherty dd99a9f5a5 docs(audit): hash-chain tamper evidence shipped, default off (ScadaBridge#41)
Component-AuditLog Security & Tamper-Evidence describes the per-month chain,
the chaining job and its lock, both purge paths (checkpoint anchor,
tombstones), pre-existing rows, verification and what an unsigned chain does
not prove. alog.md §11.4/§16, CLAUDE.md, HighLevelReqs AL-11/12,
Component-CLI, Component-ManagementService, docs/components/CLI.md and the
CLI README drop the "no-op placeholder" wording.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 16:33:11 -05:00
Joseph Doherty 548bb9a725 feat(cli): working audit verify-chain via the VerifyAuditChain management command (ScadaBridge#41)
verify-chain was a no-op stub that printed "not enabled" and exited 0 without
contacting the server. It now sends VerifyAuditChain(Month) over
POST /management and maps the verdict to the exit code: 0 intact or not
chained, 3 broken (first break: partition, ChainSeq, EventId, kind), 1 error,
2 unauthorized. Table and JSON renderings.

ManagementActor: new HandleVerifyAuditChain arm, gated on the OperationalAudit
roles (Administrator, Viewer) and refusing a site-scoped caller because a
month's chain spans every site; INVALID_ARGUMENT for a bad month, CHAIN_BUSY
when the chain lock is held; long-running Ask timeout. Result enums serialise
as names.

Parity: RequiredRoleMatrixTests freezes the gate; ManagementSurfaceParityTests
now probes `audit verify-chain` (the one audit leaf that is a management
contract) instead of skipping the whole audit group.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 16:33:11 -05:00
Joseph Doherty 063fb21815 fix(native-alarms): a 4-column CSV import keeps stored MES overrides on the CLI path too (ScadaBridge#104)
The bulk SetInstanceNativeAlarmSourceOverridesCommand handler always wrote the
entry's IsMesReportableOverride, so a CLI import of a legacy 4-column CSV (no
MesReportable column) reset every imported source's MES override to inherit. The
Central UI avoided it only by looking up the stored flag client-side.

NativeAlarmSourceOverrideEntry gains an additive trailing KeepStoredMesReportable
flag: when set, the server leaves an existing row's MES override untouched (a new
row inherits) and ignores IsMesReportableOverride; unset keeps the old meaning
(null = clear to inherit), so older senders are unaffected. The CLI and the Central
UI now build their entries through one rule (ToOverrideEntry on each row type):
no MesReportable column => KeepStoredMesReportable. CLI README and
Component-CentralUI updated.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 16:04:28 -05:00
Joseph Doherty 62fe1e685d feat(native-alarms): optional MesReportable column in the native-source override CSV import (ScadaBridge#102 item 4)
CLI import no longer has to reset MES overrides to inherit; a legacy 4-column file
still parses and the Central UI import keeps the stored flag for it.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 15:33:31 -05:00
Joseph Doherty 3dd492a77f feat(inbound-api): surface method compile errors to operators — api-method validate/recompile + compile state on get (ScadaBridge#75)
The data plane answers a broken inbound method with the generic "Script
compilation failed for this method"; the Roslyn diagnostic reached only the
central log, and the known-bad cache could not be observed or reset per method.

- InboundScriptExecutor keeps the diagnostics with each known-bad record and
  exposes GetCompileState(name, currentScript): IsCompiled / IsKnownBad /
  LastCompileError, judged against the current script text (a record for older
  text is reported as not compiled yet).
- Management: ValidateApiMethodCommand (compile the stored script, return
  ApiMethodCompileResult, register nothing, cache untouched) and
  RecompileApiMethodCommand (CompileAndRegister without re-sending the entity,
  drops the node's ApiMethodCache row, audited "Recompile"). Both Designer, like
  UpdateApiMethod; dispatched through the shared ApiMethodCompileOperations.
  GetApiMethod keeps the entity fields top-level and adds isCompiled,
  isKnownBad, lastCompileError and compileStateNode. Compile state is per
  central node and every result names the node.
- CLI: api-method validate --id, api-method recompile --id.
- Central UI: the API-method edit page shows the saved script's compile state
  and last error, with Validate and Recompile actions (same operations, same
  Designer gate, fresh DI scope).
- RequiredRoleMatrixTests entries; ManagementSurfaceParityTests covers the new
  CLI leaves; ApiMethodCompileDiagnosticsTests + an executor CompileState test.
- Docs: CLI README, Component-CLI/ManagementService/InboundAPI, components/*.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 15:30:46 -05:00
Joseph Doherty f98369a786 feat(native-alarms): per-binding IsMesReportable on native alarm source bindings (ScadaBridge#59)
TemplateNativeAlarmSource.IsMesReportable + tri-state InstanceNativeAlarmSourceOverride
.IsMesReportableOverride flatten onto ResolvedNativeAlarmSource; NativeAlarmActor stamps every
mirrored condition and InstanceActor the quiet-binding placeholder, so ScriptAlarm,
Alarms.MesReportableAsync() and the #60 wire fields carry it. Additive to the 900-999 severity band.
Migration AddNativeAlarmSourceMesReportable. Management commands, CLI --mes-reportable, Central UI
(template dialog checkbox, instance override select), Transport DTOs/importer/diff and the revision
hash (true-only, so existing hashes are unchanged) carry it.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 15:09:54 -05:00
Joseph Doherty d50433410d feat(health): surface pending-updates counts on the Health dashboard and CLI (ScadaBridge#63)
Health page gains PendingUpdatesHealthTiles (stale instances / sites with stale
artifacts, Deployer+Administrator only, site-scope narrowed, shared cache). New
GetPendingUpdates management command (Deployer) and 'health pending-updates' CLI
leaf. Flatten/cache computation untouched; the command reads
IPendingUpdatesEvaluator directly because the UI cache is not reachable from
ManagementService.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 13:18:54 -05:00
Joseph Doherty 3808b49fc7 docs: record area-subtree deploy, Topology preload and the #38 rename trap in CLAUDE.md; fix stale deploy-site client-timeout line in the CLI README 2026-10-22 14:37:16 -04:00
Joseph Doherty 1d38fb6c90 fix(cli): deploy area names a malformed summary as an invalid area deployment summary (Gitea #40 review) 2026-10-22 14:27:26 -04:00
Joseph Doherty 74d5fba2b0 feat(cli): deploy area --id <areaId> [--force] bulk-deploys an area subtree with deploy site's deadline and exit-code contract (Gitea #40) 2026-10-22 14:27:26 -04:00
Joseph Doherty baab6ac78e fix(management): ListInstances pages, site-scopes and prefix-searches in SQL (review 2026-10-21 08/P2) 2026-10-22 09:32:14 -04:00
Joseph Doherty e968566e9d fix(cli): no redirects on the management client, race-safe token refresh, verbose Basic-fallback note (review 2026-10-21 05/U1, 05/P1 L1/L2/L3/L4)
- CreateDefaultHttpClient sets AllowAutoRedirect=false so a 307/308 cannot re-POST the
  /auth/token credential form elsewhere.
- A forced re-exchange after a 401 carries the refused token; inside the gate it reuses
  a token another caller already re-exchanged instead of binding again. The BasicOnly
  switch is written under the same gate.
- SCADABRIDGE_VERBOSE=1 prints a stderr note when a fresh token is refused and the CLI
  falls back to Basic.
- debug stream's AccessTokenProvider retries the exchange once and documents the
  null-token degradation (the hub refuses, automatic reconnect retries).
- Tests: concurrent refusal reuses the peer refresh (2 exchanges, not 3); real-socket
  redirect is not followed.
2026-10-22 08:25:35 -04:00
Joseph Doherty 1b3928a0a2 fix(security): finish the bearer-token design and cache role mappings (review 2026-10-21 05/U1, 05/P1)
- ManagementAuthenticator accepts Authorization: Bearer <jwt> alongside Basic on
  /management, the audit REST endpoints and the debug-stream hub. Tokens are
  validated locally (HS256 only, alg=none rejected, issuer/audience, lifetime with
  30 s skew); roles and site scope inside a valid token are authoritative for its
  lifetime, so a bearer call costs no LDAP bind and no role re-map.
- /auth/token returns expires_in + management_bearer:true, carries username, display
  name, roles and permitted sites, and answers 404 under DisableLogin.
- CLI exchanges credentials once per process (in-memory CliTokenStore), sends Bearer,
  re-exchanges once on 401 then falls back to Basic; older/login-disabled servers are
  Basic-only; bad credentials are not retried with Basic. Debug stream uses the token
  via AccessTokenProvider. SCADABRIDGE_AUTH_MODE=basic opts out.
- StartupValidator fails a Central boot on a JwtSigningKey under 32 UTF-8 bytes;
  fixed the false 'fail-fast at construction' comment. Removed the dead
  RefreshToken/RecordActivity/ShouldRefresh/IsIdleTimedOut members and the inert
  JwtRefreshThresholdMinutes; JwtExpiryMinutes now documented as the bearer lifetime
  (validated > 0).
- RoleMapper reads through a process-wide RoleMappingCache (RoleMappingCacheSeconds,
  default 30 s): one refresh loads the table plus every Deployer mapping's scope
  rules, Invalidate() drops it (generation-guarded against in-flight refreshes).
- Component-Security.md / Component-CLI.md / CLI README describe the real model
  (cookie claims ticket, bearer for CLI) instead of a cookie-embedded JWT.
2026-10-22 08:25:35 -04:00
Joseph Doherty 75149e83b0 fix(deps): clear new NuGet audit advisories (Pkcs, Cryptography.Xml, SignalR.Common)
Targeted transitive pins per the house pattern, no family-wide bump, no suppression:
Cryptography.Xml 10.0.10->10.0.13; Pkcs pinned 10.0.13 in ConfigurationDatabase,
ExternalSystemGateway, NotificationService, Playwright tests; SignalR.Common 10.0.13 in CLI.
2026-10-22 06:15:25 -04:00
Joseph Doherty 0ffd4bdbea docs(src): fix CommentChecker findings across src/ — 1,032 → 5 (all false positives)
Comment/XML-doc changes only, no code logic:
- strip project-tracking references (WP*, P*-*, row-NN, #NN, Task N) from
  comments, keeping the explanatory rationale
- add missing summaries, <param>, <returns>, <typeparam> and <inheritdoc/>
- replace misused <inheritdoc/> with real docs; where <inheritdoc/> was
  combined with useful remarks, move the rationale into // comments
- drop stale replicationActor <param> on DeploymentManagerActor ctor

Remaining 5 are false positives (CWE-1236 x2, failures-1, year-0001, M365).
Build 0 errors / 0 warnings.
2026-10-07 14:12:25 -04:00
Joseph Doherty ef03a0dca7 fix(cli): artifact-deploy commands exit 1 when any site failed
site deploy-artifacts and deploy artifacts rendered the per-site summary and
always exited 0, so a script could not tell a failed site (unreachable,
missing or unreadable secret, partial apply) from a clean deploy. Both now
route through the failureCount rule deploy site already used.
2026-09-28 21:23:52 -04:00
Joseph Doherty 6dc784bca4 docs(cli): site list omits the artifact-revision fields until the first deploy (not null)
Found by docker/verify-pending-updates.sh step A: the management API
serializer uses WhenWritingNull, so a never-deployed site has no such keys.

Claude-Session: https://claude.ai/code/session_01Y5ef34iUUMjmwnMpL9xgmB
2026-09-14 14:13:01 -04:00
Joseph Doherty d8979664f1 docs: site artifact revision, pending-updates indicator, rail scroll pane
Claude-Session: https://claude.ai/code/session_01Y5ef34iUUMjmwnMpL9xgmB
2026-09-14 14:01:07 -04:00
Joseph Doherty 6ac95ea888 fix(template-engine): carry IsMesReportable onto derived-template placeholder alarms and through descendant reconcile; final-review doc fixes
Claude-Session: https://claude.ai/code/session_01Y5ef34iUUMjmwnMpL9xgmB
2026-09-11 10:45:09 -04:00
Joseph Doherty f1a7c34b69 feat(cli): --mes-reportable on template alarm add/update and instance alarm-override set
Claude-Session: https://claude.ai/code/session_01Y5ef34iUUMjmwnMpL9xgmB
2026-09-11 10:12:47 -04:00
Joseph Doherty 2b9a7c3ca0 Synchronize CLI documentation with management command parity 2026-09-10 01:20:12 -04:00
Joseph Doherty 2750bd2809 Complete management API and CLI command parity 2026-09-10 00:51:45 -04:00
Joseph Doherty 9ee402bbcd Resolve all 46 low-severity architecture review findings 2026-09-09 22:56:36 -04:00
Joseph Doherty 9a87d7a328 Resolve all 33 medium-severity architecture review findings
Second slice of the 2026-09-09 whole-system review remediation
(docs/plans/2026-09-09-arch-review.md §10), on top of the High slice 146a1323.

Runtime/devices: SR-2 alarm on-trigger integration scope; SR-3 correlated NativeAlarmActor
subscribe retry; SR-4 immutable audited DB target; SR-5 ordered InstanceActor stop/flush;
SR-12 rehydration fence; SR-13 S&F gate recheck per delivery; DC-4 no OPC UA security
downgrade; DC-5 alarm snapshot for every new subscriber; ORCH-1 sync-dispose joins the pump.

Management/UI: MG-2 site scope on audit tree + Flow Debugger Replay; MG-3 stale-probe memo
removed; MG-4 server-owned bulk-deploy deadline; UI-5 sandbox runs in a killable child
process (new CentralUI.SandboxWorker project); UI-14 bounded sandbox output/IPC.

Edge/audit: EI-1 relay audit ownership; EI-2 SkipBodyCapture suppresses error text; EI-3/EI-11
error-field redaction + caps; EI-4 single relay deadline, pre-execution-only retry; EI-5
authenticated relay provenance; EI-10 body cap before audit read.

Persistence/contracts: DL-1 global Instance.UniqueName unique index + conflict translation;
DL-2 DataConnection protocol config encrypted at rest (migration
20260909203046_EnforceGlobalInstanceNamesAndProtectProtocolConfiguration — coordinated
central upgrade, old binaries cannot read protected rows); DL-3 bounded Replay paging;
DL-6 cross-site Overwrite of a deployed instance rejected; XC-4 proto descriptor contract test.

Config/hygiene: XC-1 central listener isolation by local port; XC-2/XC-3 inert SMTP + cron
keys removed from rig settings; CM-2 env2 bootstrap guard enabled; DC-6 deploy site exit code;
DC-7 notifier retries only explicit standby rejection; XC-5 ClusterClient purged from docs.

Gates recorded in the report: build 0/0, 9 397 passed / 0 failed, rig redeployed on all
eight nodes with the migration applied. 46 Low findings remain open.
2026-09-09 22:05:30 -04:00
Joseph Doherty 97f01ae98e fix(flow-debugger): final-review remediation — 2000-row trace pull, shared ExceptionHeader parser, surrogate-safe clip, wording, alarm source, register renumber (I1/I2/M1-M8) 2026-09-09 13:30:48 -04:00
Joseph Doherty c1b1702397 fix(cli): single-key note rows render as raw indented lines under their step (Task 14 review) 2026-09-09 12:15:05 -04:00
Joseph Doherty c6529fa996 feat(cli): flow-debug logs column and --show-logs 2026-09-09 12:03:42 -04:00
Joseph Doherty 061f2b722b docs: run-script help names the server ceiling; Open in Debug View applies to script entries
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
2026-09-08 01:39:46 -04:00
Joseph Doherty 45c6b27d99 fix(cli): flow-debug help text names the script entry; replay role note; run-script guard tests
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
2026-09-08 01:17:32 -04:00
Joseph Doherty 067a858ce9 feat(cli): flow-debug run-script — run a site script by name through the flow debugger
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
2026-09-08 01:13:15 -04:00
Joseph Doherty f17bae093a fix(flow-debugger): review follow-ups — redact runner-synthesized rows, spawner rule, budget doc, CLI ceiling note (arch-review remediation)
Code review of the merged arch-review remediation (Tasks 1-13) found four items.

1. FlowDebugRunner returned its OWN synthesized rows un-redacted. RunMethodAsync's
   inbound row and RunValueChangeAsync's root row were handed to WriteCentralAsync
   (whose writer redacts a private copy) and then put RAW into the returned Events —
   so a Live run's caller saw the unfiltered row while Replay of the same execution
   showed the redacted one, defeating F1's "Live shows what Replay reads back". Both
   methods now resolve IAuditRedactor once per run and place the filtered copy into
   the result (including the early-return path on a failed attribute write); the raw
   row still reaches the writer, which redacts itself.

2. FlowGraphBuilder's spawner search degenerated to "the last step of the parent".
   Since executions are ordered as contiguous blocks, position[p] < childPosition
   holds for EVERY parent step, so a run shaped ScriptRun -> Write(A) -> Wait ->
   Write(B) whose Write(A) fired the cascade drew the Spawn edge from Write(B). New
   rule, in order: (a) the last non-async, non-Response parent step occurring at or
   before the child's first step — parent and child of a trigger cascade share the
   site clock, so this picks the write that actually fired it; (b) the position rule
   as fallback, which is the mixed-clock case F5 was written for (a child clock
   behind its parent's, where no parent step satisfies (a)); (c) parentChain[0].
   The class remark's stale "Replay order is unchanged" claim is replaced with what
   actually happens to a central Database.* call made after Route.To().Call.

3. FlowTraceCollector's byte budget counts UTF-16 chars of DetailsJson only, so
   "well under the 4 MB gRPC default" overstated it. Documented honestly: ~1 MB of
   UTF-8 for ASCII payloads, ~3 MB worst case for non-ASCII, both under 4 MB before
   framing — and raise it only against a real serialized-size measurement.

4. CLI --timeout-seconds help + README row now say the management endpoint caps
   flow-debug run-method at ManagementService:LongRunningCommandTimeout (default
   5 min), so a larger client value still 504s at the server's ceiling.

Deferred-work register rows 112 (site-side inline trace truncation is invisible to
the operator — needs a proto field) and 113 (the F2 method-entry site scope gates
routed traffic only; ApiMethods have no site affinity to scope against) record what
this review deliberately did not fix.

Tests: FlowGraphBuilderTests gains the same-clock Write(A)/Wait/Write(B) cascade
(red before the rule change, and the clock-behind test still passes on rule (b));
FlowDebugRunnerTests extends the redactor test to the inbound row and adds two
value-change cases (settled poll, failed write). Commons 793, ManagementService 615,
CentralUI 1264, CLI 462 — all green.

Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
2026-09-06 14:44:34 -04:00
Joseph Doherty e15aa0e430 docs: Flow Debugger design reconciled with the implementation (arch-review F12, F13)
The Flow Debugger design doc still described the pre-remediation shape: a
debug-stream subscription for the value-change entry, Deployer everywhere, an
unredacted inline trace copy and "~7 audit rows per call". Reconciled with what
shipped on this branch (Tasks 1-12):

- design D6 / §5.4 / §9 / §10: RequireFlowDebugger page policy with Deployer-only
  execution (F3), the PullTraceEvents polling path with AttributeValueUpdate
  fields 7/8 recorded as RESERVED and unconsumed (F12), the redacting + byte-
  capped shared FlowTraceCollector (F1), route-time site scope (F2), the derived
  client waits (F4), the 7-10 row estimate, the doubled central audit write per
  routed call, and the inbound-row-after-the-run ordering.
- sitestream.proto: fields 7/8 marked reserved-but-populated, numbers not to be
  reused (comment only; no regeneration needed).
- Component-InboundAPI: SkipBodyCapture on the RouteCall row (F11),
  SchemaRefResolver's public visibility, WithSiteScope / SITE_SCOPE_DENIED (F2),
  Replay's wider role set (F3).
- Component-CentralUI: page policy vs Deployer-only controls (F3), route-time
  site scope (F2), client wait derivation (F4).
- "7-10 audit rows per call (3 of them trace-only)" everywhere the estimate is
  quoted: the method form, both CLI --trace-all-calls help strings, the CLI
  README, the ApiMethod entity doc, the manual migration script and the two
  Flow Debugger plans.
- arch-review doc: new §7 Resolution listing F1-F13 with the fixing commit, plus
  the tracking-store test race fixed alongside them (86451382).

Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
2026-09-06 14:35:10 -04:00
Joseph Doherty 97b0a4ba16 fix(central-ui,cli): Flow Debugger client waits derive from the method timeout / wait window instead of a fixed 2 minutes (arch-review F4)
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
2026-09-06 14:13:02 -04:00
Joseph Doherty 7e3e809a70 feat(flow-debugger): FlowStep carries the audit CorrelationId; CLI table adds a correlation column
Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
2026-09-06 10:58:50 -04:00
Joseph Doherty 3c9f9da818 feat(flow-debugger): outbound calls on the External lane — CLI projection, specs, rig demo + verification step I
CLI flow-debug table now projects through FlowGraphBuilder (one row per step,
outbound lifecycles folded, new 'lifecycle' column; per-event fallback kept).
Specs: Component-CentralUI (four lanes, node anatomy, pending refresh),
Component-AuditLog (central method-script Database.* now audited; replay reads
the three outbound channels), Component-InboundAPI (DbOutbound row per helper
call), Component-ScriptAnalysis (REQ-SA-7 kinds + Async edges), design doc
v1.1 status line, CLAUDE.md bullet, deferred-work register row 111 (Retry/
Discard + ?correlationId= deep links deferred). Rig: seed-flow-outbound-demo.sh
(RigRest/RigDb/FlowDemo + OutboundDemo script + FlowOutboundDemo method/key)
and verify-delmia-notifier.sh step I (live rows inline, replay until terminal).
Rig verification result follows in its own commit.

Claude-Session: https://claude.ai/code/session_01NcMG6xXdsJbmD2wYAgkK29
2026-09-06 02:10:50 -04:00
Joseph Doherty 0fb7e16ced feat(cli): flow-debug run-method|run-value-change|replay
New top-level command group for the Flow Debugger (design 2026-09-05 §7.1). run-method /
run-value-change post the FlowDebugRun* management commands with a 2-minute client wait;
replay posts FlowDebugReplayCommand. A table projection unwraps the FlowDebugRunResult
envelope to one row per step (occurred | kind | lane | target | durationMs | status |
error) using the Commons FlowStepProjector's kind/lane resolution, with a trailing note
row for an incomplete run (error code verbatim, e.g. STANDBY_NODE) or a truncated event
list. --params must be a JSON object (INVALID_PARAMS otherwise). README section added;
command-tree test bumped to 18 groups.

Claude-Session: https://claude.ai/code/session_01GSLkkHKz9JrAtmJjmQK3cP
2026-09-06 00:11:42 -04:00
Joseph Doherty 23157d2bcf feat(inbound-api): ApiMethod.TraceAllCalls (entity, migration, mgmt command, CLI --trace-all-calls)
Flow Debugger (design 2026-09-05 §5.2): bit NOT NULL DEFAULT 0 column + EF migration
20260906034113 with its docs/plans/sql production twin. CreateApiMethodCommand gains
TraceAllCalls=false; UpdateApiMethodCommand gains bool? TraceAllCalls (null = unchanged so an
older CLI never resets it). Management result echoes the flag; CLI create/update expose
--trace-all-calls; README carries the ~7-rows-per-call warning.

Claude-Session: https://claude.ai/code/session_01GSLkkHKz9JrAtmJjmQK3cP
2026-09-05 23:42:52 -04:00
Joseph Doherty a26edffd4c docs: create-time connection bindings (Central UI, DeploymentManager preview flatten, CLI cross-ref)
Claude-Session: https://claude.ai/code/session_01G2GHPnbbKGvyyfbr2XBRVH
2026-08-21 06:14:55 -04:00
Joseph Doherty 5e04e81ba0 docs: grouped connection bindings — Central UI Configure page, ResolvedAttribute.DefaultDataSourceReference, CLI instance bind-module; correct the stale 'set-bindings REPLACES all' claim (server upserts) in README + help text 2026-08-20 16:01:33 -04:00
Joseph Doherty 02a374259b feat(cli): instance bind-module — rebind one composed module's members to a new object path via GetInstance → SetConnectionBindings upsert; --dry-run 2026-08-20 15:55:09 -04:00
Joseph Doherty b34ec10f8b fix(deploymentmanager): bulk site deploy skips up-to-date instances and un-poisons the shared DbContext (arch-review Low P7-5 + P7-6)
P7-6 (behaviour fix, plan decision 1): DeploySiteAsync redeployed EVERY
deployable instance unconditionally. A redeploy is not idempotent at the site —
it resets static attribute overrides and clears native alarm state by design — so
one site-wide push destroyed live operational state on every unchanged instance
in the batch, which is normally most of them. Bulk deploy now pre-filters with
IStaleInstanceProbe (the same seam the bundle importer and Deployments page use;
its memo timing was fixed by the Medium wave's P7-4): an instance whose freshly
computed revision hash still matches its DeployedConfigSnapshot is reported in the
existing additive SkippedInstances list with reason "up-to-date" and never
flattened, staged or sent. The filter is deliberately conservative — every
uncertainty deploys: only an Enabled instance qualifies, a missing snapshot
disqualifies, a prior record that ShouldQuerySiteBeforeRedeploy flags goes through
prepare so the reconcile that HEALS it still runs, and a null/faulting probe means
"indeterminate". New additive MgmtDeploySiteCommand.Force (default false = the new
filtered behaviour) plus CLI `deploy site --force` restores the unconditional push
for the case central cannot see (restored volume, rebuilt node). Composes with the
Medium wave's P7-2: Disabled wins — force suppresses the up-to-date filter only,
never the Disabled skip.

P7-5 (verified, then fixed): the cascade REPRODUCES. All three bulk phases share
one scoped DbContext, and a failed SaveChanges does not untrack the offending
entity — EF re-attempts it on the next save, including MarkDeploymentFailedAsync's
cleanup write. Measured on a real SQLite-backed ScadaBridgeDbContext: a permanent
save failure on instance 1's post-success side effects failed instances 2, 3 and 4
and left all three deployment records stranded InProgress. MarkDeploymentFailedAsync
now detaches the pending changes (new IDeploymentManagerRepository.DetachPendingChanges)
when the cleanup save fails and retries the Failed write once on the clean tracker,
so later instances save cleanly and the record reaches a terminal status. The
DeployFailed audit row moved outside the status-write try (best-effort, must not be
lost because the status write failed).

Tests: 5 new DeploySiteAsyncTests (skip + reason + no site round-trip; --force
deploys; Disabled skipped under both force values; the three not-enough-to-skip
guards), 2 new IntegrationTests over a real EF context (the poisoned-context
cascade, and the narrow EF tracking fact the fix rests on), CLI --force flag
parse test, and a MgmtDeploySite wire-default test. Proof of failure: both P7-6
behaviour tests and both P7-5 assertions fail against the unfixed code.
2026-08-19 16:41:04 -04:00
Joseph Doherty 9a3f006721 fix(management)+fix(cli): neutralize CSV formula injection + align export columns; deadline + Ctrl-C on audit export streaming (arch-review 2026-08-19 P6-7 + P6-8)
P6-7 (CWE-1236): AuditEndpoints CSV export quoted cells on comma/quote/CR/LF
but never neutralized a leading =, +, -, @, TAB, or CR — inbound-API-caller
text landing in Target/ErrorMessage/RequestSummary/ResponseSummary/Extra
could execute as a formula/DDE trigger when opened in Excel. AppendField now
prefixes a literal single quote on free-text columns only (never the
GUID/enum/numeric/date/bool columns the exporter formats itself), applied
before RFC 4180 quoting. The CSV was also missing ExecutionId/
ParentExecutionId/SourceNode that the JSONL export already carried; both are
now 24 columns in the same AuditExportRow order. A second, independently
maintained CSV writer (CentralUI's AuditLogExportService, feeding the Audit
Log page's bulk-export button) had the identical gap and got the identical
fix + column alignment.

P6-8: `audit export` was the one CLI HTTP path with no deadline at all — a
residual gap from the ManagementHttpClient timeout-honesty fix, which made
HttpClient.Timeout infinite and the per-call token the single deadline
everywhere except this streamed GET. RunExportAsync now takes a headers-
received deadline (CancelAfter, reusing ManagementHttpClient.
DefaultConnectTimeout) around the initial GetAsync, an inactivity deadline
around the streaming copy (rearmed on every successful read, so large exports
are bounded by silence, not total size), and an external CancellationToken
wired to Console.CancelKeyPress in AuditCommands so Ctrl-C cancels promptly.
A stalled/faulted stream now produces a friendly error noting the partial
output file instead of hanging or dumping a stack trace.
2026-08-19 15:01:45 -04:00
Joseph Doherty f33f25a10b fix(auth+cli): fail-closed site-scope gate (row 98); case-insensitive audit filter values with a strict server-side check (row 99)
Register row 98 — the site-scope authorization gate was FAIL-OPEN on an
unresolvable target. The resolve-then-check shape was written as
`if (target != null) EnforceSiteScope(...)`, so a null resolve skipped the
check entirely and a site-scoped caller passed the gate for any id in the
system. Authorization now fails CLOSED: a target that cannot be PROVEN to sit
inside the caller's scope is treated as outside it.

The defect turned out to have THREE more instances than the register recorded.
Besides `EnforceSiteScopeForInstance` and `EnforceSiteScopeForIdentifier`, the
same inline shape sat in `HandleGetInstance`, `HandleGetSite` and
`HandleGetDataConnection` — read handlers that resolve their own target and so
never call the helpers. Those were the sharper half of the bug: they answered
success-with-null for a nonexistent id and Unauthorized for an out-of-scope
one, making the pair an id-enumeration oracle over the whole fleet. All five
now route through one shared `DenyUnresolvedTargetIfScoped` arm.

Two properties are load-bearing and pinned by tests:
  - System-wide callers (Administrator, or a Deployer with no site grants) are
    untouched — they return at the guards above the lookup and keep the
    handler's ordinary "not found" / null answer.
  - The denial is indistinguishable from an out-of-scope denial: same
    ManagementUnauthorized shape, and a message that never states whether the
    target exists.

Register row 99 — CLI `audit query --status delivered` failed client-side while
the server would have accepted it: `AcceptOnlyFromAmong` matches ordinally
against the capitalised literals. Same treatment row 88 gave `deploy status` —
the client-side gate is removed, the valid set moves into the option help text,
and the server becomes the single authoritative check.

That required ADDING the server check, not just relying on one: the shared
audit filter parser was LAX, silently dropping an unrecognised value and
leaving the dimension unconstrained — so removing the CLI gate alone would have
turned a loud client-side failure into a plausible, complete-looking page that
did not answer the question asked. New `ParseEnumListStrict` (case-insensitive,
rejects ordinals — `Enum.TryParse` accepts "3", and a filter is a name-based
contract) backs the ManagementService /api/audit/query + /api/audit/export
endpoints, which now return 400 BAD_REQUEST naming the valid set.

Other AcceptOnlyFromAmong sites surveyed in the audit group: `--channel`,
`--kind`, `--status` on BOTH `query` and `export` (6 uses) plus `export
--format` (rejected `--format CSV` though the server lowercases it) — all seven
fixed here. Deliberately left alone: `template folder reorder --direction`
(pure client-side literal, no server-side parse to defer to) and the global
`--format json|table` (CLI-local output selection, never sent to a server).

The CentralUI AuditLogPage drill-in parser and the CentralUI export endpoint it
feeds deliberately KEEP the lax parse — they read a query string a human may
have bookmarked, where a stale fragment must still render the page. Both
contracts now live side by side in AuditQueryParamParsers and are documented as
non-reconcilable.

Docs: Component-ManagementService.md gains a "Site-scope gate: fail-closed on
an unresolvable target" section and a strict-filter note on the audit
endpoints; Component-Security.md records the fail-closed shape under Deployer
site scoping; the CLI README marks the audit filter values case-insensitive and
states the no-silent-drop / no-ordinals contract.

Tests: 5 new ManagementActorTests fail before the row-98 fix (verified by
reverting ManagementActor.cs) and cover scoped+nonexistent -> denied,
system-wide+nonexistent -> unchanged (both Administrator and no-grant Deployer),
scoped+in-scope -> allowed, and the indistinguishability property. 21 CLI tests
fail before the row-99 fix. Four pre-existing tests that pinned the removed
client-side gate were rewritten to pin the new contract.
2026-08-17 06:05:29 -04:00
Joseph Doherty 9f67147e7b refactor(deployments): one paged read of DeploymentRecords + close its three drifts
Deferred-work register rows 37, 86, 87, 88 — the structural duplication and the
three behavioral divergences it had accumulated, fixed together.

Row 37 (structural). `IDeploymentManagerRepository.QueryDeploymentSummariesAsync`
is DELETED. It and `QueryDeploymentListPageAsync` were two paged reads of the same
insert-only table with different contracts — raw offset vs page number, no total vs
total, no instance join vs joined, instance-id-list scoping vs site-id scoping. The
CLI/HTTP `QueryDeployments` handler now goes through `QueryDeploymentListPageAsync`
with a `DeploymentListFilter`, exactly like the Central UI page. The
`DeploymentRecordSummary` projection went with it (`DeploymentListRow` is the single
row shape); its `MaxPageSize` const moved to `DeploymentListPage`, which also gains
`DefaultPageSize`.

Row 86 (Medium, authorization — the headline). `SiteScopeService.ResolveAsync` was
claims-only, so an Administrator carrying `SiteId` claims saw the whole fleet through
the CLI (`ManagementActor.EnforceSiteScope` has always exempted Administrator) and
only their granted sites on the Deployments page. One user, one dataset, two answers.
Administrator = system-wide is the intended semantic on both sides; the UI was the
defect. The role claim is read explicitly with an OrdinalIgnoreCase comparison rather
than via `IsInRole`, which compares claim VALUES ordinally and only resolves when the
identity carries the matching roleType — the two sides must not disagree on a
differently-cased role name. An Administrator's site claims are not collected at all,
so the invariant "system-wide => empty permitted-id set" holds for every caller.
Designer is NOT exempt: the class doc claimed "any Admin/Design user" is system-wide,
which was true of neither side, so the doc was corrected rather than the bypass
widened.

Row 87 (Low). The scoped branch used to resolve the site grant into an instance-id
list by loading EVERY instance in the system, non-transactionally, on every scoped
query; the grant now rides as `SiteIdScope` and is resolved by the query's own join.
An unparseable permitted-site id is DROPPED, never ignored — dropping can only narrow
a grant, discarding it would widen it to fleet-wide. And `PageSize <= 0` no longer
means UNLIMITED down at the repository (`--page-size 0` materialized the whole
retention window): it means "unspecified" and takes the CLI's own default of 50,
capped at 1000. Bounding without telling would be silent truncation, so the handler
now returns a `DeploymentQueryResult` envelope carrying `totalCount` plus the page
coordinates actually applied — with a CLI table projection (mirroring row 57's
`AuditConfigTableProjection`) that unwraps it and appends a "showing N of M" note.

Row 88 (Low, UX). An unrecognised `--status` fell back to "no filter", so a typo like
`--status Sucess` returned every status — a complete-looking answer to a question the
caller never asked. It is now a `ManagementError` naming the valid values. Parsing
stays case-insensitive; a numeric string is rejected too, since `Enum.TryParse`
accepts "7" and yields an undefined value (guarded with `Enum.IsDefined`). Loud
failure over silent full results, per row 46.

Tests: 7 new `SiteScopeServiceTests` pinning Administrator = system-wide (verified
failing 6/7 against the pre-fix service, the Designer pin passing both ways); the
ManagementActor deployment suite migrated onto the consolidated query with new pins
for the status error, case-insensitive parse, numeric rejection, page-size default/
cap/floor, empty-scope-on-garbage-grant, and "reads no instances at all"; a repository
test for site-scope+status composing over a bounded page; and 6 CLI tests for the new
table projection.
2026-08-17 05:15:18 -04:00
Joseph Doherty 4ae57bdb6b docs(cli): row 76 — audit-config query JSON sample shows afterStateJson + bundleImportId
The `entries` array is a serialized `AuditLogEntry`, so both fields are always
on the wire; the README sample showed neither, which read as if the payload
were the seven columns the table projection happens to keep. Row 57 added that
projection and the table section already names both omitted fields — the JSON
sample above it was the stale half.

Added both to the sample and documented their nullability, which is the part a
script author actually needs: `afterStateJson` is the post-change snapshot
embedded as a string and is null for deletes (no after-state), `bundleImportId`
is the import session id when the change came from `transport import` and null
for a direct change. Also grew the sample from one entry to two so it now
mirrors the table-output sample directly below it row for row — the same 412
Template/Created and 413 ExternalSystem/Deleted pair — which is what makes the
"the table drops these two fields" sentence checkable against the JSON above it.

Docs only.
2026-08-15 21:17:34 -04:00