Commit Graph

1990 Commits

Author SHA1 Message Date
Joseph Doherty c4d7ec9b96 feat(audit): off-box chain-head export, verify-heads, maintenance login (ScadaBridge#107)
Item 1 (owner decision: no signing). AuditLog:HashChain:HeadExport (default
off): both centrals periodically read every month's chain head (LastSeq,
LastHash, purge anchor, chained rows, tombstones; one snapshot under the
chain lock SHARED) and append the changed heads to a sink the DB login cannot
write - File (append-only JSONL per node per day, local or UNC) or Syslog
(RFC 5424 over UDP / TCP octet counting). `audit verify-heads --path` reads
those records back (file or syslog-collector lines) and the new
VerifyAuditChainHeads command checks each against the database: Matches /
Extended / Purged, or Mismatch / Regressed / MissingHead / MissingLink /
CountMismatch (exit 3). AuditChainHeadStoreTests forges a chain with the
production hasher: verify-chain says Intact, verify-heads says Mismatch.

Item 3. AuditLog:MaintenanceConnectionString (${secret:}-capable, unset =
today's single login) runs the chain job, chain verify / head export, the
partition switch-out + archive and the per-channel purge on
AuditMaintenanceContext (keyed audit-maintenance repository/archive, purge
actor prefers them). AuditMaintenanceLoginTests runs the hardened setup on
SQL Server: writer (datareader/datawriter/audit_writer) ingests and is
refused chaining and deleting; the chainer+purger login does the full cycle.
Two-login grants documented in Component-AuditLog.md.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 23:16:05 -05:00
Joseph Doherty 5e7d5389d4 feat(audit): flip AuditLog clustered key to (OccurredAtUtc, EventId) (ScadaBridge#111)
Variant B2 of the #78 benchmark: migration FlipAuditLogClusteredKey drops the
twelve non-clustered indexes, re-adds PK_AuditLog as (OccurredAtUtc, EventId)
on ps_AuditLog_Month, adds the aligned unique UX_AuditLog_EventId_Occurred
(EventId, OccurredAtUtc) for the idempotency probe, and rebuilds the twelve
indexes unchanged. Guarded (re-runnable) Up, inverse-rebuild Down; production
scripts docs/plans/sql/FlipAuditLogClusteredKey{,.rollback}.sql.

The partition-switch purge staging table now reads the live clustered key
order from sys.index_columns, so it switches either shape (old binaries
hard-code the old order: ship with the binaries, both centrals stopped).

Evidence: 300k rows, page 1 90,532 -> 34 logical reads (ordered backward
clustered scan under Top), keyset page 90,532 -> 46, probe 14 -> 14;
FlipAuditLogClusteredKeyMigrationTests pins indexes, grants, Down/Up, the
staging key (mutation: hard-coded old key fails SWITCH) and the read ratio.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 22:52:16 -05:00
Joseph Doherty 559e02f5c8 feat(transport): production-boundary signing advisory, secret-key proof, runbook (#110)
Owner decisions on #110: one signing key per environment; TrustedKeys stay hand-edited;
Signing:ProductionBoundary (new, default off) logs a startup warning and reports a Degraded
bundle-signing-policy /health/ready entry while RequireSignedBundles is off; the private key
is read via ${secret:} (proved end to end, never logged); cluster pull decided not to be
built; differential deletions stay deferred. Adds docs/operations/bundle-signing-runbook.md.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 22:50:18 -05:00
Joseph Doherty a4806576af feat(pending-updates): short-circuit an unchanged fleet on a DB rowversion watermark (#62)
Owner decision (b), 2026-11-09. PendingUpdatesEvaluator re-flattened every
deployed instance on each cache-TTL expiry even when nothing had changed.

- Migration AddConfigGraphRowVersions: a non-concurrency rowversion column
  + index on the 18 configuration tables the evaluation reads (Templates +
  5 member tables, Instances + 4 override/binding tables, DataConnections,
  DeployedConfigSnapshots, Sites, SharedScripts, ExternalSystemDefinitions,
  ExternalSystemMethods, DatabaseConnectionDefinitions). SQL Server only in
  the model; SQLite test contexts are untouched. Script:
  docs/plans/sql/AddConfigGraphRowVersions.sql. Joins the coordinated
  central-pair migration window (#107 item 4).
- IConfigGraphWatermarkReader / ConfigGraphWatermarkReader: ONE query of
  MAX(RowVersion) + COUNT_BIG(*) per table (count catches deletes, which
  raise no rowversion). rowversion is engine-bumped for any writer, so a
  peer central node's edit moves it — not the per-process watermark MG-3
  removed.
- The secret store (outside the DB) gets its own stamp: SHA-256 over
  ListAsync(includeDeleted: true) metadata.
- Evaluator re-serves the last CLEAN snapshot while watermark + secret
  stamp are unchanged; every PendingUpdatesFullEvaluationEvery'th (default
  10, validated >= 1, 1 = off) is full regardless — the safety net for a
  same-KEK re-wrap or an input table missing from the set. Always full when
  the token is unavailable/unreadable, after an evaluation that could not
  compute a site's artifact hash, on Re-check, and after a deploy completion.
  It only re-serves a real earlier answer, never manufactures one.

Tests: SQL Server (real migrated DB) — token stable without writes, moves
for a peer-connection INSERT/UPDATE and a non-max DELETE, moves for a local
EF write, a detached Update still saves (not a concurrency token), and a
guard that every table the evaluator's repository reads touch is covered.
Evaluator — reuse without flatten (re-served answer still shows the stale
instance), watermark move, 4 secret changes, every-Nth, 5 no-token modes,
unclean evaluation not reused, cache Re-check and deploy completion force
full. Mutation-proven: token compare, clean check, cache force flag, row
count, and an uncovered read each turn tests red.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 22:38:55 -05:00
Joseph Doherty 63ec45e2e3 fix(flow-debugger): refuse method-entry Live runs for site-scoped users (#55)
Owner decision (b), 2026-11-09. An inbound API method has no site
affinity, so the F2 route-time scope only bounded routed SITE traffic;
the method's central-side effects (Database.*, ExternalSystem,
notifications) still ran for a scoped Deployer. FlowDebugRunner now
refuses FlowDebugRunMethodCommand for a site-scoped caller (site ids,
not Administrator) before the script executes, audits the refusal as a
failed InboundRequest under the operator's name, and raises
SiteScopeViolationException -> ManagementUnauthorized, so the Central
UI and the CLI (same /management command) share the gate.

The page disables the "Inbound API method" entry for scoped Deployers
with the reason (tooltip + inline note), starts them on script entry,
and ignores ?methodId= / a stored method choice. Replay and the
instance-scoped script / value-change Live runs are unchanged. The
route-time scope is kept as defence in depth.

Tests: scoped refused (in- and out-of-scope routing; script never runs,
refusal audited), unscoped/Administrator allowed, scoped Replay allowed,
scoped in-scope value-change Live still runs; bUnit for the disabled
entry, the drill-in, and the unscoped control. Mutation-proven (gate
and UI flag reverted -> 4 red).

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 22:22:44 -05:00
Joseph Doherty 205a06a0f1 feat(transport): optional detached bundle signing with a pluggable key source (ScadaBridge#45)
Exports can now carry a detached ECDSA P-256 signature over the exact
manifest.json bytes in a new optional manifest.sig zip entry. The manifest
already pins the content hash, so a verified signature covers the whole
bundle, including the plaintext source environment the confirm step relies
on. Off by default (ScadaBridge:Transport:Signing): with the defaults,
exports are unsigned and imports accept signed and unsigned bundles.

- The bundle format does not change: bundleFormatVersion stays 1,
  manifest.json and the AES-GCM associated data are untouched, and older
  importers open entries by name and ignore manifest.sig. Three entries fit
  within the default MaxBundleEntryCount of 4.
- Import policy runs after the content-hash check and before any passphrase
  work. A signature that fails to verify against a trusted key is always
  refused. RequireSignedBundles additionally refuses unsigned bundles and
  bundles signed by an untrusted key. A refusal is BundleSignatureException
  (Commons), shown on the UI upload step and on CLI preview/import.
- The signature status, key id and signer are recorded on BundleSession and
  shown in the import wizard. They are also returned on PreviewBundleResult
  and recorded on the BundleImported audit row. BundleExported records
  Signed/SigningKeyId.
- Key material is pluggable. IBundleSigningKeySource and IBundleTrustStore
  are TryAdd'd; the defaults read Signing:PrivateKey (meant to be a
  ${secret:} reference) and Signing:TrustedKeys. Key ids are SPKI
  fingerprints. Startup validation rejects unparseable or non-P-256 keys.
- Compatibility proof: the checked-in fixtures pre-signing-plaintext and
  pre-signing-encrypted were exported by origin/main 3ff463b6, before this
  change, and are imported end to end with signing off and with signing
  configured.

The design leaves key-per-env vs per-org, trust distribution and rotation,
and the production default for RequireSignedBundles open. These are listed
in Component-Transport.md.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 17:42:28 -05:00
Joseph Doherty 1ae4c7a4cb fix(auditlog): pull clients attach the site PSK inside TLS on https endpoints (ScadaBridge#108)
GrpcPullAuditEventsInvoker, GrpcPullSiteCallsInvoker and
GrpcPullTraceEventsInvoker called WithSiteCredentials without the endpoint,
so they composed the PSK over insecure channel credentials and threw at
channel construction for any https site address. They now pass the endpoint
(same change as 9138f3ed): https composes over SecureSsl, http is unchanged.

An internal ConfigureHandler seam lets tests trust a generated certificate;
production leaves it null.

Tests: real Kestrel HTTP/2-over-TLS with the real ControlPlaneAuthInterceptor
for all three pulls (correct PSK accepted, wrong PSK PermissionDenied,
untrusted certificate fails). Reverting the fix turns all five red.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 17:24:22 -05:00
Joseph Doherty 7912d793e2 fix(host): central https URLs get TLS from a node certificate or refuse to boot (ScadaBridge#108)
Central re-bound its HTTP surface from ASPNETCORE_URLS keeping only the
port, so https://+:443 silently served plaintext HTTP. The parser now keeps
the scheme (ParseHttpBindEndpoints); an https entry binds UseHttps with the
node certificate from the new ScadaBridge:Node:Tls section (CertificatePath
+ CertificatePassword, or CertificateThumbprint + StoreName + StoreLocation;
default unset) and the node refuses to boot, with a clear error, when https
is requested without a certificate. http entries bind exactly as before.

Tests: scheme parsing, refuse-to-boot, http endpoint config identical to the
legacy inline binding (reflected effective Kestrel listen options), and a
real Kestrel HTTPS listener with a generated .pfx serving h2 and HTTP/1.1 by
ALPN while plaintext to the TLS port fails. Reverting the fix turns the
refuse-to-boot and real-listener tests red.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 17:22:57 -05:00
Joseph Doherty d1fc336e18 feat(management): parquet audit export and audit archive list/verify (ScadaBridge#42)
GET /api/audit/export?format=parquet returned 501. It now streams the same 24
columns as csv/jsonl (AuditExportRow, CsvHeader order) as one Parquet file,
one 10,000-row row group at a time. Each row group is encoded into a small
non-seekable staging buffer and copied to the response asynchronously, because
Kestrel rejects the synchronous writes the Parquet writer issues.

New management commands, with CLI parity:
- ListAuditArchives -> `audit archive list` (AuditArchiveListResult: enabled
  flag + every manifest).
- VerifyAuditArchive(Name) -> `audit archive verify --name` (files, chain,
  purge anchor); exit 0 verified, 3 not verified, 1 error, 2 unauthorized.
Both carry verify-chain's gate: OperationalAudit roles (Administrator, Viewer),
site-scoped non-Administrator refused because an archive spans every site.
VerifyAuditArchive uses the long-running Ask timeout. Error codes
ARCHIVE_NOT_CONFIGURED, INVALID_ARGUMENT, NOT_FOUND.

RequiredRoleMatrixTests freezes both gates; ManagementSurfaceParityTests now
probes `audit archive list|verify` alongside `audit verify-chain`.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 17:12:12 -05:00
Joseph Doherty 7be4f19861 feat(audit): Parquet cold archive of expired AuditLog partitions before the purge (ScadaBridge#42)
Audit roadmap T2 (alog.md §15.2): with AuditLog:Archive:Enabled (default off)
the purge actor writes each expired monthly partition to Parquet under an
absolute AuditLog:Archive:Directory before switching it out, so audit rows
outlive RetentionDays.

- Parquet.Net 6.1.0 (fully managed; NuGetAudit clean) in central package
  management.
- AuditArchiveParquet: data file = the ten stored columns, OccurredAtUtc as a
  UTC microsecond timestamp plus exact OccurredAtUtcTicks, and ChainSeq/RowHash;
  tombstone file = the month's per-channel purge tombstones. Buffered row
  groups; the files are plain Parquet (DuckDB/pyarrow read them).
- AuditLogPartitionArchive (IAuditLogArchive): reads the partition's rows,
  chain heads and tombstones in one transaction under the chain lock SHARED,
  writes {Directory}/{YYYY}/auditlog-{YYYY-MM}-{stamp}Z.parquet,
  .tombstones.parquet and a .manifest.json (counts, SHA-256 per file, each
  month's head and prior anchor) via tmp + rename, manifest last. Archives are
  never overwritten.
- Archive before purge: SwitchOutPartitionAsync gains an internal guarded
  overload. Under the EXCLUSIVE chain lock it re-counts the partition and
  re-reads each month's LastSeq and THROWs 51002/51003 (nothing purged) unless
  both still equal the archive's, so the switch's checkpoint anchor is exactly
  the archive's chain end. The manifest is then marked Purged.
- Fail closed: a partition that cannot be archived is not purged; the failure
  takes the existing AuditLogPurgeFailedEvent / PurgeFailures path. Success
  publishes AuditLogArchivedEvent.
- Verify: file hashes and counts against the manifest, the chain recomputed
  from the archived rows and tombstones (AuditChainVerifier), and the chain end
  against the database anchor (Matches / Superseded / NotPurged / Mismatch).
- AuditLogArchiveOptionsValidator: Directory required when enabled and must be
  absolute; RowGroupSize, timeouts bounded.

No migration: the purger role already reads AuditLog, the head and tombstones.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 17:12:12 -05:00
Joseph Doherty 9138f3edf8 fix(grpc): control-plane PSK credentials work on https endpoints (ScadaBridge#64)
WithSiteCredentials always composed the per-site PSK over
ChannelCredentials.Insecure. Grpc.Net.Client refuses that on an https
address ("insecure channel credentials and can't use a HttpClient with a
'https' scheme"), so every https control-plane endpoint threw at channel
construction - in SitePairChannelProvider the throw was caught and the
site address logged as malformed, i.e. a TLS site was silently
undialable. The old comment claiming TLS needed no change here was wrong.

WithSiteCredentials now takes the endpoint: https composes over
ChannelCredentials.SecureSsl (PSK retained inside TLS, no insecure
opt-in); anything else, including callers that omit it, keeps the
previous composition unchanged. SitePairChannelProvider,
SiteStreamGrpcClient and CentralChannelProvider pass their endpoint. The
three AuditLog pull clients are deliberately left for after #42.

Tests: real Kestrel HTTP/2-over-TLS (ALPN) with the real
ControlPlaneAuthInterceptor - correct PSK accepted, wrong PSK still
PermissionDenied, untrusted server cert fails with an
AuthenticationException cause, the pre-fix composition throws on https,
http composition unchanged; plus each production builder accepts https
(mutation check: disabling the TLS branch turns those red).

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 17:06:01 -05:00
Joseph Doherty 2e6102d48f feat(cluster): opt-in Akka.Remote TLS block, default off (ScadaBridge#64)
ScadaBridge:Cluster:RemotingTls (ClusterRemotingTlsOptions) renders
akka.remote.dot-netty.tcp enable-ssl + ssl { certificate, mutual auth,
hostname validation } only when Enabled. Chain validation is always on
(no suppress-validation switch). Certificate is a .pfx path + password
(${secret:} reference) or an OS-store thumbprint.

ClusterOptionsValidator, TLS on only: exactly one certificate source, a
legal store location, and akka.ssl.tcp:// on every seed - Akka.Remote
advertises akka.ssl.tcp addresses once SSL is on, so an akka.tcp seed is
never answered and the join retries forever with no startup error.

ClusterBootstrapGuard.SeedPattern accepted only one scheme segment, so
akka.ssl.tcp seeds parsed as nothing and the guard would silently never
engage on a TLS pair; it now accepts any number.

Default HOCON is byte-identical: 12 rendered documents (3 downing
strategies x guard on/off x Central/Site) captured before the change and
diffed after, all identical; a permanent test pins that a disabled block
with every field set renders the default document. A real ActorSystem
booted on the emitted TLS document comes up as akka.ssl.tcp (mutation
check: forcing enable-ssl=false turns it red).

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 17:06:01 -05:00
Joseph Doherty ab561faa43 feat(cluster): opt-in leader-auto-down strategy - in-repo port of Akka AutoDowning (ScadaBridge#106)
Adds LeaderAutoDowningProvider (Host/Actors), a line-for-line port of Akka.NET
1.5.62 AutoDowning/AutoDown/AutoDownBase read from the decompiled assembly: the
leader among the reachable members downs a peer unreachable for StableAfter; a
non-leader records it as pending and downs it on becoming leader; reachable-again
or removed members are forgotten; Down/Exiting members are skipped. Its window is
carried under scadabridge.cluster.leader-auto-down.unreachable-after, so no
deprecated auto-down-unreachable-after key is emitted and Akka's boot-time
"keep-majority will be used instead" warning goes away. DownRemovalMargin reads
akka.cluster.down-removal-margin exactly as AutoDowning does (default off).

New SplitBrainResolverStrategy value "leader-auto-down", wired through BuildHocon
and ClusterOptionsValidator. The default stays "auto-down"; auto-down and
keep-oldest HOCON is byte-identical to before. Placed in Host, not
ClusterInfrastructure, per the Code Placement rule (ClusterInfrastructure stays
Akka-free).

Tests: HOCON + validator units; LeaderAutoDownTests ports Akka's AutoDownSpec
cases (leader-only, pending-on-leader-change, losing leadership, reachable again,
removed, Down/Exiting skipped, single Down per member); LeaderAutoDownFailoverTests
clones SbrFailoverTests' oldest-crash and younger-crash cases on a real two-node
cluster plus a test that Akka actually loaded the provider and its downing actor.
Mutation-checked: a provider that never calls Cluster.Down fails both failover
tests; never-downs, leader-check-dropped, reachable-ignored and pending-not-flushed
mutations each fail the unit tests.

Remaining gate before switching the default: the docker failover drill
(DRILL_MODE=active and standby mode) on leader-auto-down.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 16:46:58 -05:00
Joseph Doherty 548bb9a725 feat(cli): working audit verify-chain via the VerifyAuditChain management command (ScadaBridge#41)
verify-chain was a no-op stub that printed "not enabled" and exited 0 without
contacting the server. It now sends VerifyAuditChain(Month) over
POST /management and maps the verdict to the exit code: 0 intact or not
chained, 3 broken (first break: partition, ChainSeq, EventId, kind), 1 error,
2 unauthorized. Table and JSON renderings.

ManagementActor: new HandleVerifyAuditChain arm, gated on the OperationalAudit
roles (Administrator, Viewer) and refusing a site-scoped caller because a
month's chain spans every site; INVALID_ARGUMENT for a bad month, CHAIN_BUSY
when the chain lock is held; long-running Ask timeout. Result enums serialise
as names.

Parity: RequiredRoleMatrixTests freezes the gate; ManagementSurfaceParityTests
now probes `audit verify-chain` (the one audit leaf that is a management
contract) instead of skipping the whole audit group.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 16:33:11 -05:00
Joseph Doherty fd4be80913 feat(audit): SHA-256 hash chain over the central AuditLog (ScadaBridge#41)
Audit roadmap T1 (alog.md §11.4): one chain per UTC calendar month of
OccurredAtUtc. Each chained row carries ChainSeq (gap-free within its month)
and RowHash = SHA-256(prev.RowHash || canonical(row)); the v1 canonical form
(AuditChainHasher) is pinned by a known-answer vector computed independently.

- Migration AddAuditLogHashChain: nullable ChainSeq/RowHash on dbo.AuditLog
  (metadata-only add), AuditLogChainHead (per-month chain end + partition-purge
  checkpoint anchor), AuditLogChainTombstone, two partition-aligned filtered
  indexes, scadabridge_audit_chainer role with a column-level UPDATE grant.
  Production script docs/plans/sql/AddAuditLogHashChain.sql.
- AuditLogHashChain chains unchained rows oldest month first under an
  exclusive sp_getapplock (zero timeout, so a dual-active central pair still
  yields one chain), with an optimistic head check and a ChainSeq IS NULL
  row-count check that roll a pass back instead of leaving a gap. The ingest
  path is untouched.
- Purges keep the chain verifiable: the partition switch records the month's
  checkpoint anchor and drops its tombstones; the per-channel DELETE writes a
  tombstone per chained row in the same transaction (OUTPUT INTO). Both, and
  the SourceNode backfill, hold the chain lock; the backfill skips chained
  rows. The switch staging table gains the two new columns.
- AuditChainVerifier reports the first break (partition, ChainSeq, EventId,
  kind) for edited, deleted, reordered rows and tail truncation.
- AuditLogHashChainService (central, default OFF via AuditLog:HashChain).

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 16:33:11 -05:00
Joseph Doherty a2c3a2d2b4 fix(management): case-variant instance override names are rejected with a did-you-mean hint (ScadaBridge#105)
Verified the ManagementActor bulk attribute override and single/bulk native alarm source
override paths match names ordinally and already reject a case-variant name (nothing stored);
the error now names the single case-insensitive match. Tests pin the rejection.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 16:23:37 -05:00
Joseph Doherty 0c2e852a62 fix(inbound-api): API-method recompile/invalidate reaches the peer central node; stale doc; locked-override name rule (ScadaBridge#103)
(a) Inbound API compile state (#75) is per central node: a recompile cleared the
known-bad record only on the node that answered, and the peer kept refusing the
unchanged script with SCRIPT_COMPILE_FAILED (the per-request revision check only
notices a changed script). Create / update / delete (ManagementActor post-commit),
RecompileApiMethod, and API methods written by a Transport bundle import now publish
InboundApiMethodInvalidated on the central cache-invalidation topic (#100 item 3).
The new InboundApiCacheInvalidationHandler drops that method's compiled handler,
known-bad record and cached ApiMethod row on the peer, which compiles the current
text on its next request.

(b) docs/components/InboundAPI.md no longer claims a direct DB edit stays stale
until restart: the per-request revision check picks it up once the node's cached
row expires (ApiMethodCacheTtl, 5 s). Component-InboundAPI, Component-Host and the
#85 cache audit describe the peer invalidation.

(c) Decision: the import's locked-override warnings keep EXACT (ordinal) name
matching, because FlatteningService matches override names ordinally. An override
differing from a locked member only by case never reaches that member's lock check
(the flattener drops it as unresolved), so a "targets a LOCKED member" warning for it
would be false. The comparer is now named (LockedOverrideNameComparer) and documented,
and pinned by tests on both sides (flattener + import).

Tests: the peer's known-bad record is cleared by a recompile (and validate leaves it
alone); create/update/delete and the import publish the invalidation; the handler
drops only the named method; composition root registers the handler; case-only
override gets no lock warning; the flattener matches override names exactly.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 16:04:28 -05:00
Joseph Doherty 153609ef8a feat(central): one central cache-invalidation topic; role-mapping and external-definition caches follow peer edits (ScadaBridge#100 item 3, ScadaBridge#83)
The #96 site-psk-invalidation topic was the first cross-node invalidation broadcast.
It is generalised into ONE central DistributedPubSub topic, central-cache-invalidation,
carrying typed messages (Commons ICentralCacheInvalidation: SitePskInvalidated,
RoleMappingsInvalidated, ExternalDefinitionsInvalidated, InboundApiMethodInvalidated)
in an envelope stamped with the publisher's process-unique origin id.

- Host: CentralCacheInvalidationBus (ICentralCacheInvalidationPublisher, no-op until
  the mediator is set, never throws) and CentralCacheInvalidationSubscriber, which
  skips its own node's envelopes and hands the peer's to every registered
  ICentralCacheInvalidationHandler (a failing handler does not stop the rest).
  Replaces SitePskInvalidationSubscriber.
- Migration of #96: SitePskRotationInvalidator publishes through the bus and is the
  PSK handler; SitePskInvalidated keeps its type name and is also published bare on
  site-psk-invalidation, which the subscriber still listens to, so a mixed-version
  central pair keeps exchanging PSK invalidations. Remove after one release.
- Audit D1: RoleMappingCache is a handler; ManagementActor's post-commit role-mapping
  invalidation also publishes RoleMappingsInvalidated, so a revocation is immediate on
  the peer instead of after RoleMappingCacheSeconds.
- Audit D3 / #83 (peer half): ExternalDefinitionLookupCache is a handler; the 9
  ManagementActor handlers and the Transport bundle importer publish
  ExternalDefinitionsInvalidated after commit. (#83's local import half was already in.)

TTLs remain the fallback for a lost broadcast or a direct SQL edit. Tests: bus and
subscriber unit tests (origin filter, legacy topic), peer-node tests through the real
caches in ManagementService.Tests, importer publish, composition root, and a real
two-node cluster from the production HOCON (CentralCacheInvalidationTwoNodeTests)
proving every message type crosses with the production serializer. Docs: Host,
Communication, Security, ManagementService, ESG, and the #85 cache audit.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 16:04:28 -05:00
Joseph Doherty 063fb21815 fix(native-alarms): a 4-column CSV import keeps stored MES overrides on the CLI path too (ScadaBridge#104)
The bulk SetInstanceNativeAlarmSourceOverridesCommand handler always wrote the
entry's IsMesReportableOverride, so a CLI import of a legacy 4-column CSV (no
MesReportable column) reset every imported source's MES override to inherit. The
Central UI avoided it only by looking up the stored flag client-side.

NativeAlarmSourceOverrideEntry gains an additive trailing KeepStoredMesReportable
flag: when set, the server leaves an existing row's MES override untouched (a new
row inherits) and ignores IsMesReportableOverride; unset keeps the old meaning
(null = clear to inherit), so older senders are unaffected. The CLI and the Central
UI now build their entries through one rule (ToOverrideEntry on each row type):
no MesReportable column => KeepStoredMesReportable. CLI README and
Component-CentralUI updated.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 16:04:28 -05:00
Joseph Doherty 47f3a703bf perf(search): prefix-by-default Deployments and config-audit search; bounded Contains opt-in (ScadaBridge#50)
Deployments free-text search and the config-audit EntityName filter no longer
emit LIKE '%term%' by default. Prefix (LIKE 'term%') is the default; substring
is an explicit opt-in (DeploymentSearchMode.Contains / entityNameContains)
always bounded by a DeployedAt/Timestamp window (explicit, else 30 days).
Migration PrefixSearchIndexes adds DeployedBy, RevisionHash and EntityName
indexes. Deployments page gains a 'Match anywhere' checkbox.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 15:42:56 -05:00
Joseph Doherty 0ce6633270 feat(flow-debugger): vertical trace list view alongside the swimlane (ScadaBridge#53)
Direction B: the same FlowGraph steps as a filterable list (lane column, lane filter,
failures-only), toggled from the flow panel toolbar. The mockup canvas is not in the repo,
so this is the minimal faithful version.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 15:33:31 -05:00
Joseph Doherty 62fe1e685d feat(native-alarms): optional MesReportable column in the native-source override CSV import (ScadaBridge#102 item 4)
CLI import no longer has to reset MES overrides to inherit; a legacy 4-column file
still parses and the Central UI import keeps the stored flag for it.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 15:33:31 -05:00
Joseph Doherty b540d8ac2c feat(central-ui): MES badge on live alarm rows and template native-source list (ScadaBridge#102 item 3)
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 15:33:31 -05:00
Joseph Doherty 3b0c2b3bfa fix(template-engine): computed-alarm IsMesReportable in revision hash and diff, true-only (ScadaBridge#102 item 2)
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 15:33:31 -05:00
Joseph Doherty 3dd492a77f feat(inbound-api): surface method compile errors to operators — api-method validate/recompile + compile state on get (ScadaBridge#75)
The data plane answers a broken inbound method with the generic "Script
compilation failed for this method"; the Roslyn diagnostic reached only the
central log, and the known-bad cache could not be observed or reset per method.

- InboundScriptExecutor keeps the diagnostics with each known-bad record and
  exposes GetCompileState(name, currentScript): IsCompiled / IsKnownBad /
  LastCompileError, judged against the current script text (a record for older
  text is reported as not compiled yet).
- Management: ValidateApiMethodCommand (compile the stored script, return
  ApiMethodCompileResult, register nothing, cache untouched) and
  RecompileApiMethodCommand (CompileAndRegister without re-sending the entity,
  drops the node's ApiMethodCache row, audited "Recompile"). Both Designer, like
  UpdateApiMethod; dispatched through the shared ApiMethodCompileOperations.
  GetApiMethod keeps the entity fields top-level and adds isCompiled,
  isKnownBad, lastCompileError and compileStateNode. Compile state is per
  central node and every result names the node.
- CLI: api-method validate --id, api-method recompile --id.
- Central UI: the API-method edit page shows the saved script's compile state
  and last error, with Validate and Recompile actions (same operations, same
  Designer gate, fresh DI scope).
- RequiredRoleMatrixTests entries; ManagementSurfaceParityTests covers the new
  CLI leaves; ApiMethodCompileDiagnosticsTests + an executor CompileState test.
- Docs: CLI README, Component-CLI/ManagementService/InboundAPI, components/*.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 15:30:46 -05:00
Joseph Doherty 65ffa1dad4 test(dcl): barrier instead of a 1 s window in DisconnectedWrites_AreRejectedAndNeverReplayed (ScadaBridge#101)
The reconnect:false case sent its writes straight after ActorOf and gave each
reply a fixed 1 s, so the window also covered the actor's start and first
dispatch. A slow start (cold JIT, starved pool in a full solution run) missed
it. The reconnect:true case was shielded by its 3 s wait for Connected.

The test now waits until the connect attempt the writes race against has been
issued (ConnectAsync entered, held open by the gate), so the actor is provably
mid-attempt and cannot connect before the test releases it. Reply bounds are
10 s barriers, not part of the property: a write that was queued instead of
rejected can only be answered after the gate opens, which never happens first.

Evidence: with a 1.2 s slow actor start modelled (UpdateConnectionHealth
Connecting sleeps), the old body passed 0/30 and the new one 30/30. Unmodified,
the new test passed 40/40 under 14 busy-loop CPU load. Mutations still caught:
Connecting dropping writes (timeout) and Connecting performing writes.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 15:30:46 -05:00
Joseph Doherty 6ff5efd3f4 fix(transport): case-insensitive name lookups inside a record on import (ScadaBridge#100)
Item 2 of #100, following the #93/#95 pattern (ToNameLookup: OrdinalIgnoreCase,
first wins, never throws):

- Overwrite child sync matches template attributes, alarms, scripts and native
  alarm sources, and external-system methods, case-insensitively, like their
  (parent, Name) unique indexes under SQL Server's collation. A member that
  differs only in case is updated in place instead of deleted and re-added
  (which lost the row id and could collide with the unique index).
- ResolveAlarmScriptLinksAsync resolves the alarm and its on-trigger script
  case-insensitively; AlarmsEqual compares the script name the same way.
- ArtifactDiff (preview) matches child collections case-insensitively so it
  agrees with the sync; notification recipients keep the ordinal match (no
  unique index, replaced wholesale).
- The rewire passes found the template a DTO was imported onto with an
  ordinal index. Since #93 an Overwrite can match a differently-cased target and
  keeps its spelling, so the passes silently skipped it and the alarm on-trigger
  links the sync had cleared stayed null. TryGetImported adds a case-insensitive
  (base-first) fallback; composed/base template references keep the exact
  lookup (item 1 of #100 is still undecided).

Test: ImportNameLookupTests.Members_inside_a_record_match_case_insensitively,
over a proxy that gives the by-name template reads SQL Server's collation.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 15:30:46 -05:00
Joseph Doherty f5eed5e8fd feat(audit): persist OPC UA certificate-trust decisions to the central audit log (ScadaBridge#43)
New AuditChannel.Security with AuditKind.CertTrust / CertTrustRemove (appended, ordinals stable).
Both central entry points - the management actor handlers and the Central UI CertManagementService -
write a best-effort ICentralAuditWriter row after the site outcome is known (Delivered when all
reachable site nodes acked, Failed otherwise, incl. relay timeouts), naming actor, site, thumbprint
and connection. Shared builder: CertTrustAuditEvents.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 15:17:03 -05:00
Joseph Doherty f98369a786 feat(native-alarms): per-binding IsMesReportable on native alarm source bindings (ScadaBridge#59)
TemplateNativeAlarmSource.IsMesReportable + tri-state InstanceNativeAlarmSourceOverride
.IsMesReportableOverride flatten onto ResolvedNativeAlarmSource; NativeAlarmActor stamps every
mirrored condition and InstanceActor the quiet-binding placeholder, so ScriptAlarm,
Alarms.MesReportableAsync() and the #60 wire fields carry it. Additive to the 900-999 severity band.
Migration AddNativeAlarmSourceMesReportable. Management commands, CLI --mes-reportable, Central UI
(template dialog checkbox, instance override select), Transport DTOs/importer/diff and the revision
hash (true-only, so existing hashes are unchanged) carry it.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 15:09:54 -05:00
Joseph Doherty baf89b3df7 test+docs: serialise data-connection gauge tests, add metrics instrument reference (ScadaBridge#99)
DataConnectionGaugeTests and ExtendedInstrumentTests both mutate the process-global data
connection registry; put them in a non-parallel xUnit collection. Add docs/operations/metrics-reference.md.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 14:50:58 -05:00
Joseph Doherty 7848a3fe7c fix(transport): case-insensitive remaining library lookups; base-only template reads for import targets (ScadaBridge#95)
- Preview matches template folders, shared scripts, external systems, SMTP
  (by Host) and SMS (by AccountSid) with the #93 ToNameLookup comparer
  (OrdinalIgnoreCase, first wins, never throws), matching SQL Server's
  case-insensitive collation, so a differently-cased target row is reported as
  a conflict instead of New.
- Apply-side in-memory lookups do the same: ApplyTemplateFoldersAsync, the
  template folder-id map, ResolveFolderParentEdgesAsync, ApplySmtpConfigsAsync
  and ApplySmsConfigsAsync. An Overwrite now updates the existing row instead of
  adding a duplicate (folder names are only unique per parent, so the old
  ToDictionary could also throw).
- New ITemplateEngineRepository.GetBaseTemplatesWithChildrenAsync filters
  IsDerived in SQL. Preview and the Overwrite load in ApplyTemplatesAsync use it,
  so a derived slot template that shares a bundled name is never the import
  target. GetTemplatesWithChildrenAsync is unchanged for the id-keyed callers
  (export closure, rewire passes), which can need derived rows.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 14:30:17 -05:00
Joseph Doherty 3a31dd2de7 fix(host): follow an in-place site PSK rotation without a central restart (ScadaBridge#96)
SitePskProvider cached each site's gRPC preshared key forever; only site removal
invalidated it, so rotating SB-GRPC-PSK-{site} broke the central<->site link on
both central nodes until they restarted. The cached key is now dropped when:

- the secret is written through /admin/secrets: central replaces the secrets
  library's ISecretCacheInvalidator with SitePskRotationInvalidator, which also
  publishes SitePskInvalidated on a new DistributedPubSub topic
  (site-psk-invalidation) so the peer central node drops its copy and its
  resolver cache entry too (SitePskInvalidationSubscriber);
- a site rejects a central->site call with PermissionDenied/Unauthenticated:
  WithSiteCredentials wraps every per-site channel's handler in
  PskRejectionRefreshHandler, so the next call re-reads the key (covers
  out-of-process writes such as the secret CLI);
- a site->central call presents a token that does not match the cached key:
  CentralControlAuthInterceptor invalidates and re-reads once before refusing;
- ScadaBridge:Communication:SitePsks reloads.

Invalidation is case-insensitive because SecretName lowercases names. Rotation
runbook (topology-guide), Component-Communication/Host and the #85 cache audit
(D2) updated.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 14:30:17 -05:00
Joseph Doherty 7f479f3a5c feat(logging): stamp ExecutionId/InstanceName/ScriptName on the Serilog LogContext per run (ScadaBridge#56)
The site script and alarm on-trigger launchers push the run identity onto LogContext for the
run's lifetime and the host now enables Enrich.FromLogContext(), so log lines written on the
run's own async call chain carry the id. Bounded part only: LogContext does not cross an Akka
message hop, so Instance Actor/DCL/S&F lines and a site_events execution-id column remain open
(documented in the step-logs design doc section 9).

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 14:20:02 -05:00
Joseph Doherty 5ef27fd9a3 feat(telemetry): deploy-failure, S&F activity, script-run and DCL-transition instruments (ScadaBridge#81)
Adds four instruments on the already-allow-listed ZB.MOM.WW.ScadaBridge meter:
scadabridge.deployments.failed{reason}, scadabridge.store_and_forward.activity{action,category},
scadabridge.site.script.runs{outcome} + scadabridge.site.script.duration{outcome} (ms),
scadabridge.site.data_connections.transitions{state}. Queue depth, DCL connection state and
deploy-applied already existed. No new Meter, so ZbTelemetryOptions.Meters needs no change.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 14:17:53 -05:00
Joseph Doherty 7892c622cf fix(health): site /health/active uses the shared role-scoped ActiveNodeHealthCheck (ScadaBridge#97)
Replace the private SitePairActiveNodeHealthCheck (503 whenever no ActorSystem, no data)
with the shared ZB.MOM.WW.Health ActiveNodeHealthCheck constructed with the site-{SiteId}
role: Degraded (200) during startup per the Health spec, and it publishes
selfAddress/activeRole/activeNode so the Overview dashboard can name the active site node.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 14:13:53 -05:00
Joseph Doherty a16b7b4e65 fix(central): audit per-node central caches; management API drops ApiMethodCache entries (ScadaBridge#85)
Audit (docs/plans/2026-11-09-central-cache-staleness-audit.md) of every central in-process
cache holding configuration or secret state, with its staleness bound on the peer central
node. Every config cache with peer staleness is TTL-bounded (5-60 s) and Traefik routes to the
active node only, so no peer case was clearly a bug and no cross-node broadcast is added; the
doc names the existing DistributedPubSub mediator (health replication topic) as the mechanism
if one is wanted. Owner decisions listed: RoleMappingCache 30 s revocation lag on the peer,
SitePskProvider caching a PSK forever across an in-place rotation (both nodes), and the #83
ExternalDefinitionLookupCache 10 s bound.

Fixed: ManagementActor API-method create/update/delete invalidated only the compiled handler,
not ApiMethodCache, so the node that made the change kept serving the pre-change row for up to
the 5 s TTL (a deleted method stayed callable; a cached not-found hid a new one). The handlers
now drop the entry post-commit, like the role-mapping cache.

Tests: ApiMethodCacheInvalidationTests (create/update/delete fail without the fix; a
negative control checks other methods stay cached).

(ScadaBridge#85)

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 13:54:29 -05:00
Joseph Doherty 23c35df234 refactor(inbound-api): use the shared ZB.MOM.WW.Health IActiveNodeGate (ScadaBridge#80)
The InboundAPI carried its own IActiveNodeGate (implemented by Host/Health/ActiveNodeGate)
whose rule already matched the shared Health 0.3.0 AkkaActiveNodeGate — both route to
ClusterActiveNode (oldest Up member). The private interface and implementation are deleted;
InboundAPI references ZB.MOM.WW.Health (0.3.0, already pinned) and InboundApiEndpointFilter,
AuditWriteMiddleware, InboundMethodWarmupService and FlowDebugRunner consume
ZB.MOM.WW.Health.IActiveNodeGate. The Central branch of Program.cs registers
AkkaActiveNodeGate (unscoped, like the active-node health check).

Warm-up behaviour is kept: the shared gate answers false while no ActorSystem resolves, while
this member is not Up, and on any cluster-init fault. It resolves the ActorSystem through the
existing DI singleton bridge (GetOrCreateActorSystem), the same path the health probes use.

Tests: CompositionRoot asserts the shared gate type (fails with the registration removed);
the evaluator wiring test now drives AkkaActiveNodeGate; new warm-up test (no ActorSystem ->
not active). Nothing was missing from the Health 0.3.0 package.

(ScadaBridge#80)

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 13:54:29 -05:00
Joseph Doherty 1ee2e171cf refactor(deployment): move PendingUpdatesCache to DeploymentManager so the CLI shares it (ScadaBridge#91)
`health pending-updates` (#63) called IPendingUpdatesEvaluator.EvaluateAsync directly
from ManagementActor because the cache lived in CentralUI, so every CLI call was a full
uncached fleet flatten. IPendingUpdatesCache/PendingUpdatesCache now live in
DeploymentManager.PendingUpdates and are registered by AddDeploymentManager; the
SingleFlightMemo<T> primitive moves to Commons.Caching (public) so both CentralUI's KPI /
alarm-summary caches and DeploymentManager can use it. HandleGetPendingUpdates reads the
shared cache; the UI circuits and the CLI now share one memo per node.

Tests: ManagementActor repeated-calls test (fails against the direct evaluator call:
2 evaluations instead of 1), AddDeploymentManager singleton-registration test; the cache
and memo tests move with their types.

(ScadaBridge#91)

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 13:54:29 -05:00
Joseph Doherty c7d868c52e fix(transport): case-insensitive name lookups, batch preview lookups, correct flattener doc (ScadaBridge#93)
- ApplyTemplatesAsync/PreviewAsync template lookups use a case-insensitive,
  never-throwing comparer (SQL Server collation); base template wins over derived.
  Base names are already unique case-insensitively via the filtered unique index
  on Template.Name, derived templates are not covered by it.
- Preview reads database connections, notification lists and API methods with one
  GetAll query each instead of one by-name query per bundled row.
- Component-TemplateEngine.md: the flattener does not call GetAllTemplatesAsync;
  it loads chains via GetTemplateWithChildrenAsync.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 13:46:45 -05:00
Joseph Doherty bec07378cc test(deployment): de-flake unreachable-site reconcile breaker test; assert structurally, not by 2 s wall clock (ScadaBridge#92)
Under a loaded full-project run the serial prepare of 6 instances with a 300 ms
reconcile deadline took 2.6 s and tripped the fixed 2 s cap (5/10 full runs
failed under CPU load). Deadline raised to 2 s so it cannot expire before the
query is sent; the elapsed bound is now derived from the deadline (must beat
(n-1) deadlines) while the one-query assertion proves the breaker.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 13:44:02 -05:00
Joseph Doherty 7153a1974e perf(transport): make large-bundle apply linear in the template rewire passes (ScadaBridge#47)
Measured first: a scratch scaling probe (seed -> export -> load/preview/apply
into a fresh target and over itself, in-memory provider, every repository
call counted, DetectChanges counted via ChangeTracker.DetectingAllChanges)
showed load, preview and the instance pass scale linearly (500 / 1000 / 2000
instances apply in 0.13 / 0.27 / 0.35 s), but the post-flush template
rewire passes were quadratic: 100 / 400 / 800 composing, derived templates
applied in 0.5 / 8.1 / 26.5 s, with DetectChanges running 5N+10 times and
GetTemplateWithChildrenAsync 2N+2 times.

Cause: ResolveAlarmScriptLinksAsync, ResolveCompositionEdgesAsync and
ResolveInheritanceEdgesAsync found each template with
_dbContext.Templates.Local.FirstOrDefault(...) - per template, and per
composition edge / base lookup. Every DbSet.Local access runs a full
DetectChanges over everything the import has staged. The first two passes
also re-read every template's children one query at a time (a split query
per call on SQL Server).

Fix (no bundle-format or semantic change):
- each pass indexes the tracked templates by name once
  (SnapshotLocalTemplatesByName; ordinal, first wins like FirstOrDefault).
  These passes never stage a template or rename one, so the index stays
  exact for the pass;
- the alarm and composition passes read their templates' children in one
  batched GetTemplatesWithChildrenAsync, matched back by the stub's id;
- the alarm pass only loads templates whose bundled alarms name an
  on-trigger script (the only ones it can rewire).

Result on the same probe: 100 / 400 / 800 templates apply in
0.13 / 0.62 / 0.64 s; DetectChanges is a constant 9; per-template children
reads 0, batched reads 1 (add) / 2 (overwrite).

Evidence: ImportApplyScalingTests (Transport.IntegrationTests) exports 5 and
40 composing, derived templates with alarm -> script links, imports each
into an empty target (Add) and again (Overwrite), and asserts the imported
inheritance / composition / alarm-script links, zero per-template children
reads, at most 3 batched reads, and an identical DetectChanges count for 5
and 40 templates. Fails on the old importer (12 per-template reads), and
with a per-DTO Local snapshot reintroduced in one pass (DetectChanges 51
vs 16). Transport unit (198) and integration (138) suites pass.

Not changed: preview's per-name lookups for database connections,
notification lists and API methods stay per-name - moving them to an
in-memory ordinal match could change name matching under a case-insensitive
SQL collation.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 13:30:27 -05:00
Joseph Doherty 51d2f6c906 refactor(deployment): injectable terminal-purge rate limiter; test the swallowed failure (ScadaBridge#90)
DeploymentService.TryPurgeTerminalDeploymentRecordsAsync rate-limited its
opportunistic retention sweep with a process-wide static tick field, so only
one test in the assembly could drive the sweep and its swallow-on-failure path
was untested.

The limit now lives in TerminalPurgeRateLimiter: a TimeProvider-driven
compare-and-swap claim (MinInterval 6 h) registered by AddDeploymentManager as
a DI singleton, so every scoped DeploymentService still shares one limit per
node. DeploymentService takes it as an optional trailing constructor parameter;
a hand-built instance gets its own limiter. The sweep's cutoff now comes from
the same clock. Behaviour is otherwise unchanged.

Evidence (TerminalPurgeRateLimiterTests):
- claim once per interval on an injected clock; 16 concurrent claimants ->
  exactly one wins;
- two services with their own limiters both sweep, two sharing one sweep once;
- cutoff = injected now - retention;
- a failing artifact batch is logged as a warning with the exception and
  swallowed, the already-purged terminal rows are still reported, the next
  call inside the interval does not retry and the first call after it does;
  a first-batch failure returns 0 and skips the artifact purge;
- AddDeploymentManager registers the limiter as a singleton shared across scopes.
Mutation-checked: a process-wide static limiter fails 5 tests (incl. the
existing artifact-purge test); narrowing the catch fails both failure tests.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 13:30:27 -05:00
Joseph Doherty 6070cc2b9d perf(templates): resync and resolved-members read only the chain/subtree (ScadaBridge#89)
The last two whole-library template reads are gone. Both fed the
inheritance-only TemplateInheritanceResolver, which walks one template's
parent chain and never follows compositions.

- TemplateService.ResyncInheritedMembersAsync (operator repair, CLI + editor)
  now reads #86's GetTemplateReconcileClosureAsync (derived subtree + the
  template's chain, tracked). When nothing derives from the template the
  closure is empty, so it falls back to the new tracked
  GetTemplateChainAsync - resyncing a leaf reads only its own chain.
- ManagementActor.HandleGetResolvedTemplateMembers now reads
  GetTemplateChainAsync(id, trackChanges: false): the chain alone, untracked,
  with script bodies (the resolver projects script code, so #66's body-free
  analysis closure would not be equivalent).

The chain walk is factored out of TemplateReconcileClosure into
CollectChainAsync (one load per level, stops on parent cycles and dangling
parents) and shared by both.

Evidence:
- TemplateChainReadTests (#86 fixture): for every template plus an unknown
  id, the closure-backed resync issues exactly the writes, audit row and
  result of the whole-graph resync with GetAllTemplatesAsync wired to throw;
  the resolver answers identically from the chain and the whole graph.
  17 of these fail against the old service; 13 fail with the leaf chain
  fallback removed (mutation-checked).
- TemplateChainRepositoryTests (SQLite, recorded SQL): a 4-deep chain loads
  4 templates / 8 attributes instead of 46 / 92, one split-query set per
  level, untracked on request and tracked for the resync.
- ResolvedTemplateMembersReadTests: the actor answers from the untracked
  chain read and never calls GetAllTemplatesAsync (fails on the old actor).

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 13:30:26 -05:00
Joseph Doherty d50433410d feat(health): surface pending-updates counts on the Health dashboard and CLI (ScadaBridge#63)
Health page gains PendingUpdatesHealthTiles (stale instances / sites with stale
artifacts, Deployer+Administrator only, site-scope narrowed, shared cache). New
GetPendingUpdates management command (Deployer) and 'health pending-updates' CLI
leaf. Flatten/cache computation untouched; the command reads
IPendingUpdatesEvaluator directly because the UI cache is not reachable from
ManagementService.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 13:18:54 -05:00
Joseph Doherty 004bf3cf06 feat(health): show site event-log/audit write and redaction failures on the Health dashboard (ScadaBridge#88)
New SiteWriteFailureBadges component renders warning badges for
SiteEventLogWriteFailures, SiteAuditWriteFailures and AuditRedactionFailure
when > 0 on online sites. The CLI 'health summary' already serialises the full
latest report, so these counters are visible there unchanged.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 13:13:58 -05:00
Joseph Doherty c4186509f6 refactor(health): move ISiteHealthCollector to Commons; Communication drops HealthMonitoring reference (ScadaBridge#87)
Interface now lives in Commons.Interfaces.Services; AuditLog no longer gets it
transitively through Communication. Implementation stays in HealthMonitoring.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 13:13:58 -05:00
Joseph Doherty e5ad4c9f53 test(site-call-audit, deployment): cover error, retry and state-transition branches (ScadaBridge#73)
Backfills the branches arch-review 08 flagged as thin, chosen for what they
decide rather than for count. Every new test was mutation-checked: the
guarded line was broken and the test went red.

SiteCallAudit (SiteCallAuditFailurePathTests, 17 cases, in-memory - the
18 MSSQL-backed actor tests skip without a database, so the ingest and read
fault paths had no runnable coverage):
- a failed page write holds the reconciliation cursor; the next tick re-pulls
  the same window and advances only after the write lands
- the pinned latch publishes Pinned=true once across many pinned ticks and
  Pinned=false once when the site starts honouring after_id
- a site-enumeration fault skips one tick without wedging the single-flight
  guard
- an endless backlog drains at most 50 pages per tick, unpinned, and the next
  tick resumes from the advanced cursor
- purge single-flight: ticks during a blocked purge are dropped
- ingest fault replies Accepted=false and the next upsert is accepted
- detail / KPI / per-site / per-node / query faults map to Success=false with
  the root-cause message and an empty payload; the actor keeps serving
- query page size clamped to [1, 200]; an empty page carries no cursor
- Discard NotParked / OperationFailed (site message verbatim) with no audit
  row; Discard before transport registration is SiteUnreachable
- a throwing audit sink leaves an Applied relay Applied; a missing mirror row
  or unknown channel still audits the operator, falling back to ApiOutbound

DeploymentManager (DeploymentServiceFailurePathTests, 11 cases):
- a site that answers Failed commits a Failed record with the site's message
  (no timeout marker), leaves instance state and the deployed snapshot
  untouched, and audits Deploy (not DeployFailed)
- a prior non-timeout Failed record skips the query-before-redeploy
- a failing Failed-status write detaches pending changes and retries once
  (none when nothing was detached); the caller still gets the original error
- Disable / Enable / Delete answered with Success=false leave the instance
  unchanged, audit the attempt, and surface the site's message or the default

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 13:10:31 -05:00
Joseph Doherty c29b99c63b perf(templates): descendant reconcile loads only the subtree + chain (ScadaBridge#86)
TemplateService.ReconcileDescendantsAsync - run after every base member
add/update/delete, template and native-alarm-source alike - loaded the whole
tracked template graph (every template, member and script body) on each call,
which after #66 was the dominant cost of those paths. It now calls the new
ITemplateEngineRepository.GetTemplateReconcileClosureAsync: the same tracked,
fully-included shape as GetAllTemplatesAsync, restricted to the edited
template's derived subtree plus its own inheritance chain. The reconcile is
inheritance-only (TemplateInheritanceResolver never follows compositions) and
every descendant's chain runs through the edited template, so that set is
everything it reads.

The walk (Commons TemplateReconcileClosure, the descendant-direction twin of
#66's TemplateAnalysisClosure) loads one children batch per subtree depth and
one batch per ancestor level, terminates on parent cycles and stops at
dangling parents. A template with no children - the common case - costs one
empty probe and loads nothing.

Evidence:
- TemplateReconcileClosureTests: for every template of a graph with a deep
  subtree, an ancestor above the edited template, a composed module,
  look-alike unrelated trees, a parent cycle and a dangling parent, the
  closure-backed reconcile issues exactly the writes (add/update/delete,
  audit, save) of the old whole-graph reconcile, and the resolver answers
  identically for every descendant. Fails with the old service (whole-graph
  load wired to throw) and with the ancestor walk removed (mutation-checked).
- TemplateReconcileClosureRepositoryTests (SQLite, recorded SQL): 5 templates
  / 10 attributes instead of 47 / 94, tracked with script bodies; a leaf costs
  one probe query and tracks nothing.
- TemplateServiceTests: attribute add + update propagate two levels down with
  GetAllTemplatesAsync wired to throw; existing propagation tests now run
  against the closure via ServeReconcileClosureFromFullGraph.

ResyncInheritedMembersAsync (operator action) still reads the whole graph.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 13:10:31 -05:00
Joseph Doherty 0eac956d25 feat(health): report permanently rejected site audit rows on the site health report (ScadaBridge#69)
SiteAuditBacklogSnapshot gains an optional trailing RejectedCount (gauge of
Rejected forward-state rows, read from the site store), carried on the wire as
SiteAuditBacklogSnapshotDto.rejected_count = 4 (additive; old sites read as 0,
old central ignores it). Health page shows an 'Audit rows rejected' badge.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 13:05:48 -05:00
Joseph Doherty 3ae3f1bcef refactor(health): move ICentralHealthAggregator and SiteHealthState to Commons (ScadaBridge#71)
Interface goes to Commons.Interfaces.Services and the state record to
Commons.Messages.Health, so CentralCommunicationActor no longer needs the
HealthMonitoring namespace. Implementation stays in HealthMonitoring.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 13:05:48 -05:00