Auth 0.4.1 (BindMode stays ServiceAccount; empty password now refused in the library too; Username canonicalised from UserNameAttribute). No API breaks, no test changes needed.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Evaluated against the current pages and their data sources. The notification
outbox is central-owned, offset-paged and has its own status lifecycle. Site
calls are site-owned and keyset-paged, with a different lifecycle and a
Retry/Discard that relays to the owning site. A union grid would need a
two-cursor merge, a status filter that does not line up, and buttons that hide
different semantics. The revisit trigger (operator confusion reports) has not
fired.
Recommends against building it now. If confusion is reported, start with
cross-links and a combined "stuck items" Health card.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Neither feature is specified well enough to build safely, so this records the
open questions, the options and a recommendation instead of code.
- Pull: the existing CLI can already orchestrate export from one cluster and
import into another with per-cluster auth and audit. A server-side pull
needs decisions on the machine credential, source-side authorisation,
network direction and passphrase handling. Recommendation: orchestrate it
with the CLI and RequireSignedBundles; write no code now.
- Differential: imports already auto-skip Identical rows, so a full bundle
applies only what changed. The baseline, deletions and target drift are
undefined. Recommendation: keep YAGNI; if it is ever needed, use
per-entry content hashes plus --since, and never tombstones.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Exports can now carry a detached ECDSA P-256 signature over the exact
manifest.json bytes in a new optional manifest.sig zip entry. The manifest
already pins the content hash, so a verified signature covers the whole
bundle, including the plaintext source environment the confirm step relies
on. Off by default (ScadaBridge:Transport:Signing): with the defaults,
exports are unsigned and imports accept signed and unsigned bundles.
- The bundle format does not change: bundleFormatVersion stays 1,
manifest.json and the AES-GCM associated data are untouched, and older
importers open entries by name and ignore manifest.sig. Three entries fit
within the default MaxBundleEntryCount of 4.
- Import policy runs after the content-hash check and before any passphrase
work. A signature that fails to verify against a trusted key is always
refused. RequireSignedBundles additionally refuses unsigned bundles and
bundles signed by an untrusted key. A refusal is BundleSignatureException
(Commons), shown on the UI upload step and on CLI preview/import.
- The signature status, key id and signer are recorded on BundleSession and
shown in the import wizard. They are also returned on PreviewBundleResult
and recorded on the BundleImported audit row. BundleExported records
Signed/SigningKeyId.
- Key material is pluggable. IBundleSigningKeySource and IBundleTrustStore
are TryAdd'd; the defaults read Signing:PrivateKey (meant to be a
${secret:} reference) and Signing:TrustedKeys. Key ids are SPKI
fingerprints. Startup validation rejects unparseable or non-P-256 keys.
- Compatibility proof: the checked-in fixtures pre-signing-plaintext and
pre-signing-encrypted were exported by origin/main 3ff463b6, before this
change, and are imported end to end with signing off and with signing
configured.
The design leaves key-per-env vs per-org, trust distribution and rotation,
and the production default for RequireSignedBundles open. These are listed
in Component-Transport.md.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
GrpcPullAuditEventsInvoker, GrpcPullSiteCallsInvoker and
GrpcPullTraceEventsInvoker called WithSiteCredentials without the endpoint,
so they composed the PSK over insecure channel credentials and threw at
channel construction for any https site address. They now pass the endpoint
(same change as 9138f3ed): https composes over SecureSsl, http is unchanged.
An internal ConfigureHandler seam lets tests trust a generated certificate;
production leaves it null.
Tests: real Kestrel HTTP/2-over-TLS with the real ControlPlaneAuthInterceptor
for all three pulls (correct PSK accepted, wrong PSK PermissionDenied,
untrusted certificate fails). Reverting the fix turns all five red.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Central re-bound its HTTP surface from ASPNETCORE_URLS keeping only the
port, so https://+:443 silently served plaintext HTTP. The parser now keeps
the scheme (ParseHttpBindEndpoints); an https entry binds UseHttps with the
node certificate from the new ScadaBridge:Node:Tls section (CertificatePath
+ CertificatePassword, or CertificateThumbprint + StoreName + StoreLocation;
default unset) and the node refuses to boot, with a clear error, when https
is requested without a certificate. http entries bind exactly as before.
Tests: scheme parsing, refuse-to-boot, http endpoint config identical to the
legacy inline binding (reflected effective Kestrel listen options), and a
real Kestrel HTTPS listener with a generated .pfx serving h2 and HTTP/1.1 by
ALPN while plaintext to the TLS port fails. Reverting the fix turns the
refuse-to-boot and real-listener tests red.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Component-AuditLog Retention & Purge describes the archive: directory and
layout, the three files, consistency with the hash chain (SHARED read,
guarded switch, anchor = archived chain end), fail-closed purge, retention
interplay (per-channel overrides are not archived, their tombstones are),
verify/read, and the AuditLog:Archive configuration. alog.md §13.6/§15.2/§16,
CLAUDE.md, HighLevelReqs AL-12, Component-CLI, Component-ManagementService,
docs/components/{CLI,ManagementService}.md and the CLI README drop the
"deferred / 501" wording and document `audit archive list|verify` and the
parquet export.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
GET /api/audit/export?format=parquet returned 501. It now streams the same 24
columns as csv/jsonl (AuditExportRow, CsvHeader order) as one Parquet file,
one 10,000-row row group at a time. Each row group is encoded into a small
non-seekable staging buffer and copied to the response asynchronously, because
Kestrel rejects the synchronous writes the Parquet writer issues.
New management commands, with CLI parity:
- ListAuditArchives -> `audit archive list` (AuditArchiveListResult: enabled
flag + every manifest).
- VerifyAuditArchive(Name) -> `audit archive verify --name` (files, chain,
purge anchor); exit 0 verified, 3 not verified, 1 error, 2 unauthorized.
Both carry verify-chain's gate: OperationalAudit roles (Administrator, Viewer),
site-scoped non-Administrator refused because an archive spans every site.
VerifyAuditArchive uses the long-running Ask timeout. Error codes
ARCHIVE_NOT_CONFIGURED, INVALID_ARGUMENT, NOT_FOUND.
RequiredRoleMatrixTests freezes both gates; ManagementSurfaceParityTests now
probes `audit archive list|verify` alongside `audit verify-chain`.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Audit roadmap T2 (alog.md §15.2): with AuditLog:Archive:Enabled (default off)
the purge actor writes each expired monthly partition to Parquet under an
absolute AuditLog:Archive:Directory before switching it out, so audit rows
outlive RetentionDays.
- Parquet.Net 6.1.0 (fully managed; NuGetAudit clean) in central package
management.
- AuditArchiveParquet: data file = the ten stored columns, OccurredAtUtc as a
UTC microsecond timestamp plus exact OccurredAtUtcTicks, and ChainSeq/RowHash;
tombstone file = the month's per-channel purge tombstones. Buffered row
groups; the files are plain Parquet (DuckDB/pyarrow read them).
- AuditLogPartitionArchive (IAuditLogArchive): reads the partition's rows,
chain heads and tombstones in one transaction under the chain lock SHARED,
writes {Directory}/{YYYY}/auditlog-{YYYY-MM}-{stamp}Z.parquet,
.tombstones.parquet and a .manifest.json (counts, SHA-256 per file, each
month's head and prior anchor) via tmp + rename, manifest last. Archives are
never overwritten.
- Archive before purge: SwitchOutPartitionAsync gains an internal guarded
overload. Under the EXCLUSIVE chain lock it re-counts the partition and
re-reads each month's LastSeq and THROWs 51002/51003 (nothing purged) unless
both still equal the archive's, so the switch's checkpoint anchor is exactly
the archive's chain end. The manifest is then marked Purged.
- Fail closed: a partition that cannot be archived is not purged; the failure
takes the existing AuditLogPurgeFailedEvent / PurgeFailures path. Success
publishes AuditLogArchivedEvent.
- Verify: file hashes and counts against the manifest, the chain recomputed
from the archived rows and tombstones (AuditChainVerifier), and the chain end
against the database anchor (Matches / Superseded / NotPurged / Mismatch).
- AuditLogArchiveOptionsValidator: Directory required when enabled and must be
absolute; RowGroupSize, timeouts bounded.
No migration: the purger role already reads AuditLog, the head and tombstones.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Per-hop current state with code evidence (browser/Traefik/Kestrel, CLI,
central<->site gRPC + PSK, LocalDb sync, Akka remoting, OPC UA /
MxGateway), the recommended mode per hop (Kestrel HTTPS + ALPN, gRPC over
TLS with validation and the PSK retained, Akka DotNetty SSL with a cert
per node and mutual auth), certificate issuance/storage/rotation, rig and
wonder impact, a staged rollout, rollback, a test plan and 13 open owner
decisions. Findings: the central ASPNETCORE_URLS re-bind drops the
scheme (an https URL binds plaintext); Akka.NET has no secure-cookie, so
mTLS is the only remoting authentication. Component-ClusterInfrastructure
documents the new RemotingTls option.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
WithSiteCredentials always composed the per-site PSK over
ChannelCredentials.Insecure. Grpc.Net.Client refuses that on an https
address ("insecure channel credentials and can't use a HttpClient with a
'https' scheme"), so every https control-plane endpoint threw at channel
construction - in SitePairChannelProvider the throw was caught and the
site address logged as malformed, i.e. a TLS site was silently
undialable. The old comment claiming TLS needed no change here was wrong.
WithSiteCredentials now takes the endpoint: https composes over
ChannelCredentials.SecureSsl (PSK retained inside TLS, no insecure
opt-in); anything else, including callers that omit it, keeps the
previous composition unchanged. SitePairChannelProvider,
SiteStreamGrpcClient and CentralChannelProvider pass their endpoint. The
three AuditLog pull clients are deliberately left for after #42.
Tests: real Kestrel HTTP/2-over-TLS (ALPN) with the real
ControlPlaneAuthInterceptor - correct PSK accepted, wrong PSK still
PermissionDenied, untrusted server cert fails with an
AuthenticationException cause, the pre-fix composition throws on https,
http composition unchanged; plus each production builder accepts https
(mutation check: disabling the TLS branch turns those red).
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
ScadaBridge:Cluster:RemotingTls (ClusterRemotingTlsOptions) renders
akka.remote.dot-netty.tcp enable-ssl + ssl { certificate, mutual auth,
hostname validation } only when Enabled. Chain validation is always on
(no suppress-validation switch). Certificate is a .pfx path + password
(${secret:} reference) or an OS-store thumbprint.
ClusterOptionsValidator, TLS on only: exactly one certificate source, a
legal store location, and akka.ssl.tcp:// on every seed - Akka.Remote
advertises akka.ssl.tcp addresses once SSL is on, so an akka.tcp seed is
never answered and the join retries forever with no startup error.
ClusterBootstrapGuard.SeedPattern accepted only one scheme segment, so
akka.ssl.tcp seeds parsed as nothing and the guard would silently never
engage on a TLS pair; it now accepts any number.
Default HOCON is byte-identical: 12 rendered documents (3 downing
strategies x guard on/off x Central/Site) captured before the change and
diffed after, all identical; a permanent test pins that a disabled block
with every field set renders the default document. A real ActorSystem
booted on the emitted TLS document comes up as akka.ssl.tcp (mutation
check: forcing enable-ssl=false turns it red).
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Adds LeaderAutoDowningProvider (Host/Actors), a line-for-line port of Akka.NET
1.5.62 AutoDowning/AutoDown/AutoDownBase read from the decompiled assembly: the
leader among the reachable members downs a peer unreachable for StableAfter; a
non-leader records it as pending and downs it on becoming leader; reachable-again
or removed members are forgotten; Down/Exiting members are skipped. Its window is
carried under scadabridge.cluster.leader-auto-down.unreachable-after, so no
deprecated auto-down-unreachable-after key is emitted and Akka's boot-time
"keep-majority will be used instead" warning goes away. DownRemovalMargin reads
akka.cluster.down-removal-margin exactly as AutoDowning does (default off).
New SplitBrainResolverStrategy value "leader-auto-down", wired through BuildHocon
and ClusterOptionsValidator. The default stays "auto-down"; auto-down and
keep-oldest HOCON is byte-identical to before. Placed in Host, not
ClusterInfrastructure, per the Code Placement rule (ClusterInfrastructure stays
Akka-free).
Tests: HOCON + validator units; LeaderAutoDownTests ports Akka's AutoDownSpec
cases (leader-only, pending-on-leader-change, losing leadership, reachable again,
removed, Down/Exiting skipped, single Down per member); LeaderAutoDownFailoverTests
clones SbrFailoverTests' oldest-crash and younger-crash cases on a real two-node
cluster plus a test that Akka actually loaded the provider and its downing actor.
Mutation-checked: a provider that never calls Cluster.Down fails both failover
tests; never-downs, leader-check-dropped, reachable-ignored and pending-not-flushed
mutations each fail the unit tests.
Remaining gate before switching the default: the docker failover drill
(DRILL_MODE=active and standby mode) on leader-auto-down.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Component-AuditLog Security & Tamper-Evidence describes the per-month chain,
the chaining job and its lock, both purge paths (checkpoint anchor,
tombstones), pre-existing rows, verification and what an unsigned chain does
not prove. alog.md §11.4/§16, CLAUDE.md, HighLevelReqs AL-11/12,
Component-CLI, Component-ManagementService, docs/components/CLI.md and the
CLI README drop the "no-op placeholder" wording.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
verify-chain was a no-op stub that printed "not enabled" and exited 0 without
contacting the server. It now sends VerifyAuditChain(Month) over
POST /management and maps the verdict to the exit code: 0 intact or not
chained, 3 broken (first break: partition, ChainSeq, EventId, kind), 1 error,
2 unauthorized. Table and JSON renderings.
ManagementActor: new HandleVerifyAuditChain arm, gated on the OperationalAudit
roles (Administrator, Viewer) and refusing a site-scoped caller because a
month's chain spans every site; INVALID_ARGUMENT for a bad month, CHAIN_BUSY
when the chain lock is held; long-running Ask timeout. Result enums serialise
as names.
Parity: RequiredRoleMatrixTests freezes the gate; ManagementSurfaceParityTests
now probes `audit verify-chain` (the one audit leaf that is a management
contract) instead of skipping the whole audit group.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Audit roadmap T1 (alog.md §11.4): one chain per UTC calendar month of
OccurredAtUtc. Each chained row carries ChainSeq (gap-free within its month)
and RowHash = SHA-256(prev.RowHash || canonical(row)); the v1 canonical form
(AuditChainHasher) is pinned by a known-answer vector computed independently.
- Migration AddAuditLogHashChain: nullable ChainSeq/RowHash on dbo.AuditLog
(metadata-only add), AuditLogChainHead (per-month chain end + partition-purge
checkpoint anchor), AuditLogChainTombstone, two partition-aligned filtered
indexes, scadabridge_audit_chainer role with a column-level UPDATE grant.
Production script docs/plans/sql/AddAuditLogHashChain.sql.
- AuditLogHashChain chains unchained rows oldest month first under an
exclusive sp_getapplock (zero timeout, so a dual-active central pair still
yields one chain), with an optimistic head check and a ChainSeq IS NULL
row-count check that roll a pass back instead of leaving a gap. The ingest
path is untouched.
- Purges keep the chain verifiable: the partition switch records the month's
checkpoint anchor and drops its tombstones; the per-channel DELETE writes a
tombstone per chained row in the same transaction (OUTPUT INTO). Both, and
the SourceNode backfill, hold the chain lock; the backfill skips chained
rows. The switch staging table gains the two new columns.
- AuditChainVerifier reports the first break (partition, ChainSeq, EventId,
kind) for edited, deleted, reordered rows and tail truncation.
- AuditLogHashChainService (central, default OFF via AuditLog:HashChain).
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Verified the ManagementActor bulk attribute override and single/bulk native alarm source
override paths match names ordinally and already reject a case-variant name (nothing stored);
the error now names the single case-insensitive match. Tests pin the rejection.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
(a) Inbound API compile state (#75) is per central node: a recompile cleared the
known-bad record only on the node that answered, and the peer kept refusing the
unchanged script with SCRIPT_COMPILE_FAILED (the per-request revision check only
notices a changed script). Create / update / delete (ManagementActor post-commit),
RecompileApiMethod, and API methods written by a Transport bundle import now publish
InboundApiMethodInvalidated on the central cache-invalidation topic (#100 item 3).
The new InboundApiCacheInvalidationHandler drops that method's compiled handler,
known-bad record and cached ApiMethod row on the peer, which compiles the current
text on its next request.
(b) docs/components/InboundAPI.md no longer claims a direct DB edit stays stale
until restart: the per-request revision check picks it up once the node's cached
row expires (ApiMethodCacheTtl, 5 s). Component-InboundAPI, Component-Host and the
#85 cache audit describe the peer invalidation.
(c) Decision: the import's locked-override warnings keep EXACT (ordinal) name
matching, because FlatteningService matches override names ordinally. An override
differing from a locked member only by case never reaches that member's lock check
(the flattener drops it as unresolved), so a "targets a LOCKED member" warning for it
would be false. The comparer is now named (LockedOverrideNameComparer) and documented,
and pinned by tests on both sides (flattener + import).
Tests: the peer's known-bad record is cleared by a recompile (and validate leaves it
alone); create/update/delete and the import publish the invalidation; the handler
drops only the named method; composition root registers the handler; case-only
override gets no lock warning; the flattener matches override names exactly.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
The #96 site-psk-invalidation topic was the first cross-node invalidation broadcast.
It is generalised into ONE central DistributedPubSub topic, central-cache-invalidation,
carrying typed messages (Commons ICentralCacheInvalidation: SitePskInvalidated,
RoleMappingsInvalidated, ExternalDefinitionsInvalidated, InboundApiMethodInvalidated)
in an envelope stamped with the publisher's process-unique origin id.
- Host: CentralCacheInvalidationBus (ICentralCacheInvalidationPublisher, no-op until
the mediator is set, never throws) and CentralCacheInvalidationSubscriber, which
skips its own node's envelopes and hands the peer's to every registered
ICentralCacheInvalidationHandler (a failing handler does not stop the rest).
Replaces SitePskInvalidationSubscriber.
- Migration of #96: SitePskRotationInvalidator publishes through the bus and is the
PSK handler; SitePskInvalidated keeps its type name and is also published bare on
site-psk-invalidation, which the subscriber still listens to, so a mixed-version
central pair keeps exchanging PSK invalidations. Remove after one release.
- Audit D1: RoleMappingCache is a handler; ManagementActor's post-commit role-mapping
invalidation also publishes RoleMappingsInvalidated, so a revocation is immediate on
the peer instead of after RoleMappingCacheSeconds.
- Audit D3 / #83 (peer half): ExternalDefinitionLookupCache is a handler; the 9
ManagementActor handlers and the Transport bundle importer publish
ExternalDefinitionsInvalidated after commit. (#83's local import half was already in.)
TTLs remain the fallback for a lost broadcast or a direct SQL edit. Tests: bus and
subscriber unit tests (origin filter, legacy topic), peer-node tests through the real
caches in ManagementService.Tests, importer publish, composition root, and a real
two-node cluster from the production HOCON (CentralCacheInvalidationTwoNodeTests)
proving every message type crosses with the production serializer. Docs: Host,
Communication, Security, ManagementService, ESG, and the #85 cache audit.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
The bulk SetInstanceNativeAlarmSourceOverridesCommand handler always wrote the
entry's IsMesReportableOverride, so a CLI import of a legacy 4-column CSV (no
MesReportable column) reset every imported source's MES override to inherit. The
Central UI avoided it only by looking up the stored flag client-side.
NativeAlarmSourceOverrideEntry gains an additive trailing KeepStoredMesReportable
flag: when set, the server leaves an existing row's MES override untouched (a new
row inherits) and ignores IsMesReportableOverride; unset keeps the old meaning
(null = clear to inherit), so older senders are unaffected. The CLI and the Central
UI now build their entries through one rule (ToOverrideEntry on each row type):
no MesReportable column => KeepStoredMesReportable. CLI README and
Component-CentralUI updated.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Deployments free-text search and the config-audit EntityName filter no longer
emit LIKE '%term%' by default. Prefix (LIKE 'term%') is the default; substring
is an explicit opt-in (DeploymentSearchMode.Contains / entityNameContains)
always bounded by a DeployedAt/Timestamp window (explicit, else 30 days).
Migration PrefixSearchIndexes adds DeployedBy, RevisionHash and EntityName
indexes. Deployments page gains a 'Match anywhere' checkbox.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Direction B: the same FlowGraph steps as a filterable list (lane column, lane filter,
failures-only), toggled from the flow panel toolbar. The mockup canvas is not in the repo,
so this is the minimal faithful version.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
CLI import no longer has to reset MES overrides to inherit; a legacy 4-column file
still parses and the Central UI import keeps the stored flag for it.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
The data plane answers a broken inbound method with the generic "Script
compilation failed for this method"; the Roslyn diagnostic reached only the
central log, and the known-bad cache could not be observed or reset per method.
- InboundScriptExecutor keeps the diagnostics with each known-bad record and
exposes GetCompileState(name, currentScript): IsCompiled / IsKnownBad /
LastCompileError, judged against the current script text (a record for older
text is reported as not compiled yet).
- Management: ValidateApiMethodCommand (compile the stored script, return
ApiMethodCompileResult, register nothing, cache untouched) and
RecompileApiMethodCommand (CompileAndRegister without re-sending the entity,
drops the node's ApiMethodCache row, audited "Recompile"). Both Designer, like
UpdateApiMethod; dispatched through the shared ApiMethodCompileOperations.
GetApiMethod keeps the entity fields top-level and adds isCompiled,
isKnownBad, lastCompileError and compileStateNode. Compile state is per
central node and every result names the node.
- CLI: api-method validate --id, api-method recompile --id.
- Central UI: the API-method edit page shows the saved script's compile state
and last error, with Validate and Recompile actions (same operations, same
Designer gate, fresh DI scope).
- RequiredRoleMatrixTests entries; ManagementSurfaceParityTests covers the new
CLI leaves; ApiMethodCompileDiagnosticsTests + an executor CompileState test.
- Docs: CLI README, Component-CLI/ManagementService/InboundAPI, components/*.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
The reconnect:false case sent its writes straight after ActorOf and gave each
reply a fixed 1 s, so the window also covered the actor's start and first
dispatch. A slow start (cold JIT, starved pool in a full solution run) missed
it. The reconnect:true case was shielded by its 3 s wait for Connected.
The test now waits until the connect attempt the writes race against has been
issued (ConnectAsync entered, held open by the gate), so the actor is provably
mid-attempt and cannot connect before the test releases it. Reply bounds are
10 s barriers, not part of the property: a write that was queued instead of
rejected can only be answered after the gate opens, which never happens first.
Evidence: with a 1.2 s slow actor start modelled (UpdateConnectionHealth
Connecting sleeps), the old body passed 0/30 and the new one 30/30. Unmodified,
the new test passed 40/40 under 14 busy-loop CPU load. Mutations still caught:
Connecting dropping writes (timeout) and Connecting performing writes.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Item 2 of #100, following the #93/#95 pattern (ToNameLookup: OrdinalIgnoreCase,
first wins, never throws):
- Overwrite child sync matches template attributes, alarms, scripts and native
alarm sources, and external-system methods, case-insensitively, like their
(parent, Name) unique indexes under SQL Server's collation. A member that
differs only in case is updated in place instead of deleted and re-added
(which lost the row id and could collide with the unique index).
- ResolveAlarmScriptLinksAsync resolves the alarm and its on-trigger script
case-insensitively; AlarmsEqual compares the script name the same way.
- ArtifactDiff (preview) matches child collections case-insensitively so it
agrees with the sync; notification recipients keep the ordinal match (no
unique index, replaced wholesale).
- The rewire passes found the template a DTO was imported onto with an
ordinal index. Since #93 an Overwrite can match a differently-cased target and
keeps its spelling, so the passes silently skipped it and the alarm on-trigger
links the sync had cleared stayed null. TryGetImported adds a case-insensitive
(base-first) fallback; composed/base template references keep the exact
lookup (item 1 of #100 is still undecided).
Test: ImportNameLookupTests.Members_inside_a_record_match_case_insensitively,
over a proxy that gives the by-name template reads SQL Server's collation.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
New AuditChannel.Security with AuditKind.CertTrust / CertTrustRemove (appended, ordinals stable).
Both central entry points - the management actor handlers and the Central UI CertManagementService -
write a best-effort ICentralAuditWriter row after the site outcome is known (Delivered when all
reachable site nodes acked, Failed otherwise, incl. relay timeouts), naming actor, site, thumbprint
and connection. Shared builder: CertTrustAuditEvents.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
TemplateNativeAlarmSource.IsMesReportable + tri-state InstanceNativeAlarmSourceOverride
.IsMesReportableOverride flatten onto ResolvedNativeAlarmSource; NativeAlarmActor stamps every
mirrored condition and InstanceActor the quiet-binding placeholder, so ScriptAlarm,
Alarms.MesReportableAsync() and the #60 wire fields carry it. Additive to the 900-999 severity band.
Migration AddNativeAlarmSourceMesReportable. Management commands, CLI --mes-reportable, Central UI
(template dialog checkbox, instance override select), Transport DTOs/importer/diff and the revision
hash (true-only, so existing hashes are unchanged) carry it.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
DataConnectionGaugeTests and ExtendedInstrumentTests both mutate the process-global data
connection registry; put them in a non-parallel xUnit collection. Add docs/operations/metrics-reference.md.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Fixes the SQLite connection-pool double-handout (dotnet/efcore#39009). EF Core family,
AspNetCore 10.0.7 packages and Microsoft.Data.SqlClient (6.1.1 -> 6.1.7, EF SqlServer floor)
move in lockstep to avoid NU1109 downgrades.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
- Preview matches template folders, shared scripts, external systems, SMTP
(by Host) and SMS (by AccountSid) with the #93 ToNameLookup comparer
(OrdinalIgnoreCase, first wins, never throws), matching SQL Server's
case-insensitive collation, so a differently-cased target row is reported as
a conflict instead of New.
- Apply-side in-memory lookups do the same: ApplyTemplateFoldersAsync, the
template folder-id map, ResolveFolderParentEdgesAsync, ApplySmtpConfigsAsync
and ApplySmsConfigsAsync. An Overwrite now updates the existing row instead of
adding a duplicate (folder names are only unique per parent, so the old
ToDictionary could also throw).
- New ITemplateEngineRepository.GetBaseTemplatesWithChildrenAsync filters
IsDerived in SQL. Preview and the Overwrite load in ApplyTemplatesAsync use it,
so a derived slot template that shares a bundled name is never the import
target. GetTemplatesWithChildrenAsync is unchanged for the id-keyed callers
(export closure, rewire passes), which can need derived rows.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
SitePskProvider cached each site's gRPC preshared key forever; only site removal
invalidated it, so rotating SB-GRPC-PSK-{site} broke the central<->site link on
both central nodes until they restarted. The cached key is now dropped when:
- the secret is written through /admin/secrets: central replaces the secrets
library's ISecretCacheInvalidator with SitePskRotationInvalidator, which also
publishes SitePskInvalidated on a new DistributedPubSub topic
(site-psk-invalidation) so the peer central node drops its copy and its
resolver cache entry too (SitePskInvalidationSubscriber);
- a site rejects a central->site call with PermissionDenied/Unauthenticated:
WithSiteCredentials wraps every per-site channel's handler in
PskRejectionRefreshHandler, so the next call re-reads the key (covers
out-of-process writes such as the secret CLI);
- a site->central call presents a token that does not match the cached key:
CentralControlAuthInterceptor invalidates and re-reads once before refusing;
- ScadaBridge:Communication:SitePsks reloads.
Invalidation is case-insensitive because SecretName lowercases names. Rotation
runbook (topology-guide), Component-Communication/Host and the #85 cache audit
(D2) updated.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
The site script and alarm on-trigger launchers push the run identity onto LogContext for the
run's lifetime and the host now enables Enrich.FromLogContext(), so log lines written on the
run's own async call chain carry the id. Bounded part only: LogContext does not cross an Akka
message hop, so Instance Actor/DCL/S&F lines and a site_events execution-id column remain open
(documented in the step-logs design doc section 9).
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Adds four instruments on the already-allow-listed ZB.MOM.WW.ScadaBridge meter:
scadabridge.deployments.failed{reason}, scadabridge.store_and_forward.activity{action,category},
scadabridge.site.script.runs{outcome} + scadabridge.site.script.duration{outcome} (ms),
scadabridge.site.data_connections.transitions{state}. Queue depth, DCL connection state and
deploy-applied already existed. No new Meter, so ZbTelemetryOptions.Meters needs no change.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Replace the private SitePairActiveNodeHealthCheck (503 whenever no ActorSystem, no data)
with the shared ZB.MOM.WW.Health ActiveNodeHealthCheck constructed with the site-{SiteId}
role: Degraded (200) during startup per the Health spec, and it publishes
selfAddress/activeRole/activeNode so the Overview dashboard can name the active site node.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Audit (docs/plans/2026-11-09-central-cache-staleness-audit.md) of every central in-process
cache holding configuration or secret state, with its staleness bound on the peer central
node. Every config cache with peer staleness is TTL-bounded (5-60 s) and Traefik routes to the
active node only, so no peer case was clearly a bug and no cross-node broadcast is added; the
doc names the existing DistributedPubSub mediator (health replication topic) as the mechanism
if one is wanted. Owner decisions listed: RoleMappingCache 30 s revocation lag on the peer,
SitePskProvider caching a PSK forever across an in-place rotation (both nodes), and the #83
ExternalDefinitionLookupCache 10 s bound.
Fixed: ManagementActor API-method create/update/delete invalidated only the compiled handler,
not ApiMethodCache, so the node that made the change kept serving the pre-change row for up to
the 5 s TTL (a deleted method stayed callable; a cached not-found hid a new one). The handlers
now drop the entry post-commit, like the role-mapping cache.
Tests: ApiMethodCacheInvalidationTests (create/update/delete fail without the fix; a
negative control checks other methods stay cached).
(ScadaBridge#85)
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
The InboundAPI carried its own IActiveNodeGate (implemented by Host/Health/ActiveNodeGate)
whose rule already matched the shared Health 0.3.0 AkkaActiveNodeGate — both route to
ClusterActiveNode (oldest Up member). The private interface and implementation are deleted;
InboundAPI references ZB.MOM.WW.Health (0.3.0, already pinned) and InboundApiEndpointFilter,
AuditWriteMiddleware, InboundMethodWarmupService and FlowDebugRunner consume
ZB.MOM.WW.Health.IActiveNodeGate. The Central branch of Program.cs registers
AkkaActiveNodeGate (unscoped, like the active-node health check).
Warm-up behaviour is kept: the shared gate answers false while no ActorSystem resolves, while
this member is not Up, and on any cluster-init fault. It resolves the ActorSystem through the
existing DI singleton bridge (GetOrCreateActorSystem), the same path the health probes use.
Tests: CompositionRoot asserts the shared gate type (fails with the registration removed);
the evaluator wiring test now drives AkkaActiveNodeGate; new warm-up test (no ActorSystem ->
not active). Nothing was missing from the Health 0.3.0 package.
(ScadaBridge#80)
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
`health pending-updates` (#63) called IPendingUpdatesEvaluator.EvaluateAsync directly
from ManagementActor because the cache lived in CentralUI, so every CLI call was a full
uncached fleet flatten. IPendingUpdatesCache/PendingUpdatesCache now live in
DeploymentManager.PendingUpdates and are registered by AddDeploymentManager; the
SingleFlightMemo<T> primitive moves to Commons.Caching (public) so both CentralUI's KPI /
alarm-summary caches and DeploymentManager can use it. HandleGetPendingUpdates reads the
shared cache; the UI circuits and the CLI now share one memo per node.
Tests: ManagementActor repeated-calls test (fails against the direct evaluator call:
2 evaluations instead of 1), AddDeploymentManager singleton-registration test; the cache
and memo tests move with their types.
(ScadaBridge#91)
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
- ApplyTemplatesAsync/PreviewAsync template lookups use a case-insensitive,
never-throwing comparer (SQL Server collation); base template wins over derived.
Base names are already unique case-insensitively via the filtered unique index
on Template.Name, derived templates are not covered by it.
- Preview reads database connections, notification lists and API methods with one
GetAll query each instead of one by-name query per bundled row.
- Component-TemplateEngine.md: the flattener does not call GetAllTemplatesAsync;
it loads chains via GetTemplateWithChildrenAsync.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Under a loaded full-project run the serial prepare of 6 instances with a 300 ms
reconcile deadline took 2.6 s and tripped the fixed 2 s cap (5/10 full runs
failed under CPU load). Deadline raised to 2 s so it cannot expire before the
query is sent; the elapsed bound is now derived from the deadline (must beat
(n-1) deadlines) while the one-query assertion proves the breaker.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Measured first: a scratch scaling probe (seed -> export -> load/preview/apply
into a fresh target and over itself, in-memory provider, every repository
call counted, DetectChanges counted via ChangeTracker.DetectingAllChanges)
showed load, preview and the instance pass scale linearly (500 / 1000 / 2000
instances apply in 0.13 / 0.27 / 0.35 s), but the post-flush template
rewire passes were quadratic: 100 / 400 / 800 composing, derived templates
applied in 0.5 / 8.1 / 26.5 s, with DetectChanges running 5N+10 times and
GetTemplateWithChildrenAsync 2N+2 times.
Cause: ResolveAlarmScriptLinksAsync, ResolveCompositionEdgesAsync and
ResolveInheritanceEdgesAsync found each template with
_dbContext.Templates.Local.FirstOrDefault(...) - per template, and per
composition edge / base lookup. Every DbSet.Local access runs a full
DetectChanges over everything the import has staged. The first two passes
also re-read every template's children one query at a time (a split query
per call on SQL Server).
Fix (no bundle-format or semantic change):
- each pass indexes the tracked templates by name once
(SnapshotLocalTemplatesByName; ordinal, first wins like FirstOrDefault).
These passes never stage a template or rename one, so the index stays
exact for the pass;
- the alarm and composition passes read their templates' children in one
batched GetTemplatesWithChildrenAsync, matched back by the stub's id;
- the alarm pass only loads templates whose bundled alarms name an
on-trigger script (the only ones it can rewire).
Result on the same probe: 100 / 400 / 800 templates apply in
0.13 / 0.62 / 0.64 s; DetectChanges is a constant 9; per-template children
reads 0, batched reads 1 (add) / 2 (overwrite).
Evidence: ImportApplyScalingTests (Transport.IntegrationTests) exports 5 and
40 composing, derived templates with alarm -> script links, imports each
into an empty target (Add) and again (Overwrite), and asserts the imported
inheritance / composition / alarm-script links, zero per-template children
reads, at most 3 batched reads, and an identical DetectChanges count for 5
and 40 templates. Fails on the old importer (12 per-template reads), and
with a per-DTO Local snapshot reintroduced in one pass (DetectChanges 51
vs 16). Transport unit (198) and integration (138) suites pass.
Not changed: preview's per-name lookups for database connections,
notification lists and API methods stay per-name - moving them to an
in-memory ordinal match could change name matching under a case-insensitive
SQL collation.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Joseph Doherty