Owner decision 2026-11-15, at IT security's request, pending a family-wide general solution.
Removed: the chaining service, hasher, verifier, chain head/tombstone entities and store, the #139
chain-start cutoff, AuditLog:HashChain (incl. ChainBacklog and HeadExport with its File/Syslog sinks
and export service), the audit-chain-head-export health check, the VerifyAuditChain /
VerifyAuditChainHeads management commands and the `audit verify-chain` / `audit verify-heads` CLI
commands, and the two-login plumbing (AuditLog:MaintenanceConnectionString, AuditMaintenanceContext,
the keyed audit-maintenance repository/archive). Purge, archive, retention, partition roll-forward
and backfill-source-node all run on the runtime connection again, as before #107.
Parquet archive (#42) kept, chain-free: format v2 (ten stored columns, no tombstone file, manifest
without chain state), the switch guard is the row count only (51003 gone), verify = data-file size,
SHA-256 and row count, and superseded-stale cleanup proves coverage by EventId. Format-v1 archives
still list, verify and are cleaned up (their tombstone file too).
Migration RemoveAuditLogHashChain drops the chain indexes, the scadabridge_audit_chainer role
(members first), both chain tables, ChainSeq/RowHash and the purger's AuditMaintenanceGrants grants;
writer/purger roles stay. Down restores them. Every chain-column reference is EXEC-deferred for the
idempotent script (#126); FlipAuditLogClusteredKey's two chain index statements get the same
deferral, behaviour unchanged, so re-running the idempotent script on a migrated database works.
The partition-switch staging table adds the chain columns only when the live table has them, so the
new binary purges either schema. Deploy gate: old binaries map the chain columns, so the migration
ships with the new binaries, both centrals stopped.
A leftover AuditLog:HashChain section or MaintenanceConnectionString (DEV/QA still carry
AuditLog__HashChain__Enabled=true) is ignored with one startup warning (RetiredAuditSettingsWarning).
Claude-Session: https://claude.ai/code/session_01GT67AFKC45MBz1wPZnVbX8
Owner decision on #107 (2026-11-13): enabling AuditLog:HashChain must not
chain the historical backlog. main chained every unchained row oldest-first.
- The first chaining pass on a pristine database (no head, no chained row)
writes a chain-start anchor: AuditLogChainTombstone (cutoff month,
ChainSeq 0), PurgedAtUtc = DB SYSUTCDATETIME(), RowHash =
SHA-256("ScadaBridge.AuditLog.ChainCutoff.v1" || YYYY-MM || ticks).
Written once under the exclusive chain lock, so both centrals agree
whatever their clocks/config. No migration: existing table + grants.
- Only rows with OccurredAtUtc >= cutoff are chained. The anchor hash is
the cutoff month's genesis, binding the cutoff into the chain.
- AuditLog:HashChain:ChainBacklog (default false) restores chain-everything,
read only on the very first enablement. Legacy chains (heads, no anchor)
keep chaining everything.
- verify-chain splits unchained rows into UnchainedRows vs PreCutoffRows
(+ ChainFromUtc); new breaks ChainedBeforeCutoff (row re-dated before the
cutoff, or cutoff moved forward) and CutoffAnchorInvalid (anchor hash /
month / multiplicity). Archive verify uses the anchor from its own
tombstone file or the DB.
- Partition switch-out keeps the anchor (deletes ChainSeq > 0 only); the
head store / export / verify-heads tombstone count excludes it.
- CLI verify-chain table prints Chain from + Pre-cutoff rows.
Tests: AuditLogHashChainCutoffTests (7, MSSQL), 6 verifier unit tests,
service/CLI/management wire tests; 9 mutations all killed.
Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
Prerequisite for OtOpcUa Warm redundancy (lmxopcua#717): a Warm standby keeps a
healthy session but serves BadOutOfService at a standby ServiceLevel (90/50, boot 1),
so a connection that lands there sat on Bad quality forever.
- OPC UA connections with a backup poll Server_ServiceLevel (i=2267) on the connected
endpoint (IServiceLevelReadable, implemented by OpcUaDataConnection; never raises
Disconnected). A level < 200 sustained for the degraded window (default 10 s), or
the first sample after any (re)connect, triggers a probe of the other endpoint
(short-lived session: connect, read, dispose). Switch only when it reports a
Healthy (>= 200) higher level, and not within the minimum dwell (default 60 s) of
the previous ServiceLevel switch. Both degraded, other unreachable or level
unknown: stay put. 250/240 (today's OtOpcUa) never starts a streak.
- The switch reuses the failover path: SwitchToOtherEndpoint() now shared by all
three triggers, then BecomeReconnecting(ReconnectCause.ServiceLevelFailover) -
zero delay, no unstable accounting, ReSubscribeAll on the new endpoint.
- Config: OpcUaEndpointConfig.ServiceLevelFailover {enabled, degradedWindowSeconds,
minDwellSeconds}, read from the PRIMARY endpoint's JSON only; null = defaults
(enabled, 10, 60). Inside the encrypted protocol JSON, so no schema change or
migration. Central UI: fields in the Backup card next to Failover Retry Count.
Global: ScadaBridge:DataConnection:ServiceLevelPollInterval (2 s) and
ServiceLevelProbeTimeout (15 s).
- Tests: actor tests per trigger/non-trigger with positive controls, an unsolicited
promotion RefreshStart/RefreshEnd test at client and actor level, adapter tests,
and a real in-process OPC UA server pair end to end. All 15 mutants killed.
- Docs: Component-DataConnectionLayer, components/DataConnectionLayer, Component-CLI.
Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
AlarmConditionState.Severity mixed two conventions: OPC UA rows carried Part 9
Severity (higher = more severe) while MxGateway native alarms and computed
alarms carried a raw AVEVA-style priority (1 = most urgent). Every consumer
ranks higher as worse, so Worst Severity, the severity filter/sort and the
site event-log Error threshold picked the LEAST urgent MxGateway/computed
alarms.
- AlarmPriorityBands: the OtOpcUa GalaxyAlarmPriority table (lmxopcua#887,
b61f3e05), value for value: 1-250 -> 1000, 251-500 -> 750, 501-750 -> 500,
751-999 -> 250; <= 0 clamps to Critical, > 999 to Low.
- MxGatewayAlarmMapper bands the gateway severity (= AVEVA priority);
AlarmConditionStateFactory.ForComputed bands PriorityLevel. OPC UA unchanged.
- AlarmConditionState gains trailing Priority (int?): the raw source number,
exactly the pre-#129 Severity. AlarmStateChanged.Priority and the wire
AlarmStateUpdate.priority keep that meaning (no proto change); central derives
Severity by kind (SeverityFromPriority), so an older site decodes correctly.
- ScriptAlarm gains trailing Priority. The native MES 900-999 band and the MES
API Severity/MinSeverity/MaxSeverity key off Priority; rig seed scripts moved
from a.Severity to a.Priority, so MES output is unchanged.
- AlarmActor raise logs Error when the banded severity >= 700 (same threshold
as NativeAlarmActor): priority 1-500 Error, 501+ Warning (was raw >= 700).
- NativeAlarmActor upgrades pre-#129 native_alarm_state rows on rehydrate.
- Debug snapshot decode restores Condition.Priority from the row's priority.
- Docs: Commons, DCL, SiteRuntime, Communication, CentralUI, InboundAPI,
HighLevelReqs, MES alarm-status plan, CLAUDE.md.
Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
#120: Akka ILoggingAdapter lines used to bypass Serilog and go to stdout only.
- BuildHocon sets akka.loggers = [Host.Actors.SerilogAkkaLogger] and keeps akka.loglevel = INFO,
so Akka Debug chatter is still filtered at the adapter. The INFO+ lines that used to reach
stdout now go through the Serilog pipeline instead, with no extra volume.
- Lines reach the file sinks with the node enrichers. SourceContext is the logging class's full
name, AkkaSource is the actor path, Akka's own timestamp is kept, and format arguments become
structured properties.
- The Akka logger actor writes on its own thread, after the logging actor's LogContext is gone.
To keep the ExecutionId, the two DCL actors log through ExecutionAwareLogging.GetLogger(Context),
which records ExecutionLogContext.Current (a new AsyncLocal mirror of the pushed id) in an
ExecutionTaggedLogMessage. The logger actor pushes that id again while it writes the line.
#56 remainder (coordinator decision: no replicated schema change):
- The host pipeline gains ExecutionLogCaptureSink.
- While a run is traced, its step logger registers its ExecutionId: the site ScriptLogger from the
launcher (plus each inline CallShared child id), and the central InboundScriptLogger from the
executor.
- Each tagged line becomes a ScriptLog row with LogSource "node" and the new
ScriptTraceExtra.LogCategory.
- The rows count against the existing MaxLogLinesPerRun cap, are clipped at 2 000 chars, ride the
low-priority collector lane and pass the same redactor. Untraced runs cost one volatile read.
- Lines are never copied twice. The step loggers' own forwards run inside
ExecutionLogCapture.Suppress(), and the sink suppresses itself while it calls a target. The
registration is declared inside the launcher's and executor's try, so it ends before the catch
blocks log the outcome, and FlushDropped closes it.
- FlowGraphBuilder lists node lines in FlowGraph.NodeLogs and never folds them onto a step.
- The Flow Debugger shows them in a Node log card (FlowNodeLogPanel): grouped per execution under
its run step, with an All | Warn + filter. Live and Replay both read the same rows, so site
scope and role rules apply unchanged.
- CLI --show-logs prints them as a trailing block.
Tests: Commons ExecutionLogCaptureTests, ExecutionLogContext.Current, FlowGraphBuilder node lines.
SiteRuntime NodeLogCaptureTests covers cross-actor capture via the launcher, untraced runs
capturing nothing, no double capture of script and runtime lines or of the launcher failure line,
the shared cap, collector redaction, shared-script child ids and FlushDropped closing capture.
InboundNodeLogCaptureTests covers the central path. DCL ExecutionAwareLoggingTests covers the
DataConnectionActor write-path Debug line being tagged and an uncorrelated line not being tagged.
Host SerilogAkkaLoggerTests covers mapping, the ExecutionId crossing the logger hop, HOCON,
ActorSystem boot and the Akka line reaching a traced run's step log end to end. Host
LoggerConfigurationTests checks that the sink is wired. bUnit FlowNodeLogPanelTests and a CLI
projection test cover the UI and CLI. Every new assertion was mutation-proven.
Docs: step-logs design section 12, Component-SiteRuntime, CentralUI, InboundAPI, Host, CLI README,
CLAUDE.md.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
LogContext is an AsyncLocal and does not cross an Akka message hop, so the run's id now rides the
message: Commons.Observability.IExecutionCorrelated (explicit interface member, no wire change) on
SetStaticAttributeCommand, AttributeValueChanged (not for an inferred writer), ScriptCallRequest and
the four RouteTo*Requests, plus a new additive SourceExecutionId on GetAttributeRequest,
WaitForAttributeRequest, WriteAttributeBatchRequest, WriteTagRequest and WriteTagBatchRequest.
ScriptRuntimeContext stamps its id on the requests it sends, the Instance Actor hands it on to the
DCL, and the Instance, Script, Alarm, Deployment Manager and both DCL actors re-push it per message in
AroundReceive (ExecutionLogContext.PushFor: one type test when the message names no execution). The
inbound executor pushes the request's ExecutionId for the method execution.
Tests (TestKit): Instance Actor line carries the message's id, no leak from the sender's ambient
context, scope ends with the message, per-write ids inside a batched SetAttributes, ids forwarded on
DCL writes, ScriptRuntimeContext stamps; DCL adapter lines on single/batch writes carry the id;
inbound executor lines carry the request id; Commons mapping + no serialized member. Each mutation-
proved. Still open on #56: a queryable per-execution sink and a Flow Debugger section. The DCL
actors' own Akka _log lines bypass Serilog (filed #120).
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
#113: AuditLogPartitionMaintenance (monthly SPLIT RANGE) and the
backfill-source-node endpoint now use the AuditLog:MaintenanceConnectionString
login when set (unset = unchanged). Migration AuditMaintenanceGrants gives
scadabridge_audit_purger ALTER ANY DATASPACE and column-level UPDATE (SourceNode).
#114: archive cleanup deletes the writer's own *.tmp files and manifest-less
data files older than 24 h inside {Directory}/{YYYY} (DeletedOrphan, dry-run aware).
#115: head-export failures bump scadabridge.audit.chain_head_export.failures and
Degrade the /health/ready entry audit-chain-head-export (failing or stale > 2 intervals).
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Owner decisions recorded: the archive destination stays a directory/UNC
(no object-storage client; share + permissions named at enablement),
parquet-tools-only reading (no re-import), unsigned like the chain (#107).
AuditLog:Archive:RetentionDays (default 0 = never delete; else must exceed
AuditLog:RetentionDays, counted from the archived month's end) and
DeleteSupersededStaleArchives (default true) run after every purge tick.
A retried purge's stale Archived archive is deleted only when a later,
verifying, Purged archive of the same partition provably holds every row:
same prior anchor and the stale chain-end hash at its chain-end sequence
(row or tombstone), every unchained row present by EventId; otherwise it is
reported KeptStale. A sole copy left at Archived is never touched. Deletion
removes only the archive name's own three files in its folder, manifest
first, never a manifest-supplied path. `audit archive cleanup [--dry-run]`
= CleanupAuditArchives (Administrator).
AuditLogArchiveCleanupTests (MSSQL) cover retried purge, kept stale copy,
sole copy, retention and the path guard; each safety rule mutation-checked.
Follow-ups filed: #113, #114, #115.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Item 1 (owner decision: no signing). AuditLog:HashChain:HeadExport (default
off): both centrals periodically read every month's chain head (LastSeq,
LastHash, purge anchor, chained rows, tombstones; one snapshot under the
chain lock SHARED) and append the changed heads to a sink the DB login cannot
write - File (append-only JSONL per node per day, local or UNC) or Syslog
(RFC 5424 over UDP / TCP octet counting). `audit verify-heads --path` reads
those records back (file or syslog-collector lines) and the new
VerifyAuditChainHeads command checks each against the database: Matches /
Extended / Purged, or Mismatch / Regressed / MissingHead / MissingLink /
CountMismatch (exit 3). AuditChainHeadStoreTests forges a chain with the
production hasher: verify-chain says Intact, verify-heads says Mismatch.
Item 3. AuditLog:MaintenanceConnectionString (${secret:}-capable, unset =
today's single login) runs the chain job, chain verify / head export, the
partition switch-out + archive and the per-channel purge on
AuditMaintenanceContext (keyed audit-maintenance repository/archive, purge
actor prefers them). AuditMaintenanceLoginTests runs the hardened setup on
SQL Server: writer (datareader/datawriter/audit_writer) ingests and is
refused chaining and deleting; the chainer+purger login does the full cycle.
Two-login grants documented in Component-AuditLog.md.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Variant B2 of the #78 benchmark: migration FlipAuditLogClusteredKey drops the
twelve non-clustered indexes, re-adds PK_AuditLog as (OccurredAtUtc, EventId)
on ps_AuditLog_Month, adds the aligned unique UX_AuditLog_EventId_Occurred
(EventId, OccurredAtUtc) for the idempotency probe, and rebuilds the twelve
indexes unchanged. Guarded (re-runnable) Up, inverse-rebuild Down; production
scripts docs/plans/sql/FlipAuditLogClusteredKey{,.rollback}.sql.
The partition-switch purge staging table now reads the live clustered key
order from sys.index_columns, so it switches either shape (old binaries
hard-code the old order: ship with the binaries, both centrals stopped).
Evidence: 300k rows, page 1 90,532 -> 34 logical reads (ordered backward
clustered scan under Top), keyset page 90,532 -> 46, probe 14 -> 14;
FlipAuditLogClusteredKeyMigrationTests pins indexes, grants, Down/Up, the
staging key (mutation: hard-coded old key fails SWITCH) and the read ratio.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Owner decision (b), 2026-11-09. PendingUpdatesEvaluator re-flattened every
deployed instance on each cache-TTL expiry even when nothing had changed.
- Migration AddConfigGraphRowVersions: a non-concurrency rowversion column
+ index on the 18 configuration tables the evaluation reads (Templates +
5 member tables, Instances + 4 override/binding tables, DataConnections,
DeployedConfigSnapshots, Sites, SharedScripts, ExternalSystemDefinitions,
ExternalSystemMethods, DatabaseConnectionDefinitions). SQL Server only in
the model; SQLite test contexts are untouched. Script:
docs/plans/sql/AddConfigGraphRowVersions.sql. Joins the coordinated
central-pair migration window (#107 item 4).
- IConfigGraphWatermarkReader / ConfigGraphWatermarkReader: ONE query of
MAX(RowVersion) + COUNT_BIG(*) per table (count catches deletes, which
raise no rowversion). rowversion is engine-bumped for any writer, so a
peer central node's edit moves it — not the per-process watermark MG-3
removed.
- The secret store (outside the DB) gets its own stamp: SHA-256 over
ListAsync(includeDeleted: true) metadata.
- Evaluator re-serves the last CLEAN snapshot while watermark + secret
stamp are unchanged; every PendingUpdatesFullEvaluationEvery'th (default
10, validated >= 1, 1 = off) is full regardless — the safety net for a
same-KEK re-wrap or an input table missing from the set. Always full when
the token is unavailable/unreadable, after an evaluation that could not
compute a site's artifact hash, on Re-check, and after a deploy completion.
It only re-serves a real earlier answer, never manufactures one.
Tests: SQL Server (real migrated DB) — token stable without writes, moves
for a peer-connection INSERT/UPDATE and a non-max DELETE, moves for a local
EF write, a detached Update still saves (not a concurrency token), and a
guard that every table the evaluator's repository reads touch is covered.
Evaluator — reuse without flatten (re-served answer still shows the stale
instance), watermark move, 4 secret changes, every-Nth, 5 no-token modes,
unclean evaluation not reused, cache Re-check and deploy completion force
full. Mutation-proven: token compare, clean check, cache force flag, row
count, and an uncovered read each turn tests red.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Exports can now carry a detached ECDSA P-256 signature over the exact
manifest.json bytes in a new optional manifest.sig zip entry. The manifest
already pins the content hash, so a verified signature covers the whole
bundle, including the plaintext source environment the confirm step relies
on. Off by default (ScadaBridge:Transport:Signing): with the defaults,
exports are unsigned and imports accept signed and unsigned bundles.
- The bundle format does not change: bundleFormatVersion stays 1,
manifest.json and the AES-GCM associated data are untouched, and older
importers open entries by name and ignore manifest.sig. Three entries fit
within the default MaxBundleEntryCount of 4.
- Import policy runs after the content-hash check and before any passphrase
work. A signature that fails to verify against a trusted key is always
refused. RequireSignedBundles additionally refuses unsigned bundles and
bundles signed by an untrusted key. A refusal is BundleSignatureException
(Commons), shown on the UI upload step and on CLI preview/import.
- The signature status, key id and signer are recorded on BundleSession and
shown in the import wizard. They are also returned on PreviewBundleResult
and recorded on the BundleImported audit row. BundleExported records
Signed/SigningKeyId.
- Key material is pluggable. IBundleSigningKeySource and IBundleTrustStore
are TryAdd'd; the defaults read Signing:PrivateKey (meant to be a
${secret:} reference) and Signing:TrustedKeys. Key ids are SPKI
fingerprints. Startup validation rejects unparseable or non-P-256 keys.
- Compatibility proof: the checked-in fixtures pre-signing-plaintext and
pre-signing-encrypted were exported by origin/main 3ff463b6, before this
change, and are imported end to end with signing off and with signing
configured.
The design leaves key-per-env vs per-org, trust distribution and rotation,
and the production default for RequireSignedBundles open. These are listed
in Component-Transport.md.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
GET /api/audit/export?format=parquet returned 501. It now streams the same 24
columns as csv/jsonl (AuditExportRow, CsvHeader order) as one Parquet file,
one 10,000-row row group at a time. Each row group is encoded into a small
non-seekable staging buffer and copied to the response asynchronously, because
Kestrel rejects the synchronous writes the Parquet writer issues.
New management commands, with CLI parity:
- ListAuditArchives -> `audit archive list` (AuditArchiveListResult: enabled
flag + every manifest).
- VerifyAuditArchive(Name) -> `audit archive verify --name` (files, chain,
purge anchor); exit 0 verified, 3 not verified, 1 error, 2 unauthorized.
Both carry verify-chain's gate: OperationalAudit roles (Administrator, Viewer),
site-scoped non-Administrator refused because an archive spans every site.
VerifyAuditArchive uses the long-running Ask timeout. Error codes
ARCHIVE_NOT_CONFIGURED, INVALID_ARGUMENT, NOT_FOUND.
RequiredRoleMatrixTests freezes both gates; ManagementSurfaceParityTests now
probes `audit archive list|verify` alongside `audit verify-chain`.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Audit roadmap T2 (alog.md §15.2): with AuditLog:Archive:Enabled (default off)
the purge actor writes each expired monthly partition to Parquet under an
absolute AuditLog:Archive:Directory before switching it out, so audit rows
outlive RetentionDays.
- Parquet.Net 6.1.0 (fully managed; NuGetAudit clean) in central package
management.
- AuditArchiveParquet: data file = the ten stored columns, OccurredAtUtc as a
UTC microsecond timestamp plus exact OccurredAtUtcTicks, and ChainSeq/RowHash;
tombstone file = the month's per-channel purge tombstones. Buffered row
groups; the files are plain Parquet (DuckDB/pyarrow read them).
- AuditLogPartitionArchive (IAuditLogArchive): reads the partition's rows,
chain heads and tombstones in one transaction under the chain lock SHARED,
writes {Directory}/{YYYY}/auditlog-{YYYY-MM}-{stamp}Z.parquet,
.tombstones.parquet and a .manifest.json (counts, SHA-256 per file, each
month's head and prior anchor) via tmp + rename, manifest last. Archives are
never overwritten.
- Archive before purge: SwitchOutPartitionAsync gains an internal guarded
overload. Under the EXCLUSIVE chain lock it re-counts the partition and
re-reads each month's LastSeq and THROWs 51002/51003 (nothing purged) unless
both still equal the archive's, so the switch's checkpoint anchor is exactly
the archive's chain end. The manifest is then marked Purged.
- Fail closed: a partition that cannot be archived is not purged; the failure
takes the existing AuditLogPurgeFailedEvent / PurgeFailures path. Success
publishes AuditLogArchivedEvent.
- Verify: file hashes and counts against the manifest, the chain recomputed
from the archived rows and tombstones (AuditChainVerifier), and the chain end
against the database anchor (Matches / Superseded / NotPurged / Mismatch).
- AuditLogArchiveOptionsValidator: Directory required when enabled and must be
absolute; RowGroupSize, timeouts bounded.
No migration: the purger role already reads AuditLog, the head and tombstones.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
verify-chain was a no-op stub that printed "not enabled" and exited 0 without
contacting the server. It now sends VerifyAuditChain(Month) over
POST /management and maps the verdict to the exit code: 0 intact or not
chained, 3 broken (first break: partition, ChainSeq, EventId, kind), 1 error,
2 unauthorized. Table and JSON renderings.
ManagementActor: new HandleVerifyAuditChain arm, gated on the OperationalAudit
roles (Administrator, Viewer) and refusing a site-scoped caller because a
month's chain spans every site; INVALID_ARGUMENT for a bad month, CHAIN_BUSY
when the chain lock is held; long-running Ask timeout. Result enums serialise
as names.
Parity: RequiredRoleMatrixTests freezes the gate; ManagementSurfaceParityTests
now probes `audit verify-chain` (the one audit leaf that is a management
contract) instead of skipping the whole audit group.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Audit roadmap T1 (alog.md §11.4): one chain per UTC calendar month of
OccurredAtUtc. Each chained row carries ChainSeq (gap-free within its month)
and RowHash = SHA-256(prev.RowHash || canonical(row)); the v1 canonical form
(AuditChainHasher) is pinned by a known-answer vector computed independently.
- Migration AddAuditLogHashChain: nullable ChainSeq/RowHash on dbo.AuditLog
(metadata-only add), AuditLogChainHead (per-month chain end + partition-purge
checkpoint anchor), AuditLogChainTombstone, two partition-aligned filtered
indexes, scadabridge_audit_chainer role with a column-level UPDATE grant.
Production script docs/plans/sql/AddAuditLogHashChain.sql.
- AuditLogHashChain chains unchained rows oldest month first under an
exclusive sp_getapplock (zero timeout, so a dual-active central pair still
yields one chain), with an optimistic head check and a ChainSeq IS NULL
row-count check that roll a pass back instead of leaving a gap. The ingest
path is untouched.
- Purges keep the chain verifiable: the partition switch records the month's
checkpoint anchor and drops its tombstones; the per-channel DELETE writes a
tombstone per chained row in the same transaction (OUTPUT INTO). Both, and
the SourceNode backfill, hold the chain lock; the backfill skips chained
rows. The switch staging table gains the two new columns.
- AuditChainVerifier reports the first break (partition, ChainSeq, EventId,
kind) for edited, deleted, reordered rows and tail truncation.
- AuditLogHashChainService (central, default OFF via AuditLog:HashChain).
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
The #96 site-psk-invalidation topic was the first cross-node invalidation broadcast.
It is generalised into ONE central DistributedPubSub topic, central-cache-invalidation,
carrying typed messages (Commons ICentralCacheInvalidation: SitePskInvalidated,
RoleMappingsInvalidated, ExternalDefinitionsInvalidated, InboundApiMethodInvalidated)
in an envelope stamped with the publisher's process-unique origin id.
- Host: CentralCacheInvalidationBus (ICentralCacheInvalidationPublisher, no-op until
the mediator is set, never throws) and CentralCacheInvalidationSubscriber, which
skips its own node's envelopes and hands the peer's to every registered
ICentralCacheInvalidationHandler (a failing handler does not stop the rest).
Replaces SitePskInvalidationSubscriber.
- Migration of #96: SitePskRotationInvalidator publishes through the bus and is the
PSK handler; SitePskInvalidated keeps its type name and is also published bare on
site-psk-invalidation, which the subscriber still listens to, so a mixed-version
central pair keeps exchanging PSK invalidations. Remove after one release.
- Audit D1: RoleMappingCache is a handler; ManagementActor's post-commit role-mapping
invalidation also publishes RoleMappingsInvalidated, so a revocation is immediate on
the peer instead of after RoleMappingCacheSeconds.
- Audit D3 / #83 (peer half): ExternalDefinitionLookupCache is a handler; the 9
ManagementActor handlers and the Transport bundle importer publish
ExternalDefinitionsInvalidated after commit. (#83's local import half was already in.)
TTLs remain the fallback for a lost broadcast or a direct SQL edit. Tests: bus and
subscriber unit tests (origin filter, legacy topic), peer-node tests through the real
caches in ManagementService.Tests, importer publish, composition root, and a real
two-node cluster from the production HOCON (CentralCacheInvalidationTwoNodeTests)
proving every message type crosses with the production serializer. Docs: Host,
Communication, Security, ManagementService, ESG, and the #85 cache audit.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
The bulk SetInstanceNativeAlarmSourceOverridesCommand handler always wrote the
entry's IsMesReportableOverride, so a CLI import of a legacy 4-column CSV (no
MesReportable column) reset every imported source's MES override to inherit. The
Central UI avoided it only by looking up the stored flag client-side.
NativeAlarmSourceOverrideEntry gains an additive trailing KeepStoredMesReportable
flag: when set, the server leaves an existing row's MES override untouched (a new
row inherits) and ignores IsMesReportableOverride; unset keeps the old meaning
(null = clear to inherit), so older senders are unaffected. The CLI and the Central
UI now build their entries through one rule (ToOverrideEntry on each row type):
no MesReportable column => KeepStoredMesReportable. CLI README and
Component-CentralUI updated.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Deployments free-text search and the config-audit EntityName filter no longer
emit LIKE '%term%' by default. Prefix (LIKE 'term%') is the default; substring
is an explicit opt-in (DeploymentSearchMode.Contains / entityNameContains)
always bounded by a DeployedAt/Timestamp window (explicit, else 30 days).
Migration PrefixSearchIndexes adds DeployedBy, RevisionHash and EntityName
indexes. Deployments page gains a 'Match anywhere' checkbox.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
CLI import no longer has to reset MES overrides to inherit; a legacy 4-column file
still parses and the Central UI import keeps the stored flag for it.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
The data plane answers a broken inbound method with the generic "Script
compilation failed for this method"; the Roslyn diagnostic reached only the
central log, and the known-bad cache could not be observed or reset per method.
- InboundScriptExecutor keeps the diagnostics with each known-bad record and
exposes GetCompileState(name, currentScript): IsCompiled / IsKnownBad /
LastCompileError, judged against the current script text (a record for older
text is reported as not compiled yet).
- Management: ValidateApiMethodCommand (compile the stored script, return
ApiMethodCompileResult, register nothing, cache untouched) and
RecompileApiMethodCommand (CompileAndRegister without re-sending the entity,
drops the node's ApiMethodCache row, audited "Recompile"). Both Designer, like
UpdateApiMethod; dispatched through the shared ApiMethodCompileOperations.
GetApiMethod keeps the entity fields top-level and adds isCompiled,
isKnownBad, lastCompileError and compileStateNode. Compile state is per
central node and every result names the node.
- CLI: api-method validate --id, api-method recompile --id.
- Central UI: the API-method edit page shows the saved script's compile state
and last error, with Validate and Recompile actions (same operations, same
Designer gate, fresh DI scope).
- RequiredRoleMatrixTests entries; ManagementSurfaceParityTests covers the new
CLI leaves; ApiMethodCompileDiagnosticsTests + an executor CompileState test.
- Docs: CLI README, Component-CLI/ManagementService/InboundAPI, components/*.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
New AuditChannel.Security with AuditKind.CertTrust / CertTrustRemove (appended, ordinals stable).
Both central entry points - the management actor handlers and the Central UI CertManagementService -
write a best-effort ICentralAuditWriter row after the site outcome is known (Delivered when all
reachable site nodes acked, Failed otherwise, incl. relay timeouts), naming actor, site, thumbprint
and connection. Shared builder: CertTrustAuditEvents.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
TemplateNativeAlarmSource.IsMesReportable + tri-state InstanceNativeAlarmSourceOverride
.IsMesReportableOverride flatten onto ResolvedNativeAlarmSource; NativeAlarmActor stamps every
mirrored condition and InstanceActor the quiet-binding placeholder, so ScriptAlarm,
Alarms.MesReportableAsync() and the #60 wire fields carry it. Additive to the 900-999 severity band.
Migration AddNativeAlarmSourceMesReportable. Management commands, CLI --mes-reportable, Central UI
(template dialog checkbox, instance override select), Transport DTOs/importer/diff and the revision
hash (true-only, so existing hashes are unchanged) carry it.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
- Preview matches template folders, shared scripts, external systems, SMTP
(by Host) and SMS (by AccountSid) with the #93 ToNameLookup comparer
(OrdinalIgnoreCase, first wins, never throws), matching SQL Server's
case-insensitive collation, so a differently-cased target row is reported as
a conflict instead of New.
- Apply-side in-memory lookups do the same: ApplyTemplateFoldersAsync, the
template folder-id map, ResolveFolderParentEdgesAsync, ApplySmtpConfigsAsync
and ApplySmsConfigsAsync. An Overwrite now updates the existing row instead of
adding a duplicate (folder names are only unique per parent, so the old
ToDictionary could also throw).
- New ITemplateEngineRepository.GetBaseTemplatesWithChildrenAsync filters
IsDerived in SQL. Preview and the Overwrite load in ApplyTemplatesAsync use it,
so a derived slot template that shares a bundled name is never the import
target. GetTemplatesWithChildrenAsync is unchanged for the id-keyed callers
(export closure, rewire passes), which can need derived rows.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Adds four instruments on the already-allow-listed ZB.MOM.WW.ScadaBridge meter:
scadabridge.deployments.failed{reason}, scadabridge.store_and_forward.activity{action,category},
scadabridge.site.script.runs{outcome} + scadabridge.site.script.duration{outcome} (ms),
scadabridge.site.data_connections.transitions{state}. Queue depth, DCL connection state and
deploy-applied already existed. No new Meter, so ZbTelemetryOptions.Meters needs no change.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
`health pending-updates` (#63) called IPendingUpdatesEvaluator.EvaluateAsync directly
from ManagementActor because the cache lived in CentralUI, so every CLI call was a full
uncached fleet flatten. IPendingUpdatesCache/PendingUpdatesCache now live in
DeploymentManager.PendingUpdates and are registered by AddDeploymentManager; the
SingleFlightMemo<T> primitive moves to Commons.Caching (public) so both CentralUI's KPI /
alarm-summary caches and DeploymentManager can use it. HandleGetPendingUpdates reads the
shared cache; the UI circuits and the CLI now share one memo per node.
Tests: ManagementActor repeated-calls test (fails against the direct evaluator call:
2 evaluations instead of 1), AddDeploymentManager singleton-registration test; the cache
and memo tests move with their types.
(ScadaBridge#91)
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
The last two whole-library template reads are gone. Both fed the
inheritance-only TemplateInheritanceResolver, which walks one template's
parent chain and never follows compositions.
- TemplateService.ResyncInheritedMembersAsync (operator repair, CLI + editor)
now reads #86's GetTemplateReconcileClosureAsync (derived subtree + the
template's chain, tracked). When nothing derives from the template the
closure is empty, so it falls back to the new tracked
GetTemplateChainAsync - resyncing a leaf reads only its own chain.
- ManagementActor.HandleGetResolvedTemplateMembers now reads
GetTemplateChainAsync(id, trackChanges: false): the chain alone, untracked,
with script bodies (the resolver projects script code, so #66's body-free
analysis closure would not be equivalent).
The chain walk is factored out of TemplateReconcileClosure into
CollectChainAsync (one load per level, stops on parent cycles and dangling
parents) and shared by both.
Evidence:
- TemplateChainReadTests (#86 fixture): for every template plus an unknown
id, the closure-backed resync issues exactly the writes, audit row and
result of the whole-graph resync with GetAllTemplatesAsync wired to throw;
the resolver answers identically from the chain and the whole graph.
17 of these fail against the old service; 13 fail with the leaf chain
fallback removed (mutation-checked).
- TemplateChainRepositoryTests (SQLite, recorded SQL): a 4-deep chain loads
4 templates / 8 attributes instead of 46 / 92, one split-query set per
level, untracked on request and tracked for the resync.
- ResolvedTemplateMembersReadTests: the actor answers from the untracked
chain read and never calls GetAllTemplatesAsync (fails on the old actor).
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Health page gains PendingUpdatesHealthTiles (stale instances / sites with stale
artifacts, Deployer+Administrator only, site-scope narrowed, shared cache). New
GetPendingUpdates management command (Deployer) and 'health pending-updates' CLI
leaf. Flatten/cache computation untouched; the command reads
IPendingUpdatesEvaluator directly because the UI cache is not reachable from
ManagementService.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Interface now lives in Commons.Interfaces.Services; AuditLog no longer gets it
transitively through Communication. Implementation stays in HealthMonitoring.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
TemplateService.ReconcileDescendantsAsync - run after every base member
add/update/delete, template and native-alarm-source alike - loaded the whole
tracked template graph (every template, member and script body) on each call,
which after #66 was the dominant cost of those paths. It now calls the new
ITemplateEngineRepository.GetTemplateReconcileClosureAsync: the same tracked,
fully-included shape as GetAllTemplatesAsync, restricted to the edited
template's derived subtree plus its own inheritance chain. The reconcile is
inheritance-only (TemplateInheritanceResolver never follows compositions) and
every descendant's chain runs through the edited template, so that set is
everything it reads.
The walk (Commons TemplateReconcileClosure, the descendant-direction twin of
#66's TemplateAnalysisClosure) loads one children batch per subtree depth and
one batch per ancestor level, terminates on parent cycles and stops at
dangling parents. A template with no children - the common case - costs one
empty probe and loads nothing.
Evidence:
- TemplateReconcileClosureTests: for every template of a graph with a deep
subtree, an ancestor above the edited template, a composed module,
look-alike unrelated trees, a parent cycle and a dangling parent, the
closure-backed reconcile issues exactly the writes (add/update/delete,
audit, save) of the old whole-graph reconcile, and the resolver answers
identically for every descendant. Fails with the old service (whole-graph
load wired to throw) and with the ancestor walk removed (mutation-checked).
- TemplateReconcileClosureRepositoryTests (SQLite, recorded SQL): 5 templates
/ 10 attributes instead of 47 / 94, tracked with script bodies; a leaf costs
one probe query and tracks nothing.
- TemplateServiceTests: attribute add + update propagate two levels down with
GetAllTemplatesAsync wired to throw; existing propagation tests now run
against the closure via ServeReconcileClosureFromFullGraph.
ResyncInheritedMembersAsync (operator action) still reads the whole graph.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
SiteAuditBacklogSnapshot gains an optional trailing RejectedCount (gauge of
Rejected forward-state rows, read from the site store), carried on the wire as
SiteAuditBacklogSnapshotDto.rejected_count = 4 (additive; old sites read as 0,
old central ignores it). Health page shows an 'Audit rows rejected' badge.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Interface goes to Commons.Interfaces.Services and the state record to
Commons.Messages.Health, so CentralCommunicationActor no longer needs the
HealthMonitoring namespace. Implementation stays in HealthMonitoring.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Attribute/alarm/script add (collision check) and update (parent-lock lookup)
used GetAllTemplatesForAnalysisAsync, materialising every template, attribute,
alarm and script in the library per edit. They now call the new
ITemplateEngineRepository.GetTemplateAnalysisClosureAsync: the same untracked,
body-free analysis projection, restricted to the template's inheritance chain
plus the composition closure of every chain member - exactly what
TemplateResolver and CollisionDetector walk (a composed module's own parent
chain is never climbed, so it is not loaded). Adds use the edited template's
closure, updates the parent's.
The walk (Commons TemplateAnalysisClosure) loads one batch per closure depth,
requests each id once, terminates on cycles and skips dangling edges.
Evidence:
- TemplateAnalysisClosureTests: for every template of a graph with chains,
nested + diamond compositions, a derived module, cycles, dangling edges and
colliding unrelated templates, ResolveAllMembers / DetectCollisions /
FindMemberByCanonicalName answer identically from the closure and the whole
graph (mutation-checked).
- TemplateAnalysisClosureRepositoryTests (SQLite, recorded SQL): 6 templates /
12 attributes loaded instead of 47 / 94, in 4 queries (one per depth),
untracked, script bodies never selected.
- TemplateServiceTests: two new tests with the whole-graph load wired to throw
(fail on the old service); every existing add/update test now runs against
the closure via AnalysisClosureMock.
ReconcileDescendantsAsync still loads the tracked full graph; not in scope.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
The live AlarmStateUpdate already carried ack_time (24) but the Debug View
snapshot DTO did not, so snapshot-seeded alarm rows showed a null ack time
until the next live update. Additive google.protobuf.Timestamp ack_time = 21
(absent => null, including from a pre-#84 site), vendored C# regenerated with
docker/regen-proto.sh sitecommand, mapped both ways in SiteCommandDtoMapper.
The native golden sample now sets AckTime (the computed one leaves it null),
so the round-trip golden pins both the present and the absent decode.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
The bundle importer writes external systems, methods and database connections
through the raw DbContext, bypassing the ManagementActor handlers that
invalidate ExternalDefinitionLookupCache (#65), so the importing node served
pre-import definitions for up to DefinitionCacheTtl. New Commons seam
IExternalDefinitionCacheInvalidator, implemented by the cache and registered as
the same singleton; BundleImporter takes it optionally and invalidates once
after every committed import (never on rollback; never throws).
Peer-central-node invalidation (the other half of #83) is NOT done here: the
issue leaves open whether the 10 s TTL bound is acceptable for the peer.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
ScriptCallResult and RouteToCallResponse gain trailing Truncated/LogsTruncated,
filled from the site FlowTraceCollector after Complete(); RouteToCallResponseDto
fields 8/9 carry them (sitecommand regen). FlowDebugRunner ORs them into
FlowDebugRunResult on the script entry, and on the method entry through a new
optional truncation sink on RouteHelper.WithTraceSink (carried through every
With* clone). Previously a site run whose collector dropped rows came back as a
silently short list. Older sites send false.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Central's Debug View live stream and Alarm Summary cache always saw false
because neither wire message had the field. Additive proto fields, vendored
C# regenerated with docker/regen-proto.sh (sitestream + sitecommand), mapped
both ways in StreamRelayActor/SiteStreamGrpcClient and SiteCommandDtoMapper.
The live stream publishes Alarm Actor events, which never carried the flag,
so InstanceActor.HandleAlarmStateChanged now stamps it before retaining and
publishing. A pre-#60 site sends the proto3 default. No UI badge yet.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
SiteScriptCompileCache now records each eviction sweep on scadabridge.site.script_compile_cache.evictions
(ScadaBridgeTelemetry). It rides the existing ZB.MOM.WW.ScadaBridge meter, which is already in
SiteServiceRegistration.ObservedMeters, so no new allowlist entry is needed; a Host test pins that
the meter stays allowlisted.