feat(audit)!: remove the AuditLog hash chain, head export and two-login maintenance hardening

Owner decision 2026-11-15, at IT security's request, pending a family-wide general solution.

Removed: the chaining service, hasher, verifier, chain head/tombstone entities and store, the #139
chain-start cutoff, AuditLog:HashChain (incl. ChainBacklog and HeadExport with its File/Syslog sinks
and export service), the audit-chain-head-export health check, the VerifyAuditChain /
VerifyAuditChainHeads management commands and the `audit verify-chain` / `audit verify-heads` CLI
commands, and the two-login plumbing (AuditLog:MaintenanceConnectionString, AuditMaintenanceContext,
the keyed audit-maintenance repository/archive). Purge, archive, retention, partition roll-forward
and backfill-source-node all run on the runtime connection again, as before #107.

Parquet archive (#42) kept, chain-free: format v2 (ten stored columns, no tombstone file, manifest
without chain state), the switch guard is the row count only (51003 gone), verify = data-file size,
SHA-256 and row count, and superseded-stale cleanup proves coverage by EventId. Format-v1 archives
still list, verify and are cleaned up (their tombstone file too).

Migration RemoveAuditLogHashChain drops the chain indexes, the scadabridge_audit_chainer role
(members first), both chain tables, ChainSeq/RowHash and the purger's AuditMaintenanceGrants grants;
writer/purger roles stay. Down restores them. Every chain-column reference is EXEC-deferred for the
idempotent script (#126); FlipAuditLogClusteredKey's two chain index statements get the same
deferral, behaviour unchanged, so re-running the idempotent script on a migrated database works.
The partition-switch staging table adds the chain columns only when the live table has them, so the
new binary purges either schema. Deploy gate: old binaries map the chain columns, so the migration
ships with the new binaries, both centrals stopped.

A leftover AuditLog:HashChain section or MaintenanceConnectionString (DEV/QA still carry
AuditLog__HashChain__Enabled=true) is ignored with one startup warning (RetiredAuditSettingsWarning).

Claude-Session: https://claude.ai/code/session_01GT67AFKC45MBz1wPZnVbX8
This commit is contained in:
Joseph Doherty
2026-11-15 17:37:33 -05:00
parent 125db03232
commit ed2ed0a367
82 changed files with 3203 additions and 7603 deletions
@@ -30,8 +30,6 @@ public sealed class AuditLogArchiveOptionsValidator : OptionsValidatorBase<Audit
$"AuditLog:Archive:{nameof(AuditLogArchiveOptions.RowGroupSize)} ({options.RowGroupSize}) must be between 1000 and 1000000.");
builder.RequireThat(options.ReadCommandTimeoutMinutes > 0,
$"AuditLog:Archive:{nameof(AuditLogArchiveOptions.ReadCommandTimeoutMinutes)} ({options.ReadCommandTimeoutMinutes}) must be > 0.");
builder.RequireThat(options.ChainLockTimeoutSeconds > 0,
$"AuditLog:Archive:{nameof(AuditLogArchiveOptions.ChainLockTimeoutSeconds)} ({options.ChainLockTimeoutSeconds}) must be > 0.");
builder.RequireThat(options.RetentionDays is >= 0 and <= 36_500,
$"AuditLog:Archive:{nameof(AuditLogArchiveOptions.RetentionDays)} ({options.RetentionDays}) must be between 0 (never delete) and 36500.");
}
@@ -1,99 +0,0 @@
using ZB.MOM.WW.Configuration;
using ZB.MOM.WW.ScadaBridge.AuditLog.Central.HeadExport;
namespace ZB.MOM.WW.ScadaBridge.AuditLog.Central;
/// <summary>
/// Options for the central <see cref="AuditLogHashChainService"/> (section
/// <c>AuditLog:HashChain</c>) — hash-chain tamper evidence for the AuditLog
/// (audit roadmap T1, alog.md §11.4, ScadaBridge#41).
/// </summary>
/// <remarks>
/// <para>
/// <b>Default OFF</b> (alog.md §11.4: "Default off … to avoid operational burden"). When
/// enabled, both central nodes run the job; the SQL application lock means only one of
/// them chains at a time, so a dual-active pair still produces one gap-free chain per month.
/// </para>
/// <para>
/// <b>Chain start (ScadaBridge#139).</b> Enabling it on an existing database does NOT chain the
/// historical backlog: the first chaining pass records a chain-start anchor in the database at
/// the database's current UTC time, and only rows dated from then on are chained. Older rows
/// stay unchained and are reported by <c>audit verify-chain</c> as pre-cutoff rows, never as a
/// break. The anchor is written once and is the same for both central nodes. Set
/// <see cref="ChainBacklog"/> to chain the backlog instead (only effective on the very first
/// enablement). Run <c>audit backfill-source-node</c> BEFORE enabling either way: a chained
/// row's SourceNode is frozen.
/// </para>
/// </remarks>
public sealed class AuditLogHashChainOptions
{
/// <summary>Whether the chaining job runs (default false).</summary>
public bool Enabled { get; set; }
/// <summary>
/// Chain the historical backlog on first enablement instead of starting the chain at the
/// enablement instant (default false — owner decision on ScadaBridge#107, 2026-11-13: no
/// backlog chaining). Only read while nothing has been chained yet: once the database holds
/// a chain-start anchor or any chained row, the decision is fixed and this value is ignored.
/// Set it the same on both central nodes; if they differ, whichever chains first decides.
/// </summary>
public bool ChainBacklog { get; set; }
/// <summary>Seconds between chaining ticks (default 60).</summary>
public int IntervalSeconds { get; set; } = 60;
/// <summary>Rows linked per transaction (default 1000).</summary>
public int BatchSize { get; set; } = 1000;
/// <summary>Upper bound on batches per tick so a backlog yields between ticks (default 100).</summary>
public int MaxBatchesPerTick { get; set; } = 100;
/// <summary>
/// Off-box export of the per-month chain heads (<c>AuditLog:HashChain:HeadExport</c>,
/// ScadaBridge#107 item 1) — the chain's external anchor. Default off.
/// </summary>
public AuditChainHeadExportOptions HeadExport { get; set; } = new();
}
/// <summary>Validates <see cref="AuditLogHashChainOptions"/> at host startup.</summary>
public sealed class AuditLogHashChainOptionsValidator : OptionsValidatorBase<AuditLogHashChainOptions>
{
/// <inheritdoc />
protected override void Validate(ValidationBuilder builder, AuditLogHashChainOptions options)
{
builder.RequireThat(options.IntervalSeconds > 0,
$"AuditLog:HashChain:{nameof(AuditLogHashChainOptions.IntervalSeconds)} ({options.IntervalSeconds}) must be > 0.");
builder.RequireThat(options.BatchSize is > 0 and <= 10_000,
$"AuditLog:HashChain:{nameof(AuditLogHashChainOptions.BatchSize)} ({options.BatchSize}) must be between 1 and 10000.");
builder.RequireThat(options.MaxBatchesPerTick > 0,
$"AuditLog:HashChain:{nameof(AuditLogHashChainOptions.MaxBatchesPerTick)} ({options.MaxBatchesPerTick}) must be > 0.");
var export = options.HeadExport ?? new AuditChainHeadExportOptions();
if (!export.Enabled)
{
return;
}
const string section = "AuditLog:HashChain:HeadExport";
builder.RequireThat(export.IntervalMinutes > 0,
$"{section}:IntervalMinutes ({export.IntervalMinutes}) must be > 0.");
if (export.Sink == AuditChainHeadSinkKind.File)
{
var dir = export.File?.Directory;
builder.RequireThat(!string.IsNullOrWhiteSpace(dir) && Path.IsPathFullyQualified(dir),
$"{section}:File:Directory ('{dir}') must be an absolute path (local or UNC) when Sink=File.");
}
else
{
var syslog = export.Syslog ?? new AuditChainHeadSyslogSinkOptions();
builder.RequireThat(!string.IsNullOrWhiteSpace(syslog.Host),
$"{section}:Syslog:Host is required when Sink=Syslog.");
builder.RequireThat(syslog.Port is > 0 and <= 65535,
$"{section}:Syslog:Port ({syslog.Port}) must be between 1 and 65535.");
builder.RequireThat(syslog.Facility is >= 0 and <= 23,
$"{section}:Syslog:Facility ({syslog.Facility}) must be between 0 and 23.");
builder.RequireThat(syslog.TimeoutSeconds > 0,
$"{section}:Syslog:TimeoutSeconds ({syslog.TimeoutSeconds}) must be > 0.");
}
}
}
@@ -1,151 +0,0 @@
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
namespace ZB.MOM.WW.ScadaBridge.AuditLog.Central;
/// <summary>
/// Central hosted service that links newly written AuditLog rows into their month's
/// SHA-256 hash chain (audit roadmap T1, alog.md §11.4, ScadaBridge#41). The chaining
/// job of alog.md §11.4, run incrementally — every <see cref="AuditLogHashChainOptions.IntervalSeconds"/>
/// rather than only once a month has closed, so a row is tamper-evident within about a
/// minute of reaching central instead of up to a month later.
/// </summary>
/// <remarks>
/// <para>
/// Same shape as <see cref="AuditLogPartitionMaintenanceService"/>: a background loop, a
/// fresh DI scope per tick, every failure logged and contained. Runs on BOTH central nodes;
/// <see cref="IAuditLogHashChain.ChainPendingAsync"/> takes an exclusive SQL application
/// lock with a zero timeout, so the peer's pass skips instead of racing (and correctness
/// does not depend on Akka's view of which node is active).
/// </para>
/// </remarks>
public sealed class AuditLogHashChainService : IHostedService, IDisposable
{
private readonly IServiceScopeFactory _scopeFactory;
private readonly IOptions<AuditLogHashChainOptions> _options;
private readonly ILogger<AuditLogHashChainService> _logger;
private CancellationTokenSource? _cts;
private Task? _loop;
/// <summary>Initializes the service.</summary>
/// <param name="scopeFactory">Scope factory used to resolve the scoped chain implementation per tick.</param>
/// <param name="options">Hash-chain options.</param>
/// <param name="logger">Logger.</param>
public AuditLogHashChainService(
IServiceScopeFactory scopeFactory,
IOptions<AuditLogHashChainOptions> options,
ILogger<AuditLogHashChainService> logger)
{
_scopeFactory = scopeFactory ?? throw new ArgumentNullException(nameof(scopeFactory));
_options = options ?? throw new ArgumentNullException(nameof(options));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
}
/// <inheritdoc />
public Task StartAsync(CancellationToken cancellationToken)
{
if (!_options.Value.Enabled)
{
_logger.LogInformation("AuditLog hash chain disabled (AuditLog:HashChain:Enabled=false); no chaining job started.");
return Task.CompletedTask;
}
var cts = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
_cts = cts;
var token = cts.Token;
_loop = Task.Run(() => RunLoopAsync(token), CancellationToken.None);
return Task.CompletedTask;
}
private async Task RunLoopAsync(CancellationToken ct)
{
while (!ct.IsCancellationRequested)
{
await TickAsync(ct).ConfigureAwait(false);
try
{
await Task.Delay(TimeSpan.FromSeconds(_options.Value.IntervalSeconds), ct).ConfigureAwait(false);
}
catch (OperationCanceledException)
{
break;
}
}
}
/// <summary>
/// Runs one tick: chains batches until the backlog is empty, the peer holds the lock,
/// or <see cref="AuditLogHashChainOptions.MaxBatchesPerTick"/> is reached.
/// </summary>
/// <param name="ct">Cancellation token.</param>
/// <returns>The number of rows chained in this tick.</returns>
internal async Task<long> TickAsync(CancellationToken ct)
{
long total = 0;
try
{
var options = _options.Value;
await using var scope = _scopeFactory.CreateAsyncScope();
var chain = scope.ServiceProvider.GetRequiredService<IAuditLogHashChain>();
for (var i = 0; i < options.MaxBatchesPerTick && !ct.IsCancellationRequested; i++)
{
var result = await chain.ChainPendingAsync(options.BatchSize, options.ChainBacklog, ct).ConfigureAwait(false);
total += result.RowsChained;
if (result.LockBusy || result.RowsChained == 0)
{
break;
}
}
if (total > 0)
{
_logger.LogDebug("AuditLog hash chain linked {Rows} rows this tick.", total);
}
}
catch (OperationCanceledException) when (ct.IsCancellationRequested)
{
// Shutdown.
}
catch (Exception ex)
{
// Contained: a transient SQL failure must not end the loop; the next tick
// resumes from wherever the last committed pass left the chain.
_logger.LogError(ex, "AuditLog hash-chain tick failed after linking {Rows} rows.", total);
}
return total;
}
/// <inheritdoc />
public Task StopAsync(CancellationToken cancellationToken)
{
try
{
_cts?.Cancel();
}
catch (ObjectDisposedException)
{
// Dispose already cancelled the loop.
}
return _loop ?? Task.CompletedTask;
}
/// <inheritdoc />
public void Dispose()
{
try
{
_cts?.Cancel();
}
catch (ObjectDisposedException)
{
// Idempotent.
}
_cts?.Dispose();
}
}
@@ -49,9 +49,7 @@ namespace ZB.MOM.WW.ScadaBridge.AuditLog.Central;
/// </para>
/// <para>
/// <b>DI scopes.</b> <see cref="IAuditLogRepository"/> is a scoped EF Core
/// service registered by <c>AddConfigurationDatabase</c> — the
/// <see cref="AuditMaintenanceContext.ServiceKey"/>-keyed one, on the AuditLog maintenance
/// connection (ScadaBridge#107 item 3), when registered. The singleton
/// service registered by <c>AddConfigurationDatabase</c>. The singleton
/// opens one DI scope per tick and reuses the same repository across every
/// boundary in that tick — mirrors the
/// <see cref="SiteAuditReconciliationActor"/> pattern.
@@ -152,11 +150,7 @@ public class AuditLogPurgeActor : ReceiveActor
IAuditLogRepository repository;
try
{
// The maintenance-connection repository when the host registered one
// (AddConfigurationDatabase, ScadaBridge#107 item 3 — on the separate maintenance
// login if AuditLog:MaintenanceConnectionString is set), else the plain one.
repository = scope.ServiceProvider.GetKeyedService<IAuditLogRepository>(AuditMaintenanceContext.ServiceKey)
?? scope.ServiceProvider.GetRequiredService<IAuditLogRepository>();
repository = scope.ServiceProvider.GetRequiredService<IAuditLogRepository>();
}
catch (Exception ex)
{
@@ -210,8 +204,7 @@ public class AuditLogPurgeActor : ReceiveActor
{
try
{
archive = scope.ServiceProvider.GetKeyedService<IAuditLogArchive>(AuditMaintenanceContext.ServiceKey)
?? scope.ServiceProvider.GetRequiredService<IAuditLogArchive>();
archive = scope.ServiceProvider.GetRequiredService<IAuditLogArchive>();
}
catch (Exception ex)
{
@@ -305,8 +298,7 @@ public class AuditLogPurgeActor : ReceiveActor
return;
}
archive = services.GetKeyedService<IAuditLogArchive>(AuditMaintenanceContext.ServiceKey)
?? services.GetRequiredService<IAuditLogArchive>();
archive = services.GetRequiredService<IAuditLogArchive>();
}
var result = await archive.CleanupAsync(dryRun: false).ConfigureAwait(false);
@@ -1,87 +0,0 @@
namespace ZB.MOM.WW.ScadaBridge.AuditLog.Central.HeadExport;
/// <summary>Where exported chain heads go.</summary>
public enum AuditChainHeadSinkKind
{
/// <summary>Append-only JSON-lines files in a directory (local or UNC).</summary>
File,
/// <summary>RFC 5424 syslog over UDP or TCP.</summary>
Syslog,
}
/// <summary>Syslog transport.</summary>
public enum AuditSyslogProtocol
{
/// <summary>One datagram per record (RFC 5426).</summary>
Udp,
/// <summary>Octet-counted frames on one TCP connection per export (RFC 6587).</summary>
Tcp,
}
/// <summary>
/// Off-box export of the AuditLog hash-chain heads (section <c>AuditLog:HashChain:HeadExport</c>,
/// ScadaBridge#107 item 1). <b>Default OFF.</b> Every <see cref="IntervalMinutes"/> both central
/// nodes read each month's chain head (month, last sequence + hash, partition-purge anchor,
/// chained-row and tombstone counts) and append the heads that changed since their previous
/// export to a sink the database login cannot write. <c>audit verify-heads</c> later compares
/// those records with the database: a chain someone rewrote and re-hashed inside SQL Server no
/// longer carries the hash that was exported.
/// </summary>
public sealed class AuditChainHeadExportOptions
{
/// <summary>Whether the export runs (default false).</summary>
public bool Enabled { get; set; }
/// <summary>Minutes between exports (default 60).</summary>
public int IntervalMinutes { get; set; } = 60;
/// <summary>
/// Re-export unchanged heads every tick too (default false: only heads whose sequence,
/// anchor or counts moved since this node's previous export, plus every head on the first
/// tick after start).
/// </summary>
public bool ExportUnchanged { get; set; }
/// <summary>Which sink (default <see cref="AuditChainHeadSinkKind.File"/>).</summary>
public AuditChainHeadSinkKind Sink { get; set; } = AuditChainHeadSinkKind.File;
/// <summary>File sink settings.</summary>
public AuditChainHeadFileSinkOptions File { get; set; } = new();
/// <summary>Syslog sink settings.</summary>
public AuditChainHeadSyslogSinkOptions Syslog { get; set; } = new();
}
/// <summary>File sink: <c>{Directory}/{yyyy}/audit-chain-heads-{yyyy-MM-dd}-{node}.jsonl</c>, opened for append only.</summary>
public sealed class AuditChainHeadFileSinkOptions
{
/// <summary>
/// Absolute directory (local path or UNC share). Give the service account create + append
/// rights only; the SQL Server login has no access to it by construction.
/// </summary>
public string? Directory { get; set; }
}
/// <summary>Syslog sink (RFC 5424).</summary>
public sealed class AuditChainHeadSyslogSinkOptions
{
/// <summary>Collector host name or address.</summary>
public string? Host { get; set; }
/// <summary>Collector port (default 514).</summary>
public int Port { get; set; } = 514;
/// <summary>Transport (default UDP).</summary>
public AuditSyslogProtocol Protocol { get; set; } = AuditSyslogProtocol.Udp;
/// <summary>Syslog facility 0–23 (default 13, "log audit").</summary>
public int Facility { get; set; } = 13;
/// <summary>APP-NAME field (default <c>scadabridge</c>).</summary>
public string AppName { get; set; } = "scadabridge";
/// <summary>Send/connect timeout in seconds (default 10).</summary>
public int TimeoutSeconds { get; set; } = 10;
}
@@ -1,205 +0,0 @@
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Audit;
using ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Maintenance;
namespace ZB.MOM.WW.ScadaBridge.AuditLog.Central.HeadExport;
/// <summary>
/// Central hosted service that exports the AuditLog hash-chain heads off-box
/// (<c>AuditLog:HashChain:HeadExport</c>, ScadaBridge#107 item 1). Default off.
/// </summary>
/// <remarks>
/// <para>
/// Runs on both central nodes (each record names its node). A tick reads every month's head in
/// one consistent snapshot (<see cref="IAuditChainHeadStore.ReadHeadsAsync"/>, chain lock
/// SHARED) and writes the heads that changed since this node's last successful export — all of
/// them on the first tick after start — to the configured <see cref="IAuditChainHeadSink"/>.
/// A failed write is logged and retried on the next tick with the same heads (the in-memory
/// "last exported" state only advances on success). No signing: the protection is that the
/// sink sits outside the database login's reach.
/// </para>
/// </remarks>
public sealed class AuditChainHeadExportService : IHostedService, IDisposable
{
private readonly IServiceScopeFactory _scopeFactory;
private readonly AuditChainHeadExportOptions _options;
private readonly IAuditChainHeadSink? _sink;
private readonly string? _node;
private readonly TimeProvider _time;
private readonly ILogger<AuditChainHeadExportService> _logger;
private readonly AuditChainHeadExportStatus _status;
private readonly Dictionary<DateTime, (long Seq, long? Purged, long Links)> _lastExported = new();
private CancellationTokenSource? _cts;
private Task? _loop;
/// <summary>Initializes the service.</summary>
/// <param name="scopeFactory">Scope factory for the scoped head store.</param>
/// <param name="options">Hash-chain options (the <c>HeadExport</c> sub-section is used).</param>
/// <param name="sink">The sink (null when the export is disabled).</param>
/// <param name="nodeName">This node's name for the records.</param>
/// <param name="logger">Logger.</param>
/// <param name="time">Clock.</param>
/// <param name="status">Health status the tick outcome is recorded on (ScadaBridge#115).</param>
public AuditChainHeadExportService(
IServiceScopeFactory scopeFactory,
IOptions<AuditLogHashChainOptions> options,
IAuditChainHeadSink? sink,
string? nodeName,
ILogger<AuditChainHeadExportService> logger,
TimeProvider? time = null,
AuditChainHeadExportStatus? status = null)
{
_status = status ?? new AuditChainHeadExportStatus();
_scopeFactory = scopeFactory ?? throw new ArgumentNullException(nameof(scopeFactory));
_options = (options ?? throw new ArgumentNullException(nameof(options))).Value.HeadExport;
_sink = sink;
_node = nodeName;
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
_time = time ?? TimeProvider.System;
}
/// <inheritdoc />
public Task StartAsync(CancellationToken cancellationToken)
{
if (!_options.Enabled || _sink is null)
{
_logger.LogInformation("AuditLog chain-head export disabled (AuditLog:HashChain:HeadExport:Enabled=false).");
return Task.CompletedTask;
}
_status.MarkStarted(_options.IntervalMinutes, _time.GetUtcNow().UtcDateTime);
_logger.LogInformation("AuditLog chain-head export every {Minutes} min to {Sink}.", _options.IntervalMinutes, _sink.Describe);
var cts = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
_cts = cts;
var token = cts.Token;
_loop = Task.Run(() => RunLoopAsync(token), CancellationToken.None);
return Task.CompletedTask;
}
private async Task RunLoopAsync(CancellationToken ct)
{
while (!ct.IsCancellationRequested)
{
await TickAsync(ct).ConfigureAwait(false);
try
{
await Task.Delay(TimeSpan.FromMinutes(_options.IntervalMinutes), ct).ConfigureAwait(false);
}
catch (OperationCanceledException)
{
break;
}
}
}
/// <summary>Runs one export tick.</summary>
/// <param name="ct">Cancellation token.</param>
/// <returns>The number of records written (0 on failure or nothing changed).</returns>
internal async Task<int> TickAsync(CancellationToken ct)
{
if (_sink is null)
{
return 0;
}
try
{
IReadOnlyList<AuditChainHeadSnapshot> heads;
await using (var scope = _scopeFactory.CreateAsyncScope())
{
heads = await scope.ServiceProvider.GetRequiredService<IAuditChainHeadStore>()
.ReadHeadsAsync(ct).ConfigureAwait(false);
}
var now = _time.GetUtcNow().UtcDateTime;
var changed = heads
.Where(h => _options.ExportUnchanged
|| !_lastExported.TryGetValue(h.Month, out var last)
|| last != (h.LastSeq, h.PurgedThroughSeq, h.ChainedRows + h.Tombstones))
.ToList();
if (changed.Count == 0)
{
_status.RecordSuccess(now);
return 0;
}
var records = changed.Select(h => ToRecord(h, now, _node)).ToList();
await _sink.WriteAsync(records, ct).ConfigureAwait(false);
foreach (var h in changed)
{
_lastExported[h.Month] = (h.LastSeq, h.PurgedThroughSeq, h.ChainedRows + h.Tombstones);
}
_status.RecordSuccess(now);
_logger.LogDebug("Exported {Count} AuditLog chain head(s) to {Sink}.", records.Count, _sink.Describe);
return records.Count;
}
catch (OperationCanceledException) when (ct.IsCancellationRequested)
{
return 0;
}
catch (Exception ex)
{
// Contained: the next tick retries the same heads (nothing was marked exported).
_logger.LogError(ex, "AuditLog chain-head export to {Sink} failed; retrying next tick.", _sink.Describe);
_status.RecordFailure(_time.GetUtcNow().UtcDateTime, ex.Message);
return 0;
}
}
/// <summary>Maps a database head to its exported record.</summary>
/// <param name="head">The head.</param>
/// <param name="exportedAtUtc">Export time.</param>
/// <param name="node">Exporting node.</param>
/// <returns>The record.</returns>
public static AuditChainHeadRecord ToRecord(AuditChainHeadSnapshot head, DateTime exportedAtUtc, string? node)
{
ArgumentNullException.ThrowIfNull(head);
return new AuditChainHeadRecord(
AuditChainHeadRecord.CurrentVersion,
AuditChainHasher.PartitionLabel(head.Month),
head.LastSeq,
AuditChainHeadComparer.ToHex(head.LastHash),
head.PurgedThroughSeq,
head.PurgedThroughHash is null ? null : AuditChainHeadComparer.ToHex(head.PurgedThroughHash),
head.ChainedRows,
head.Tombstones,
head.UpdatedAtUtc,
exportedAtUtc,
node);
}
/// <inheritdoc />
public Task StopAsync(CancellationToken cancellationToken)
{
try
{
_cts?.Cancel();
}
catch (ObjectDisposedException)
{
// Dispose already cancelled the loop.
}
return _loop ?? Task.CompletedTask;
}
/// <inheritdoc />
public void Dispose()
{
try
{
_cts?.Cancel();
}
catch (ObjectDisposedException)
{
// Idempotent.
}
_cts?.Dispose();
}
}
@@ -1,91 +0,0 @@
using System.Diagnostics.Metrics;
using ZB.MOM.WW.ScadaBridge.Commons.Observability;
namespace ZB.MOM.WW.ScadaBridge.AuditLog.Central.HeadExport;
/// <summary>
/// Health signal of the off-box chain-head export (ScadaBridge#115). Updated by
/// <see cref="AuditChainHeadExportService"/> on every tick; read by the Host's
/// <c>audit-chain-head-export</c> /health/ready check. A sink that stays down used to be a
/// log line only, silently stopping the external anchor.
/// </summary>
public sealed class AuditChainHeadExportStatus
{
private static readonly Meter Meter = new(ScadaBridgeTelemetry.MeterName);
private static readonly Counter<long> FailuresCounter =
Meter.CreateCounter<long>("scadabridge.audit.chain_head_export.failures", unit: "1",
description: "AuditLog chain-head export ticks that failed to read or write the heads (sink offline / unreachable); retried next tick.");
private readonly object _gate = new();
private long _totalFailures;
private int _consecutiveFailures;
private DateTime? _lastSuccessUtc;
private DateTime? _lastFailureUtc;
private string? _lastError;
private DateTime? _startedUtc;
/// <summary>Whether the export is running at all (false = disabled, nothing to report).</summary>
public bool Enabled { get; private set; }
/// <summary>Minutes between export ticks (for the staleness test).</summary>
public int IntervalMinutes { get; private set; } = 60;
/// <summary>Total failed ticks since process start.</summary>
public long TotalFailures { get { lock (_gate) { return _totalFailures; } } }
/// <summary>Failed ticks since the last success.</summary>
public int ConsecutiveFailures { get { lock (_gate) { return _consecutiveFailures; } } }
/// <summary>UTC time of the last successful tick, or null.</summary>
public DateTime? LastSuccessUtc { get { lock (_gate) { return _lastSuccessUtc; } } }
/// <summary>UTC time of the last failed tick, or null.</summary>
public DateTime? LastFailureUtc { get { lock (_gate) { return _lastFailureUtc; } } }
/// <summary>Message of the last failure (never a secret: the exception message only), or null.</summary>
public string? LastError { get { lock (_gate) { return _lastError; } } }
/// <summary>UTC time the export loop started, or null.</summary>
public DateTime? StartedUtc { get { lock (_gate) { return _startedUtc; } } }
/// <summary>Marks the export as running.</summary>
/// <param name="intervalMinutes">Tick interval.</param>
/// <param name="nowUtc">Start time.</param>
public void MarkStarted(int intervalMinutes, DateTime nowUtc)
{
lock (_gate)
{
Enabled = true;
IntervalMinutes = intervalMinutes;
_startedUtc = nowUtc;
}
}
/// <summary>Records a successful tick.</summary>
/// <param name="nowUtc">Time of the tick.</param>
public void RecordSuccess(DateTime nowUtc)
{
lock (_gate)
{
_consecutiveFailures = 0;
_lastSuccessUtc = nowUtc;
}
}
/// <summary>Records a failed tick and bumps the failure counter metric.</summary>
/// <param name="nowUtc">Time of the tick.</param>
/// <param name="error">The failure message.</param>
public void RecordFailure(DateTime nowUtc, string error)
{
lock (_gate)
{
_totalFailures++;
_consecutiveFailures++;
_lastFailureUtc = nowUtc;
_lastError = error;
}
FailuresCounter.Add(1);
}
}
@@ -1,21 +0,0 @@
using System.Text.Json;
using System.Text.Json.Serialization;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Audit;
namespace ZB.MOM.WW.ScadaBridge.AuditLog.Central.HeadExport;
/// <summary>The one JSON shape every sink writes (and <c>audit verify-heads</c> reads).</summary>
public static class AuditChainHeadJson
{
/// <summary>camelCase, compact, nulls kept so every record has every field.</summary>
public static readonly JsonSerializerOptions Options = new(JsonSerializerDefaults.Web)
{
WriteIndented = false,
DefaultIgnoreCondition = JsonIgnoreCondition.Never,
};
/// <summary>Serialises one record to a single line.</summary>
/// <param name="record">The record.</param>
/// <returns>Compact JSON.</returns>
public static string Serialize(AuditChainHeadRecord record) => JsonSerializer.Serialize(record, Options);
}
@@ -1,79 +0,0 @@
using System.Text;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Audit;
namespace ZB.MOM.WW.ScadaBridge.AuditLog.Central.HeadExport;
/// <summary>
/// Appends exported heads as JSON lines to <c>{Directory}/{yyyy}/audit-chain-heads-{yyyy-MM-dd}-{node}.jsonl</c>.
/// </summary>
/// <remarks>
/// The file is only ever opened with <see cref="FileMode.Append"/> — never truncated, rewritten
/// or deleted by ScadaBridge — so the share can grant the service account create + append data
/// only. One file per node per UTC day keeps the two central nodes from interleaving writes in
/// one file. Each batch is flushed to disk before the call returns.
/// </remarks>
public sealed class FileAuditChainHeadSink : IAuditChainHeadSink
{
private readonly string _directory;
private readonly string _node;
private readonly TimeProvider _time;
/// <summary>Creates the sink.</summary>
/// <param name="directory">Absolute export directory.</param>
/// <param name="node">This node's name (part of the file name).</param>
/// <param name="time">Clock (file-per-day naming).</param>
public FileAuditChainHeadSink(string directory, string node, TimeProvider? time = null)
{
ArgumentException.ThrowIfNullOrWhiteSpace(directory);
if (!Path.IsPathFullyQualified(directory))
{
throw new ArgumentException($"Head-export directory '{directory}' must be absolute.", nameof(directory));
}
_directory = directory;
_node = SafeName(string.IsNullOrWhiteSpace(node) ? "node" : node);
_time = time ?? TimeProvider.System;
}
/// <inheritdoc />
public string Describe => _directory;
/// <summary>The file a write at <paramref name="nowUtc"/> lands in.</summary>
/// <param name="nowUtc">The write time.</param>
/// <returns>The full path.</returns>
public string PathFor(DateTime nowUtc) => Path.Combine(
_directory, nowUtc.ToString("yyyy", System.Globalization.CultureInfo.InvariantCulture),
$"audit-chain-heads-{nowUtc:yyyy-MM-dd}-{_node}.jsonl");
/// <inheritdoc />
public async Task WriteAsync(IReadOnlyList<AuditChainHeadRecord> records, CancellationToken ct)
{
ArgumentNullException.ThrowIfNull(records);
if (records.Count == 0)
{
return;
}
var path = PathFor(_time.GetUtcNow().UtcDateTime);
Directory.CreateDirectory(Path.GetDirectoryName(path)!);
var text = new StringBuilder();
foreach (var record in records)
{
text.Append(AuditChainHeadJson.Serialize(record)).Append('\n');
}
var bytes = new UTF8Encoding(encoderShouldEmitUTF8Identifier: false).GetBytes(text.ToString());
await using var stream = new FileStream(path, FileMode.Append, FileAccess.Write, FileShare.Read, 4096, useAsync: true);
await stream.WriteAsync(bytes, ct).ConfigureAwait(false);
await stream.FlushAsync(ct).ConfigureAwait(false);
stream.Flush(flushToDisk: true);
}
private static string SafeName(string value)
{
var invalid = Path.GetInvalidFileNameChars();
var chars = value.Select(c => invalid.Contains(c) || c == ' ' ? '_' : c).ToArray();
return new string(chars);
}
}
@@ -1,16 +0,0 @@
using ZB.MOM.WW.ScadaBridge.Commons.Types.Audit;
namespace ZB.MOM.WW.ScadaBridge.AuditLog.Central.HeadExport;
/// <summary>An off-box destination for exported chain heads (ScadaBridge#107 item 1).</summary>
public interface IAuditChainHeadSink
{
/// <summary>Short description for logs (e.g. the directory or <c>host:port/udp</c>).</summary>
string Describe { get; }
/// <summary>Writes the records; throws when they could not all be delivered.</summary>
/// <param name="records">The records, oldest month first.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>A task that completes when the records are durably handed off.</returns>
Task WriteAsync(IReadOnlyList<AuditChainHeadRecord> records, CancellationToken ct);
}
@@ -1,108 +0,0 @@
using System.Globalization;
using System.Net.Sockets;
using System.Text;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Audit;
namespace ZB.MOM.WW.ScadaBridge.AuditLog.Central.HeadExport;
/// <summary>
/// Sends exported heads as RFC 5424 syslog messages — one message per head, the JSON record as
/// MSG — over UDP (RFC 5426, one datagram each) or TCP (RFC 6587 octet-counting framing, one
/// connection per export).
/// </summary>
/// <remarks>
/// Message: <c>&lt;PRI&gt;1 TIMESTAMP HOSTNAME APP-NAME PROCID chainhead - {json}</c>, severity
/// Notice (5), PRI = facility × 8 + 5. No structured data (a private enterprise number would be
/// needed); the collector keeps the JSON body, which <c>audit verify-heads</c> reads back from
/// any line that contains it. UDP is fire-and-forget — use TCP where a lost datagram matters.
/// </remarks>
public sealed class SyslogAuditChainHeadSink : IAuditChainHeadSink
{
/// <summary>MSGID of every head message.</summary>
public const string MessageId = "chainhead";
private const int SeverityNotice = 5;
private readonly AuditChainHeadSyslogSinkOptions _options;
private readonly string _hostName;
private readonly TimeProvider _time;
/// <summary>Creates the sink.</summary>
/// <param name="options">Syslog settings.</param>
/// <param name="hostName">HOSTNAME field (defaults to the machine name).</param>
/// <param name="time">Clock for TIMESTAMP.</param>
public SyslogAuditChainHeadSink(AuditChainHeadSyslogSinkOptions options, string? hostName = null, TimeProvider? time = null)
{
_options = options ?? throw new ArgumentNullException(nameof(options));
ArgumentException.ThrowIfNullOrWhiteSpace(options.Host);
_hostName = Token(hostName ?? Environment.MachineName, 255);
_time = time ?? TimeProvider.System;
}
/// <inheritdoc />
public string Describe => $"{_options.Host}:{_options.Port}/{_options.Protocol.ToString().ToLowerInvariant()}";
/// <summary>Formats one RFC 5424 message (no framing).</summary>
/// <param name="record">The record.</param>
/// <returns>The message text.</returns>
public string Format(AuditChainHeadRecord record)
{
var pri = (_options.Facility * 8) + SeverityNotice;
var timestamp = _time.GetUtcNow().UtcDateTime.ToString("yyyy-MM-dd'T'HH:mm:ss.fff'Z'", CultureInfo.InvariantCulture);
return string.Create(CultureInfo.InvariantCulture,
$"<{pri}>1 {timestamp} {_hostName} {Token(_options.AppName, 48)} {Environment.ProcessId} {MessageId} - {AuditChainHeadJson.Serialize(record)}");
}
/// <inheritdoc />
public async Task WriteAsync(IReadOnlyList<AuditChainHeadRecord> records, CancellationToken ct)
{
ArgumentNullException.ThrowIfNull(records);
if (records.Count == 0)
{
return;
}
using var timeout = CancellationTokenSource.CreateLinkedTokenSource(ct);
timeout.CancelAfter(TimeSpan.FromSeconds(Math.Max(1, _options.TimeoutSeconds)));
var token = timeout.Token;
if (_options.Protocol == AuditSyslogProtocol.Udp)
{
using var udp = new UdpClient();
udp.Connect(_options.Host!, _options.Port);
foreach (var record in records)
{
var bytes = Encoding.UTF8.GetBytes(Format(record));
await udp.SendAsync(bytes, token).ConfigureAwait(false);
}
return;
}
using var tcp = new TcpClient();
await tcp.ConnectAsync(_options.Host!, _options.Port, token).ConfigureAwait(false);
var stream = tcp.GetStream();
foreach (var record in records)
{
var message = Encoding.UTF8.GetBytes(Format(record));
var frame = Encoding.ASCII.GetBytes(message.Length.ToString(CultureInfo.InvariantCulture) + " ");
await stream.WriteAsync(frame, token).ConfigureAwait(false);
await stream.WriteAsync(message, token).ConfigureAwait(false);
}
await stream.FlushAsync(token).ConfigureAwait(false);
tcp.Client.Shutdown(SocketShutdown.Send);
}
// RFC 5424 header fields are PRINTUSASCII (33-126), no spaces; "-" when empty.
private static string Token(string? value, int max)
{
if (string.IsNullOrEmpty(value))
{
return "-";
}
var chars = value.Where(c => c is >= '!' and <= '~').Take(max).ToArray();
return chars.Length == 0 ? "-" : new string(chars);
}
}
@@ -0,0 +1,73 @@
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
namespace ZB.MOM.WW.ScadaBridge.AuditLog.Central;
/// <summary>
/// Logs one warning at central startup for each AuditLog setting that no longer does anything.
/// </summary>
/// <remarks>
/// The hash chain (<c>AuditLog:HashChain</c>, including its head export) and the two-login
/// maintenance hardening (<c>AuditLog:MaintenanceConnectionString</c>) were removed on
/// 2026-11-15 at IT security's request. Nothing validates unknown keys, so a node still carrying
/// them (for example <c>AuditLog__HashChain__Enabled=true</c> in its service environment) boots
/// normally; this warning tells the operator the keys can be deleted.
/// </remarks>
public sealed class RetiredAuditSettingsWarning : IHostedService
{
/// <summary>The retired hash-chain section.</summary>
public const string HashChainSection = "AuditLog:HashChain";
/// <summary>The retired maintenance-login connection string.</summary>
public const string MaintenanceConnectionStringKey = "AuditLog:MaintenanceConnectionString";
private readonly IConfiguration _configuration;
private readonly ILogger<RetiredAuditSettingsWarning> _logger;
/// <summary>Initializes a new instance of the <see cref="RetiredAuditSettingsWarning"/> class.</summary>
/// <param name="configuration">Application configuration.</param>
/// <param name="logger">Logger.</param>
public RetiredAuditSettingsWarning(IConfiguration configuration, ILogger<RetiredAuditSettingsWarning> logger)
{
_configuration = configuration ?? throw new ArgumentNullException(nameof(configuration));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
}
/// <summary>The retired keys present in <paramref name="configuration"/>.</summary>
/// <param name="configuration">The configuration to inspect.</param>
/// <returns>Each retired key or section that is set.</returns>
public static IReadOnlyList<string> FindRetired(IConfiguration configuration)
{
ArgumentNullException.ThrowIfNull(configuration);
var found = new List<string>();
if (configuration.GetSection(HashChainSection).Exists())
{
found.Add(HashChainSection);
}
if (!string.IsNullOrWhiteSpace(configuration[MaintenanceConnectionStringKey]))
{
found.Add(MaintenanceConnectionStringKey);
}
return found;
}
/// <inheritdoc />
public Task StartAsync(CancellationToken cancellationToken)
{
foreach (var key in FindRetired(_configuration))
{
_logger.LogWarning(
"{Setting} is no longer supported and is ignored: the AuditLog hash chain and the two-login "
+ "maintenance hardening were removed on 2026-11-15. Delete the setting.",
key);
}
return Task.CompletedTask;
}
/// <inheritdoc />
public Task StopAsync(CancellationToken cancellationToken) => Task.CompletedTask;
}
@@ -52,15 +52,6 @@ public static class ServiceCollectionExtensions
/// <summary>Configuration section bound to <see cref="AuditLogPartitionMaintenanceOptions"/>.</summary>
public const string PartitionMaintenanceSectionName = "AuditLog:PartitionMaintenance";
/// <summary>
/// Optional connection string for the AuditLog maintenance paths — the hash-chain job and both
/// purge paths (ScadaBridge#107 item 3). Unset: they run on <c>ScadaBridge:Database:ConfigurationDb</c>.
/// </summary>
public const string MaintenanceConnectionStringKey = "AuditLog:MaintenanceConnectionString";
/// <summary>Configuration section bound to <see cref="AuditLogHashChainOptions"/> (ScadaBridge#41).</summary>
public const string HashChainSectionName = "AuditLog:HashChain";
/// <summary>
/// Configuration section bound to
/// <see cref="ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Maintenance.AuditLogArchiveOptions"/> (ScadaBridge#42).
@@ -410,34 +401,9 @@ public static class ServiceCollectionExtensions
AuditLogPartitionMaintenanceOptionsValidator>(config, PartitionMaintenanceSectionName);
services.AddHostedService<AuditLogPartitionMaintenanceService>();
// Hash-chain tamper evidence (ScadaBridge#41). Default OFF; the hosted service is
// always registered and returns from StartAsync without a loop when disabled.
// IAuditLogHashChain is registered by AddConfigurationDatabase (central only).
services.AddValidatedOptions<AuditLogHashChainOptions, AuditLogHashChainOptionsValidator>(
config, HashChainSectionName);
services.AddHostedService<AuditLogHashChainService>();
// Off-box chain-head export (ScadaBridge#107 item 1, AuditLog:HashChain:HeadExport,
// default off). The sink is built from the bound options; null when disabled.
services.AddSingleton<Central.HeadExport.AuditChainHeadExportStatus>();
services.AddHostedService(sp =>
{
var options = sp.GetRequiredService<Microsoft.Extensions.Options.IOptions<AuditLogHashChainOptions>>();
var node = sp.GetService<INodeIdentityProvider>()?.NodeName;
var export = options.Value.HeadExport;
Central.HeadExport.IAuditChainHeadSink? sink = !export.Enabled ? null : export.Sink switch
{
Central.HeadExport.AuditChainHeadSinkKind.Syslog => new Central.HeadExport.SyslogAuditChainHeadSink(export.Syslog),
_ => new Central.HeadExport.FileAuditChainHeadSink(export.File.Directory!, node ?? Environment.MachineName),
};
return new Central.HeadExport.AuditChainHeadExportService(
sp.GetRequiredService<IServiceScopeFactory>(),
options,
sink,
node,
sp.GetRequiredService<ILogger<Central.HeadExport.AuditChainHeadExportService>>(),
status: sp.GetRequiredService<Central.HeadExport.AuditChainHeadExportStatus>());
});
// The hash chain and the two-login maintenance hardening were removed on 2026-11-15;
// a node still carrying their settings boots normally and is told to delete them.
services.AddHostedService<RetiredAuditSettingsWarning>();
// Parquet cold archive (ScadaBridge#42). Default OFF; when enabled the purge actor
// archives each expired partition before switching it out. IAuditLogArchive is
@@ -11,16 +11,16 @@ namespace ZB.MOM.WW.ScadaBridge.CLI.Commands;
/// </summary>
/// <remarks>
/// <c>verify</c> exit codes: <c>0</c> verified, <see cref="NotVerifiedExitCode"/> when the
/// files, the chain or the purge anchor disagree, <c>1</c> on a request error, <c>2</c> on an
/// data file disagrees with its manifest, <c>1</c> on a request error, <c>2</c> on an
/// authorization failure.
/// </remarks>
public static class AuditArchiveHelpers
{
/// <summary>Process exit code when an archive failed verification (tamper evidence).</summary>
/// <summary>Process exit code when an archive failed verification.</summary>
public const int NotVerifiedExitCode = 3;
/// <summary>
/// Client deadline for <c>verify</c>: the server hashes and re-chains a whole month under
/// Client deadline for <c>verify</c>: the server hashes and reads a whole month under
/// the long-running management Ask (5 min); the client waits a little longer.
/// </summary>
public static readonly TimeSpan VerifyRequestTimeout = TimeSpan.FromMinutes(6);
@@ -50,11 +50,11 @@ public static class AuditArchiveHelpers
return 0;
}
output.WriteLine($"{"NAME",-42} {"PARTITION",-9} {"STATUS",-8} {"ROWS",12} {"CHAINED",12} {"BYTES",14}");
output.WriteLine($"{"NAME",-42} {"PARTITION",-9} {"STATUS",-8} {"ROWS",12} {"BYTES",14}");
foreach (var a in archives.EnumerateArray())
{
output.WriteLine(string.Create(CultureInfo.InvariantCulture,
$"{Str(a, "name"),-42} {Str(a, "partition"),-9} {Str(a, "status"),-8} {Num(a, "rowCount"),12} {Num(a, "chainedRowCount"),12} {Num(a, "dataBytes"),14}"));
$"{Str(a, "name"),-42} {Str(a, "partition"),-9} {Str(a, "status"),-8} {Num(a, "rowCount"),12} {Num(a, "dataBytes"),14}"));
}
output.Flush();
@@ -90,20 +90,6 @@ public static class AuditArchiveHelpers
}
output.WriteLine($"Rows read : {Num(root, "rowsRead")}");
if (root.TryGetProperty("chains", out var chains) && chains.ValueKind == JsonValueKind.Array)
{
foreach (var c in chains.EnumerateArray())
{
var chain = c.GetProperty("chain");
output.WriteLine($"Chain {Str(chain, "partition")}: {Str(chain, "status")}, {Num(chain, "rowsVerified")} rows verified, "
+ $"{Num(chain, "tombstonesApplied")} tombstones, {Num(chain, "unchainedRows")} unchained; anchor {Str(c, "anchor")}");
output.WriteLine($" {Str(c, "anchorDetail")}");
if (chain.TryGetProperty("firstBreak", out var brk) && brk.ValueKind == JsonValueKind.Object)
{
output.WriteLine($" first break: chainSeq {Num(brk, "chainSeq")}, eventId {Str(brk, "eventId", "-")}, {Str(brk, "kind")} — {Str(brk, "detail")}");
}
}
}
output.Flush();
return verified ? 0 : NotVerifiedExitCode;
@@ -8,17 +8,14 @@ namespace ZB.MOM.WW.ScadaBridge.CLI.Commands;
/// The <c>scadabridge audit</c> command group. Provides read access to
/// the centralized append-only Audit Log via the REST endpoints
/// (<c>GET /api/audit/query</c>, <c>GET /api/audit/export</c>,
/// <c>GET /api/audit/tree</c>), plus <c>verify-chain</c>, which sends the
/// <c>VerifyAuditChain</c> management command (hash-chain tamper evidence, ScadaBridge#41),
/// <c>verify-heads</c>, which checks off-box exported chain heads with
/// <c>VerifyAuditChainHeads</c> (ScadaBridge#107), and <c>archive list</c> / <c>archive verify</c>
/// <c>GET /api/audit/tree</c>), plus <c>archive list</c> / <c>archive verify</c>
/// / <c>archive cleanup</c>, which send <c>ListAuditArchives</c> / <c>VerifyAuditArchive</c>
/// (Parquet cold archive, ScadaBridge#42) / <c>CleanupAuditArchives</c> (ScadaBridge#109).
/// </summary>
public static class AuditCommands
{
/// <summary>
/// Builds the <c>audit</c> command group with query, export, tree, and verify-chain
/// Builds the <c>audit</c> command group with query, export, tree and archive
/// sub-commands.
/// </summary>
/// <param name="urlOption">Global <c>--url</c> option for the management API endpoint.</param>
@@ -33,8 +30,6 @@ public static class AuditCommands
command.Add(BuildQuery(urlOption, formatOption, usernameOption, passwordOption));
command.Add(BuildExport(urlOption, formatOption, usernameOption, passwordOption));
command.Add(BuildTree(urlOption, formatOption, usernameOption, passwordOption));
command.Add(BuildVerifyChain(urlOption, formatOption, usernameOption, passwordOption));
command.Add(BuildVerifyHeads(urlOption, formatOption, usernameOption, passwordOption));
command.Add(BuildBackfillSourceNode(urlOption, formatOption, usernameOption, passwordOption));
command.Add(BuildArchive(urlOption, formatOption, usernameOption, passwordOption));
@@ -293,87 +288,10 @@ public static class AuditCommands
return cmd;
}
private static Command BuildVerifyChain(Option<string> urlOption, Option<string> formatOption, Option<string> usernameOption, Option<string> passwordOption)
{
var monthOption = new Option<string>("--month") { Description = "UTC calendar month to verify (YYYY-MM)", Required = true };
var cmd = new Command("verify-chain")
{
Description = "Verify the audit log hash chain for a month; exit 3 and report the first break if it was tampered with",
};
cmd.Add(monthOption);
cmd.SetAction(async (ParseResult result) =>
{
var month = result.GetValue(monthOption)!;
if (!AuditVerifyChainHelpers.IsValidMonth(month))
{
OutputFormatter.WriteError(
$"Invalid month '{month}'. Expected YYYY-MM (e.g. 2026-05).", "INVALID_ARGUMENT");
return 1;
}
var format = AuditCommandHelpers.ResolveFormat(result, formatOption);
return await CommandHelpers.ExecuteCommandAsync(
result, urlOption, formatOption, usernameOption, passwordOption,
new VerifyAuditChainCommand(month),
timeout: AuditVerifyChainHelpers.RequestTimeout,
onSuccess: json => AuditVerifyChainHelpers.Render(json, format, Console.Out));
});
return cmd;
}
/// <summary>
/// Builds the <c>audit archive</c> group (ScadaBridge#42): <c>list</c> and <c>verify</c>
/// over the AuditLog Parquet cold archives on the central archive directory.
/// Builds the <c>audit archive</c> group (ScadaBridge#42): <c>list</c>, <c>verify</c> and
/// <c>cleanup</c> over the AuditLog Parquet cold archives on the central archive directory.
/// </summary>
private static Command BuildVerifyHeads(Option<string> urlOption, Option<string> formatOption, Option<string> usernameOption, Option<string> passwordOption)
{
var pathOption = new Option<string>("--path")
{
Description = "Export file, or directory of *.jsonl/*.log/*.txt export files (file sink or syslog collector output)",
Required = true,
};
var cmd = new Command("verify-heads")
{
Description = "Check AuditLog chain heads exported off-box against the database chain; exit 3 if any no longer matches",
};
cmd.Add(pathOption);
cmd.SetAction(async (ParseResult result) =>
{
AuditVerifyHeadsHelpers.ReadResult read;
try
{
read = AuditVerifyHeadsHelpers.ReadRecords(result.GetValue(pathOption)!);
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
{
OutputFormatter.WriteError(ex.Message, "INVALID_ARGUMENT");
return 1;
}
if (read.Records.Count == 0)
{
OutputFormatter.WriteError(
$"No chain-head records found in {read.Files} file(s) ({read.LinesSkipped} unparsable line(s)).",
"INVALID_ARGUMENT");
return 1;
}
if (read.LinesSkipped > 0)
{
Console.Error.WriteLine($"warning: skipped {read.LinesSkipped} line(s) without a chain-head record.");
}
var format = AuditCommandHelpers.ResolveFormat(result, formatOption);
return await CommandHelpers.ExecuteCommandAsync(
result, urlOption, formatOption, usernameOption, passwordOption,
new VerifyAuditChainHeadsCommand(read.Records),
timeout: AuditVerifyHeadsHelpers.RequestTimeout,
onSuccess: json => AuditVerifyHeadsHelpers.Render(json, format, Console.Out));
});
return cmd;
}
private static Command BuildArchive(Option<string> urlOption, Option<string> formatOption, Option<string> usernameOption, Option<string> passwordOption)
{
var group = new Command("archive") { Description = "List, verify and clean up the AuditLog Parquet cold archives" };
@@ -396,7 +314,7 @@ public static class AuditCommands
};
var verify = new Command("verify")
{
Description = "Verify an archive's files, its hash chain and its purge anchor; exit 3 if it does not verify",
Description = "Verify an archive's data file against its manifest; exit 3 if it does not verify",
};
verify.Add(nameOption);
verify.SetAction(async (ParseResult result) =>
@@ -1,88 +0,0 @@
using System.Globalization;
using System.Text.Json;
namespace ZB.MOM.WW.ScadaBridge.CLI.Commands;
/// <summary>
/// Helpers for the <c>audit verify-chain</c> subcommand (ScadaBridge#41). The command sends
/// the <c>VerifyAuditChain</c> management command for one UTC calendar month and reports
/// whether that month's AuditLog hash chain is intact, or the first break (chain partition,
/// <c>ChainSeq</c>, <c>EventId</c>, kind).
/// </summary>
/// <remarks>
/// Exit codes: <c>0</c> intact (or nothing chained yet), <see cref="BrokenExitCode"/> when
/// the chain is broken, <c>1</c> on a request error, <c>2</c> on an authorization failure.
/// </remarks>
public static class AuditVerifyChainHelpers
{
/// <summary>Process exit code when verification found a break (tamper evidence).</summary>
public const int BrokenExitCode = 3;
/// <summary>
/// Client deadline. The server bounds the call with the long-running management Ask
/// (5 min); the client waits a little longer so the server's answer always arrives.
/// </summary>
public static readonly TimeSpan RequestTimeout = TimeSpan.FromMinutes(6);
/// <summary>
/// Returns true if <paramref name="month"/> is a well-formed <c>YYYY-MM</c> value
/// with a real month (01-12). A malformed month (e.g. <c>2026-13</c>) is rejected.
/// </summary>
/// <param name="month">The month string to validate in YYYY-MM format.</param>
/// <returns><c>true</c> if the string is a well-formed YYYY-MM value with a real month; otherwise <c>false</c>.</returns>
public static bool IsValidMonth(string? month)
=> !string.IsNullOrWhiteSpace(month)
&& DateTime.TryParseExact(month, "yyyy-MM", CultureInfo.InvariantCulture,
DateTimeStyles.None, out _);
/// <summary>
/// Renders a <c>VerifyAuditChain</c> result and maps it to the process exit code.
/// <c>json</c> prints the server body verbatim; <c>table</c> prints a short report.
/// </summary>
/// <param name="json">The success body (an <c>AuditChainVerificationResult</c>, camelCase).</param>
/// <param name="format">Resolved output format.</param>
/// <param name="output">Writer for the report.</param>
/// <returns><c>0</c> when intact / not chained, <see cref="BrokenExitCode"/> when broken.</returns>
public static int Render(string json, string format, TextWriter output)
{
using var doc = JsonDocument.Parse(json);
var root = doc.RootElement;
var status = root.TryGetProperty("status", out var s) ? s.GetString() ?? "" : "";
var broken = string.Equals(status, "Broken", StringComparison.OrdinalIgnoreCase);
if (!string.Equals(format, "table", StringComparison.OrdinalIgnoreCase))
{
output.WriteLine(json);
output.Flush();
return broken ? BrokenExitCode : 0;
}
output.WriteLine($"Partition : {Str(root, "partition")}");
output.WriteLine($"Status : {status}");
output.WriteLine($"Rows verified : {Num(root, "rowsVerified")}");
output.WriteLine($"Tombstones applied: {Num(root, "tombstonesApplied")}");
output.WriteLine($"Purged through : {Num(root, "purgedThroughSeq", "-")}");
output.WriteLine($"Head sequence : {Num(root, "headSeq", "-")}");
output.WriteLine($"Unchained rows : {Num(root, "unchainedRows")}");
output.WriteLine($"Chain from : {Str(root, "chainFromUtc", "- (whole table in scope)")}");
output.WriteLine($"Pre-cutoff rows : {Num(root, "preCutoffRows")}");
if (root.TryGetProperty("firstBreak", out var brk) && brk.ValueKind == JsonValueKind.Object)
{
output.WriteLine("First break:");
output.WriteLine($" partition : {Str(brk, "partition")}");
output.WriteLine($" chainSeq : {Num(brk, "chainSeq")}");
output.WriteLine($" eventId : {Str(brk, "eventId", "-")}");
output.WriteLine($" kind : {Str(brk, "kind")}");
output.WriteLine($" detail : {Str(brk, "detail")}");
}
output.Flush();
return broken ? BrokenExitCode : 0;
}
private static string Str(JsonElement e, string name, string missing = "")
=> e.TryGetProperty(name, out var v) && v.ValueKind == JsonValueKind.String ? v.GetString() ?? missing : missing;
private static string Num(JsonElement e, string name, string missing = "0")
=> e.TryGetProperty(name, out var v) && v.ValueKind == JsonValueKind.Number ? v.GetInt64().ToString(CultureInfo.InvariantCulture) : missing;
}
@@ -1,160 +0,0 @@
using System.Globalization;
using System.Text.Json;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Audit;
namespace ZB.MOM.WW.ScadaBridge.CLI.Commands;
/// <summary>
/// Helpers for <c>audit verify-heads</c> (ScadaBridge#107 item 1): read AuditLog chain heads
/// exported off-box (the <c>AuditLog:HashChain:HeadExport</c> JSON lines — straight from the
/// file sink, or from a syslog collector's log), send them in a
/// <c>VerifyAuditChainHeads</c> management command, and render the verdicts.
/// </summary>
/// <remarks>
/// Exit codes: <c>0</c> every exported head is consistent with the database,
/// <see cref="FailedExitCode"/> at least one is not (tamper evidence), <c>1</c> on a request
/// or input error, <c>2</c> on an authorization failure.
/// </remarks>
public static class AuditVerifyHeadsHelpers
{
/// <summary>Process exit code when a head check failed.</summary>
public const int FailedExitCode = 3;
/// <summary>Client deadline (the server bounds the call with the long-running Ask).</summary>
public static readonly TimeSpan RequestTimeout = TimeSpan.FromMinutes(6);
private static readonly JsonSerializerOptions ReadOptions = new(JsonSerializerDefaults.Web);
/// <summary>The result of reading export files.</summary>
/// <param name="Records">Distinct records (by month, sequence, hash), latest export kept.</param>
/// <param name="LinesRead">Lines that held a head record.</param>
/// <param name="LinesSkipped">Non-empty lines without a parsable record.</param>
/// <param name="Files">Files read.</param>
public sealed record ReadResult(IReadOnlyList<AuditChainHeadRecord> Records, int LinesRead, int LinesSkipped, int Files);
/// <summary>
/// Reads every record from <paramref name="path"/> — one file, or every <c>*.jsonl</c> /
/// <c>*.log</c> / <c>*.txt</c> file below a directory. Each line's JSON object is taken from
/// its first <c>{</c> to its last <c>}</c>, so syslog-prefixed lines parse too.
/// </summary>
/// <param name="path">File or directory.</param>
/// <returns>The records read.</returns>
/// <exception cref="FileNotFoundException">The path does not exist.</exception>
public static ReadResult ReadRecords(string path)
{
string[] files;
if (File.Exists(path))
{
files = [path];
}
else if (Directory.Exists(path))
{
files = Directory.EnumerateFiles(path, "*", SearchOption.AllDirectories)
.Where(f => f.EndsWith(".jsonl", StringComparison.OrdinalIgnoreCase)
|| f.EndsWith(".log", StringComparison.OrdinalIgnoreCase)
|| f.EndsWith(".txt", StringComparison.OrdinalIgnoreCase))
.Order(StringComparer.Ordinal)
.ToArray();
}
else
{
throw new FileNotFoundException($"'{path}' is neither a file nor a directory.", path);
}
var records = new List<AuditChainHeadRecord>();
int read = 0, skipped = 0;
foreach (var file in files)
{
foreach (var line in File.ReadLines(file))
{
if (string.IsNullOrWhiteSpace(line))
{
continue;
}
var record = TryParseLine(line);
if (record is null)
{
skipped++;
continue;
}
read++;
records.Add(record);
}
}
var distinct = records
.GroupBy(r => (r.Month, r.LastSeq, Hash: r.LastHash.ToLowerInvariant()))
.Select(g => g.OrderByDescending(r => r.ExportedAtUtc).First())
.OrderBy(r => r.Month, StringComparer.Ordinal)
.ThenBy(r => r.LastSeq)
.ToList();
return new ReadResult(distinct, read, skipped, files.Length);
}
/// <summary>Parses one export line; null when it holds no valid record.</summary>
/// <param name="line">The line.</param>
/// <returns>The record, or null.</returns>
public static AuditChainHeadRecord? TryParseLine(string line)
{
var start = line.IndexOf('{');
var end = line.LastIndexOf('}');
if (start < 0 || end <= start)
{
return null;
}
try
{
var record = JsonSerializer.Deserialize<AuditChainHeadRecord>(line.AsSpan(start, end - start + 1), ReadOptions);
return record is { V: >= 1, Month.Length: 7, LastHash.Length: > 0 } && record.LastSeq > 0 ? record : null;
}
catch (JsonException)
{
return null;
}
}
/// <summary>Renders a <c>VerifyAuditChainHeads</c> result and maps it to the exit code.</summary>
/// <param name="json">Success body (an <c>AuditChainHeadVerificationResult</c>, camelCase).</param>
/// <param name="format">Resolved output format.</param>
/// <param name="output">Writer for the report.</param>
/// <returns><c>0</c> when verified, <see cref="FailedExitCode"/> otherwise.</returns>
public static int Render(string json, string format, TextWriter output)
{
using var doc = JsonDocument.Parse(json);
var root = doc.RootElement;
var verified = root.TryGetProperty("verified", out var v) && v.ValueKind == JsonValueKind.True;
if (!string.Equals(format, "table", StringComparison.OrdinalIgnoreCase))
{
output.WriteLine(json);
output.Flush();
return verified ? 0 : FailedExitCode;
}
output.WriteLine($"Verified : {(verified ? "yes" : "NO")}");
output.WriteLine($"Heads checked : {Num(root, "recordsChecked")}");
if (root.TryGetProperty("checks", out var checks) && checks.ValueKind == JsonValueKind.Array)
{
output.WriteLine($"{"MONTH",-8} {"EXPORTED SEQ",12} {"DB SEQ",8} {"STATUS",-13} DETAIL");
foreach (var c in checks.EnumerateArray())
{
output.WriteLine(string.Create(CultureInfo.InvariantCulture,
$"{Str(c, "month"),-8} {Num(c, "exportedSeq"),12} {Num(c, "databaseSeq", "-"),8} {Str(c, "status"),-13} {Str(c, "detail")}"));
}
}
output.Flush();
return verified ? 0 : FailedExitCode;
}
private static string Str(JsonElement e, string name)
=> e.TryGetProperty(name, out var v) && v.ValueKind == JsonValueKind.String ? v.GetString() ?? "" : "";
private static string Num(JsonElement e, string name, string missing = "0")
=> e.TryGetProperty(name, out var v) && v.ValueKind == JsonValueKind.Number
? v.GetInt64().ToString(CultureInfo.InvariantCulture)
: missing;
}
@@ -1,11 +0,0 @@
namespace ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
/// <summary>DI keys for the AuditLog maintenance-login registrations (ScadaBridge#107/#113).</summary>
public static class AuditLogServiceKeys
{
/// <summary>
/// Key of the maintenance-login <see cref="IAuditLogRepository"/> / <c>IAuditLogArchive</c>:
/// the connection from <c>AuditLog:MaintenanceConnectionString</c>, or the runtime connection when unset.
/// </summary>
public const string Maintenance = "audit-maintenance";
}
@@ -1,28 +0,0 @@
using ZB.MOM.WW.ScadaBridge.Commons.Types.Audit;
namespace ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
/// <summary>
/// Reads the AuditLog hash-chain heads for off-box export and checks exported heads against
/// the database chain (ScadaBridge#107 item 1 — the external anchor of the #41 chain).
/// </summary>
public interface IAuditChainHeadStore
{
/// <summary>
/// Reads every month's chain head with its live chained-row and tombstone counts, in one
/// consistent snapshot (under the chain lock SHARED, so no chaining pass or purge is half-way).
/// </summary>
/// <param name="ct">Cancellation token.</param>
/// <returns>One snapshot per month that has a head, oldest first.</returns>
Task<IReadOnlyList<AuditChainHeadSnapshot>> ReadHeadsAsync(CancellationToken ct = default);
/// <summary>
/// Checks exported head records against the database: for each distinct exported head, does
/// the database still carry the exported hash at the exported sequence?
/// </summary>
/// <param name="records">Exported records (duplicates are collapsed).</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>The per-record verdicts.</returns>
Task<AuditChainHeadVerificationResult> VerifyExportedHeadsAsync(
IReadOnlyList<AuditChainHeadRecord> records, CancellationToken ct = default);
}
@@ -10,16 +10,10 @@ namespace ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
/// <remarks>
/// <para>
/// <b>Archive before purge.</b> <see cref="ArchivePartitionAsync"/> writes the partition's
/// rows (with their hash-chain columns), the month's chain head and its per-channel purge
/// tombstones; <see cref="PurgeArchivedPartitionAsync"/> then switches the partition out
/// inside a guard that refuses to drop it if the partition gained or lost a row, or its
/// chain advanced, since the archive was written. A partition is therefore never purged
/// without an archive that holds exactly the rows the purge drops.
/// </para>
/// <para>
/// <b>Hash chain.</b> The switch records each month's chain end as the checkpoint anchor
/// (ScadaBridge#41); the archive records the same chain end, so <see cref="VerifyAsync"/>
/// can recompute the archived chain and tie its end to the anchor still in the database.
/// rows; <see cref="PurgeArchivedPartitionAsync"/> then switches the partition out inside a
/// guard that refuses to drop it if the partition gained or lost a row since the archive was
/// written. A partition is therefore never purged without an archive that holds exactly the
/// rows the purge drops.
/// </para>
/// </remarks>
public interface IAuditLogArchive
@@ -58,8 +52,7 @@ public interface IAuditLogArchive
Task<IReadOnlyList<AuditArchiveManifest>> ListAsync(CancellationToken ct = default);
/// <summary>
/// Verifies one archive: file integrity against the manifest, the hash chain recomputed
/// over the archived rows, and the chain end against the database's checkpoint anchor.
/// Verifies one archive: data-file integrity (size, SHA-256, row count) against the manifest.
/// </summary>
/// <param name="name">The archive name (as <see cref="ListAsync"/> reports it).</param>
/// <param name="ct">Cancellation token.</param>
@@ -70,7 +63,7 @@ public interface IAuditLogArchive
/// <summary>
/// Archive cleanup (ScadaBridge#109): deletes superseded stale archives a retried purge left
/// behind (when <c>AuditLog:Archive:DeleteSupersededStaleArchives</c>), and archives past
/// <c>AuditLog:Archive:RetentionDays</c> (0 = never). Only ever deletes the three files of a
/// <c>AuditLog:Archive:RetentionDays</c> (0 = never). Only ever deletes the files of a
/// well-formed archive inside the configured directory, manifest first.
/// </summary>
/// <param name="dryRun">Report only; delete nothing.</param>
@@ -81,7 +74,7 @@ public interface IAuditLogArchive
/// <summary>
/// Thrown by <see cref="IAuditLogArchive.PurgeArchivedPartitionAsync"/> when the partition
/// gained or lost rows, or its hash chain advanced, after the archive was written. Nothing
/// gained or lost rows after the archive was written. Nothing
/// is purged; the next purge tick writes a fresh archive.
/// </summary>
public sealed class AuditArchiveStaleException : Exception
@@ -1,58 +0,0 @@
using ZB.MOM.WW.ScadaBridge.Commons.Types.Audit;
namespace ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
/// <summary>
/// Hash-chain tamper evidence for the central <c>AuditLog</c> table (audit roadmap T1,
/// alog.md §11.4, ScadaBridge#41).
/// </summary>
/// <remarks>
/// <para>
/// One chain per <b>partition</b> = UTC calendar month of <c>OccurredAtUtc</c> (the same
/// month the <c>ps_AuditLog_Month</c> scheme partitions on). Each chained row carries a
/// <c>ChainSeq</c> (1, 2, 3 … within its month) and
/// <c>RowHash = SHA-256(prev.RowHash || canonical(row))</c>, where <c>prev</c> is the row
/// at <c>ChainSeq - 1</c> (32 zero bytes for the first). The canonical serialisation
/// covers every writable column plus the month and sequence number.
/// </para>
/// <para>
/// Rows are linked by a <b>chaining job</b>, not by the hot-path writers: two central
/// nodes and the site-ingest path all INSERT concurrently, so a write-time chain would
/// need a global serialisation point on every audit write. The job serialises itself
/// with an exclusive SQL Server application lock, so even a dual-active central pair
/// produces one gap-free chain per month.
/// </para>
/// </remarks>
public interface IAuditLogHashChain
{
/// <summary>
/// Links up to <paramref name="batchSize"/> unchained rows (oldest
/// <c>OccurredAtUtc</c> first) onto the end of their month's chain, in one transaction.
/// Returns immediately with <see cref="AuditChainBatchResult.LockBusy"/> set when another
/// chainer holds the chain lock.
/// </summary>
/// <remarks>
/// <b>Chain start (ScadaBridge#139).</b> On a pristine database (nothing ever chained) and
/// with <paramref name="chainBacklog"/> false, the first pass records a chain-start anchor
/// at the database's current UTC time; from then on only rows dated at or after it are
/// chained, whatever either node passes later. With <paramref name="chainBacklog"/> true on
/// that first pass, no anchor is written and every row is chained, oldest month first. A
/// database that already has a chain and no anchor keeps chaining every row.
/// </remarks>
/// <param name="batchSize">Maximum rows to chain in this pass.</param>
/// <param name="chainBacklog">
/// Only consulted on a pristine database: true chains the historical backlog instead of
/// starting the chain now.
/// </param>
/// <param name="ct">Cancellation token.</param>
/// <returns>How many rows were chained, and whether the lock was busy.</returns>
Task<AuditChainBatchResult> ChainPendingAsync(int batchSize, bool chainBacklog = false, CancellationToken ct = default);
/// <summary>
/// Verifies one month's chain end to end and reports the first break.
/// </summary>
/// <param name="monthStartUtc">The first instant of the month (UTC, day 1, midnight).</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>The verification result.</returns>
Task<AuditChainVerificationResult> VerifyMonthAsync(DateTime monthStartUtc, CancellationToken ct = default);
}
@@ -158,11 +158,6 @@ public interface IAuditLogRepository
/// <c>OccurredAtUtc</c> is strictly older than <paramref name="threshold"/>, in
/// bounded batches of <paramref name="batchSize"/> rows, looping until no further
/// rows match. Returns the total number of rows deleted across all batches.
/// <para>
/// Hash chain (ScadaBridge#41): every CHAINED row the DELETE removes leaves a
/// <c>dbo.AuditLogChainTombstone</c> (month, ChainSeq, RowHash) written in the same
/// transaction, under the exclusive chain lock, so the month's chain still verifies.
/// </para>
/// </summary>
/// <remarks>
/// <para>
@@ -25,23 +25,6 @@ public record QueryAuditLogCommand(
/// <param name="TotalCount">Total number of rows matching the filters (unpaged).</param>
public record AuditLogQueryResult(IReadOnlyList<AuditLogEntry> Entries, int TotalCount);
/// <summary>
/// Verifies the central AuditLog hash chain for one UTC calendar month (audit roadmap T1,
/// ScadaBridge#41). Read-only; gated on the OperationalAudit roles and refused for a
/// site-scoped caller, because a month's chain spans every site.
/// </summary>
/// <param name="Month">The month to verify, <c>YYYY-MM</c>.</param>
public record VerifyAuditChainCommand(string Month);
/// <summary>
/// Checks AuditLog chain heads exported off-box (<c>AuditLog:HashChain:HeadExport</c>,
/// ScadaBridge#107 item 1) against the database chain: does the database still carry each
/// exported hash at its exported sequence? Read-only; same gate as
/// <see cref="VerifyAuditChainCommand"/>. The CLI reads the records from the export files.
/// </summary>
/// <param name="Records">The exported head records (at most 50,000; duplicates are collapsed).</param>
public record VerifyAuditChainHeadsCommand(IReadOnlyList<AuditChainHeadRecord> Records);
/// <summary>
/// Lists the AuditLog Parquet cold archives under the configured archive directory (audit
/// roadmap T2, ScadaBridge#42). Read-only; OperationalAudit roles, refused for a site-scoped
@@ -50,9 +33,8 @@ public record VerifyAuditChainHeadsCommand(IReadOnlyList<AuditChainHeadRecord> R
public record ListAuditArchivesCommand;
/// <summary>
/// Verifies one AuditLog Parquet archive (ScadaBridge#42): file integrity against its
/// manifest, the hash chain recomputed over the archived rows, and the chain end against the
/// checkpoint anchor the partition purge left. Read-only; same gate as
/// Verifies one AuditLog Parquet archive (ScadaBridge#42): data-file integrity against its
/// manifest (size, SHA-256, row count). Read-only; same gate as
/// <see cref="ListAuditArchivesCommand"/>.
/// </summary>
/// <param name="Name">The archive name, as <c>audit archive list</c> reports it.</param>
@@ -11,8 +11,7 @@ public enum AuditArchiveStatus
/// <summary>
/// The archive files are durable but the partition switch that should follow has not
/// been confirmed: either it failed (the archive went stale and a later tick wrote a
/// fresh one) or the node stopped between the switch and the manifest update. Verify
/// tells the two apart: only the second one's chain end matches the database anchor.
/// fresh one) or the node stopped between the switch and the manifest update.
/// </summary>
Archived,
@@ -20,48 +19,25 @@ public enum AuditArchiveStatus
Purged,
}
/// <summary>
/// One chain partition (UTC calendar month) as it stood when its rows were archived: the
/// chain head and the checkpoint anchor an earlier partition purge had left, so the archived
/// rows can be re-verified with <c>AuditChainVerifier</c> long after the database dropped them.
/// </summary>
/// <param name="Partition">The chain partition, <c>YYYY-MM</c>.</param>
/// <param name="LastSeq">The head's last sequence number at archive time.</param>
/// <param name="LastHash">The head's last hash at archive time (lowercase hex).</param>
/// <param name="PriorPurgedThroughSeq">The checkpoint anchor an earlier purge of this month left, or null.</param>
/// <param name="PriorPurgedThroughHash">The hash at <paramref name="PriorPurgedThroughSeq"/> (lowercase hex), or null.</param>
/// <param name="TombstoneCount">Per-channel purge tombstones of this month carried in the archive.</param>
public sealed record AuditArchiveChainState(
string Partition,
long LastSeq,
string LastHash,
long? PriorPurgedThroughSeq,
string? PriorPurgedThroughHash,
long TombstoneCount);
/// <summary>
/// The JSON manifest written beside every AuditLog Parquet archive (ScadaBridge#42). It is
/// written LAST, so its presence is the commit marker for the archive's two Parquet files.
/// written LAST, so its presence is the commit marker for the archive's Parquet data file.
/// A format-v1 manifest (written while the audit hash chain existed, before 2026-11-15) also
/// carries chain and tombstone fields; they are ignored when it is read.
/// </summary>
/// <param name="FormatVersion">Manifest/file layout version (currently 1).</param>
/// <param name="Name">Archive name, <c>auditlog-YYYY-MM-yyyyMMddTHHmmssfffZ</c>; the file stem of all three files.</param>
/// <param name="FormatVersion">Manifest/file layout version (currently 2; 1 = written with the hash chain).</param>
/// <param name="Name">Archive name, <c>auditlog-YYYY-MM-yyyyMMddTHHmmssfffZ</c>; the file stem of the archive's files.</param>
/// <param name="PartitionBoundaryUtc">The <c>pf_AuditLog_Month</c> lower boundary of the archived partition.</param>
/// <param name="Partition">The archived partition, <c>YYYY-MM</c>.</param>
/// <param name="ArchivedAtUtc">When the archive was written.</param>
/// <param name="Status">Whether the partition switch that follows the archive has been confirmed.</param>
/// <param name="PurgedAtUtc">When the partition switch completed, or null.</param>
/// <param name="RowCount">Rows in the data file.</param>
/// <param name="ChainedRowCount">Rows that carried a ChainSeq/RowHash.</param>
/// <param name="UnchainedRowCount">Rows the chaining job had not reached (not covered by the chain).</param>
/// <param name="MinOccurredAtUtc">Earliest OccurredAtUtc in the data file, or null when empty.</param>
/// <param name="MaxOccurredAtUtc">Latest OccurredAtUtc in the data file, or null when empty.</param>
/// <param name="DataFile">Data file name, relative to the manifest's directory.</param>
/// <param name="DataSha256">SHA-256 of the data file (lowercase hex).</param>
/// <param name="DataBytes">Size of the data file in bytes.</param>
/// <param name="TombstoneFile">Tombstone file name, relative to the manifest's directory.</param>
/// <param name="TombstoneSha256">SHA-256 of the tombstone file (lowercase hex).</param>
/// <param name="TombstoneCount">Tombstones in the tombstone file.</param>
/// <param name="Chains">One entry per chain partition that had a head when the rows were archived.</param>
public sealed record AuditArchiveManifest(
int FormatVersion,
string Name,
@@ -71,66 +47,23 @@ public sealed record AuditArchiveManifest(
AuditArchiveStatus Status,
DateTime? PurgedAtUtc,
long RowCount,
long ChainedRowCount,
long UnchainedRowCount,
DateTime? MinOccurredAtUtc,
DateTime? MaxOccurredAtUtc,
string DataFile,
string DataSha256,
long DataBytes,
string TombstoneFile,
string TombstoneSha256,
long TombstoneCount,
IReadOnlyList<AuditArchiveChainState> Chains);
long DataBytes);
/// <summary>
/// How an archive's recorded chain end relates to the checkpoint anchor the partition purge
/// left on the database's chain head.
/// </summary>
[JsonConverter(typeof(JsonStringEnumConverter<AuditArchiveAnchorStatus>))]
public enum AuditArchiveAnchorStatus
{
/// <summary>The database anchor is exactly the archive's chain end (sequence and hash).</summary>
Matches,
/// <summary>
/// The database anchor is past the archive's chain end: rows that arrived late for this
/// month were chained and purged (and archived) again afterwards.
/// </summary>
Superseded,
/// <summary>The partition has not been purged: the archive's rows are still in <c>dbo.AuditLog</c>.</summary>
NotPurged,
/// <summary>
/// The anchor disagrees with the archive (an earlier sequence number, a different hash, or
/// no chain head at all): the archive or the head was altered.
/// </summary>
Mismatch,
}
/// <summary>Verification of one chain partition inside an archive.</summary>
/// <param name="Chain">The chain recomputed over the archived rows, the archived head and tombstones.</param>
/// <param name="Anchor">How the archived chain end relates to the database's checkpoint anchor.</param>
/// <param name="AnchorDetail">Human-readable explanation of <paramref name="Anchor"/>.</param>
public sealed record AuditArchiveChainVerification(
AuditChainVerificationResult Chain,
AuditArchiveAnchorStatus Anchor,
string AnchorDetail);
/// <summary>
/// Result of <c>audit archive verify</c> (ScadaBridge#42): the archive files match their
/// manifest, every archived chain recomputes to the archived head, and each chain end is the
/// anchor the partition purge left in the database.
/// Result of <c>audit archive verify</c> (ScadaBridge#42): the data file matches its manifest
/// (size, SHA-256 and row count).
/// </summary>
/// <param name="Name">The archive verified.</param>
/// <param name="Partition">The archived partition, <c>YYYY-MM</c>.</param>
/// <param name="ManifestStatus">The manifest's lifecycle status.</param>
/// <param name="Verified">True when the files are intact, no chain is broken and no anchor mismatches.</param>
/// <param name="FilesIntact">True when both files' size and SHA-256 and the row/tombstone counts match the manifest.</param>
/// <param name="Verified">True when the data file is intact.</param>
/// <param name="FilesIntact">True when the data file's size, SHA-256 and row count match the manifest.</param>
/// <param name="FileProblem">Why <paramref name="FilesIntact"/> is false, or null.</param>
/// <param name="RowsRead">Rows read from the data file.</param>
/// <param name="Chains">Per chain partition verification.</param>
public sealed record AuditArchiveVerificationResult(
string Name,
string Partition,
@@ -138,8 +71,7 @@ public sealed record AuditArchiveVerificationResult(
bool Verified,
bool FilesIntact,
string? FileProblem,
long RowsRead,
IReadOnlyList<AuditArchiveChainVerification> Chains);
long RowsRead);
/// <summary>Result of <c>audit archive list</c> (ScadaBridge#42).</summary>
/// <param name="Enabled">Whether the retention purge is archiving partitions (<c>AuditLog:Archive:Enabled</c>).</param>
@@ -159,8 +91,7 @@ public enum AuditArchiveCleanupAction
/// <summary>
/// Deleted as a superseded stale archive: status <c>Archived</c> (its partition switch never
/// completed — a retried purge), and a LATER, verified, <c>Purged</c> archive of the same
/// partition provably holds every row it holds (same chain through its chain end, every
/// unchained row present by EventId).
/// partition provably holds every row it holds (every row present by EventId).
/// </summary>
DeletedSuperseded,
@@ -172,7 +103,7 @@ public enum AuditArchiveCleanupAction
/// <summary>
/// Deleted as an orphan of an interrupted archive write (ScadaBridge#114): a temp file of the
/// archive writer (<c>*.parquet.tmp</c>, <c>*.tombstones.parquet.tmp</c>, <c>*.manifest.json.tmp</c>)
/// archive writer (<c>*.parquet.tmp</c>, <c>*.manifest.json.tmp</c>, or a format-v1 <c>*.tombstones.parquet.tmp</c>)
/// or a data file whose <c>.manifest.json</c> was never written, older than the safe age.
/// </summary>
DeletedOrphan,
@@ -1,126 +0,0 @@
using System.Text.Json.Serialization;
namespace ZB.MOM.WW.ScadaBridge.Commons.Types.Audit;
/// <summary>
/// One month's AuditLog hash-chain head as exported off-box (ScadaBridge#107 item 1). The
/// export is the external anchor the in-database chain lacks: a principal able to rewrite rows,
/// recompute every later link and update <c>AuditLogChainHead</c> produces a chain that
/// <c>verify-chain</c> accepts, but cannot change a head hash that already left the box.
/// </summary>
/// <remarks>
/// Serialised as one JSON object per line (camelCase) by both export sinks; the
/// <c>audit verify-heads</c> CLI reads those lines back and the server compares them with the
/// database. Hashes are lowercase hex. The format is versioned by <see cref="V"/>.
/// </remarks>
/// <param name="V">Record format version (1).</param>
/// <param name="Month">Chain partition, the UTC calendar month <c>YYYY-MM</c>.</param>
/// <param name="LastSeq">The head's last sequence number — links written for the month so far.</param>
/// <param name="LastHash">The head's last link hash (hex).</param>
/// <param name="PurgedThroughSeq">The partition-purge checkpoint anchor's sequence, or null.</param>
/// <param name="PurgedThroughHash">The anchor's hash (hex), or null.</param>
/// <param name="ChainedRows">Chained rows of the month still in <c>dbo.AuditLog</c> at export time.</param>
/// <param name="Tombstones">Per-channel purge tombstones of the month at export time.</param>
/// <param name="HeadUpdatedAtUtc">When the head row was last advanced.</param>
/// <param name="ExportedAtUtc">When this record was exported.</param>
/// <param name="Node">The central node that exported it.</param>
public sealed record AuditChainHeadRecord(
int V,
string Month,
long LastSeq,
string LastHash,
long? PurgedThroughSeq,
string? PurgedThroughHash,
long ChainedRows,
long Tombstones,
DateTime HeadUpdatedAtUtc,
DateTime ExportedAtUtc,
string? Node)
{
/// <summary>The current record format version.</summary>
public const int CurrentVersion = 1;
}
/// <summary>Outcome of comparing one exported head with the database chain.</summary>
[JsonConverter(typeof(JsonStringEnumConverter<AuditChainHeadStatus>))]
public enum AuditChainHeadStatus
{
/// <summary>The database head is exactly the exported head (same sequence, same hash, same row + tombstone count).</summary>
Matches,
/// <summary>The chain has grown since the export and the link at the exported sequence still carries the exported hash.</summary>
Extended,
/// <summary>
/// The link at the exported sequence was removed by a monthly partition purge (it is at or
/// before the checkpoint anchor and not the anchor itself), so the database cannot confirm
/// it; verify the month's Parquet archive instead (<c>audit archive verify</c>).
/// </summary>
Purged,
/// <summary>The link at the exported sequence carries a different hash: the chain was rewritten.</summary>
Mismatch,
/// <summary>The database head's sequence is BELOW the exported one: links were removed and the head rolled back.</summary>
Regressed,
/// <summary>The month's head row is gone although links had been exported for it.</summary>
MissingHead,
/// <summary>No row, tombstone or anchor carries the exported sequence any more: a chained row was deleted outside the purge paths.</summary>
MissingLink,
/// <summary>
/// Same head as exported, no partition purge since, but chained rows + tombstones differ
/// from the exported counts: a chained row was removed or its link cleared.
/// </summary>
CountMismatch,
}
/// <summary>One exported head checked against the database.</summary>
/// <param name="Month">Chain partition (<c>YYYY-MM</c>).</param>
/// <param name="ExportedSeq">The exported head's sequence.</param>
/// <param name="ExportedAtUtc">When it was exported.</param>
/// <param name="Node">Which node exported it.</param>
/// <param name="Status">The verdict.</param>
/// <param name="DatabaseSeq">The database head's current sequence, or null when it has none.</param>
/// <param name="Detail">Human-readable explanation.</param>
public sealed record AuditChainHeadCheck(
string Month,
long ExportedSeq,
DateTime ExportedAtUtc,
string? Node,
AuditChainHeadStatus Status,
long? DatabaseSeq,
string Detail)
{
/// <summary>True for the verdicts that are tamper evidence.</summary>
[JsonIgnore]
public bool IsFailure => Status is AuditChainHeadStatus.Mismatch or AuditChainHeadStatus.Regressed
or AuditChainHeadStatus.MissingHead or AuditChainHeadStatus.MissingLink or AuditChainHeadStatus.CountMismatch;
}
/// <summary>Result of <c>audit verify-heads</c>.</summary>
/// <param name="Verified">True when no check is a failure.</param>
/// <param name="RecordsChecked">Distinct exported heads checked.</param>
/// <param name="Checks">Every check, failures first, then by month and sequence.</param>
public sealed record AuditChainHeadVerificationResult(bool Verified, int RecordsChecked, IReadOnlyList<AuditChainHeadCheck> Checks);
/// <summary>The database's current chain head for a month, as read for export.</summary>
/// <param name="Month">First instant of the month (UTC).</param>
/// <param name="LastSeq">Head sequence.</param>
/// <param name="LastHash">Head hash.</param>
/// <param name="PurgedThroughSeq">Checkpoint anchor sequence, or null.</param>
/// <param name="PurgedThroughHash">Checkpoint anchor hash, or null.</param>
/// <param name="ChainedRows">Chained rows of the month still present.</param>
/// <param name="Tombstones">Tombstones of the month.</param>
/// <param name="UpdatedAtUtc">When the head row was last advanced.</param>
public sealed record AuditChainHeadSnapshot(
DateTime Month,
long LastSeq,
byte[] LastHash,
long? PurgedThroughSeq,
byte[]? PurgedThroughHash,
long ChainedRows,
long Tombstones,
DateTime UpdatedAtUtc);
@@ -1,126 +0,0 @@
using System.Text.Json.Serialization;
namespace ZB.MOM.WW.ScadaBridge.Commons.Types.Audit;
/// <summary>
/// Overall outcome of verifying one month's AuditLog hash chain (audit roadmap T1,
/// ScadaBridge#41).
/// </summary>
[JsonConverter(typeof(JsonStringEnumConverter<AuditChainStatus>))]
public enum AuditChainStatus
{
/// <summary>Every chained row, tombstone and the chain head agree.</summary>
Intact,
/// <summary>The chain is broken; <see cref="AuditChainVerificationResult.FirstBreak"/> names where.</summary>
Broken,
/// <summary>The month has no chained rows and no chain head — nothing to verify yet.</summary>
NotChained,
}
/// <summary>How the first chain break manifested.</summary>
[JsonConverter(typeof(JsonStringEnumConverter<AuditChainBreakKind>))]
public enum AuditChainBreakKind
{
/// <summary>
/// A row's recomputed hash differs from its stored <c>RowHash</c>: the row's content
/// was edited, or rows were reordered (their <c>ChainSeq</c> values swapped).
/// </summary>
HashMismatch,
/// <summary>
/// A sequence number is absent and no purge tombstone covers it: a chained row was
/// deleted outside the sanctioned purge path, or its <c>ChainSeq</c> was cleared.
/// </summary>
MissingRow,
/// <summary>Two rows carry the same <c>ChainSeq</c>.</summary>
DuplicateSequence,
/// <summary>A chained row carries a <c>ChainSeq</c> but no <c>RowHash</c> (or vice versa).</summary>
IncompleteLink,
/// <summary>
/// The chain ends before (or after) the sequence/hash the chain head records: rows were
/// deleted from the tail, or the head itself was altered.
/// </summary>
HeadMismatch,
/// <summary>
/// A chained row is dated before the chain cutoff (ScadaBridge#139). Rows before the cutoff
/// are never chained, so the row's <c>OccurredAtUtc</c> was moved back (to hide it among the
/// unprotected pre-cutoff rows) or the cutoff itself was moved forward.
/// </summary>
ChainedBeforeCutoff,
/// <summary>
/// The chain-start (cutoff) anchor is inconsistent (ScadaBridge#139): its stored hash does not
/// match its cutoff, it sits in the wrong month, or more than one anchor exists. The cutoff
/// that decides which rows are in scope cannot be trusted.
/// </summary>
CutoffAnchorInvalid,
}
/// <summary>
/// The first point at which a month's chain fails verification.
/// </summary>
/// <param name="Partition">The chain partition — the UTC calendar month, formatted <c>YYYY-MM</c>.</param>
/// <param name="ChainSeq">The sequence number at which the break was detected.</param>
/// <param name="EventId">
/// The <c>EventId</c> of the row at (or, for <see cref="AuditChainBreakKind.MissingRow"/>,
/// immediately after) the break; null when no row is present there (tail truncation).
/// </param>
/// <param name="Kind">How the break manifested.</param>
/// <param name="Detail">Human-readable explanation.</param>
public sealed record AuditChainBreak(
string Partition,
long ChainSeq,
Guid? EventId,
AuditChainBreakKind Kind,
string Detail);
/// <summary>
/// Result of <c>audit verify-chain --month YYYY-MM</c>: whether the month's AuditLog hash
/// chain is intact, and if not, the first break.
/// </summary>
/// <param name="Partition">The chain partition verified (<c>YYYY-MM</c>).</param>
/// <param name="Status">Overall outcome.</param>
/// <param name="RowsVerified">Chained rows whose hash was recomputed and matched before the first break (or in total when intact).</param>
/// <param name="TombstonesApplied">Sequence numbers bridged by a sanctioned purge tombstone (per-channel retention).</param>
/// <param name="PurgedThroughSeq">
/// The checkpoint anchor left by a monthly partition purge: the last sequence number the
/// purge removed. Verification starts immediately after it. Null when the month was never purged.
/// </param>
/// <param name="HeadSeq">The last sequence number the chain head records, or null when the month has no head.</param>
/// <param name="UnchainedRows">
/// Rows in the month, at or after the chain cutoff, that the chaining job has not reached yet
/// (written since the last tick, or written while chaining was disabled). They are not covered
/// by the chain yet.
/// </param>
/// <param name="FirstBreak">The first break, or null when <see cref="Status"/> is not <see cref="AuditChainStatus.Broken"/>.</param>
/// <param name="PreCutoffRows">
/// Rows in the month dated before the chain cutoff (ScadaBridge#139). They are out of scope by
/// design — the chain covers only rows written from its first enablement on — so they are
/// never chained and never a break. Zero when the chain has no cutoff.
/// </param>
/// <param name="ChainFromUtc">
/// The chain cutoff: the instant the chain was first enabled without backlog chaining. Null
/// when the chain covers the whole table (legacy / <c>ChainBacklog</c> enablement).
/// </param>
public sealed record AuditChainVerificationResult(
string Partition,
AuditChainStatus Status,
long RowsVerified,
long TombstonesApplied,
long? PurgedThroughSeq,
long? HeadSeq,
long UnchainedRows,
AuditChainBreak? FirstBreak,
long PreCutoffRows = 0,
DateTime? ChainFromUtc = null);
/// <summary>Outcome of one chaining pass over unchained AuditLog rows.</summary>
/// <param name="RowsChained">Rows linked into a chain by this pass.</param>
/// <param name="LockBusy">True when another chainer (the peer central node) held the chain lock, so this pass did nothing.</param>
public sealed record AuditChainBatchResult(int RowsChained, bool LockBusy);
@@ -169,17 +169,6 @@ public class AuditLogEntityTypeConfiguration : IEntityTypeConfiguration<AuditLog
stored: false)
.ValueGeneratedOnAddOrUpdate();
// ── Hash-chain link (ScadaBridge#41) ─────────────────────────────────
//
// Written only by the chaining job (column-level UPDATE grant to
// scadabridge_audit_chainer; the writer role keeps its table-wide DENY UPDATE).
// binary(32) on SQL Server; BLOB on the SQLite test provider.
builder.Property(e => e.ChainSeq);
builder.Property(e => e.RowHash)
.HasMaxLength(32)
.IsFixedLength();
// ── Keys + indexes ───────────────────────────────────────────────────
// Clustered PK (OccurredAtUtc, EventId) — FlipAuditLogClusteredKey (ScadaBridge#111,
@@ -262,46 +251,5 @@ public class AuditLogEntityTypeConfiguration : IEntityTypeConfiguration<AuditLog
builder.HasIndex(e => new { e.Actor, e.OccurredAtUtc })
.IsDescending(false, true)
.HasDatabaseName("IX_AuditLog_Actor_Occurred");
// Hash-chain indexes (AddAuditLogHashChain migration, raw SQL ON ps_AuditLog_Month
// so they stay partition-aligned). Unchained: the chaining job's work queue —
// filtered to ChainSeq IS NULL, so it shrinks to the rows written since the last
// tick. ChainSeq: verification streams one month's chain in sequence order.
builder.HasIndex(e => new { e.OccurredAtUtc, e.EventId })
.HasFilter("[ChainSeq] IS NULL")
.HasDatabaseName("IX_AuditLog_Unchained");
builder.HasIndex(e => e.ChainSeq)
.HasFilter("[ChainSeq] IS NOT NULL")
.HasDatabaseName("IX_AuditLog_ChainSeq");
}
}
/// <summary>Maps <see cref="AuditLogChainHead"/> to <c>dbo.AuditLogChainHead</c> (ScadaBridge#41).</summary>
public class AuditLogChainHeadEntityTypeConfiguration : IEntityTypeConfiguration<AuditLogChainHead>
{
/// <summary>Applies the EF Core type configuration for <see cref="AuditLogChainHead"/>.</summary>
/// <param name="builder">The entity type builder to configure.</param>
public void Configure(EntityTypeBuilder<AuditLogChainHead> builder)
{
builder.ToTable("AuditLogChainHead");
builder.HasKey(e => e.ChainMonth);
builder.Property(e => e.ChainMonth).HasColumnType("date");
builder.Property(e => e.LastHash).HasMaxLength(32).IsFixedLength().IsRequired();
builder.Property(e => e.PurgedThroughHash).HasMaxLength(32).IsFixedLength();
}
}
/// <summary>Maps <see cref="AuditLogChainTombstone"/> to <c>dbo.AuditLogChainTombstone</c> (ScadaBridge#41).</summary>
public class AuditLogChainTombstoneEntityTypeConfiguration : IEntityTypeConfiguration<AuditLogChainTombstone>
{
/// <summary>Applies the EF Core type configuration for <see cref="AuditLogChainTombstone"/>.</summary>
/// <param name="builder">The entity type builder to configure.</param>
public void Configure(EntityTypeBuilder<AuditLogChainTombstone> builder)
{
builder.ToTable("AuditLogChainTombstone");
builder.HasKey(e => new { e.ChainMonth, e.ChainSeq });
builder.Property(e => e.ChainMonth).HasColumnType("date");
builder.Property(e => e.RowHash).HasMaxLength(32).IsFixedLength().IsRequired();
}
}
@@ -1,61 +0,0 @@
namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Entities;
/// <summary>
/// One row per chain partition (UTC calendar month) of the AuditLog hash chain
/// (ScadaBridge#41): where the chain currently ends, plus the checkpoint anchor a
/// monthly partition purge leaves behind.
/// </summary>
/// <remarks>
/// The head is what lets verification detect <b>tail truncation</b> — deleting the last
/// rows of a month leaves a chain that is internally consistent but shorter than the
/// head says. It is written by the chaining job (<see cref="LastSeq"/>/<see cref="LastHash"/>)
/// and by the partition-switch purge (<see cref="PurgedThroughSeq"/>/<see cref="PurgedThroughHash"/>).
/// </remarks>
public sealed class AuditLogChainHead
{
/// <summary>First day of the chain's month (UTC). Primary key.</summary>
public DateTime ChainMonth { get; set; }
/// <summary>Sequence number of the last row linked into the chain.</summary>
public long LastSeq { get; set; }
/// <summary>RowHash of the last row linked into the chain (32 bytes).</summary>
public byte[] LastHash { get; set; } = [];
/// <summary>
/// Checkpoint anchor: the last sequence number removed by a monthly partition-switch
/// purge. Verification resumes at <c>PurgedThroughSeq + 1</c> with
/// <see cref="PurgedThroughHash"/> as the previous link. Null when never purged.
/// </summary>
public long? PurgedThroughSeq { get; set; }
/// <summary>RowHash at <see cref="PurgedThroughSeq"/>; null when never purged.</summary>
public byte[]? PurgedThroughHash { get; set; }
/// <summary>UTC time the head last moved.</summary>
public DateTime UpdatedAtUtc { get; set; }
}
/// <summary>
/// The link a sanctioned per-channel retention DELETE removed from the middle of a chain
/// (ScadaBridge#41). Written in the SAME statement batch as the DELETE (via
/// <c>OUTPUT … INTO</c>) so the purge can never remove a chained row without leaving its
/// hash behind; verification bridges the gap with it.
/// </summary>
public sealed class AuditLogChainTombstone
{
/// <summary>First day of the chain's month (UTC). Part of the primary key.</summary>
public DateTime ChainMonth { get; set; }
/// <summary>Sequence number of the purged row. Part of the primary key.</summary>
public long ChainSeq { get; set; }
/// <summary>RowHash of the purged row (32 bytes).</summary>
public byte[] RowHash { get; set; } = [];
/// <summary>The purged row's EventId, kept for the operator's benefit.</summary>
public Guid EventId { get; set; }
/// <summary>UTC time of the purge.</summary>
public DateTime PurgedAtUtc { get; set; }
}
@@ -138,19 +138,4 @@ public sealed record AuditLogRow
: null;
}
private readonly DateTime? _ingestedAtUtc;
// ── Hash-chain link (audit roadmap T1, ScadaBridge#41) ────────────────────
/// <summary>
/// Position of this row in its month's hash chain (1-based, gap-free per UTC calendar
/// month of <see cref="OccurredAtUtc"/>); null until the chaining job reaches the row.
/// Written ONLY by the chaining job — never by the ingest/writer path.
/// </summary>
public long? ChainSeq { get; init; }
/// <summary>
/// <c>SHA-256(prev.RowHash || canonical(row))</c> — 32 bytes; null until chained. See
/// <c>AuditChainHasher</c> for the canonical serialisation.
/// </summary>
public byte[]? RowHash { get; init; }
}
@@ -1,3 +1,5 @@
using System.Data;
using System.Data.Common;
using System.Globalization;
using System.Runtime.CompilerServices;
using System.Security.Cryptography;
@@ -7,50 +9,58 @@ using Parquet.Schema;
namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Maintenance;
/// <summary>
/// One per-channel purge tombstone carried in an archive (ScadaBridge#41/#42): the link a
/// sanctioned per-channel retention DELETE removed from the middle of a month's chain.
/// One archived <c>dbo.AuditLog</c> row: the ten stored (writable) columns. The persisted
/// computed columns (Kind, Status, SourceSiteId, …) are derived from <c>DetailsJson</c> and
/// are not carried.
/// </summary>
/// <param name="ChainMonth">The chain partition's first instant (UTC).</param>
/// <param name="ChainSeq">The purged row's sequence number.</param>
/// <param name="RowHash">The purged row's 32-byte hash.</param>
/// <param name="EventId">The purged row's EventId.</param>
/// <param name="PurgedAtUtc">When the per-channel purge removed it.</param>
public sealed record AuditArchiveTombstone(
DateTime ChainMonth,
long ChainSeq,
byte[] RowHash,
/// <param name="EventId">The row's EventId.</param>
/// <param name="OccurredAtUtc">When the event occurred (UTC, <c>datetime2(7)</c> precision).</param>
/// <param name="Actor">Actor, or null.</param>
/// <param name="Action">Action.</param>
/// <param name="Outcome">Outcome.</param>
/// <param name="Category">Channel (the canonical <c>Category</c> column).</param>
/// <param name="Target">Target, or null.</param>
/// <param name="SourceNode">Node of origin, or null.</param>
/// <param name="CorrelationId">Correlation id, or null.</param>
/// <param name="DetailsJson">Details JSON, or null.</param>
public sealed record AuditArchiveRow(
Guid EventId,
DateTime PurgedAtUtc);
DateTime OccurredAtUtc,
string? Actor,
string Action,
string Outcome,
string Category,
string? Target,
string? SourceNode,
Guid? CorrelationId,
string? DetailsJson);
/// <summary>
/// The Parquet layout of an AuditLog archive (audit roadmap T2, ScadaBridge#42), format v1.
/// The Parquet layout of an AuditLog archive (audit roadmap T2, ScadaBridge#42), format v2.
/// </summary>
/// <remarks>
/// <para>
/// <b>Data file</b> — one row per archived <c>dbo.AuditLog</c> row, the ten stored (writable)
/// columns plus the two hash-chain columns, so a row can be re-hashed with
/// <see cref="AuditChainHasher"/> exactly as the database chained it:
/// <c>EventId</c> (string, <c>D</c> format), <c>OccurredAtUtc</c> (timestamp, UTC, microseconds —
/// for analytics), <c>OccurredAtUtcTicks</c> (int64 — the exact <c>datetime2(7)</c> value the hash
/// covers), <c>Actor</c>, <c>Action</c>, <c>Outcome</c>, <c>Category</c>, <c>Target</c>,
/// <c>SourceNode</c>, <c>CorrelationId</c>, <c>DetailsJson</c> (strings), <c>ChainSeq</c> (int64,
/// null when unchained) and <c>RowHash</c> (binary, null when unchained). The persisted computed
/// columns (Kind, Status, SourceSiteId, …) are not stored: they are derived from
/// <c>DetailsJson</c>, which the hash covers.
/// columns: <c>EventId</c> (string, <c>D</c> format), <c>OccurredAtUtc</c> (timestamp, UTC,
/// microseconds — for analytics), <c>OccurredAtUtcTicks</c> (int64 — the exact
/// <c>datetime2(7)</c> value), <c>Actor</c>, <c>Action</c>, <c>Outcome</c>, <c>Category</c>,
/// <c>Target</c>, <c>SourceNode</c>, <c>CorrelationId</c>, <c>DetailsJson</c> (strings).
/// </para>
/// <para>
/// <b>Tombstone file</b> — <c>ChainMonth</c> (<c>YYYY-MM</c>), <c>ChainSeq</c>, <c>RowHash</c>,
/// <c>EventId</c>, <c>PurgedAtUtc</c>.
/// <b>Format v1</b> (written before 2026-11-15, when the audit hash chain was removed) also
/// carried <c>ChainSeq</c>/<c>RowHash</c> columns and a <c>.tombstones.parquet</c> sibling. The
/// reader reads a v1 data file by column name and ignores those columns; cleanup still deletes a
/// v1 archive's tombstone file with it.
/// </para>
/// <para>
/// Both are plain Parquet: DuckDB (<c>SELECT * FROM 'auditlog-….parquet'</c>), pandas/pyarrow
/// and Spark read them without any ScadaBridge code.
/// Plain Parquet: DuckDB (<c>SELECT * FROM 'auditlog-….parquet'</c>), pandas/pyarrow and Spark
/// read it without any ScadaBridge code.
/// </para>
/// </remarks>
public static class AuditArchiveParquet
{
/// <summary>The archive layout version written into the file metadata and the manifest.</summary>
public const int FormatVersion = 1;
public const int FormatVersion = 2;
/// <summary>Key-value metadata key carrying <see cref="FormatVersion"/>.</summary>
public const string FormatVersionKey = "scadabridge.auditlog.archive.formatVersion";
@@ -70,24 +80,60 @@ public static class AuditArchiveParquet
private static readonly DataField<string> SourceNodeField = new("SourceNode", true);
private static readonly DataField<string> CorrelationIdField = new("CorrelationId", true);
private static readonly DataField<string> DetailsJsonField = new("DetailsJson", true);
private static readonly DataField<long> ChainSeqField = new("ChainSeq", true);
private static readonly DataField<byte[]> RowHashField = new("RowHash", true);
private static readonly DataField<string> TombstoneMonthField = new("ChainMonth", false);
private static readonly DataField<long> TombstoneSeqField = new("ChainSeq", false);
private static readonly DataField<byte[]> TombstoneHashField = new("RowHash", false);
private static readonly DataField<string> TombstoneEventIdField = new("EventId", false);
private static readonly DateTimeDataField TombstonePurgedAtField =
new("PurgedAtUtc", DateTimeFormat.Timestamp, isAdjustedToUTC: true, unit: DateTimeTimeUnit.Micros, isNullable: false);
/// <summary>Suffix of the per-channel-purge tombstone file a format-v1 archive carried.</summary>
public const string LegacyTombstoneSuffix = ".tombstones.parquet";
/// <summary>The ten stored <c>dbo.AuditLog</c> columns, in table order (see <see cref="ReadRow"/>).</summary>
internal const string RowColumns =
"EventId, OccurredAtUtc, Actor, Action, Outcome, Category, Target, SourceNode, CorrelationId, DetailsJson";
/// <summary>The data-file schema.</summary>
public static ParquetSchema DataSchema { get; } = new(
EventIdField, OccurredAtField, OccurredAtTicksField, ActorField, ActionField, OutcomeField, CategoryField,
TargetField, SourceNodeField, CorrelationIdField, DetailsJsonField, ChainSeqField, RowHashField);
TargetField, SourceNodeField, CorrelationIdField, DetailsJsonField);
/// <summary>The tombstone-file schema.</summary>
public static ParquetSchema TombstoneSchema { get; } = new(
TombstoneMonthField, TombstoneSeqField, TombstoneHashField, TombstoneEventIdField, TombstonePurgedAtField);
/// <summary>The first instant (UTC) of the calendar month holding <paramref name="occurredAtUtc"/>.</summary>
/// <param name="occurredAtUtc">A UTC instant.</param>
/// <returns>Day 1, midnight UTC.</returns>
public static DateTime MonthOf(DateTime occurredAtUtc)
=> new(occurredAtUtc.Year, occurredAtUtc.Month, 1, 0, 0, 0, DateTimeKind.Utc);
/// <summary>Formats a month as <c>YYYY-MM</c>.</summary>
/// <param name="monthStartUtc">The month's first instant.</param>
/// <returns>The partition label.</returns>
public static string PartitionLabel(DateTime monthStartUtc)
=> monthStartUtc.ToString("yyyy-MM", CultureInfo.InvariantCulture);
/// <summary>Reads one row selected as <see cref="RowColumns"/>, strictly in ordinal order (SequentialAccess-safe).</summary>
/// <param name="reader">The reader, positioned on a row.</param>
/// <returns>The row.</returns>
internal static AuditArchiveRow ReadRow(DbDataReader reader) => new(
reader.GetGuid(0),
DateTime.SpecifyKind(reader.GetDateTime(1), DateTimeKind.Utc),
reader.IsDBNull(2) ? null : reader.GetString(2),
reader.GetString(3),
reader.GetString(4),
reader.GetString(5),
reader.IsDBNull(6) ? null : reader.GetString(6),
reader.IsDBNull(7) ? null : reader.GetString(7),
reader.IsDBNull(8) ? null : reader.GetGuid(8),
reader.IsDBNull(9) ? null : reader.GetString(9));
/// <summary>Opens <paramref name="conn"/> when it is not already open.</summary>
/// <param name="conn">The connection.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>True when this call opened it (the caller closes it).</returns>
internal static async Task<bool> OpenAsync(DbConnection conn, CancellationToken ct)
{
if (conn.State == ConnectionState.Open)
{
return false;
}
await conn.OpenAsync(ct).ConfigureAwait(false);
return true;
}
/// <summary>Opens a buffered writer of archive data rows over <paramref name="output"/>.</summary>
/// <param name="output">Destination stream (need not be seekable).</param>
@@ -102,24 +148,11 @@ public static class AuditArchiveParquet
return new DataWriter(writer, rowGroupSize);
}
/// <summary>Opens a buffered writer of tombstones over <paramref name="output"/>.</summary>
/// <param name="output">Destination stream (need not be seekable).</param>
/// <param name="partition">The archived partition (<c>YYYY-MM</c>), stamped into the file metadata.</param>
/// <param name="rowGroupSize">Rows per Parquet row group.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>The writer; dispose it to flush the last row group and the footer.</returns>
public static async Task<TombstoneWriter> CreateTombstoneWriterAsync(Stream output, string partition, int rowGroupSize, CancellationToken ct = default)
{
var writer = await ParquetWriter.CreateAsync(TombstoneSchema, output, cancellationToken: ct).ConfigureAwait(false);
writer.CustomMetadata = Metadata(partition);
return new TombstoneWriter(writer, rowGroupSize);
}
/// <summary>Streams the rows of an archive data file in file order.</summary>
/// <param name="input">A readable, seekable stream over the data file.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>The rows, as the chain hasher sees them.</returns>
public static async IAsyncEnumerable<AuditChainRow> ReadRowsAsync(Stream input, [EnumeratorCancellation] CancellationToken ct = default)
/// <returns>The rows.</returns>
public static async IAsyncEnumerable<AuditArchiveRow> ReadRowsAsync(Stream input, [EnumeratorCancellation] CancellationToken ct = default)
{
await using var reader = await ParquetReader.CreateAsync(input, leaveStreamOpen: true, cancellationToken: ct).ConfigureAwait(false);
for (var g = 0; g < reader.RowGroupCount; g++)
@@ -137,14 +170,10 @@ public static class AuditArchiveParquet
var nodes = await Strings(group, SourceNodeField, n, ct).ConfigureAwait(false);
var correlations = await Strings(group, CorrelationIdField, n, ct).ConfigureAwait(false);
var details = await Strings(group, DetailsJsonField, n, ct).ConfigureAwait(false);
var seqs = new long?[n];
await group.ReadAsync<long>(ChainSeqField, seqs, cancellationToken: ct).ConfigureAwait(false);
var hashes = new byte[]?[n];
await group.ReadAsync(RowHashField, hashes!, cancellationToken: ct).ConfigureAwait(false);
for (var i = 0; i < n; i++)
{
yield return new AuditChainRow(
yield return new AuditArchiveRow(
Guid.Parse(eventIds[i]!),
new DateTime(ticks[i], DateTimeKind.Utc),
actors[i],
@@ -154,46 +183,12 @@ public static class AuditArchiveParquet
targets[i],
nodes[i],
correlations[i] is { } c ? Guid.Parse(c) : null,
details[i],
seqs[i],
hashes[i]);
details[i]);
}
}
}
/// <summary>Streams the tombstones of an archive tombstone file.</summary>
/// <param name="input">A readable, seekable stream over the tombstone file.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>The tombstones.</returns>
public static async IAsyncEnumerable<AuditArchiveTombstone> ReadTombstonesAsync(Stream input, [EnumeratorCancellation] CancellationToken ct = default)
{
await using var reader = await ParquetReader.CreateAsync(input, leaveStreamOpen: true, cancellationToken: ct).ConfigureAwait(false);
for (var g = 0; g < reader.RowGroupCount; g++)
{
using var group = reader.OpenRowGroupReader(g);
var n = checked((int)group.RowCount);
var months = await Strings(group, TombstoneMonthField, n, ct).ConfigureAwait(false);
var seqs = new long[n];
await group.ReadAsync<long>(TombstoneSeqField, seqs, cancellationToken: ct).ConfigureAwait(false);
var hashes = new byte[]?[n];
await group.ReadAsync(TombstoneHashField, hashes!, cancellationToken: ct).ConfigureAwait(false);
var eventIds = await Strings(group, TombstoneEventIdField, n, ct).ConfigureAwait(false);
var purgedAt = new DateTime[n];
await group.ReadAsync<DateTime>(TombstonePurgedAtField, purgedAt, cancellationToken: ct).ConfigureAwait(false);
for (var i = 0; i < n; i++)
{
yield return new AuditArchiveTombstone(
ParseMonth(months[i]!),
seqs[i],
hashes[i]!,
Guid.Parse(eventIds[i]!),
DateTime.SpecifyKind(purgedAt[i], DateTimeKind.Utc));
}
}
}
/// <summary>Parses a <c>YYYY-MM</c> chain partition label to its first instant (UTC).</summary>
/// <summary>Parses a <c>YYYY-MM</c> partition label to its first instant (UTC).</summary>
/// <param name="partition">The label.</param>
/// <returns>Day 1, midnight UTC.</returns>
public static DateTime ParseMonth(string partition)
@@ -220,14 +215,14 @@ public static class AuditArchiveParquet
{
private readonly ParquetWriter _writer;
private readonly int _rowGroupSize;
private readonly List<AuditChainRow> _buffer;
private readonly List<AuditArchiveRow> _buffer;
private bool _disposed;
internal DataWriter(ParquetWriter writer, int rowGroupSize)
{
_writer = writer;
_rowGroupSize = Math.Max(1, rowGroupSize);
_buffer = new List<AuditChainRow>(Math.Min(_rowGroupSize, 65_536));
_buffer = new List<AuditArchiveRow>(Math.Min(_rowGroupSize, 65_536));
}
/// <summary>Rows added so far.</summary>
@@ -237,7 +232,7 @@ public static class AuditArchiveParquet
/// <param name="row">The row.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>A task that completes when the row is buffered (or its row group written).</returns>
public async Task AddAsync(AuditChainRow row, CancellationToken ct = default)
public async Task AddAsync(AuditArchiveRow row, CancellationToken ct = default)
{
ArgumentNullException.ThrowIfNull(row);
_buffer.Add(row);
@@ -283,75 +278,6 @@ public static class AuditArchiveParquet
await group.WriteAsync(SourceNodeField, rows.Select(r => r.SourceNode).ToArray()!).ConfigureAwait(false);
await group.WriteAsync(CorrelationIdField, rows.Select(r => r.CorrelationId?.ToString("D")).ToArray()!).ConfigureAwait(false);
await group.WriteAsync(DetailsJsonField, rows.Select(r => r.DetailsJson).ToArray()!).ConfigureAwait(false);
await group.WriteAsync<long>(ChainSeqField, rows.Select(r => r.ChainSeq).ToArray(), cancellationToken: ct).ConfigureAwait(false);
await group.WriteAsync(RowHashField, rows.Select(r => r.RowHash).ToArray()!).ConfigureAwait(false);
}
_buffer.Clear();
}
}
/// <summary>Buffers tombstones and writes them a row group at a time.</summary>
public sealed class TombstoneWriter : IAsyncDisposable
{
private readonly ParquetWriter _writer;
private readonly int _rowGroupSize;
private readonly List<AuditArchiveTombstone> _buffer = [];
private bool _disposed;
internal TombstoneWriter(ParquetWriter writer, int rowGroupSize)
{
_writer = writer;
_rowGroupSize = Math.Max(1, rowGroupSize);
}
/// <summary>Tombstones added so far.</summary>
public long Count { get; private set; }
/// <summary>Adds one tombstone, flushing a row group when the buffer is full.</summary>
/// <param name="tombstone">The tombstone.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>A task that completes when the tombstone is buffered (or its row group written).</returns>
public async Task AddAsync(AuditArchiveTombstone tombstone, CancellationToken ct = default)
{
ArgumentNullException.ThrowIfNull(tombstone);
_buffer.Add(tombstone);
Count++;
if (_buffer.Count >= _rowGroupSize)
{
await FlushAsync(ct).ConfigureAwait(false);
}
}
/// <summary>Writes any buffered tombstones as a row group, then the file footer.</summary>
/// <returns>A task that completes when the file is finished.</returns>
public async ValueTask DisposeAsync()
{
if (_disposed)
{
return;
}
_disposed = true;
await FlushAsync(CancellationToken.None).ConfigureAwait(false);
await _writer.DisposeAsync().ConfigureAwait(false);
}
private async Task FlushAsync(CancellationToken ct)
{
if (_buffer.Count == 0)
{
return;
}
var rows = _buffer;
using (var group = _writer.CreateRowGroup())
{
await group.WriteAsync(TombstoneMonthField, rows.Select(t => AuditChainHasher.PartitionLabel(t.ChainMonth)).ToArray()).ConfigureAwait(false);
await group.WriteAsync<long>(TombstoneSeqField, rows.Select(t => t.ChainSeq).ToArray(), cancellationToken: ct).ConfigureAwait(false);
await group.WriteAsync(TombstoneHashField, rows.Select(t => t.RowHash).ToArray()).ConfigureAwait(false);
await group.WriteAsync(TombstoneEventIdField, rows.Select(t => t.EventId.ToString("D")).ToArray()).ConfigureAwait(false);
await group.WriteAsync<DateTime>(TombstonePurgedAtField, rows.Select(t => t.PurgedAtUtc).ToArray(), cancellationToken: ct).ConfigureAwait(false);
}
_buffer.Clear();
@@ -1,184 +0,0 @@
using System.Buffers.Binary;
using System.Globalization;
using System.Security.Cryptography;
using System.Text;
namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Maintenance;
/// <summary>
/// The stored column values of one <c>dbo.AuditLog</c> row as the hash chain sees them
/// (ScadaBridge#41). Strings are the raw column values exactly as SQL Server returns them
/// (e.g. <see cref="Outcome"/> / <see cref="Category"/> are the stored enum NAMES), so
/// the hash never depends on how a later build happens to map them to CLR enums.
/// </summary>
/// <param name="EventId">The row's EventId.</param>
/// <param name="OccurredAtUtc">The row's OccurredAtUtc (datetime2(7) — full tick precision).</param>
/// <param name="Actor">Actor column.</param>
/// <param name="Action">Action column.</param>
/// <param name="Outcome">Outcome column.</param>
/// <param name="Category">Category (channel) column.</param>
/// <param name="Target">Target column.</param>
/// <param name="SourceNode">SourceNode column.</param>
/// <param name="CorrelationId">CorrelationId column.</param>
/// <param name="DetailsJson">DetailsJson column.</param>
/// <param name="ChainSeq">Stored ChainSeq, or null when unchained.</param>
/// <param name="RowHash">Stored RowHash, or null when unchained.</param>
public sealed record AuditChainRow(
Guid EventId,
DateTime OccurredAtUtc,
string? Actor,
string Action,
string Outcome,
string Category,
string? Target,
string? SourceNode,
Guid? CorrelationId,
string? DetailsJson,
long? ChainSeq = null,
byte[]? RowHash = null);
/// <summary>
/// Canonical serialisation + SHA-256 link computation for the AuditLog hash chain
/// (alog.md §11.4, ScadaBridge#41): <c>RowHash = SHA-256(prev.RowHash || canonical(row))</c>.
/// </summary>
/// <remarks>
/// <para>
/// <b>Canonical form (v1), all integers big-endian:</b> the ASCII domain tag
/// <c>"ScadaBridge.AuditLog.Chain.v1"</c>; the chain partition (<c>YYYY-MM</c>); the
/// 64-bit <c>ChainSeq</c>; then the ten writable columns in table order — EventId (as its
/// lowercase <c>D</c>-format string), OccurredAtUtc (64-bit ticks), Actor, Action, Outcome,
/// Category, Target, SourceNode, CorrelationId (<c>D</c>-format string), DetailsJson.
/// Every string-valued field is encoded as one presence byte (<c>0</c> = SQL NULL,
/// <c>1</c> = present) followed, when present, by a 32-bit byte length and its UTF-8 bytes —
/// so NULL and the empty string hash differently, and no field boundary is ambiguous.
/// </para>
/// <para>
/// The partition and sequence number are inside the hashed bytes so a row moved to a
/// different position, or into another month's chain, fails verification on its own,
/// independently of the previous-hash link. The computed columns (Kind, Status, …) are
/// derived from DetailsJson, so hashing DetailsJson covers them.
/// </para>
/// <para>
/// Changing anything here invalidates every chain already written. A new form needs a new
/// domain tag and a migration story; never edit v1 in place.
/// </para>
/// </remarks>
public static class AuditChainHasher
{
/// <summary>Length of a link hash in bytes.</summary>
public const int HashLength = 32;
private static readonly byte[] DomainTag = Encoding.ASCII.GetBytes("ScadaBridge.AuditLog.Chain.v1");
/// <summary>The previous-hash value the first row of every chain links to (32 zero bytes).</summary>
public static ReadOnlySpan<byte> GenesisHash => new byte[HashLength];
/// <summary>Returns the first instant (UTC) of the month containing <paramref name="occurredAtUtc"/>.</summary>
/// <param name="occurredAtUtc">A UTC timestamp.</param>
/// <returns>Day 1, 00:00:00 UTC of that month.</returns>
public static DateTime MonthOf(DateTime occurredAtUtc)
=> new(occurredAtUtc.Year, occurredAtUtc.Month, 1, 0, 0, 0, DateTimeKind.Utc);
/// <summary>Formats a chain partition as <c>YYYY-MM</c>.</summary>
/// <param name="monthStartUtc">The month's first instant.</param>
/// <returns>The partition label.</returns>
public static string PartitionLabel(DateTime monthStartUtc)
=> monthStartUtc.ToString("yyyy-MM", CultureInfo.InvariantCulture);
/// <summary>Computes the link hash for <paramref name="row"/> at <paramref name="chainSeq"/>.</summary>
/// <param name="previousHash">The RowHash of the row at <c>chainSeq - 1</c>, or <see cref="GenesisHash"/>.</param>
/// <param name="monthStartUtc">The chain partition the row belongs to.</param>
/// <param name="chainSeq">The row's position in the chain.</param>
/// <param name="row">The row's stored column values.</param>
/// <returns>The 32-byte SHA-256 link hash.</returns>
public static byte[] ComputeRowHash(ReadOnlySpan<byte> previousHash, DateTime monthStartUtc, long chainSeq, AuditChainRow row)
{
ArgumentNullException.ThrowIfNull(row);
if (previousHash.Length != HashLength)
{
throw new ArgumentException($"Previous hash must be {HashLength} bytes.", nameof(previousHash));
}
using var hash = IncrementalHash.CreateHash(HashAlgorithmName.SHA256);
hash.AppendData(previousHash);
hash.AppendData(Canonicalize(monthStartUtc, chainSeq, row));
return hash.GetHashAndReset();
}
/// <summary>Builds the canonical v1 byte form of <paramref name="row"/> at its chain position.</summary>
/// <param name="monthStartUtc">The chain partition.</param>
/// <param name="chainSeq">The row's position in the chain.</param>
/// <param name="row">The row's stored column values.</param>
/// <returns>The canonical bytes (excluding the previous hash).</returns>
public static byte[] Canonicalize(DateTime monthStartUtc, long chainSeq, AuditChainRow row)
{
ArgumentNullException.ThrowIfNull(row);
using var buffer = new MemoryStream(256 + (row.DetailsJson?.Length ?? 0) * 3);
buffer.Write(DomainTag);
WriteString(buffer, PartitionLabel(monthStartUtc));
WriteInt64(buffer, chainSeq);
WriteString(buffer, row.EventId.ToString("D"));
WriteInt64(buffer, row.OccurredAtUtc.Ticks);
WriteString(buffer, row.Actor);
WriteString(buffer, row.Action);
WriteString(buffer, row.Outcome);
WriteString(buffer, row.Category);
WriteString(buffer, row.Target);
WriteString(buffer, row.SourceNode);
WriteString(buffer, row.CorrelationId?.ToString("D"));
WriteString(buffer, row.DetailsJson);
return buffer.ToArray();
}
/// <summary>
/// The <c>ChainSeq</c> of the chain-start (cutoff) anchor in <c>dbo.AuditLogChainTombstone</c>
/// (ScadaBridge#139). Real links start at 1, so slot 0 is "the link before the first row":
/// the anchor's <c>RowHash</c> is the genesis hash of the cutoff month's chain.
/// </summary>
public const long CutoffAnchorSeq = 0;
/// <summary>The fixed <c>EventId</c> the chain-start anchor row carries, so it is recognisable by eye.</summary>
public static readonly Guid CutoffAnchorEventId = new("5cadab1d-c4a1-4c0f-a000-000000000139");
private static readonly byte[] CutoffDomainTag = Encoding.ASCII.GetBytes("ScadaBridge.AuditLog.ChainCutoff.v1");
/// <summary>
/// Computes the chain-start anchor hash for a cutoff instant (ScadaBridge#139):
/// <c>SHA-256("ScadaBridge.AuditLog.ChainCutoff.v1" || YYYY-MM || ticks)</c>, big-endian like
/// the row form. It is both the anchor row's stored <c>RowHash</c> and the previous-hash the
/// first row of the cutoff month links to, so moving the cutoff breaks that month's chain.
/// </summary>
/// <param name="chainFromUtc">The cutoff: rows with an earlier <c>OccurredAtUtc</c> are never chained.</param>
/// <returns>The 32-byte anchor hash.</returns>
public static byte[] ComputeCutoffAnchorHash(DateTime chainFromUtc)
{
using var buffer = new MemoryStream(64);
buffer.Write(CutoffDomainTag);
WriteString(buffer, PartitionLabel(MonthOf(chainFromUtc)));
WriteInt64(buffer, chainFromUtc.Ticks);
return SHA256.HashData(buffer.ToArray());
}
private static void WriteInt64(Stream stream, long value)
{
Span<byte> bytes = stackalloc byte[8];
BinaryPrimitives.WriteInt64BigEndian(bytes, value);
stream.Write(bytes);
}
private static void WriteString(Stream stream, string? value)
{
if (value is null)
{
stream.WriteByte(0);
return;
}
stream.WriteByte(1);
var bytes = Encoding.UTF8.GetBytes(value);
Span<byte> length = stackalloc byte[4];
BinaryPrimitives.WriteInt32BigEndian(length, bytes.Length);
stream.Write(length);
stream.Write(bytes);
}
}
@@ -1,139 +0,0 @@
using System.Globalization;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Audit;
namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Maintenance;
/// <summary>
/// Pure comparison of one exported chain head (ScadaBridge#107 item 1) with the database's
/// current state for that month. Kept free of I/O so every verdict is unit-testable; the
/// database side supplies the head snapshot and, when needed, the link stored at the
/// exported sequence number.
/// </summary>
public static class AuditChainHeadComparer
{
/// <summary>What the database stores at one chain sequence number.</summary>
/// <param name="RowHash">The <c>RowHash</c> of the live row carrying the sequence, or null.</param>
/// <param name="TombstoneHash">The <c>RowHash</c> of a per-channel purge tombstone carrying it, or null.</param>
public readonly record struct StoredLink(byte[]? RowHash, byte[]? TombstoneHash);
/// <summary>Compares one exported head with the database.</summary>
/// <param name="record">The exported head.</param>
/// <param name="head">The database head for the record's month, or null when there is none.</param>
/// <param name="linkAt">Reads the stored link at a sequence number (only called when the chain has grown).</param>
/// <returns>The verdict.</returns>
public static async Task<AuditChainHeadCheck> CompareAsync(
AuditChainHeadRecord record, AuditChainHeadSnapshot? head, Func<long, Task<StoredLink>> linkAt)
{
ArgumentNullException.ThrowIfNull(record);
ArgumentNullException.ThrowIfNull(linkAt);
AuditChainHeadCheck Result(AuditChainHeadStatus status, string detail) =>
new(record.Month, record.LastSeq, record.ExportedAtUtc, record.Node, status, head?.LastSeq, detail);
if (!TryParseHex(record.LastHash, out var exported))
{
return Result(AuditChainHeadStatus.Mismatch, $"Exported hash '{record.LastHash}' is not 32 bytes of hex.");
}
if (head is null)
{
return Result(AuditChainHeadStatus.MissingHead,
$"{record.Month} has no chain head in the database, but {record.LastSeq} links were exported for it.");
}
if (head.LastSeq < record.LastSeq)
{
return Result(AuditChainHeadStatus.Regressed,
$"Database head is at sequence {head.LastSeq}, below the exported {record.LastSeq}: links were removed and the head rolled back.");
}
if (head.LastSeq == record.LastSeq)
{
if (!exported.AsSpan().SequenceEqual(head.LastHash))
{
return Result(AuditChainHeadStatus.Mismatch,
$"Database head hash at sequence {head.LastSeq} is {ToHex(head.LastHash)}, exported {record.LastHash}: the chain was rewritten.");
}
// No link was added since the export. Unless a partition purge set (or moved) the
// anchor since, rows + tombstones are invariant: the per-channel purge turns a row
// into a tombstone, nothing else may remove a chained row.
if (head.PurgedThroughSeq == record.PurgedThroughSeq
&& head.ChainedRows + head.Tombstones != record.ChainedRows + record.Tombstones)
{
return Result(AuditChainHeadStatus.CountMismatch,
$"Chained rows + tombstones are {head.ChainedRows} + {head.Tombstones} in the database, {record.ChainedRows} + {record.Tombstones} exported.");
}
return Result(AuditChainHeadStatus.Matches, "Database head equals the exported head.");
}
// The chain has grown: the link at the exported sequence must still carry the exported hash.
if (head.PurgedThroughSeq is { } anchor && record.LastSeq <= anchor)
{
if (record.LastSeq < anchor)
{
return Result(AuditChainHeadStatus.Purged,
$"Sequence {record.LastSeq} was removed by a partition purge (anchor at {anchor}); verify the month's archive.");
}
return head.PurgedThroughHash is { } anchorHash && exported.AsSpan().SequenceEqual(anchorHash)
? Result(AuditChainHeadStatus.Extended, $"Exported head is the partition-purge anchor at {anchor}.")
: Result(AuditChainHeadStatus.Mismatch, $"Partition-purge anchor at {anchor} does not carry the exported hash.");
}
var link = await linkAt(record.LastSeq).ConfigureAwait(false);
var stored = link.RowHash ?? link.TombstoneHash;
if (stored is null)
{
return Result(AuditChainHeadStatus.MissingLink,
$"No row, tombstone or anchor carries sequence {record.LastSeq}: a chained row was deleted outside the purge paths.");
}
return exported.AsSpan().SequenceEqual(stored)
? Result(AuditChainHeadStatus.Extended,
$"Chain grew to {head.LastSeq}; the {(link.RowHash is null ? "tombstone" : "row")} at {record.LastSeq} still carries the exported hash.")
: Result(AuditChainHeadStatus.Mismatch,
$"The {(link.RowHash is null ? "tombstone" : "row")} at sequence {record.LastSeq} carries {ToHex(stored)}, exported {record.LastHash}: the chain was rewritten.");
}
/// <summary>Lowercase hex of a hash.</summary>
/// <param name="bytes">The bytes.</param>
/// <returns>Hex string.</returns>
public static string ToHex(byte[] bytes) => Convert.ToHexString(bytes).ToLowerInvariant();
/// <summary>Parses a <c>YYYY-MM</c> month into its first instant (UTC).</summary>
/// <param name="month">The month text.</param>
/// <param name="monthStart">The parsed month.</param>
/// <returns>True when well-formed.</returns>
public static bool TryParseMonth(string? month, out DateTime monthStart)
{
if (DateTime.TryParseExact(month, "yyyy-MM", CultureInfo.InvariantCulture, DateTimeStyles.None, out var parsed))
{
monthStart = new DateTime(parsed.Year, parsed.Month, 1, 0, 0, 0, DateTimeKind.Utc);
return true;
}
monthStart = default;
return false;
}
private static bool TryParseHex(string? hex, out byte[] bytes)
{
bytes = [];
if (hex is null || hex.Length != AuditChainHasher.HashLength * 2)
{
return false;
}
try
{
bytes = Convert.FromHexString(hex);
return true;
}
catch (FormatException)
{
return false;
}
}
}
@@ -1,252 +0,0 @@
using ZB.MOM.WW.ScadaBridge.Commons.Types.Audit;
namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Maintenance;
/// <summary>A month's chain head as verification sees it (ScadaBridge#41).</summary>
/// <param name="LastSeq">Last sequence number linked.</param>
/// <param name="LastHash">RowHash at <paramref name="LastSeq"/>.</param>
/// <param name="PurgedThroughSeq">Checkpoint anchor left by a partition-switch purge, or null.</param>
/// <param name="PurgedThroughHash">RowHash at <paramref name="PurgedThroughSeq"/>, or null.</param>
public sealed record AuditChainHeadState(
long LastSeq,
byte[] LastHash,
long? PurgedThroughSeq,
byte[]? PurgedThroughHash);
/// <summary>
/// Streaming verifier for one month's AuditLog hash chain (ScadaBridge#41). Feed it the
/// month's chained rows in <c>ChainSeq</c> order via <see cref="Accept"/>, then call
/// <see cref="Complete"/>. Storage-agnostic so the detection rules are unit-testable
/// without SQL Server.
/// </summary>
/// <remarks>
/// <para>
/// Verification starts at the checkpoint anchor (<c>PurgedThroughSeq + 1</c>, previous
/// link <c>PurgedThroughHash</c>) when a partition purge left one, otherwise at sequence 1
/// with the genesis hash. Each row's hash is recomputed from the previous link and its own
/// stored columns; a sequence number with no row is bridged only by a purge tombstone. At
/// the end the chain must land exactly on the head's <c>(LastSeq, LastHash)</c>, which is
/// what catches deletion of the newest rows.
/// </para>
/// <para>
/// Detection, by tampering act: an <b>edited</b> row → <see cref="AuditChainBreakKind.HashMismatch"/>
/// at that row; a <b>deleted</b> row → <see cref="AuditChainBreakKind.MissingRow"/> at its
/// sequence number (or <see cref="AuditChainBreakKind.HeadMismatch"/> if it was the tail);
/// <b>reordered</b> rows (ChainSeq values swapped) → <see cref="AuditChainBreakKind.HashMismatch"/>
/// at the first moved row, because the sequence number is inside the hashed bytes.
/// </para>
/// <para>
/// <b>Chain start (cutoff, ScadaBridge#139).</b> When the chain was enabled with no backlog,
/// a link-0 anchor in the tombstones of the cutoff month is that month's genesis hash (instead
/// of 32 zero bytes), and no row before the cutoff is ever chained. So a chained row whose
/// <c>OccurredAtUtc</c> is before the cutoff is a break (<see cref="AuditChainBreakKind.ChainedBeforeCutoff"/>):
/// either the row's timestamp was moved back to hide it among the unprotected rows, or the
/// cutoff was moved forward. An anchor that does not hash to its own cutoff is reported as
/// <see cref="AuditChainBreakKind.CutoffAnchorInvalid"/> by the caller.
/// </para>
/// </remarks>
public sealed class AuditChainVerifier
{
private readonly DateTime _monthStartUtc;
private readonly string _partition;
private readonly AuditChainHeadState? _head;
private readonly IReadOnlyDictionary<long, byte[]> _tombstones;
private readonly long _startAfter;
private readonly DateTime? _chainFromUtc;
private byte[] _previous;
private long _expected;
private long _verified;
private long _tombstonesApplied;
private AuditChainBreak? _break;
/// <summary>Initializes a verifier for one month.</summary>
/// <param name="monthStartUtc">The chain partition (first instant of the month, UTC).</param>
/// <param name="head">The month's chain head, or null when none exists.</param>
/// <param name="tombstones">
/// Purge tombstones for the month, keyed by ChainSeq; a key of
/// <see cref="AuditChainHasher.CutoffAnchorSeq"/> is the chain-start anchor (the month's genesis).
/// </param>
/// <param name="chainFromUtc">The chain cutoff, or null when the whole table is in scope (legacy / backlog chaining).</param>
/// <param name="anchorProblem">When non-null, the chain-start anchor is invalid; verification fails with this detail.</param>
public AuditChainVerifier(
DateTime monthStartUtc,
AuditChainHeadState? head,
IReadOnlyDictionary<long, byte[]> tombstones,
DateTime? chainFromUtc = null,
string? anchorProblem = null)
{
_monthStartUtc = monthStartUtc;
_partition = AuditChainHasher.PartitionLabel(monthStartUtc);
_head = head;
_tombstones = tombstones ?? throw new ArgumentNullException(nameof(tombstones));
if (head?.PurgedThroughSeq is { } purgedThrough && head.PurgedThroughHash is { } anchor)
{
_startAfter = purgedThrough;
_previous = anchor;
}
else
{
_startAfter = 0;
_previous = _tombstones.TryGetValue(AuditChainHasher.CutoffAnchorSeq, out var cutoffAnchor)
? cutoffAnchor
: AuditChainHasher.GenesisHash.ToArray();
}
_expected = _startAfter + 1;
_chainFromUtc = chainFromUtc;
if (anchorProblem is not null)
{
Fail(AuditChainHasher.CutoffAnchorSeq, AuditChainHasher.CutoffAnchorEventId,
AuditChainBreakKind.CutoffAnchorInvalid, anchorProblem);
}
}
/// <summary>The chain cutoff this verification applies, or null.</summary>
public DateTime? ChainFromUtc => _chainFromUtc;
/// <summary>The chain partition label (<c>YYYY-MM</c>).</summary>
public string Partition => _partition;
/// <summary>The checkpoint anchor this verification resumed after, or null.</summary>
public long? PurgedThroughSeq => _head?.PurgedThroughSeq;
/// <summary>True once a break has been found; further rows are ignored.</summary>
public bool IsBroken => _break is not null;
/// <summary>Feeds the next chained row (ascending ChainSeq).</summary>
/// <param name="row">The row; its <see cref="AuditChainRow.ChainSeq"/> must be non-null.</param>
/// <returns>False once a break has been found, so the caller can stop reading.</returns>
public bool Accept(AuditChainRow row)
{
ArgumentNullException.ThrowIfNull(row);
if (_break is not null)
{
return false;
}
var seq = row.ChainSeq ?? throw new ArgumentException("Only chained rows can be verified.", nameof(row));
// Rows at or below a partition-purge anchor cannot exist (the switch removed the
// whole month's partition); ignore rather than mis-report them.
if (seq <= _startAfter)
{
return true;
}
if (_chainFromUtc is { } chainFrom && row.OccurredAtUtc < chainFrom)
{
return Fail(seq, row.EventId, AuditChainBreakKind.ChainedBeforeCutoff,
$"Row at ChainSeq {seq} is dated {row.OccurredAtUtc:O}, before the chain cutoff {chainFrom:O}: "
+ "no row before the cutoff is ever chained, so its timestamp was moved back or the cutoff was moved forward.");
}
if (seq < _expected)
{
return Fail(seq, row.EventId, AuditChainBreakKind.DuplicateSequence,
$"ChainSeq {seq} appears more than once.");
}
while (_expected < seq)
{
if (!_tombstones.TryGetValue(_expected, out var tombstone))
{
return Fail(_expected, row.EventId, AuditChainBreakKind.MissingRow,
$"ChainSeq {_expected} is missing and no purge tombstone covers it; "
+ $"the next chained row is ChainSeq {seq} (EventId {row.EventId}).");
}
_previous = tombstone;
_tombstonesApplied++;
_expected++;
}
if (row.RowHash is not { Length: AuditChainHasher.HashLength } stored)
{
return Fail(seq, row.EventId, AuditChainBreakKind.IncompleteLink,
$"Row at ChainSeq {seq} has a sequence number but no 32-byte RowHash.");
}
var computed = AuditChainHasher.ComputeRowHash(_previous, _monthStartUtc, seq, row);
if (!computed.AsSpan().SequenceEqual(stored))
{
return Fail(seq, row.EventId, AuditChainBreakKind.HashMismatch,
$"Recomputed hash of the row at ChainSeq {seq} does not match its stored RowHash: "
+ "the row was edited, moved to a different position, or an earlier link was altered.");
}
_previous = stored;
_expected++;
_verified++;
return true;
}
/// <summary>Finishes verification against the chain head.</summary>
/// <param name="unchainedRows">Rows in the month, at or after the cutoff, not yet chained (reported, never a break).</param>
/// <param name="preCutoffRows">Rows in the month before the chain cutoff: out of scope, never chained (reported, never a break).</param>
/// <returns>The verification result.</returns>
public AuditChainVerificationResult Complete(long unchainedRows, long preCutoffRows = 0)
{
if (_break is null)
{
// A purge tombstone may cover the newest links.
while (_tombstones.TryGetValue(_expected, out var tombstone))
{
_previous = tombstone;
_tombstonesApplied++;
_expected++;
}
var lastSeq = _expected - 1;
if (_head is null)
{
if (lastSeq > _startAfter)
{
Fail(lastSeq, null, AuditChainBreakKind.HeadMismatch,
$"Chained rows exist through ChainSeq {lastSeq} but the month has no chain head.");
}
else
{
return Result(AuditChainStatus.NotChained, unchainedRows, preCutoffRows);
}
}
else if (lastSeq < _head.LastSeq)
{
Fail(lastSeq + 1, null, AuditChainBreakKind.HeadMismatch,
$"The chain ends at ChainSeq {lastSeq} but the head records {_head.LastSeq}: "
+ $"ChainSeq {lastSeq + 1}..{_head.LastSeq} are missing (rows deleted from the end).");
}
else if (lastSeq > _head.LastSeq)
{
Fail(_head.LastSeq + 1, null, AuditChainBreakKind.HeadMismatch,
$"The chain continues to ChainSeq {lastSeq} past the head's {_head.LastSeq}.");
}
else if (!_previous.AsSpan().SequenceEqual(_head.LastHash))
{
Fail(lastSeq, null, AuditChainBreakKind.HeadMismatch,
$"The hash at ChainSeq {lastSeq} does not match the chain head's recorded hash.");
}
}
return Result(_break is null ? AuditChainStatus.Intact : AuditChainStatus.Broken, unchainedRows, preCutoffRows);
}
private AuditChainVerificationResult Result(AuditChainStatus status, long unchainedRows, long preCutoffRows)
=> new(
_partition,
status,
_verified,
_tombstonesApplied,
_head?.PurgedThroughSeq,
_head?.LastSeq,
unchainedRows,
_break,
preCutoffRows,
_chainFromUtc);
private bool Fail(long seq, Guid? eventId, AuditChainBreakKind kind, string detail)
{
_break ??= new AuditChainBreak(_partition, seq, eventId, kind, detail);
return false;
}
}
@@ -38,12 +38,6 @@ public sealed class AuditLogArchiveOptions
/// </summary>
public int ReadCommandTimeoutMinutes { get; set; } = 60;
/// <summary>
/// How long, in seconds, archiving waits for a chaining pass or another purge to release
/// the hash-chain lock before giving up for this tick (default 300).
/// </summary>
public int ChainLockTimeoutSeconds { get; set; } = 300;
/// <summary>
/// Archive retention in days, counted from the END of the archived month (ScadaBridge#109).
/// <c>0</c> (the default) = archives are never deleted — until compliance names a period.
@@ -1,729 +0,0 @@
using System.Data;
using System.Data.Common;
using System.Text;
using Microsoft.Data.SqlClient;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Logging.Abstractions;
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Audit;
namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Maintenance;
/// <summary>
/// The SQL Server application lock that serialises every writer of AuditLog hash-chain
/// state (ScadaBridge#41): the chaining job and both purge paths take it EXCLUSIVE;
/// verification takes it SHARED so it reads a chain no writer is half-way through.
/// </summary>
public static class AuditChainLock
{
/// <summary>The <c>sp_getapplock</c> resource name.</summary>
public const string Resource = "ScadaBridge.AuditLogHashChain";
}
/// <summary>
/// SQL Server implementation of <see cref="IAuditLogHashChain"/> (ScadaBridge#41,
/// alog.md §11.4). See the interface for the chain model.
/// </summary>
/// <remarks>
/// <para>
/// <b>Concurrency.</b> Two central nodes and the site-ingest path INSERT into
/// <c>dbo.AuditLog</c> concurrently; none of them touches the chain. The chaining pass runs
/// in one transaction that first takes the exclusive <see cref="AuditChainLock"/> with a
/// zero timeout (the peer node's pass simply skips its tick), then reads the head row with
/// <c>UPDLOCK, HOLDLOCK</c> and advances it with an optimistic <c>LastSeq</c> check. Every
/// link UPDATE is guarded by <c>ChainSeq IS NULL</c> and its row count is checked, so a row
/// purged or chained underneath the pass rolls the whole pass back instead of leaving a gap.
/// A row INSERTed while the pass runs is invisible to it (READ COMMITTED) and is chained on
/// a later pass with a later sequence number — chain order is chaining order, not
/// <c>OccurredAtUtc</c> order, which is why late-arriving site rows need no special case.
/// </para>
/// <para>
/// <b>Chain start (ScadaBridge#139).</b> Unless the caller asks for backlog chaining, the first
/// pass on a pristine database (no chain head, no chained row) records a <b>chain-start
/// anchor</b> and only rows dated at or after it are ever chained; the historical backlog stays
/// unchained and out of scope. The anchor is one row in <c>dbo.AuditLogChainTombstone</c> at
/// <c>ChainSeq = 0</c> of the cutoff month: <c>PurgedAtUtc</c> holds the cutoff (the database's
/// <c>SYSUTCDATETIME()</c> at first enablement, so both central nodes see the same instant
/// whatever their own clocks say), <c>RowHash</c> holds
/// <see cref="AuditChainHasher.ComputeCutoffAnchorHash"/> of it, and that hash is the genesis
/// the cutoff month's first row links to. It is written once, under the exclusive chain lock,
/// and never updated; the partition-switch purge keeps it. A database that already has a
/// chain (heads or chained rows) and no anchor keeps chaining every row, as before.
/// </para>
/// <para>
/// <b>Append-only.</b> The only AuditLog columns this class writes are <c>ChainSeq</c> and
/// <c>RowHash</c>, and only while they are NULL. In production that UPDATE runs under the
/// <c>scadabridge_audit_chainer</c> role, which holds a COLUMN-level UPDATE grant on just
/// those two columns; the ingest writer role keeps its table-wide DENY UPDATE.
/// </para>
/// </remarks>
public sealed class AuditLogHashChain : IAuditLogHashChain, IAuditChainHeadStore
{
// 4 parameters per row; 200 rows = 800 parameters, far under SQL Server's 2,100 cap.
private const int UpdateChunkRows = 200;
/// <summary>The ten writable AuditLog columns, in table order — the hashed row content.</summary>
internal const string RowColumns =
"EventId, OccurredAtUtc, Actor, Action, Outcome, Category, Target, SourceNode, CorrelationId, DetailsJson";
private readonly ScadaBridgeDbContext _context;
private readonly ILogger<AuditLogHashChain> _logger;
/// <summary>Initializes a new instance of the <see cref="AuditLogHashChain"/> class.</summary>
/// <param name="context">The database context whose connection is used.</param>
/// <param name="logger">Optional logger.</param>
public AuditLogHashChain(ScadaBridgeDbContext context, ILogger<AuditLogHashChain>? logger = null)
{
_context = context ?? throw new ArgumentNullException(nameof(context));
_logger = logger ?? NullLogger<AuditLogHashChain>.Instance;
}
/// <summary>
/// Command timeout for verification statements, in seconds. A month can hold millions
/// of rows; the management Ask (long-running, 5 min) is the real deadline.
/// </summary>
public int VerifyCommandTimeoutSeconds { get; init; } = 300;
/// <summary>
/// Test seam: the cutoff a new chain-start anchor records. Null (production) uses the
/// database's <c>SYSUTCDATETIME()</c> so both central nodes agree regardless of node clocks.
/// </summary>
internal Func<DateTime>? AnchorClock { get; init; }
/// <summary>How long verification waits for an in-flight chaining/purge pass to release the lock.</summary>
public TimeSpan VerifyLockTimeout { get; init; } = TimeSpan.FromSeconds(60);
/// <inheritdoc />
public async Task<AuditChainBatchResult> ChainPendingAsync(int batchSize, bool chainBacklog = false, CancellationToken ct = default)
{
if (batchSize <= 0)
{
throw new ArgumentOutOfRangeException(nameof(batchSize), batchSize, "Batch size must be > 0.");
}
var conn = _context.Database.GetDbConnection();
var openedHere = await OpenAsync(conn, ct).ConfigureAwait(false);
try
{
await using var tx = await conn.BeginTransactionAsync(IsolationLevel.ReadCommitted, ct).ConfigureAwait(false);
if (!await TryAcquireLockAsync(conn, tx, "Exclusive", TimeSpan.Zero, ct).ConfigureAwait(false))
{
await tx.RollbackAsync(ct).ConfigureAwait(false);
return new AuditChainBatchResult(0, LockBusy: true);
}
// Chain start (ScadaBridge#139): the cutoff, recorded on the first pass.
var cutoff = await ReadChainerCutoffAsync(conn, tx, ct).ConfigureAwait(false);
if (cutoff is null && !chainBacklog && await IsPristineAsync(conn, tx, ct).ConfigureAwait(false))
{
cutoff = await CreateCutoffAnchorAsync(conn, tx, ct).ConfigureAwait(false);
}
// Oldest month that still has unchained rows in scope. One month per pass keeps the
// candidate read partition-eliminated and index-ordered (no cross-partition sort).
DateTime? oldest;
await using (var cmd = Command(conn, tx,
"SELECT TOP (1) OccurredAtUtc FROM dbo.AuditLog WHERE ChainSeq IS NULL AND OccurredAtUtc >= @chainFrom ORDER BY OccurredAtUtc;"))
{
AddChainFrom(cmd, cutoff?.ChainFromUtc);
var scalar = await cmd.ExecuteScalarAsync(ct).ConfigureAwait(false);
oldest = scalar is DateTime dt ? DateTime.SpecifyKind(dt, DateTimeKind.Utc) : null;
}
if (oldest is null)
{
await tx.CommitAsync(ct).ConfigureAwait(false);
return new AuditChainBatchResult(0, LockBusy: false);
}
var month = AuditChainHasher.MonthOf(oldest.Value);
var candidates = new List<AuditChainRow>(Math.Min(batchSize, 4096));
await using (var cmd = Command(conn, tx,
$"SELECT TOP (@batch) {RowColumns} FROM dbo.AuditLog " +
"WHERE ChainSeq IS NULL AND OccurredAtUtc >= @monthStart AND OccurredAtUtc < @monthEnd " +
"AND OccurredAtUtc >= @chainFrom " +
"ORDER BY OccurredAtUtc, EventId;"))
{
AddInt(cmd, "@batch", batchSize);
AddMonthRange(cmd, month);
AddChainFrom(cmd, cutoff?.ChainFromUtc);
await using var reader = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false);
while (await reader.ReadAsync(ct).ConfigureAwait(false))
{
candidates.Add(ReadRow(reader, withLink: false));
}
}
// Current chain end for the month, locked until commit.
long? headSeq = null;
byte[] previous = AuditChainHasher.GenesisHash.ToArray();
await using (var cmd = Command(conn, tx,
"SELECT LastSeq, LastHash FROM dbo.AuditLogChainHead WITH (UPDLOCK, HOLDLOCK) WHERE ChainMonth = @month;"))
{
AddDate(cmd, "@month", month);
await using var reader = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false);
if (await reader.ReadAsync(ct).ConfigureAwait(false))
{
headSeq = reader.GetInt64(0);
previous = (byte[])reader.GetValue(1);
}
}
// The cutoff month's chain starts from the anchor, not from 32 zero bytes, so the
// cutoff is inside every later link of that month.
if (headSeq is null && cutoff is not null && cutoff.Month == month)
{
previous = cutoff.AnchorHash.ToArray();
}
var seq = headSeq ?? 0;
var links = new List<(AuditChainRow Row, long Seq, byte[] Hash)>(candidates.Count);
foreach (var row in candidates)
{
seq++;
var hash = AuditChainHasher.ComputeRowHash(previous, month, seq, row);
links.Add((row, seq, hash));
previous = hash;
}
for (var offset = 0; offset < links.Count; offset += UpdateChunkRows)
{
var chunk = links.Skip(offset).Take(UpdateChunkRows).ToList();
var updated = await WriteLinksAsync(conn, tx, chunk, ct).ConfigureAwait(false);
if (updated != chunk.Count)
{
// A candidate was purged (or chained by a pass that bypassed the lock)
// between the read and the write. Linking the rest would leave a gap.
_logger.LogWarning(
"AuditLog hash chain: {Updated} of {Expected} links written for {Partition}; rolling the pass back to retry.",
updated, chunk.Count, AuditChainHasher.PartitionLabel(month));
await tx.RollbackAsync(ct).ConfigureAwait(false);
return new AuditChainBatchResult(0, LockBusy: false);
}
}
if (links.Count > 0 && !await AdvanceHeadAsync(conn, tx, month, headSeq, seq, previous, ct).ConfigureAwait(false))
{
_logger.LogWarning(
"AuditLog hash chain: head for {Partition} moved underneath the pass; rolling back to retry.",
AuditChainHasher.PartitionLabel(month));
await tx.RollbackAsync(ct).ConfigureAwait(false);
return new AuditChainBatchResult(0, LockBusy: false);
}
await tx.CommitAsync(ct).ConfigureAwait(false);
return new AuditChainBatchResult(links.Count, LockBusy: false);
}
finally
{
if (openedHere)
{
await conn.CloseAsync().ConfigureAwait(false);
}
}
}
/// <inheritdoc />
public async Task<AuditChainVerificationResult> VerifyMonthAsync(DateTime monthStartUtc, CancellationToken ct = default)
{
var month = AuditChainHasher.MonthOf(DateTime.SpecifyKind(monthStartUtc, DateTimeKind.Utc));
var conn = _context.Database.GetDbConnection();
var openedHere = await OpenAsync(conn, ct).ConfigureAwait(false);
try
{
await using var tx = await conn.BeginTransactionAsync(IsolationLevel.ReadCommitted, ct).ConfigureAwait(false);
if (!await TryAcquireLockAsync(conn, tx, "Shared", VerifyLockTimeout, ct).ConfigureAwait(false))
{
await tx.RollbackAsync(ct).ConfigureAwait(false);
throw new TimeoutException(
$"The AuditLog hash chain was busy (chaining or purge in progress) for {VerifyLockTimeout.TotalSeconds:0}s; retry.");
}
AuditChainHeadState? head = null;
await using (var cmd = Command(conn, tx,
"SELECT LastSeq, LastHash, PurgedThroughSeq, PurgedThroughHash FROM dbo.AuditLogChainHead WHERE ChainMonth = @month;"))
{
AddDate(cmd, "@month", month);
await using var reader = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false);
if (await reader.ReadAsync(ct).ConfigureAwait(false))
{
head = new AuditChainHeadState(
reader.GetInt64(0),
(byte[])reader.GetValue(1),
reader.IsDBNull(2) ? null : reader.GetInt64(2),
reader.IsDBNull(3) ? null : (byte[])reader.GetValue(3));
}
}
var cutoff = await ReadVerifierCutoffAsync(conn, tx, ct).ConfigureAwait(false);
var tombstones = new Dictionary<long, byte[]>();
await using (var cmd = Command(conn, tx,
"SELECT ChainSeq, RowHash FROM dbo.AuditLogChainTombstone WHERE ChainMonth = @month;"))
{
AddDate(cmd, "@month", month);
await using var reader = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false);
while (await reader.ReadAsync(ct).ConfigureAwait(false))
{
tombstones[reader.GetInt64(0)] = (byte[])reader.GetValue(1);
}
}
// Unchained rows split at the cutoff: pending (in scope, not reached yet) versus
// pre-cutoff (out of scope by design, never chained).
long unchained = 0, preCutoff = 0;
await using (var cmd = Command(conn, tx,
"SELECT COUNT_BIG(CASE WHEN OccurredAtUtc >= @chainFrom THEN 1 END), COUNT_BIG(CASE WHEN OccurredAtUtc < @chainFrom THEN 1 END) " +
"FROM dbo.AuditLog WHERE ChainSeq IS NULL AND OccurredAtUtc >= @monthStart AND OccurredAtUtc < @monthEnd;"))
{
cmd.CommandTimeout = VerifyCommandTimeoutSeconds;
AddMonthRange(cmd, month);
AddChainFrom(cmd, cutoff.ChainFromUtc);
await using var reader = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false);
if (await reader.ReadAsync(ct).ConfigureAwait(false))
{
unchained = reader.GetInt64(0);
preCutoff = reader.GetInt64(1);
}
}
var verifier = new AuditChainVerifier(month, head, tombstones, cutoff.ChainFromUtc, cutoff.Problem);
await using (var cmd = Command(conn, tx,
$"SELECT {RowColumns}, ChainSeq, RowHash FROM dbo.AuditLog " +
"WHERE ChainSeq IS NOT NULL AND OccurredAtUtc >= @monthStart AND OccurredAtUtc < @monthEnd " +
"ORDER BY ChainSeq;"))
{
cmd.CommandTimeout = VerifyCommandTimeoutSeconds;
AddMonthRange(cmd, month);
await using var reader = await cmd.ExecuteReaderAsync(CommandBehavior.SequentialAccess, ct).ConfigureAwait(false);
while (await reader.ReadAsync(ct).ConfigureAwait(false))
{
if (!verifier.Accept(ReadRow(reader, withLink: true)))
{
break;
}
}
}
await tx.CommitAsync(ct).ConfigureAwait(false);
return verifier.Complete(unchained, preCutoff);
}
finally
{
if (openedHere)
{
await conn.CloseAsync().ConfigureAwait(false);
}
}
}
/// <inheritdoc />
public async Task<IReadOnlyList<AuditChainHeadSnapshot>> ReadHeadsAsync(CancellationToken ct = default)
{
var conn = _context.Database.GetDbConnection();
var openedHere = await OpenAsync(conn, ct).ConfigureAwait(false);
try
{
await using var tx = await conn.BeginTransactionAsync(IsolationLevel.ReadCommitted, ct).ConfigureAwait(false);
await AcquireSharedOrThrowAsync(conn, tx, ct).ConfigureAwait(false);
var heads = await ReadHeadsCoreAsync(conn, tx, ct).ConfigureAwait(false);
await tx.CommitAsync(ct).ConfigureAwait(false);
return heads;
}
finally
{
if (openedHere)
{
await conn.CloseAsync().ConfigureAwait(false);
}
}
}
/// <inheritdoc />
public async Task<AuditChainHeadVerificationResult> VerifyExportedHeadsAsync(
IReadOnlyList<AuditChainHeadRecord> records, CancellationToken ct = default)
{
ArgumentNullException.ThrowIfNull(records);
// Collapse duplicates (both centrals export, and every unchanged re-export repeats a
// record): one check per distinct (month, sequence, hash) claim.
var distinct = records
.GroupBy(r => (r.Month, r.LastSeq, Hash: r.LastHash?.ToLowerInvariant()))
.Select(g => g.OrderByDescending(r => r.ExportedAtUtc).First())
.ToList();
var conn = _context.Database.GetDbConnection();
var openedHere = await OpenAsync(conn, ct).ConfigureAwait(false);
try
{
await using var tx = await conn.BeginTransactionAsync(IsolationLevel.ReadCommitted, ct).ConfigureAwait(false);
await AcquireSharedOrThrowAsync(conn, tx, ct).ConfigureAwait(false);
var heads = (await ReadHeadsCoreAsync(conn, tx, ct).ConfigureAwait(false))
.ToDictionary(h => AuditChainHasher.PartitionLabel(h.Month), StringComparer.Ordinal);
var checks = new List<AuditChainHeadCheck>(distinct.Count);
foreach (var record in distinct)
{
if (!AuditChainHeadComparer.TryParseMonth(record.Month, out var month))
{
checks.Add(new AuditChainHeadCheck(record.Month ?? "", record.LastSeq, record.ExportedAtUtc, record.Node,
AuditChainHeadStatus.Mismatch, null, $"Exported month '{record.Month}' is not YYYY-MM."));
continue;
}
heads.TryGetValue(AuditChainHasher.PartitionLabel(month), out var head);
checks.Add(await AuditChainHeadComparer.CompareAsync(
record, head, seq => ReadStoredLinkAsync(conn, tx, month, seq, ct)).ConfigureAwait(false));
}
await tx.CommitAsync(ct).ConfigureAwait(false);
var ordered = checks
.OrderByDescending(c => c.IsFailure)
.ThenBy(c => c.Month, StringComparer.Ordinal)
.ThenBy(c => c.ExportedSeq)
.ToList();
return new AuditChainHeadVerificationResult(ordered.All(c => !c.IsFailure), ordered.Count, ordered);
}
finally
{
if (openedHere)
{
await conn.CloseAsync().ConfigureAwait(false);
}
}
}
private async Task AcquireSharedOrThrowAsync(DbConnection conn, DbTransaction tx, CancellationToken ct)
{
if (!await TryAcquireLockAsync(conn, tx, "Shared", VerifyLockTimeout, ct).ConfigureAwait(false))
{
await tx.RollbackAsync(ct).ConfigureAwait(false);
throw new TimeoutException(
$"The AuditLog hash chain was busy (chaining or purge in progress) for {VerifyLockTimeout.TotalSeconds:0}s; retry.");
}
}
private async Task<IReadOnlyList<AuditChainHeadSnapshot>> ReadHeadsCoreAsync(DbConnection conn, DbTransaction tx, CancellationToken ct)
{
// Per month: the head row, the chained rows still present (the filtered ChainSeq index,
// partition-eliminated by the month range) and the tombstones. The head table holds one
// row per month ever chained, so this is a few dozen rows.
var heads = new List<AuditChainHeadSnapshot>();
await using var cmd = Command(conn, tx, @"
SELECT h.ChainMonth, h.LastSeq, h.LastHash, h.PurgedThroughSeq, h.PurgedThroughHash, h.UpdatedAtUtc,
(SELECT COUNT_BIG(*) FROM dbo.AuditLog a
WHERE a.ChainSeq IS NOT NULL
AND a.OccurredAtUtc >= CAST(h.ChainMonth AS datetime2(7))
AND a.OccurredAtUtc < DATEADD(MONTH, 1, CAST(h.ChainMonth AS datetime2(7)))) AS ChainedRows,
(SELECT COUNT_BIG(*) FROM dbo.AuditLogChainTombstone t WHERE t.ChainMonth = h.ChainMonth AND t.ChainSeq > 0) AS Tombstones
FROM dbo.AuditLogChainHead h
ORDER BY h.ChainMonth;");
cmd.CommandTimeout = VerifyCommandTimeoutSeconds;
await using var reader = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false);
while (await reader.ReadAsync(ct).ConfigureAwait(false))
{
heads.Add(new AuditChainHeadSnapshot(
DateTime.SpecifyKind(reader.GetDateTime(0), DateTimeKind.Utc),
reader.GetInt64(1),
(byte[])reader.GetValue(2),
reader.IsDBNull(3) ? null : reader.GetInt64(3),
reader.IsDBNull(4) ? null : (byte[])reader.GetValue(4),
reader.GetInt64(6),
reader.GetInt64(7),
DateTime.SpecifyKind(reader.GetDateTime(5), DateTimeKind.Utc)));
}
return heads;
}
private async Task<AuditChainHeadComparer.StoredLink> ReadStoredLinkAsync(
DbConnection conn, DbTransaction tx, DateTime month, long seq, CancellationToken ct)
{
byte[]? rowHash;
await using (var cmd = Command(conn, tx,
"SELECT TOP (1) RowHash FROM dbo.AuditLog WHERE ChainSeq = @seq AND OccurredAtUtc >= @monthStart AND OccurredAtUtc < @monthEnd;"))
{
cmd.CommandTimeout = VerifyCommandTimeoutSeconds;
cmd.Parameters.Add(new SqlParameter("@seq", SqlDbType.BigInt) { Value = seq });
AddMonthRange(cmd, month);
rowHash = await cmd.ExecuteScalarAsync(ct).ConfigureAwait(false) as byte[];
}
byte[]? tombstoneHash = null;
if (rowHash is null)
{
await using var cmd = Command(conn, tx,
"SELECT RowHash FROM dbo.AuditLogChainTombstone WHERE ChainMonth = @month AND ChainSeq = @seq;");
AddDate(cmd, "@month", month);
cmd.Parameters.Add(new SqlParameter("@seq", SqlDbType.BigInt) { Value = seq });
tombstoneHash = await cmd.ExecuteScalarAsync(ct).ConfigureAwait(false) as byte[];
}
return new AuditChainHeadComparer.StoredLink(rowHash, tombstoneHash);
}
/// <summary>The chain-start anchor as the chainer uses it (ScadaBridge#139).</summary>
/// <param name="ChainFromUtc">The cutoff.</param>
/// <param name="Month">The cutoff month (the anchor's <c>ChainMonth</c>).</param>
/// <param name="AnchorHash">The anchor's stored hash: the cutoff month's genesis.</param>
private sealed record ChainCutoff(DateTime ChainFromUtc, DateTime Month, byte[] AnchorHash);
/// <summary>The cutoff as verification sees it (ScadaBridge#139).</summary>
/// <param name="ChainFromUtc">The cutoff, or null when the chain covers the whole table.</param>
/// <param name="Problem">Why the anchor cannot be trusted, or null.</param>
private sealed record VerifierCutoff(DateTime? ChainFromUtc, string? Problem);
/// <summary>
/// Reads the chain-start anchor for the chaining pass. It can only sit in a month at or
/// before the earliest chain head (it is written before anything is chained, and nothing
/// before it ever is), so the read is bounded to those months rather than scanning every
/// per-channel purge tombstone each tick. Verification reads them all.
/// </summary>
private static async Task<ChainCutoff?> ReadChainerCutoffAsync(DbConnection conn, DbTransaction tx, CancellationToken ct)
{
await using var cmd = Command(conn, tx,
"SELECT TOP (1) ChainMonth, RowHash, PurgedAtUtc FROM dbo.AuditLogChainTombstone " +
"WHERE ChainSeq = 0 AND ChainMonth <= ISNULL((SELECT MIN(ChainMonth) FROM dbo.AuditLogChainHead), '9999-12-01') " +
"ORDER BY PurgedAtUtc, ChainMonth;");
await using var reader = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false);
if (!await reader.ReadAsync(ct).ConfigureAwait(false))
{
return null;
}
return new ChainCutoff(
DateTime.SpecifyKind(reader.GetDateTime(2), DateTimeKind.Utc),
DateTime.SpecifyKind(reader.GetDateTime(0), DateTimeKind.Utc),
(byte[])reader.GetValue(1));
}
/// <summary>
/// Reads every chain-start anchor and checks it: exactly one, in its cutoff's month, hashing
/// to its own cutoff. The earliest anchor's cutoff is used either way, matching the chainer.
/// </summary>
private async Task<VerifierCutoff> ReadVerifierCutoffAsync(DbConnection conn, DbTransaction tx, CancellationToken ct)
{
var anchors = new List<(DateTime Month, byte[] Hash, DateTime ChainFrom)>();
await using (var cmd = Command(conn, tx,
"SELECT ChainMonth, RowHash, PurgedAtUtc FROM dbo.AuditLogChainTombstone WHERE ChainSeq = 0 ORDER BY PurgedAtUtc, ChainMonth;"))
{
cmd.CommandTimeout = VerifyCommandTimeoutSeconds;
await using var reader = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false);
while (await reader.ReadAsync(ct).ConfigureAwait(false))
{
anchors.Add((
DateTime.SpecifyKind(reader.GetDateTime(0), DateTimeKind.Utc),
(byte[])reader.GetValue(1),
DateTime.SpecifyKind(reader.GetDateTime(2), DateTimeKind.Utc)));
}
}
if (anchors.Count == 0)
{
return new VerifierCutoff(null, null);
}
var (anchorMonth, hash, chainFrom) = anchors[0];
string? problem = null;
if (anchors.Count > 1)
{
problem = $"{anchors.Count} chain-start anchors exist (cutoffs {string.Join(", ", anchors.Select(a => a.ChainFrom.ToString("O", System.Globalization.CultureInfo.InvariantCulture)))}); "
+ "exactly one is ever written.";
}
else if (anchorMonth != AuditChainHasher.MonthOf(chainFrom))
{
problem = $"The chain-start anchor is filed under {AuditChainHasher.PartitionLabel(anchorMonth)} but its cutoff {chainFrom:O} is in another month.";
}
else if (!AuditChainHasher.ComputeCutoffAnchorHash(chainFrom).AsSpan().SequenceEqual(hash))
{
problem = $"The chain-start anchor's hash does not match its cutoff {chainFrom:O}: the cutoff or the anchor was altered.";
}
return new VerifierCutoff(chainFrom, problem);
}
/// <summary>True when nothing has ever been chained: no chain head and no chained row.</summary>
private static async Task<bool> IsPristineAsync(DbConnection conn, DbTransaction tx, CancellationToken ct)
{
await using var cmd = Command(conn, tx,
"SELECT CASE WHEN EXISTS (SELECT 1 FROM dbo.AuditLogChainHead) " +
"OR EXISTS (SELECT 1 FROM dbo.AuditLog WHERE ChainSeq IS NOT NULL) THEN 0 ELSE 1 END;");
return Convert.ToInt32(await cmd.ExecuteScalarAsync(ct).ConfigureAwait(false)) == 1;
}
/// <summary>
/// Writes the chain-start anchor at the database's current UTC time (ScadaBridge#139). Runs
/// inside the chaining transaction, under the exclusive chain lock, so exactly one node
/// writes it and the peer's next pass reads it.
/// </summary>
private async Task<ChainCutoff> CreateCutoffAnchorAsync(DbConnection conn, DbTransaction tx, CancellationToken ct)
{
DateTime chainFrom;
if (AnchorClock is { } clock)
{
chainFrom = DateTime.SpecifyKind(clock(), DateTimeKind.Utc);
}
else
{
await using var cmd = Command(conn, tx, "SELECT SYSUTCDATETIME();");
chainFrom = DateTime.SpecifyKind((DateTime)(await cmd.ExecuteScalarAsync(ct).ConfigureAwait(false))!, DateTimeKind.Utc);
}
var month = AuditChainHasher.MonthOf(chainFrom);
var hash = AuditChainHasher.ComputeCutoffAnchorHash(chainFrom);
await using (var cmd = Command(conn, tx,
"INSERT INTO dbo.AuditLogChainTombstone (ChainMonth, ChainSeq, RowHash, EventId, PurgedAtUtc) " +
"VALUES (@month, 0, @hash, @eventId, @chainFrom);"))
{
AddDate(cmd, "@month", month);
cmd.Parameters.Add(new SqlParameter("@hash", SqlDbType.Binary, AuditChainHasher.HashLength) { Value = hash });
cmd.Parameters.Add(new SqlParameter("@eventId", SqlDbType.UniqueIdentifier) { Value = AuditChainHasher.CutoffAnchorEventId });
cmd.Parameters.Add(new SqlParameter("@chainFrom", SqlDbType.DateTime2) { Scale = 7, Value = chainFrom });
await cmd.ExecuteNonQueryAsync(ct).ConfigureAwait(false);
}
_logger.LogInformation(
"AuditLog hash chain started at {ChainFrom:O}: only rows dated from then on are chained; earlier rows stay unchained and out of scope (ScadaBridge#139).",
chainFrom);
return new ChainCutoff(chainFrom, month, hash);
}
private static void AddChainFrom(DbCommand cmd, DateTime? chainFromUtc)
=> cmd.Parameters.Add(new SqlParameter("@chainFrom", SqlDbType.DateTime2) { Scale = 7, Value = chainFromUtc ?? DateTime.MinValue });
/// <summary>
/// Takes the chain application lock inside <paramref name="tx"/> (released at commit /
/// rollback). Returns false when the lock could not be granted within <paramref name="timeout"/>.
/// </summary>
internal static async Task<bool> TryAcquireLockAsync(
DbConnection conn, DbTransaction tx, string mode, TimeSpan timeout, CancellationToken ct)
{
await using var cmd = Command(conn, tx,
"DECLARE @result int; " +
"EXEC @result = sp_getapplock @Resource = @resource, @LockMode = @mode, @LockOwner = 'Transaction', @LockTimeout = @timeoutMs; " +
"SELECT @result;");
cmd.Parameters.Add(new SqlParameter("@resource", SqlDbType.NVarChar, 255) { Value = AuditChainLock.Resource });
cmd.Parameters.Add(new SqlParameter("@mode", SqlDbType.VarChar, 32) { Value = mode });
AddInt(cmd, "@timeoutMs", (int)Math.Min(int.MaxValue, timeout.TotalMilliseconds));
if (timeout > TimeSpan.Zero)
{
cmd.CommandTimeout = (int)Math.Ceiling(timeout.TotalSeconds) + 30;
}
var result = Convert.ToInt32(await cmd.ExecuteScalarAsync(ct).ConfigureAwait(false));
return result >= 0;
}
private static async Task<int> WriteLinksAsync(
DbConnection conn, DbTransaction tx, IReadOnlyList<(AuditChainRow Row, long Seq, byte[] Hash)> chunk, CancellationToken ct)
{
var sql = new StringBuilder();
// The ONE sanctioned UPDATE against AuditLog outside the maintenance backfill: it
// writes only the two chain columns, only while they are still NULL.
sql.Append("UPDATE dbo.AuditLog SET ChainSeq = v.ChainSeq, RowHash = v.RowHash\n"); // AUDIT-PURGE-ALLOWED: hash-chain link (ChainSeq/RowHash only, NULL→value), chaining job
sql.Append("FROM dbo.AuditLog INNER JOIN (VALUES ");
await using var cmd = conn.CreateCommand();
cmd.Transaction = tx;
for (var i = 0; i < chunk.Count; i++)
{
if (i > 0)
{
sql.Append(", ");
}
sql.Append($"(@e{i}, @o{i}, @s{i}, @h{i})");
cmd.Parameters.Add(new SqlParameter($"@e{i}", SqlDbType.UniqueIdentifier) { Value = chunk[i].Row.EventId });
cmd.Parameters.Add(new SqlParameter($"@o{i}", SqlDbType.DateTime2) { Scale = 7, Value = chunk[i].Row.OccurredAtUtc });
cmd.Parameters.Add(new SqlParameter($"@s{i}", SqlDbType.BigInt) { Value = chunk[i].Seq });
cmd.Parameters.Add(new SqlParameter($"@h{i}", SqlDbType.Binary, AuditChainHasher.HashLength) { Value = chunk[i].Hash });
}
sql.Append(") AS v (EventId, OccurredAtUtc, ChainSeq, RowHash)\n");
sql.Append("ON dbo.AuditLog.EventId = v.EventId AND dbo.AuditLog.OccurredAtUtc = v.OccurredAtUtc\n");
sql.Append("WHERE dbo.AuditLog.ChainSeq IS NULL;");
cmd.CommandText = sql.ToString();
return await cmd.ExecuteNonQueryAsync(ct).ConfigureAwait(false);
}
private static async Task<bool> AdvanceHeadAsync(
DbConnection conn, DbTransaction tx, DateTime month, long? oldSeq, long newSeq, byte[] newHash, CancellationToken ct)
{
await using var cmd = Command(conn, tx, oldSeq is null
? "INSERT INTO dbo.AuditLogChainHead (ChainMonth, LastSeq, LastHash, UpdatedAtUtc) VALUES (@month, @seq, @hash, SYSUTCDATETIME());"
: "UPDATE dbo.AuditLogChainHead SET LastSeq = @seq, LastHash = @hash, UpdatedAtUtc = SYSUTCDATETIME() " +
"WHERE ChainMonth = @month AND LastSeq = @oldSeq;");
AddDate(cmd, "@month", month);
cmd.Parameters.Add(new SqlParameter("@seq", SqlDbType.BigInt) { Value = newSeq });
cmd.Parameters.Add(new SqlParameter("@hash", SqlDbType.Binary, AuditChainHasher.HashLength) { Value = newHash });
if (oldSeq is not null)
{
cmd.Parameters.Add(new SqlParameter("@oldSeq", SqlDbType.BigInt) { Value = oldSeq.Value });
}
return await cmd.ExecuteNonQueryAsync(ct).ConfigureAwait(false) == 1;
}
/// <summary>
/// Reads one row selected as <see cref="RowColumns"/> (plus <c>ChainSeq, RowHash</c> when
/// <paramref name="withLink"/>), strictly in ordinal order so it is safe under SequentialAccess.
/// </summary>
internal static AuditChainRow ReadRow(DbDataReader reader, bool withLink)
{
// Ordinals follow RowColumns (+ ChainSeq, RowHash). Read strictly in order — the
// verification reader is opened with SequentialAccess.
var eventId = reader.GetGuid(0);
var occurred = DateTime.SpecifyKind(reader.GetDateTime(1), DateTimeKind.Utc);
var actor = reader.IsDBNull(2) ? null : reader.GetString(2);
var action = reader.GetString(3);
var outcome = reader.GetString(4);
var category = reader.GetString(5);
var target = reader.IsDBNull(6) ? null : reader.GetString(6);
var sourceNode = reader.IsDBNull(7) ? null : reader.GetString(7);
Guid? correlationId = reader.IsDBNull(8) ? null : reader.GetGuid(8);
var details = reader.IsDBNull(9) ? null : reader.GetString(9);
long? chainSeq = null;
byte[]? rowHash = null;
if (withLink)
{
chainSeq = reader.IsDBNull(10) ? null : reader.GetInt64(10);
rowHash = reader.IsDBNull(11) ? null : (byte[])reader.GetValue(11);
}
return new AuditChainRow(eventId, occurred, actor, action, outcome, category, target, sourceNode,
correlationId, details, chainSeq, rowHash);
}
private static DbCommand Command(DbConnection conn, DbTransaction tx, string sql)
{
var cmd = conn.CreateCommand();
cmd.Transaction = tx;
cmd.CommandText = sql;
return cmd;
}
private static void AddInt(DbCommand cmd, string name, int value)
=> cmd.Parameters.Add(new SqlParameter(name, SqlDbType.Int) { Value = value });
private static void AddDate(DbCommand cmd, string name, DateTime month)
=> cmd.Parameters.Add(new SqlParameter(name, SqlDbType.Date) { Value = month.Date });
private static void AddMonthRange(DbCommand cmd, DateTime month)
{
cmd.Parameters.Add(new SqlParameter("@monthStart", SqlDbType.DateTime2) { Scale = 7, Value = month });
cmd.Parameters.Add(new SqlParameter("@monthEnd", SqlDbType.DateTime2) { Scale = 7, Value = month.AddMonths(1) });
}
internal static async Task<bool> OpenAsync(DbConnection conn, CancellationToken ct)
{
if (conn.State == ConnectionState.Open)
{
return false;
}
await conn.OpenAsync(ct).ConfigureAwait(false);
return true;
}
}
@@ -12,16 +12,13 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Maintenance;
/// <remarks>
/// <para>
/// <b>Superseded stale archives.</b> A purge whose guard finds the partition changed after it
/// was archived (51002/51003) purges nothing and leaves its archive at status
/// was archived (51002) purges nothing and leaves its archive at status
/// <see cref="AuditArchiveStatus.Archived"/>; the next tick writes a fresh archive and purges
/// with it. The stale one is redundant only if a LATER archive B of the same partition, which
/// is <see cref="AuditArchiveStatus.Purged"/> and verifies, holds everything the stale archive A
/// holds. That is proven, not assumed: for every chain month of A, B starts from the same
/// earlier purge anchor and carries A's chain-end hash at A's chain-end sequence (as a row or a
/// tombstone) — so, the chain being a hash chain B itself verifies, every chained row of A is
/// in B; and every UNCHAINED row of A is found in B (row or tombstone) by EventId. Anything
/// else — notably an unchained row a per-channel purge removed in between, which leaves no
/// tombstone — keeps A (<see cref="AuditArchiveCleanupAction.KeptStale"/>). An archive left at
/// holds. That is proven, not assumed: every row of A is found in B by EventId. Anything
/// else — notably a row a per-channel purge removed in between — keeps A
/// (<see cref="AuditArchiveCleanupAction.KeptStale"/>). An archive left at
/// <c>Archived</c> because the node stopped between the switch and the manifest update has no
/// later purged archive and is kept: it is the only copy.
/// </para>
@@ -31,8 +28,9 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Maintenance;
/// an <c>Archived</c> one — when its partition no longer holds rows in <c>dbo.AuditLog</c>.
/// </para>
/// <para>
/// <b>Safe deletion.</b> Only the three files a well-formed archive name implies
/// (<c>{name}.manifest.json</c>, <c>{name}.parquet</c>, <c>{name}.tombstones.parquet</c>) in the
/// <b>Safe deletion.</b> Only the files a well-formed archive name implies
/// (<c>{name}.manifest.json</c>, <c>{name}.parquet</c>, and a format-v1 archive's
/// <c>{name}.tombstones.parquet</c>) in the
/// archive's own <c>{Directory}/{YYYY}</c> folder are deleted — never a path read from the
/// manifest — and the manifest goes FIRST, so an interrupted deletion leaves leftovers that are
/// no longer an archive rather than a manifest naming missing files.
@@ -41,10 +39,10 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Maintenance;
public sealed partial class AuditLogPartitionArchive
{
/// <summary>
/// Upper bound on unchained rows compared by EventId when proving a stale archive redundant;
/// a larger set keeps the archive rather than spend the memory.
/// Upper bound on rows compared by EventId when proving a stale archive redundant;
/// a larger archive is kept rather than spend the memory.
/// </summary>
internal const long MaxUnchainedRowsCompared = 5_000_000;
internal const long MaxRowsCompared = 5_000_000;
/// <inheritdoc />
public async Task<AuditArchiveCleanupResult> CleanupAsync(bool dryRun, CancellationToken ct = default)
@@ -104,7 +102,7 @@ public sealed partial class AuditLogPartitionArchive
var now = _time.GetUtcNow().UtcDateTime;
foreach (var manifest in manifests.Where(m => !deleted.Contains(m.Name)))
{
var monthEnd = AuditChainHasher.MonthOf(manifest.PartitionBoundaryUtc).AddMonths(1);
var monthEnd = AuditArchiveParquet.MonthOf(manifest.PartitionBoundaryUtc).AddMonths(1);
if (monthEnd.AddDays(_options.RetentionDays) > now)
{
continue;
@@ -230,102 +228,44 @@ public sealed partial class AuditLogPartitionArchive
/// </summary>
private async Task<string?> ProveCoveredAsync(AuditArchiveManifest stale, AuditArchiveManifest later, CancellationToken ct)
{
// Same chain generation and at least as far, for every chain month the stale archive has.
var laterChains = later.Chains.ToDictionary(c => c.Partition, StringComparer.Ordinal);
var targets = new Dictionary<(DateTime Month, long Seq), string>();
foreach (var chain in stale.Chains)
if (stale.RowCount > MaxRowsCompared)
{
if (!laterChains.TryGetValue(chain.Partition, out var next))
{
return $"{later.Name} has no chain for {chain.Partition}.";
}
if (next.PriorPurgedThroughSeq != chain.PriorPurgedThroughSeq || next.LastSeq < chain.LastSeq)
{
return $"{later.Name}'s chain for {chain.Partition} does not extend this archive's (prior anchor or end differs).";
}
if (chain.LastSeq > (chain.PriorPurgedThroughSeq ?? 0))
{
targets[(AuditArchiveParquet.ParseMonth(chain.Partition), chain.LastSeq)] = chain.LastHash.ToLowerInvariant();
}
return $"{stale.RowCount} rows exceed the {MaxRowsCompared} compared by EventId.";
}
if (stale.UnchainedRowCount > MaxUnchainedRowsCompared)
{
return $"{stale.UnchainedRowCount} unchained rows exceed the {MaxUnchainedRowsCompared} compared by EventId.";
}
// The later archive must itself be intact and tied to the database.
// The later archive must itself be intact.
var verification = await VerifyAsync(later.Name, ct).ConfigureAwait(false);
if (!verification.Verified)
{
return $"{later.Name} does not verify ({verification.FileProblem ?? "chain or anchor problem"}).";
return $"{later.Name} does not verify ({verification.FileProblem}).";
}
var directory = ArchiveDirectory(stale);
var unchained = new HashSet<Guid>();
await using (var input = File.OpenRead(Path.Combine(directory, stale.Name + ".parquet")))
var missing = new HashSet<Guid>();
await using (var input = File.OpenRead(Path.Combine(ArchiveDirectory(stale), stale.Name + ".parquet")))
{
await foreach (var row in AuditArchiveParquet.ReadRowsAsync(input, ct).ConfigureAwait(false))
{
if (row.ChainSeq is null)
{
unchained.Add(row.EventId);
}
missing.Add(row.EventId);
}
}
var laterDirectory = ArchiveDirectory(later);
await using (var input = File.OpenRead(Path.Combine(laterDirectory, later.Name + ".parquet")))
await using (var input = File.OpenRead(Path.Combine(ArchiveDirectory(later), later.Name + ".parquet")))
{
await foreach (var row in AuditArchiveParquet.ReadRowsAsync(input, ct).ConfigureAwait(false))
{
unchained.Remove(row.EventId);
if (row.ChainSeq is { } seq
&& targets.TryGetValue((AuditChainHasher.MonthOf(row.OccurredAtUtc), seq), out var expected))
{
if (row.RowHash is null || !string.Equals(Convert.ToHexStringLower(row.RowHash), expected, StringComparison.Ordinal))
{
return $"{later.Name} carries a different hash at the stale archive's chain end.";
}
targets.Remove((AuditChainHasher.MonthOf(row.OccurredAtUtc), seq));
}
missing.Remove(row.EventId);
}
}
await using (var input = File.OpenRead(Path.Combine(laterDirectory, later.Name + ".tombstones.parquet")))
{
await foreach (var tombstone in AuditArchiveParquet.ReadTombstonesAsync(input, ct).ConfigureAwait(false))
{
unchained.Remove(tombstone.EventId);
if (targets.TryGetValue((tombstone.ChainMonth, tombstone.ChainSeq), out var expected))
{
if (!string.Equals(Convert.ToHexStringLower(tombstone.RowHash), expected, StringComparison.Ordinal))
{
return $"{later.Name} carries a different hash at the stale archive's chain end.";
}
targets.Remove((tombstone.ChainMonth, tombstone.ChainSeq));
}
}
}
if (targets.Count > 0)
{
return $"{later.Name} does not contain the stale archive's chain end.";
}
return unchained.Count == 0
return missing.Count == 0
? null
: $"{unchained.Count} unchained row(s) of this archive are not in {later.Name} (e.g. removed by a per-channel purge, which leaves no tombstone for an unchained row).";
: $"{missing.Count} row(s) of this archive are not in {later.Name} (e.g. removed by a per-channel purge in between).";
}
private async Task<bool> PartitionHasRowsAsync(DateTime boundary, CancellationToken ct)
{
var conn = _context.Database.GetDbConnection();
var openedHere = await AuditLogHashChain.OpenAsync(conn, ct).ConfigureAwait(false);
var openedHere = await AuditArchiveParquet.OpenAsync(conn, ct).ConfigureAwait(false);
try
{
await using var cmd = conn.CreateCommand();
@@ -354,7 +294,7 @@ public sealed partial class AuditLogPartitionArchive
}
/// <summary>
/// Deletes exactly the archive's three files, manifest first. File names are derived from
/// Deletes exactly the archive's files (a format-v1 archive also has a tombstone file), manifest first. File names are derived from
/// the validated archive name, never taken from the manifest.
/// </summary>
private void DeleteArchiveFiles(AuditArchiveManifest manifest)
@@ -21,8 +21,7 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Maintenance;
/// </summary>
/// <remarks>
/// <para>
/// <b>Layout.</b> <c>{Directory}/{YYYY}/{name}.parquet</c> (rows),
/// <c>{name}.tombstones.parquet</c> (per-channel purge tombstones) and
/// <b>Layout.</b> <c>{Directory}/{YYYY}/{name}.parquet</c> (rows) and
/// <c>{name}.manifest.json</c>, where <c>name</c> is
/// <c>auditlog-{YYYY-MM}-{archivedAt:yyyyMMddTHHmmssfff}Z</c>. Each file is written to a
/// <c>.tmp</c> sibling, flushed to disk and renamed; the manifest goes last, so a manifest
@@ -30,10 +29,9 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Maintenance;
/// new archive, and the superseded one keeps status <see cref="AuditArchiveStatus.Archived"/>.
/// </para>
/// <para>
/// <b>Consistency with the hash chain.</b> The partition is read inside one transaction that
/// holds the chain lock SHARED, so no chaining pass or purge moves a head or writes a
/// tombstone while the rows, heads and tombstones are read. The purge then re-checks the
/// row count and heads under the EXCLUSIVE lock (<see cref="AuditPartitionSwitchGuard"/>).
/// <b>Consistency with the purge.</b> The partition is read inside one transaction; the purge
/// then re-counts the partition inside the switch transaction
/// (<see cref="AuditPartitionSwitchGuard"/>) and purges nothing when the count moved.
/// </para>
/// </remarks>
public sealed partial class AuditLogPartitionArchive : IAuditLogArchive
@@ -86,93 +84,27 @@ public sealed partial class AuditLogPartitionArchive : IAuditLogArchive
}
var boundary = DateTime.SpecifyKind(partitionBoundaryUtc, DateTimeKind.Utc);
var partition = AuditChainHasher.PartitionLabel(AuditChainHasher.MonthOf(boundary));
var partition = AuditArchiveParquet.PartitionLabel(AuditArchiveParquet.MonthOf(boundary));
var archivedAt = _time.GetUtcNow().UtcDateTime;
var name = $"auditlog-{partition}-{archivedAt.ToString("yyyyMMdd'T'HHmmssfff", CultureInfo.InvariantCulture)}Z";
var directory = Path.Combine(_options.Directory!, partition[..4]);
Directory.CreateDirectory(directory);
var dataFile = name + ".parquet";
var tombstoneFile = name + ".tombstones.parquet";
var dataTmp = Path.Combine(directory, dataFile + ".tmp");
var tombstoneTmp = Path.Combine(directory, tombstoneFile + ".tmp");
var chains = new List<AuditArchiveChainState>();
long rowCount = 0, chained = 0;
long rowCount = 0;
DateTime? minOccurred = null, maxOccurred = null;
string dataSha, tombstoneSha;
long dataBytes, tombstoneCount;
string dataSha;
long dataBytes;
try
{
var conn = _context.Database.GetDbConnection();
var openedHere = await AuditLogHashChain.OpenAsync(conn, ct).ConfigureAwait(false);
var openedHere = await AuditArchiveParquet.OpenAsync(conn, ct).ConfigureAwait(false);
try
{
await using var tx = await conn.BeginTransactionAsync(IsolationLevel.ReadCommitted, ct).ConfigureAwait(false);
if (!await AuditLogHashChain.TryAcquireLockAsync(
conn, tx, "Shared", TimeSpan.FromSeconds(Math.Max(1, _options.ChainLockTimeoutSeconds)), ct)
.ConfigureAwait(false))
{
await tx.RollbackAsync(ct).ConfigureAwait(false);
throw new TimeoutException(
$"The AuditLog hash-chain lock stayed busy for {_options.ChainLockTimeoutSeconds}s; partition {partition} not archived this tick.");
}
// 1. Chain heads of every month the partition holds (normally exactly one).
var tombstoneCounts = new Dictionary<DateTime, long>();
var heads = new List<(DateTime Month, long LastSeq, byte[] LastHash, long? PurgedSeq, byte[]? PurgedHash)>();
await using (var cmd = Command(conn, tx,
"SELECT ChainMonth, LastSeq, LastHash, PurgedThroughSeq, PurgedThroughHash FROM dbo.AuditLogChainHead " +
"WHERE $partition.pf_AuditLog_Month(CAST(ChainMonth AS datetime2(7))) = $partition.pf_AuditLog_Month(@boundary) " +
"ORDER BY ChainMonth;", boundary))
{
await using var reader = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false);
while (await reader.ReadAsync(ct).ConfigureAwait(false))
{
heads.Add((
DateTime.SpecifyKind(reader.GetDateTime(0), DateTimeKind.Utc),
reader.GetInt64(1),
(byte[])reader.GetValue(2),
reader.IsDBNull(3) ? null : reader.GetInt64(3),
reader.IsDBNull(4) ? null : (byte[])reader.GetValue(4)));
}
}
// 2. Per-channel purge tombstones (verification bridges gaps with them).
await using (var file = NewFile(tombstoneTmp))
{
var hashing = new HashingWriteStream(file);
await using (var writer = await AuditArchiveParquet
.CreateTombstoneWriterAsync(hashing, partition, _options.RowGroupSize, ct).ConfigureAwait(false))
{
await using var cmd = Command(conn, tx,
"SELECT ChainMonth, ChainSeq, RowHash, EventId, PurgedAtUtc FROM dbo.AuditLogChainTombstone " +
"WHERE $partition.pf_AuditLog_Month(CAST(ChainMonth AS datetime2(7))) = $partition.pf_AuditLog_Month(@boundary) " +
"ORDER BY ChainMonth, ChainSeq;", boundary);
await using var reader = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false);
while (await reader.ReadAsync(ct).ConfigureAwait(false))
{
var tombstone = new AuditArchiveTombstone(
DateTime.SpecifyKind(reader.GetDateTime(0), DateTimeKind.Utc),
reader.GetInt64(1),
(byte[])reader.GetValue(2),
reader.GetGuid(3),
DateTime.SpecifyKind(reader.GetDateTime(4), DateTimeKind.Utc));
tombstoneCounts[tombstone.ChainMonth] = tombstoneCounts.GetValueOrDefault(tombstone.ChainMonth) + 1;
await writer.AddAsync(tombstone, ct).ConfigureAwait(false);
}
tombstoneCount = writer.Count;
}
await hashing.FlushAsync(ct).ConfigureAwait(false);
file.Flush(flushToDisk: true);
tombstoneSha = hashing.HashHex();
}
// 3. The rows, grouped by chain month and in chain order within it (unchained
// rows last), which is the order verification needs.
await using (var file = NewFile(dataTmp))
{
var hashing = new HashingWriteStream(file);
@@ -180,20 +112,14 @@ public sealed partial class AuditLogPartitionArchive : IAuditLogArchive
.CreateDataWriterAsync(hashing, partition, _options.RowGroupSize, ct).ConfigureAwait(false))
{
await using var cmd = Command(conn, tx,
$"SELECT {AuditLogHashChain.RowColumns}, ChainSeq, RowHash FROM dbo.AuditLog " +
$"SELECT {AuditArchiveParquet.RowColumns} FROM dbo.AuditLog " +
"WHERE $PARTITION.pf_AuditLog_Month(OccurredAtUtc) = $partition.pf_AuditLog_Month(@boundary) " +
"ORDER BY DATEFROMPARTS(YEAR(OccurredAtUtc), MONTH(OccurredAtUtc), 1), " +
"CASE WHEN ChainSeq IS NULL THEN 1 ELSE 0 END, ChainSeq, OccurredAtUtc, EventId;", boundary);
"ORDER BY OccurredAtUtc, EventId;", boundary);
cmd.CommandTimeout = Math.Max(1, _options.ReadCommandTimeoutMinutes) * 60;
await using var reader = await cmd.ExecuteReaderAsync(CommandBehavior.SequentialAccess, ct).ConfigureAwait(false);
while (await reader.ReadAsync(ct).ConfigureAwait(false))
{
var row = AuditLogHashChain.ReadRow(reader, withLink: true);
if (row.ChainSeq is not null)
{
chained++;
}
var row = AuditArchiveParquet.ReadRow(reader);
minOccurred = minOccurred is { } lo && lo <= row.OccurredAtUtc ? lo : row.OccurredAtUtc;
maxOccurred = maxOccurred is { } hi && hi >= row.OccurredAtUtc ? hi : row.OccurredAtUtc;
await writer.AddAsync(row, ct).ConfigureAwait(false);
@@ -209,17 +135,6 @@ public sealed partial class AuditLogPartitionArchive : IAuditLogArchive
}
await tx.CommitAsync(ct).ConfigureAwait(false);
foreach (var head in heads)
{
chains.Add(new AuditArchiveChainState(
AuditChainHasher.PartitionLabel(head.Month),
head.LastSeq,
Convert.ToHexStringLower(head.LastHash),
head.PurgedSeq,
head.PurgedHash is null ? null : Convert.ToHexStringLower(head.PurgedHash),
tombstoneCounts.GetValueOrDefault(head.Month)));
}
}
finally
{
@@ -230,12 +145,10 @@ public sealed partial class AuditLogPartitionArchive : IAuditLogArchive
}
File.Move(dataTmp, Path.Combine(directory, dataFile));
File.Move(tombstoneTmp, Path.Combine(directory, tombstoneFile));
}
catch
{
TryDelete(dataTmp);
TryDelete(tombstoneTmp);
throw;
}
@@ -248,22 +161,16 @@ public sealed partial class AuditLogPartitionArchive : IAuditLogArchive
AuditArchiveStatus.Archived,
PurgedAtUtc: null,
rowCount,
chained,
rowCount - chained,
minOccurred,
maxOccurred,
dataFile,
dataSha,
dataBytes,
tombstoneFile,
tombstoneSha,
tombstoneCount,
chains);
dataBytes);
await WriteManifestAsync(directory, manifest, ct).ConfigureAwait(false);
_logger.LogInformation(
"Archived AuditLog partition {Partition} to {Archive}: {Rows} rows ({Chained} chained), {Tombstones} tombstones, {Bytes} bytes.",
partition, name, rowCount, chained, tombstoneCount, dataBytes);
"Archived AuditLog partition {Partition} to {Archive}: {Rows} rows, {Bytes} bytes.",
partition, name, rowCount, dataBytes);
return manifest;
}
@@ -272,9 +179,7 @@ public sealed partial class AuditLogPartitionArchive : IAuditLogArchive
AuditArchiveManifest manifest, TimeSpan? commandTimeout = null, CancellationToken ct = default)
{
ArgumentNullException.ThrowIfNull(manifest);
var guard = new AuditPartitionSwitchGuard(
manifest.RowCount,
manifest.Chains.Select(c => (AuditArchiveParquet.ParseMonth(c.Partition), c.LastSeq)).ToList());
var guard = new AuditPartitionSwitchGuard(manifest.RowCount);
long purged;
try
@@ -283,8 +188,7 @@ public sealed partial class AuditLogPartitionArchive : IAuditLogArchive
.SwitchOutPartitionAsync(manifest.PartitionBoundaryUtc, guard, commandTimeout, ct)
.ConfigureAwait(false);
}
catch (Exception ex) when (FindSqlError(ex) is { } number
&& number is AuditPartitionSwitchGuard.RowCountChangedError or AuditPartitionSwitchGuard.ChainAdvancedError)
catch (Exception ex) when (FindSqlError(ex) is AuditPartitionSwitchGuard.RowCountChangedError)
{
throw new AuditArchiveStaleException(
$"Partition {manifest.Partition} changed after archive {manifest.Name} was written; nothing was purged, "
@@ -302,8 +206,7 @@ public sealed partial class AuditLogPartitionArchive : IAuditLogArchive
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
{
// The rows are gone and the archive files are complete; only the status flag is
// stale. Verify still ties the archive to the purge anchor (anchor Matches).
// The rows are gone and the archive file is complete; only the status flag is stale.
_logger.LogError(ex,
"AuditLog partition {Partition} was purged but archive {Archive}'s manifest could not be marked Purged.",
manifest.Partition, manifest.Name);
@@ -365,85 +268,20 @@ public sealed partial class AuditLogPartitionArchive : IAuditLogArchive
var manifest = await ReadManifestAsync(manifestPath, ct).ConfigureAwait(false);
var problems = new List<string>();
var dataPath = Path.Combine(directory, Path.GetFileName(manifest.DataFile));
var tombstonePath = Path.Combine(directory, Path.GetFileName(manifest.TombstoneFile));
await CheckFileAsync(dataPath, manifest.DataSha256, manifest.DataBytes, problems, ct).ConfigureAwait(false);
await CheckFileAsync(tombstonePath, manifest.TombstoneSha256, null, problems, ct).ConfigureAwait(false);
if (problems.Count > 0)
{
return new AuditArchiveVerificationResult(
manifest.Name, manifest.Partition, manifest.Status, Verified: false, FilesIntact: false,
string.Join(" ", problems), RowsRead: 0, []);
}
// Tombstones per month, then one streaming verifier per chain month.
var tombstones = new Dictionary<DateTime, Dictionary<long, byte[]>>();
long tombstonesRead = 0;
DateTime? chainFrom = null;
await using (var input = File.OpenRead(tombstonePath))
{
await foreach (var t in AuditArchiveParquet.ReadTombstonesAsync(input, ct).ConfigureAwait(false))
{
if (!tombstones.TryGetValue(t.ChainMonth, out var byMonth))
{
tombstones[t.ChainMonth] = byMonth = [];
}
byMonth[t.ChainSeq] = t.RowHash;
tombstonesRead++;
if (t.ChainSeq == AuditChainHasher.CutoffAnchorSeq)
{
// The chain-start anchor (ScadaBridge#139) travels with the archive of its month.
chainFrom = chainFrom is { } seen && seen <= t.PurgedAtUtc ? seen : t.PurgedAtUtc;
}
}
}
// Otherwise the cutoff comes from the database's anchor, which outlives every purge.
chainFrom ??= await ReadChainFromAsync(ct).ConfigureAwait(false);
var states = manifest.Chains.ToDictionary(c => AuditArchiveParquet.ParseMonth(c.Partition));
var verifiers = new SortedDictionary<DateTime, AuditChainVerifier>();
var unchained = new Dictionary<DateTime, long>();
var preCutoff = new Dictionary<DateTime, long>();
AuditChainVerifier VerifierFor(DateTime month)
{
if (!verifiers.TryGetValue(month, out var verifier))
{
AuditChainHeadState? head = states.TryGetValue(month, out var s)
? new AuditChainHeadState(
s.LastSeq,
Convert.FromHexString(s.LastHash),
s.PriorPurgedThroughSeq,
s.PriorPurgedThroughHash is null ? null : Convert.FromHexString(s.PriorPurgedThroughHash))
: null;
verifier = new AuditChainVerifier(month, head, tombstones.GetValueOrDefault(month) ?? [], chainFrom);
verifiers[month] = verifier;
}
return verifier;
}
foreach (var month in states.Keys)
{
VerifierFor(month);
string.Join(" ", problems), RowsRead: 0);
}
long rowsRead = 0;
await using (var input = File.OpenRead(dataPath))
{
await foreach (var row in AuditArchiveParquet.ReadRowsAsync(input, ct).ConfigureAwait(false))
await foreach (var _ in AuditArchiveParquet.ReadRowsAsync(input, ct).ConfigureAwait(false))
{
rowsRead++;
var month = AuditChainHasher.MonthOf(row.OccurredAtUtc);
if (row.ChainSeq is null)
{
var bucket = chainFrom is { } cutoff && row.OccurredAtUtc < cutoff ? preCutoff : unchained;
bucket[month] = bucket.GetValueOrDefault(month) + 1;
VerifierFor(month);
continue;
}
VerifierFor(month).Accept(row);
}
}
@@ -452,126 +290,10 @@ public sealed partial class AuditLogPartitionArchive : IAuditLogArchive
problems.Add($"The data file holds {rowsRead} rows; the manifest records {manifest.RowCount}.");
}
if (tombstonesRead != manifest.TombstoneCount)
{
problems.Add($"The tombstone file holds {tombstonesRead} tombstones; the manifest records {manifest.TombstoneCount}.");
}
var anchors = await ReadAnchorsAsync(verifiers.Keys, ct).ConfigureAwait(false);
var results = new List<AuditArchiveChainVerification>();
foreach (var (month, verifier) in verifiers)
{
var chain = verifier.Complete(unchained.GetValueOrDefault(month), preCutoff.GetValueOrDefault(month));
var (anchor, detail) = CompareAnchor(states.GetValueOrDefault(month), anchors.GetValueOrDefault(month));
results.Add(new AuditArchiveChainVerification(chain, anchor, detail));
}
var filesIntact = problems.Count == 0;
var verified = filesIntact
&& results.All(r => r.Chain.Status != AuditChainStatus.Broken && r.Anchor != AuditArchiveAnchorStatus.Mismatch);
return new AuditArchiveVerificationResult(
manifest.Name, manifest.Partition, manifest.Status, verified, filesIntact,
filesIntact ? null : string.Join(" ", problems), rowsRead, results);
}
/// <summary>
/// Compares an archived chain end with the database's current checkpoint anchor for the month.
/// </summary>
internal static (AuditArchiveAnchorStatus Status, string Detail) CompareAnchor(
AuditArchiveChainState? archived, (bool HeadExists, long? PurgedSeq, byte[]? PurgedHash) database)
{
if (archived is null)
{
// No chain head existed when the rows were archived (chaining off, or the month was
// never reached): nothing to tie. A later anchor can only come from a later purge.
return database.PurgedSeq is null
? (AuditArchiveAnchorStatus.Matches, "No hash chain existed for this month when it was archived.")
: (AuditArchiveAnchorStatus.Superseded,
$"No chain at archive time; a later purge anchored the month at ChainSeq {database.PurgedSeq}.");
}
if (!database.HeadExists)
{
return (AuditArchiveAnchorStatus.Mismatch,
$"The archive records a chain through ChainSeq {archived.LastSeq} but the database has no chain head for {archived.Partition}.");
}
if (database.PurgedSeq is null)
{
return (AuditArchiveAnchorStatus.NotPurged,
$"{archived.Partition} has not been purged: the archived rows are still in dbo.AuditLog.");
}
if (database.PurgedSeq == archived.LastSeq)
{
return database.PurgedHash is not null
&& Convert.ToHexStringLower(database.PurgedHash) == archived.LastHash.ToLowerInvariant()
? (AuditArchiveAnchorStatus.Matches,
$"The archived chain ends at the purge anchor (ChainSeq {archived.LastSeq}).")
: (AuditArchiveAnchorStatus.Mismatch,
$"The purge anchor is at ChainSeq {archived.LastSeq} but its hash differs from the archive's chain end.");
}
return database.PurgedSeq > archived.LastSeq
? (AuditArchiveAnchorStatus.Superseded,
$"The purge anchor (ChainSeq {database.PurgedSeq}) is past this archive's chain end ({archived.LastSeq}): "
+ "late rows were purged and archived again afterwards.")
: (AuditArchiveAnchorStatus.Mismatch,
$"The purge anchor (ChainSeq {database.PurgedSeq}) is before this archive's chain end ({archived.LastSeq}).");
}
/// <summary>The database's chain cutoff (ScadaBridge#139), or null when the chain has none.</summary>
private async Task<DateTime?> ReadChainFromAsync(CancellationToken ct)
{
var conn = _context.Database.GetDbConnection();
var openedHere = await AuditLogHashChain.OpenAsync(conn, ct).ConfigureAwait(false);
try
{
await using var cmd = conn.CreateCommand();
cmd.CommandText = "SELECT MIN(PurgedAtUtc) FROM dbo.AuditLogChainTombstone WHERE ChainSeq = 0;";
return await cmd.ExecuteScalarAsync(ct).ConfigureAwait(false) is DateTime dt
? DateTime.SpecifyKind(dt, DateTimeKind.Utc)
: null;
}
finally
{
if (openedHere)
{
await conn.CloseAsync().ConfigureAwait(false);
}
}
}
private async Task<Dictionary<DateTime, (bool HeadExists, long? PurgedSeq, byte[]? PurgedHash)>> ReadAnchorsAsync(
IEnumerable<DateTime> months, CancellationToken ct)
{
var result = new Dictionary<DateTime, (bool, long?, byte[]?)>();
var conn = _context.Database.GetDbConnection();
var openedHere = await AuditLogHashChain.OpenAsync(conn, ct).ConfigureAwait(false);
try
{
foreach (var month in months)
{
await using var cmd = conn.CreateCommand();
cmd.CommandText = "SELECT PurgedThroughSeq, PurgedThroughHash FROM dbo.AuditLogChainHead WHERE ChainMonth = @month;";
cmd.Parameters.Add(new SqlParameter("@month", SqlDbType.Date) { Value = month.Date });
await using var reader = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false);
result[month] = await reader.ReadAsync(ct).ConfigureAwait(false)
? (true,
reader.IsDBNull(0) ? null : reader.GetInt64(0),
reader.IsDBNull(1) ? null : (byte[])reader.GetValue(1))
: (false, null, null);
}
}
finally
{
if (openedHere)
{
await conn.CloseAsync().ConfigureAwait(false);
}
}
return result;
manifest.Name, manifest.Partition, manifest.Status, filesIntact, filesIntact,
filesIntact ? null : string.Join(" ", problems), rowsRead);
}
private static async Task CheckFileAsync(string path, string expectedSha, long? expectedBytes, List<string> problems, CancellationToken ct)
@@ -1,83 +0,0 @@
using Microsoft.AspNetCore.DataProtection;
using Microsoft.EntityFrameworkCore;
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Maintenance;
/// <summary>
/// The connection the AuditLog maintenance paths run on — the hash-chain job and both purge
/// paths (partition switch-out with its archive, per-channel DELETE) — ScadaBridge#107 item 3.
/// </summary>
/// <remarks>
/// <para>
/// With <c>AuditLog:MaintenanceConnectionString</c> unset (the default), this is the scope's
/// ordinary <see cref="ScadaBridgeDbContext"/>: one login for everything, exactly as before.
/// With it set, the maintenance paths get their OWN context on that connection string, so a
/// DBA can give the runtime (writer) login only <c>scadabridge_audit_writer</c> — whose DENYs
/// stop it writing chain state or deleting audit rows — and a separate maintenance login the
/// <c>scadabridge_audit_chainer</c> + <c>scadabridge_audit_purger</c> roles. The string is read
/// after the pre-host <c>${secret:}</c> expansion, so it may (and in production should) be a
/// secret reference.
/// </para>
/// <para>Scoped: an owned context is disposed with the scope; a borrowed one is left to DI.</para>
/// </remarks>
public sealed class AuditMaintenanceContext : IAsyncDisposable, IDisposable
{
/// <summary>DI key of the maintenance-login <c>IAuditLogRepository</c> and <c>IAuditLogArchive</c>.</summary>
public const string ServiceKey = AuditLogServiceKeys.Maintenance;
private readonly bool _owned;
private AuditMaintenanceContext(ScadaBridgeDbContext context, bool owned)
{
Context = context;
_owned = owned;
}
/// <summary>The context to run maintenance SQL on.</summary>
public ScadaBridgeDbContext Context { get; }
/// <summary>True when <see cref="Context"/> is on the separate maintenance connection string.</summary>
public bool UsesSeparateLogin => _owned;
/// <summary>Wraps the scope's ordinary context (no separate maintenance login configured).</summary>
/// <param name="context">The scope's context.</param>
/// <returns>The wrapper.</returns>
public static AuditMaintenanceContext Shared(ScadaBridgeDbContext context)
=> new(context ?? throw new ArgumentNullException(nameof(context)), owned: false);
/// <summary>Opens an owned context on the maintenance connection string.</summary>
/// <param name="connectionString">The maintenance connection string.</param>
/// <param name="protectionProvider">Data protection for secret-bearing columns (same as the runtime context).</param>
/// <returns>The wrapper; disposing it disposes the context.</returns>
public static AuditMaintenanceContext Separate(string connectionString, IDataProtectionProvider? protectionProvider)
{
ArgumentException.ThrowIfNullOrWhiteSpace(connectionString);
var options = new DbContextOptionsBuilder<ScadaBridgeDbContext>()
.UseSqlServer(connectionString, sql => sql.EnableRetryOnFailure(
maxRetryCount: 5, maxRetryDelay: TimeSpan.FromSeconds(30), errorNumbersToAdd: null))
.ConfigureWarnings(w => w.Ignore(
Microsoft.EntityFrameworkCore.Diagnostics.RelationalEventId.PendingModelChangesWarning))
.Options;
var context = protectionProvider is null
? new ScadaBridgeDbContext(options)
: new ScadaBridgeDbContext(options, protectionProvider);
return new AuditMaintenanceContext(context, owned: true);
}
/// <inheritdoc />
public ValueTask DisposeAsync() => _owned ? Context.DisposeAsync() : ValueTask.CompletedTask;
/// <inheritdoc />
public void Dispose()
{
if (_owned)
{
Context.Dispose();
}
}
}
/// <summary>The configured maintenance connection string (null = use the runtime connection).</summary>
/// <param name="ConnectionString">The connection string, or null.</param>
public sealed record AuditMaintenanceConnection(string? ConnectionString);
@@ -8,49 +8,28 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Maintenance;
/// partition-switch transaction so the purge drops exactly the rows the archive holds.
/// </summary>
/// <param name="ExpectedRows">Rows the archive holds for the partition.</param>
/// <param name="ExpectedHeads">Each chain month's <c>LastSeq</c> at archive time.</param>
public sealed record AuditPartitionSwitchGuard(long ExpectedRows, IReadOnlyList<(DateTime ChainMonth, long LastSeq)> ExpectedHeads)
public sealed record AuditPartitionSwitchGuard(long ExpectedRows)
{
/// <summary>SQL error number THROWn when the partition row count changed.</summary>
public const int RowCountChangedError = 51002;
/// <summary>SQL error number THROWn when a chain head in the partition changed.</summary>
public const int ChainAdvancedError = 51003;
/// <summary>
/// Renders the guard as T-SQL for the switch batch. Every value is a number or a
/// <c>yyyy-MM-dd</c> date produced here, so inlining them as literals is injection-safe and
/// keeps the switch a parameterless <c>ExecuteSqlRaw</c> batch.
/// Renders the guard as T-SQL for the switch batch. The only value is a number produced
/// here, so inlining it as a literal is injection-safe and keeps the switch a
/// parameterless <c>ExecuteSqlRaw</c> batch.
/// </summary>
/// <param name="monthBoundaryLiteral">The boundary literal the switch batch already uses.</param>
/// <returns>The guard statements (run after the chain lock, inside the TRY).</returns>
/// <returns>The guard statements (run inside the switch TRY, before the SWITCH).</returns>
internal string ToSql(string monthBoundaryLiteral)
{
var sql = new StringBuilder();
sql.AppendLine();
sql.AppendLine(" -- 0b. Archive guard (ScadaBridge#42): the archive must hold exactly the rows this");
sql.AppendLine(" -- switch drops. Under the exclusive chain lock, so nothing chains or per-channel");
sql.AppendLine(" -- purges between this check and the SWITCH.");
sql.AppendLine(" -- 0. Archive guard (ScadaBridge#42): the archive must hold exactly the rows this");
sql.AppendLine(" -- switch drops.");
sql.AppendLine(CultureInfo.InvariantCulture, $" DECLARE @guardPartition int = $partition.pf_AuditLog_Month('{monthBoundaryLiteral}');");
sql.AppendLine(" IF (SELECT COUNT_BIG(*) FROM dbo.AuditLog WHERE $PARTITION.pf_AuditLog_Month(OccurredAtUtc) = @guardPartition)");
sql.AppendLine(CultureInfo.InvariantCulture, $" <> {ExpectedRows}");
sql.AppendLine(CultureInfo.InvariantCulture, $" THROW {RowCountChangedError}, 'AuditLog archive is stale: the partition row count changed after it was archived.', 1;");
sql.AppendLine(" DECLARE @guardHeads TABLE (ChainMonth date NOT NULL PRIMARY KEY, LastSeq bigint NOT NULL);");
foreach (var (month, lastSeq) in ExpectedHeads)
{
sql.AppendLine(CultureInfo.InvariantCulture,
$" INSERT INTO @guardHeads (ChainMonth, LastSeq) VALUES ('{month:yyyy-MM-dd}', {lastSeq});");
}
sql.AppendLine(" IF EXISTS (");
sql.AppendLine(" SELECT h.ChainMonth, h.LastSeq FROM dbo.AuditLogChainHead h");
sql.AppendLine(" WHERE $partition.pf_AuditLog_Month(CAST(h.ChainMonth AS datetime2(7))) = @guardPartition");
sql.AppendLine(" EXCEPT SELECT ChainMonth, LastSeq FROM @guardHeads)");
sql.AppendLine(" OR EXISTS (");
sql.AppendLine(" SELECT ChainMonth, LastSeq FROM @guardHeads");
sql.AppendLine(" EXCEPT SELECT h.ChainMonth, h.LastSeq FROM dbo.AuditLogChainHead h");
sql.AppendLine(" WHERE $partition.pf_AuditLog_Month(CAST(h.ChainMonth AS datetime2(7))) = @guardPartition)");
sql.AppendLine(CultureInfo.InvariantCulture, $" THROW {ChainAdvancedError}, 'AuditLog archive is stale: the hash chain advanced after it was archived.', 1;");
return sql.ToString();
}
}
@@ -69,8 +69,11 @@ CREATE NONCLUSTERED INDEX IX_AuditLog_ParentExecution ON dbo.AuditLog (ParentExe
CREATE NONCLUSTERED INDEX IX_AuditLog_Node_Occurred ON dbo.AuditLog (SourceNode, OccurredAtUtc) ON ps_AuditLog_Month(OccurredAtUtc);
CREATE NONCLUSTERED INDEX IX_AuditLog_Kind_Occurred ON dbo.AuditLog (Kind ASC, OccurredAtUtc DESC) ON ps_AuditLog_Month(OccurredAtUtc);
CREATE NONCLUSTERED INDEX IX_AuditLog_Actor_Occurred ON dbo.AuditLog (Actor ASC, OccurredAtUtc DESC) ON ps_AuditLog_Month(OccurredAtUtc);
CREATE NONCLUSTERED INDEX IX_AuditLog_Unchained ON dbo.AuditLog (OccurredAtUtc, EventId) WHERE ChainSeq IS NULL ON ps_AuditLog_Month(OccurredAtUtc);
CREATE NONCLUSTERED INDEX IX_AuditLog_ChainSeq ON dbo.AuditLog (ChainSeq) WHERE ChainSeq IS NOT NULL ON ps_AuditLog_Month(OccurredAtUtc);";
-- The two hash-chain indexes go through EXEC (compiled only when run): RemoveAuditLogHashChain
-- (2026-11-15) drops ChainSeq, and the idempotent script compiles this migration's batch against
-- that schema on a re-run, where a direct reference is Msg 207 (ScadaBridge#126). Same indexes.
EXEC(N'CREATE NONCLUSTERED INDEX IX_AuditLog_Unchained ON dbo.AuditLog (OccurredAtUtc, EventId) WHERE ChainSeq IS NULL ON ps_AuditLog_Month(OccurredAtUtc);');
EXEC(N'CREATE NONCLUSTERED INDEX IX_AuditLog_ChainSeq ON dbo.AuditLog (ChainSeq) WHERE ChainSeq IS NOT NULL ON ps_AuditLog_Month(OccurredAtUtc);');";
/// <summary>Drops every non-clustered index on dbo.AuditLog (the twelve above plus the B2 unique index), each existence-guarded.</summary>
internal const string DropNonClusteredIndexesSql = @"
@@ -0,0 +1,153 @@
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
{
/// <summary>
/// Removes the AuditLog hash chain (ScadaBridge#41, #107, #139) and the two-login maintenance
/// hardening (#113), at IT security's request (2026-11-15), pending a family-wide general
/// solution. Drops what <c>AddAuditLogHashChain</c> and <c>AuditMaintenanceGrants</c> added:
/// <list type="bullet">
/// <item>the two partition-aligned filtered indexes <c>IX_AuditLog_Unchained</c> and
/// <c>IX_AuditLog_ChainSeq</c>;</item>
/// <item>the <c>scadabridge_audit_chainer</c> role (members removed first) and its
/// column-level grant on <c>(ChainSeq, RowHash)</c>;</item>
/// <item><c>dbo.AuditLogChainHead</c> and <c>dbo.AuditLogChainTombstone</c> (their grants
/// and the writer role's DENYs on them go with the tables);</item>
/// <item><c>dbo.AuditLog.ChainSeq</c> and <c>RowHash</c>;</item>
/// <item>the two grants <c>AuditMaintenanceGrants</c> gave <c>scadabridge_audit_purger</c>
/// (<c>ALTER ANY DATASPACE</c>, <c>UPDATE (SourceNode)</c>).</item>
/// </list>
/// <para>
/// <c>scadabridge_audit_purger</c> and <c>scadabridge_audit_writer</c> themselves are KEPT:
/// they predate the chain (<c>AddAuditLogTable</c>, <c>FixAuditPurgerRoleGrants</c>) and are
/// the original append-only design.
/// </para>
/// <para>
/// Every statement is existence-guarded, and every reference to a chain column goes through
/// <c>EXEC</c>: <c>dotnet ef migrations script --idempotent</c> puts a migration in one
/// batch, and SQL Server compiles the batch before any guard runs, so a direct reference to
/// a column that is (or is not yet) there fails with Msg 207 (ScadaBridge#126).
/// </para>
/// <para>
/// <b>Deploy gate.</b> Binaries before this change map <c>ChainSeq</c>/<c>RowHash</c> and read
/// the chain tables, so they cannot run against the migrated schema: apply with both centrals
/// stopped, together with the new binaries. The new binaries purge either schema (the
/// partition-switch staging table copies the chain columns only when the live table has them).
/// </para>
/// </summary>
public partial class RemoveAuditLogHashChain : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.Sql(@"
IF EXISTS (SELECT 1 FROM sys.indexes WHERE name = 'IX_AuditLog_ChainSeq' AND object_id = OBJECT_ID('dbo.AuditLog'))
EXEC(N'DROP INDEX IX_AuditLog_ChainSeq ON dbo.AuditLog;');
IF EXISTS (SELECT 1 FROM sys.indexes WHERE name = 'IX_AuditLog_Unchained' AND object_id = OBJECT_ID('dbo.AuditLog'))
EXEC(N'DROP INDEX IX_AuditLog_Unchained ON dbo.AuditLog;');");
migrationBuilder.Sql(@"
IF DATABASE_PRINCIPAL_ID('scadabridge_audit_chainer') IS NOT NULL
BEGIN
IF COL_LENGTH('dbo.AuditLog', 'ChainSeq') IS NOT NULL
EXEC(N'REVOKE UPDATE (ChainSeq, RowHash) ON dbo.AuditLog FROM scadabridge_audit_chainer;');
EXEC(N'REVOKE SELECT ON dbo.AuditLog FROM scadabridge_audit_chainer;');
DECLARE @dropMembers nvarchar(max) = N'';
SELECT @dropMembers = @dropMembers + N'ALTER ROLE scadabridge_audit_chainer DROP MEMBER ' + QUOTENAME(m.name) + N';' + CHAR(10)
FROM sys.database_role_members rm
JOIN sys.database_principals r ON r.principal_id = rm.role_principal_id
JOIN sys.database_principals m ON m.principal_id = rm.member_principal_id
WHERE r.name = N'scadabridge_audit_chainer';
EXEC sp_executesql @dropMembers;
EXEC(N'DROP ROLE scadabridge_audit_chainer;');
END");
migrationBuilder.Sql(@"
IF OBJECT_ID('dbo.AuditLogChainHead', 'U') IS NOT NULL
EXEC(N'DROP TABLE dbo.AuditLogChainHead;');
IF OBJECT_ID('dbo.AuditLogChainTombstone', 'U') IS NOT NULL
EXEC(N'DROP TABLE dbo.AuditLogChainTombstone;');");
migrationBuilder.Sql(@"
IF COL_LENGTH('dbo.AuditLog', 'ChainSeq') IS NOT NULL
EXEC(N'ALTER TABLE dbo.AuditLog DROP COLUMN ChainSeq;');
IF COL_LENGTH('dbo.AuditLog', 'RowHash') IS NOT NULL
EXEC(N'ALTER TABLE dbo.AuditLog DROP COLUMN RowHash;');");
migrationBuilder.Sql(@"
IF DATABASE_PRINCIPAL_ID('scadabridge_audit_purger') IS NOT NULL
BEGIN
EXEC(N'REVOKE ALTER ANY DATASPACE FROM scadabridge_audit_purger;');
EXEC(N'REVOKE UPDATE (SourceNode) ON dbo.AuditLog FROM scadabridge_audit_purger;');
END");
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
// Recreates exactly what AddAuditLogHashChain and AuditMaintenanceGrants created, so
// the migrations before this one (and their Down methods) find the schema they expect.
// The chain itself is not restored: every row comes back unchained.
migrationBuilder.Sql(@"
IF COL_LENGTH('dbo.AuditLog', 'ChainSeq') IS NULL
EXEC(N'ALTER TABLE dbo.AuditLog ADD ChainSeq bigint NULL;');
IF COL_LENGTH('dbo.AuditLog', 'RowHash') IS NULL
EXEC(N'ALTER TABLE dbo.AuditLog ADD RowHash binary(32) NULL;');");
migrationBuilder.Sql(@"
IF OBJECT_ID('dbo.AuditLogChainHead', 'U') IS NULL
EXEC(N'CREATE TABLE dbo.AuditLogChainHead (
ChainMonth date NOT NULL,
LastSeq bigint NOT NULL,
LastHash binary(32) NOT NULL,
PurgedThroughSeq bigint NULL,
PurgedThroughHash binary(32) NULL,
UpdatedAtUtc datetime2 NOT NULL,
CONSTRAINT PK_AuditLogChainHead PRIMARY KEY (ChainMonth));');
IF OBJECT_ID('dbo.AuditLogChainTombstone', 'U') IS NULL
EXEC(N'CREATE TABLE dbo.AuditLogChainTombstone (
ChainMonth date NOT NULL,
ChainSeq bigint NOT NULL,
RowHash binary(32) NOT NULL,
EventId uniqueidentifier NOT NULL,
PurgedAtUtc datetime2 NOT NULL,
CONSTRAINT PK_AuditLogChainTombstone PRIMARY KEY (ChainMonth, ChainSeq));');");
migrationBuilder.Sql(@"
IF NOT EXISTS (SELECT 1 FROM sys.indexes WHERE name = 'IX_AuditLog_Unchained' AND object_id = OBJECT_ID('dbo.AuditLog'))
EXEC(N'CREATE NONCLUSTERED INDEX IX_AuditLog_Unchained
ON dbo.AuditLog (OccurredAtUtc, EventId)
WHERE ChainSeq IS NULL
ON ps_AuditLog_Month(OccurredAtUtc);');
IF NOT EXISTS (SELECT 1 FROM sys.indexes WHERE name = 'IX_AuditLog_ChainSeq' AND object_id = OBJECT_ID('dbo.AuditLog'))
EXEC(N'CREATE NONCLUSTERED INDEX IX_AuditLog_ChainSeq
ON dbo.AuditLog (ChainSeq)
WHERE ChainSeq IS NOT NULL
ON ps_AuditLog_Month(OccurredAtUtc);');");
migrationBuilder.Sql(@"
IF DATABASE_PRINCIPAL_ID('scadabridge_audit_chainer') IS NULL
EXEC sp_executesql N'CREATE ROLE scadabridge_audit_chainer';
EXEC(N'GRANT SELECT ON dbo.AuditLog TO scadabridge_audit_chainer;
GRANT UPDATE (ChainSeq, RowHash) ON dbo.AuditLog TO scadabridge_audit_chainer;
GRANT SELECT, INSERT, UPDATE ON dbo.AuditLogChainHead TO scadabridge_audit_chainer;
GRANT SELECT ON dbo.AuditLogChainTombstone TO scadabridge_audit_chainer;');
IF DATABASE_PRINCIPAL_ID('scadabridge_audit_purger') IS NULL
EXEC sp_executesql N'CREATE ROLE scadabridge_audit_purger';
EXEC(N'GRANT SELECT, UPDATE ON dbo.AuditLogChainHead TO scadabridge_audit_purger;
GRANT SELECT, INSERT, DELETE ON dbo.AuditLogChainTombstone TO scadabridge_audit_purger;
GRANT ALTER ANY DATASPACE TO scadabridge_audit_purger;
GRANT UPDATE (SourceNode) ON dbo.AuditLog TO scadabridge_audit_purger;');
IF DATABASE_PRINCIPAL_ID('scadabridge_audit_writer') IS NULL
EXEC sp_executesql N'CREATE ROLE scadabridge_audit_writer';
EXEC(N'DENY INSERT, UPDATE, DELETE ON dbo.AuditLogChainHead TO scadabridge_audit_writer;
DENY INSERT, UPDATE, DELETE ON dbo.AuditLogChainTombstone TO scadabridge_audit_writer;');");
}
}
}
@@ -1922,61 +1922,6 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
b.ToTable("BundleUnlockAttempt", (string)null);
});
modelBuilder.Entity("ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Entities.AuditLogChainHead", b =>
{
b.Property<DateTime>("ChainMonth")
.HasColumnType("date");
b.Property<byte[]>("LastHash")
.IsRequired()
.HasMaxLength(32)
.HasColumnType("binary(32)")
.IsFixedLength();
b.Property<long>("LastSeq")
.HasColumnType("bigint");
b.Property<byte[]>("PurgedThroughHash")
.HasMaxLength(32)
.HasColumnType("binary(32)")
.IsFixedLength();
b.Property<long?>("PurgedThroughSeq")
.HasColumnType("bigint");
b.Property<DateTime>("UpdatedAtUtc")
.HasColumnType("datetime2");
b.HasKey("ChainMonth");
b.ToTable("AuditLogChainHead", (string)null);
});
modelBuilder.Entity("ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Entities.AuditLogChainTombstone", b =>
{
b.Property<DateTime>("ChainMonth")
.HasColumnType("date");
b.Property<long>("ChainSeq")
.HasColumnType("bigint");
b.Property<Guid>("EventId")
.HasColumnType("uniqueidentifier");
b.Property<DateTime>("PurgedAtUtc")
.HasColumnType("datetime2");
b.Property<byte[]>("RowHash")
.IsRequired()
.HasMaxLength(32)
.HasColumnType("binary(32)")
.IsFixedLength();
b.HasKey("ChainMonth", "ChainSeq");
b.ToTable("AuditLogChainTombstone", (string)null);
});
modelBuilder.Entity("ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Entities.AuditLogRow", b =>
{
b.Property<DateTime>("OccurredAtUtc")
@@ -1995,9 +1940,6 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
.HasMaxLength(256)
.HasColumnType("nvarchar(256)");
b.Property<long?>("ChainSeq")
.HasColumnType("bigint");
b.Property<string>("Channel")
.IsRequired()
.HasMaxLength(32)
@@ -2040,11 +1982,6 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
.HasColumnType("uniqueidentifier")
.HasComputedColumnSql("CAST(JSON_VALUE(DetailsJson,'$.parentExecutionId') AS uniqueidentifier)", true);
b.Property<byte[]>("RowHash")
.HasMaxLength(32)
.HasColumnType("binary(32)")
.IsFixedLength();
b.Property<string>("SourceNode")
.HasMaxLength(64)
.IsUnicode(false)
@@ -2071,10 +2008,6 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
b.HasKey("OccurredAtUtc", "EventId");
b.HasIndex("ChainSeq")
.HasDatabaseName("IX_AuditLog_ChainSeq")
.HasFilter("[ChainSeq] IS NOT NULL");
b.HasIndex("CorrelationId")
.HasDatabaseName("IX_AuditLog_CorrelationId")
.HasFilter("[CorrelationId] IS NOT NULL");
@@ -2103,10 +2036,6 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
.IsDescending(false, true)
.HasDatabaseName("IX_AuditLog_Kind_Occurred");
b.HasIndex("OccurredAtUtc", "EventId")
.HasDatabaseName("IX_AuditLog_Unchained")
.HasFilter("[ChainSeq] IS NULL");
b.HasIndex("SourceNode", "OccurredAtUtc")
.HasDatabaseName("IX_AuditLog_Node_Occurred");
@@ -49,22 +49,6 @@ public class AuditLogRepository : IAuditLogRepository
private const string CanonicalColumnList =
"EventId, OccurredAtUtc, Actor, Action, Outcome, Category, Target, SourceNode, CorrelationId, DetailsJson";
// Hash chain (ScadaBridge#41). Wraps a maintenance-path batch in a transaction holding
// the EXCLUSIVE chain application lock (Maintenance.AuditChainLock), so it never
// interleaves with a chaining pass or a verification. NOCOUNT is ON for everything but
// the one mutating statement the caller places between prefix and suffix, so
// ExecuteNonQuery still returns exactly that statement's row count. XACT_ABORT rolls
// the transaction back on any error or client timeout (attention).
private const string ChainLockedBatchPrefix =
"SET XACT_ABORT ON;\nSET NOCOUNT ON;\nBEGIN TRANSACTION;\n" +
"DECLARE @chainLock int;\n" +
"EXEC @chainLock = sp_getapplock @Resource = N'" + Maintenance.AuditChainLock.Resource + "', @LockMode = 'Exclusive', @LockOwner = 'Transaction', @LockTimeout = -1;\n" +
"IF @chainLock < 0 THROW 51000, 'AuditLog hash-chain lock was not granted.', 1;\n" +
"SET NOCOUNT OFF;\n";
private const string ChainLockedBatchSuffix =
"\nSET NOCOUNT ON;\nCOMMIT TRANSACTION;";
private readonly ScadaBridgeDbContext _context;
private readonly ILogger<AuditLogRepository> _logger;
@@ -494,11 +478,10 @@ VALUES
/// <summary>
/// The partition switch-out, optionally guarded by an archive's expectations
/// (ScadaBridge#42). With a <paramref name="guard"/>, the switch transaction — after it
/// holds the exclusive chain lock — re-counts the partition and re-reads its months'
/// chain heads, and THROWs (error 51002 / 51003, nothing purged) when either differs
/// from what the archive captured. That is what makes "archive, then purge" safe
/// against a row arriving, a per-channel purge or a chaining pass in between.
/// (ScadaBridge#42). With a <paramref name="guard"/>, the switch transaction re-counts
/// the partition and THROWs (error 51002, nothing purged) when the count differs from
/// what the archive captured, so "archive, then purge" never drops a row the archive
/// does not hold.
/// </summary>
/// <param name="monthBoundary">The partition's lower boundary.</param>
/// <param name="guard">Archive expectations, or null for the unguarded purge.</param>
@@ -556,21 +539,15 @@ VALUES
WHERE $PARTITION.pf_AuditLog_Month(OccurredAtUtc) =
$partition.pf_AuditLog_Month('{monthBoundaryStr}');";
// ScadaBridge#42 archive guard (empty for the unguarded purge). Runs under the chain
// lock, so no chaining pass or per-channel purge can move the partition between the
// check and the SWITCH; an INSERT racing it is caught by the next tick's archive.
// ScadaBridge#42 archive guard (empty for the unguarded purge). The per-channel purge
// and this switch run on the same purge actor, one after the other, so nothing else
// removes rows between the check and the SWITCH; an INSERT racing it is caught by the
// next tick's archive.
var guardSql = guard is null ? string.Empty : guard.ToSql(monthBoundaryStr);
var sql = $@"
BEGIN TRY
BEGIN TRANSACTION;
-- 0. Hash chain (ScadaBridge#41): hold the exclusive chain lock for the whole
-- switch so no chaining pass / verification interleaves with the anchor
-- update in step 3b.
DECLARE @chainLock int;
EXEC @chainLock = sp_getapplock @Resource = N'{Maintenance.AuditChainLock.Resource}', @LockMode = 'Exclusive', @LockOwner = 'Transaction', @LockTimeout = -1;
IF @chainLock < 0 THROW 51000, 'AuditLog hash-chain lock was not granted.', 1;
{guardSql}
-- 1. Defensive cleanup for databases created before
-- AlignAuditLogEventIdUniqueness: the migration drops the
@@ -588,8 +565,7 @@ VALUES
-- expression text + PERSISTED) or ALTER TABLE ... SWITCH PARTITION
-- rejects the operation with msg 4904/4948. The ordinal order also
-- matches dbo.AuditLog_v2 (the CollapseAuditLogToCanonical migration):
-- 10 canonical columns first, then the 6 computed columns, then the two
-- hash-chain columns AddAuditLogHashChain appended (ScadaBridge#41).
-- 10 canonical columns first, then the 6 computed columns.
CREATE TABLE dbo.[{stagingTableName}] (
EventId uniqueidentifier NOT NULL,
OccurredAtUtc datetime2(7) NOT NULL,
@@ -606,11 +582,15 @@ VALUES
SourceSiteId AS JSON_VALUE(DetailsJson,'$.sourceSiteId') PERSISTED,
ExecutionId AS CAST(JSON_VALUE(DetailsJson,'$.executionId') AS uniqueidentifier) PERSISTED,
ParentExecutionId AS CAST(JSON_VALUE(DetailsJson,'$.parentExecutionId') AS uniqueidentifier) PERSISTED,
IngestedAtUtc AS CAST(SWITCHOFFSET(CAST(JSON_VALUE(DetailsJson,'$.ingestedAtUtc') AS datetimeoffset), 0) AS datetime2(7)),
ChainSeq bigint NULL,
RowHash binary(32) NULL
IngestedAtUtc AS CAST(SWITCHOFFSET(CAST(JSON_VALUE(DetailsJson,'$.ingestedAtUtc') AS datetimeoffset), 0) AS datetime2(7))
) ON [PRIMARY];
-- 2a. A database that has not yet run RemoveAuditLogHashChain still carries the two
-- columns AddAuditLogHashChain appended; SWITCH needs them on staging too. Read
-- the live table instead of assuming, so this binary purges either schema.
IF COL_LENGTH('dbo.AuditLog', 'ChainSeq') IS NOT NULL
EXEC(N'ALTER TABLE dbo.[{stagingTableName}] ADD ChainSeq bigint NULL, RowHash binary(32) NULL;');
-- 2b. The staging clustered key must match dbo.AuditLog's or SWITCH refuses
-- (msg 4913/4947). FlipAuditLogClusteredKey (ScadaBridge#111) changed it from
-- (EventId, OccurredAtUtc) to (OccurredAtUtc, EventId), so read the live key
@@ -631,19 +611,6 @@ VALUES
DECLARE @sql nvarchar(max) = 'ALTER TABLE dbo.AuditLog SWITCH PARTITION ' + CAST(@partitionNumber AS nvarchar(10)) + ' TO dbo.[{stagingTableName}];';
EXEC sp_executesql @sql;
-- 3b. Hash chain (ScadaBridge#41): the months this partition held are gone as a
-- whole, so record each month's current chain end as its checkpoint anchor
-- (verification resumes after it; a late-arriving row chains on from it)
-- and drop the per-channel tombstones the anchor now subsumes. The chain-start
-- anchor (ChainSeq 0, ScadaBridge#139) is NOT a purge tombstone: it records the
-- cutoff for the life of the database, so it stays.
UPDATE dbo.AuditLogChainHead
SET PurgedThroughSeq = LastSeq, PurgedThroughHash = LastHash, UpdatedAtUtc = SYSUTCDATETIME()
WHERE $partition.pf_AuditLog_Month(CAST(ChainMonth AS datetime2(7))) = @partitionNumber;
DELETE FROM dbo.AuditLogChainTombstone
WHERE $partition.pf_AuditLog_Month(CAST(ChainMonth AS datetime2(7))) = @partitionNumber
AND ChainSeq > 0;
-- 4. Drop staging — the rows are discarded here. This is the purge.
DROP TABLE dbo.[{stagingTableName}];
@@ -771,22 +738,8 @@ VALUES
// The trailing AUDIT-PURGE-ALLOWED marker on the DELETE line below is the
// single narrow exemption the append-only CI guard (AuditLogAppendOnlyGuardTests)
// recognizes; any other UPDATE/DELETE targeting AuditLog still trips the guard.
//
// Hash chain (ScadaBridge#41): a row deleted from the middle of a month's chain must
// leave its link behind or verification reports the gap as tampering. The DELETE
// OUTPUTs every removed row into a table variable and the same transaction writes a
// tombstone (month, ChainSeq, RowHash) for each CHAINED one; the whole batch runs
// under the exclusive chain lock so a chaining pass or a verification never sees the
// row gone without its tombstone. Rows not yet chained need no tombstone.
const string deleteBatchSql =
ChainLockedBatchPrefix +
"DECLARE @purged TABLE (EventId uniqueidentifier NOT NULL, OccurredAtUtc datetime2(7) NOT NULL, ChainSeq bigint NULL, RowHash binary(32) NULL);\n" +
"DELETE TOP (@batch) FROM dbo.AuditLog OUTPUT deleted.EventId, deleted.OccurredAtUtc, deleted.ChainSeq, deleted.RowHash INTO @purged WHERE Category = @channel AND OccurredAtUtc < @threshold;\n" + // AUDIT-PURGE-ALLOWED: per-channel retention override, maintenance path
"SET NOCOUNT ON;\n" +
"INSERT INTO dbo.AuditLogChainTombstone (ChainMonth, ChainSeq, RowHash, EventId, PurgedAtUtc)\n" +
"SELECT DATEFROMPARTS(YEAR(OccurredAtUtc), MONTH(OccurredAtUtc), 1), ChainSeq, RowHash, EventId, SYSUTCDATETIME()\n" +
"FROM @purged WHERE ChainSeq IS NOT NULL AND RowHash IS NOT NULL;\n" +
ChainLockedBatchSuffix;
"DELETE TOP (@batch) FROM dbo.AuditLog WHERE Category = @channel AND OccurredAtUtc < @threshold;"; // AUDIT-PURGE-ALLOWED: per-channel retention override, maintenance path
long totalDeleted = 0;
@@ -1462,16 +1415,8 @@ VALUES
// The trailing AUDIT-PURGE-ALLOWED marker on the UPDATE line below is the
// single narrow exemption the append-only CI guard (AuditLogAppendOnlyGuardTests)
// recognises for an UPDATE; any other UPDATE targeting AuditLog still trips the guard.
//
// Hash chain (ScadaBridge#41): SourceNode is inside every chained row's hash, so the
// backfill touches ONLY unchained rows (ChainSeq IS NULL) and runs under the
// exclusive chain lock — a chaining pass can never hash a row whose SourceNode this
// batch is about to change. Run the backfill BEFORE enabling AuditLog:HashChain;
// once a row is chained its SourceNode is frozen.
const string updateBatchSql =
ChainLockedBatchPrefix +
"UPDATE TOP (@batch) dbo.AuditLog SET SourceNode = @sentinel WHERE SourceNode IS NULL AND OccurredAtUtc < @before AND ChainSeq IS NULL;" + // AUDIT-PURGE-ALLOWED: SourceNode sentinel backfill, maintenance path
ChainLockedBatchSuffix;
"UPDATE TOP (@batch) dbo.AuditLog SET SourceNode = @sentinel WHERE SourceNode IS NULL AND OccurredAtUtc < @before;"; // AUDIT-PURGE-ALLOWED: SourceNode sentinel backfill, maintenance path
long totalUpdated = 0;
@@ -139,10 +139,6 @@ public class ScadaBridgeDbContext : DbContext, IDataProtectionKeyContext
public DbSet<AuditLogEntry> AuditLogEntries => Set<AuditLogEntry>();
/// <summary>Gets the set of audit log rows (central <c>dbo.AuditLog</c> persistence shape; mapped to/from the canonical record at the repository boundary).</summary>
public DbSet<AuditLogRow> AuditLogs => Set<AuditLogRow>();
/// <summary>Gets the per-month AuditLog hash-chain heads (ScadaBridge#41).</summary>
public DbSet<AuditLogChainHead> AuditLogChainHeads => Set<AuditLogChainHead>();
/// <summary>Gets the AuditLog hash-chain purge tombstones (ScadaBridge#41).</summary>
public DbSet<AuditLogChainTombstone> AuditLogChainTombstones => Set<AuditLogChainTombstone>();
/// <summary>Gets the set of site calls.</summary>
public DbSet<SiteCall> SiteCalls => Set<SiteCall>();
@@ -20,14 +20,9 @@ public static class ServiceCollectionExtensions
/// </summary>
/// <param name="services">The service collection to register into.</param>
/// <param name="connectionString">SQL Server connection string for the central configuration database.</param>
/// <param name="auditMaintenanceConnectionString">
/// Optional separate connection string for the AuditLog maintenance paths — hash-chain job
/// and both purge paths (<c>AuditLog:MaintenanceConnectionString</c>, ScadaBridge#107 item 3).
/// Null or blank runs them on <paramref name="connectionString"/> as before.
/// </param>
/// <returns>The same <paramref name="services"/> collection, for chaining.</returns>
public static IServiceCollection AddConfigurationDatabase(
this IServiceCollection services, string connectionString, string? auditMaintenanceConnectionString = null)
this IServiceCollection services, string connectionString)
{
// The DbContext is constructed via the (options, IDataProtectionProvider) overload so
// secret-bearing configuration columns are encrypted at rest. AddDataProtection below
@@ -115,37 +110,7 @@ public static class ServiceCollectionExtensions
// hosted service. Scoped because the implementation reuses the per-scope
// ScadaBridgeDbContext for raw-SQL execution; the hosted service opens a
// fresh scope on each tick (mirrors AuditLogPurgeActor / AuditLogIngestActor).
// Runs on the maintenance connection when AuditLog:MaintenanceConnectionString is set
// (ScadaBridge#113), else on the scope's own context as before.
services.AddScoped<IPartitionMaintenance>(sp => new AuditLogPartitionMaintenance(
sp.GetRequiredService<AuditMaintenanceContext>().Context,
sp.GetService<Microsoft.Extensions.Logging.ILogger<AuditLogPartitionMaintenance>>()));
// AuditLog maintenance connection (ScadaBridge#107 item 3): the chain job and both purge
// paths run on AuditMaintenanceContext — the scope's own context unless a separate
// maintenance login is configured, in which case an owned context on that string.
services.AddSingleton(new AuditMaintenanceConnection(
string.IsNullOrWhiteSpace(auditMaintenanceConnectionString) ? null : auditMaintenanceConnectionString));
services.AddScoped(sp =>
{
var maintenance = sp.GetRequiredService<AuditMaintenanceConnection>().ConnectionString;
return maintenance is null
? AuditMaintenanceContext.Shared(sp.GetRequiredService<ScadaBridgeDbContext>())
: AuditMaintenanceContext.Separate(maintenance, sp.GetService<IDataProtectionProvider>());
});
services.AddKeyedScoped<IAuditLogRepository>(AuditMaintenanceContext.ServiceKey, (sp, _) => new AuditLogRepository(
sp.GetRequiredService<AuditMaintenanceContext>().Context,
sp.GetService<Microsoft.Extensions.Logging.ILogger<AuditLogRepository>>()));
// AuditLog hash-chain tamper evidence (ScadaBridge#41): chained by the central
// AuditLogHashChainService, verified by the VerifyAuditChain management command; the
// same instance reads/verifies the off-box head export (#107 item 1). Both on the
// maintenance connection.
services.AddScoped(sp => new AuditLogHashChain(
sp.GetRequiredService<AuditMaintenanceContext>().Context,
sp.GetService<Microsoft.Extensions.Logging.ILogger<AuditLogHashChain>>()));
services.AddScoped<IAuditLogHashChain>(sp => sp.GetRequiredService<AuditLogHashChain>());
services.AddScoped<IAuditChainHeadStore>(sp => sp.GetRequiredService<AuditLogHashChain>());
services.AddScoped<IPartitionMaintenance, AuditLogPartitionMaintenance>();
// AuditLog Parquet cold archive (ScadaBridge#42): written by the purge actor before a
// partition switch, listed/verified by the ListAuditArchives / VerifyAuditArchive
@@ -155,12 +120,6 @@ public static class ServiceCollectionExtensions
sp.GetRequiredService<ScadaBridgeDbContext>(),
sp.GetService<Microsoft.Extensions.Options.IOptions<AuditLogArchiveOptions>>()?.Value ?? new AuditLogArchiveOptions(),
sp.GetService<Microsoft.Extensions.Logging.ILogger<AuditLogPartitionArchive>>()));
// The purge actor's archive-then-purge runs on the maintenance connection (#107 item 3);
// list/verify from the management surface stay on the runtime connection (reads only).
services.AddKeyedScoped<IAuditLogArchive>(AuditMaintenanceContext.ServiceKey, (sp, _) => new AuditLogPartitionArchive(
sp.GetRequiredService<AuditMaintenanceContext>().Context,
sp.GetService<Microsoft.Extensions.Options.IOptions<AuditLogArchiveOptions>>()?.Value ?? new AuditLogArchiveOptions(),
sp.GetService<Microsoft.Extensions.Logging.ILogger<AuditLogPartitionArchive>>()));
// Opt-in config-audit retention (ScadaBridge:ConfigAudit:RetentionDays, default 0 = off).
// Options binding to configuration is done by the host; without it the defaults apply
@@ -1,63 +0,0 @@
using Microsoft.Extensions.Diagnostics.HealthChecks;
using ZB.MOM.WW.ScadaBridge.AuditLog.Central.HeadExport;
namespace ZB.MOM.WW.ScadaBridge.Host.Health;
/// <summary>
/// /health/ready entry (ScadaBridge#115): Degraded while the off-box AuditLog chain-head export
/// (<c>AuditLog:HashChain:HeadExport</c>) is failing, or has had no success for more than two
/// intervals. Degraded maps to HTTP 200, so it shows on the Overview dashboard without taking
/// the node out of rotation. Healthy when the export is disabled.
/// </summary>
public sealed class AuditChainHeadExportHealthCheck : IHealthCheck
{
private readonly AuditChainHeadExportStatus _status;
private readonly TimeProvider _time;
/// <summary>Initializes the check.</summary>
/// <param name="status">The export status singleton.</param>
/// <param name="time">Clock (defaults to the system clock).</param>
public AuditChainHeadExportHealthCheck(AuditChainHeadExportStatus status, TimeProvider? time = null)
{
_status = status ?? throw new ArgumentNullException(nameof(status));
_time = time ?? TimeProvider.System;
}
/// <inheritdoc />
public Task<HealthCheckResult> CheckHealthAsync(
HealthCheckContext context, CancellationToken cancellationToken = default)
{
if (!_status.Enabled)
{
return Task.FromResult(HealthCheckResult.Healthy("Chain-head export disabled."));
}
var now = _time.GetUtcNow().UtcDateTime;
var data = new Dictionary<string, object>(StringComparer.Ordinal)
{
["totalFailures"] = _status.TotalFailures,
["consecutiveFailures"] = _status.ConsecutiveFailures,
["intervalMinutes"] = _status.IntervalMinutes,
};
if (_status.LastSuccessUtc is { } ok)
{
data["lastSuccessUtc"] = ok;
}
if (_status.ConsecutiveFailures > 0)
{
return Task.FromResult(HealthCheckResult.Degraded(
$"Chain-head export failing ({_status.ConsecutiveFailures} consecutive): {_status.LastError}", data: data));
}
var limit = TimeSpan.FromMinutes(2L * _status.IntervalMinutes);
var reference = _status.LastSuccessUtc ?? _status.StartedUtc;
if (reference is { } r && now - r > limit)
{
return Task.FromResult(HealthCheckResult.Degraded(
$"No successful chain-head export for more than 2 intervals (last {(_status.LastSuccessUtc is null ? "never" : _status.LastSuccessUtc.Value.ToString("O"))}).", data: data));
}
return Task.FromResult(HealthCheckResult.Healthy("Chain-head export OK.", data));
}
}
+1 -12
View File
@@ -541,12 +541,7 @@ try
var configDbConnectionString = configuration["ScadaBridge:Database:ConfigurationDb"]
?? throw new InvalidOperationException("ScadaBridge:Database:ConfigurationDb connection string is required for Central role.");
// Optional second login for the AuditLog maintenance paths (chain job + both purge
// paths, ScadaBridge#107 item 3). Read from the pre-host-expanded configuration, so it
// may be a ${secret:} reference; unset = everything on the runtime connection, as before.
builder.Services.AddConfigurationDatabase(
configDbConnectionString,
configuration[ZB.MOM.WW.ScadaBridge.AuditLog.ServiceCollectionExtensions.MaintenanceConnectionStringKey]);
builder.Services.AddConfigurationDatabase(configDbConnectionString);
builder.Services.AddOptions<ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Maintenance.ConfigAuditOptions>()
.Bind(configuration.GetSection(ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Maintenance.ConfigAuditOptions.SectionName))
.ValidateOnStart();
@@ -589,12 +584,6 @@ try
"bundle-signing-policy",
failureStatus: null,
tags: new[] { ZbHealthTags.Ready })
// ScadaBridge#115: Degraded (HTTP 200) while the off-box chain-head export is failing
// or stale; healthy when the export is disabled.
.AddTypeActivatedCheck<AuditChainHeadExportHealthCheck>(
"audit-chain-head-export",
failureStatus: null,
tags: new[] { ZbHealthTags.Ready })
// Oldest-Up-member check (review 01 [High]), unscoped: every central member competes for
// the one active slot. Since ZB.MOM.WW.Health 0.3.0 this IS the shared check — it selects
// by age, so the host-local OldestNodeActiveHealthCheck that existed only to avoid the old
@@ -426,10 +426,7 @@ public static class AuditEndpoints
statusCode: 400);
}
// Maintenance login when AuditLog:MaintenanceConnectionString is set (ScadaBridge#113);
// otherwise the keyed registration resolves to the runtime connection's context.
var repo = context.RequestServices.GetKeyedService<IAuditLogRepository>(AuditLogServiceKeys.Maintenance)
?? context.RequestServices.GetRequiredService<IAuditLogRepository>();
var repo = context.RequestServices.GetRequiredService<IAuditLogRepository>();
var rowsUpdated = await repo.BackfillSourceNodeAsync(sentinel, beforeUtc.Value, batchSize, context.RequestAborted);
return Results.Json(new
@@ -246,7 +246,7 @@ public class ManagementActor : ReceiveActor
// three Live run commands stay DeployerOnly.
private static readonly string[] FlowDebuggerReplayRoles = AuthorizationPolicies.FlowDebuggerAccessRoles;
// ScadaBridge#41: verify-chain is an audit read — the OperationalAudit role set.
// AuditLog archive list/verify are audit reads — the OperationalAudit role set.
private static readonly string[] OperationalAuditReaders = AuthorizationPolicies.OperationalAuditRoles;
// arch-review C6: area management is reconciled to any-of [Designer, Deployer].
@@ -387,16 +387,8 @@ public class ManagementActor : ReceiveActor
// (arch-review UA1).
ListSecuredWritesCommand => SecuredWriteReaders,
// AuditLog hash-chain verification (ScadaBridge#41): an audit read, so the
// OperationalAudit roles (alog.md §15.1 — verify-chain carries audit query's
// permission). A site-scoped caller is refused in the handler: a month's chain
// spans every site.
VerifyAuditChainCommand => OperationalAuditReaders,
// Off-box chain-head check (ScadaBridge#107 item 1): the same audit read.
VerifyAuditChainHeadsCommand => OperationalAuditReaders,
// AuditLog Parquet archives (ScadaBridge#42): audit reads with verify-chain's gate —
// OperationalAudit roles, site-scoped callers refused in the handler.
// AuditLog Parquet archives (ScadaBridge#42): audit reads — OperationalAudit roles
// (alog.md §15.1, audit query's permission), site-scoped callers refused in the handler.
ListAuditArchivesCommand => OperationalAuditReaders,
// Archive cleanup deletes archive files (ScadaBridge#109): Administrator only.
CleanupAuditArchivesCommand => AdminOnly,
@@ -572,8 +564,6 @@ public class ManagementActor : ReceiveActor
// Audit Log
QueryAuditLogCommand cmd => await HandleQueryAuditLog(sp, cmd),
VerifyAuditChainCommand cmd => await HandleVerifyAuditChain(sp, cmd, user),
VerifyAuditChainHeadsCommand cmd => await HandleVerifyAuditChainHeads(sp, cmd, user),
ListAuditArchivesCommand => await HandleListAuditArchives(sp, user),
VerifyAuditArchiveCommand cmd => await HandleVerifyAuditArchive(sp, cmd, user),
CleanupAuditArchivesCommand cmd => await HandleCleanupAuditArchives(sp, cmd, user),
@@ -3230,42 +3220,6 @@ public class ManagementActor : ReceiveActor
return new AuditLogQueryResult(entries, totalCount);
}
/// <summary>
/// Verifies one month of the AuditLog hash chain (ScadaBridge#41). The month must be
/// <c>YYYY-MM</c>; a site-scoped, non-Administrator caller is refused because the chain
/// covers every site's rows and a break report names rows by EventId.
/// </summary>
private static async Task<object?> HandleVerifyAuditChain(
IServiceProvider sp, VerifyAuditChainCommand cmd, AuthenticatedUser user)
{
if (string.IsNullOrWhiteSpace(cmd.Month)
|| !DateTime.TryParseExact(cmd.Month, "yyyy-MM", System.Globalization.CultureInfo.InvariantCulture,
System.Globalization.DateTimeStyles.None, out var month))
{
throw new ManagementCommandException(
$"Invalid month '{cmd.Month}'. Expected YYYY-MM (e.g. 2026-05).", "INVALID_ARGUMENT");
}
if (user.PermittedSiteIds.Length > 0
&& !user.Roles.Contains(Roles.Administrator, StringComparer.OrdinalIgnoreCase))
{
throw new SiteScopeViolationException(
"Access denied: the audit hash chain spans every site; verify-chain needs an "
+ "OperationalAudit role that is not scoped to specific sites.");
}
var chain = sp.GetRequiredService<IAuditLogHashChain>();
try
{
return await chain.VerifyMonthAsync(
new DateTime(month.Year, month.Month, 1, 0, 0, 0, DateTimeKind.Utc));
}
catch (TimeoutException ex)
{
throw new ManagementCommandException(ex.Message, "CHAIN_BUSY");
}
}
/// <summary>
/// Lists the AuditLog Parquet archives (ScadaBridge#42). Refused for a site-scoped,
/// non-Administrator caller: an archive holds every site's rows.
@@ -3276,51 +3230,8 @@ public class ManagementActor : ReceiveActor
return new AuditArchiveListResult(archive.IsEnabled, await archive.ListAsync());
}
/// <summary>Upper bound on records per <see cref="VerifyAuditChainHeadsCommand"/>.</summary>
internal const int MaxChainHeadRecords = 50_000;
/// <summary>
/// Checks exported AuditLog chain heads against the database (ScadaBridge#107 item 1). Same
/// gate as verify-chain: a site-scoped, non-Administrator caller is refused because the
/// chain spans every site.
/// </summary>
private static async Task<object?> HandleVerifyAuditChainHeads(
IServiceProvider sp, VerifyAuditChainHeadsCommand cmd, AuthenticatedUser user)
{
if (cmd.Records is null || cmd.Records.Count == 0)
{
throw new ManagementCommandException("No exported chain-head records were supplied.", "INVALID_ARGUMENT");
}
if (cmd.Records.Count > MaxChainHeadRecords)
{
throw new ManagementCommandException(
$"{cmd.Records.Count} records exceed the {MaxChainHeadRecords} limit; verify fewer export files at a time.",
"INVALID_ARGUMENT");
}
if (user.PermittedSiteIds.Length > 0
&& !user.Roles.Contains(Roles.Administrator, StringComparer.OrdinalIgnoreCase))
{
throw new SiteScopeViolationException(
"Access denied: the audit hash chain spans every site; verify-heads needs an "
+ "OperationalAudit role that is not scoped to specific sites.");
}
var store = sp.GetRequiredService<IAuditChainHeadStore>();
try
{
return await store.VerifyExportedHeadsAsync(cmd.Records);
}
catch (TimeoutException ex)
{
throw new ManagementCommandException(ex.Message, "CHAIN_BUSY");
}
}
/// <summary>
/// Verifies one AuditLog Parquet archive (ScadaBridge#42): files against the manifest,
/// the chain recomputed from the archived rows, the chain end against the purge anchor.
/// Verifies one AuditLog Parquet archive (ScadaBridge#42): the data file against its manifest.
/// </summary>
private static async Task<object?> HandleVerifyAuditArchive(
IServiceProvider sp, VerifyAuditArchiveCommand cmd, AuthenticatedUser user)
@@ -3341,16 +3252,12 @@ public class ManagementActor : ReceiveActor
}
/// <summary>
/// Runs the archive cleanup now (ScadaBridge#109), on the maintenance-connection archive when
/// registered. Dry run reports only.
/// Runs the archive cleanup now (ScadaBridge#109). Dry run reports only.
/// </summary>
private static async Task<object?> HandleCleanupAuditArchives(
IServiceProvider sp, CleanupAuditArchivesCommand cmd, AuthenticatedUser user)
{
RequireAuditArchive(sp, user);
var archive = sp.GetKeyedService<IAuditLogArchive>(
ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Maintenance.AuditMaintenanceContext.ServiceKey)
?? sp.GetRequiredService<IAuditLogArchive>();
var archive = RequireAuditArchive(sp, user);
return await archive.CleanupAsync(cmd.DryRun);
}
@@ -60,13 +60,7 @@ public static class ManagementEndpoints
// script entry (design 2026-09-08): a direct call lasts up to the script's timeout
// (+ grace), a write kind up to its wait window.
typeof(FlowDebugRunScriptCommand),
// AuditLog hash-chain verification (ScadaBridge#41) recomputes one hash per row of
// a whole month — millions of rows on a busy central.
typeof(VerifyAuditChainCommand),
// Off-box chain-head check (ScadaBridge#107): one link lookup per exported head, under
// the chain lock SHARED (waits behind a chaining pass).
typeof(VerifyAuditChainHeadsCommand),
// AuditLog archive verification (ScadaBridge#42) hashes and re-chains a whole month's
// AuditLog archive verification (ScadaBridge#42) hashes and reads a whole month's
// Parquet archive.
typeof(VerifyAuditArchiveCommand),
// Archive cleanup proves stale archives redundant by reading Parquet files (ScadaBridge#109).