Joseph Doherty
7be4f19861
Audit roadmap T2 (alog.md §15.2): with AuditLog:Archive:Enabled (default off)
the purge actor writes each expired monthly partition to Parquet under an
absolute AuditLog:Archive:Directory before switching it out, so audit rows
outlive RetentionDays.
- Parquet.Net 6.1.0 (fully managed; NuGetAudit clean) in central package
management.
- AuditArchiveParquet: data file = the ten stored columns, OccurredAtUtc as a
UTC microsecond timestamp plus exact OccurredAtUtcTicks, and ChainSeq/RowHash;
tombstone file = the month's per-channel purge tombstones. Buffered row
groups; the files are plain Parquet (DuckDB/pyarrow read them).
- AuditLogPartitionArchive (IAuditLogArchive): reads the partition's rows,
chain heads and tombstones in one transaction under the chain lock SHARED,
writes {Directory}/{YYYY}/auditlog-{YYYY-MM}-{stamp}Z.parquet,
.tombstones.parquet and a .manifest.json (counts, SHA-256 per file, each
month's head and prior anchor) via tmp + rename, manifest last. Archives are
never overwritten.
- Archive before purge: SwitchOutPartitionAsync gains an internal guarded
overload. Under the EXCLUSIVE chain lock it re-counts the partition and
re-reads each month's LastSeq and THROWs 51002/51003 (nothing purged) unless
both still equal the archive's, so the switch's checkpoint anchor is exactly
the archive's chain end. The manifest is then marked Purged.
- Fail closed: a partition that cannot be archived is not purged; the failure
takes the existing AuditLogPurgeFailedEvent / PurgeFailures path. Success
publishes AuditLogArchivedEvent.
- Verify: file hashes and counts against the manifest, the chain recomputed
from the archived rows and tombstones (AuditChainVerifier), and the chain end
against the database anchor (Matches / Superseded / NotPurged / Mismatch).
- AuditLogArchiveOptionsValidator: Directory required when enabled and must be
absolute; RowGroupSize, timeouts bounded.
No migration: the purger role already reads AuditLog, the head and tombstones.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
40 lines
1.8 KiB
XML
40 lines
1.8 KiB
XML
<Project Sdk="Microsoft.NET.Sdk">
|
|
|
|
<PropertyGroup>
|
|
<TargetFramework>net10.0</TargetFramework>
|
|
<ImplicitUsings>enable</ImplicitUsings>
|
|
<Nullable>enable</Nullable>
|
|
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
|
|
</PropertyGroup>
|
|
|
|
<ItemGroup>
|
|
<InternalsVisibleTo Include="ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests" />
|
|
</ItemGroup>
|
|
|
|
<ItemGroup>
|
|
<PackageReference Include="Parquet.Net" />
|
|
<PackageReference Include="Microsoft.EntityFrameworkCore" />
|
|
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" />
|
|
<PackageReference Include="Microsoft.EntityFrameworkCore.Design">
|
|
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
|
|
<PrivateAssets>all</PrivateAssets>
|
|
</PackageReference>
|
|
<PackageReference Include="Microsoft.Extensions.Configuration.Json" />
|
|
<PackageReference Include="Microsoft.Extensions.DependencyInjection.Abstractions" />
|
|
<PackageReference Include="Microsoft.Extensions.Logging.Abstractions" />
|
|
<PackageReference Include="Microsoft.Extensions.Options" />
|
|
<PackageReference Include="Microsoft.AspNetCore.DataProtection" />
|
|
<PackageReference Include="Microsoft.AspNetCore.DataProtection.EntityFrameworkCore" />
|
|
<!-- Direct pin of a transitive dependency of Microsoft.AspNetCore.DataProtection:
|
|
10.0.7 carries four NU1903 high-severity advisories that break fresh restores
|
|
under TreatWarningsAsErrors. See Directory.Packages.props for the rationale. -->
|
|
<PackageReference Include="System.Security.Cryptography.Xml" />
|
|
<PackageReference Include="System.Security.Cryptography.Pkcs" />
|
|
</ItemGroup>
|
|
|
|
<ItemGroup>
|
|
<ProjectReference Include="../ZB.MOM.WW.ScadaBridge.Commons/ZB.MOM.WW.ScadaBridge.Commons.csproj" />
|
|
</ItemGroup>
|
|
|
|
</Project>
|