dohertj2/ScadaBridge
1
0
Fork 0
Code Issues 15 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
1ae4c7a4cbb7c2e62ca5f2d766eb22061cbe4707
ScadaBridge/Directory.Packages.props
T
Joseph Doherty 7be4f19861 feat(audit): Parquet cold archive of expired AuditLog partitions before the purge (ScadaBridge#42) 
Audit roadmap T2 (alog.md §15.2): with AuditLog:Archive:Enabled (default off)
the purge actor writes each expired monthly partition to Parquet under an
absolute AuditLog:Archive:Directory before switching it out, so audit rows
outlive RetentionDays.

- Parquet.Net 6.1.0 (fully managed; NuGetAudit clean) in central package
  management.
- AuditArchiveParquet: data file = the ten stored columns, OccurredAtUtc as a
  UTC microsecond timestamp plus exact OccurredAtUtcTicks, and ChainSeq/RowHash;
  tombstone file = the month's per-channel purge tombstones. Buffered row
  groups; the files are plain Parquet (DuckDB/pyarrow read them).
- AuditLogPartitionArchive (IAuditLogArchive): reads the partition's rows,
  chain heads and tombstones in one transaction under the chain lock SHARED,
  writes {Directory}/{YYYY}/auditlog-{YYYY-MM}-{stamp}Z.parquet,
  .tombstones.parquet and a .manifest.json (counts, SHA-256 per file, each
  month's head and prior anchor) via tmp + rename, manifest last. Archives are
  never overwritten.
- Archive before purge: SwitchOutPartitionAsync gains an internal guarded
  overload. Under the EXCLUSIVE chain lock it re-counts the partition and
  re-reads each month's LastSeq and THROWs 51002/51003 (nothing purged) unless
  both still equal the archive's, so the switch's checkpoint anchor is exactly
  the archive's chain end. The manifest is then marked Purged.
- Fail closed: a partition that cannot be archived is not purged; the failure
  takes the existing AuditLogPurgeFailedEvent / PurgeFailures path. Success
  publishes AuditLogArchivedEvent.
- Verify: file hashes and counts against the manifest, the chain recomputed
  from the archived rows and tombstones (AuditChainVerifier), and the chain end
  against the database anchor (Matches / Superseded / NotPurged / Mismatch).
- AuditLogArchiveOptionsValidator: Directory required when enabled and must be
  absolute; RowGroupSize, timeouts bounded.

No migration: the purger role already reads AuditLog, the head and tombstones.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 17:12:12 -05:00

210 lines
14 KiB
XML
Raw Blame History

<Project>
<PropertyGroup>
<ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
<CentralPackageTransitivePinningEnabled>true</CentralPackageTransitivePinningEnabled>
</PropertyGroup>
<ItemGroup>
<PackageVersion Include="Akka" Version="1.5.62" />
<PackageVersion Include="Akka.Cluster" Version="1.5.62" />
<PackageVersion Include="Akka.Cluster.Tools" Version="1.5.62" />
<PackageVersion Include="Akka.Remote" Version="1.5.62" />
<PackageVersion Include="Akka.Streams" Version="1.5.62" />
<PackageVersion Include="Akka.Streams.TestKit" Version="1.5.62" />
<PackageVersion Include="Akka.TestKit.Xunit2" Version="1.5.62" />
<PackageVersion Include="bunit" Version="2.7.2" />
<PackageVersion Include="coverlet.collector" Version="6.0.4" />
<PackageVersion Include="FluentAssertions" Version="8.3.0" />
<!--
gRPC stack raised 2.71.0 -> 2.76.0 (Protobuf 3.29.3 -> 3.34.1) on 2026-07-19.
Forced by ZB.MOM.WW.LocalDb.Replication 0.1.0, whose nuspec floors Grpc.AspNetCore,
Grpc.Net.Client, Grpc.Core.Api and Grpc.Tools at 2.76.0 and Google.Protobuf at 3.34.1.
Below that floor restore fails NU1605 (package downgrade), so this is not optional for
LocalDb adoption.
The SQLitePCLRaw note below deferred exactly this bump as belonging in "their own
reviewed commit" rather than smuggled under a security fix - so it IS its own commit.
Direct consumer surface is narrow: Grpc.AspNetCore only in Host, Grpc.Net.Client only
in Communication. Grpc.Core.Api is pinned explicitly here to keep the whole family on
one version rather than letting it float in transitively.
SqlClient is pinned to EF SqlServer 10.0.13's 6.1.7 floor (was 6.1.1 under EF 10.0.7; raised with the Gitea #98 Sqlite/EF 10.0.13 bump).
Newtonsoft.Json remains a separate data-access compatibility decision.
-->
<PackageVersion Include="Google.Protobuf" Version="3.34.1" />
<PackageVersion Include="Grpc.AspNetCore" Version="2.76.0" />
<PackageVersion Include="Grpc.Net.Client" Version="2.76.0" />
<PackageVersion Include="Grpc.Core.Api" Version="2.76.0" />
<PackageVersion Include="Grpc.Tools" Version="2.76.0" />
<PackageVersion Include="MailKit" Version="4.16.0" />
<PackageVersion Include="Microsoft.AspNetCore.Authentication.JwtBearer" Version="10.0.13" />
<PackageVersion Include="Microsoft.AspNetCore.Authorization" Version="10.0.13" />
<PackageVersion Include="Microsoft.AspNetCore.DataProtection" Version="10.0.13" />
<PackageVersion Include="Microsoft.AspNetCore.DataProtection.EntityFrameworkCore" Version="10.0.13" />
<PackageVersion Include="Microsoft.AspNetCore.Mvc.Testing" Version="10.0.13" />
<PackageVersion Include="Microsoft.AspNetCore.TestHost" Version="10.0.13" />
<PackageVersion Include="Microsoft.AspNetCore.SignalR.Client" Version="10.0.13" />
<PackageVersion Include="Microsoft.AspNetCore.SignalR.Common" Version="10.0.13" />
<PackageVersion Include="Microsoft.CodeAnalysis.CSharp.Scripting" Version="5.0.0" />
<PackageVersion Include="Microsoft.CodeAnalysis.CSharp.Workspaces" Version="5.0.0" />
<PackageVersion Include="Microsoft.Data.SqlClient" Version="6.1.7" />
<PackageVersion Include="Microsoft.Data.Sqlite" Version="10.0.13" />
<PackageVersion Include="Microsoft.EntityFrameworkCore" Version="10.0.13" />
<PackageVersion Include="Microsoft.EntityFrameworkCore.Design" Version="10.0.13" />
<PackageVersion Include="Microsoft.EntityFrameworkCore.InMemory" Version="10.0.13" />
<PackageVersion Include="Microsoft.EntityFrameworkCore.Sqlite" Version="10.0.13" />
<PackageVersion Include="Microsoft.EntityFrameworkCore.SqlServer" Version="10.0.13" />
<PackageVersion Include="Microsoft.Extensions.Configuration.Json" Version="10.0.13" />
<PackageVersion Include="Microsoft.Extensions.DependencyInjection" Version="10.0.13" />
<PackageVersion Include="Microsoft.Extensions.DependencyInjection.Abstractions" Version="10.0.13" />
<PackageVersion Include="Microsoft.Extensions.Hosting.Abstractions" Version="10.0.13" />
<PackageVersion Include="Microsoft.Extensions.Hosting.WindowsServices" Version="10.0.13" />
<PackageVersion Include="Microsoft.Extensions.Http" Version="10.0.13" />
<PackageVersion Include="Microsoft.Extensions.Logging" Version="10.0.13" />
<PackageVersion Include="Microsoft.Extensions.Logging.Abstractions" Version="10.0.13" />
<PackageVersion Include="Microsoft.Extensions.Options" Version="10.0.13" />
<PackageVersion Include="Microsoft.Extensions.Options.ConfigurationExtensions" Version="10.0.13" />
<PackageVersion Include="Microsoft.NET.Test.Sdk" Version="17.14.1" />
<PackageVersion Include="Microsoft.Playwright" Version="1.58.0" />
<PackageVersion Include="Moq" Version="4.20.72" />
<!-- 13.0.4 is the floor OPCFoundation.NetStandard.Opc.Ua.Core 1.5.378.106 demands; surfaced when
CentralPackageTransitivePinningEnabled was turned on (Gitea #79). Also used by the
Commons.Tests contract-lock tests;
used by Commons.Tests contract-lock tests that exercise the Akka default JSON serializer. -->
<PackageVersion Include="Newtonsoft.Json" Version="13.0.4" />
<PackageVersion Include="Novell.Directory.Ldap.NETStandard" Version="3.6.0" />
<PackageVersion Include="NSubstitute" Version="5.3.0" />
<PackageVersion Include="OPCFoundation.NetStandard.Opc.Ua.Client" Version="1.5.378.106" />
<PackageVersion Include="OpenTelemetry.Api" Version="1.15.3" />
<!-- Parquet audit archival + the parquet format of `audit export` (ScadaBridge#42). 6.1.0 is fully
managed (Snappier / ZstdSharp / K4os LZ4, no native IronCompress) and NuGetAudit-clean. -->
<PackageVersion Include="Parquet.Net" Version="6.1.0" />
<PackageVersion Include="Serilog" Version="4.3.1" />
<PackageVersion Include="Serilog.AspNetCore" Version="10.0.0" />
<PackageVersion Include="Serilog.Sinks.Console" Version="6.1.1" />
<PackageVersion Include="Serilog.Sinks.File" Version="7.0.0" />
<PackageVersion Include="System.CommandLine" Version="2.0.5" />
<PackageVersion Include="System.IdentityModel.Tokens.Jwt" Version="8.11.0" />
<PackageVersion Include="xunit" Version="2.9.3" />
<PackageVersion Include="xunit.runner.visualstudio" Version="3.1.4" />
<!--
Xunit.SkippableFact provides [SkippableFact] + Skip.IfNot/Skip.If for
xunit v2. The native Skip API (Assert.Skip / Assert.SkipUnless /
Assert.SkipWhen) only exists in xunit v3; xunit 2.9.x lacks it. Used by
Bundle C MSSQL integration tests in ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests
to mark tests as Skipped (not silently Passed) when MSSQL is unreachable.
-->
<PackageVersion Include="Xunit.SkippableFact" Version="1.5.61" />
<PackageVersion Include="ZB.MOM.WW.Health" Version="0.3.0" />
<PackageVersion Include="ZB.MOM.WW.Health.Akka" Version="0.3.0" />
<PackageVersion Include="ZB.MOM.WW.Health.EntityFrameworkCore" Version="0.3.0" />
<PackageVersion Include="ZB.MOM.WW.Telemetry" Version="0.1.0" />
<PackageVersion Include="ZB.MOM.WW.Telemetry.Serilog" Version="0.1.0" />
<PackageVersion Include="ZB.MOM.WW.MxGateway.Client" Version="0.2.1" />
<PackageVersion Include="ZB.MOM.WW.MxGateway.Contracts" Version="0.2.1" />
<PackageVersion Include="ZB.MOM.WW.Configuration" Version="0.1.0" />
<PackageVersion Include="ZB.MOM.WW.Auth.Abstractions" Version="0.3.0" />
<PackageVersion Include="ZB.MOM.WW.Auth.Ldap" Version="0.3.0" />
<PackageVersion Include="ZB.MOM.WW.Auth.ApiKeys" Version="0.3.0" />
<PackageVersion Include="ZB.MOM.WW.Auth.ApiKeys.SqlServer" Version="0.3.0" />
<PackageVersion Include="ZB.MOM.WW.Auth.AspNetCore" Version="0.3.0" />
<PackageVersion Include="ZB.MOM.WW.Audit" Version="0.1.0" />
<PackageVersion Include="ZB.MOM.WW.Theme" Version="0.5.0" />
<PackageVersion Include="ZB.MOM.WW.Secrets" Version="0.6.2" />
<PackageVersion Include="ZB.MOM.WW.Secrets.Abstractions" Version="0.6.2" />
<PackageVersion Include="ZB.MOM.WW.Secrets.Ui" Version="0.6.2" />
<PackageVersion Include="ZB.MOM.WW.Secrets.Replicator.SqlServer" Version="0.6.2" />
<PackageVersion Include="ZB.MOM.WW.LocalDb" Version="0.3.4" />
<PackageVersion Include="ZB.MOM.WW.LocalDb.Replication" Version="0.3.4" />
<PackageVersion Include="ZB.MOM.WW.LocalDb.Contracts" Version="0.3.4" />
</ItemGroup>
<!--
GHSA-2m69-gcr7-jv3q (NU1903, high) is on the NATIVE library SQLitePCLRaw.lib.e_sqlite3
2.1.11, pulled in TRANSITIVELY by Microsoft.Data.Sqlite / EntityFrameworkCore.Sqlite.
FIXED PROPERLY 2026-07-18, replacing a NuGetAuditSuppress that had been masking it. The
suppression's stated rationale — "the only patched native lib is the SQLitePCLRaw 3.x line" —
was WRONG: 2.1.12 patches this advisory within the 2.1.x line, so no risky force-override of
the whole family to 3.x is needed. Verified empirically: with the suppression removed and the
pin in place, a forced restore reports no NU1903.
Why the pin and not a Microsoft.Data.Sqlite bump: bumping Sqlite does NOT clear it (even
10.0.10 still references 2.1.11). Pinning the native lib is the fix. Same approach as
ZB.MOM.WW.Secrets and ZB.MOM.WW.LocalDb.
Why this mattered more than it looked: only the Host project resolved a patched 2.1.12 (via
ZB.MOM.WW.Auth.ApiKeys). AuditLog, SiteEventLogging, SiteRuntime, StoreAndForward and the
test projects — the whole site-local SQLite layer — were still resolving the vulnerable
2.1.11, and the suppression hid that everywhere.
Delivered since Gitea #79 via CentralPackageTransitivePinningEnabled (top of this file): the
PackageVersion below is applied to the transitive SQLitePCLRaw.lib.e_sqlite3 in EVERY project,
so the per-project <PackageReference> pins (16 csproj files) were removed. Enabling it forced
centrally-defined versions to be a floor for transitive resolution, which required:
Newtonsoft.Json 13.0.3 -> 13.0.4 (OPC UA core floor) and Microsoft.Extensions.* 10.0.7 ->
10.0.13 (the SignalR.Common 10.0.13 security pin in the CLI floors Options/DI at 10.0.13).
Google.Protobuf, Grpc.* and SqlClient had already been raised for other reasons.
-->
<!-- Gitea #98 (scadaproj#26): Microsoft.Data.Sqlite/EF Core 10.0.13 fix the connection-pool double-handout
(dotnet/efcore#39009) and pull SQLitePCLRaw 2.1.13, so this pin moved 2.1.12 -> 2.1.13 (2.1.12 = NU1109). -->
<ItemGroup>
<PackageVersion Include="SQLitePCLRaw.lib.e_sqlite3" Version="2.1.13" />
</ItemGroup>
<!--
Four NU1903 high-severity advisories (GHSA-23rf-6693-g89p, GHSA-8q5v-6pqq-x66h,
GHSA-cvvh-rhrc-wg4q, GHSA-g8r8-53c2-pm3f) landed in the NuGet audit data against
System.Security.Cryptography.Xml 10.0.7, pulled in TRANSITIVELY by
Microsoft.AspNetCore.DataProtection 10.0.7 (ConfigurationDatabase's DataProtection
key storage). With TreatWarningsAsErrors any FRESH restore — notably the docker
image build — went red (surfaced 2026-07-21; local builds had cached audit data).
Same pattern as SQLitePCLRaw above: pin the vulnerable transitive package to its
patched version (10.0.10) with an explicit <PackageReference> in the one project
where the chain enters (ConfigurationDatabase; every other resolver — AuditLog,
SiteCallAudit, Transport, PerformanceTests, tests — reaches it through that
ProjectReference). Bumping the DataProtection parent instead was tried and
rejected: 10.0.10 floors Microsoft.Extensions.* and (via the EFCore adapter)
Microsoft.EntityFrameworkCore at 10.0.10, forcing a family-wide servicing bump
(NU1605 downgrade errors) that belongs in its own reviewed commit.
-->
<ItemGroup>
<PackageVersion Include="System.Security.Cryptography.Xml" Version="10.0.13" />
<PackageVersion Include="System.Security.Cryptography.Pkcs" Version="10.0.13" />
</ItemGroup>
<!--
2026-10-22 NuGet audit refresh (fresh restore went red under TreatWarningsAsErrors):
- System.Security.Cryptography.Xml 10.0.10 now carries GHSA-7hqp-7c5m-4vq6,
GHSA-c399-9853-6jmj, GHSA-r955-8q5j-3h83 (high): pin raised 10.0.10 -> 10.0.13.
- System.Security.Cryptography.Pkcs (GHSA-2fmv-rv2c-59x7 moderate, GHSA-846f-f764-76mx
high) reaches the graph as 9.0.4 / 10.0.0 / 10.0.10 via SqlClient 6.1.1, MimeKit and
System.Security.Cryptography.Xml: pinned to 10.0.13 with explicit PackageReferences in
ConfigurationDatabase, ExternalSystemGateway, NotificationService and the Playwright tests,
the projects where those chains enter.
- Microsoft.AspNetCore.SignalR.Common 10.0.7 (GHSA-334f-vvp4-7g6p, high) comes in
transitively through SignalR.Client in the CLI: pinned to 10.0.13 in the CLI only.
Same house pattern as above: targeted transitive pins, no family-wide servicing bump, no
NuGetAuditSuppress/NoWarn.
-->
<!--
GHSA-pgww-w46g-26qg (NU1902, moderate) on AngleSharp, reached only transitively via bunit
in ZB.MOM.WW.ScadaBridge.CentralUI.Tests. With TreatWarningsAsErrors it made the WHOLE
SOLUTION BUILD RED, so no suite could run and every "0 warnings / green" gate was
unverifiable.
Resolved with a scoped NuGetAuditSuppress in that ONE test project (see its csproj for the
full rationale) — deliberately NOT a version pin here. A pin was tried first and reverted:
AngleSharp >= 1.5.0 is the only patched line, and it changes IHtmlCollection<T>'s indexer,
which makes every bunit build throw MissingMethodException at runtime (33 render tests
failed). Bumping bunit does not help either — even 2.7.2 resolves AngleSharp 1.4.0, which
is still vulnerable. There is no working patched combination upstream today.
-->
</Project>
Reference in New Issue View Git Blame Copy Permalink