chore(audit): ScadaBridge C7 — perf re-baseline + CollapseAuditLogToCanonical projection test + index-test fix + dead-cref cleanup (Task 2.5)
Perf re-baseline (HotPathLatencyTests): empirical p95 on Apple M-series Release build: 4KB DetailsJson slow path ≈14 µs, small-DetailsJson no-redactors ≈2 µs, true no-op fast path ≈0 µs. Thresholds updated: 200 µs / 30 µs / 5 µs (≈15× headroom for contested CI runners). Old thresholds (50 µs / 10 µs) were set for the pre-C3 typed-field path; canonical JSON parse+rewrite is empirically faster. Adds a third test (Filter_Apply_NoDetailsJson_FastPath) that asserts same-instance return on the DetailsJson-null + within-cap fast path. Env-var overrides retained. CollapseAuditLogToCanonicalMigrationTests (new): three MSSQL-gated [SkippableFact] tests verifying Action/Category/Outcome projection, NULL Actor, DetailsJson codec round-trip, and all six persisted computed columns (Kind/Status/SourceSiteId/ ExecutionId/ParentExecutionId) for ApiOutbound, InboundAuthFailure, and Failed- status rows. AddAuditLogTableMigrationTests: rename CreatesFiveNamedIndexes → CreatesNineNamedIndexes; expand coverage from 5 original indexes to all 9 named non-clustered indexes present after CollapseAuditLogToCanonical (adds IX_AuditLog_Execution, IX_AuditLog_ParentExecution, IX_AuditLog_Node_Occurred, UX_AuditLog_EventId). Dead-cref cleanup: zero references to the deleted IAuditPayloadFilter / DefaultAuditPayloadFilter / SafeDefaultAuditPayloadFilter types remain in any .cs file (source or test). 26 occurrences across 13 files replaced with correct references to IAuditRedactor / ScadaBridgeAuditRedactor / SafeDefaultAuditRedactor or reworded as plain prose. Residual sweep: no unused transitional code found beyond the acknowledged "C3 transitional shim" comments on IngestedAtUtc stamping (active code, not dead).
This commit is contained in:
Joseph Doherty
@@ -117,10 +117,10 @@ public class AuditLogIngestActor : ReceiveActor
|
||||
// Resolve the repository for the whole batch — one DbContext per
|
||||
// message, mirroring NotificationOutboxActor. The injected-repository
|
||||
// mode (Bundle D tests) skips the scope entirely.
|
||||
// Bundle C (M5-T6): the IAuditPayloadFilter is also resolved from the
|
||||
// Bundle C (M5-T6): the IAuditRedactor is also resolved from the
|
||||
// per-message scope when one is available so the row is truncated +
|
||||
// redacted before InsertIfNotExistsAsync. The single-repository test
|
||||
// ctor has no service provider — it falls through with no filter,
|
||||
// ctor has no service provider — it falls through with no redactor,
|
||||
// which preserves the small-payload assumptions baked into the
|
||||
// existing D2 fixtures.
|
||||
// AuditLog-003: use CreateAsyncScope + await using so scoped EF Core
|
||||
|
||||
@@ -5,10 +5,10 @@ namespace ZB.MOM.WW.ScadaBridge.AuditLog.Central;
|
||||
/// <summary>
|
||||
/// Audit Log (#23) M6 Bundle E (T9) — bridges
|
||||
/// <see cref="IAuditRedactionFailureCounter"/> (incremented by
|
||||
/// <see cref="DefaultAuditPayloadFilter"/> every time a header / body / SQL
|
||||
/// parameter redactor stage throws and the filter has to over-redact the
|
||||
/// offending field) into <see cref="AuditCentralHealthSnapshot"/> so the
|
||||
/// failure surfaces on the central health surface as
|
||||
/// <see cref="ZB.MOM.WW.ScadaBridge.AuditLog.Redaction.ScadaBridgeAuditRedactor"/> every time
|
||||
/// a header / body / SQL parameter redactor stage throws and the redactor has
|
||||
/// to over-redact the offending field) into <see cref="AuditCentralHealthSnapshot"/>
|
||||
/// so the failure surfaces on the central health surface as
|
||||
/// <c>AuditCentralHealthSnapshot.AuditRedactionFailure</c>.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
|
||||
@@ -82,11 +82,11 @@ public sealed class CentralAuditWriter : ICentralAuditWriter
|
||||
_services = services ?? throw new ArgumentNullException(nameof(services));
|
||||
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
|
||||
// AuditLog-008: never default to null — over-redact instead.
|
||||
// C3 (Task 2.5): the canonical IAuditRedactor replaces the legacy
|
||||
// IAuditPayloadFilter. SafeDefaultAuditRedactor applies HTTP header
|
||||
// redaction with hard-coded sensitive defaults so a composition root
|
||||
// that omits the real redactor still scrubs Authorization / X-Api-Key /
|
||||
// Cookie / Set-Cookie before persistence.
|
||||
// C3 (Task 2.5): wired via the canonical IAuditRedactor seam.
|
||||
// SafeDefaultAuditRedactor applies HTTP header redaction with
|
||||
// hard-coded sensitive defaults so a composition root that omits the
|
||||
// real redactor still scrubs Authorization / X-Api-Key / Cookie /
|
||||
// Set-Cookie before persistence.
|
||||
_redactor = redactor ?? SafeDefaultAuditRedactor.Instance;
|
||||
_failureCounter = failureCounter ?? new NoOpCentralAuditWriteFailureCounter();
|
||||
_nodeIdentity = nodeIdentity;
|
||||
|
||||
@@ -8,13 +8,11 @@ using Microsoft.Extensions.Logging;
|
||||
namespace ZB.MOM.WW.ScadaBridge.AuditLog.Payload;
|
||||
|
||||
/// <summary>
|
||||
/// Pure, stateless redaction + truncation primitives shared by the legacy
|
||||
/// <see cref="DefaultAuditPayloadFilter"/> (which operates on the
|
||||
/// <see cref="ZB.MOM.WW.ScadaBridge.Commons.Entities.Audit.AuditEvent"/> entity)
|
||||
/// and the canonical <see cref="ScadaBridgeAuditRedactor"/> (which operates on
|
||||
/// <c>ZB.MOM.WW.Audit.AuditEvent</c> + its <c>DetailsJson</c>). Extracted in
|
||||
/// ScadaBridge audit re-architecture stage C2 (Task 2.5) so the byte-exact
|
||||
/// redaction logic lives in ONE place and the two code paths can never drift.
|
||||
/// Pure, stateless redaction + truncation primitives used by
|
||||
/// <see cref="ZB.MOM.WW.ScadaBridge.AuditLog.Redaction.ScadaBridgeAuditRedactor"/>
|
||||
/// (which operates on <c>ZB.MOM.WW.Audit.AuditEvent</c> + its <c>DetailsJson</c>).
|
||||
/// Extracted in ScadaBridge audit re-architecture stage C2 (Task 2.5) so the
|
||||
/// byte-exact redaction logic lives in ONE place.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
@@ -26,10 +24,11 @@ namespace ZB.MOM.WW.ScadaBridge.AuditLog.Payload;
|
||||
/// any DI / health-metric coupling.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// The regex CACHE and per-call options resolution deliberately stay in
|
||||
/// <see cref="DefaultAuditPayloadFilter"/> — they carry per-instance state
|
||||
/// (lazy compile, 100 ms compile budget, sentinel entries) that the safety-net
|
||||
/// tests pin to that class. This helper only holds the stateless stages that
|
||||
/// The regex CACHE and per-call options resolution live in
|
||||
/// <see cref="ZB.MOM.WW.ScadaBridge.AuditLog.Payload.AuditRegexCache"/> /
|
||||
/// <see cref="ZB.MOM.WW.ScadaBridge.AuditLog.Redaction.ScadaBridgeAuditRedactor"/>
|
||||
/// — they carry per-instance state (lazy compile, 100 ms compile budget,
|
||||
/// sentinel entries). This helper only holds the stateless stages that
|
||||
/// operate once the compiled regex set / redact list / cap has already been
|
||||
/// resolved.
|
||||
/// </para>
|
||||
|
||||
@@ -5,12 +5,11 @@ using Microsoft.Extensions.Logging;
|
||||
namespace ZB.MOM.WW.ScadaBridge.AuditLog.Payload;
|
||||
|
||||
/// <summary>
|
||||
/// Per-instance compiled-regex cache for audit body / SQL-parameter redactors.
|
||||
/// Extracted in ScadaBridge audit re-architecture stage C2 (Task 2.5) so the
|
||||
/// legacy <see cref="DefaultAuditPayloadFilter"/> and the canonical
|
||||
/// <see cref="ZB.MOM.WW.ScadaBridge.AuditLog.Redaction.ScadaBridgeAuditRedactor"/>
|
||||
/// share the SAME compile rules (50 ms per-match timeout, 100 ms compile budget,
|
||||
/// invalid-pattern sentinel) rather than duplicating the logic.
|
||||
/// Per-instance compiled-regex cache for audit body / SQL-parameter redactors
|
||||
/// used by <see cref="ZB.MOM.WW.ScadaBridge.AuditLog.Redaction.ScadaBridgeAuditRedactor"/>.
|
||||
/// Extracted in ScadaBridge audit re-architecture stage C2 (Task 2.5) to
|
||||
/// centralize compile rules (50 ms per-match timeout, 100 ms compile budget,
|
||||
/// invalid-pattern sentinel).
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
namespace ZB.MOM.WW.ScadaBridge.AuditLog.Payload;
|
||||
|
||||
/// <summary>
|
||||
/// Counter sink invoked by <see cref="DefaultAuditPayloadFilter"/> every time
|
||||
/// a redactor (header / body regex / SQL parameter) throws and the filter has
|
||||
/// to over-redact the offending field with the
|
||||
/// Counter sink invoked by <see cref="ZB.MOM.WW.ScadaBridge.AuditLog.Redaction.ScadaBridgeAuditRedactor"/>
|
||||
/// every time a redactor (header / body regex / SQL parameter) throws and the
|
||||
/// redactor has to over-redact the offending field with the
|
||||
/// <c><redacted: redactor error></c> marker. Bundle C bridges this into
|
||||
/// the Site Health Monitoring report payload as <c>AuditRedactionFailure</c>.
|
||||
/// </summary>
|
||||
|
||||
@@ -7,10 +7,9 @@ using static ZB.MOM.WW.ScadaBridge.AuditLog.Payload.AuditRedactionPrimitives;
|
||||
namespace ZB.MOM.WW.ScadaBridge.AuditLog.Redaction;
|
||||
|
||||
/// <summary>
|
||||
/// Canonical-record analogue of <see cref="SafeDefaultAuditPayloadFilter"/> for
|
||||
/// stage C2 (Task 2.5): a minimal always-safe <see cref="IAuditRedactor"/>
|
||||
/// fallback for composition roots that bypass the full
|
||||
/// <see cref="ScadaBridgeAuditRedactor"/>. Performs line-oriented HTTP header
|
||||
/// Minimal always-safe <see cref="IAuditRedactor"/> fallback for composition
|
||||
/// roots that bypass the full <see cref="ScadaBridgeAuditRedactor"/>.
|
||||
/// Performs line-oriented HTTP header
|
||||
/// redaction for the always-sensitive defaults (Authorization, X-Api-Key,
|
||||
/// Cookie, Set-Cookie) on the <c>RequestSummary</c> / <c>ResponseSummary</c>
|
||||
/// fields carried inside <c>ZB.MOM.WW.Audit.AuditEvent.DetailsJson</c>. Does NOT
|
||||
|
||||
@@ -11,23 +11,17 @@ using ZB.MOM.WW.ScadaBridge.Commons.Types.Enums;
|
||||
namespace ZB.MOM.WW.ScadaBridge.AuditLog.Redaction;
|
||||
|
||||
/// <summary>
|
||||
/// Canonical <see cref="IAuditRedactor"/> implementation for ScadaBridge — the
|
||||
/// stage-C2 port of <see cref="DefaultAuditPayloadFilter"/> onto
|
||||
/// <c>ZB.MOM.WW.Audit.AuditEvent</c> and its <see cref="AuditEvent.DetailsJson"/>
|
||||
/// Canonical <see cref="IAuditRedactor"/> implementation for ScadaBridge —
|
||||
/// operates on <c>ZB.MOM.WW.Audit.AuditEvent</c> and its <see cref="AuditEvent.DetailsJson"/>
|
||||
/// payload bag. The ScadaBridge request/response/error/extra summaries travel
|
||||
/// inside <c>DetailsJson</c> as a <see cref="AuditDetails"/> record (serialized
|
||||
/// by <see cref="AuditDetailsCodec"/>); this redactor deserializes them, applies
|
||||
/// the SAME header → body-regex → SQL-parameter → byte-safe truncation pipeline
|
||||
/// the legacy filter applies, re-serializes, and returns a filtered COPY.
|
||||
/// the header → body-regex → SQL-parameter → byte-safe truncation pipeline,
|
||||
/// re-serializes, and returns a filtered COPY.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// Additive only: the legacy <see cref="IAuditPayloadFilter"/> pipeline stays in
|
||||
/// place and wired until stage C3 swaps the record type; this redactor is the
|
||||
/// canonical-record analogue exercised in isolation by the C2 unit tests.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// Cap selection is faithful to the legacy filter, translated onto canonical
|
||||
/// Cap selection is faithful to the original pipeline, translated onto canonical
|
||||
/// fields:
|
||||
/// <list type="bullet">
|
||||
/// <item>The <c>ApiInbound</c> branch keys on <see cref="AuditEvent.Category"/>
|
||||
@@ -49,7 +43,7 @@ namespace ZB.MOM.WW.ScadaBridge.AuditLog.Redaction;
|
||||
/// <para>
|
||||
/// MUST NOT throw — wrapped in try/catch; over-redacts (drops ALL sensitive free-text
|
||||
/// fields to a safe marker) on any internal failure, mirroring
|
||||
/// <see cref="SafeDefaultAuditPayloadFilter"/>.
|
||||
/// <see cref="SafeDefaultAuditRedactor"/>.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
public sealed class ScadaBridgeAuditRedactor : IAuditRedactor
|
||||
@@ -238,8 +232,7 @@ public sealed class ScadaBridgeAuditRedactor : IAuditRedactor
|
||||
/// <summary>
|
||||
/// Combine the global and per-target body-redactor lists, returning the
|
||||
/// compiled-regex set to apply. Patterns that failed compilation are
|
||||
/// silently skipped. Identical resolution to
|
||||
/// <see cref="DefaultAuditPayloadFilter"/>.
|
||||
/// silently skipped.
|
||||
/// </summary>
|
||||
private IReadOnlyList<Regex> ResolveBodyRegexes(AuditLogOptions opts, string? target)
|
||||
{
|
||||
@@ -283,7 +276,6 @@ public sealed class ScadaBridgeAuditRedactor : IAuditRedactor
|
||||
/// Resolve the per-connection SQL parameter redaction regex for the given
|
||||
/// target. Connection key = everything before the first <c>.</c> in
|
||||
/// <paramref name="target"/>. Patterns are forced case-insensitive.
|
||||
/// Identical resolution to <see cref="DefaultAuditPayloadFilter"/>.
|
||||
/// </summary>
|
||||
private bool TryGetSqlParamRedactor(AuditLogOptions opts, string? target, out Regex? regex)
|
||||
{
|
||||
|
||||
@@ -72,14 +72,11 @@ public static class ServiceCollectionExtensions
|
||||
// validator (a strict improvement over the previous AddSingleton).
|
||||
services.AddValidatedOptions<AuditLogOptions, AuditLogOptionsValidator>(config, ConfigSectionName);
|
||||
|
||||
// C3 (Task 2.5): the canonical IAuditRedactor replaces the legacy
|
||||
// IAuditPayloadFilter in the writer pipeline. ScadaBridgeAuditRedactor
|
||||
// is the port of DefaultAuditPayloadFilter onto the canonical record +
|
||||
// its DetailsJson payload bag — same truncation + header / body /
|
||||
// SQL-parameter redaction, applied between event construction and
|
||||
// persistence. Singleton — stateless; the IOptionsMonitor dependency
|
||||
// picks up hot reloads on its own. The old IAuditPayloadFilter classes
|
||||
// are retained but no longer wired into any pipeline (C7 deletes them).
|
||||
// C3 (Task 2.5): the canonical IAuditRedactor is wired as
|
||||
// ScadaBridgeAuditRedactor — same truncation + header / body /
|
||||
// SQL-parameter redaction as the original pipeline, applied between
|
||||
// event construction and persistence. Singleton — stateless; the
|
||||
// IOptionsMonitor dependency picks up hot reloads on its own.
|
||||
services.AddSingleton<IAuditRedactor, ScadaBridgeAuditRedactor>();
|
||||
|
||||
// M5 Bundle B: per-stage redactor-failure counter. NoOp default;
|
||||
@@ -234,7 +231,7 @@ public static class ServiceCollectionExtensions
|
||||
/// real <see cref="HealthMetricsAuditWriteFailureCounter"/> /
|
||||
/// <see cref="HealthMetricsAuditRedactionFailureCounter"/> bridges so the
|
||||
/// FallbackAuditWriter primary-failure counter AND the
|
||||
/// DefaultAuditPayloadFilter redactor-failure counter both surface in the
|
||||
/// <see cref="ScadaBridgeAuditRedactor"/> redactor-failure counter both surface in the
|
||||
/// site health report payload as
|
||||
/// <c>SiteHealthReport.SiteAuditWriteFailures</c> +
|
||||
/// <c>SiteHealthReport.AuditRedactionFailure</c>.
|
||||
|
||||
@@ -36,14 +36,15 @@ public sealed class FallbackAuditWriter : IAuditWriter
|
||||
private readonly SemaphoreSlim _drainGate = new(1, 1);
|
||||
|
||||
/// <summary>
|
||||
/// Bundle C (M5-T6) wires the singleton <see cref="IAuditPayloadFilter"/>
|
||||
/// Bundle C (M5-T6) wires the singleton <see cref="IAuditRedactor"/>
|
||||
/// here so every event written via the site hot path is truncated +
|
||||
/// header/body/SQL-param redacted before it hits both the primary SQLite
|
||||
/// writer AND the ring fallback. The parameter is optional (defaults to
|
||||
/// no filtering) so the long tail of test composition roots that don't
|
||||
/// care about the filter need no change — the production
|
||||
/// the always-safe <see cref="SafeDefaultAuditRedactor"/>) so the long
|
||||
/// tail of test composition roots that don't care about the redactor need
|
||||
/// no change — the production
|
||||
/// <see cref="ServiceCollectionExtensions.AddAuditLog"/> registration
|
||||
/// always passes the real filter through.
|
||||
/// always passes the real redactor through.
|
||||
/// </summary>
|
||||
/// <param name="primary">The primary audit writer (typically the SQLite writer).</param>
|
||||
/// <param name="ring">Drop-oldest ring buffer used to stash events when the primary fails.</param>
|
||||
@@ -62,14 +63,13 @@ public sealed class FallbackAuditWriter : IAuditWriter
|
||||
_failureCounter = failureCounter ?? throw new ArgumentNullException(nameof(failureCounter));
|
||||
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
|
||||
// AuditLog-008: never default to a null redactor — over-redact instead.
|
||||
// C3 (Task 2.5): the canonical IAuditRedactor replaces the legacy
|
||||
// IAuditPayloadFilter. SafeDefaultAuditRedactor performs HTTP header
|
||||
// redaction with the hard-coded sensitive defaults (Authorization,
|
||||
// X-Api-Key, Cookie, Set-Cookie) on the DetailsJson summaries so a test
|
||||
// composition root that doesn't bind the real options never persists
|
||||
// those headers verbatim. The full ScadaBridgeAuditRedactor (truncation
|
||||
// + body / SQL-param redaction) is wired by AddAuditLog and takes
|
||||
// precedence.
|
||||
// C3 (Task 2.5): wired via the canonical IAuditRedactor seam.
|
||||
// SafeDefaultAuditRedactor performs HTTP header redaction with the
|
||||
// hard-coded sensitive defaults (Authorization, X-Api-Key, Cookie,
|
||||
// Set-Cookie) on the DetailsJson summaries so a test composition root
|
||||
// that doesn't bind the real options never persists those headers
|
||||
// verbatim. The full ScadaBridgeAuditRedactor (truncation + body /
|
||||
// SQL-param redaction) is wired by AddAuditLog and takes precedence.
|
||||
_redactor = redactor ?? SafeDefaultAuditRedactor.Instance;
|
||||
}
|
||||
|
||||
|
||||
+4
-4
@@ -6,10 +6,10 @@ namespace ZB.MOM.WW.ScadaBridge.AuditLog.Site;
|
||||
/// <summary>
|
||||
/// Audit Log (#23) M5 Bundle C — bridges
|
||||
/// <see cref="IAuditRedactionFailureCounter"/> (incremented by
|
||||
/// <see cref="DefaultAuditPayloadFilter"/> every time a header / body / SQL
|
||||
/// parameter redactor stage throws and the filter has to over-redact the
|
||||
/// offending field) into <see cref="ISiteHealthCollector"/> so the count
|
||||
/// surfaces in the site health report payload as
|
||||
/// <see cref="ZB.MOM.WW.ScadaBridge.AuditLog.Redaction.ScadaBridgeAuditRedactor"/> every time
|
||||
/// a header / body / SQL parameter redactor stage throws and the redactor has
|
||||
/// to over-redact the offending field) into <see cref="ISiteHealthCollector"/>
|
||||
/// so the count surfaces in the site health report payload as
|
||||
/// <c>SiteHealthReport.AuditRedactionFailure</c>.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
|
||||
@@ -59,9 +59,9 @@ namespace ZB.MOM.WW.ScadaBridge.InboundAPI.Middleware;
|
||||
/// <c>cap + 1</c> bytes per body even when the client streams hundreds of MiB.
|
||||
/// The downstream handler and the real client still see every byte; only the
|
||||
/// audit copy is bounded. The cap is also enforced again by
|
||||
/// <see cref="ZB.MOM.WW.ScadaBridge.AuditLog.Payload.DefaultAuditPayloadFilter"/> (which OR's
|
||||
/// <see cref="ZB.MOM.WW.ScadaBridge.AuditLog.Redaction.ScadaBridgeAuditRedactor"/> (which OR's
|
||||
/// in its own <see cref="AuditEvent.PayloadTruncated"/> determination), so a
|
||||
/// row truncated here remains truncated even if the filter is bypassed.
|
||||
/// row truncated here remains truncated even if the redactor is bypassed.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
public sealed class AuditWriteMiddleware
|
||||
@@ -118,9 +118,8 @@ public sealed class AuditWriteMiddleware
|
||||
{
|
||||
var sw = Stopwatch.StartNew();
|
||||
|
||||
// Per-request hot read of the inbound cap — mirrors the convention used
|
||||
// by DefaultAuditPayloadFilter so a live config change picks up on the
|
||||
// next request without re-resolving the singleton.
|
||||
// Per-request hot read of the inbound cap so a live config change
|
||||
// picks up on the next request without re-resolving the singleton.
|
||||
var cap = _options.CurrentValue.InboundMaxBytes;
|
||||
|
||||
// Audit Log #23 (ParentExecutionId): mint the inbound request's per-request
|
||||
@@ -428,7 +427,7 @@ public sealed class AuditWriteMiddleware
|
||||
/// and the boundary stands as-is.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// Mirrors the algorithm in <c>DefaultAuditPayloadFilter.TruncateUtf8</c>;
|
||||
/// Mirrors the algorithm in <see cref="ZB.MOM.WW.ScadaBridge.AuditLog.Payload.AuditRedactionPrimitives.TruncateUtf8"/>;
|
||||
/// kept local to avoid a backwards project reference from
|
||||
/// ZB.MOM.WW.ScadaBridge.AuditLog into ZB.MOM.WW.ScadaBridge.InboundAPI.
|
||||
/// </remarks>
|
||||
|
||||
Reference in New Issue
Block a user