Merge branch 'feature/inbound-api-full-response-audit': inbound API full request/response audit capture
Inbound API audit rows (Channel = ApiInbound) now capture request and response bodies in full up to a configurable 1 MiB per-body ceiling (AuditLog:InboundMaxBytes), instead of the global 8 KiB / 64 KiB caps that other audit channels use. Implements the M5-deferred response-body capture in AuditWriteMiddleware via a write-only Stream wrapper that forwards every byte to the framework's response sink while bounding the audit copy at the capture site (ArrayPool-backed for the request side). Other channels untouched.
This commit is contained in:
@@ -19,9 +19,11 @@ receive" debugging path.
|
|||||||
For `Channel = ApiInbound` rows only, capture `RequestSummary` and
|
For `Channel = ApiInbound` rows only, capture `RequestSummary` and
|
||||||
`ResponseSummary` verbatim up to a hard per-body ceiling of **1 MB**
|
`ResponseSummary` verbatim up to a hard per-body ceiling of **1 MB**
|
||||||
(configurable). The 8 KB / 64 KB default/error caps that apply to other channels
|
(configurable). The 8 KB / 64 KB default/error caps that apply to other channels
|
||||||
do not apply here. All other channels (`ApiOutbound`, `DbOutbound`,
|
do not apply here. The carve-out is channel-scoped (NOT kind-scoped): every
|
||||||
`Notification`, cached-call lifecycle, `InboundAuthFailure`) keep the existing
|
`Channel = ApiInbound` row uses the inbound ceiling regardless of `Kind`, so
|
||||||
policy unchanged.
|
`InboundAuthFailure` rows pick up the same ceiling as `InboundRequest`. All
|
||||||
|
other channels (`ApiOutbound`, `DbOutbound`, `Notification`, cached-call
|
||||||
|
lifecycle) keep the existing policy unchanged.
|
||||||
|
|
||||||
## Capture Policy Change
|
## Capture Policy Change
|
||||||
|
|
||||||
@@ -124,8 +126,9 @@ same "options validation" path used for other AuditLog settings.
|
|||||||
routinely-huge responses, operators use the existing per-target body redactor
|
routinely-huge responses, operators use the existing per-target body redactor
|
||||||
to compress them, or lower the global ceiling.
|
to compress them, or lower the global ceiling.
|
||||||
- **Changes to other channels' caps.** `ApiOutbound`, `DbOutbound`,
|
- **Changes to other channels' caps.** `ApiOutbound`, `DbOutbound`,
|
||||||
`Notification`, cached-call lifecycle rows, and `InboundAuthFailure` keep the
|
`Notification`, and cached-call lifecycle rows keep the existing 8 KB / 64 KB
|
||||||
existing 8 KB / 64 KB policy.
|
policy. (`InboundAuthFailure` rows carry `Channel = ApiInbound` and so fall
|
||||||
|
under the inbound ceiling like every other inbound row.)
|
||||||
|
|
||||||
## Acceptance Criteria
|
## Acceptance Criteria
|
||||||
|
|
||||||
|
|||||||
@@ -95,8 +95,8 @@ row per lifecycle event across all channels.
|
|||||||
| `DurationMs` | `int` NULL | Call / attempt duration. |
|
| `DurationMs` | `int` NULL | Call / attempt duration. |
|
||||||
| `ErrorMessage` | `nvarchar(1024)` NULL | Truncated; `ErrorDetail` for full text. |
|
| `ErrorMessage` | `nvarchar(1024)` NULL | Truncated; `ErrorDetail` for full text. |
|
||||||
| `ErrorDetail` | `nvarchar(max)` NULL | Optional full exception text on failures. |
|
| `ErrorDetail` | `nvarchar(max)` NULL | Optional full exception text on failures. |
|
||||||
| `RequestSummary` | `nvarchar(max)` NULL | Truncated request payload (configurable cap). Headers redacted. |
|
| `RequestSummary` | `nvarchar(max)` NULL | Truncated request payload (configurable cap). Headers redacted. For `Channel = ApiInbound`, captured in full up to `AuditLog:InboundMaxBytes` (default 1 MiB) — see Payload Capture Policy. |
|
||||||
| `ResponseSummary` | `nvarchar(max)` NULL | Truncated response payload. Full on errors. |
|
| `ResponseSummary` | `nvarchar(max)` NULL | Truncated response payload. For `Channel = ApiInbound`, captured in full up to `AuditLog:InboundMaxBytes` (default 1 MiB). For other channels, capped at `DefaultCapBytes` by default and `ErrorCapBytes` on error rows. |
|
||||||
| `PayloadTruncated` | `bit` | Set if either summary was truncated. |
|
| `PayloadTruncated` | `bit` | Set if either summary was truncated. |
|
||||||
| `Extra` | `nvarchar(max)` NULL | Channel-specific JSON for fields we don't promote to columns. |
|
| `Extra` | `nvarchar(max)` NULL | Channel-specific JSON for fields we don't promote to columns. |
|
||||||
|
|
||||||
@@ -262,6 +262,7 @@ operational `SiteCalls` shape for the dispatcher and UI.
|
|||||||
|
|
||||||
- **Default cap** — 8 KB for each of `RequestSummary` and `ResponseSummary`;
|
- **Default cap** — 8 KB for each of `RequestSummary` and `ResponseSummary`;
|
||||||
raised to 64 KB on any error row (`Status IN ('Failed', 'Parked', 'Discarded')`).
|
raised to 64 KB on any error row (`Status IN ('Failed', 'Parked', 'Discarded')`).
|
||||||
|
- **Inbound API exception.** For `Channel = ApiInbound`, `RequestSummary` and `ResponseSummary` are captured in full up to a per-body hard ceiling of 1 MiB (configurable via `AuditLog:InboundMaxBytes`; default 1 048 576 bytes; min 8 192; max 16 777 216). The 8 KiB / 64 KiB default/error caps that apply to other channels do not apply here. `PayloadTruncated = 1` is set only when the inbound ceiling is hit — verbatim capture is the normal case. The ceiling applies independently to each body. Header redaction and per-target body redactors still run before persistence.
|
||||||
- **Truncation** — UTF-8 byte-safe; `PayloadTruncated = 1` when applied. Full
|
- **Truncation** — UTF-8 byte-safe; `PayloadTruncated = 1` when applied. Full
|
||||||
bodies are never stored.
|
bodies are never stored.
|
||||||
- **HTTP headers** — `Authorization`, `Cookie`, `Set-Cookie`, `X-API-Key`, and
|
- **HTTP headers** — `Authorization`, `Cookie`, `Set-Cookie`, `X-API-Key`, and
|
||||||
|
|||||||
@@ -116,7 +116,7 @@ API method scripts are compiled at central startup — all method definitions ar
|
|||||||
|
|
||||||
## API Call Logging
|
## API Call Logging
|
||||||
|
|
||||||
- **Every request — success or failure — emits one `ApiInbound.Completed` row** to `ICentralAuditWriter` from request middleware before the HTTP response is flushed. The row captures the API key **name** (never the key material), remote IP, user-agent, response status, duration, and truncated request/response bodies per the Audit Log capture policy (see Component-AuditLog.md, Payload Capture Policy). This supersedes the earlier failures-only stance: operational API traffic is now part of the centralized audit log, so configuration changes and call activity share a single retention/query surface.
|
- **Every request — success or failure — emits one `ApiInbound.Completed` row** to `ICentralAuditWriter` from request middleware before the HTTP response is flushed. The row captures the API key **name** (never the key material), remote IP, user-agent, response status, duration, and the request/response bodies. Bodies are captured in full up to `AuditLog:InboundMaxBytes` (default 1 MiB); `PayloadTruncated = 1` only when that ceiling is hit. Header redaction and per-target body redactors still apply (see Component-AuditLog.md, Payload Capture Policy). This supersedes the earlier failures-only stance: operational API traffic is now part of the centralized audit log, so configuration changes and call activity share a single retention/query surface.
|
||||||
- Script execution errors (500 responses) remain captured on the same `ApiInbound.Completed` row (response status + error fields) rather than emitting a separate failure-only event.
|
- Script execution errors (500 responses) remain captured on the same `ApiInbound.Completed` row (response status + error fields) rather than emitting a separate failure-only event.
|
||||||
- **Fail-soft semantics.** The audit write is synchronous (inline before the response is flushed), but failures are caught: a write that throws is logged and increments `CentralAuditWriteFailures` (see Health Monitoring #11) and the request still returns its normal HTTP response. A failed audit append never turns a successful API call into an error returned to the caller.
|
- **Fail-soft semantics.** The audit write is synchronous (inline before the response is flushed), but failures are caught: a write that throws is logged and increments `CentralAuditWriteFailures` (see Health Monitoring #11) and the request still returns its normal HTTP response. A failed audit append never turns a successful API call into an error returned to the caller.
|
||||||
- No rate limiting — this is a private API in a controlled industrial environment with a known set of callers. Misbehaving callers are handled operationally (disable the API key).
|
- No rate limiting — this is a private API in a controlled industrial environment with a known set of callers. Misbehaving callers are handled operationally (disable the API key).
|
||||||
@@ -199,7 +199,7 @@ Inbound API scripts **cannot** call shared scripts directly — shared scripts a
|
|||||||
- **Communication Layer**: Routes requests to sites when method implementations need site data.
|
- **Communication Layer**: Routes requests to sites when method implementations need site data.
|
||||||
- **Security & Auth**: API key validation (separate from LDAP/AD — API uses key-based auth).
|
- **Security & Auth**: API key validation (separate from LDAP/AD — API uses key-based auth).
|
||||||
- **Configuration Database (via IAuditService)**: All API key and method definition changes are audit logged.
|
- **Configuration Database (via IAuditService)**: All API key and method definition changes are audit logged.
|
||||||
- **Audit Log (#23)**: Every inbound API request emits an `ApiInbound.Completed` row via `ICentralAuditWriter` from request middleware (non-blocking for the HTTP response). Payload truncation/redaction follows the Audit Log Payload Capture Policy.
|
- **Audit Log (#23)**: Every inbound API request emits an `ApiInbound.Completed` row via `ICentralAuditWriter` from request middleware (non-blocking for the HTTP response). Request and response bodies are captured in full up to `AuditLog:InboundMaxBytes` (default 1 MiB) per the Audit Log Payload Capture Policy; redaction (headers + per-target body redactors) still applies before persistence.
|
||||||
- **Cluster Infrastructure**: API is hosted on the active central node and fails over with it.
|
- **Cluster Infrastructure**: API is hosted on the active central node and fails over with it.
|
||||||
|
|
||||||
## Interactions
|
## Interactions
|
||||||
|
|||||||
@@ -1,3 +1,6 @@
|
|||||||
|
using ScadaLink.Commons.Entities.Audit;
|
||||||
|
using ScadaLink.Commons.Types.Enums;
|
||||||
|
|
||||||
namespace ScadaLink.AuditLog.Configuration;
|
namespace ScadaLink.AuditLog.Configuration;
|
||||||
|
|
||||||
/// <summary>
|
/// <summary>
|
||||||
@@ -33,4 +36,14 @@ public sealed class AuditLogOptions
|
|||||||
|
|
||||||
/// <summary>Central retention window in days (default 365, range [30, 3650]).</summary>
|
/// <summary>Central retention window in days (default 365, range [30, 3650]).</summary>
|
||||||
public int RetentionDays { get; set; } = 365;
|
public int RetentionDays { get; set; } = 365;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Per-body byte ceiling applied to <see cref="AuditEvent.RequestSummary"/> and
|
||||||
|
/// <see cref="AuditEvent.ResponseSummary"/> for <see cref="AuditChannel.ApiInbound"/> rows
|
||||||
|
/// (default 1 MiB). The 8 KiB / 64 KiB default/error caps that apply to other channels
|
||||||
|
/// do not apply here — inbound traffic captures verbatim up to this ceiling and only
|
||||||
|
/// then sets <see cref="AuditEvent.PayloadTruncated"/>. See
|
||||||
|
/// <c>docs/plans/2026-05-23-inbound-api-full-response-audit-design.md</c>.
|
||||||
|
/// </summary>
|
||||||
|
public int InboundMaxBytes { get; set; } = 1_048_576;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -21,6 +21,12 @@ public sealed class AuditLogOptionsValidator : IValidateOptions<AuditLogOptions>
|
|||||||
/// <summary>Inclusive upper bound for <see cref="AuditLogOptions.RetentionDays"/>.</summary>
|
/// <summary>Inclusive upper bound for <see cref="AuditLogOptions.RetentionDays"/>.</summary>
|
||||||
public const int MaxRetentionDays = 3650;
|
public const int MaxRetentionDays = 3650;
|
||||||
|
|
||||||
|
/// <summary>Inclusive lower bound for <see cref="AuditLogOptions.InboundMaxBytes"/> (8 KiB).</summary>
|
||||||
|
public const int MinInboundMaxBytes = 8_192;
|
||||||
|
|
||||||
|
/// <summary>Inclusive upper bound for <see cref="AuditLogOptions.InboundMaxBytes"/> (16 MiB).</summary>
|
||||||
|
public const int MaxInboundMaxBytes = 16_777_216;
|
||||||
|
|
||||||
/// <inheritdoc />
|
/// <inheritdoc />
|
||||||
public ValidateOptionsResult Validate(string? name, AuditLogOptions options)
|
public ValidateOptionsResult Validate(string? name, AuditLogOptions options)
|
||||||
{
|
{
|
||||||
@@ -50,6 +56,13 @@ public sealed class AuditLogOptionsValidator : IValidateOptions<AuditLogOptions>
|
|||||||
$"must be in [{MinRetentionDays}, {MaxRetentionDays}] days.");
|
$"must be in [{MinRetentionDays}, {MaxRetentionDays}] days.");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if (options.InboundMaxBytes < MinInboundMaxBytes || options.InboundMaxBytes > MaxInboundMaxBytes)
|
||||||
|
{
|
||||||
|
failures.Add(
|
||||||
|
$"AuditLog:{nameof(AuditLogOptions.InboundMaxBytes)} ({options.InboundMaxBytes}) " +
|
||||||
|
$"must be in [{MinInboundMaxBytes}, {MaxInboundMaxBytes}] bytes.");
|
||||||
|
}
|
||||||
|
|
||||||
return failures.Count == 0
|
return failures.Count == 0
|
||||||
? ValidateOptionsResult.Success
|
? ValidateOptionsResult.Success
|
||||||
: ValidateOptionsResult.Fail(failures);
|
: ValidateOptionsResult.Fail(failures);
|
||||||
|
|||||||
@@ -118,7 +118,14 @@ public sealed class DefaultAuditPayloadFilter : IAuditPayloadFilter
|
|||||||
try
|
try
|
||||||
{
|
{
|
||||||
var opts = _options.CurrentValue;
|
var opts = _options.CurrentValue;
|
||||||
var cap = IsErrorStatus(rawEvent.Status) ? opts.ErrorCapBytes : opts.DefaultCapBytes;
|
// Inbound API gets a dedicated, larger ceiling — request/response bodies are
|
||||||
|
// captured verbatim up to InboundMaxBytes (default 1 MiB) so support can
|
||||||
|
// replay exactly what the caller sent and what we returned. Other channels
|
||||||
|
// keep the global 8 KiB / 64 KiB policy.
|
||||||
|
// See docs/plans/2026-05-23-inbound-api-full-response-audit-design.md.
|
||||||
|
var cap = rawEvent.Channel == AuditChannel.ApiInbound
|
||||||
|
? opts.InboundMaxBytes
|
||||||
|
: (IsErrorStatus(rawEvent.Status) ? opts.ErrorCapBytes : opts.DefaultCapBytes);
|
||||||
|
|
||||||
// --- Header-redaction stage (runs BEFORE truncation) ----------
|
// --- Header-redaction stage (runs BEFORE truncation) ----------
|
||||||
var request = RedactHeaders(rawEvent.RequestSummary, opts.HeaderRedactList);
|
var request = RedactHeaders(rawEvent.RequestSummary, opts.HeaderRedactList);
|
||||||
|
|||||||
@@ -1,8 +1,11 @@
|
|||||||
|
using System.Buffers;
|
||||||
using System.Diagnostics;
|
using System.Diagnostics;
|
||||||
using System.Text;
|
using System.Text;
|
||||||
using System.Text.Json;
|
using System.Text.Json;
|
||||||
using Microsoft.AspNetCore.Http;
|
using Microsoft.AspNetCore.Http;
|
||||||
using Microsoft.Extensions.Logging;
|
using Microsoft.Extensions.Logging;
|
||||||
|
using Microsoft.Extensions.Options;
|
||||||
|
using ScadaLink.AuditLog.Configuration;
|
||||||
using ScadaLink.Commons.Entities.Audit;
|
using ScadaLink.Commons.Entities.Audit;
|
||||||
using ScadaLink.Commons.Interfaces.Services;
|
using ScadaLink.Commons.Interfaces.Services;
|
||||||
using ScadaLink.Commons.Types.Enums;
|
using ScadaLink.Commons.Types.Enums;
|
||||||
@@ -42,11 +45,22 @@ namespace ScadaLink.InboundAPI.Middleware;
|
|||||||
/// <para>
|
/// <para>
|
||||||
/// <b>Body capture.</b> The request body is buffered via
|
/// <b>Body capture.</b> The request body is buffered via
|
||||||
/// <see cref="HttpRequestRewindExtensions.EnableBuffering(HttpRequest)"/> then
|
/// <see cref="HttpRequestRewindExtensions.EnableBuffering(HttpRequest)"/> then
|
||||||
/// rewound so the downstream endpoint handler still sees the full payload.
|
/// rewound so the downstream endpoint handler still sees the full payload. The
|
||||||
/// Response body capture is deferred to M5 — wrapping <c>Response.Body</c>
|
/// response body is captured by wrapping <see cref="HttpResponse.Body"/> in a
|
||||||
/// requires a memory-stream swap that interacts awkwardly with Minimal API's
|
/// forwarding stream that mirrors writes to the original sink (transparent to
|
||||||
/// <c>Results.Json</c>/<c>Results.Text</c> writers; the M4 deliverable emits
|
/// the real client) while capturing a bounded copy for audit.
|
||||||
/// the audit row with <see cref="AuditEvent.ResponseSummary"/> left null.
|
/// </para>
|
||||||
|
///
|
||||||
|
/// <para>
|
||||||
|
/// <b>Bounded capture at the source.</b> Both the request- and response-body
|
||||||
|
/// audit copies are bounded at <see cref="AuditLogOptions.InboundMaxBytes"/>
|
||||||
|
/// (default 1 MiB) AT THE CAPTURE SITE — we never buffer more than
|
||||||
|
/// <c>cap + 1</c> bytes per body even when the client streams hundreds of MiB.
|
||||||
|
/// The downstream handler and the real client still see every byte; only the
|
||||||
|
/// audit copy is bounded. The cap is also enforced again by
|
||||||
|
/// <see cref="ScadaLink.AuditLog.Payload.DefaultAuditPayloadFilter"/> (which OR's
|
||||||
|
/// in its own <see cref="AuditEvent.PayloadTruncated"/> determination), so a
|
||||||
|
/// row truncated here remains truncated even if the filter is bypassed.
|
||||||
/// </para>
|
/// </para>
|
||||||
/// </summary>
|
/// </summary>
|
||||||
public sealed class AuditWriteMiddleware
|
public sealed class AuditWriteMiddleware
|
||||||
@@ -74,21 +88,29 @@ public sealed class AuditWriteMiddleware
|
|||||||
private readonly RequestDelegate _next;
|
private readonly RequestDelegate _next;
|
||||||
private readonly ICentralAuditWriter _auditWriter;
|
private readonly ICentralAuditWriter _auditWriter;
|
||||||
private readonly ILogger<AuditWriteMiddleware> _logger;
|
private readonly ILogger<AuditWriteMiddleware> _logger;
|
||||||
|
private readonly IOptionsMonitor<AuditLogOptions> _options;
|
||||||
|
|
||||||
public AuditWriteMiddleware(
|
public AuditWriteMiddleware(
|
||||||
RequestDelegate next,
|
RequestDelegate next,
|
||||||
ICentralAuditWriter auditWriter,
|
ICentralAuditWriter auditWriter,
|
||||||
ILogger<AuditWriteMiddleware> logger)
|
ILogger<AuditWriteMiddleware> logger,
|
||||||
|
IOptionsMonitor<AuditLogOptions> options)
|
||||||
{
|
{
|
||||||
_next = next ?? throw new ArgumentNullException(nameof(next));
|
_next = next ?? throw new ArgumentNullException(nameof(next));
|
||||||
_auditWriter = auditWriter ?? throw new ArgumentNullException(nameof(auditWriter));
|
_auditWriter = auditWriter ?? throw new ArgumentNullException(nameof(auditWriter));
|
||||||
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
|
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
|
||||||
|
_options = options ?? throw new ArgumentNullException(nameof(options));
|
||||||
}
|
}
|
||||||
|
|
||||||
public async Task InvokeAsync(HttpContext ctx)
|
public async Task InvokeAsync(HttpContext ctx)
|
||||||
{
|
{
|
||||||
var sw = Stopwatch.StartNew();
|
var sw = Stopwatch.StartNew();
|
||||||
|
|
||||||
|
// Per-request hot read of the inbound cap — mirrors the convention used
|
||||||
|
// by DefaultAuditPayloadFilter so a live config change picks up on the
|
||||||
|
// next request without re-resolving the singleton.
|
||||||
|
var cap = _options.CurrentValue.InboundMaxBytes;
|
||||||
|
|
||||||
// Audit Log #23 (ParentExecutionId): mint the inbound request's per-request
|
// Audit Log #23 (ParentExecutionId): mint the inbound request's per-request
|
||||||
// ExecutionId ONCE, here at the start of the request, and stash it on
|
// ExecutionId ONCE, here at the start of the request, and stash it on
|
||||||
// HttpContext.Items. Two consumers share this single id:
|
// HttpContext.Items. Two consumers share this single id:
|
||||||
@@ -106,7 +128,16 @@ public sealed class AuditWriteMiddleware
|
|||||||
// of the pipeline for us — but we also rewind to position 0 after our
|
// of the pipeline for us — but we also rewind to position 0 after our
|
||||||
// own read so the very next reader starts from the top.
|
// own read so the very next reader starts from the top.
|
||||||
ctx.Request.EnableBuffering();
|
ctx.Request.EnableBuffering();
|
||||||
var requestBody = await ReadBufferedRequestBodyAsync(ctx.Request).ConfigureAwait(false);
|
var (requestBody, requestTruncated) =
|
||||||
|
await ReadBufferedRequestBodyAsync(ctx.Request, cap).ConfigureAwait(false);
|
||||||
|
|
||||||
|
// Response body — wrap Response.Body in a forwarding stream that mirrors
|
||||||
|
// every write to the original sink (transparent to the real client)
|
||||||
|
// while capturing AT MOST `cap + 1` bytes for the audit copy. The
|
||||||
|
// original Response.Body is restored in the finally block.
|
||||||
|
var originalResponseBody = ctx.Response.Body;
|
||||||
|
using var captureStream = new CapturedResponseStream(originalResponseBody, cap);
|
||||||
|
ctx.Response.Body = captureStream;
|
||||||
|
|
||||||
Exception? thrown = null;
|
Exception? thrown = null;
|
||||||
try
|
try
|
||||||
@@ -123,7 +154,20 @@ public sealed class AuditWriteMiddleware
|
|||||||
finally
|
finally
|
||||||
{
|
{
|
||||||
sw.Stop();
|
sw.Stop();
|
||||||
EmitInboundAudit(ctx, sw.ElapsedMilliseconds, thrown, requestBody);
|
|
||||||
|
// Restore the original stream and resolve the captured audit copy.
|
||||||
|
// The forwarding wrapper has already written every byte to the
|
||||||
|
// original sink; this just pulls back the bounded UTF-8 string.
|
||||||
|
ctx.Response.Body = originalResponseBody;
|
||||||
|
var (responseBody, responseTruncated) = captureStream.GetCapturedBody();
|
||||||
|
|
||||||
|
EmitInboundAudit(
|
||||||
|
ctx,
|
||||||
|
sw.ElapsedMilliseconds,
|
||||||
|
thrown,
|
||||||
|
requestBody,
|
||||||
|
responseBody,
|
||||||
|
requestTruncated || responseTruncated);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -136,7 +180,9 @@ public sealed class AuditWriteMiddleware
|
|||||||
HttpContext ctx,
|
HttpContext ctx,
|
||||||
long durationMs,
|
long durationMs,
|
||||||
Exception? thrown,
|
Exception? thrown,
|
||||||
string? requestBody)
|
string? requestBody,
|
||||||
|
string? responseBody,
|
||||||
|
bool payloadTruncated)
|
||||||
{
|
{
|
||||||
try
|
try
|
||||||
{
|
{
|
||||||
@@ -187,9 +233,8 @@ public sealed class AuditWriteMiddleware
|
|||||||
DurationMs = (int)Math.Min(durationMs, int.MaxValue),
|
DurationMs = (int)Math.Min(durationMs, int.MaxValue),
|
||||||
ErrorMessage = thrown?.Message,
|
ErrorMessage = thrown?.Message,
|
||||||
RequestSummary = requestBody,
|
RequestSummary = requestBody,
|
||||||
// Response body capture is deferred to M5 (see XML doc above).
|
ResponseSummary = responseBody,
|
||||||
ResponseSummary = null,
|
PayloadTruncated = payloadTruncated,
|
||||||
PayloadTruncated = false,
|
|
||||||
Extra = extra,
|
Extra = extra,
|
||||||
// Central direct-write — no site-local forwarding state.
|
// Central direct-write — no site-local forwarding state.
|
||||||
ForwardState = null,
|
ForwardState = null,
|
||||||
@@ -210,39 +255,113 @@ public sealed class AuditWriteMiddleware
|
|||||||
}
|
}
|
||||||
|
|
||||||
/// <summary>
|
/// <summary>
|
||||||
/// Reads the buffered request body fully into a string and rewinds the
|
/// Reads the buffered request body up to <paramref name="capBytes"/> bytes
|
||||||
/// stream so the downstream handler sees the unconsumed payload. Returns
|
/// into a string for the audit copy and rewinds the stream so the
|
||||||
/// null for empty/missing bodies so the audit row's
|
/// downstream handler sees the unconsumed payload. Returns
|
||||||
|
/// <c>(null, false)</c> for empty/missing bodies so the audit row's
|
||||||
/// <see cref="AuditEvent.RequestSummary"/> stays null rather than
|
/// <see cref="AuditEvent.RequestSummary"/> stays null rather than
|
||||||
/// containing an empty string.
|
/// containing an empty string.
|
||||||
/// </summary>
|
/// </summary>
|
||||||
private static async Task<string?> ReadBufferedRequestBodyAsync(HttpRequest request)
|
/// <remarks>
|
||||||
|
/// Reads AT MOST <c>cap + 1</c> bytes from the request stream into a
|
||||||
|
/// scratch buffer; if the extra byte arrives the body is over the cap and
|
||||||
|
/// the returned string is UTF-8 byte-safe truncated to exactly
|
||||||
|
/// <c>cap</c> bytes with <c>truncated = true</c>. The cap applies only to
|
||||||
|
/// the audit copy — the request stream is always rewound to position 0
|
||||||
|
/// afterwards so the framework's next reader (the endpoint handler's
|
||||||
|
/// JSON parser) sees the full body.
|
||||||
|
/// </remarks>
|
||||||
|
private static async Task<(string? body, bool truncated)> ReadBufferedRequestBodyAsync(
|
||||||
|
HttpRequest request,
|
||||||
|
int capBytes)
|
||||||
{
|
{
|
||||||
if (request.ContentLength is 0)
|
if (request.ContentLength is 0)
|
||||||
{
|
{
|
||||||
return null;
|
return (null, false);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Read AT MOST cap + 1 bytes — the extra byte tells us the body was
|
||||||
|
// over the cap without forcing us to allocate the whole payload. Rent
|
||||||
|
// the scratch buffer from the shared ArrayPool so we don't allocate
|
||||||
|
// (and immediately discard) `cap + 1` bytes per request — the pool
|
||||||
|
// may hand back a buffer LARGER than `limit`, so we treat `limit`
|
||||||
|
// (not `buffer.Length`) as the read ceiling.
|
||||||
|
var limit = capBytes + 1;
|
||||||
|
var buffer = ArrayPool<byte>.Shared.Rent(limit);
|
||||||
try
|
try
|
||||||
{
|
{
|
||||||
request.Body.Position = 0;
|
request.Body.Position = 0;
|
||||||
using var reader = new StreamReader(
|
|
||||||
request.Body,
|
var total = 0;
|
||||||
Encoding.UTF8,
|
while (total < limit)
|
||||||
detectEncodingFromByteOrderMarks: false,
|
{
|
||||||
bufferSize: 1024,
|
var read = await request.Body
|
||||||
leaveOpen: true);
|
.ReadAsync(buffer.AsMemory(total, limit - total))
|
||||||
var content = await reader.ReadToEndAsync().ConfigureAwait(false);
|
.ConfigureAwait(false);
|
||||||
request.Body.Position = 0;
|
if (read == 0)
|
||||||
return string.IsNullOrEmpty(content) ? null : content;
|
{
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
total += read;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (total == 0)
|
||||||
|
{
|
||||||
|
return (null, false);
|
||||||
|
}
|
||||||
|
|
||||||
|
var truncated = total > capBytes;
|
||||||
|
var bytesForString = truncated ? capBytes : total;
|
||||||
|
var content = DecodeUtf8Bounded(buffer, bytesForString, cutAtValidBytes: truncated);
|
||||||
|
return (string.IsNullOrEmpty(content) ? null : content, truncated);
|
||||||
}
|
}
|
||||||
catch
|
catch
|
||||||
{
|
{
|
||||||
// A failed body read must not abort the request — fall through
|
// A failed body read must not abort the request — fall through
|
||||||
// with a null RequestSummary; the audit row still records the
|
// with a null RequestSummary; the audit row still records the
|
||||||
// outcome.
|
// outcome.
|
||||||
return null;
|
return (null, false);
|
||||||
}
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
// Even on a thrown read, the downstream handler must see the full
|
||||||
|
// body from position 0 — never let a failed audit copy leak a
|
||||||
|
// truncated view. A rewind failure is swallowed: best-effort,
|
||||||
|
// same philosophy as the rest of the file.
|
||||||
|
try { request.Body.Position = 0; } catch { /* swallow */ }
|
||||||
|
ArrayPool<byte>.Shared.Return(buffer);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// UTF-8 byte-safe decode of <paramref name="validBytes"/> bytes from
|
||||||
|
/// <paramref name="bytes"/>. When <paramref name="cutAtValidBytes"/> is
|
||||||
|
/// <c>true</c> the input is the result of a hard byte-count truncation, so
|
||||||
|
/// we walk back from <c>validBytes</c> while the byte is a continuation
|
||||||
|
/// byte (<c>byte & 0xC0 == 0x80</c>) to avoid splitting a multi-byte
|
||||||
|
/// codepoint. When <c>false</c> the caller is decoding the full payload
|
||||||
|
/// and the boundary stands as-is.
|
||||||
|
/// </summary>
|
||||||
|
/// <remarks>
|
||||||
|
/// Mirrors the algorithm in <c>DefaultAuditPayloadFilter.TruncateUtf8</c>;
|
||||||
|
/// kept local to avoid a backwards project reference from
|
||||||
|
/// ScadaLink.AuditLog into ScadaLink.InboundAPI.
|
||||||
|
/// </remarks>
|
||||||
|
private static string DecodeUtf8Bounded(byte[] bytes, int validBytes, bool cutAtValidBytes)
|
||||||
|
{
|
||||||
|
if (validBytes <= 0)
|
||||||
|
{
|
||||||
|
return string.Empty;
|
||||||
|
}
|
||||||
|
var boundary = validBytes;
|
||||||
|
if (cutAtValidBytes)
|
||||||
|
{
|
||||||
|
while (boundary > 0 && (bytes[boundary] & 0xC0) == 0x80)
|
||||||
|
{
|
||||||
|
boundary--;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return Encoding.UTF8.GetString(bytes, 0, boundary);
|
||||||
}
|
}
|
||||||
|
|
||||||
/// <summary>
|
/// <summary>
|
||||||
@@ -321,4 +440,153 @@ public sealed class AuditWriteMiddleware
|
|||||||
|
|
||||||
return path[(lastSlash + 1)..];
|
return path[(lastSlash + 1)..];
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Write-only forwarding <see cref="Stream"/> wrapper that mirrors every
|
||||||
|
/// write to the inner ASP.NET <see cref="HttpResponse.Body"/> (so the real
|
||||||
|
/// client receives all bytes) while capturing AT MOST <c>cap + 1</c> bytes
|
||||||
|
/// into a private bounded <see cref="MemoryStream"/> for the audit copy.
|
||||||
|
/// </summary>
|
||||||
|
/// <remarks>
|
||||||
|
/// <para>
|
||||||
|
/// The inner sink is owned by the framework and is NOT disposed when this
|
||||||
|
/// wrapper is disposed — we only own the capture <see cref="MemoryStream"/>.
|
||||||
|
/// </para>
|
||||||
|
/// <para>
|
||||||
|
/// All Write overloads forward to the inner stream FIRST, then capture the
|
||||||
|
/// remaining quota. If the inner sink throws (e.g. the client disconnects),
|
||||||
|
/// the exception is allowed to propagate — capture is best-effort, the
|
||||||
|
/// real I/O is authoritative. The handler-throws-mid-response test
|
||||||
|
/// (<c>ResponseBody_OnHandlerThrow_BodyCapturedUpToTheThrow</c>) verifies
|
||||||
|
/// that captured bytes up to the throw are still recoverable.
|
||||||
|
/// </para>
|
||||||
|
/// </remarks>
|
||||||
|
private sealed class CapturedResponseStream : Stream
|
||||||
|
{
|
||||||
|
private readonly Stream _inner;
|
||||||
|
private readonly int _capBytes;
|
||||||
|
private readonly MemoryStream _captured;
|
||||||
|
private bool _disposed;
|
||||||
|
|
||||||
|
public CapturedResponseStream(Stream inner, int capBytes)
|
||||||
|
{
|
||||||
|
_inner = inner ?? throw new ArgumentNullException(nameof(inner));
|
||||||
|
_capBytes = Math.Max(0, capBytes);
|
||||||
|
// Capture up to cap + 1 bytes so we can detect the over-cap case
|
||||||
|
// without growing the buffer further.
|
||||||
|
_captured = new MemoryStream();
|
||||||
|
}
|
||||||
|
|
||||||
|
public override bool CanRead => false;
|
||||||
|
public override bool CanSeek => false;
|
||||||
|
public override bool CanWrite => true;
|
||||||
|
|
||||||
|
public override long Length =>
|
||||||
|
throw new NotSupportedException("CapturedResponseStream is write-only.");
|
||||||
|
|
||||||
|
public override long Position
|
||||||
|
{
|
||||||
|
get => throw new NotSupportedException("CapturedResponseStream is write-only.");
|
||||||
|
set => throw new NotSupportedException("CapturedResponseStream is write-only.");
|
||||||
|
}
|
||||||
|
|
||||||
|
public override void Flush() => _inner.Flush();
|
||||||
|
|
||||||
|
public override Task FlushAsync(CancellationToken cancellationToken) =>
|
||||||
|
_inner.FlushAsync(cancellationToken);
|
||||||
|
|
||||||
|
public override int Read(byte[] buffer, int offset, int count) =>
|
||||||
|
throw new NotSupportedException("CapturedResponseStream is write-only.");
|
||||||
|
|
||||||
|
public override long Seek(long offset, SeekOrigin origin) =>
|
||||||
|
throw new NotSupportedException("CapturedResponseStream is write-only.");
|
||||||
|
|
||||||
|
public override void SetLength(long value) =>
|
||||||
|
throw new NotSupportedException("CapturedResponseStream is write-only.");
|
||||||
|
|
||||||
|
public override void Write(byte[] buffer, int offset, int count)
|
||||||
|
{
|
||||||
|
// Forward to the real sink FIRST — the client must never miss
|
||||||
|
// bytes if capture throws.
|
||||||
|
_inner.Write(buffer, offset, count);
|
||||||
|
CaptureBytes(buffer.AsSpan(offset, count));
|
||||||
|
}
|
||||||
|
|
||||||
|
public override void Write(ReadOnlySpan<byte> buffer)
|
||||||
|
{
|
||||||
|
_inner.Write(buffer);
|
||||||
|
CaptureBytes(buffer);
|
||||||
|
}
|
||||||
|
|
||||||
|
public override async Task WriteAsync(
|
||||||
|
byte[] buffer, int offset, int count, CancellationToken cancellationToken)
|
||||||
|
{
|
||||||
|
await _inner.WriteAsync(buffer.AsMemory(offset, count), cancellationToken)
|
||||||
|
.ConfigureAwait(false);
|
||||||
|
CaptureBytes(buffer.AsSpan(offset, count));
|
||||||
|
}
|
||||||
|
|
||||||
|
public override async ValueTask WriteAsync(
|
||||||
|
ReadOnlyMemory<byte> buffer, CancellationToken cancellationToken = default)
|
||||||
|
{
|
||||||
|
await _inner.WriteAsync(buffer, cancellationToken).ConfigureAwait(false);
|
||||||
|
CaptureBytes(buffer.Span);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Capture up to <c>cap + 1</c> bytes total into the private
|
||||||
|
/// <see cref="MemoryStream"/>. Once the cap quota is reached, further
|
||||||
|
/// bytes are silently dropped from the audit copy (the real sink has
|
||||||
|
/// already received them upstream of this call).
|
||||||
|
/// </summary>
|
||||||
|
private void CaptureBytes(ReadOnlySpan<byte> span)
|
||||||
|
{
|
||||||
|
if (span.Length == 0)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
var quota = (_capBytes + 1) - (int)_captured.Length;
|
||||||
|
if (quota <= 0)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
var take = Math.Min(quota, span.Length);
|
||||||
|
_captured.Write(span[..take]);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Returns the captured response body as a UTF-8 string (byte-safe
|
||||||
|
/// truncated to <c>cap</c> bytes) and a flag indicating whether the
|
||||||
|
/// audit copy hit the cap. Returns <c>(null, false)</c> when no bytes
|
||||||
|
/// were captured, mirroring the request-body empty contract.
|
||||||
|
/// </summary>
|
||||||
|
public (string? body, bool truncated) GetCapturedBody()
|
||||||
|
{
|
||||||
|
var length = (int)_captured.Length;
|
||||||
|
if (length == 0)
|
||||||
|
{
|
||||||
|
return (null, false);
|
||||||
|
}
|
||||||
|
var truncated = length > _capBytes;
|
||||||
|
var bytes = _captured.GetBuffer();
|
||||||
|
var bytesForString = truncated ? _capBytes : length;
|
||||||
|
var content = DecodeUtf8Bounded(bytes, bytesForString, cutAtValidBytes: truncated);
|
||||||
|
return (string.IsNullOrEmpty(content) ? null : content, truncated);
|
||||||
|
}
|
||||||
|
|
||||||
|
protected override void Dispose(bool disposing)
|
||||||
|
{
|
||||||
|
if (!_disposed)
|
||||||
|
{
|
||||||
|
if (disposing)
|
||||||
|
{
|
||||||
|
// Own only the capture stream; the inner sink belongs to
|
||||||
|
// the framework's response pipeline.
|
||||||
|
_captured.Dispose();
|
||||||
|
}
|
||||||
|
_disposed = true;
|
||||||
|
}
|
||||||
|
base.Dispose(disposing);
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -14,6 +14,9 @@
|
|||||||
<ItemGroup>
|
<ItemGroup>
|
||||||
<ProjectReference Include="../ScadaLink.Commons/ScadaLink.Commons.csproj" />
|
<ProjectReference Include="../ScadaLink.Commons/ScadaLink.Commons.csproj" />
|
||||||
<ProjectReference Include="../ScadaLink.Communication/ScadaLink.Communication.csproj" />
|
<ProjectReference Include="../ScadaLink.Communication/ScadaLink.Communication.csproj" />
|
||||||
|
<!-- AuditWriteMiddleware reads AuditLogOptions.InboundMaxBytes to bound
|
||||||
|
per-request request/response audit capture at the source. -->
|
||||||
|
<ProjectReference Include="../ScadaLink.AuditLog/ScadaLink.AuditLog.csproj" />
|
||||||
</ItemGroup>
|
</ItemGroup>
|
||||||
|
|
||||||
<ItemGroup>
|
<ItemGroup>
|
||||||
|
|||||||
@@ -45,7 +45,8 @@ public class AuditLogOptionsBindingTests
|
|||||||
"RedactSqlParamsMatching": "@token|@secret"
|
"RedactSqlParamsMatching": "@token|@secret"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"RetentionDays": 180
|
"RetentionDays": 180,
|
||||||
|
"InboundMaxBytes": 524288
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
""";
|
""";
|
||||||
@@ -64,6 +65,7 @@ public class AuditLogOptionsBindingTests
|
|||||||
Assert.Equal(4096, opts.DefaultCapBytes);
|
Assert.Equal(4096, opts.DefaultCapBytes);
|
||||||
Assert.Equal(32768, opts.ErrorCapBytes);
|
Assert.Equal(32768, opts.ErrorCapBytes);
|
||||||
Assert.Equal(180, opts.RetentionDays);
|
Assert.Equal(180, opts.RetentionDays);
|
||||||
|
Assert.Equal(524_288, opts.InboundMaxBytes);
|
||||||
|
|
||||||
// HeaderRedactList: the Microsoft.Extensions.Configuration list binder
|
// HeaderRedactList: the Microsoft.Extensions.Configuration list binder
|
||||||
// APPENDS to the default list, so we assert containment rather than
|
// APPENDS to the default list, so we assert containment rather than
|
||||||
|
|||||||
@@ -0,0 +1,53 @@
|
|||||||
|
using ScadaLink.AuditLog.Configuration;
|
||||||
|
|
||||||
|
namespace ScadaLink.AuditLog.Tests.Configuration;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Task 1 of <c>docs/plans/2026-05-23-inbound-api-full-response-audit.md</c>:
|
||||||
|
/// pins the <see cref="AuditLogOptions.InboundMaxBytes"/> default to 1 MiB and
|
||||||
|
/// the validator bounds to <c>[8 KiB, 16 MiB]</c>. The inbound channel needs a
|
||||||
|
/// much larger ceiling than the 8 KiB / 64 KiB default/error caps that other
|
||||||
|
/// channels use, but unbounded would let any caller flood the central
|
||||||
|
/// <c>AuditLog</c> table with arbitrarily large bodies — hence the upper bound.
|
||||||
|
/// Companion to <see cref="AuditLogOptionsTests"/> which covers the existing
|
||||||
|
/// cap-bytes + retention invariants.
|
||||||
|
/// </summary>
|
||||||
|
public class AuditLogOptionsValidatorTests
|
||||||
|
{
|
||||||
|
[Fact]
|
||||||
|
public void Validate_InboundMaxBytes_DefaultOptions_IsOneMebibyte()
|
||||||
|
{
|
||||||
|
// The doc'd default per docs/plans/2026-05-23-inbound-api-full-response-audit-design.md
|
||||||
|
// is 1 048 576 bytes (1 MiB). Pin it so a config drift is a test failure,
|
||||||
|
// not a silent operational surprise.
|
||||||
|
var opts = new AuditLogOptions();
|
||||||
|
Assert.Equal(1_048_576, opts.InboundMaxBytes);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Theory]
|
||||||
|
[InlineData(8_192)] // documented min
|
||||||
|
[InlineData(1_048_576)] // default
|
||||||
|
[InlineData(16_777_216)] // documented max
|
||||||
|
public void Validate_InboundMaxBytes_InRange_Passes(int value)
|
||||||
|
{
|
||||||
|
var validator = new AuditLogOptionsValidator();
|
||||||
|
var opts = new AuditLogOptions { InboundMaxBytes = value };
|
||||||
|
Assert.True(validator.Validate(null, opts).Succeeded);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Theory]
|
||||||
|
[InlineData(0)]
|
||||||
|
[InlineData(8_191)]
|
||||||
|
[InlineData(16_777_217)]
|
||||||
|
[InlineData(int.MaxValue)]
|
||||||
|
public void Validate_InboundMaxBytes_OutOfRange_Fails(int value)
|
||||||
|
{
|
||||||
|
var validator = new AuditLogOptionsValidator();
|
||||||
|
var opts = new AuditLogOptions { InboundMaxBytes = value };
|
||||||
|
var result = validator.Validate(null, opts);
|
||||||
|
Assert.False(result.Succeeded);
|
||||||
|
Assert.Contains(
|
||||||
|
result.Failures!,
|
||||||
|
f => f.Contains(nameof(AuditLogOptions.InboundMaxBytes), StringComparison.Ordinal));
|
||||||
|
}
|
||||||
|
}
|
||||||
133
tests/ScadaLink.AuditLog.Tests/Payload/InboundChannelCapTests.cs
Normal file
133
tests/ScadaLink.AuditLog.Tests/Payload/InboundChannelCapTests.cs
Normal file
@@ -0,0 +1,133 @@
|
|||||||
|
using System.Text;
|
||||||
|
using Microsoft.Extensions.Logging.Abstractions;
|
||||||
|
using Microsoft.Extensions.Options;
|
||||||
|
using ScadaLink.AuditLog.Configuration;
|
||||||
|
using ScadaLink.AuditLog.Payload;
|
||||||
|
using ScadaLink.Commons.Entities.Audit;
|
||||||
|
using ScadaLink.Commons.Types.Enums;
|
||||||
|
|
||||||
|
namespace ScadaLink.AuditLog.Tests.Payload;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Pins the docs/plans/2026-05-23-inbound-api-full-response-audit-design.md
|
||||||
|
/// inbound carve-out: ApiInbound rows use InboundMaxBytes (default 1 MiB) for
|
||||||
|
/// RequestSummary / ResponseSummary truncation, NOT DefaultCapBytes /
|
||||||
|
/// ErrorCapBytes. Other channels keep the existing caps.
|
||||||
|
/// </summary>
|
||||||
|
/// <remarks>
|
||||||
|
/// Uses a file-local <see cref="StaticMonitor"/> helper mirroring the
|
||||||
|
/// convention in the sibling Payload tests (TruncationTests,
|
||||||
|
/// FilterIntegrationTests, BodyRegexRedactionTests, etc.) — the
|
||||||
|
/// <c>TestOptionsMonitor<T></c> helper referenced by the plan is a
|
||||||
|
/// private nested class inside <c>AuditLogOptionsBindingTests</c> and thus
|
||||||
|
/// not reachable from this file.
|
||||||
|
/// </remarks>
|
||||||
|
public class InboundChannelCapTests
|
||||||
|
{
|
||||||
|
private static AuditEvent MakeInbound(
|
||||||
|
AuditStatus status,
|
||||||
|
string? request = null,
|
||||||
|
string? response = null) =>
|
||||||
|
new()
|
||||||
|
{
|
||||||
|
EventId = Guid.NewGuid(),
|
||||||
|
OccurredAtUtc = DateTime.UtcNow,
|
||||||
|
Channel = AuditChannel.ApiInbound,
|
||||||
|
Kind = AuditKind.InboundRequest,
|
||||||
|
Status = status,
|
||||||
|
RequestSummary = request,
|
||||||
|
ResponseSummary = response,
|
||||||
|
};
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void ApiInbound_Delivered_RequestBody_BelowInboundMaxBytes_NotTruncated()
|
||||||
|
{
|
||||||
|
// Body well above the legacy 8 KiB default cap but under the 1 MiB
|
||||||
|
// inbound ceiling — must NOT truncate.
|
||||||
|
var body = new string('a', 100_000);
|
||||||
|
var opts = new AuditLogOptions(); // defaults
|
||||||
|
var filter = new DefaultAuditPayloadFilter(
|
||||||
|
new StaticMonitor(opts),
|
||||||
|
NullLogger<DefaultAuditPayloadFilter>.Instance);
|
||||||
|
|
||||||
|
var result = filter.Apply(MakeInbound(AuditStatus.Delivered, request: body));
|
||||||
|
|
||||||
|
Assert.False(result.PayloadTruncated);
|
||||||
|
Assert.Equal(100_000, Encoding.UTF8.GetByteCount(result.RequestSummary!));
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void ApiInbound_Delivered_ResponseBody_BelowInboundMaxBytes_NotTruncated()
|
||||||
|
{
|
||||||
|
var body = new string('a', 100_000);
|
||||||
|
var opts = new AuditLogOptions();
|
||||||
|
var filter = new DefaultAuditPayloadFilter(
|
||||||
|
new StaticMonitor(opts),
|
||||||
|
NullLogger<DefaultAuditPayloadFilter>.Instance);
|
||||||
|
|
||||||
|
var result = filter.Apply(MakeInbound(AuditStatus.Delivered, response: body));
|
||||||
|
|
||||||
|
Assert.False(result.PayloadTruncated);
|
||||||
|
Assert.Equal(100_000, Encoding.UTF8.GetByteCount(result.ResponseSummary!));
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void ApiInbound_Failed_BodyAboveInboundMaxBytes_TruncatedToInboundMaxBytes()
|
||||||
|
{
|
||||||
|
// Even on error rows, the inbound cap is InboundMaxBytes (NOT ErrorCapBytes).
|
||||||
|
var opts = new AuditLogOptions { InboundMaxBytes = 16_384 };
|
||||||
|
var oversized = new string('z', 50_000);
|
||||||
|
var filter = new DefaultAuditPayloadFilter(
|
||||||
|
new StaticMonitor(opts),
|
||||||
|
NullLogger<DefaultAuditPayloadFilter>.Instance);
|
||||||
|
|
||||||
|
var result = filter.Apply(MakeInbound(AuditStatus.Failed, response: oversized));
|
||||||
|
|
||||||
|
Assert.True(result.PayloadTruncated);
|
||||||
|
Assert.True(Encoding.UTF8.GetByteCount(result.ResponseSummary!) <= 16_384);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void ApiOutbound_StillUsesDefaultCap_NotInboundMaxBytes()
|
||||||
|
{
|
||||||
|
// Regression guard: lifting the inbound cap MUST NOT change other
|
||||||
|
// channels. An ApiOutbound 100 KB body still hits the 8 KiB cap.
|
||||||
|
var opts = new AuditLogOptions();
|
||||||
|
var body = new string('a', 100_000);
|
||||||
|
var filter = new DefaultAuditPayloadFilter(
|
||||||
|
new StaticMonitor(opts),
|
||||||
|
NullLogger<DefaultAuditPayloadFilter>.Instance);
|
||||||
|
|
||||||
|
var evt = new AuditEvent
|
||||||
|
{
|
||||||
|
EventId = Guid.NewGuid(),
|
||||||
|
OccurredAtUtc = DateTime.UtcNow,
|
||||||
|
Channel = AuditChannel.ApiOutbound,
|
||||||
|
Kind = AuditKind.ApiCall,
|
||||||
|
Status = AuditStatus.Delivered,
|
||||||
|
RequestSummary = body,
|
||||||
|
};
|
||||||
|
var result = filter.Apply(evt);
|
||||||
|
|
||||||
|
Assert.True(result.PayloadTruncated);
|
||||||
|
Assert.True(Encoding.UTF8.GetByteCount(result.RequestSummary!) <= opts.DefaultCapBytes);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// IOptionsMonitor test double — returns the same snapshot on every read,
|
||||||
|
/// no change-token plumbing required for these tests. Mirrors the helper
|
||||||
|
/// used in <c>TruncationTests</c>, <c>FilterIntegrationTests</c>, etc.
|
||||||
|
/// </summary>
|
||||||
|
private sealed class StaticMonitor : IOptionsMonitor<AuditLogOptions>
|
||||||
|
{
|
||||||
|
private readonly AuditLogOptions _value;
|
||||||
|
|
||||||
|
public StaticMonitor(AuditLogOptions value) => _value = value;
|
||||||
|
|
||||||
|
public AuditLogOptions CurrentValue => _value;
|
||||||
|
|
||||||
|
public AuditLogOptions Get(string? name) => _value;
|
||||||
|
|
||||||
|
public IDisposable? OnChange(Action<AuditLogOptions, string?> listener) => null;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -4,6 +4,8 @@ using System.Text.Json;
|
|||||||
using Microsoft.AspNetCore.Http;
|
using Microsoft.AspNetCore.Http;
|
||||||
using Microsoft.AspNetCore.Routing;
|
using Microsoft.AspNetCore.Routing;
|
||||||
using Microsoft.Extensions.Logging.Abstractions;
|
using Microsoft.Extensions.Logging.Abstractions;
|
||||||
|
using Microsoft.Extensions.Options;
|
||||||
|
using ScadaLink.AuditLog.Configuration;
|
||||||
using ScadaLink.Commons.Entities.Audit;
|
using ScadaLink.Commons.Entities.Audit;
|
||||||
using ScadaLink.Commons.Interfaces.Services;
|
using ScadaLink.Commons.Interfaces.Services;
|
||||||
using ScadaLink.Commons.Types.Enums;
|
using ScadaLink.Commons.Types.Enums;
|
||||||
@@ -79,8 +81,32 @@ public class AuditWriteMiddlewareTests
|
|||||||
|
|
||||||
private static AuditWriteMiddleware CreateMiddleware(
|
private static AuditWriteMiddleware CreateMiddleware(
|
||||||
RequestDelegate next,
|
RequestDelegate next,
|
||||||
ICentralAuditWriter writer) =>
|
ICentralAuditWriter writer,
|
||||||
new(next, writer, NullLogger<AuditWriteMiddleware>.Instance);
|
AuditLogOptions? options = null) =>
|
||||||
|
new(
|
||||||
|
next,
|
||||||
|
writer,
|
||||||
|
NullLogger<AuditWriteMiddleware>.Instance,
|
||||||
|
new StaticAuditLogOptionsMonitor(options ?? new AuditLogOptions()));
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// File-local <see cref="IOptionsMonitor{TOptions}"/> test double — returns the
|
||||||
|
/// same snapshot on every read, no change-token plumbing required. Mirrors the
|
||||||
|
/// <c>StaticMonitor</c> pattern in
|
||||||
|
/// <c>tests/ScadaLink.AuditLog.Tests/Payload/InboundChannelCapTests.cs</c>.
|
||||||
|
/// </summary>
|
||||||
|
private sealed class StaticAuditLogOptionsMonitor : IOptionsMonitor<AuditLogOptions>
|
||||||
|
{
|
||||||
|
private readonly AuditLogOptions _value;
|
||||||
|
|
||||||
|
public StaticAuditLogOptionsMonitor(AuditLogOptions value) => _value = value;
|
||||||
|
|
||||||
|
public AuditLogOptions CurrentValue => _value;
|
||||||
|
|
||||||
|
public AuditLogOptions Get(string? name) => _value;
|
||||||
|
|
||||||
|
public IDisposable? OnChange(Action<AuditLogOptions, string?> listener) => null;
|
||||||
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------
|
// ---------------------------------------------------------------------
|
||||||
// 1. Happy path — InboundRequest/Delivered/HttpStatus 200
|
// 1. Happy path — InboundRequest/Delivered/HttpStatus 200
|
||||||
@@ -487,4 +513,180 @@ public class AuditWriteMiddlewareTests
|
|||||||
Assert.NotNull(evt.DurationMs);
|
Assert.NotNull(evt.DurationMs);
|
||||||
Assert.True(evt.DurationMs >= 0);
|
Assert.True(evt.DurationMs >= 0);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------
|
||||||
|
// Response body capture — Audit Log #23 (inbound full-response feature).
|
||||||
|
// Until the M5-deferred work landed, ResponseSummary was always null.
|
||||||
|
// These tests pin the new contract: the middleware wraps Response.Body,
|
||||||
|
// runs the pipeline, copies the buffered bytes back to the real stream,
|
||||||
|
// and stashes a UTF-8 string copy on ResponseSummary.
|
||||||
|
// ---------------------------------------------------------------------
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task ResponseBody_IsCaptured_OnResponseSummary()
|
||||||
|
{
|
||||||
|
var writer = new RecordingAuditWriter();
|
||||||
|
var ctx = BuildContext();
|
||||||
|
var responseJson = "{\"result\":42}";
|
||||||
|
var mw = CreateMiddleware(async hc =>
|
||||||
|
{
|
||||||
|
hc.Response.StatusCode = 200;
|
||||||
|
hc.Response.ContentType = "application/json";
|
||||||
|
await hc.Response.WriteAsync(responseJson);
|
||||||
|
}, writer);
|
||||||
|
|
||||||
|
await mw.InvokeAsync(ctx);
|
||||||
|
|
||||||
|
var evt = Assert.Single(writer.Events);
|
||||||
|
Assert.Equal(responseJson, evt.ResponseSummary);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task ResponseBody_IsForwardedToOriginalStream_DownstreamReadersSeeIt()
|
||||||
|
{
|
||||||
|
// Wrapping the response body must be TRANSPARENT — the real client
|
||||||
|
// stream still receives every byte the pipeline wrote.
|
||||||
|
var writer = new RecordingAuditWriter();
|
||||||
|
var ctx = BuildContext();
|
||||||
|
var captured = new MemoryStream();
|
||||||
|
ctx.Response.Body = captured; // simulate the client/test sink
|
||||||
|
|
||||||
|
var responseJson = "{\"ok\":true}";
|
||||||
|
var mw = CreateMiddleware(async hc =>
|
||||||
|
{
|
||||||
|
hc.Response.StatusCode = 200;
|
||||||
|
await hc.Response.WriteAsync(responseJson);
|
||||||
|
}, writer);
|
||||||
|
|
||||||
|
await mw.InvokeAsync(ctx);
|
||||||
|
|
||||||
|
Assert.Equal(responseJson, Encoding.UTF8.GetString(captured.ToArray()));
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task ResponseBody_Empty_LeavesResponseSummaryNull()
|
||||||
|
{
|
||||||
|
// No bytes written => null, not empty-string. Mirrors the request-body
|
||||||
|
// contract in ReadBufferedRequestBodyAsync.
|
||||||
|
var writer = new RecordingAuditWriter();
|
||||||
|
var ctx = BuildContext();
|
||||||
|
var mw = CreateMiddleware(hc =>
|
||||||
|
{
|
||||||
|
hc.Response.StatusCode = 204;
|
||||||
|
return Task.CompletedTask;
|
||||||
|
}, writer);
|
||||||
|
|
||||||
|
await mw.InvokeAsync(ctx);
|
||||||
|
|
||||||
|
var evt = Assert.Single(writer.Events);
|
||||||
|
Assert.Null(evt.ResponseSummary);
|
||||||
|
Assert.Equal(204, evt.HttpStatus);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task ResponseBody_OnHandlerThrow_BodyCapturedUpToTheThrow()
|
||||||
|
{
|
||||||
|
// If the handler writes some bytes then throws, the audit row still
|
||||||
|
// surfaces whatever the framework had flushed. The middleware re-throws
|
||||||
|
// (audit is best-effort, the request's error path stays authoritative).
|
||||||
|
var writer = new RecordingAuditWriter();
|
||||||
|
var ctx = BuildContext();
|
||||||
|
var boom = new InvalidOperationException("kaboom");
|
||||||
|
var mw = CreateMiddleware(async hc =>
|
||||||
|
{
|
||||||
|
hc.Response.StatusCode = 500;
|
||||||
|
await hc.Response.WriteAsync("partial");
|
||||||
|
throw boom;
|
||||||
|
}, writer);
|
||||||
|
|
||||||
|
var thrown = await Assert.ThrowsAsync<InvalidOperationException>(
|
||||||
|
() => mw.InvokeAsync(ctx));
|
||||||
|
Assert.Same(boom, thrown);
|
||||||
|
|
||||||
|
var evt = Assert.Single(writer.Events);
|
||||||
|
Assert.Equal(AuditStatus.Failed, evt.Status);
|
||||||
|
Assert.Equal("partial", evt.ResponseSummary);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------
|
||||||
|
// Bounded audit capture — memory safety follow-up. The capture site now
|
||||||
|
// honours AuditLogOptions.InboundMaxBytes at READ time (not just at
|
||||||
|
// filter-time), so a 500 MiB body cannot transiently allocate 500 MiB of
|
||||||
|
// string. The cap is local to the AUDIT copy; downstream readers and the
|
||||||
|
// real client still see every byte.
|
||||||
|
// ---------------------------------------------------------------------
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task RequestBody_AboveInboundMaxBytes_TruncatedToCap_PayloadTruncatedTrue()
|
||||||
|
{
|
||||||
|
// 4 KiB cap, 20 KB body — the audit copy must be UTF-8 byte-safe
|
||||||
|
// capped at 4 KiB AND PayloadTruncated must flip, while the
|
||||||
|
// downstream handler still sees the full 20 KB payload.
|
||||||
|
const int cap = 4096;
|
||||||
|
var bigBody = new string('a', 20_000);
|
||||||
|
var writer = new RecordingAuditWriter();
|
||||||
|
var ctx = BuildContext(body: bigBody);
|
||||||
|
|
||||||
|
string? observedAfterMiddleware = null;
|
||||||
|
var mw = CreateMiddleware(
|
||||||
|
async hc =>
|
||||||
|
{
|
||||||
|
using var reader = new StreamReader(hc.Request.Body);
|
||||||
|
observedAfterMiddleware = await reader.ReadToEndAsync();
|
||||||
|
hc.Response.StatusCode = 200;
|
||||||
|
},
|
||||||
|
writer,
|
||||||
|
options: new AuditLogOptions { InboundMaxBytes = cap });
|
||||||
|
|
||||||
|
await mw.InvokeAsync(ctx);
|
||||||
|
|
||||||
|
// (iii) Downstream handler still sees the FULL body — the cap applied
|
||||||
|
// only to the audit copy.
|
||||||
|
Assert.Equal(bigBody, observedAfterMiddleware);
|
||||||
|
|
||||||
|
var evt = Assert.Single(writer.Events);
|
||||||
|
// (i) Audit copy bounded at cap bytes (UTF-8 byte count).
|
||||||
|
Assert.NotNull(evt.RequestSummary);
|
||||||
|
Assert.True(
|
||||||
|
Encoding.UTF8.GetByteCount(evt.RequestSummary!) <= cap,
|
||||||
|
$"RequestSummary byte count {Encoding.UTF8.GetByteCount(evt.RequestSummary!)} exceeded cap {cap}");
|
||||||
|
// (ii) Truncation flag set by the middleware (the filter will OR its
|
||||||
|
// own determination on top, but the middleware MUST set it itself).
|
||||||
|
Assert.True(evt.PayloadTruncated);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task ResponseBody_AboveInboundMaxBytes_TruncatedToCap_ClientStillReceivesAllBytes_PayloadTruncatedTrue()
|
||||||
|
{
|
||||||
|
// 4 KiB cap, 20 KB response — the test sink (acts as the real client)
|
||||||
|
// MUST receive all 20 KB while the audit copy is bounded at 4 KiB.
|
||||||
|
const int cap = 4096;
|
||||||
|
var bigResponse = new string('b', 20_000);
|
||||||
|
var writer = new RecordingAuditWriter();
|
||||||
|
var ctx = BuildContext();
|
||||||
|
var captured = new MemoryStream();
|
||||||
|
ctx.Response.Body = captured; // stand-in for the client sink
|
||||||
|
|
||||||
|
var mw = CreateMiddleware(
|
||||||
|
async hc =>
|
||||||
|
{
|
||||||
|
hc.Response.StatusCode = 200;
|
||||||
|
await hc.Response.WriteAsync(bigResponse);
|
||||||
|
},
|
||||||
|
writer,
|
||||||
|
options: new AuditLogOptions { InboundMaxBytes = cap });
|
||||||
|
|
||||||
|
await mw.InvokeAsync(ctx);
|
||||||
|
|
||||||
|
// Client sink received every byte — the forwarding wrap is transparent.
|
||||||
|
Assert.Equal(bigResponse, Encoding.UTF8.GetString(captured.ToArray()));
|
||||||
|
|
||||||
|
var evt = Assert.Single(writer.Events);
|
||||||
|
// Audit copy bounded at cap bytes.
|
||||||
|
Assert.NotNull(evt.ResponseSummary);
|
||||||
|
Assert.True(
|
||||||
|
Encoding.UTF8.GetByteCount(evt.ResponseSummary!) <= cap,
|
||||||
|
$"ResponseSummary byte count {Encoding.UTF8.GetByteCount(evt.ResponseSummary!)} exceeded cap {cap}");
|
||||||
|
Assert.True(evt.PayloadTruncated);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -6,6 +6,8 @@ using Microsoft.AspNetCore.TestHost;
|
|||||||
using Microsoft.Extensions.DependencyInjection;
|
using Microsoft.Extensions.DependencyInjection;
|
||||||
using Microsoft.Extensions.Hosting;
|
using Microsoft.Extensions.Hosting;
|
||||||
using Microsoft.Extensions.Logging.Abstractions;
|
using Microsoft.Extensions.Logging.Abstractions;
|
||||||
|
using Microsoft.Extensions.Options;
|
||||||
|
using ScadaLink.AuditLog.Configuration;
|
||||||
using ScadaLink.Commons.Entities.Audit;
|
using ScadaLink.Commons.Entities.Audit;
|
||||||
using ScadaLink.Commons.Interfaces.Services;
|
using ScadaLink.Commons.Interfaces.Services;
|
||||||
using ScadaLink.Commons.Types.Enums;
|
using ScadaLink.Commons.Types.Enums;
|
||||||
@@ -145,7 +147,8 @@ public class MiddlewareOrderTests
|
|||||||
// instantiates the type correctly.
|
// instantiates the type correctly.
|
||||||
_ => Task.CompletedTask,
|
_ => Task.CompletedTask,
|
||||||
writer,
|
writer,
|
||||||
NullLogger<AuditWriteMiddleware>.Instance));
|
NullLogger<AuditWriteMiddleware>.Instance,
|
||||||
|
new StaticAuditLogOptionsMonitor(new AuditLogOptions())));
|
||||||
services.AddRouting();
|
services.AddRouting();
|
||||||
services.AddAuthorization();
|
services.AddAuthorization();
|
||||||
services.AddAuthentication("TestScheme")
|
services.AddAuthentication("TestScheme")
|
||||||
@@ -233,4 +236,22 @@ public class MiddlewareOrderTests
|
|||||||
return Task.FromResult(AuthenticateResult.Success(ticket));
|
return Task.FromResult(AuthenticateResult.Success(ticket));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// File-local <see cref="IOptionsMonitor{TOptions}"/> test double — returns the
|
||||||
|
/// same snapshot on every read. Mirrors the helper in
|
||||||
|
/// <c>AuditWriteMiddlewareTests</c>.
|
||||||
|
/// </summary>
|
||||||
|
private sealed class StaticAuditLogOptionsMonitor : IOptionsMonitor<AuditLogOptions>
|
||||||
|
{
|
||||||
|
private readonly AuditLogOptions _value;
|
||||||
|
|
||||||
|
public StaticAuditLogOptionsMonitor(AuditLogOptions value) => _value = value;
|
||||||
|
|
||||||
|
public AuditLogOptions CurrentValue => _value;
|
||||||
|
|
||||||
|
public AuditLogOptions Get(string? name) => _value;
|
||||||
|
|
||||||
|
public IDisposable? OnChange(Action<AuditLogOptions, string?> listener) => null;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user