11 Commits

Author SHA1 Message Date
Joseph Doherty e3345a0fc1 Merge branch 'feature/inbound-api-full-response-audit': inbound API full request/response audit capture
Inbound API audit rows (Channel = ApiInbound) now capture request and
response bodies in full up to a configurable 1 MiB per-body ceiling
(AuditLog:InboundMaxBytes), instead of the global 8 KiB / 64 KiB caps
that other audit channels use. Implements the M5-deferred response-body
capture in AuditWriteMiddleware via a write-only Stream wrapper that
forwards every byte to the framework's response sink while bounding the
audit copy at the capture site (ArrayPool-backed for the request side).
Other channels untouched.
2026-05-23 09:49:52 -04:00
Joseph Doherty e6ccee1a16 refactor(inboundapi): pool the request audit buffer + reset Position in finally 2026-05-23 09:46:53 -04:00
Joseph Doherty e567eb334c docs(audit): drop stale InboundAuthFailure exclusion from design doc
The design doc claimed (in two places) that InboundAuthFailure rows
were excluded from the inbound full-body carve-out — but the actual
implementation gates the carve-out on Channel == ApiInbound, NOT Kind.
Every audit row the InboundAPI middleware emits (whether
Kind = InboundRequest or Kind = InboundAuthFailure) carries
Channel = ApiInbound, so both Kinds receive the inbound ceiling. That
is the intended behaviour: an auth-failure row's request body is
exactly the body the operator wants to see in full when investigating
a rejected request.

Update both occurrences (Decision block + Not in Scope block) to say
the carve-out applies to all Channel = ApiInbound rows regardless of
Kind. Pure documentation change — no code drift.
2026-05-23 09:25:23 -04:00
Joseph Doherty 7d87994ac0 feat(inboundapi): bound audit capture at InboundMaxBytes (memory safety)
AuditWriteMiddleware previously buffered the FULL request and response
bodies into memory and only let DefaultAuditPayloadFilter trim them
after persistence. A 500 MiB upload allocated 500 MiB of MemoryStream
plus 1 GiB of UTF-16 string transiently before the filter pulled it
back to the 1 MiB inbound ceiling — the cap was real on the persisted
row but not at the capture site.

Inject IOptionsMonitor<AuditLogOptions> and read InboundMaxBytes
per-request (same convention as DefaultAuditPayloadFilter so a live
config change picks up the next request). The request reader now pulls
at most cap + 1 bytes into a UTF-8 byte-safe-truncated string and
rewinds the stream so the endpoint handler still sees the full body.
The response wrap is a new CapturedResponseStream that forwards every
Write / WriteAsync to the real sink (the client still receives all
bytes) while capturing at most cap + 1 bytes for the audit copy. The
middleware now sets PayloadTruncated itself when either body hit the
cap; the filter still OR's its own determination on top.

Adds a project reference from ScadaLink.InboundAPI to
ScadaLink.AuditLog so AuditLogOptions resolves. AuditLog does NOT
reference InboundAPI back, so no cycle is introduced.

Tests:
 - All 21 existing AuditWriteMiddlewareTests still pass (the helper
   gains an optional AuditLogOptions argument; default is the standard
   1 MiB ceiling so existing small-body tests are unaffected).
 - MiddlewareOrderTests' construction site updated for the new ctor
   arg; a StaticAuditLogOptionsMonitor file-local double mirrors the
   InboundChannelCapTests pattern.
 - New RequestBody_AboveInboundMaxBytes_TruncatedToCap_PayloadTruncatedTrue
   pins a 4 KiB cap against a 20 KB body: audit copy <= 4 KiB,
   PayloadTruncated = true, downstream handler reads the full 20 KB.
 - New ResponseBody_AboveInboundMaxBytes_TruncatedToCap_ClientStillReceivesAllBytes_PayloadTruncatedTrue
   pins the same shape on the response side: client sink receives
   20 KB, audit copy <= 4 KiB, PayloadTruncated = true.

InboundAPI test count: 133 -> 135.
2026-05-23 09:25:00 -04:00
Joseph Doherty 651c4b6833 docs(inboundapi): note request/response bodies captured in full to InboundMaxBytes 2026-05-23 06:09:10 -04:00
Joseph Doherty 7efb004a02 docs(audit): schema + Payload Capture Policy note inbound full-body carve-out 2026-05-23 06:07:11 -04:00
Joseph Doherty a8d2e13d4e feat(inboundapi): AuditWriteMiddleware captures response body on ApiInbound audit rows 2026-05-23 06:00:24 -04:00
Joseph Doherty 7b619d711d feat(auditlog): payload filter uses InboundMaxBytes for ApiInbound rows 2026-05-23 05:55:03 -04:00
Joseph Doherty c5b27361c0 feat(auditlog): add AuditLog:InboundMaxBytes option (default 1 MiB, [8 KiB, 16 MiB]) 2026-05-23 05:39:50 -04:00
Joseph Doherty 441ec087a7 docs(audit): implementation plan — full request/response capture for inbound API audit rows
Plan companion to the 2026-05-23 design doc. Seven tasks (#0 prep, #1-3
implementation TDD, #4-5 doc updates, #6 final sweep). Tracks via
.tasks.json for resumability.
2026-05-23 05:36:08 -04:00
Joseph Doherty 0670864160 docs(audit): design — full request/response capture for inbound API rows
Carve-out from Payload Capture Policy: ApiInbound rows capture
RequestSummary and ResponseSummary in full up to a configurable 1 MB
per-body ceiling (AuditLog:InboundMaxBytes), instead of the global 8 KB /
64 KB caps. No schema change; existing redaction (headers + per-target
body redactors) still applies before persistence.
2026-05-23 05:28:34 -04:00
15 changed files with 1658 additions and 36 deletions
@@ -0,0 +1,148 @@
# Inbound API: Full Request/Response Capture in Audit Log
**Date:** 2026-05-23
**Status:** Approved (brainstorming complete)
**Affects:** Component-AuditLog (#23), Component-InboundAPI (#14)
## Problem
Today the centralized Audit Log captures inbound API request and response bodies
into `RequestSummary` / `ResponseSummary`, but with the global Payload Capture
Policy cap — 8 KB by default, 64 KB on error rows. For inbound API traffic this
is too tight: operators routinely need to replay exactly what an external caller
sent and exactly what we returned. Truncation defeats both replay and the most
common "why did this script see that input / what did the caller actually
receive" debugging path.
## Decision
For `Channel = ApiInbound` rows only, capture `RequestSummary` and
`ResponseSummary` verbatim up to a hard per-body ceiling of **1 MB**
(configurable). The 8 KB / 64 KB default/error caps that apply to other channels
do not apply here. The carve-out is channel-scoped (NOT kind-scoped): every
`Channel = ApiInbound` row uses the inbound ceiling regardless of `Kind`, so
`InboundAuthFailure` rows pick up the same ceiling as `InboundRequest`. All
other channels (`ApiOutbound`, `DbOutbound`, `Notification`, cached-call
lifecycle) keep the existing policy unchanged.
## Capture Policy Change
The Payload Capture Policy in `Component-AuditLog.md` gains an Inbound API
carve-out:
> **Inbound API exception.** For `Channel = ApiInbound`, `RequestSummary` and
> `ResponseSummary` are captured in full up to a per-body hard ceiling of 1 MB
> (configurable via `AuditLog:InboundMaxBytes`; default 1 048 576 bytes; min
> 8 192; max 16 777 216). The 8 KB / 64 KB default/error caps that apply to
> other channels do not apply here. `PayloadTruncated = 1` is set only when the
> 1 MB ceiling is hit — verbatim capture is the normal case.
The rest of the policy is unchanged:
- Header redact list (`Authorization`, `Cookie`, `Set-Cookie`, `X-API-Key`,
configured regex) still applies.
- Per-target body redactors (regex → replacement, keyed by inbound method name)
still run before persistence.
- The redactor-error safety net (`<redacted: redactor error>` plus
`AuditRedactionFailure` health metric increment) still applies.
- UTF-8 byte-safe truncation when the 1 MB ceiling *is* hit.
The ceiling applies independently to the request body and the response body —
each gets its own 1 MB budget on a given audit row.
## Schema
No schema change. `RequestSummary` and `ResponseSummary` are already
`nvarchar(max)`; SQL Server transparently stores LOB content out-of-row, so
larger row payloads are paid for only when the column is read. Only the column
description text changes to reflect the inbound carve-out.
## Ingestion Path
Unchanged. Inbound rows are already a central direct-write from the request-
handler middleware via `ICentralAuditWriter` before the HTTP response is
flushed, and audit-write failure is already fail-soft (logged + increments
`CentralAuditWriteFailures`, never fails the user-facing request).
The only code change at the write site is the cap selection:
```text
maxBytes = channel == ApiInbound
? options.InboundMaxBytes // default 1 MB
: isErrorRow ? 64*1024 : 8*1024; // existing policy
```
Redactors run before the cap; the cap is the final byte-budget step before the
INSERT.
## Configuration
New option on the existing `AuditLog` options class:
| Key | Default | Min | Max | Description |
|---|---|---|---|---|
| `AuditLog:InboundMaxBytes` | `1048576` | `8192` | `16777216` | Per-body ceiling for `ApiInbound` `RequestSummary` / `ResponseSummary`. Truncation past this is the only case where `PayloadTruncated` is set on an inbound row. |
Bounds enforced on options binding; out-of-range values fail startup with the
same "options validation" path used for other AuditLog settings.
## Doc Edits
1. **`Component-AuditLog.md`**
- `RequestSummary` and `ResponseSummary` rows in the schema table: amend
descriptions to note the `ApiInbound` carve-out (full capture up to
`InboundMaxBytes`, default 1 MB).
- Payload Capture Policy section: add the **Inbound API exception**
paragraph above; add `AuditLog:InboundMaxBytes` to the configuration knobs
list.
2. **`Component-InboundAPI.md`**
- Line ~119 (audit row description): "truncated request/response bodies per
the Audit Log capture policy" → "request/response bodies captured in full
up to the configured `AuditLog:InboundMaxBytes` ceiling (default 1 MB);
`PayloadTruncated = 1` only when that ceiling is hit".
- Line ~202 (Dependencies → Audit Log): mirror the wording adjustment.
## Operational Trade-offs
- **Storage growth.** At 365-day retention, full-body capture on every inbound
request can grow `AuditLog` significantly compared to today's 8 KB cap.
Operators tune by lowering `InboundMaxBytes`, shortening retention via
`AuditLog:RetentionDays`, or — once per-target redaction is configured for
chatty methods — applying body redactors to drop noise. Monthly partition
purge keeps reclamation cheap regardless of row size.
- **No new health metric.** Hitting the 1 MB ceiling is reflected in the
existing `PayloadTruncated` bit; no separate counter in v1. If ceiling-hits
become a real operational signal, an `AuditInboundCeilingHits` metric can be
added later without schema change.
- **Append-only and audit role.** The `scadalink_audit_writer` role already
permits `INSERT` only — full-body rows don't change the security model.
## Not in Scope (Deferred)
- **Structured response capture.** `ResponseSummary` stays a single string;
response status code remains in `HttpStatus`. No separate columns for response
headers or content type. Inbound request headers remain uncaptured.
- **Per-method opt-out** from full capture. If specific methods produce
routinely-huge responses, operators use the existing per-target body redactor
to compress them, or lower the global ceiling.
- **Changes to other channels' caps.** `ApiOutbound`, `DbOutbound`,
`Notification`, and cached-call lifecycle rows keep the existing 8 KB / 64 KB
policy. (`InboundAuthFailure` rows carry `Channel = ApiInbound` and so fall
under the inbound ceiling like every other inbound row.)
## Acceptance Criteria
- [ ] `AuditLog:InboundMaxBytes` option exists on the AuditLog options class,
with the documented default and bounds, validated at startup.
- [ ] Inbound request middleware writes `RequestSummary` and `ResponseSummary`
using the inbound ceiling instead of the 8 KB / 64 KB defaults.
- [ ] Other channels' rows (e.g. an `ApiOutbound.ApiCall` over the limit) still
truncate at 8 KB (64 KB on error rows) — regression-tested.
- [ ] `PayloadTruncated = 1` on an inbound row iff request body or response
body exceeded `InboundMaxBytes`.
- [ ] Header redaction list and per-target body redactors still apply to
inbound rows.
- [ ] Redactor failure on an inbound row still produces `<redacted: redactor
error>` and increments `AuditRedactionFailure`.
- [ ] `Component-AuditLog.md` and `Component-InboundAPI.md` updated as
described in **Doc Edits**.
@@ -0,0 +1,745 @@
# Inbound API: Full Request/Response Audit Capture — Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:subagent-driven-development to execute this plan task-by-task (fresh implementer per task + spec review + code-quality review).
**Goal:** Inbound API audit rows (`AuditChannel.ApiInbound`) capture the FULL request and response body verbatim up to a configurable 1 MB per-body ceiling, instead of the global 8 KB / 64 KB caps. Other channels are untouched.
**Architecture:** Two changes. (1) `AuditLogOptions` gains an `InboundMaxBytes` knob; `DefaultAuditPayloadFilter` branches on `Channel == ApiInbound` to use it. (2) `AuditWriteMiddleware` finally implements the M5-deferred response-body capture — wraps `HttpContext.Response.Body` with a buffering `MemoryStream` swap, reads it after the pipeline runs, restores and flushes the original body. The redaction stages (headers, body regexes, SQL params) keep their existing semantics; only the truncation cap changes for ApiInbound rows. Validated design: `docs/plans/2026-05-23-inbound-api-full-response-audit-design.md`.
**Tech Stack:** .NET 10, xUnit, ASP.NET Core Minimal API, `Microsoft.Extensions.Options.IOptionsMonitor`.
**Ground rules (every task):** create + work on branch `feature/inbound-api-full-response-audit` — never commit to `main`. TDD: failing test first, then minimal implementation, then verify. Edit in place; never edit `infra/*`, `alog.md`, or `docker/*` unless a task names them (none here). Stage with explicit `git add <path>` — never `git add .` / `commit -am`. Solution stays green: `dotnet build ScadaLink.slnx` 0 warnings (`TreatWarningsAsErrors` on). Do not push.
---
## Task 0: Prep — branch, baseline build
**Files:** none.
**Steps:**
1. `git status --short` — confirm you are starting from the `main` revision that already contains commit `0670864` (`docs(audit): design — full request/response capture for inbound API rows`).
2. `git checkout -b feature/inbound-api-full-response-audit`.
3. `git branch --show-current` — expect `feature/inbound-api-full-response-audit`.
4. `dotnet build ScadaLink.slnx` from repo root — expect 0 warnings, 0 errors.
**Acceptance:** on the feature branch; solution builds clean.
**Commit:** none (no changes yet).
---
## Task 1: Add `InboundMaxBytes` to `AuditLogOptions` (TDD)
**What:** New `int InboundMaxBytes` property on `AuditLogOptions` with default 1 048 576 bytes, validated to `[8192, 16777216]`.
**Files:**
- Modify: `src/ScadaLink.AuditLog/Configuration/AuditLogOptions.cs` — add property + XML doc.
- Modify: `src/ScadaLink.AuditLog/Configuration/AuditLogOptionsValidator.cs` — add min/max constants + validation branch.
- Modify: `tests/ScadaLink.AuditLog.Tests/Configuration/AuditLogOptionsBindingTests.cs` — extend the binding test to assert the new field round-trips from JSON.
- Test (new): `tests/ScadaLink.AuditLog.Tests/Configuration/AuditLogOptionsValidatorTests.cs` — if this file does not exist, create it with the four cases below; if a validator-tests file already exists (search for it under `tests/ScadaLink.AuditLog.Tests/Configuration/`), extend it instead.
**Step 1: Write the failing tests**
Add to a validator-tests file (create if missing — namespace `ScadaLink.AuditLog.Tests.Configuration`):
```csharp
public class AuditLogOptionsValidatorTests
{
[Fact]
public void Validate_InboundMaxBytes_DefaultOptions_IsOneMebibyte()
{
// The doc'd default per docs/plans/2026-05-23-inbound-api-full-response-audit-design.md
// is 1 048 576 bytes (1 MiB). Pin it so a config drift is a test failure,
// not a silent operational surprise.
var opts = new AuditLogOptions();
Assert.Equal(1_048_576, opts.InboundMaxBytes);
}
[Theory]
[InlineData(8_192)] // documented min
[InlineData(1_048_576)] // default
[InlineData(16_777_216)] // documented max
public void Validate_InboundMaxBytes_InRange_Passes(int value)
{
var validator = new AuditLogOptionsValidator();
var opts = new AuditLogOptions { InboundMaxBytes = value };
Assert.True(validator.Validate(null, opts).Succeeded);
}
[Theory]
[InlineData(0)]
[InlineData(8_191)]
[InlineData(16_777_217)]
[InlineData(int.MaxValue)]
public void Validate_InboundMaxBytes_OutOfRange_Fails(int value)
{
var validator = new AuditLogOptionsValidator();
var opts = new AuditLogOptions { InboundMaxBytes = value };
var result = validator.Validate(null, opts);
Assert.False(result.Succeeded);
Assert.Contains(
result.Failures!,
f => f.Contains(nameof(AuditLogOptions.InboundMaxBytes), StringComparison.Ordinal));
}
}
```
Extend `AuditLogOptionsBindingTests.AuditLog_Section_Binds_AllFields` — add `"InboundMaxBytes": 524288` to the JSON literal and a matching `Assert.Equal(524_288, opts.InboundMaxBytes)`.
**Step 2: Run tests — confirm they fail**
```
dotnet test tests/ScadaLink.AuditLog.Tests \
--filter "FullyQualifiedName~AuditLogOptionsValidatorTests|FullyQualifiedName~AuditLogOptionsBindingTests"
```
Expected: the new validator tests fail (no `InboundMaxBytes` property), the binding test fails (property does not bind).
**Step 3: Add the property**
In `AuditLogOptions.cs`, insert after `RetentionDays`:
```csharp
/// <summary>
/// Per-body byte ceiling applied to <see cref="AuditEvent.RequestSummary"/> and
/// <see cref="AuditEvent.ResponseSummary"/> for <see cref="AuditChannel.ApiInbound"/> rows
/// (default 1 MiB). The 8 KiB / 64 KiB default/error caps that apply to other channels
/// do not apply here — inbound traffic captures verbatim up to this ceiling and only
/// then sets <see cref="AuditEvent.PayloadTruncated"/>. See
/// <c>docs/plans/2026-05-23-inbound-api-full-response-audit-design.md</c>.
/// </summary>
public int InboundMaxBytes { get; set; } = 1_048_576;
```
**Step 4: Add the validator branch**
In `AuditLogOptionsValidator.cs`, add the constants beside the existing retention bounds:
```csharp
public const int MinInboundMaxBytes = 8_192;
public const int MaxInboundMaxBytes = 16_777_216;
```
Add the validation block inside `Validate` (after the retention check, before the `return`):
```csharp
if (options.InboundMaxBytes < MinInboundMaxBytes || options.InboundMaxBytes > MaxInboundMaxBytes)
{
failures.Add(
$"AuditLog:{nameof(AuditLogOptions.InboundMaxBytes)} ({options.InboundMaxBytes}) " +
$"must be in [{MinInboundMaxBytes}, {MaxInboundMaxBytes}] bytes.");
}
```
**Step 5: Run tests — confirm they pass**
```
dotnet test tests/ScadaLink.AuditLog.Tests
```
Expected: all green, including the extended binding test and the new validator tests.
**Step 6: Build the whole solution**
```
dotnet build ScadaLink.slnx
```
Expected: 0 warnings, 0 errors.
**Step 7: Commit**
```
git add src/ScadaLink.AuditLog/Configuration/AuditLogOptions.cs \
src/ScadaLink.AuditLog/Configuration/AuditLogOptionsValidator.cs \
tests/ScadaLink.AuditLog.Tests/Configuration/AuditLogOptionsBindingTests.cs \
tests/ScadaLink.AuditLog.Tests/Configuration/AuditLogOptionsValidatorTests.cs
git commit -m "feat(auditlog): add AuditLog:InboundMaxBytes option (default 1 MiB, [8 KiB, 16 MiB])"
```
---
## Task 2: Wire `InboundMaxBytes` into `DefaultAuditPayloadFilter` (TDD)
**What:** When `AuditEvent.Channel == AuditChannel.ApiInbound`, the filter selects `InboundMaxBytes` as the truncation cap instead of `DefaultCapBytes` / `ErrorCapBytes`. Redaction stages run unchanged.
**Files:**
- Modify: `src/ScadaLink.AuditLog/Payload/DefaultAuditPayloadFilter.cs` — change the one cap-selection line.
- Test (new): `tests/ScadaLink.AuditLog.Tests/Payload/InboundChannelCapTests.cs` — pin the new behaviour.
**Step 1: Write the failing tests**
Create `tests/ScadaLink.AuditLog.Tests/Payload/InboundChannelCapTests.cs`:
```csharp
using System.Text;
using Microsoft.Extensions.Logging.Abstractions;
using ScadaLink.AuditLog.Configuration;
using ScadaLink.AuditLog.Payload;
using ScadaLink.AuditLog.Tests.Configuration; // for TestOptionsMonitor — confirm namespace via existing file
using ScadaLink.Commons.Entities.Audit;
using ScadaLink.Commons.Types.Enums;
namespace ScadaLink.AuditLog.Tests.Payload;
/// <summary>
/// Pins the docs/plans/2026-05-23-inbound-api-full-response-audit-design.md
/// inbound carve-out: ApiInbound rows use InboundMaxBytes (default 1 MiB) for
/// RequestSummary / ResponseSummary truncation, NOT DefaultCapBytes /
/// ErrorCapBytes. Other channels keep the existing caps.
/// </summary>
public class InboundChannelCapTests
{
private static AuditEvent MakeInbound(
AuditStatus status,
string? request = null,
string? response = null) =>
new()
{
EventId = Guid.NewGuid(),
OccurredAtUtc = DateTime.UtcNow,
Channel = AuditChannel.ApiInbound,
Kind = status == AuditStatus.Delivered
? AuditKind.InboundRequest
: AuditKind.InboundRequest,
Status = status,
RequestSummary = request,
ResponseSummary = response,
};
[Fact]
public void ApiInbound_Delivered_RequestBody_BelowInboundMaxBytes_NotTruncated()
{
// Body well above the legacy 8 KiB default cap but under the 1 MiB
// inbound ceiling — must NOT truncate.
var body = new string('a', 100_000);
var opts = new AuditLogOptions(); // defaults
var filter = new DefaultAuditPayloadFilter(
new TestOptionsMonitor<AuditLogOptions>(opts),
NullLogger<DefaultAuditPayloadFilter>.Instance);
var result = filter.Apply(MakeInbound(AuditStatus.Delivered, request: body));
Assert.False(result.PayloadTruncated);
Assert.Equal(body.Length, result.RequestSummary!.Length);
}
[Fact]
public void ApiInbound_Delivered_ResponseBody_BelowInboundMaxBytes_NotTruncated()
{
var body = new string('a', 100_000);
var opts = new AuditLogOptions();
var filter = new DefaultAuditPayloadFilter(
new TestOptionsMonitor<AuditLogOptions>(opts),
NullLogger<DefaultAuditPayloadFilter>.Instance);
var result = filter.Apply(MakeInbound(AuditStatus.Delivered, response: body));
Assert.False(result.PayloadTruncated);
Assert.Equal(body.Length, result.ResponseSummary!.Length);
}
[Fact]
public void ApiInbound_Failed_BodyAboveInboundMaxBytes_TruncatedToInboundMaxBytes()
{
// Even on error rows, the inbound cap is InboundMaxBytes (NOT ErrorCapBytes).
var opts = new AuditLogOptions { InboundMaxBytes = 16_384 };
var oversized = new string('z', 50_000);
var filter = new DefaultAuditPayloadFilter(
new TestOptionsMonitor<AuditLogOptions>(opts),
NullLogger<DefaultAuditPayloadFilter>.Instance);
var result = filter.Apply(MakeInbound(AuditStatus.Failed, response: oversized));
Assert.True(result.PayloadTruncated);
Assert.True(Encoding.UTF8.GetByteCount(result.ResponseSummary!) <= 16_384);
}
[Fact]
public void ApiOutbound_StillUsesDefaultCap_NotInboundMaxBytes()
{
// Regression guard: lifting the inbound cap MUST NOT change other
// channels. An ApiOutbound 100 KB body still hits the 8 KiB cap.
var opts = new AuditLogOptions();
var body = new string('a', 100_000);
var filter = new DefaultAuditPayloadFilter(
new TestOptionsMonitor<AuditLogOptions>(opts),
NullLogger<DefaultAuditPayloadFilter>.Instance);
var evt = new AuditEvent
{
EventId = Guid.NewGuid(),
OccurredAtUtc = DateTime.UtcNow,
Channel = AuditChannel.ApiOutbound,
Kind = AuditKind.ApiCall,
Status = AuditStatus.Delivered,
RequestSummary = body,
};
var result = filter.Apply(evt);
Assert.True(result.PayloadTruncated);
Assert.True(Encoding.UTF8.GetByteCount(result.RequestSummary!) <= opts.DefaultCapBytes);
}
}
```
Verify the `TestOptionsMonitor<T>` helper lives in `tests/ScadaLink.AuditLog.Tests/`. Grep:
```
grep -rn "class TestOptionsMonitor" tests/ScadaLink.AuditLog.Tests
```
If its namespace differs from `ScadaLink.AuditLog.Tests.Configuration`, update the `using` accordingly.
**Step 2: Run tests — confirm they fail**
```
dotnet test tests/ScadaLink.AuditLog.Tests \
--filter "FullyQualifiedName~InboundChannelCapTests"
```
Expected: the inbound tests fail (filter still applies the 8 KiB cap to ApiInbound). The `ApiOutbound_StillUsesDefaultCap` test SHOULD pass even before the change — that's the regression baseline and stays green after.
**Step 3: Add the channel branch to the filter**
In `DefaultAuditPayloadFilter.cs`, replace the single line:
```csharp
var cap = IsErrorStatus(rawEvent.Status) ? opts.ErrorCapBytes : opts.DefaultCapBytes;
```
with:
```csharp
// Inbound API gets a dedicated, larger ceiling — request/response bodies are
// captured verbatim up to InboundMaxBytes (default 1 MiB) so support can
// replay exactly what the caller sent and what we returned. Other channels
// keep the global 8 KiB / 64 KiB policy.
// See docs/plans/2026-05-23-inbound-api-full-response-audit-design.md.
var cap = rawEvent.Channel == AuditChannel.ApiInbound
? opts.InboundMaxBytes
: (IsErrorStatus(rawEvent.Status) ? opts.ErrorCapBytes : opts.DefaultCapBytes);
```
**Step 4: Run tests — confirm they pass**
```
dotnet test tests/ScadaLink.AuditLog.Tests
```
Expected: all green. The existing `FilterIntegrationTests`, `BodyRegexRedactionTests`, `RedactionSafetyNetTests`, `SqlParamRedactionTests`, etc., MUST stay passing — the change is channel-scoped and the non-inbound cases never see the new branch.
**Step 5: Build the whole solution**
```
dotnet build ScadaLink.slnx
```
Expected: 0 warnings.
**Step 6: Commit**
```
git add src/ScadaLink.AuditLog/Payload/DefaultAuditPayloadFilter.cs \
tests/ScadaLink.AuditLog.Tests/Payload/InboundChannelCapTests.cs
git commit -m "feat(auditlog): payload filter uses InboundMaxBytes for ApiInbound rows"
```
---
## Task 3: Capture the response body in `AuditWriteMiddleware` (TDD)
**What:** Implement the M5-deferred response-body capture. Wrap `HttpContext.Response.Body` with a buffering `MemoryStream` BEFORE `_next(ctx)`, restore + copy the buffered bytes back to the original stream AFTER the pipeline runs, then read the buffer as UTF-8 into `ResponseSummary` on the audit event. The `AuditEvent.ResponseSummary = null` line goes away.
**Files:**
- Modify: `src/ScadaLink.InboundAPI/Middleware/AuditWriteMiddleware.cs`.
- Modify: `tests/ScadaLink.InboundAPI.Tests/Middleware/AuditWriteMiddlewareTests.cs` — extend.
**Step 1: Write the failing tests**
Append to `AuditWriteMiddlewareTests.cs` (before the closing class brace), keeping the existing `BuildContext` / `CreateMiddleware` helpers:
```csharp
// ---------------------------------------------------------------------
// Response body capture — Audit Log #23 (inbound full-response feature).
// Until the M5-deferred work landed, ResponseSummary was always null.
// These tests pin the new contract: the middleware wraps Response.Body,
// runs the pipeline, copies the buffered bytes back to the real stream,
// and stashes a UTF-8 string copy on ResponseSummary.
// ---------------------------------------------------------------------
[Fact]
public async Task ResponseBody_IsCaptured_OnResponseSummary()
{
var writer = new RecordingAuditWriter();
var ctx = BuildContext();
var responseJson = "{\"result\":42}";
var mw = CreateMiddleware(async hc =>
{
hc.Response.StatusCode = 200;
hc.Response.ContentType = "application/json";
await hc.Response.WriteAsync(responseJson);
}, writer);
await mw.InvokeAsync(ctx);
var evt = Assert.Single(writer.Events);
Assert.Equal(responseJson, evt.ResponseSummary);
}
[Fact]
public async Task ResponseBody_IsForwardedToOriginalStream_DownstreamReadersSeeIt()
{
// Wrapping the response body must be TRANSPARENT — the real client
// stream still receives every byte the pipeline wrote.
var writer = new RecordingAuditWriter();
var ctx = BuildContext();
var captured = new MemoryStream();
ctx.Response.Body = captured; // simulate the client/test sink
var responseJson = "{\"ok\":true}";
var mw = CreateMiddleware(async hc =>
{
hc.Response.StatusCode = 200;
await hc.Response.WriteAsync(responseJson);
}, writer);
await mw.InvokeAsync(ctx);
Assert.Equal(responseJson, Encoding.UTF8.GetString(captured.ToArray()));
}
[Fact]
public async Task ResponseBody_Empty_LeavesResponseSummaryNull()
{
// No bytes written => null, not empty-string. Mirrors the request-body
// contract in ReadBufferedRequestBodyAsync.
var writer = new RecordingAuditWriter();
var ctx = BuildContext();
var mw = CreateMiddleware(hc =>
{
hc.Response.StatusCode = 204;
return Task.CompletedTask;
}, writer);
await mw.InvokeAsync(ctx);
var evt = Assert.Single(writer.Events);
Assert.Null(evt.ResponseSummary);
Assert.Equal(204, evt.HttpStatus);
}
[Fact]
public async Task ResponseBody_OnHandlerThrow_BodyCapturedUpToTheThrow()
{
// If the handler writes some bytes then throws, the audit row still
// surfaces whatever the framework had flushed. The middleware re-throws
// (audit is best-effort, the request's error path stays authoritative).
var writer = new RecordingAuditWriter();
var ctx = BuildContext();
var boom = new InvalidOperationException("kaboom");
var mw = CreateMiddleware(async hc =>
{
hc.Response.StatusCode = 500;
await hc.Response.WriteAsync("partial");
throw boom;
}, writer);
var thrown = await Assert.ThrowsAsync<InvalidOperationException>(
() => mw.InvokeAsync(ctx));
Assert.Same(boom, thrown);
var evt = Assert.Single(writer.Events);
Assert.Equal(AuditStatus.Failed, evt.Status);
Assert.Equal("partial", evt.ResponseSummary);
}
```
**Step 2: Run tests — confirm they fail**
```
dotnet test tests/ScadaLink.InboundAPI.Tests \
--filter "FullyQualifiedName~AuditWriteMiddlewareTests"
```
Expected: the four new tests fail (ResponseSummary stays null today). Pre-existing tests still pass (they don't assert ResponseSummary).
**Step 3: Implement response capture in the middleware**
Edit `AuditWriteMiddleware.cs`:
(a) Update the XML doc at the top — remove the "Response body capture is deferred to M5…" paragraph (lines 42-50 in the current file). Replace with:
```csharp
/// <para>
/// <b>Body capture.</b> The request body is buffered via
/// <see cref="HttpRequestRewindExtensions.EnableBuffering(HttpRequest)"/> then
/// rewound so the downstream endpoint handler still sees the full payload. The
/// response body is captured by swapping <see cref="HttpResponse.Body"/> for a
/// <see cref="MemoryStream"/> before the pipeline runs; after the pipeline
/// returns, the buffered bytes are copied to the original stream (transparent
/// to the real client) and read into <see cref="AuditEvent.ResponseSummary"/>.
/// Truncation to the configured inbound ceiling happens in
/// <see cref="ScadaLink.AuditLog.Payload.DefaultAuditPayloadFilter"/>; the
/// middleware itself stores the full buffered content.
/// </para>
```
(b) Rewrite `InvokeAsync` so the response stream is swapped, the buffer is read post-pipeline (in `finally`, even on a thrown handler), and the original stream receives the bytes back:
```csharp
public async Task InvokeAsync(HttpContext ctx)
{
var sw = Stopwatch.StartNew();
ctx.Items[InboundExecutionIdItemKey] = Guid.NewGuid();
// Request body — buffer for both audit + downstream handler.
ctx.Request.EnableBuffering();
var requestBody = await ReadBufferedRequestBodyAsync(ctx.Request).ConfigureAwait(false);
// Response body — swap in a MemoryStream so the pipeline writes are
// captured. The original Response.Body is restored in the finally block,
// and the captured bytes are copied back to it so the real client still
// receives every byte (transparent wrap). The captured string is then
// available for the audit row.
var originalResponseBody = ctx.Response.Body;
using var responseBuffer = new MemoryStream();
ctx.Response.Body = responseBuffer;
string? responseBody = null;
Exception? thrown = null;
try
{
await _next(ctx).ConfigureAwait(false);
}
catch (Exception ex)
{
thrown = ex;
throw;
}
finally
{
sw.Stop();
// Whatever the handler managed to write — full success, partial
// success before throwing, or nothing at all — copy back to the
// original stream and read for audit.
responseBody = await DrainResponseBufferAsync(responseBuffer, originalResponseBody)
.ConfigureAwait(false);
ctx.Response.Body = originalResponseBody;
EmitInboundAudit(ctx, sw.ElapsedMilliseconds, thrown, requestBody, responseBody);
}
}
```
(c) Add the new helper (place beside `ReadBufferedRequestBodyAsync`):
```csharp
/// <summary>
/// Copies the bytes buffered in <paramref name="buffer"/> to
/// <paramref name="originalBody"/> (so the real client still receives them)
/// and returns a UTF-8 string copy for <see cref="AuditEvent.ResponseSummary"/>.
/// Returns null when no bytes were written, mirroring the
/// <see cref="ReadBufferedRequestBodyAsync"/> empty-body contract.
/// </summary>
private static async Task<string?> DrainResponseBufferAsync(
MemoryStream buffer,
Stream originalBody)
{
if (buffer.Length == 0)
{
return null;
}
buffer.Position = 0;
// Copy first so the client never misses bytes even if the read for audit
// throws somehow (defensive — MemoryStream.CopyToAsync to a sink shouldn't
// throw on its own, but the original body may).
try
{
await buffer.CopyToAsync(originalBody).ConfigureAwait(false);
}
catch
{
// Best-effort: a sink that refuses our copy is the sink's problem;
// the audit still records what the handler produced. Do NOT rethrow.
}
buffer.Position = 0;
using var reader = new StreamReader(
buffer,
Encoding.UTF8,
detectEncodingFromByteOrderMarks: false,
bufferSize: 1024,
leaveOpen: true);
var content = await reader.ReadToEndAsync().ConfigureAwait(false);
return string.IsNullOrEmpty(content) ? null : content;
}
```
(d) Change `EmitInboundAudit`'s signature to take the response body, and drop the `ResponseSummary = null` line:
```csharp
private void EmitInboundAudit(
HttpContext ctx,
long durationMs,
Exception? thrown,
string? requestBody,
string? responseBody)
{
// ... unchanged up to the AuditEvent constructor ...
var evt = new AuditEvent
{
// ... existing fields ...
RequestSummary = requestBody,
ResponseSummary = responseBody, // was null in the M4 deliverable
PayloadTruncated = false,
Extra = extra,
ForwardState = null,
};
// ... unchanged fire-and-forget write ...
}
```
**Step 4: Run tests — confirm they pass**
```
dotnet test tests/ScadaLink.InboundAPI.Tests
```
Expected: all green, including the four new response-body tests AND every pre-existing middleware test.
**Step 5: Build the whole solution**
```
dotnet build ScadaLink.slnx
```
Expected: 0 warnings.
**Step 6: Commit**
```
git add src/ScadaLink.InboundAPI/Middleware/AuditWriteMiddleware.cs \
tests/ScadaLink.InboundAPI.Tests/Middleware/AuditWriteMiddlewareTests.cs
git commit -m "feat(inboundapi): AuditWriteMiddleware captures response body on ApiInbound audit rows"
```
---
## Task 4: Update `Component-AuditLog.md`
**What:** Reflect the inbound carve-out in the requirements doc — schema row descriptions + Payload Capture Policy.
**Files:**
- Modify: `docs/requirements/Component-AuditLog.md`.
**Edits (all in-place — no copies):**
1. Schema table — find the `RequestSummary` row. Change its `Description` cell from:
> Truncated request payload (configurable cap). Headers redacted.
to:
> Truncated request payload (configurable cap). Headers redacted. For `Channel = ApiInbound`, captured in full up to `AuditLog:InboundMaxBytes` (default 1 MiB) — see Payload Capture Policy.
2. Schema table — `ResponseSummary` row. Change its `Description` cell from:
> Truncated response payload. Full on errors.
to:
> Truncated response payload. For `Channel = ApiInbound`, captured in full up to `AuditLog:InboundMaxBytes` (default 1 MiB). For other channels, capped at `DefaultCapBytes` by default and `ErrorCapBytes` on error rows.
3. **Payload Capture Policy** section — after the existing **Default cap** bullet, insert:
> - **Inbound API exception.** For `Channel = ApiInbound`, `RequestSummary` and `ResponseSummary` are captured in full up to a per-body hard ceiling of 1 MiB (configurable via `AuditLog:InboundMaxBytes`; default 1 048 576 bytes; min 8 192; max 16 777 216). The 8 KiB / 64 KiB default/error caps that apply to other channels do not apply here. `PayloadTruncated = 1` is set only when the inbound ceiling is hit — verbatim capture is the normal case. The ceiling applies independently to each body. Header redaction and per-target body redactors still run before persistence.
**Verify:** `git diff docs/requirements/Component-AuditLog.md` — three edits, no other lines touched.
**Commit:**
```
git add docs/requirements/Component-AuditLog.md
git commit -m "docs(audit): schema + Payload Capture Policy note inbound full-body carve-out"
```
---
## Task 5: Update `Component-InboundAPI.md`
**What:** Update the two existing references that say "truncated request/response bodies per the Audit Log capture policy" to reflect the new wording.
**Files:**
- Modify: `docs/requirements/Component-InboundAPI.md`.
**Edits:**
1. Around line 119 (the inbound-audit description in §Operational Audit / Logging) — change:
> Every request — success or failure — emits one `ApiInbound.Completed` row to `ICentralAuditWriter` from request middleware before the HTTP response is flushed. The row captures the API key **name** (never the key material), remote IP, user-agent, response status, duration, and truncated request/response bodies per the Audit Log capture policy (see Component-AuditLog.md, Payload Capture Policy).
to:
> Every request — success or failure — emits one `ApiInbound.Completed` row to `ICentralAuditWriter` from request middleware before the HTTP response is flushed. The row captures the API key **name** (never the key material), remote IP, user-agent, response status, duration, and the request/response bodies. Bodies are captured in full up to `AuditLog:InboundMaxBytes` (default 1 MiB); `PayloadTruncated = 1` only when that ceiling is hit. Header redaction and per-target body redactors still apply (see Component-AuditLog.md, Payload Capture Policy).
2. Around line 202 (Dependencies → Audit Log) — change:
> **Audit Log (#23)**: Every inbound API request emits an `ApiInbound.Completed` row via `ICentralAuditWriter` from request middleware (non-blocking for the HTTP response). Payload truncation/redaction follows the Audit Log Payload Capture Policy.
to:
> **Audit Log (#23)**: Every inbound API request emits an `ApiInbound.Completed` row via `ICentralAuditWriter` from request middleware (non-blocking for the HTTP response). Request and response bodies are captured in full up to `AuditLog:InboundMaxBytes` (default 1 MiB) per the Audit Log Payload Capture Policy; redaction (headers + per-target body redactors) still applies before persistence.
**Verify:**
```
grep -nE "truncated request/response|InboundMaxBytes" docs/requirements/Component-InboundAPI.md
```
Expected: no "truncated request/response" hits remain; two "InboundMaxBytes" hits land on the updated lines.
**Commit:**
```
git add docs/requirements/Component-InboundAPI.md
git commit -m "docs(inboundapi): note request/response bodies captured in full to InboundMaxBytes"
```
---
## Task 6: Final solution build + full test run + branch summary
**What:** Confirm the cumulative change is green end-to-end and summarise the branch.
**Steps:**
1. `dotnet build ScadaLink.slnx` from the repo root — expect 0 warnings, 0 errors.
2. Run the affected test projects:
```
dotnet test tests/ScadaLink.AuditLog.Tests
dotnet test tests/ScadaLink.InboundAPI.Tests
```
Expect both green. (The wider solution test run is optional but cheap — `dotnet test ScadaLink.slnx` if you want full coverage.)
3. `git log --oneline main..HEAD` — expect exactly five commits:
- feat(auditlog): add AuditLog:InboundMaxBytes option …
- feat(auditlog): payload filter uses InboundMaxBytes for ApiInbound rows
- feat(inboundapi): AuditWriteMiddleware captures response body on ApiInbound audit rows
- docs(audit): schema + Payload Capture Policy note inbound full-body carve-out
- docs(inboundapi): note request/response bodies captured in full to InboundMaxBytes
4. `git status --short` — expect a clean tree (no uncommitted files; pre-existing uncommitted files from `git status` at session start may still be present and are unrelated to this work — leave them).
**Acceptance:**
- All acceptance criteria in `docs/plans/2026-05-23-inbound-api-full-response-audit-design.md` met.
- Solution builds clean.
- All targeted tests pass.
- Five commits on the feature branch, no commits on `main`, branch not pushed.
**Commit:** none. Do not merge or push — that is the user's call.
---
## Notes for the executor
- The `ResponseSummary = null` comment in `AuditWriteMiddleware.cs` (line ~191 today) is the smoking-gun: response capture was always intended, just deferred. The design doc explicitly authorises closing that gap.
- The `TestOptionsMonitor<T>` helper is already used by `AuditLogOptionsBindingTests.cs` — reuse it; do not introduce a second one. If its public location moves between when this plan was written and execution, `grep -rn "class TestOptionsMonitor" tests/` and adjust the `using`.
- `appsettings.json` files in `docker/central-node-a` and `docker/central-node-b` do not currently override `AuditLog:*` — leave them alone. The 1 MiB default takes effect automatically from `AuditLogOptions`.
- No EF migration is needed — schema is unchanged (`nvarchar(max)` already).
- No new health metric — `PayloadTruncated = 1` carries the ceiling-hit signal. The design doc explicitly defers a dedicated `AuditInboundCeilingHits` counter.
@@ -0,0 +1,13 @@
{
"planPath": "docs/plans/2026-05-23-inbound-api-full-response-audit.md",
"tasks": [
{"id": 1, "subject": "Task 0: Prep — branch, baseline build", "status": "pending"},
{"id": 2, "subject": "Task 1: Add InboundMaxBytes to AuditLogOptions (TDD)", "status": "pending", "blockedBy": [1]},
{"id": 3, "subject": "Task 2: Wire InboundMaxBytes into DefaultAuditPayloadFilter (TDD)", "status": "pending", "blockedBy": [2]},
{"id": 4, "subject": "Task 3: Capture response body in AuditWriteMiddleware (TDD)", "status": "pending", "blockedBy": [3]},
{"id": 5, "subject": "Task 4: Update Component-AuditLog.md", "status": "pending", "blockedBy": [4]},
{"id": 6, "subject": "Task 5: Update Component-InboundAPI.md", "status": "pending", "blockedBy": [5]},
{"id": 7, "subject": "Task 6: Final build + full test run + branch summary", "status": "pending", "blockedBy": [6]}
],
"lastUpdated": "2026-05-23"
}
+3 -2
View File
@@ -95,8 +95,8 @@ row per lifecycle event across all channels.
| `DurationMs` | `int` NULL | Call / attempt duration. |
| `ErrorMessage` | `nvarchar(1024)` NULL | Truncated; `ErrorDetail` for full text. |
| `ErrorDetail` | `nvarchar(max)` NULL | Optional full exception text on failures. |
| `RequestSummary` | `nvarchar(max)` NULL | Truncated request payload (configurable cap). Headers redacted. |
| `ResponseSummary` | `nvarchar(max)` NULL | Truncated response payload. Full on errors. |
| `RequestSummary` | `nvarchar(max)` NULL | Truncated request payload (configurable cap). Headers redacted. For `Channel = ApiInbound`, captured in full up to `AuditLog:InboundMaxBytes` (default 1 MiB) — see Payload Capture Policy. |
| `ResponseSummary` | `nvarchar(max)` NULL | Truncated response payload. For `Channel = ApiInbound`, captured in full up to `AuditLog:InboundMaxBytes` (default 1 MiB). For other channels, capped at `DefaultCapBytes` by default and `ErrorCapBytes` on error rows. |
| `PayloadTruncated` | `bit` | Set if either summary was truncated. |
| `Extra` | `nvarchar(max)` NULL | Channel-specific JSON for fields we don't promote to columns. |
@@ -262,6 +262,7 @@ operational `SiteCalls` shape for the dispatcher and UI.
- **Default cap** — 8 KB for each of `RequestSummary` and `ResponseSummary`;
raised to 64 KB on any error row (`Status IN ('Failed', 'Parked', 'Discarded')`).
- **Inbound API exception.** For `Channel = ApiInbound`, `RequestSummary` and `ResponseSummary` are captured in full up to a per-body hard ceiling of 1 MiB (configurable via `AuditLog:InboundMaxBytes`; default 1 048 576 bytes; min 8 192; max 16 777 216). The 8 KiB / 64 KiB default/error caps that apply to other channels do not apply here. `PayloadTruncated = 1` is set only when the inbound ceiling is hit — verbatim capture is the normal case. The ceiling applies independently to each body. Header redaction and per-target body redactors still run before persistence.
- **Truncation** — UTF-8 byte-safe; `PayloadTruncated = 1` when applied. Full
bodies are never stored.
- **HTTP headers** — `Authorization`, `Cookie`, `Set-Cookie`, `X-API-Key`, and
+2 -2
View File
@@ -116,7 +116,7 @@ API method scripts are compiled at central startup — all method definitions ar
## API Call Logging
- **Every request — success or failure — emits one `ApiInbound.Completed` row** to `ICentralAuditWriter` from request middleware before the HTTP response is flushed. The row captures the API key **name** (never the key material), remote IP, user-agent, response status, duration, and truncated request/response bodies per the Audit Log capture policy (see Component-AuditLog.md, Payload Capture Policy). This supersedes the earlier failures-only stance: operational API traffic is now part of the centralized audit log, so configuration changes and call activity share a single retention/query surface.
- **Every request — success or failure — emits one `ApiInbound.Completed` row** to `ICentralAuditWriter` from request middleware before the HTTP response is flushed. The row captures the API key **name** (never the key material), remote IP, user-agent, response status, duration, and the request/response bodies. Bodies are captured in full up to `AuditLog:InboundMaxBytes` (default 1 MiB); `PayloadTruncated = 1` only when that ceiling is hit. Header redaction and per-target body redactors still apply (see Component-AuditLog.md, Payload Capture Policy). This supersedes the earlier failures-only stance: operational API traffic is now part of the centralized audit log, so configuration changes and call activity share a single retention/query surface.
- Script execution errors (500 responses) remain captured on the same `ApiInbound.Completed` row (response status + error fields) rather than emitting a separate failure-only event.
- **Fail-soft semantics.** The audit write is synchronous (inline before the response is flushed), but failures are caught: a write that throws is logged and increments `CentralAuditWriteFailures` (see Health Monitoring #11) and the request still returns its normal HTTP response. A failed audit append never turns a successful API call into an error returned to the caller.
- No rate limiting — this is a private API in a controlled industrial environment with a known set of callers. Misbehaving callers are handled operationally (disable the API key).
@@ -199,7 +199,7 @@ Inbound API scripts **cannot** call shared scripts directly — shared scripts a
- **Communication Layer**: Routes requests to sites when method implementations need site data.
- **Security & Auth**: API key validation (separate from LDAP/AD — API uses key-based auth).
- **Configuration Database (via IAuditService)**: All API key and method definition changes are audit logged.
- **Audit Log (#23)**: Every inbound API request emits an `ApiInbound.Completed` row via `ICentralAuditWriter` from request middleware (non-blocking for the HTTP response). Payload truncation/redaction follows the Audit Log Payload Capture Policy.
- **Audit Log (#23)**: Every inbound API request emits an `ApiInbound.Completed` row via `ICentralAuditWriter` from request middleware (non-blocking for the HTTP response). Request and response bodies are captured in full up to `AuditLog:InboundMaxBytes` (default 1 MiB) per the Audit Log Payload Capture Policy; redaction (headers + per-target body redactors) still applies before persistence.
- **Cluster Infrastructure**: API is hosted on the active central node and fails over with it.
## Interactions
@@ -1,3 +1,6 @@
using ScadaLink.Commons.Entities.Audit;
using ScadaLink.Commons.Types.Enums;
namespace ScadaLink.AuditLog.Configuration;
/// <summary>
@@ -33,4 +36,14 @@ public sealed class AuditLogOptions
/// <summary>Central retention window in days (default 365, range [30, 3650]).</summary>
public int RetentionDays { get; set; } = 365;
/// <summary>
/// Per-body byte ceiling applied to <see cref="AuditEvent.RequestSummary"/> and
/// <see cref="AuditEvent.ResponseSummary"/> for <see cref="AuditChannel.ApiInbound"/> rows
/// (default 1 MiB). The 8 KiB / 64 KiB default/error caps that apply to other channels
/// do not apply here — inbound traffic captures verbatim up to this ceiling and only
/// then sets <see cref="AuditEvent.PayloadTruncated"/>. See
/// <c>docs/plans/2026-05-23-inbound-api-full-response-audit-design.md</c>.
/// </summary>
public int InboundMaxBytes { get; set; } = 1_048_576;
}
@@ -21,6 +21,12 @@ public sealed class AuditLogOptionsValidator : IValidateOptions<AuditLogOptions>
/// <summary>Inclusive upper bound for <see cref="AuditLogOptions.RetentionDays"/>.</summary>
public const int MaxRetentionDays = 3650;
/// <summary>Inclusive lower bound for <see cref="AuditLogOptions.InboundMaxBytes"/> (8 KiB).</summary>
public const int MinInboundMaxBytes = 8_192;
/// <summary>Inclusive upper bound for <see cref="AuditLogOptions.InboundMaxBytes"/> (16 MiB).</summary>
public const int MaxInboundMaxBytes = 16_777_216;
/// <inheritdoc />
public ValidateOptionsResult Validate(string? name, AuditLogOptions options)
{
@@ -50,6 +56,13 @@ public sealed class AuditLogOptionsValidator : IValidateOptions<AuditLogOptions>
$"must be in [{MinRetentionDays}, {MaxRetentionDays}] days.");
}
if (options.InboundMaxBytes < MinInboundMaxBytes || options.InboundMaxBytes > MaxInboundMaxBytes)
{
failures.Add(
$"AuditLog:{nameof(AuditLogOptions.InboundMaxBytes)} ({options.InboundMaxBytes}) " +
$"must be in [{MinInboundMaxBytes}, {MaxInboundMaxBytes}] bytes.");
}
return failures.Count == 0
? ValidateOptionsResult.Success
: ValidateOptionsResult.Fail(failures);
@@ -118,7 +118,14 @@ public sealed class DefaultAuditPayloadFilter : IAuditPayloadFilter
try
{
var opts = _options.CurrentValue;
var cap = IsErrorStatus(rawEvent.Status) ? opts.ErrorCapBytes : opts.DefaultCapBytes;
// Inbound API gets a dedicated, larger ceiling — request/response bodies are
// captured verbatim up to InboundMaxBytes (default 1 MiB) so support can
// replay exactly what the caller sent and what we returned. Other channels
// keep the global 8 KiB / 64 KiB policy.
// See docs/plans/2026-05-23-inbound-api-full-response-audit-design.md.
var cap = rawEvent.Channel == AuditChannel.ApiInbound
? opts.InboundMaxBytes
: (IsErrorStatus(rawEvent.Status) ? opts.ErrorCapBytes : opts.DefaultCapBytes);
// --- Header-redaction stage (runs BEFORE truncation) ----------
var request = RedactHeaders(rawEvent.RequestSummary, opts.HeaderRedactList);
@@ -1,8 +1,11 @@
using System.Buffers;
using System.Diagnostics;
using System.Text;
using System.Text.Json;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using ScadaLink.AuditLog.Configuration;
using ScadaLink.Commons.Entities.Audit;
using ScadaLink.Commons.Interfaces.Services;
using ScadaLink.Commons.Types.Enums;
@@ -42,11 +45,22 @@ namespace ScadaLink.InboundAPI.Middleware;
/// <para>
/// <b>Body capture.</b> The request body is buffered via
/// <see cref="HttpRequestRewindExtensions.EnableBuffering(HttpRequest)"/> then
/// rewound so the downstream endpoint handler still sees the full payload.
/// Response body capture is deferred to M5 — wrapping <c>Response.Body</c>
/// requires a memory-stream swap that interacts awkwardly with Minimal API's
/// <c>Results.Json</c>/<c>Results.Text</c> writers; the M4 deliverable emits
/// the audit row with <see cref="AuditEvent.ResponseSummary"/> left null.
/// rewound so the downstream endpoint handler still sees the full payload. The
/// response body is captured by wrapping <see cref="HttpResponse.Body"/> in a
/// forwarding stream that mirrors writes to the original sink (transparent to
/// the real client) while capturing a bounded copy for audit.
/// </para>
///
/// <para>
/// <b>Bounded capture at the source.</b> Both the request- and response-body
/// audit copies are bounded at <see cref="AuditLogOptions.InboundMaxBytes"/>
/// (default 1 MiB) AT THE CAPTURE SITE — we never buffer more than
/// <c>cap + 1</c> bytes per body even when the client streams hundreds of MiB.
/// The downstream handler and the real client still see every byte; only the
/// audit copy is bounded. The cap is also enforced again by
/// <see cref="ScadaLink.AuditLog.Payload.DefaultAuditPayloadFilter"/> (which OR's
/// in its own <see cref="AuditEvent.PayloadTruncated"/> determination), so a
/// row truncated here remains truncated even if the filter is bypassed.
/// </para>
/// </summary>
public sealed class AuditWriteMiddleware
@@ -74,21 +88,29 @@ public sealed class AuditWriteMiddleware
private readonly RequestDelegate _next;
private readonly ICentralAuditWriter _auditWriter;
private readonly ILogger<AuditWriteMiddleware> _logger;
private readonly IOptionsMonitor<AuditLogOptions> _options;
public AuditWriteMiddleware(
RequestDelegate next,
ICentralAuditWriter auditWriter,
ILogger<AuditWriteMiddleware> logger)
ILogger<AuditWriteMiddleware> logger,
IOptionsMonitor<AuditLogOptions> options)
{
_next = next ?? throw new ArgumentNullException(nameof(next));
_auditWriter = auditWriter ?? throw new ArgumentNullException(nameof(auditWriter));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
_options = options ?? throw new ArgumentNullException(nameof(options));
}
public async Task InvokeAsync(HttpContext ctx)
{
var sw = Stopwatch.StartNew();
// Per-request hot read of the inbound cap — mirrors the convention used
// by DefaultAuditPayloadFilter so a live config change picks up on the
// next request without re-resolving the singleton.
var cap = _options.CurrentValue.InboundMaxBytes;
// Audit Log #23 (ParentExecutionId): mint the inbound request's per-request
// ExecutionId ONCE, here at the start of the request, and stash it on
// HttpContext.Items. Two consumers share this single id:
@@ -106,7 +128,16 @@ public sealed class AuditWriteMiddleware
// of the pipeline for us — but we also rewind to position 0 after our
// own read so the very next reader starts from the top.
ctx.Request.EnableBuffering();
var requestBody = await ReadBufferedRequestBodyAsync(ctx.Request).ConfigureAwait(false);
var (requestBody, requestTruncated) =
await ReadBufferedRequestBodyAsync(ctx.Request, cap).ConfigureAwait(false);
// Response body — wrap Response.Body in a forwarding stream that mirrors
// every write to the original sink (transparent to the real client)
// while capturing AT MOST `cap + 1` bytes for the audit copy. The
// original Response.Body is restored in the finally block.
var originalResponseBody = ctx.Response.Body;
using var captureStream = new CapturedResponseStream(originalResponseBody, cap);
ctx.Response.Body = captureStream;
Exception? thrown = null;
try
@@ -123,7 +154,20 @@ public sealed class AuditWriteMiddleware
finally
{
sw.Stop();
EmitInboundAudit(ctx, sw.ElapsedMilliseconds, thrown, requestBody);
// Restore the original stream and resolve the captured audit copy.
// The forwarding wrapper has already written every byte to the
// original sink; this just pulls back the bounded UTF-8 string.
ctx.Response.Body = originalResponseBody;
var (responseBody, responseTruncated) = captureStream.GetCapturedBody();
EmitInboundAudit(
ctx,
sw.ElapsedMilliseconds,
thrown,
requestBody,
responseBody,
requestTruncated || responseTruncated);
}
}
@@ -136,7 +180,9 @@ public sealed class AuditWriteMiddleware
HttpContext ctx,
long durationMs,
Exception? thrown,
string? requestBody)
string? requestBody,
string? responseBody,
bool payloadTruncated)
{
try
{
@@ -187,9 +233,8 @@ public sealed class AuditWriteMiddleware
DurationMs = (int)Math.Min(durationMs, int.MaxValue),
ErrorMessage = thrown?.Message,
RequestSummary = requestBody,
// Response body capture is deferred to M5 (see XML doc above).
ResponseSummary = null,
PayloadTruncated = false,
ResponseSummary = responseBody,
PayloadTruncated = payloadTruncated,
Extra = extra,
// Central direct-write — no site-local forwarding state.
ForwardState = null,
@@ -210,39 +255,113 @@ public sealed class AuditWriteMiddleware
}
/// <summary>
/// Reads the buffered request body fully into a string and rewinds the
/// stream so the downstream handler sees the unconsumed payload. Returns
/// null for empty/missing bodies so the audit row's
/// Reads the buffered request body up to <paramref name="capBytes"/> bytes
/// into a string for the audit copy and rewinds the stream so the
/// downstream handler sees the unconsumed payload. Returns
/// <c>(null, false)</c> for empty/missing bodies so the audit row's
/// <see cref="AuditEvent.RequestSummary"/> stays null rather than
/// containing an empty string.
/// </summary>
private static async Task<string?> ReadBufferedRequestBodyAsync(HttpRequest request)
/// <remarks>
/// Reads AT MOST <c>cap + 1</c> bytes from the request stream into a
/// scratch buffer; if the extra byte arrives the body is over the cap and
/// the returned string is UTF-8 byte-safe truncated to exactly
/// <c>cap</c> bytes with <c>truncated = true</c>. The cap applies only to
/// the audit copy — the request stream is always rewound to position 0
/// afterwards so the framework's next reader (the endpoint handler's
/// JSON parser) sees the full body.
/// </remarks>
private static async Task<(string? body, bool truncated)> ReadBufferedRequestBodyAsync(
HttpRequest request,
int capBytes)
{
if (request.ContentLength is 0)
{
return null;
return (null, false);
}
// Read AT MOST cap + 1 bytes — the extra byte tells us the body was
// over the cap without forcing us to allocate the whole payload. Rent
// the scratch buffer from the shared ArrayPool so we don't allocate
// (and immediately discard) `cap + 1` bytes per request — the pool
// may hand back a buffer LARGER than `limit`, so we treat `limit`
// (not `buffer.Length`) as the read ceiling.
var limit = capBytes + 1;
var buffer = ArrayPool<byte>.Shared.Rent(limit);
try
{
request.Body.Position = 0;
using var reader = new StreamReader(
request.Body,
Encoding.UTF8,
detectEncodingFromByteOrderMarks: false,
bufferSize: 1024,
leaveOpen: true);
var content = await reader.ReadToEndAsync().ConfigureAwait(false);
request.Body.Position = 0;
return string.IsNullOrEmpty(content) ? null : content;
var total = 0;
while (total < limit)
{
var read = await request.Body
.ReadAsync(buffer.AsMemory(total, limit - total))
.ConfigureAwait(false);
if (read == 0)
{
break;
}
total += read;
}
if (total == 0)
{
return (null, false);
}
var truncated = total > capBytes;
var bytesForString = truncated ? capBytes : total;
var content = DecodeUtf8Bounded(buffer, bytesForString, cutAtValidBytes: truncated);
return (string.IsNullOrEmpty(content) ? null : content, truncated);
}
catch
{
// A failed body read must not abort the request — fall through
// with a null RequestSummary; the audit row still records the
// outcome.
return null;
return (null, false);
}
finally
{
// Even on a thrown read, the downstream handler must see the full
// body from position 0 — never let a failed audit copy leak a
// truncated view. A rewind failure is swallowed: best-effort,
// same philosophy as the rest of the file.
try { request.Body.Position = 0; } catch { /* swallow */ }
ArrayPool<byte>.Shared.Return(buffer);
}
}
/// <summary>
/// UTF-8 byte-safe decode of <paramref name="validBytes"/> bytes from
/// <paramref name="bytes"/>. When <paramref name="cutAtValidBytes"/> is
/// <c>true</c> the input is the result of a hard byte-count truncation, so
/// we walk back from <c>validBytes</c> while the byte is a continuation
/// byte (<c>byte &amp; 0xC0 == 0x80</c>) to avoid splitting a multi-byte
/// codepoint. When <c>false</c> the caller is decoding the full payload
/// and the boundary stands as-is.
/// </summary>
/// <remarks>
/// Mirrors the algorithm in <c>DefaultAuditPayloadFilter.TruncateUtf8</c>;
/// kept local to avoid a backwards project reference from
/// ScadaLink.AuditLog into ScadaLink.InboundAPI.
/// </remarks>
private static string DecodeUtf8Bounded(byte[] bytes, int validBytes, bool cutAtValidBytes)
{
if (validBytes <= 0)
{
return string.Empty;
}
var boundary = validBytes;
if (cutAtValidBytes)
{
while (boundary > 0 && (bytes[boundary] & 0xC0) == 0x80)
{
boundary--;
}
}
return Encoding.UTF8.GetString(bytes, 0, boundary);
}
/// <summary>
@@ -321,4 +440,153 @@ public sealed class AuditWriteMiddleware
return path[(lastSlash + 1)..];
}
/// <summary>
/// Write-only forwarding <see cref="Stream"/> wrapper that mirrors every
/// write to the inner ASP.NET <see cref="HttpResponse.Body"/> (so the real
/// client receives all bytes) while capturing AT MOST <c>cap + 1</c> bytes
/// into a private bounded <see cref="MemoryStream"/> for the audit copy.
/// </summary>
/// <remarks>
/// <para>
/// The inner sink is owned by the framework and is NOT disposed when this
/// wrapper is disposed — we only own the capture <see cref="MemoryStream"/>.
/// </para>
/// <para>
/// All Write overloads forward to the inner stream FIRST, then capture the
/// remaining quota. If the inner sink throws (e.g. the client disconnects),
/// the exception is allowed to propagate — capture is best-effort, the
/// real I/O is authoritative. The handler-throws-mid-response test
/// (<c>ResponseBody_OnHandlerThrow_BodyCapturedUpToTheThrow</c>) verifies
/// that captured bytes up to the throw are still recoverable.
/// </para>
/// </remarks>
private sealed class CapturedResponseStream : Stream
{
private readonly Stream _inner;
private readonly int _capBytes;
private readonly MemoryStream _captured;
private bool _disposed;
public CapturedResponseStream(Stream inner, int capBytes)
{
_inner = inner ?? throw new ArgumentNullException(nameof(inner));
_capBytes = Math.Max(0, capBytes);
// Capture up to cap + 1 bytes so we can detect the over-cap case
// without growing the buffer further.
_captured = new MemoryStream();
}
public override bool CanRead => false;
public override bool CanSeek => false;
public override bool CanWrite => true;
public override long Length =>
throw new NotSupportedException("CapturedResponseStream is write-only.");
public override long Position
{
get => throw new NotSupportedException("CapturedResponseStream is write-only.");
set => throw new NotSupportedException("CapturedResponseStream is write-only.");
}
public override void Flush() => _inner.Flush();
public override Task FlushAsync(CancellationToken cancellationToken) =>
_inner.FlushAsync(cancellationToken);
public override int Read(byte[] buffer, int offset, int count) =>
throw new NotSupportedException("CapturedResponseStream is write-only.");
public override long Seek(long offset, SeekOrigin origin) =>
throw new NotSupportedException("CapturedResponseStream is write-only.");
public override void SetLength(long value) =>
throw new NotSupportedException("CapturedResponseStream is write-only.");
public override void Write(byte[] buffer, int offset, int count)
{
// Forward to the real sink FIRST — the client must never miss
// bytes if capture throws.
_inner.Write(buffer, offset, count);
CaptureBytes(buffer.AsSpan(offset, count));
}
public override void Write(ReadOnlySpan<byte> buffer)
{
_inner.Write(buffer);
CaptureBytes(buffer);
}
public override async Task WriteAsync(
byte[] buffer, int offset, int count, CancellationToken cancellationToken)
{
await _inner.WriteAsync(buffer.AsMemory(offset, count), cancellationToken)
.ConfigureAwait(false);
CaptureBytes(buffer.AsSpan(offset, count));
}
public override async ValueTask WriteAsync(
ReadOnlyMemory<byte> buffer, CancellationToken cancellationToken = default)
{
await _inner.WriteAsync(buffer, cancellationToken).ConfigureAwait(false);
CaptureBytes(buffer.Span);
}
/// <summary>
/// Capture up to <c>cap + 1</c> bytes total into the private
/// <see cref="MemoryStream"/>. Once the cap quota is reached, further
/// bytes are silently dropped from the audit copy (the real sink has
/// already received them upstream of this call).
/// </summary>
private void CaptureBytes(ReadOnlySpan<byte> span)
{
if (span.Length == 0)
{
return;
}
var quota = (_capBytes + 1) - (int)_captured.Length;
if (quota <= 0)
{
return;
}
var take = Math.Min(quota, span.Length);
_captured.Write(span[..take]);
}
/// <summary>
/// Returns the captured response body as a UTF-8 string (byte-safe
/// truncated to <c>cap</c> bytes) and a flag indicating whether the
/// audit copy hit the cap. Returns <c>(null, false)</c> when no bytes
/// were captured, mirroring the request-body empty contract.
/// </summary>
public (string? body, bool truncated) GetCapturedBody()
{
var length = (int)_captured.Length;
if (length == 0)
{
return (null, false);
}
var truncated = length > _capBytes;
var bytes = _captured.GetBuffer();
var bytesForString = truncated ? _capBytes : length;
var content = DecodeUtf8Bounded(bytes, bytesForString, cutAtValidBytes: truncated);
return (string.IsNullOrEmpty(content) ? null : content, truncated);
}
protected override void Dispose(bool disposing)
{
if (!_disposed)
{
if (disposing)
{
// Own only the capture stream; the inner sink belongs to
// the framework's response pipeline.
_captured.Dispose();
}
_disposed = true;
}
base.Dispose(disposing);
}
}
}
@@ -14,6 +14,9 @@
<ItemGroup>
<ProjectReference Include="../ScadaLink.Commons/ScadaLink.Commons.csproj" />
<ProjectReference Include="../ScadaLink.Communication/ScadaLink.Communication.csproj" />
<!-- AuditWriteMiddleware reads AuditLogOptions.InboundMaxBytes to bound
per-request request/response audit capture at the source. -->
<ProjectReference Include="../ScadaLink.AuditLog/ScadaLink.AuditLog.csproj" />
</ItemGroup>
<ItemGroup>
@@ -45,7 +45,8 @@ public class AuditLogOptionsBindingTests
"RedactSqlParamsMatching": "@token|@secret"
}
},
"RetentionDays": 180
"RetentionDays": 180,
"InboundMaxBytes": 524288
}
}
""";
@@ -64,6 +65,7 @@ public class AuditLogOptionsBindingTests
Assert.Equal(4096, opts.DefaultCapBytes);
Assert.Equal(32768, opts.ErrorCapBytes);
Assert.Equal(180, opts.RetentionDays);
Assert.Equal(524_288, opts.InboundMaxBytes);
// HeaderRedactList: the Microsoft.Extensions.Configuration list binder
// APPENDS to the default list, so we assert containment rather than
@@ -0,0 +1,53 @@
using ScadaLink.AuditLog.Configuration;
namespace ScadaLink.AuditLog.Tests.Configuration;
/// <summary>
/// Task 1 of <c>docs/plans/2026-05-23-inbound-api-full-response-audit.md</c>:
/// pins the <see cref="AuditLogOptions.InboundMaxBytes"/> default to 1 MiB and
/// the validator bounds to <c>[8 KiB, 16 MiB]</c>. The inbound channel needs a
/// much larger ceiling than the 8 KiB / 64 KiB default/error caps that other
/// channels use, but unbounded would let any caller flood the central
/// <c>AuditLog</c> table with arbitrarily large bodies — hence the upper bound.
/// Companion to <see cref="AuditLogOptionsTests"/> which covers the existing
/// cap-bytes + retention invariants.
/// </summary>
public class AuditLogOptionsValidatorTests
{
[Fact]
public void Validate_InboundMaxBytes_DefaultOptions_IsOneMebibyte()
{
// The doc'd default per docs/plans/2026-05-23-inbound-api-full-response-audit-design.md
// is 1 048 576 bytes (1 MiB). Pin it so a config drift is a test failure,
// not a silent operational surprise.
var opts = new AuditLogOptions();
Assert.Equal(1_048_576, opts.InboundMaxBytes);
}
[Theory]
[InlineData(8_192)] // documented min
[InlineData(1_048_576)] // default
[InlineData(16_777_216)] // documented max
public void Validate_InboundMaxBytes_InRange_Passes(int value)
{
var validator = new AuditLogOptionsValidator();
var opts = new AuditLogOptions { InboundMaxBytes = value };
Assert.True(validator.Validate(null, opts).Succeeded);
}
[Theory]
[InlineData(0)]
[InlineData(8_191)]
[InlineData(16_777_217)]
[InlineData(int.MaxValue)]
public void Validate_InboundMaxBytes_OutOfRange_Fails(int value)
{
var validator = new AuditLogOptionsValidator();
var opts = new AuditLogOptions { InboundMaxBytes = value };
var result = validator.Validate(null, opts);
Assert.False(result.Succeeded);
Assert.Contains(
result.Failures!,
f => f.Contains(nameof(AuditLogOptions.InboundMaxBytes), StringComparison.Ordinal));
}
}
@@ -0,0 +1,133 @@
using System.Text;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Options;
using ScadaLink.AuditLog.Configuration;
using ScadaLink.AuditLog.Payload;
using ScadaLink.Commons.Entities.Audit;
using ScadaLink.Commons.Types.Enums;
namespace ScadaLink.AuditLog.Tests.Payload;
/// <summary>
/// Pins the docs/plans/2026-05-23-inbound-api-full-response-audit-design.md
/// inbound carve-out: ApiInbound rows use InboundMaxBytes (default 1 MiB) for
/// RequestSummary / ResponseSummary truncation, NOT DefaultCapBytes /
/// ErrorCapBytes. Other channels keep the existing caps.
/// </summary>
/// <remarks>
/// Uses a file-local <see cref="StaticMonitor"/> helper mirroring the
/// convention in the sibling Payload tests (TruncationTests,
/// FilterIntegrationTests, BodyRegexRedactionTests, etc.) — the
/// <c>TestOptionsMonitor&lt;T&gt;</c> helper referenced by the plan is a
/// private nested class inside <c>AuditLogOptionsBindingTests</c> and thus
/// not reachable from this file.
/// </remarks>
public class InboundChannelCapTests
{
private static AuditEvent MakeInbound(
AuditStatus status,
string? request = null,
string? response = null) =>
new()
{
EventId = Guid.NewGuid(),
OccurredAtUtc = DateTime.UtcNow,
Channel = AuditChannel.ApiInbound,
Kind = AuditKind.InboundRequest,
Status = status,
RequestSummary = request,
ResponseSummary = response,
};
[Fact]
public void ApiInbound_Delivered_RequestBody_BelowInboundMaxBytes_NotTruncated()
{
// Body well above the legacy 8 KiB default cap but under the 1 MiB
// inbound ceiling — must NOT truncate.
var body = new string('a', 100_000);
var opts = new AuditLogOptions(); // defaults
var filter = new DefaultAuditPayloadFilter(
new StaticMonitor(opts),
NullLogger<DefaultAuditPayloadFilter>.Instance);
var result = filter.Apply(MakeInbound(AuditStatus.Delivered, request: body));
Assert.False(result.PayloadTruncated);
Assert.Equal(100_000, Encoding.UTF8.GetByteCount(result.RequestSummary!));
}
[Fact]
public void ApiInbound_Delivered_ResponseBody_BelowInboundMaxBytes_NotTruncated()
{
var body = new string('a', 100_000);
var opts = new AuditLogOptions();
var filter = new DefaultAuditPayloadFilter(
new StaticMonitor(opts),
NullLogger<DefaultAuditPayloadFilter>.Instance);
var result = filter.Apply(MakeInbound(AuditStatus.Delivered, response: body));
Assert.False(result.PayloadTruncated);
Assert.Equal(100_000, Encoding.UTF8.GetByteCount(result.ResponseSummary!));
}
[Fact]
public void ApiInbound_Failed_BodyAboveInboundMaxBytes_TruncatedToInboundMaxBytes()
{
// Even on error rows, the inbound cap is InboundMaxBytes (NOT ErrorCapBytes).
var opts = new AuditLogOptions { InboundMaxBytes = 16_384 };
var oversized = new string('z', 50_000);
var filter = new DefaultAuditPayloadFilter(
new StaticMonitor(opts),
NullLogger<DefaultAuditPayloadFilter>.Instance);
var result = filter.Apply(MakeInbound(AuditStatus.Failed, response: oversized));
Assert.True(result.PayloadTruncated);
Assert.True(Encoding.UTF8.GetByteCount(result.ResponseSummary!) <= 16_384);
}
[Fact]
public void ApiOutbound_StillUsesDefaultCap_NotInboundMaxBytes()
{
// Regression guard: lifting the inbound cap MUST NOT change other
// channels. An ApiOutbound 100 KB body still hits the 8 KiB cap.
var opts = new AuditLogOptions();
var body = new string('a', 100_000);
var filter = new DefaultAuditPayloadFilter(
new StaticMonitor(opts),
NullLogger<DefaultAuditPayloadFilter>.Instance);
var evt = new AuditEvent
{
EventId = Guid.NewGuid(),
OccurredAtUtc = DateTime.UtcNow,
Channel = AuditChannel.ApiOutbound,
Kind = AuditKind.ApiCall,
Status = AuditStatus.Delivered,
RequestSummary = body,
};
var result = filter.Apply(evt);
Assert.True(result.PayloadTruncated);
Assert.True(Encoding.UTF8.GetByteCount(result.RequestSummary!) <= opts.DefaultCapBytes);
}
/// <summary>
/// IOptionsMonitor test double — returns the same snapshot on every read,
/// no change-token plumbing required for these tests. Mirrors the helper
/// used in <c>TruncationTests</c>, <c>FilterIntegrationTests</c>, etc.
/// </summary>
private sealed class StaticMonitor : IOptionsMonitor<AuditLogOptions>
{
private readonly AuditLogOptions _value;
public StaticMonitor(AuditLogOptions value) => _value = value;
public AuditLogOptions CurrentValue => _value;
public AuditLogOptions Get(string? name) => _value;
public IDisposable? OnChange(Action<AuditLogOptions, string?> listener) => null;
}
}
@@ -4,6 +4,8 @@ using System.Text.Json;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Routing;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Options;
using ScadaLink.AuditLog.Configuration;
using ScadaLink.Commons.Entities.Audit;
using ScadaLink.Commons.Interfaces.Services;
using ScadaLink.Commons.Types.Enums;
@@ -79,8 +81,32 @@ public class AuditWriteMiddlewareTests
private static AuditWriteMiddleware CreateMiddleware(
RequestDelegate next,
ICentralAuditWriter writer) =>
new(next, writer, NullLogger<AuditWriteMiddleware>.Instance);
ICentralAuditWriter writer,
AuditLogOptions? options = null) =>
new(
next,
writer,
NullLogger<AuditWriteMiddleware>.Instance,
new StaticAuditLogOptionsMonitor(options ?? new AuditLogOptions()));
/// <summary>
/// File-local <see cref="IOptionsMonitor{TOptions}"/> test double — returns the
/// same snapshot on every read, no change-token plumbing required. Mirrors the
/// <c>StaticMonitor</c> pattern in
/// <c>tests/ScadaLink.AuditLog.Tests/Payload/InboundChannelCapTests.cs</c>.
/// </summary>
private sealed class StaticAuditLogOptionsMonitor : IOptionsMonitor<AuditLogOptions>
{
private readonly AuditLogOptions _value;
public StaticAuditLogOptionsMonitor(AuditLogOptions value) => _value = value;
public AuditLogOptions CurrentValue => _value;
public AuditLogOptions Get(string? name) => _value;
public IDisposable? OnChange(Action<AuditLogOptions, string?> listener) => null;
}
// ---------------------------------------------------------------------
// 1. Happy path — InboundRequest/Delivered/HttpStatus 200
@@ -487,4 +513,180 @@ public class AuditWriteMiddlewareTests
Assert.NotNull(evt.DurationMs);
Assert.True(evt.DurationMs >= 0);
}
// ---------------------------------------------------------------------
// Response body capture — Audit Log #23 (inbound full-response feature).
// Until the M5-deferred work landed, ResponseSummary was always null.
// These tests pin the new contract: the middleware wraps Response.Body,
// runs the pipeline, copies the buffered bytes back to the real stream,
// and stashes a UTF-8 string copy on ResponseSummary.
// ---------------------------------------------------------------------
[Fact]
public async Task ResponseBody_IsCaptured_OnResponseSummary()
{
var writer = new RecordingAuditWriter();
var ctx = BuildContext();
var responseJson = "{\"result\":42}";
var mw = CreateMiddleware(async hc =>
{
hc.Response.StatusCode = 200;
hc.Response.ContentType = "application/json";
await hc.Response.WriteAsync(responseJson);
}, writer);
await mw.InvokeAsync(ctx);
var evt = Assert.Single(writer.Events);
Assert.Equal(responseJson, evt.ResponseSummary);
}
[Fact]
public async Task ResponseBody_IsForwardedToOriginalStream_DownstreamReadersSeeIt()
{
// Wrapping the response body must be TRANSPARENT — the real client
// stream still receives every byte the pipeline wrote.
var writer = new RecordingAuditWriter();
var ctx = BuildContext();
var captured = new MemoryStream();
ctx.Response.Body = captured; // simulate the client/test sink
var responseJson = "{\"ok\":true}";
var mw = CreateMiddleware(async hc =>
{
hc.Response.StatusCode = 200;
await hc.Response.WriteAsync(responseJson);
}, writer);
await mw.InvokeAsync(ctx);
Assert.Equal(responseJson, Encoding.UTF8.GetString(captured.ToArray()));
}
[Fact]
public async Task ResponseBody_Empty_LeavesResponseSummaryNull()
{
// No bytes written => null, not empty-string. Mirrors the request-body
// contract in ReadBufferedRequestBodyAsync.
var writer = new RecordingAuditWriter();
var ctx = BuildContext();
var mw = CreateMiddleware(hc =>
{
hc.Response.StatusCode = 204;
return Task.CompletedTask;
}, writer);
await mw.InvokeAsync(ctx);
var evt = Assert.Single(writer.Events);
Assert.Null(evt.ResponseSummary);
Assert.Equal(204, evt.HttpStatus);
}
[Fact]
public async Task ResponseBody_OnHandlerThrow_BodyCapturedUpToTheThrow()
{
// If the handler writes some bytes then throws, the audit row still
// surfaces whatever the framework had flushed. The middleware re-throws
// (audit is best-effort, the request's error path stays authoritative).
var writer = new RecordingAuditWriter();
var ctx = BuildContext();
var boom = new InvalidOperationException("kaboom");
var mw = CreateMiddleware(async hc =>
{
hc.Response.StatusCode = 500;
await hc.Response.WriteAsync("partial");
throw boom;
}, writer);
var thrown = await Assert.ThrowsAsync<InvalidOperationException>(
() => mw.InvokeAsync(ctx));
Assert.Same(boom, thrown);
var evt = Assert.Single(writer.Events);
Assert.Equal(AuditStatus.Failed, evt.Status);
Assert.Equal("partial", evt.ResponseSummary);
}
// ---------------------------------------------------------------------
// Bounded audit capture — memory safety follow-up. The capture site now
// honours AuditLogOptions.InboundMaxBytes at READ time (not just at
// filter-time), so a 500 MiB body cannot transiently allocate 500 MiB of
// string. The cap is local to the AUDIT copy; downstream readers and the
// real client still see every byte.
// ---------------------------------------------------------------------
[Fact]
public async Task RequestBody_AboveInboundMaxBytes_TruncatedToCap_PayloadTruncatedTrue()
{
// 4 KiB cap, 20 KB body — the audit copy must be UTF-8 byte-safe
// capped at 4 KiB AND PayloadTruncated must flip, while the
// downstream handler still sees the full 20 KB payload.
const int cap = 4096;
var bigBody = new string('a', 20_000);
var writer = new RecordingAuditWriter();
var ctx = BuildContext(body: bigBody);
string? observedAfterMiddleware = null;
var mw = CreateMiddleware(
async hc =>
{
using var reader = new StreamReader(hc.Request.Body);
observedAfterMiddleware = await reader.ReadToEndAsync();
hc.Response.StatusCode = 200;
},
writer,
options: new AuditLogOptions { InboundMaxBytes = cap });
await mw.InvokeAsync(ctx);
// (iii) Downstream handler still sees the FULL body — the cap applied
// only to the audit copy.
Assert.Equal(bigBody, observedAfterMiddleware);
var evt = Assert.Single(writer.Events);
// (i) Audit copy bounded at cap bytes (UTF-8 byte count).
Assert.NotNull(evt.RequestSummary);
Assert.True(
Encoding.UTF8.GetByteCount(evt.RequestSummary!) <= cap,
$"RequestSummary byte count {Encoding.UTF8.GetByteCount(evt.RequestSummary!)} exceeded cap {cap}");
// (ii) Truncation flag set by the middleware (the filter will OR its
// own determination on top, but the middleware MUST set it itself).
Assert.True(evt.PayloadTruncated);
}
[Fact]
public async Task ResponseBody_AboveInboundMaxBytes_TruncatedToCap_ClientStillReceivesAllBytes_PayloadTruncatedTrue()
{
// 4 KiB cap, 20 KB response — the test sink (acts as the real client)
// MUST receive all 20 KB while the audit copy is bounded at 4 KiB.
const int cap = 4096;
var bigResponse = new string('b', 20_000);
var writer = new RecordingAuditWriter();
var ctx = BuildContext();
var captured = new MemoryStream();
ctx.Response.Body = captured; // stand-in for the client sink
var mw = CreateMiddleware(
async hc =>
{
hc.Response.StatusCode = 200;
await hc.Response.WriteAsync(bigResponse);
},
writer,
options: new AuditLogOptions { InboundMaxBytes = cap });
await mw.InvokeAsync(ctx);
// Client sink received every byte — the forwarding wrap is transparent.
Assert.Equal(bigResponse, Encoding.UTF8.GetString(captured.ToArray()));
var evt = Assert.Single(writer.Events);
// Audit copy bounded at cap bytes.
Assert.NotNull(evt.ResponseSummary);
Assert.True(
Encoding.UTF8.GetByteCount(evt.ResponseSummary!) <= cap,
$"ResponseSummary byte count {Encoding.UTF8.GetByteCount(evt.ResponseSummary!)} exceeded cap {cap}");
Assert.True(evt.PayloadTruncated);
}
}
@@ -6,6 +6,8 @@ using Microsoft.AspNetCore.TestHost;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Options;
using ScadaLink.AuditLog.Configuration;
using ScadaLink.Commons.Entities.Audit;
using ScadaLink.Commons.Interfaces.Services;
using ScadaLink.Commons.Types.Enums;
@@ -145,7 +147,8 @@ public class MiddlewareOrderTests
// instantiates the type correctly.
_ => Task.CompletedTask,
writer,
NullLogger<AuditWriteMiddleware>.Instance));
NullLogger<AuditWriteMiddleware>.Instance,
new StaticAuditLogOptionsMonitor(new AuditLogOptions())));
services.AddRouting();
services.AddAuthorization();
services.AddAuthentication("TestScheme")
@@ -233,4 +236,22 @@ public class MiddlewareOrderTests
return Task.FromResult(AuthenticateResult.Success(ticket));
}
}
/// <summary>
/// File-local <see cref="IOptionsMonitor{TOptions}"/> test double — returns the
/// same snapshot on every read. Mirrors the helper in
/// <c>AuditWriteMiddlewareTests</c>.
/// </summary>
private sealed class StaticAuditLogOptionsMonitor : IOptionsMonitor<AuditLogOptions>
{
private readonly AuditLogOptions _value;
public StaticAuditLogOptionsMonitor(AuditLogOptions value) => _value = value;
public AuditLogOptions CurrentValue => _value;
public AuditLogOptions Get(string? name) => _value;
public IDisposable? OnChange(Action<AuditLogOptions, string?> listener) => null;
}
}