feat(inbound-api): standby central node relays POST /api/* to the active peer instead of 503

A caller addressing a standby central node directly (or hitting Traefik inside its
/health/active probe window after a failover) used to get 503 STANDBY_NODE. The
standby now relays the request verbatim to the active peer over HTTP
(IStandbyRequestForwarder / HttpStandbyRequestForwarder) and replays the response;
method scripts and Route.To() still execute only on the active node.

- ScadaBridge:InboundApi:StandbyForwardPeerEndpoints (static list of the OTHER
  central node's HTTP base URL; empty = old 503 behaviour) + StandbyForwardTimeout
  (60s), both startup-validated.
- Peers tried in order: peer 503 stamped X-ScadaBridge-Standby, connection failure
  or timeout → next peer; all fail → 503 ACTIVE_NODE_UNREACHABLE. A 503 without
  the marker is the active node's own answer and is replayed.
- One-hop loop guard via X-ScadaBridge-Forwarded; 413 body cap now runs BEFORE
  the active-node check so a standby never buffers/relays an oversized body.
- AuditWriteMiddleware skips its own ApiInbound row for a relayed request so a
  call is audited once, on the node that executed it.
- Rig + env2 central nodes configured; spec (Component-InboundAPI.md → Standby
  forwarding), Component-Host.md options row, CLAUDE.md updated.
- 20 new tests (filter decision + relay mechanics + validator).

Claude-Session: https://claude.ai/code/session_019ZxXT5sxZs5XrtdwYzJDPV
This commit is contained in:
Joseph Doherty
2026-08-28 13:34:20 -04:00
parent f07ba5cff5
commit a9b23556eb
16 changed files with 839 additions and 31 deletions
@@ -48,4 +48,25 @@ public class InboundApiOptions
/// serve minutes-old method definitions.
/// </summary>
public TimeSpan ApiMethodCacheTtl { get; set; } = TimeSpan.FromSeconds(5);
/// <summary>
/// Standby-node forwarding (2026-08-28): the HTTP base URLs of the OTHER central
/// node(s) — e.g. <c>["http://scadabridge-central-b:5000"]</c> on central-a — that a
/// standby relays <c>POST /api/{methodName}</c> to instead of answering 503
/// <c>STANDBY_NODE</c>. Tried in order; the first peer that is not itself a standby
/// serves the call. Empty (the default) keeps the pre-forwarding behaviour: a standby
/// refuses with 503. Method scripts still execute only on the active node — the
/// standby is a pure relay (see <see cref="HttpStandbyRequestForwarder"/>). Each entry
/// must be an absolute <c>http</c>/<c>https</c> URL; never list this node's own address
/// (it would cost one wasted round-trip per call — the loop guard stops it bouncing).
/// </summary>
public List<string> StandbyForwardPeerEndpoints { get; set; } = new();
/// <summary>
/// Per-peer deadline for one relay attempt (connect + the active node's full method
/// execution). Must comfortably exceed <see cref="DefaultMethodTimeout"/> and the
/// longest per-method timeout in use, otherwise the standby gives up on a peer that
/// is still legitimately working and moves on to the next one. Default 60 s.
/// </summary>
public TimeSpan StandbyForwardTimeout { get; set; } = TimeSpan.FromSeconds(60);
}