fix(historian): gate historization capture on the Primary, quarantine a poison tag (#537, #538)

Both defects were found by #491's live gate against a real HistorianGateway and filed rather than
fixed at the time, because each one's fix is a design decision rather than a patch.

#537 — both nodes of a redundant pair captured every value, so the historian received duplicate
live samples for every historized tag (measured: the gateway's write counter advanced twice per
value and halved when one central was stopped).

The recorder now reads a capture gate per value, fed from the new
PrimaryGatePolicy.ShouldRecordHistorizedValues through IHistorizationRoleView — a third verdict on
the same RedundancyRoleView object, registered unconditionally on driver nodes rather than
alongside the alarm section's IRedundancyRoleView (which only exists when that section is enabled;
piggybacking is the #511 registration accident).

Two things are deliberate and easy to get backwards:

  * It gates the CAPTURE, not the drain — the opposite of the alarm sink. That queue is in LocalDb
    and replicates, so the Secondary already holds the Primary's rows. The historization outbox is
    a node-local FasterLog replicating to nobody, so a drain-only gate would leave the standby
    appending forever to a queue no one empties: unbounded disk, then a flood of stale duplicates
    the moment it is promoted. Entries a node already captured are still drained by that node.

  * The peer it stands down for must RECORD THE SAME REFS, not merely hold the same rows — a
    Primary in this node's own per-cluster mesh. A node carrying no cluster-scoped role cannot
    establish that and never stands down: under a fleet-wide election the elected Primary is
    usually running a different cluster's configuration, and deferring to it would end this
    cluster's historization silently, with the drain reporting success on an empty outbox forever.
    That is the scope mismatch that once suspended the alarm drain on every node of the rig.

An unknown role captures, matching the alarm drain — a duplicate sample is recoverable, a hole in
the trend is not.

#538 — the drain writes every per-tag group and then acks all-or-nothing (deliberate: RemoveAsync
truncates THROUGH an id, so partial acking could drop older unwritten entries). One permanently
failing tag therefore pinned the FIFO head while every healthy tag in the batch was re-submitted
to the historian on every pass, indefinitely — duplicate samples and an outbox that never
truncated, observed live at 8.3 MB from a single unprovisionable tag.

The drain now tracks consecutive failures per historian tag and abandons that tag's entries once
it exceeds MaxWriteAttempts (new, default 5), so the head advances. The counter only advances on a
pass where some OTHER tag wrote successfully: that is what separates "this tag is poison" from
"the backend is down". A count-only quarantine looks identical on the poison-tag case and silently
discards the whole backlog during an ordinary gateway restart, turning a duplicate-samples bug
into a lost-history one. Abandonment is also deferred until every failing tag is over budget, so
nothing is discarded on a pass that could not have advanced the head anyway. Each abandonment logs
a Warning naming the tag; the total rides RecorderStatus.AbandonedEntries.

Capacity also gains a non-zero default (500,000 entries) as the disk backstop for the one case the
quarantine deliberately cannot resolve — a historian refusing every write, where retaining is
correct. Set 0 to restore unbounded retention.

Every new test was falsified by reverting the production change it guards, and each one reddens
alone:

  * drop the backend-reachable evidence -> only A_backend_that_fails_every_tag_abandons_nothing
  * never abandon -> only A_permanently_failing_tag_is_abandoned_so_healthy_tags_stop_being_rewritten
  * latch the capture gate at construction -> only The_capture_gate_is_re_read_per_value...
  * treat any Primary as a scope peer -> only the two scope-mismatch cases

Full suite green. (Four OpcUaClient.Browser failures during the run were the opc-plc fixture being
down on the shared Docker host; brought it back up and they pass.)

Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo

CLAUDE.md

@@ -822,7 +822,8 @@ only the recorder + outbox knobs:
 | `CommitIntervalMs` | `100` | Periodic-mode commit cadence; required `> 0` only under `Periodic` |
 | `DrainBatchSize` | `64` | Entries peeked + written per drain pass |
 | `DrainIntervalSeconds` | `2` | Steady drain cadence (and post-success reschedule) |
-| `Capacity` | `0` | Max un-acked outbox entries before drop-oldest; `0` = unbounded |
+| `Capacity` | `500000` | Max un-acked outbox entries before drop-oldest; `0` = unbounded. Non-zero **by default since #538** — the disk backstop for a historian refusing *every* write, the one case the per-tag quarantine deliberately cannot resolve |
+| `MaxWriteAttempts` | `5` | #538: consecutive failed drain passes one tag may accumulate — counted **only across passes where other tags wrote successfully** — before its queued entries are abandoned so the FIFO head can advance |
 | `MinBackoffSeconds` | `1` | Initial retry backoff after a failed drain pass |
 | `MaxBackoffSeconds` | `30` | Cap on the exponential retry backoff |
 
@@ -925,9 +926,9 @@ to know:
   the captured context, so the caller still resumes on the actor thread. The fake must complete on a
   **pool** thread (`await Task.Delay(1).ConfigureAwait(false)`), as a real per-entry fsync does. Only
   reverting the production fix exposed that the first regression test proved nothing.
-- **Both pair nodes historize the same value** — the recorder has **no Primary gate**, unlike the alarm
-  store-and-forward sink. A redundant pair therefore sends duplicate live values per historized tag.
-  Deliberately recorded, not "fixed": gating it introduces a failover window with no capture.
+- **Both pair nodes historized the same value** — the recorder had **no Primary gate**, unlike the alarm
+  store-and-forward sink, so a redundant pair sent duplicate live values per historized tag. Filed as
+  **#537** and **fixed** — see the paragraph below.
 
 **Gate COMPLETE — #491 CLOSED 2026-08-04.** Outbox append, drain, `WriteLiveValues`,
 **restart-convergence**, `EnsureTags` provisioning (`ensured=2 failed=0`) and **read-back** all ✅. The
@@ -935,13 +936,34 @@ read-back leg ran against the licensed `wonder-sql-vd03` with the VPN up: two re
 *distinct non-zero* constants (`8224`, `120`) each read back as its own value — a `0`-valued source
 would have proved nothing (the #536 trap).
 
-⚠️ **A second defect was found and FILED, not fixed (#538): one permanently-failing tag re-writes every
-healthy tag on every drain pass, forever.** `DrainOnceAsync` writes **every** per-tag group and then acks
-**all-or-nothing**, so one bad tag stops the FIFO head advancing while the good tags' values are
-re-submitted to the historian on every retry — duplicate samples, and an outbox that never truncates
-(`Capacity` defaults to `0` = unbounded; observed at 8.3 MB). The all-or-nothing ack is deliberate
-(`RemoveAsync` truncates *through* an id, so partial acking could drop older unwritten entries) — the
-consequence is what is undocumented.
+**Both gate findings are now CLOSED — #537 (Primary gate) and #538 (poison tag). Two things to know
+before touching either.**
+
+- **#538 — the discriminator is "did another tag write this pass?", NOT a failure count.** `DrainOnceAsync`
+  writes every per-tag group and acks **all-or-nothing** (deliberate: `RemoveAsync` truncates *through* an
+  id, so partial acking could drop older unwritten entries). One permanently-failing tag therefore pinned
+  the FIFO head while every *healthy* tag was re-submitted on every pass, forever — duplicate samples, an
+  outbox that never truncated (observed 8.3 MB; signature = the gateway's `ok` **and** `error` counters
+  climbing together). The drain now quarantines a tag after `MaxWriteAttempts` consecutive failures, but
+  ⚠️ **only counts a failure on a pass where some other tag succeeded** — otherwise an ordinary gateway
+  restart (where everything fails) would discard the entire backlog, converting a duplicate-samples bug
+  into a lost-history one. A count-only quarantine passes the poison-tag test and fails nothing else, so
+  the load-bearing test is the **falsification control** (`A_backend_that_fails_every_tag_abandons_nothing`).
+  Abandonment is also deferred until *every* failing tag is over budget, so nothing is discarded on a pass
+  that could not have advanced the head anyway.
+- **#537 — it gates the CAPTURE, not the drain, and that is the opposite of the alarm sink.** The alarm
+  queue is in LocalDb and *replicates*, so the Secondary holds the Primary's rows and gating the drain
+  loses nothing. The historization outbox is a node-local FasterLog replicating to nobody: gating only its
+  drain leaves the standby appending forever to a queue no one empties (unbounded disk, then a flood of
+  stale duplicates on promotion). ⚠️ **And the peer that matters is one recording the SAME refs, not one
+  holding the same rows** — a node stands down only for a Primary in its own per-cluster mesh, and a node
+  with no cluster-scoped role never stands down at all. Under a fleet-wide election the elected Primary is
+  usually running another cluster's configuration, and deferring to it ends this cluster's historization
+  *silently* (the drain reports success on an empty outbox forever) — the same scope mismatch that once
+  suspended the alarm drain on every node of the rig at once. An unknown role captures. New verdict on
+  `RedundancyRoleView` behind **`IHistorizationRoleView`**, registered unconditionally on driver nodes
+  (NOT alongside the alarm `IRedundancyRoleView`, which only exists when that section is enabled — the
+  #511 registration accident).
 
 Also re-confirmed live: **`EnsureTags` provisioning fires only for newly-ADDED historized tags** —
 flipping an existing tag's `historianTagname` produces no provisioning at all