lmxopcua/archreview/plans/R3-01-mesh-grpc-allowanonymous-plan.md

# Design + Implementation Plan — R3-01 Mesh gRPC `AllowAnonymous`

- **Source report:** `archreview/03-server-runtime.md` §S14 (2026-08-19 round-3 re-review at `12b0035a`)
- **Round-1/2 designs consumed:** none — **S14 is NEW this round**. No prior plan, branch or issue has ever targeted it.
- **Plan verified against tree at:** master `7c77a8c7` (clean; code identical to `12b0035a`) — every anchor re-opened and re-checked 2026-08-20
- **Scope:** OVERALL prioritized action item **#1** — "`.AllowAnonymous()` on every mesh gRPC map (the PSK interceptors are the gate); one rig/gate pass with `DisableLogin=false`; a standing CI variant of `MeshGrpcBoundaryTests` with `UseAuthentication`/`UseAuthorization` + the FallbackPolicy" (03/S14, **Critical**).
- **Cross-cutting requirement (OVERALL theme 2, "verification that cannot see its own blind spot"):** the fix is one line per map; **the plan's real deliverable is the verification that would have caught it.** Two standing guards (a real-auth-stack boundary test + an endpoint-metadata census, both with falsifiability controls) and one rig change that makes the production middleware stack reachable on demand (`Security__Auth__DisableLogin` parameterised, default unchanged). A fix without those guards leaves the *next* map free to re-open the Critical.
- **Cross-cutting requirement (OVERALL theme 4, "one rule stated twice diverges"):** the five map sites move behind **one** mapper (`MapOtOpcUaMeshGrpcServices`) that both Program.cs and the census test drive, so the guard can never be testing a hand-copied shape — which is precisely how the existing listener tests (`ConfigServeListenerTests`, `LocalDbSyncListenerTests`, `MeshGrpcBoundaryTests`) came to build pipelines that omit the middleware that breaks production.

## Verification summary

S14 is **CONFIRMED at `7c77a8c7`** — not stale, not partially fixed, and every line anchor in the report is exact. One claim in the finding is **over-stated** and one is **incomplete**; both are corrected below and both change the work (they add one non-chainable map and one extra affected caller).

| Report anchor | Claim | Status at `7c77a8c7` |
|---|---|---|
| `Security/ServiceCollectionExtensions.cs:143-148` | global `FallbackPolicy = RequireAuthenticatedUser` under the cookie scheme | **Exact.** File is `src/Server/ZB.MOM.WW.OtOpcUa.**Security**/ServiceCollectionExtensions.cs` (note: not under `Host/`, as the report's shorthand reads). `AddAuthorization` at `:143`, `FallbackPolicy` built `:145-148` naming `CookieAuthenticationDefaults.AuthenticationScheme`. |
| `Security/ServiceCollectionExtensions.cs:82-98` | `DisableLogin` registers a pass-everything handler **under the cookie scheme name** | **Exact.** `authBuilder.AddScheme<AuthenticationSchemeOptions, AutoLoginAuthenticationHandler>(CookieAuthenticationDefaults.AuthenticationScheme, …)` at `:87-88`, with the "FallbackPolicy + FleetAdmin + DriverOperator … pass with all roles — zero policy changes" comment at `:84-86`. The masking mechanism is exactly as described. |
| `Host/Program.cs:520-523` | fallback registered on `hasAdmin` nodes only | **Exact.** `if (hasAdmin) { builder.Services.AddOtOpcUaAuth(builder.Configuration); … }` at `:520-523`. |
| `Host/Program.cs:732-733` | `UseAuthentication`/`UseAuthorization` in the pipeline | **Exact**, and **also inside `if (hasAdmin)`** (`:730-741`) — which is what makes the driver-only exemption structural rather than incidental. `UseAntiforgery()` follows at `:734` (harmless for gRPC: the antiforgery middleware only validates endpoints carrying `IAntiforgeryMetadata`). |
| `Host/Program.cs:724-728` | `MapStaticAssets().AllowAnonymous()` proves the author knew about the fallback | **Exact.** The comment at `:724-727` states the failure mode verbatim ("401s every CSS/JS request"). Two further precedents: `AdminUI/Api/DeployApiEndpoints.cs:61` (`.AllowAnonymous()  // gated by the API key, not the cookie auth fallback`) and `Host/Observability/ObservabilityExtensions.cs:56` (`app.MapZbMetrics().AllowAnonymous()`). **Three** call sites already apply the pattern; the mesh maps are the omission. |
| `Host/Program.cs:749, 757, 767, 782-783` | five mesh maps, none carrying `AllowAnonymous` | **Exact line-for-line**, and **complete**: `grep -rn "MapGrpcService\|MapZbLocalDbSync" src` returns exactly these five sites and nothing else. |
| `MeshGrpcBoundaryTests.cs:334-346` | builds a pipeline with interceptors but no auth stack | **Confirmed, minor drift** — the builder block is `:333-348` (`WebApplication.CreateBuilder()` at `:333`, `AddGrpc(o => o.Interceptors.Add<MeshCommandAuthInterceptor>())` at `:342`, `MapGrpcService<NodeCommandGrpcService>()` at `:347`). No `AddAuthentication`/`AddAuthorization`/`UseAuthentication`/`UseAuthorization` anywhere in the file. **Additional gap the report does not name:** the class only ever builds the **driver-node** (`NodeCommandService`) shape — the *central serve side* (`DeploymentArtifactService` + `CentralCommandService`), which is the fleet-wide choke point, has no boundary test at all. `ConfigServeListenerTests.cs:33-59` and `LocalDbSyncListenerTests.cs:25-27` build Kestrel-binding-only apps and say so in their own remarks ("shaped exactly like `Program.cs`'s … block"). |

### Which maps are actually affected — determined, not assumed

The fallback reaches an endpoint iff (a) the authorization middleware is in this node's pipeline, and (b) the endpoint carries no `IAuthorizeData` and no `IAllowAnonymous` metadata. Port is **irrelevant**: the dedicated h2c listeners (`Program.cs:692-703`) are extra Kestrel endpoints on the **same `WebApplication`**, so they share one middleware pipeline and one endpoint table. There is no `RequireHost`, no per-port branch and no second host anywhere in `Program.cs`. So:

| Map | Role gate | Affected on a login-enabled node? |
|---|---|---|
| `DeploymentArtifactGrpcService` (`:757`) | `hasAdmin && ConfigServe:GrpcListenPort > 0` | **YES — always.** Admin-only map on an admin-only pipeline. Every `FetchAndCache` node's artifact fetch. |
| `CentralCommandGrpcService` (`:767`) | same | **YES — always.** Every node's `ReportApply` deploy ack, from *every* cluster. |
| `TelemetryStreamGrpcService` (`:782`) | `hasDriver && Telemetry:GrpcListenPort > 0` | **YES on a fused `admin,driver` node** (the docker-dev central pair, and any production fused central). **NO on a driver-only node.** |
| `NodeCommandGrpcService` (`:783`) | same | **YES on a fused node / NO on driver-only** — same pipeline. Central→central command dispatch is affected; central→site dispatch is not. |
| `MapZbLocalDbSync` (`:749`) | `hasDriver && LocalDb:SyncListenPort > 0` | **Conditionally — and NOT in the current fleet.** The report lists it unqualified; in fact the only nodes that set `LocalDb__SyncListenPort` are the driver-only **site-a** pair (`docker-dev/docker-compose.yml:657`), where no authorization middleware exists. It becomes affected the moment a **fused** node sets a sync port. Fixed anyway (it is one config key away from being a live outage) — and it is the one map that **cannot** take `.AllowAnonymous()` (see below). |

**Driver-only nodes: confirmed unaffected, structurally.** `AddOtOpcUaAuth` is the only caller of `AddAuthorization` in the host graph (grep-verified), and it sits inside `if (hasAdmin)` (`:520-523`); `UseAuthentication`/`UseAuthorization` sit inside `if (hasAdmin)` (`:730-741`). A driver-only node therefore has **no authorization middleware at all** and no fallback policy to apply. The report's claim holds. (This is worth a standing test, not just a reading — a future "register auth everywhere for consistency" change would silently break every site node's telemetry and command ingress. T6.)

**Correction that changes the work:** `MapZbLocalDbSync` returns **`IEndpointRouteBuilder`**, not an `IEndpointConventionBuilder`, so `.AllowAnonymous()` **cannot be chained onto it** (verified by reading the method signature out of `ZB.MOM.WW.LocalDb.Replication` 0.2.1's metadata: `MapZbLocalDbSync : returns IEndpointRouteBuilder (IEndpointRouteBuilder)`). The library also marks nothing anonymous itself — the assembly contains no `AllowAnonymous`/`Authorize` metadata reference at all. That map needs a different mechanism (resolved below).

**Extra blast radius the finding does not spell out** (fused central, `DisableLogin=false`, `MeshTransport:Mode=Grpc`): central-1/central-2 carry `ClusterNode` rows with a `GrpcPort` (`docker-dev/seed/seed-clusters.sql:71-79`), and `CentralCommunicationActor` dials **one call per enabled row**, its own pair included. So the MAIN pair's *own* deploy dispatch 401s against its *own* `NodeCommandService`, and central's telemetry dialer 401s against its own pair's `TelemetryStreamService`. A login-enabled central pair therefore breaks the deploy plane for **all six** rig nodes, not only the four sites.

Supporting facts verified: all five interceptors are **fail-closed** — a blank/whitespace expected key rejects every call, and the presented token is compared with `CryptographicOperations.FixedTimeEquals` (`ConfigServeAuthInterceptor.cs:112-141`, `TelemetryStreamAuthInterceptor.cs:112-145`, `MeshCommandAuthInterceptor.cs:125-154`, `CentralCommandAuthInterceptor.cs:142-190`, `LocalDbSyncAuthInterceptor.cs:117-147`); each returns early on a non-matching method prefix, so the shared `AddGrpc` pipeline (`Program.cs:586-602`) is a pass-through for foreign paths. `Program.cs:574` already states the design contract in so many words: "the interceptor is the ONLY inbound auth on any endpoint (the libraries verify nothing)". Metrics are already anonymous (`ObservabilityExtensions.cs:56`); `MapOtOpcUaHealth` delegates to `MapZbHealth()` (`Health/HealthEndpoints.cs:104`) and `ZB.MOM.WW.Health` 0.3.0's assembly *does* reference `AllowAnonymous` — the census in T2 settles it either way rather than trusting the string. Rig masking confirmed: `Security__Auth__DisableLogin: "true"` on central-1 (`docker-dev/docker-compose.yml:385`) and central-2 (`:511`); both are `OTOPCUA_ROLES=admin,driver,cluster-MAIN` (`:298`, `:448`).

### Open design choices — resolved in this plan

1. **Fix mechanism: `.AllowAnonymous()` per map**, not a policy, not a scheme, not a second host. The interceptors are verified fail-closed, the codebase already applies this exact pattern at three sites for the same reason, and it is the only option that leaves the failure statuses dialers already handle untouched. Alternatives weighed in *Proposed design*.
2. **LocalDbSync mechanism: a single `app.MapGroup("").AllowAnonymous()` scoped to that one call.** The library map is not chainable (verified above). The empty prefix leaves the gRPC route patterns byte-identical; the group is deliberately **not** extended over the four chainable maps, so the deviation stays as small as the problem.
3. **Extract the five maps behind `MapOtOpcUaMeshGrpcServices`** (Host, one file) rather than testing a hand-copied shape. This is the theme-4 move and it is what makes the census a *production* assertion. Rejected alternative: leave the maps inline and have the census rebuild them in test code — that is the same construction that produced S14.
4. **Two guards, not one.** The census (metadata) catches a map added to the helper without the convention; a **Program.cs source-scan** (T7, following the in-repo idiom at `tests/Drivers/Cli/…/FlushLoggingConventionTests.cs:28-45`) catches a map added to `Program.cs` *outside* the helper. Neither subsumes the other.
5. **Assert the interceptor's detail string, never the status alone.** `ConfigServeAuthInterceptor` rejects with `StatusCode.Unauthenticated` (`:127`), and `Grpc.Net.Client` maps an HTTP **401** to `Unauthenticated` as well — so "no-PSK call fails with Unauthenticated" is **vacuous on the artifact service**: it passes identically before and after the fix. The interceptor's messages are the discriminator (`"Config-serve authentication failed."`, `"Telemetry stream authentication failed."`, `"Node command authentication failed."`).
6. **The load-bearing assertion is the ALLOW.** A blocked call and a refused call both "fail"; only a *correctly-keyed call succeeding through the real auth stack* distinguishes the fixed build. Every deny assertion in T4/T5 is paired with the positive.
7. **Rig: parameterise, don't flip.** `Security__Auth__DisableLogin: "${OTOPCUA_DISABLE_LOGIN:-true}"` on the central pair — default behaviour byte-identical, and the production-faithful mode becomes a documented one-variable flip instead of an edit-and-remember-to-revert. Rejected: making `false` the rig default (every future rig session then needs an LDAP login through a round-robin `:9200`, and the shared GLAuth becomes a hard dependency of the whole rig).
8. **No deploy-plane retry/queueing work here.** Mesh dispatch is fire-and-forget by design (CLAUDE.md §Mesh command transport); S14 is an authorization defect, not a delivery-semantics one. Out of scope.

---

## Finding 1 — 03/S14: The AdminUI cookie `FallbackPolicy` blocks every mesh gRPC ingress on a login-enabled admin node (Critical)

### Restatement

`AddOtOpcUaAuth` installs a global `FallbackPolicy = RequireAuthenticatedUser` bound to the cookie scheme (`Security/ServiceCollectionExtensions.cs:143-148`), on every `hasAdmin` node (`Program.cs:520-523`), with `UseAuthentication`/`UseAuthorization` in the pipeline (`:732-733`). ASP.NET Core's authorization middleware applies the fallback to **every endpoint that carries no authorization metadata** — gRPC endpoints included — and none of the five mesh maps carries `.AllowAnonymous()` (`:749`, `:757`, `:767`, `:782-783`). The PSK bearer is not an ASP.NET authentication scheme, and gRPC interceptors run *inside* the endpoint invocation, i.e. strictly after the authorization middleware, so a correctly-keyed node call is challenged by the cookie handler before the interceptor's verdict can matter.

On a production (login-enabled) central this is a fleet-wide deploy-plane outage: every `FetchAndCache` node's artifact fetch fails on every endpoint (→ the apply fails fail-closed to last-known-good per #485 — loud, not corrupting), every gRPC `ReportApply` ack is dropped (→ deployments seal `TimedOut`), and on a fused central the pair's own command dispatch and telemetry ingress fail too. Nothing caught it because the docker-dev rig runs `DisableLogin=true`, which registers an always-succeeding handler *under the cookie scheme name* (`:82-98`) so the FallbackPolicy passes — i.e. **every live gate of the entire seven-phase mesh program ran with the one middleware that breaks production neutralized** — and because `MeshGrpcBoundaryTests` builds its pipelines without any auth stack (`:333-348`).

### Verification

Confirmed exact at `7c77a8c7` (see the summary table for line-by-line). Three refinements found while verifying:

- **The set of affected maps is role-shaped**, not uniform: two are always affected on a login-enabled node (`DeploymentArtifactGrpcService`, `CentralCommandGrpcService`); two are affected only on a **fused** `admin,driver` node (`TelemetryStreamGrpcService`, `NodeCommandGrpcService`); one (`MapZbLocalDbSync`) is affected only on a fused node that also sets a sync port — a shape **no node in the fleet currently has**. Driver-only site nodes have no authorization middleware at all.
- **`MapZbLocalDbSync` is not chainable** (`returns IEndpointRouteBuilder`), so the "one-line-per-map fix" is one line for four maps and a group wrapper for the fifth.
- **The observable failure is not necessarily a 401.** `CookieAuthenticationEvents`'s default `OnRedirectToLogin` returns 401 only for an *AJAX-shaped* request (`X-Requested-With`); a gRPC call carries no such header, so the challenge is a **302 to `/login`**. Whether the caller then reports `Unknown` ("Bad gRPC response. HTTP status code: 302") or a content-type failure depends on whether the channel's handler follows the redirect. This is why T5 asserts the *interceptor's* identity rather than enumerating failure statuses, and why the live gate greps for `/login`/302/401 together rather than "401".

### Root cause

Two independent decisions, each locally correct:

1. **A global fallback policy is the right default for a Blazor admin surface** — it means a newly-added page is protected by default rather than by remembering an attribute (the same instinct as `PageAuthorizationGuardTests`). Its cost is that *every* endpoint added to that host must now opt out explicitly, and the three prior opt-outs (static assets, deploy API, metrics) were each discovered by something breaking.
2. **The mesh gRPC endpoints authenticate in interceptors** because their credential is a per-cluster PSK, not a browser identity — deliberately documented as "the interceptor is the ONLY inbound auth on any endpoint" (`Program.cs:574`). That statement is true of the *gRPC* pipeline and false of the *HTTP* pipeline that precedes it, and nothing in the build or the tests could hold those two sentences next to each other.

The verification gap is the same shape: every test that touches these endpoints builds the pipeline **by hand**, so each one reproduced the author's mental model rather than the host's actual composition.

### Proposed design

**Fix (5 map sites, one new file, no behavioural change to any authenticated surface):**

```csharp
// src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/MeshGrpcEndpoints.cs  (new)

/// <summary>
///     Maps the mesh gRPC ingress endpoints (artifact serve, central ack, telemetry stream, node
///     command) and the LocalDb sync endpoint, each marked <c>AllowAnonymous</c>.
/// </summary>
/// <remarks>
///     <para>
///         <b>Why AllowAnonymous (archreview 03/S14).</b> On an admin-role node
///         <c>AddOtOpcUaAuth</c> installs a global <c>FallbackPolicy = RequireAuthenticatedUser</c>
///         under the cookie scheme, which the authorization middleware applies to EVERY endpoint
///         without explicit authorization metadata — gRPC included. These endpoints authenticate
///         with a per-cluster PSK bearer inside a gRPC interceptor, which runs INSIDE the endpoint
///         invocation, i.e. after that middleware. Without this opt-out a correctly-keyed node call
///         is challenged by the cookie handler before the interceptor's verdict matters: artifact
///         fetches fail, deploy acks are dropped, deployments seal TimedOut. Anonymous is safe here
///         because all five interceptors fail CLOSED — an unset key rejects every call and the
///         presented token is compared with FixedTimeEquals.
///     </para>
///     <para>
///         <b>Do not "tidy" this into Program.cs.</b> The census guard
///         (<c>MeshGrpcAnonymousMetadataTests</c>) drives THIS method, so a map that moves back
///         inline leaves the guard asserting a shape production no longer has — which is exactly
///         how S14 survived seven phases of live gates.
///     </para>
/// </remarks>
public static void MapOtOpcUaMeshGrpcServices(
    this WebApplication app,
    bool hasAdmin,
    bool hasDriver,
    int syncListenPort,
    int configServeGrpcPort,
    int telemetryListenPort)
{
    if (hasDriver && syncListenPort > 0)
    {
        // MapZbLocalDbSync returns IEndpointRouteBuilder, NOT an IEndpointConventionBuilder, so
        // .AllowAnonymous() cannot be chained onto it. An empty-prefix group carries the convention
        // to whatever it maps while leaving the gRPC route patterns byte-identical.
        app.MapGroup(string.Empty).AllowAnonymous().MapZbLocalDbSync();
    }

    if (hasAdmin && configServeGrpcPort > 0)
    {
        app.MapGrpcService<DeploymentArtifactGrpcService>().AllowAnonymous();
        app.MapGrpcService<CentralCommandGrpcService>().AllowAnonymous();
    }

    if (hasDriver && telemetryListenPort > 0)
    {
        app.MapGrpcService<TelemetryStreamGrpcService>().AllowAnonymous();
        app.MapGrpcService<NodeCommandGrpcService>().AllowAnonymous();
    }
}
```

`Program.cs:743-784` collapses to one call (`app.MapOtOpcUaMeshGrpcServices(hasAdmin, hasDriver, syncListenPort, configServeGrpcPort, telemetryListenPort);`) with the existing per-map rationale comments moved verbatim onto the new method's members — **no comment is deleted**; they are the record of why each endpoint rides which listener.

**Alternatives considered:**

- ***`RequireHost` / port-scoping the mesh endpoints*** — **does not fix this.** `RequireHost` constrains endpoint *matching*, not authorization; a matched endpoint is still subject to the fallback policy. It is a plausible *complement* (it would stop a mesh service being reachable over the AdminUI's Traefik-fronted port), but the ports are already dedicated and HTTP/2-only, so it buys a second place to keep in sync for no authorization benefit. Rejected as the fix; not adopted as hardening.
- ***A named authorization policy / a real PSK authentication scheme*** — architecturally the "correct" answer (auth in middleware, where the framework expects it) and worth naming as the eventual consolidation. Rejected here: it would duplicate five per-service key resolutions (three different option sources plus `IMeshPskProvider`'s per-cluster map) into a scheme, and it would change the failure surface every dialer already handles from the interceptors' `PermissionDenied`/`Unauthenticated` gRPC statuses to a middleware 403 — i.e. it converts a Critical one-liner into a cross-cutting redesign of the mesh's auth contract. Recorded as a follow-up note, not this plan.
- ***Host the mesh services on a second `WebApplication` with its own pipeline*** — true isolation, and the only option that makes the fallback structurally unable to reach them. Rejected: `Program.cs:604-623` documents how delicate the current single-host Kestrel takeover already is (any explicit `Listen*` discards `ASPNETCORE_URLS`; the surface must be re-bound exactly once), and a second host duplicates the DI graph the services resolve from.
- ***Branch `UseAuthorization` so it does not run for mesh paths (`UseWhen`)*** — path-sniffing middleware, evaluated before the endpoint is known in the general case, and it would silently exempt anything sharing a prefix. Rejected.
- ***Change the FallbackPolicy to something narrower*** — rejected outright: the fallback is what protects every AdminUI page and hub; weakening it to fix five endpoints trades a deploy-plane outage for an authorization hole.
- ***Extend the empty-prefix group over all five maps*** — tidier, but it puts four already-chainable maps behind an untested-in-this-repo route-group/gRPC interaction for no gain. Rejected; the group is scoped to the one map that needs it.

**Guards (the actual deliverable):**

1. **`MeshGrpcAuthStackTests`** (new, `Host.IntegrationTests`) — the CI-shaped variant of `MeshGrpcBoundaryTests` the action item asks for. It builds a `WebApplication` whose auth stack is the **production registration** — `builder.Services.AddOtOpcUaAuth(config)` with `Security:Auth:DisableLogin=false`, then `app.UseAuthentication(); app.UseAuthorization();` — and maps the mesh services through `MapOtOpcUaMeshGrpcServices`. Composition constraints found while verifying (all cheap, all must be in the fixture or it will not build):
   - `AddOtOpcUaAuth` calls `AddDataProtection().PersistKeysToDbContext<OtOpcUaConfigDbContext>()` (`:73-75`) ⇒ the fixture registers an **InMemory** `OtOpcUaConfigDbContext` (the same `InMemoryConfigDbFactory` idiom `MeshGrpcBoundaryTests.cs:288-298` already uses, promoted to a shared fixture helper).
   - Set `builder.Environment.EnvironmentName = "Production"` so `ValidateOnBuild`/`ValidateScopes` do not reject services (`ILdapAuthService`'s graph) that are never resolved without a login attempt.
   - `Security:Auth:DisableLogin` must be **explicitly false** — the whole point is to exercise the cookie handler, not `AutoLoginAuthenticationHandler`. The fixture asserts the resolved scheme handler type, so a future default flip cannot silently re-mask the test.
2. **`MeshGrpcAnonymousMetadataTests`** (new, same project) — an endpoint-metadata census over `((IEndpointRouteBuilder)app).DataSources` after `MapOtOpcUaMeshGrpcServices`, in all three role shapes (admin-only, driver-only, fused). Asserts every mesh endpoint carries `IAllowAnonymous`, **and** (guard-of-the-guard) that the census found the expected endpoint count per shape — a census over zero endpoints passes vacuously, which is the exact failure class this plan exists to close. It also asserts the census is **scoped**: only the five known gRPC service prefixes are marked anonymous, so a future change that anonymises the AdminUI wholesale fails here.
3. **`MeshGrpcMapConventionTests`** (new, source scan over `Program.cs`, following `FlushLoggingConventionTests.cs:28-45`'s solution-root-walk idiom) — every `MapGrpcService<` / `MapZbLocalDbSync(` occurrence in `Program.cs` is inside `MeshGrpcEndpoints`' call, i.e. `Program.cs` contains **zero** direct map sites; the scan asserts the count is 0 and that `MapOtOpcUaMeshGrpcServices` is called exactly once. This is the guard the metadata census cannot be: it catches a map added to `Program.cs` that bypasses the helper entirely.

**Rig (theme 2):** `docker-dev/docker-compose.yml` central-1 (`:385`) and central-2 (`:511`) become `Security__Auth__DisableLogin: "${OTOPCUA_DISABLE_LOGIN:-true}"`, with a header-comment paragraph naming the production-faithful gate mode and the LDAP users it needs. Default behaviour is byte-identical.

### Implementation steps

1. `src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/MeshGrpcEndpoints.cs` — new mapper (above), carrying the five map sites' existing rationale comments plus the S14 rationale.
2. `src/Server/ZB.MOM.WW.OtOpcUa.Host/Program.cs` — replace `:743-784` with the single `MapOtOpcUaMeshGrpcServices(...)` call; keep the `syncListenPort`/`configServeGrpcPort`/`telemetryListenPort` locals and the Kestrel block (`:604-711`) untouched.
3. `docker-dev/docker-compose.yml` — parameterise `Security__Auth__DisableLogin` on central-1/central-2; document the flip in the file header.
4. `docs/security.md` — a short "Mesh gRPC ingress is AllowAnonymous by design" section: what the fallback does, why the interceptors are the gate, what fails when the opt-out is missing, and the two standing guards.
5. `docs/Configuration.md` §`MeshTransport` / §`ConfigSource`-`ConfigServe` and `docs/Telemetry.md` — one cross-reference line each, so an operator reading about the listener ports meets the constraint.

### Tests

- **`MeshGrpcAuthStackTests`** (new): real auth stack + `MapOtOpcUaMeshGrpcServices`, on an h2c Kestrel port.
  - **Positive (load-bearing):** a `DeploymentArtifactService` fetch carrying the correct `ConfigServe:ApiKey` bearer **succeeds** through `UseAuthentication`/`UseAuthorization`; likewise a `CentralCommandService.ReportApply` and a `NodeCommandService.Execute` with their keys. Before the fix these fail; after it they pass. *This is the only assertion that distinguishes the two builds.*
  - **Negative control (anti-vacuity):** the same fixture maps one extra gRPC service **without** `AllowAnonymous`; a correctly-keyed call to it must be refused by the cookie challenge. If this control passes trivially, the fixture's auth stack is not live and every other assertion in the class is void.
  - **Handler-identity guard:** assert the cookie scheme resolves to the real `CookieAuthenticationHandler`, not `AutoLoginAuthenticationHandler`.
- **`MeshGrpcAuthStackTests` (status fidelity, T5):** a **no-PSK** call under the real auth stack fails with the *interceptor's* detail string (`"Config-serve authentication failed."` / `"Telemetry stream authentication failed."` / `"Node command authentication failed."`), never a cookie challenge. ⚠️ Assert the **detail**, not only the status: `ConfigServeAuthInterceptor` uses `Unauthenticated` and an HTTP 401 maps to `Unauthenticated` too, so a status-only assertion passes on a broken build. Record the observed cookie-challenge signature (302 vs 401, and the resulting client-side status) in the plan's execution notes — it is the string the live gate greps for.
- **`MeshGrpcAnonymousMetadataTests`** (new): census as designed above — per-role endpoint sets, `IAllowAnonymous` on each, expected counts (admin-only: 2; driver-only: 2 or 3 with a sync port; fused with all three ports: 5), and the scope assertion.
- **`MeshGrpcDriverOnlyPipelineTests`** (new, T6): a driver-only shape (no `AddOtOpcUaAuth`, no `UseAuthentication`/`UseAuthorization`) serves `NodeCommandService` and `TelemetryStreamService` identically — pins the report's "site nodes unaffected" finding so that adding auth services to driver nodes fails a test instead of a fleet.
- **`MeshGrpcMapConventionTests`** (new, T7): source scan of `Program.cs`.
- **Regression:** `MeshGrpcBoundaryTests`, `ConfigServeListenerTests`, `LocalDbSyncListenerTests`, `TelemetryListenerTests` all stay green untouched (the mapper is behaviour-preserving; the boundary test keeps its own hand-built node pipeline, which is now honestly scoped to "the node side, without auth"). Full project: `dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests`.

**Live gate — docker-dev rig, and it MUST run the falsification control first.**

The rig runs locally (OrbStack on the Mac); AdminUI `http://localhost:9200` round-robins central-1/central-2 via Traefik, so **both centrals must be rebuilt/recreated together**. LDAP is the shared GLAuth at `10.100.0.35:3893` (users `alice`/`bob`, password `password`; `multi-role` for the OPC UA data plane).

*Step 0 — falsification control (pre-fix image; this is what makes the gate non-vacuous):*
```bash
# On master (WITHOUT the fix), flip the central pair to production-faithful auth:
OTOPCUA_DISABLE_LOGIN=false docker compose -f docker-dev/docker-compose.yml up -d central-1 central-2
open http://localhost:9200            # EXPECT a login form — if it auto-authenticates, the flip did not take
curl -s -X POST http://localhost:9200/api/deployments \
     -H "X-Api-Key: docker-dev-deploy-key" -H "Content-Type: application/json" \
     -d '{"CreatedBy":"r3-01-control"}'          # 202 Accepted (the deploy API is AllowAnonymous by design)
docker logs otopcua-dev-site-a-1-1 2>&1 | grep -iE "fetch|artifact|401|302|/login|Unauthenticated" | tail
docker logs otopcua-dev-central-1-1 2>&1 | grep -iE "ReportApply|ack|401|302" | tail
docker exec otopcua-dev-sql-1 /opt/mssql-tools18/bin/sqlcmd -C -S localhost -U sa -P 'OtOpcUa!Dev123' \
  -d OtOpcUa -I -Q "SELECT TOP 3 Id, Status, CreatedUtc FROM dbo.Deployment ORDER BY CreatedUtc DESC"
```
**Required outcome: FAILURE.** Deployment `Status = 4` (TimedOut; `0 Dispatching / 1 AwaitingApplyAcks / 2 Sealed / 3 PartiallyFailed / 4 TimedOut`), site-node logs showing the artifact fetch rejected at the HTTP layer, no `NodeDeploymentState` rows for this revision. A green deploy here means the flip did not take effect and the whole gate proves nothing. *(If `mssql-tools18` is absent in the image, use `/opt/mssql-tools/bin/sqlcmd` without `-C`; `-I` is mandatory for this schema or every statement fails `Msg 1934`.)*

*Step 1 — the fix:*
```bash
docker compose -f docker-dev/docker-compose.yml build central-1 central-2 site-a-1 site-a-2 site-b-1 site-b-2
OTOPCUA_DISABLE_LOGIN=false docker compose -f docker-dev/docker-compose.yml up -d
```
Sign in at `http://localhost:9200` as `alice`/`password` (proves the cookie stack is genuinely live), then repeat the headless deploy above. **Required outcome:**
1. Deployment reaches `Status = 2` (Sealed) — not 4.
2. `SELECT NodeId, RevisionHash, State FROM dbo.NodeDeploymentState` shows **all six** nodes on the new revision.
3. Site-node logs show a successful artifact fetch (SHA-256 verified) and an apply; no `/login`, 302 or 401 on any mesh path.
4. `/hosts` shows fresh driver health for all six nodes — **including central-1 and central-2**. ⚠️ Do **not** accept the `/alerts` live pill as the telemetry evidence: it is aggregate ("live when ≥1 node stream is up"), so it stays green while precisely the two fused-node streams are broken. The per-node rows are the discriminating observation.
5. A driver command from the AdminUI (`/hosts` → Restart driver) reaches a **central-pair** node — the central→own-pair `NodeCommandService` dial that step 0 broke.

*Step 2 — restore:* `docker compose -f docker-dev/docker-compose.yml up -d central-1 central-2` (no env var ⇒ `DisableLogin` back to `true`); confirm `:9200` auto-authenticates again. Record everything in `archreview/plans/STATUS.md`.

### Effort + Risk

**Effort:** S code (one new file, one `Program.cs` edit, one compose line) + M verification (three new test classes; the rig session is the long pole, and it is run twice — once to fail, once to pass).
**Risk:** **low for the fix, medium for the refactor.** The behavioural change is strictly additive: five endpoints stop being subject to a policy that was never intended to gate them, and nothing else on the host loses protection (the census's scope assertion pins that). The real risks are (a) the map extraction touching the boot path — mitigated by landing it as a behaviour-preserving commit with the listener/boundary suites green **before** any convention is added, and (b) the empty-prefix route group changing gRPC routing for LocalDbSync — mitigated by the census (metadata) plus a routing assertion in T3, with the fallback position being a cross-repo `ZB.MOM.WW.LocalDb.Replication` change to mark its own endpoints anonymous. The one thing that must not be skipped is the step-0 falsification control: a fixed build and an unflipped rig are indistinguishable at step 1.

---

## Task breakdown

Branch: one branch `fix/archreview-r3-01-mesh-grpc-allowanonymous` off master. Ordering is strictly TDD: the refactor lands first with its scaffold green, then each guard is written to FAIL against the un-fixed convention and made to pass. T4–T7 are independent of each other once T3 lands.

Verification bar: every task has a deterministic guard; T4/T5 additionally carry falsifiability controls; T10 is the merge-blocking live gate and is only meaningful if its step-0 control **fails** first.

---

### T1 — Extract `MapOtOpcUaMeshGrpcServices` (behaviour-preserving) + census scaffold

**Classification:** MEDIUM-RISK (boot-path refactor; no behaviour change)
**Estimated implement time:** 10 min
**Parallelizable with:** T8, T9; nothing else (every later task builds on the mapper)
**Files:** `src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/MeshGrpcEndpoints.cs` (new), `src/Server/ZB.MOM.WW.OtOpcUa.Host/Program.cs`, `tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests/MeshGrpcAnonymousMetadataTests.cs` (new)

1. Write the scaffold census: build a `WebApplication` per role shape (admin-only / driver-only / fused-with-all-three-ports), call `MapOtOpcUaMeshGrpcServices`, enumerate `((IEndpointRouteBuilder)app).DataSources.SelectMany(d => d.Endpoints)`, and assert the **expected endpoint counts and service prefixes** per shape (no metadata assertion yet). Run `dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests --filter "FullyQualifiedName~MeshGrpcAnonymousMetadataTests"` → **FAIL** (method does not exist).
2. Implement `MeshGrpcEndpoints` as a pure move of `Program.cs:743-784` — same gates, same order, **every existing comment carried over**. Replace the block in `Program.cs` with the single call.
3. Filter → **PASS**. Then the regression net: `dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests --filter "FullyQualifiedName~MeshGrpcBoundaryTests|FullyQualifiedName~ConfigServeListenerTests|FullyQualifiedName~LocalDbSyncListenerTests|FullyQualifiedName~TelemetryListenerTests"` → **PASS unchanged**.
4. Commit: `refactor(host): extract MapOtOpcUaMeshGrpcServices (one authority for the five mesh maps)`.

### T2 — Census asserts `IAllowAnonymous`; `.AllowAnonymous()` on the four gRPC maps

**Classification:** HIGH-RISK (the Critical's fix; changes endpoint authorization metadata)
**Estimated implement time:** 5 min
**Parallelizable with:** T8, T9; after T1
**Files:** `MeshGrpcEndpoints.cs`, `MeshGrpcAnonymousMetadataTests.cs`

1. Extend the census: every mesh endpoint must carry `IAllowAnonymous` metadata; add the **guard-of-the-guard** (the census must have found ≥1 endpoint in each shape, with the exact expected count) and the **scope** assertion (only the five known service prefixes are anonymised). Filter → **FAIL** for all five endpoints.
2. Implement `.AllowAnonymous()` on `DeploymentArtifactGrpcService`, `CentralCommandGrpcService`, `TelemetryStreamGrpcService`, `NodeCommandGrpcService`, with the S14 rationale in the method's XML doc.
3. Filter → **four PASS, LocalDbSync still FAIL** (expected — T3 owns it). Commit: `fix(host): mesh gRPC endpoints opt out of the AdminUI cookie FallbackPolicy (archreview 03/S14)`.

### T3 — LocalDbSync anonymous group (the non-chainable map)

**Classification:** medium-risk (route-group/gRPC interaction on a mapped endpoint)
**Estimated implement time:** 5 min
**Parallelizable with:** T8, T9; after T2
**Files:** `MeshGrpcEndpoints.cs`, `MeshGrpcAnonymousMetadataTests.cs`

1. Add the assertion that the `localdb_sync.v1.LocalDbSync` endpoints carry `IAllowAnonymous` **and** that their route patterns are unchanged by the group (assert the raw pattern text still begins `/localdb_sync.v1.LocalDbSync/`). Filter → **FAIL**.
2. Implement `app.MapGroup(string.Empty).AllowAnonymous().MapZbLocalDbSync()`, with the comment recording *why* (the library's map returns `IEndpointRouteBuilder`, verified against 0.2.1 metadata, and marks nothing anonymous itself).
3. Filter → **PASS**; `LocalDbSyncListenerTests` → **PASS unchanged**. Commit: `fix(host): LocalDb sync endpoint opts out of the cookie fallback via an empty-prefix anon group`.

### T4 — `MeshGrpcAuthStackTests`: a PSK call succeeds through the REAL auth stack

**Classification:** HIGH-RISK verification (this is the guard that would have caught S14)
**Estimated implement time:** ~15 min (fixture plumbing) — the deliberately-larger task
**Parallelizable with:** T6, T7, T8, T9; after T3
**Files:** `tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests/MeshGrpcAuthStackTests.cs` (new)

1. Fixture: `WebApplication` with `EnvironmentName = "Production"`, an InMemory `OtOpcUaConfigDbContext` (DataProtection's `PersistKeysToDbContext` dependency), `AddOtOpcUaAuth(config)` with `Security:Auth:DisableLogin=false`, `AddGrpc` with the production interceptors, `UseAuthentication()`/`UseAuthorization()`, `MapOtOpcUaMeshGrpcServices`, h2c Kestrel on a free port. Assert the resolved cookie-scheme handler is `CookieAuthenticationHandler` (not `AutoLoginAuthenticationHandler`).
2. Positive tests (run against **HEAD before T2** to see them FAIL, then against the fixed build): correctly-keyed `DeploymentArtifactService` fetch, `CentralCommandService.ReportApply`, and `NodeCommandService.Execute` each **succeed**.
3. Negative control: one extra gRPC service mapped **without** `AllowAnonymous` in the same fixture — a correctly-keyed call to it is refused by the cookie challenge. (If it succeeds, the fixture's auth stack is inert and the class proves nothing.)
4. Filter → **PASS**. Commit: `test(host): mesh gRPC boundary under the production auth stack (FallbackPolicy live)`.

### T5 — Status fidelity: a no-PSK call surfaces the INTERCEPTOR, not a cookie challenge

**Classification:** high-risk verification (the vacuity trap)
**Estimated implement time:** 5 min
**Parallelizable with:** T6, T7, T8, T9; after T4
**Files:** `MeshGrpcAuthStackTests.cs`

1. Add: a no-bearer call to each of the three services fails with the interceptor's **detail string** — `"Config-serve authentication failed."`, `"Telemetry stream authentication failed."`, `"Node command authentication failed."` — and a wrong-bearer call likewise. ⚠️ Assert the detail, **not** the status alone: `ConfigServeAuthInterceptor` throws `Unauthenticated` and `Grpc.Net.Client` maps HTTP 401 to `Unauthenticated`, so a status-only assertion is green on a broken build.
2. Record the observed cookie-challenge signature (302 to `/login` vs 401, and the client-side status/detail it produces) in a code comment and in this plan's execution notes — the live gate greps for it.
3. Filter → **PASS**. Commit: `test(host): no-PSK mesh calls report the interceptor's verdict, not a cookie 401/redirect`.

### T6 — Driver-only pipeline control (pins "site nodes unaffected")

**Classification:** standard (test-only)
**Estimated implement time:** 5 min
**Parallelizable with:** T4, T5, T7, T8, T9; after T3
**Files:** `tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests/MeshGrpcDriverOnlyPipelineTests.cs` (new)

1. Build the driver-only shape (no `AddOtOpcUaAuth`, no `UseAuthentication`/`UseAuthorization`) and assert `NodeCommandService`/`TelemetryStreamService` behave identically to T4's fused shape — correctly-keyed call succeeds, no-key call gets the interceptor's status. Also assert the app has **no** `IAuthorizationPolicyProvider` fallback registered, which is *why* it is unaffected — a future "register auth everywhere" change then fails here rather than in a site outage.
2. Filter → **PASS**. Commit: `test(host): driver-only nodes carry no authorization middleware (03/S14 exemption pinned)`.

### T7 — `Program.cs` source-scan convention guard

**Classification:** standard (test-only)
**Estimated implement time:** 5 min
**Parallelizable with:** T4, T5, T6, T8, T9; after T1
**Files:** `tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests/MeshGrpcMapConventionTests.cs` (new)

1. Following `tests/Drivers/Cli/ZB.MOM.WW.OtOpcUa.Driver.S7.Cli.Tests/FlushLoggingConventionTests.cs:28-45` (solution-root walk + `File.ReadAllText`): assert `Program.cs` contains **zero** `MapGrpcService<` and zero `MapZbLocalDbSync(` occurrences, and exactly one `MapOtOpcUaMeshGrpcServices(` call. Guard-of-the-guard: the located file must be non-empty and must contain `MapOtOpcUaMeshGrpcServices`, so a bad path cannot pass the test by finding nothing.
2. Filter → **PASS** (it would have FAILED at HEAD before T1 — note that in the test's remarks). Commit: `test(host): a mesh gRPC map added outside MeshGrpcEndpoints fails the build`.

### T8 — docker-dev: make the production-faithful auth mode a one-variable flip

**Classification:** standard (rig config; default behaviour unchanged)
**Estimated implement time:** 5 min
**Parallelizable with:** everything
**Files:** `docker-dev/docker-compose.yml`

1. `Security__Auth__DisableLogin: "${OTOPCUA_DISABLE_LOGIN:-true}"` on central-1 (`:385`) and central-2 (`:511`); header-comment paragraph naming the flip, the LDAP users (`alice`/`bob`, password `password`, shared GLAuth `10.100.0.35:3893`), and **why** it exists (theme 2: every mesh live gate before R3-01 ran with the blocking middleware neutralized).
2. Verify `docker compose -f docker-dev/docker-compose.yml config` resolves to `"true"` with no env var set (byte-identical default) and to `"false"` with `OTOPCUA_DISABLE_LOGIN=false`. Commit: `chore(docker-dev): parameterise Security__Auth__DisableLogin on the central pair`.

### T9 — Docs

**Classification:** standard (docs)
**Estimated implement time:** 5 min
**Parallelizable with:** everything
**Files:** `docs/security.md`, `docs/Configuration.md`, `docs/Telemetry.md`

Add "Mesh gRPC ingress is `AllowAnonymous` by design" to `docs/security.md`: what the cookie `FallbackPolicy` covers, why interceptors cannot gate what the middleware already rejected, the fail-closed property that makes anonymous safe, the affected-map table (incl. the fused-vs-driver-only distinction), and the two standing guards. One cross-reference line each in `docs/Configuration.md` §`MeshTransport` / §`ConfigSource`-`ConfigServe` and `docs/Telemetry.md`. Commit: `docs: mesh gRPC ingress is AllowAnonymous; the PSK interceptors are the gate (R3-01)`.

### T10 — LIVE GATE: docker-dev with `DisableLogin=false` — operator/rig task

**Classification:** HIGH-RISK verification gate (merge-blocking)
**Estimated implement time:** ~45 min rig session (two passes)
**Parallelizable with:** nothing (needs the built branch **and** a master build for the control)
**Files:** none (rig session; outcomes recorded in `archreview/plans/STATUS.md`)

1. **Step 0 — falsification control on the PRE-FIX build**, per Finding 1's recipe: `OTOPCUA_DISABLE_LOGIN=false` on the central pair, headless deploy, **expect `Deployment.Status = 4` (TimedOut)** with the fetch/ack rejections in the node logs. A green deploy here voids the entire gate — stop and find out why the flip did not take.
2. **Step 1 — the fixed build**: rebuild all six nodes, sign in as `alice`, headless deploy ⇒ `Status = 2` (Sealed), `NodeDeploymentState` on the new revision for **all six** nodes, no `/login`/302/401 on any mesh path, `/hosts` fresh for central-1 **and** central-2 (not the aggregate `/alerts` pill — it cannot see the fused-node streams failing), and an AdminUI driver restart reaching a central-pair node.
3. **Step 2 — restore** `DisableLogin` to `true` (no env var) and confirm `:9200` auto-authenticates.
4. Record both passes — including the exact log signature the blocked calls produced — in `STATUS.md` and in this plan's execution notes.

---

### Sequencing summary

```
T1 → T2 → T3                 (fix, sequential — same two files)
T3 → T4 → T5                 (real-auth-stack guard + status fidelity)
T3 → T6                      (driver-only control)
T1 → T7                      (source scan)
T8, T9 anytime
T10 last — and its step-0 control runs on master BEFORE the branch is merged
```

Overall effort: **~half a dev-day code+tests, plus the two-pass rig session.** The Critical itself is five lines; everything else in this plan exists because the five lines were invisible to a seven-phase live-gate program.

---

## Execution notes (T1–T9, 2026-08-20)

Branch `fix/r3-01-mesh-grpc-allowanonymous`, one commit per task as specified. T1–T9 all landed;
T10 (live gate) left `pending` for the orchestrator per the run brief. Every code-touching task
(T1–T7) was verified red-before-fix by hand — for T2–T5 by asserting against the un-fixed
`MeshGrpcEndpoints.cs`/`Program.cs` (via a temporary `git show <parent-commit>:<path>` swap,
rebuild, run, then restore — confirmed byte-identical restore via `git diff --stat` each time)
and observing the expected failure before reapplying the fix. Two things surfaced during that
process that are worth recording:

1. **The census's endpoint filter could not stay a "route pattern starts with `/`" heuristic once
   T3's empty-prefix `MapGroup` landed.** `RealMethodEndpoints` in `MeshGrpcAnonymousMetadataTests`
   originally filtered on `RoutePattern.RawText?.StartsWith('/')` to exclude grpc-dotnet's own
   per-app "Unimplemented service" catch-all and per-service "Unimplemented method for X"
   fallback (neither carries a route pattern starting with `/` in the ungrouped case). Once
   LocalDbSync moved behind `app.MapGroup(string.Empty).AllowAnonymous().MapZbLocalDbSync()`,
   those SAME fallback patterns *also* gained a leading `/` inside the group, so the heuristic
   started over-counting (`DriverOnly_WithASyncPort_AlsoMapsTheLocalDbSyncEndpoint` and
   `LocalDbSync_AllowAnonymousGroup_LeavesTheRoutePatternByteIdentical` both went red for the
   wrong reason — 3 endpoints found where 1 was expected). Fixed by matching against the exact
   known set of five method route-pattern literals instead of a shape heuristic. Not a plan defect
   — the plan didn't specify the filter mechanism — but a genuine gotcha for anyone extending this
   census: **grpc-dotnet's route-pattern-shape for its own fallback endpoints is not stable across
   a plain `MapGrpcService` vs. a grouped one**, so don't filter on shape once any mesh map moves
   behind a group.

2. **T4's fixture does not map or wire `LocalDbSyncService`'s real dependencies.** The plan's
   fixture description says "maps the mesh services through `MapOtOpcUaMeshGrpcServices`"; T4's
   `MeshGrpcAuthStackTests` calls it with `syncListenPort: 0` (LocalDbSync NOT mapped by the
   helper) and instead hand-maps the library's `LocalDbSyncService` directly, with no
   `.AllowAnonymous()`, purely as the negative control. This is a minimal, deliberate narrowing:
   no assertion in T4 or T5 exercises LocalDbSync's real RPC (the plan's own positive/negative
   lists never name it), and standing up its DI graph (`AddZbLocalDb`/`AddZbLocalDbReplication`,
   a temp-file SQLite `ILocalDb`) would have added fixture weight with nothing to verify — the
   authorization middleware refuses an unauthorized call before the service is ever constructed,
   so the DI graph doesn't need to exist for the negative control to be meaningful. Recorded here
   because it's a case where the fixture is deliberately narrower than a literal reading of the
   plan's prose, even though every named assertion is still covered.

3. **The positive/negative service lists T4 and T5 assert are asymmetric, and that asymmetry is
   the plan's own, not introduced here.** T4's positive list is {`DeploymentArtifactService.Fetch`,
   `CentralCommandService.ReportApply`, `NodeCommandService.Execute`} — Telemetry is not asserted
   positively there. T5's no-PSK detail-string list is {Config-serve, Telemetry stream, Node
   command} — `CentralCommandService`'s own detail string
   (`"Central command authentication failed."`) is never asserted anywhere in T4/T5. Implemented
   exactly as specified (no scope widening per the run brief); flagging the gap in case a
   follow-up wants a `NoOrWrongPsk_CentralCommandReportApply_FailsWithTheInterceptorsDetail` case
   for full symmetry — trivial to add given the fixture already maps and keys that service.

4. **Observed cookie-challenge signature (T5 step 2), captured directly against this repo's
   fixture stack (macOS, Kestrel h2c, `Grpc.Net.Client` over `HttpClient`/`SocketsHttpHandler`):**
   every blocked positive-test call in the red state failed as
   `RpcException(StatusCode.Internal, "Error starting gRPC call. HttpRequestException: The HTTP/2
   server reset the stream. HTTP/2 error code 'PROTOCOL_ERROR' (0x1). (HttpProtocolError)
   HttpProtocolException: The HTTP/2 server reset the stream. HTTP/2 error code 'PROTOCOL_ERROR'
   (0x1). (HttpProtocolError)")` — never `Unauthenticated`/`PermissionDenied`. The cookie handler's
   redirect/401 breaks the prior-knowledge h2c stream before any gRPC trailers are written, so the
   client sees a transport-level reset rather than a clean gRPC status. **The live gate (T10)
   should grep node/central logs and browser network traces for `PROTOCOL_ERROR`, `/login`, `302`
   and `401` together** — not for `401` alone, and not for a gRPC `Unauthenticated`/
   `PermissionDenied` status, which this environment never actually produces for the blocked case.
   (A real reverse-proxy in front of a production deployment could still surface a clean HTTP 401
   instead of a stream reset, depending on how it terminates HTTP/2 — the plan's own multi-signal
   grep advice already covers that; this note is the empirical confirmation of one concrete shape.)

5. **T6 passes identically on both the pre-fix and post-fix mapper (confirmed by the same
   temporary-swap procedure).** Driver-only nodes never register `AddOtOpcUaAuth` at all, so
   `.AllowAnonymous()` presence or absence on the mesh maps has zero effect on that shape — T6 is
   a structural pin/regression-control test, not a red-then-green TDD test for a behaviour T1–T3
   changed. This matches the plan's own "standard (test-only)" classification; recorded because
   the run brief asked for red-before-fix confirmation to be checked (and reported) for every task,
   and this is the one task where "red" never existed to begin with.

6. **T8's `docker compose config` verification ran locally on macOS (OrbStack-backed Docker
   Compose v5.1.2)** — no Windows-VM `lmxopcua-fix` helper needed, consistent with the Mac-path
   guidance in CLAUDE.md. Confirmed both branches: unset `OTOPCUA_DISABLE_LOGIN` resolves
   `Security__Auth__DisableLogin: "true"` on both central-1/central-2; `OTOPCUA_DISABLE_LOGIN=false`
   resolves `"false"` on both.

No other deviations from the plan's design or task breakdown. T10 was not run — left `pending`
per the run brief, for the orchestrator.

## Live gate result (2026-08-20) — **PASS** on docker-dev with `DisableLogin=false` (control failed first)

Rig: `docker-dev/docker-compose.yml` on OrbStack (Mac), fresh volumes (`down -v` first), six host
containers. Both runs used `OTOPCUA_DISABLE_LOGIN=false` on the central pair (verified in-container:
`Security__Auth__DisableLogin=false` on central-1/central-2; `curl -s http://localhost:9200/` → `302 →
/login?ReturnUrl=%2F` in both runs — the cookie FallbackPolicy was live for the UI throughout).
Deploys were triggered with `./docker-dev/deploy.sh` (`POST /api/deployments`, `X-Api-Key:
docker-dev-deploy-key`), which answered `202 Accepted` in both runs — the headless deploy API is
`AllowAnonymous` + key-gated and is NOT behind the cookie fallback, so the API-key path works with
login enabled (one open question from the brief, answered: not a finding).

### Step 0 — falsification control: **FAILED as required**

Control image `otopcua-host:prefix` = `53c41fc6bc01`, built from a scratch worktree of master
`f1b976f5` with the five R3-01 opt-outs reverted in `MeshGrpcEndpoints.cs` (`.AllowAnonymous()` removed
from the four `MapGrpcService` maps, the empty-prefix `MapGroup(...).AllowAnonymous()` collapsed back to a
bare `MapZbLocalDbSync()`; `git diff --stat` = 1 file, 5 insertions/5 deletions). Chosen over a checkout of
`851e4cf5` so the control image matched the R3-04 compose (site-b-2 `Cluster:Roles`-only, site-a-2
miscased pin) — the same compose both runs used. Tagged over `otopcua-host:dev` for the run; image
parity asserted on all six containers (`docker inspect --format '{{.Image}}'` = `53c41fc6bc01…` ×6).

- Deployment `26077962…` (revision `9e3540a1…`): **`Status = 4` (TimedOut)**, `SealedAtUtc = NULL`,
  six `NodeDeploymentState` rows all `Status = 0`, `AppliedAtUtc = NULL`.
- central-1 (seal): `[16:10:04 WRN] Deployment 26077962e53a4ae99538c5aaaa396a85 timed out after
  00:02:00 (0/6 acks landed). No ack from: central-1:4053, central-2:4053, site-a-1:4053,
  site-a-2:4053, site-b-1:4053, site-b-2:4053.`
- central-1 (own-pair command dial, the fused-node path): `[16:08:04 WRN] Mesh: DispatchDeployment
  (correlationId=9fe30594…) was NOT delivered to central-1:4053 (Cancelled: Bad gRPC response.
  Invalid content-type value: text/html; charset=utf-8).` — and identically for central-2:4053.
- site-a-1 (artifact fetch + ack): `[16:08:04 WRN] Artifact fetch for 26077962… from
  http://central-1:4055 failed (Cancelled); trying next endpoint.` → `… failed on ALL 2 endpoint(s);
  returning no bytes (apply failure — last-known-good is kept).` → `[16:08:04 ERR] DriverHost
  site-a-1:4053: FetchAndCache apply of 26077962… FAILED … nothing was applied. Staying on revision
  null` → `[16:08:04 WRN] Dropping the ApplyAck for 26077962… from site-a-1:4053: all 4 central
  endpoint(s) refused it (http://central-1:4055 => Cancelled; …)`.
- Server-side signature, per central (Kestrel request log, `sort | uniq -c` over the whole control run):

  | endpoint (central-1 / central-2) | responded |
  |---|---|
  | `POST /deployment_artifact.v1.DeploymentArtifactService/Fetch` | **302** ×4 / ×4 |
  | `POST /mesh_transport.v1.CentralCommandService/ReportApply` | **302** ×8 / ×8 |
  | `POST /mesh_transport.v1.NodeCommandService/Execute` | **302** ×1 / ×1 |
  | `POST /telemetry.v1.TelemetryStreamService/Subscribe` | **302** ×107 / ×105 (central's own-pair dial reconnecting every ~5 s) |

  Site nodes (no auth stack) served `NodeCommandService/Execute` → `200` — the report's "site nodes
  unaffected" finding holds live; it is the *central* side of every mesh call that was gated.

### Step 1 — the fix: **PASS**

Image `otopcua-host:dev` = `b344565cad3d`, built 12:28 local via `docker compose build` from the master
working tree. ⚠️ Master advanced from `f1b976f5` to `e280db37` (R3-02 merge + one follow-up) *while the
control was running*, and the build ran after that, so the post-fix image is master **`e280db37`** —
which contains `d79d51dd` (R3-01) unchanged; the working tree was clean at build time. Parity asserted
on all six containers (`b344565cad3d` ×6), `DisableLogin=false` still on both centrals, SQL volume kept
(so the TimedOut revision re-dispatched as a new deployment).

1. Deployment `98ea226a…` (same revision `9e3540a1…`): **`Status = 2` (Sealed)**, `CreatedAtUtc
   16:30:43.743`, `SealedAtUtc 16:30:43.920` (177 ms). Six `NodeDeploymentState` rows, **all
   `Status = 1` with `AppliedAtUtc` 16:30:43.836–.910**: central-1, central-2, site-a-1, site-a-2,
   site-b-1, site-b-2. A second deploy after recreating the centrals (`49bcf361…`) sealed identically.
2. site-a-1: `[16:30:43 INF] Fetched + verified artifact for 98ea226a… from http://central-1:4055 (1469
   bytes).` → `cached applied deployment 98ea226a… as this node's local fallback`.
3. Centrals' request log, same grep: `DeploymentArtifactService/Fetch` **200** ×4,
   `CentralCommandService/ReportApply` **200** ×8, `NodeCommandService/Execute` **200**. Count of
   `responded (302|401)|PROTOCOL_ERROR|/login|Invalid content-type` across all six node logs after the
   recreate: **0 on every node** except one `HTTP GET / responded 302` on central-2 — the curl probe of
   the AdminUI root, i.e. the FallbackPolicy still active for the UI (the intended scope). The four site
   nodes: 0.
4. Signed in through the real form at `http://localhost:9200/login` as `multi-role`/`password` (see the
   LDAP finding below for why not `alice`); session badge listed all roles. `/hosts` showed the MAIN
   cluster's `pymodbus` (`MAIN-modbus`) driver **Healthy, Last read 16:37:23 UTC then 16:37:53 UTC**
   across two auto-refreshes — that row is fed only by central's `TelemetryDial` into its *own pair's*
   `TelemetryStreamService`, the stream that answered 302 ×212 in the control. `/alerts` pill `live`
   (recorded, but per the plan not relied on). The rig seed has **no drivers on SITE-A/SITE-B**, so there
   are no per-node site rows to show; site-node telemetry is instead evidenced by central-1's dial log —
   one boot-time `Telemetry stream to node … failed (failure streak 1); reconnecting` per node at
   recreate (connection refused, peer not yet listening) and **no further failure** for the rest of the
   run, and by the `TelemetryStreamService/Subscribe` streams on the centrals never closing (0
   `responded` lines after recreate — a stream logs only when it ends).
5. `/hosts → Controls → Restart → Confirm restart` on `MAIN-modbus`: central-2 `[16:38:19 INF] AdminOps:
   RestartDriver dispatched for MAIN-modbus by multi-role`; **both** central-1 and central-2 logged
   `HTTP POST /mesh_transport.v1.NodeCommandService/Execute responded 200` followed by `DriverHost
   central-N:4053: restarting driver MAIN-modbus by request of multi-role` — the central→own-pair
   `NodeCommandService` dial that step 0 broke (`Invalid content-type value: text/html`).

LocalDb sync (site-a replicating pair, step 2 of the brief): the library logs no success line, so the
evidence is (a) an **ESTABLISHED** TCP session on site-a-2's `:9001` sync listener from site-a-1
(`/proc/net/tcp6` inside site-a-2: `…FFFF00000761A8C0:2329 ← …FFFF00000861A8C0:E276 state 01`), and
(b) exactly one `Replication session to peer http://site-a-2:9001 faulted; reconnecting.` on site-a-1
at boot (`16:29:47`, inner `SocketException (111): Connection refused` — peer not yet listening) and
none afterwards. Site nodes carry no auth stack, so this was expected to be unaffected in both runs;
it was (same shape in the control).

### Step 2 — restore

`docker compose -f docker-dev/docker-compose.yml up -d central-1 central-2` with no env var →
`Security__Auth__DisableLogin=true` in both containers, `curl -s http://localhost:9200/` → `200` (auto-
authenticated). Rig left up on `otopcua-host:dev` = `b344565cad3d` ×6. Scratch worktree
`/tmp/otopcua-prefix` removed. The `otopcua-host:prefix` image tag is left behind for re-runs.

### Findings (new, outside R3-01's fix)

1. **The rig's LDAP login is broken by the SEC-36 rotation, masked until now by `DisableLogin=true`.**
   With login enabled, every sign-in failed `/login?error=Unexpected%20authentication%20error`
   (= `LdapAuthFailure.ServiceAccountBindFailed`). The shared GLAuth at `10.100.0.35:3893` logged
   `invalid credentials binddn=cn=serviceaccount,dc=zb,dc=local src=10.100.0.199` — its
   `serviceaccount` `passsha256` (`e9c2967c…`, rotated 2026-08-07, scadaproj `aada53b`) no longer
   matches the `serviceaccount123` the compose hardcodes (`sha256 = af29d0e5…`); the directory also has
   **no `alice`** (that user is the Host.IntegrationTests harness GLAuth). Same class as the ScadaBridge
   rig breakage recorded in the scadaproj memory. The gate proceeded with a gate-only compose override
   (`Security__Ldap__Server: host.docker.internal`, pointing at the local `zb-ldap-primary` pair which
   keeps `serviceaccount123`) and `multi-role`/`password`; the override was dropped at restore. The
   compose's `Security__Ldap__*` block, the `docker-dev/README.md` "Auth" section and the plan's
   `alice`/`password` recipe all need correcting — filed as a follow-up, not fixed here.
2. **The live cookie-challenge signature differs from the unit-test one.** Execution note 4 recorded
   `StatusCode.Internal … PROTOCOL_ERROR` from the in-process fixture. On the rig the server side is a
   plain `302` (Kestrel log), and the gRPC *client* side surfaces as **`Cancelled: Bad gRPC response.
   Invalid content-type value: text/html; charset=utf-8`** (the `DispatchDeployment` warning) or a bare
   `(Cancelled)` (the fetcher's per-endpoint warning). Grep for `responded 302` on the serving side and
   `Invalid content-type value: text/html` / `Cancelled` on the dialling side; `PROTOCOL_ERROR` never
   appeared in any container log.
3. **The seed gives the site clusters no drivers**, so `/hosts` cannot show per-node driver health for
   site nodes — plan step-1 item 4 ("all six nodes") is only satisfiable for the central pair on this
   rig as seeded. Not a defect; noted so the next gate does not read "No drivers." as a telemetry break.