Merge branch 'fix/r3-01-mesh-grpc-allowanonymous' — R3-01: mesh gRPC + LocalDb sync endpoints opt out of the AdminUI cookie FallbackPolicy (PSK interceptors are the gate); real-auth-stack boundary tests + map-convention guard; rig DisableLogin parameterised (archreview 03/S14 Critical)
This commit is contained in:
Joseph Doherty
@@ -352,3 +352,87 @@ T10 last — and its step-0 control runs on master BEFORE the branch is merged
|
||||
```
|
||||
|
||||
Overall effort: **~half a dev-day code+tests, plus the two-pass rig session.** The Critical itself is five lines; everything else in this plan exists because the five lines were invisible to a seven-phase live-gate program.
|
||||
|
||||
---
|
||||
|
||||
## Execution notes (T1–T9, 2026-08-20)
|
||||
|
||||
Branch `fix/r3-01-mesh-grpc-allowanonymous`, one commit per task as specified. T1–T9 all landed;
|
||||
T10 (live gate) left `pending` for the orchestrator per the run brief. Every code-touching task
|
||||
(T1–T7) was verified red-before-fix by hand — for T2–T5 by asserting against the un-fixed
|
||||
`MeshGrpcEndpoints.cs`/`Program.cs` (via a temporary `git show <parent-commit>:<path>` swap,
|
||||
rebuild, run, then restore — confirmed byte-identical restore via `git diff --stat` each time)
|
||||
and observing the expected failure before reapplying the fix. Two things surfaced during that
|
||||
process that are worth recording:
|
||||
|
||||
1. **The census's endpoint filter could not stay a "route pattern starts with `/`" heuristic once
|
||||
T3's empty-prefix `MapGroup` landed.** `RealMethodEndpoints` in `MeshGrpcAnonymousMetadataTests`
|
||||
originally filtered on `RoutePattern.RawText?.StartsWith('/')` to exclude grpc-dotnet's own
|
||||
per-app "Unimplemented service" catch-all and per-service "Unimplemented method for X"
|
||||
fallback (neither carries a route pattern starting with `/` in the ungrouped case). Once
|
||||
LocalDbSync moved behind `app.MapGroup(string.Empty).AllowAnonymous().MapZbLocalDbSync()`,
|
||||
those SAME fallback patterns *also* gained a leading `/` inside the group, so the heuristic
|
||||
started over-counting (`DriverOnly_WithASyncPort_AlsoMapsTheLocalDbSyncEndpoint` and
|
||||
`LocalDbSync_AllowAnonymousGroup_LeavesTheRoutePatternByteIdentical` both went red for the
|
||||
wrong reason — 3 endpoints found where 1 was expected). Fixed by matching against the exact
|
||||
known set of five method route-pattern literals instead of a shape heuristic. Not a plan defect
|
||||
— the plan didn't specify the filter mechanism — but a genuine gotcha for anyone extending this
|
||||
census: **grpc-dotnet's route-pattern-shape for its own fallback endpoints is not stable across
|
||||
a plain `MapGrpcService` vs. a grouped one**, so don't filter on shape once any mesh map moves
|
||||
behind a group.
|
||||
|
||||
2. **T4's fixture does not map or wire `LocalDbSyncService`'s real dependencies.** The plan's
|
||||
fixture description says "maps the mesh services through `MapOtOpcUaMeshGrpcServices`"; T4's
|
||||
`MeshGrpcAuthStackTests` calls it with `syncListenPort: 0` (LocalDbSync NOT mapped by the
|
||||
helper) and instead hand-maps the library's `LocalDbSyncService` directly, with no
|
||||
`.AllowAnonymous()`, purely as the negative control. This is a minimal, deliberate narrowing:
|
||||
no assertion in T4 or T5 exercises LocalDbSync's real RPC (the plan's own positive/negative
|
||||
lists never name it), and standing up its DI graph (`AddZbLocalDb`/`AddZbLocalDbReplication`,
|
||||
a temp-file SQLite `ILocalDb`) would have added fixture weight with nothing to verify — the
|
||||
authorization middleware refuses an unauthorized call before the service is ever constructed,
|
||||
so the DI graph doesn't need to exist for the negative control to be meaningful. Recorded here
|
||||
because it's a case where the fixture is deliberately narrower than a literal reading of the
|
||||
plan's prose, even though every named assertion is still covered.
|
||||
|
||||
3. **The positive/negative service lists T4 and T5 assert are asymmetric, and that asymmetry is
|
||||
the plan's own, not introduced here.** T4's positive list is {`DeploymentArtifactService.Fetch`,
|
||||
`CentralCommandService.ReportApply`, `NodeCommandService.Execute`} — Telemetry is not asserted
|
||||
positively there. T5's no-PSK detail-string list is {Config-serve, Telemetry stream, Node
|
||||
command} — `CentralCommandService`'s own detail string
|
||||
(`"Central command authentication failed."`) is never asserted anywhere in T4/T5. Implemented
|
||||
exactly as specified (no scope widening per the run brief); flagging the gap in case a
|
||||
follow-up wants a `NoOrWrongPsk_CentralCommandReportApply_FailsWithTheInterceptorsDetail` case
|
||||
for full symmetry — trivial to add given the fixture already maps and keys that service.
|
||||
|
||||
4. **Observed cookie-challenge signature (T5 step 2), captured directly against this repo's
|
||||
fixture stack (macOS, Kestrel h2c, `Grpc.Net.Client` over `HttpClient`/`SocketsHttpHandler`):**
|
||||
every blocked positive-test call in the red state failed as
|
||||
`RpcException(StatusCode.Internal, "Error starting gRPC call. HttpRequestException: The HTTP/2
|
||||
server reset the stream. HTTP/2 error code 'PROTOCOL_ERROR' (0x1). (HttpProtocolError)
|
||||
HttpProtocolException: The HTTP/2 server reset the stream. HTTP/2 error code 'PROTOCOL_ERROR'
|
||||
(0x1). (HttpProtocolError)")` — never `Unauthenticated`/`PermissionDenied`. The cookie handler's
|
||||
redirect/401 breaks the prior-knowledge h2c stream before any gRPC trailers are written, so the
|
||||
client sees a transport-level reset rather than a clean gRPC status. **The live gate (T10)
|
||||
should grep node/central logs and browser network traces for `PROTOCOL_ERROR`, `/login`, `302`
|
||||
and `401` together** — not for `401` alone, and not for a gRPC `Unauthenticated`/
|
||||
`PermissionDenied` status, which this environment never actually produces for the blocked case.
|
||||
(A real reverse-proxy in front of a production deployment could still surface a clean HTTP 401
|
||||
instead of a stream reset, depending on how it terminates HTTP/2 — the plan's own multi-signal
|
||||
grep advice already covers that; this note is the empirical confirmation of one concrete shape.)
|
||||
|
||||
5. **T6 passes identically on both the pre-fix and post-fix mapper (confirmed by the same
|
||||
temporary-swap procedure).** Driver-only nodes never register `AddOtOpcUaAuth` at all, so
|
||||
`.AllowAnonymous()` presence or absence on the mesh maps has zero effect on that shape — T6 is
|
||||
a structural pin/regression-control test, not a red-then-green TDD test for a behaviour T1–T3
|
||||
changed. This matches the plan's own "standard (test-only)" classification; recorded because
|
||||
the run brief asked for red-before-fix confirmation to be checked (and reported) for every task,
|
||||
and this is the one task where "red" never existed to begin with.
|
||||
|
||||
6. **T8's `docker compose config` verification ran locally on macOS (OrbStack-backed Docker
|
||||
Compose v5.1.2)** — no Windows-VM `lmxopcua-fix` helper needed, consistent with the Mac-path
|
||||
guidance in CLAUDE.md. Confirmed both branches: unset `OTOPCUA_DISABLE_LOGIN` resolves
|
||||
`Security__Auth__DisableLogin: "true"` on both central-1/central-2; `OTOPCUA_DISABLE_LOGIN=false`
|
||||
resolves `"false"` on both.
|
||||
|
||||
No other deviations from the plan's design or task breakdown. T10 was not run — left `pending`
|
||||
per the run brief, for the orchestrator.
|
||||
|
||||
@@ -5,13 +5,13 @@
|
||||
{
|
||||
"id": "T1",
|
||||
"subject": "Extract MapOtOpcUaMeshGrpcServices from Program.cs:743-784 (behaviour-preserving) + endpoint census scaffold (03/S14)",
|
||||
"status": "pending",
|
||||
"status": "done",
|
||||
"blockedBy": []
|
||||
},
|
||||
{
|
||||
"id": "T2",
|
||||
"subject": "Census asserts IAllowAnonymous + scope/count guard-of-the-guard; .AllowAnonymous() on the four MapGrpcService maps",
|
||||
"status": "pending",
|
||||
"status": "done",
|
||||
"blockedBy": [
|
||||
"T1"
|
||||
]
|
||||
@@ -19,7 +19,7 @@
|
||||
{
|
||||
"id": "T3",
|
||||
"subject": "LocalDb sync endpoint anonymised via an empty-prefix MapGroup (MapZbLocalDbSync returns IEndpointRouteBuilder, not chainable)",
|
||||
"status": "pending",
|
||||
"status": "done",
|
||||
"blockedBy": [
|
||||
"T2"
|
||||
]
|
||||
@@ -27,7 +27,7 @@
|
||||
{
|
||||
"id": "T4",
|
||||
"subject": "MeshGrpcAuthStackTests: PSK-bearing calls succeed through the REAL AddOtOpcUaAuth + UseAuthentication/UseAuthorization stack, with an un-anonymised negative control",
|
||||
"status": "pending",
|
||||
"status": "done",
|
||||
"blockedBy": [
|
||||
"T3"
|
||||
]
|
||||
@@ -35,7 +35,7 @@
|
||||
{
|
||||
"id": "T5",
|
||||
"subject": "Status fidelity: a no-PSK call reports the interceptor's DETAIL string, not a cookie 401/302 (ConfigServe Unauthenticated collision makes status-only assertions vacuous)",
|
||||
"status": "pending",
|
||||
"status": "done",
|
||||
"blockedBy": [
|
||||
"T4"
|
||||
]
|
||||
@@ -43,7 +43,7 @@
|
||||
{
|
||||
"id": "T6",
|
||||
"subject": "MeshGrpcDriverOnlyPipelineTests: driver-only nodes carry no authorization middleware — pins the 03/S14 exemption",
|
||||
"status": "pending",
|
||||
"status": "done",
|
||||
"blockedBy": [
|
||||
"T3"
|
||||
]
|
||||
@@ -51,7 +51,7 @@
|
||||
{
|
||||
"id": "T7",
|
||||
"subject": "MeshGrpcMapConventionTests: Program.cs source scan — zero direct MapGrpcService/MapZbLocalDbSync sites, exactly one mapper call",
|
||||
"status": "pending",
|
||||
"status": "done",
|
||||
"blockedBy": [
|
||||
"T1"
|
||||
]
|
||||
@@ -59,13 +59,13 @@
|
||||
{
|
||||
"id": "T8",
|
||||
"subject": "docker-dev: Security__Auth__DisableLogin parameterised as ${OTOPCUA_DISABLE_LOGIN:-true} on central-1/central-2 + header docs",
|
||||
"status": "pending",
|
||||
"status": "done",
|
||||
"blockedBy": []
|
||||
},
|
||||
{
|
||||
"id": "T9",
|
||||
"subject": "docs/security.md \"Mesh gRPC ingress is AllowAnonymous by design\" + cross-refs in docs/Configuration.md and docs/Telemetry.md",
|
||||
"status": "pending",
|
||||
"status": "done",
|
||||
"blockedBy": []
|
||||
},
|
||||
{
|
||||
|
||||
@@ -90,6 +90,21 @@
|
||||
# prior dark-switch shape). The ClusterNode.GrpcPort column (seeded by
|
||||
# docker-dev/seed/seed-clusters.sql) points central's dial supervisor at each node's :4056.
|
||||
#
|
||||
# Production-faithful auth (R3-01, archreview 03/S14): central-1/central-2's
|
||||
# Security__Auth__DisableLogin defaults to "true" here (byte-identical to the previous hardcoded
|
||||
# value) but is now `${OTOPCUA_DISABLE_LOGIN:-true}` — a one-variable flip to the REAL cookie
|
||||
# login flow instead of an edit-and-remember-to-revert. DisableLogin registers an always-succeeding
|
||||
# handler UNDER the cookie scheme name, so every live gate run with the default masks the AdminUI
|
||||
# cookie FallbackPolicy entirely — which is exactly how 03/S14 (the FallbackPolicy blocking every
|
||||
# mesh gRPC call on a login-enabled admin node) survived seven phases of prior live gates. To drive
|
||||
# this rig production-faithfully:
|
||||
# OTOPCUA_DISABLE_LOGIN=false docker compose -f docker-dev/docker-compose.yml up -d central-1 central-2
|
||||
# Sign in as `alice` / `password` (or `bob`) against the shared GLAuth (10.100.0.35:3893,
|
||||
# dc=zb,dc=local) — the same LDAP users every other node in this file already authenticates OPC UA
|
||||
# sessions against. Restore the default with `docker compose ... up -d central-1 central-2` (no env
|
||||
# var). See docs/security.md "Mesh gRPC ingress is AllowAnonymous by design" for what this actually
|
||||
# exercises.
|
||||
#
|
||||
# Usage:
|
||||
# docker compose -f docker-dev/docker-compose.yml up -d --build
|
||||
# open http://localhost:9200 # central Blazor admin UI
|
||||
@@ -382,7 +397,7 @@ services:
|
||||
Security__Ldap__ServiceAccountDn: "cn=serviceaccount,dc=zb,dc=local"
|
||||
Security__Ldap__ServiceAccountPassword: "serviceaccount123"
|
||||
Security__DeployApiKey: "docker-dev-deploy-key"
|
||||
Security__Auth__DisableLogin: "true"
|
||||
Security__Auth__DisableLogin: "${OTOPCUA_DISABLE_LOGIN:-true}"
|
||||
# Map the shared-GLAuth data-plane groups (ReadOnly/WriteOperate/...) to data-plane role
|
||||
# strings the OPC UA session ACL gates read. The DB LdapGroupRoleMapping is AdminRole-typed
|
||||
# (AdminUI roles); this free-form GroupToRole baseline is the path for data-plane roles.
|
||||
@@ -508,7 +523,7 @@ services:
|
||||
Security__Ldap__ServiceAccountDn: "cn=serviceaccount,dc=zb,dc=local"
|
||||
Security__Ldap__ServiceAccountPassword: "serviceaccount123"
|
||||
Security__DeployApiKey: "docker-dev-deploy-key"
|
||||
Security__Auth__DisableLogin: "true"
|
||||
Security__Auth__DisableLogin: "${OTOPCUA_DISABLE_LOGIN:-true}"
|
||||
# Data-plane group→role baseline (see central-1).
|
||||
Security__Ldap__GroupToRole__ReadOnly: "ReadOnly"
|
||||
Security__Ldap__GroupToRole__WriteOperate: "WriteOperate"
|
||||
|
||||
@@ -193,6 +193,8 @@ On the docker-dev rig every node carries these keys already; the rig's compose d
|
||||
|
||||
See the [ClusterClient→gRPC migration design](plans/2026-08-06-clusterclient-to-grpc-design.md), [implementation plan](plans/2026-08-06-clusterclient-to-grpc.md), [tracking doc](plans/2026-08-06-clusterclient-to-grpc-tracking.md) and [live gate](plans/2026-08-06-clusterclient-to-grpc-live-gate.md) for the migration's phase-by-phase history — those are historical records of the cutover and are not updated further by this sweep.
|
||||
|
||||
**On an admin-role node, `NodeCommandService` and `CentralCommandService` are `AllowAnonymous` by design** — the AdminUI cookie `FallbackPolicy` would otherwise 401/redirect a correctly-PSK'd call before the interceptor above ever ran (archreview 03/S14). See [`security.md` § Mesh gRPC ingress is `AllowAnonymous` by design](security.md#mesh-grpc-ingress-is-allowanonymous-by-design).
|
||||
|
||||
### `ConfigSource` / `ConfigServe` (config fetch-and-cache)
|
||||
|
||||
- **Purpose:** [per-cluster mesh](plans/2026-07-21-per-cluster-mesh-design.md) **Phase 3**. `ConfigSource:Mode` selects where a **driver** node reads its deployed configuration: `Direct` (read `Deployment.ArtifactBlob` from central SQL, today's behaviour) or `FetchAndCache` (fetch the artifact bytes from central over a gRPC stream, verify SHA-256, cache in LocalDb, and read **only** the cache — no central-SQL config read). `ConfigServe` is the **central** (admin-role) serve side that a `FetchAndCache` node dials.
|
||||
@@ -220,6 +222,8 @@ See the [ClusterClient→gRPC migration design](plans/2026-08-06-clusterclient-t
|
||||
|
||||
**`RevisionHash == SHA-256(artifact bytes)` is load-bearing.** `ConfigComposer` computes the deployment's revision hash as the lowercase-hex SHA-256 of the exact artifact bytes, so the node verifies a fetch against the `RevisionHash` the dispatch already carries — no hash rides in the proto. If a future change makes the revision hash anything else, every fetch fails verification.
|
||||
|
||||
**`DeploymentArtifactService` is `AllowAnonymous` by design** — same reasoning as `NodeCommandService`/`CentralCommandService` above: the AdminUI cookie `FallbackPolicy` on an admin-role node would otherwise block a correctly-keyed fetch before `ConfigServeAuthInterceptor`'s verdict matters. See [`security.md` § Mesh gRPC ingress is `AllowAnonymous` by design](security.md#mesh-grpc-ingress-is-allowanonymous-by-design).
|
||||
|
||||
#### Artifact compression (2026-08-15)
|
||||
|
||||
The artifact blob is **gzip-compressed, inside the hashed bytes**. `ConfigComposer.Pack` gzips the
|
||||
|
||||
@@ -190,6 +190,14 @@ transports. ScadaBridge itself has since closed that gap with this identical
|
||||
interceptor pattern, so Phase 5 ships authenticated from the start rather than deferring auth to a
|
||||
later hardening pass. See the design doc's superseded note for the full history.
|
||||
|
||||
**On a fused `admin,driver` node, `TelemetryStreamService` and `NodeCommandService` are `AllowAnonymous`
|
||||
by design** — a *different* gate from the interceptor above. `TelemetryStreamAuthInterceptor` is the PSK
|
||||
check; on an admin-role node the AdminUI cookie `FallbackPolicy` sits in front of it and would 401/redirect
|
||||
a correctly-keyed call before the interceptor's verdict matters, because it runs in the authorization
|
||||
middleware, strictly *before* the interceptor executes (archreview 03/S14). A driver-only node never
|
||||
registers that middleware at all, so it is structurally unaffected either way. See
|
||||
[`security.md` § Mesh gRPC ingress is `AllowAnonymous` by design](security.md#mesh-grpc-ingress-is-allowanonymous-by-design).
|
||||
|
||||
## Upgrading an existing deployment: populate `ClusterNode.GrpcPort` first (#493)
|
||||
|
||||
`GrpcPort` is a **nullable column added by Phase 5**, and nothing backfills it onto rows that
|
||||
|
||||
@@ -908,6 +908,79 @@ Responses: `202 Accepted` (`{ outcome, deploymentId, revisionHash }`) when a dep
|
||||
|
||||
---
|
||||
|
||||
## Mesh gRPC ingress is `AllowAnonymous` by design
|
||||
|
||||
Archreview 03/S14 (Critical, closed R3-01). On an admin-role node `AddOtOpcUaAuth` installs a global
|
||||
`FallbackPolicy = RequireAuthenticatedUser` bound to the cookie scheme (the same fallback that protects
|
||||
every AdminUI page above). ASP.NET Core's authorization middleware applies that fallback to **every
|
||||
endpoint that carries no authorization metadata of its own** — gRPC endpoints included. The five mesh
|
||||
gRPC endpoints (config-serve artifact fetch, central command ack, telemetry stream, node command, LocalDb
|
||||
sync) authenticate with a per-cluster/per-node **PSK bearer inside a gRPC interceptor**, and that
|
||||
interceptor runs *inside* the endpoint invocation — i.e. strictly *after* the authorization middleware.
|
||||
Without an explicit `.AllowAnonymous()` opt-out, a correctly-keyed node call is challenged by the cookie
|
||||
handler before the interceptor's verdict ever matters: on a login-enabled admin node this is a fleet-wide
|
||||
deploy-plane outage (artifact fetches fail, deploy acks are dropped, deployments seal `TimedOut`), and it
|
||||
went unnoticed for seven phases of prior live gates because the docker-dev rig always ran
|
||||
`Security__Auth__DisableLogin=true` (an always-succeeding handler registered *under the cookie scheme
|
||||
name*, so the fallback trivially passes) and every prior mesh-gRPC test built its own auth-free pipeline
|
||||
by hand.
|
||||
|
||||
**Why `AllowAnonymous` is the correct fix, not a hole.** All five interceptors are fail-closed: an unset
|
||||
key rejects every call, and the presented token is compared with `CryptographicOperations.FixedTimeEquals`.
|
||||
`Program.cs:574` states the design contract directly — "the interceptor is the ONLY inbound auth on any
|
||||
endpoint (the libraries verify nothing)" — which was already true of the gRPC pipeline; the missing opt-out
|
||||
made it false of the HTTP pipeline in front of it. The pattern itself predates this fix: static assets
|
||||
(`MapStaticAssets().AllowAnonymous()`), the metrics endpoint (`MapZbMetrics().AllowAnonymous()`), and the
|
||||
headless deploy API above are all `AllowAnonymous` for the identical reason — each is gated by something
|
||||
other than the cookie, and the fallback would otherwise 401/redirect it regardless.
|
||||
|
||||
| Endpoint | Role gate | Affected without `AllowAnonymous`, on a login-enabled node |
|
||||
|---|---|---|
|
||||
| `DeploymentArtifactService.Fetch` | `hasAdmin && ConfigServe:GrpcListenPort > 0` | Always — every `FetchAndCache` node's artifact fetch. |
|
||||
| `CentralCommandService.ReportApply` | same | Always — every node's deploy ack, from every cluster. |
|
||||
| `TelemetryStreamService.Subscribe` | `hasDriver && Telemetry:GrpcListenPort > 0` | Only on a **fused** `admin,driver` node (e.g. docker-dev's central pair) — a driver-only node never registers the cookie stack at all, so it is structurally unaffected either way. |
|
||||
| `NodeCommandService.Execute` | same | Only on a fused node, same reasoning. |
|
||||
| `localdb_sync.v1.LocalDbSync/Sync` | `hasDriver && LocalDb:SyncListenPort > 0` | Only on a fused node that also sets a sync port — no node in the fleet currently has that shape, but the map is anonymised anyway. |
|
||||
|
||||
**A fused central pair is affected on its OWN dispatch, not only inbound from sites.** `CentralCommunicationActor` dials every enabled `ClusterNode` row, its own pair included, and central's telemetry dialer dials its own pair too — so a login-enabled fused central breaks its own command/telemetry plane, not just the sites'.
|
||||
|
||||
**Mapping mechanics.** All five maps live in one place, `MeshGrpcEndpoints.MapOtOpcUaMeshGrpcServices`
|
||||
(`src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/MeshGrpcEndpoints.cs`) — `Program.cs` calls it once and
|
||||
carries no map sites of its own. Four maps chain `.AllowAnonymous()` directly onto `MapGrpcService<T>()`.
|
||||
The fifth, `MapZbLocalDbSync()`, returns `IEndpointRouteBuilder` rather than an
|
||||
`IEndpointConventionBuilder`, so it cannot take the same chain; it is wrapped in an empty-prefix
|
||||
`app.MapGroup(string.Empty).AllowAnonymous().MapZbLocalDbSync()`, which carries the convention without
|
||||
changing the wire route pattern.
|
||||
|
||||
**Three standing guards**, all in `tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests/`, so a future map
|
||||
cannot silently reopen this:
|
||||
|
||||
- `MeshGrpcAnonymousMetadataTests` — an endpoint-metadata census that calls the SAME
|
||||
`MapOtOpcUaMeshGrpcServices` production helper across all three role shapes (admin-only, driver-only,
|
||||
fused) and asserts every mesh endpoint carries `IAllowAnonymous`, with an exact expected count per shape
|
||||
and a scope check that the opt-out never leaks onto an unrelated endpoint.
|
||||
- `MeshGrpcMapConventionTests` — a `Program.cs` source scan asserting zero direct `MapGrpcService</
|
||||
MapZbLocalDbSync(` occurrences and exactly one call to the mapper, so a map added straight into
|
||||
`Program.cs` (bypassing the helper, and therefore invisible to the census) fails `MeshGrpcMapConventionTests`.
|
||||
- `MeshGrpcAuthStackTests` / `MeshGrpcDriverOnlyPipelineTests` drive the mesh endpoints through the REAL
|
||||
`AddOtOpcUaAuth` + `UseAuthentication`/`UseAuthorization` pipeline (`DisableLogin=false`) and the
|
||||
driver-only (no-auth-at-all) shape respectively — the CI-shaped reproduction of the outage, with a
|
||||
correctly-keyed call **succeeding** as the load-bearing assertion (a blocked call and a refused call both
|
||||
"fail"; only a genuine success through the real cookie stack proves the opt-out works) and an
|
||||
un-anonymised negative control proving the fixture's auth stack is actually live.
|
||||
|
||||
**Rig:** `docker-dev/docker-compose.yml`'s central-1/central-2 `Security__Auth__DisableLogin` is
|
||||
`${OTOPCUA_DISABLE_LOGIN:-true}` — default unchanged, but `OTOPCUA_DISABLE_LOGIN=false docker compose
|
||||
... up -d central-1 central-2` drives the rig through the real cookie login (`alice`/`bob`, password
|
||||
`password`, against the shared GLAuth) for exactly this kind of verification, instead of every prior mesh
|
||||
live gate running with the one middleware that breaks production neutralized.
|
||||
|
||||
See `archreview/plans/R3-01-mesh-grpc-allowanonymous-plan.md` (R3-01) for the full verification and design
|
||||
record, and `docs/Configuration.md` §`MeshTransport` / §`ConfigSource`-`ConfigServe` and `docs/Telemetry.md`
|
||||
for the affected listener ports.
|
||||
|
||||
---
|
||||
|
||||
## OTOPCUA0001 Analyzer — Compile-Time Guard
|
||||
|
||||
Per-capability resilience (retry, timeout, circuit-breaker) is applied by `CapabilityInvoker` in `src/Core/ZB.MOM.WW.OtOpcUa.Core/Resilience/`. A driver-capability call made **outside** the invoker bypasses resilience entirely — which in production looks like inconsistent timeouts, un-wrapped retries, and unbounded blocking.
|
||||
|
||||
@@ -0,0 +1,125 @@
|
||||
using ZB.MOM.WW.LocalDb.Replication;
|
||||
using ZB.MOM.WW.OtOpcUa.AdminUI.Grpc;
|
||||
using ZB.MOM.WW.OtOpcUa.Host.Grpc;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.Host.Configuration;
|
||||
|
||||
/// <summary>
|
||||
/// Maps the mesh gRPC ingress endpoints (artifact serve, central ack, telemetry stream, node
|
||||
/// command) and the LocalDb sync endpoint.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// <b>Why AllowAnonymous (archreview 03/S14).</b> On an admin-role node
|
||||
/// <c>AddOtOpcUaAuth</c> installs a global <c>FallbackPolicy = RequireAuthenticatedUser</c>
|
||||
/// under the cookie scheme, which the authorization middleware applies to EVERY endpoint
|
||||
/// without explicit authorization metadata — gRPC included. These endpoints authenticate
|
||||
/// with a per-cluster PSK bearer inside a gRPC interceptor, which runs INSIDE the endpoint
|
||||
/// invocation, i.e. after that middleware. Without this opt-out a correctly-keyed node call
|
||||
/// is challenged by the cookie handler before the interceptor's verdict matters: artifact
|
||||
/// fetches fail, deploy acks are dropped, deployments seal TimedOut. Anonymous is safe here
|
||||
/// because all five interceptors fail CLOSED — an unset key rejects every call and the
|
||||
/// presented token is compared with FixedTimeEquals.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>Do not "tidy" this into Program.cs.</b> A standing census guard
|
||||
/// (<c>MeshGrpcAnonymousMetadataTests</c>) drives THIS method, so a map that moves back
|
||||
/// inline leaves the guard asserting a shape production no longer has — which is exactly
|
||||
/// how the AdminUI cookie <c>FallbackPolicy</c> silently blocked every mesh gRPC call on a
|
||||
/// login-enabled admin node for seven phases of live gates.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
public static class MeshGrpcEndpoints
|
||||
{
|
||||
/// <summary>
|
||||
/// Maps the mesh gRPC ingress endpoints and the LocalDb sync endpoint, gated on the same
|
||||
/// role + dedicated-port checks <c>Program.cs</c> used inline before this extraction.
|
||||
/// </summary>
|
||||
/// <param name="app">The built application to map endpoints on.</param>
|
||||
/// <param name="hasAdmin">Whether this node carries the <c>admin</c> role.</param>
|
||||
/// <param name="hasDriver">Whether this node carries the <c>driver</c> role.</param>
|
||||
/// <param name="syncListenPort">
|
||||
/// The dedicated h2c port bound for the LocalDb passive sync endpoint, or <c>0</c> when it is
|
||||
/// not bound.
|
||||
/// </param>
|
||||
/// <param name="configServeGrpcPort">
|
||||
/// The dedicated h2c port bound for the ConfigServe artifact + central command endpoints, or
|
||||
/// <c>0</c> when it is not bound.
|
||||
/// </param>
|
||||
/// <param name="telemetryListenPort">
|
||||
/// The dedicated h2c port bound for the telemetry stream + node command endpoints, or
|
||||
/// <c>0</c> when it is not bound.
|
||||
/// </param>
|
||||
public static void MapOtOpcUaMeshGrpcServices(
|
||||
this WebApplication app,
|
||||
bool hasAdmin,
|
||||
bool hasDriver,
|
||||
int syncListenPort,
|
||||
int configServeGrpcPort,
|
||||
int telemetryListenPort)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(app);
|
||||
|
||||
// Passive LocalDb sync endpoint. Gated on the same port check that bound the listener, so a
|
||||
// default-OFF or admin-only graph carries no sync surface whatsoever. Mapping it would be
|
||||
// safe even unauthenticated — LocalDbSyncAuthInterceptor fail-closes with no ApiKey
|
||||
// configured — but not mapping it at all is the stronger default.
|
||||
//
|
||||
// AllowAnonymous (03/S14) via an empty-prefix MapGroup: MapZbLocalDbSync returns
|
||||
// IEndpointRouteBuilder, NOT an IEndpointConventionBuilder, so .AllowAnonymous() cannot be
|
||||
// chained onto it the way the other four maps below are. An empty-prefix group carries the
|
||||
// AllowAnonymous convention to whatever it maps while leaving the gRPC route pattern
|
||||
// byte-identical to the ungrouped map (verified by
|
||||
// LocalDbSync_AllowAnonymousGroup_LeavesTheRoutePatternByteIdentical).
|
||||
if (hasDriver && syncListenPort > 0)
|
||||
{
|
||||
app.MapGroup(string.Empty).AllowAnonymous().MapZbLocalDbSync();
|
||||
}
|
||||
|
||||
// Central config-serve artifact endpoint (per-cluster mesh Phase 3). Mapped only on
|
||||
// admin-role nodes (which hold the central SQL connection) and only when a dedicated h2c
|
||||
// port is bound. The ConfigServeAuthInterceptor is the ONLY inbound auth and fail-closes
|
||||
// with no ConfigServe:ApiKey.
|
||||
if (hasAdmin && configServeGrpcPort > 0)
|
||||
{
|
||||
// AllowAnonymous (03/S14): this is every FetchAndCache node's artifact fetch. Without the
|
||||
// opt-out the AdminUI cookie FallbackPolicy 401s/redirects it before
|
||||
// ConfigServeAuthInterceptor's verdict matters — a fleet-wide deploy-plane outage on any
|
||||
// login-enabled admin node.
|
||||
app.MapGrpcService<DeploymentArtifactGrpcService>().AllowAnonymous();
|
||||
|
||||
// Central command ingress (mesh gRPC command plane): the node→central deployment ack. It
|
||||
// rides the SAME listener rather than claiming a port of its own — a second cleartext h2c
|
||||
// port would buy nothing but another firewall rule, and both endpoints are admin-role and
|
||||
// dialled by the same driver nodes. Deliberately NOT gated on MeshTransport:Mode: mapping
|
||||
// it under Dps costs nothing (nothing dials it) whereas gating it would mean flipping the
|
||||
// mode on a running fleet requires a central restart before any node can ack — the same
|
||||
// reasoning that registers CentralCommunicationActor in both modes.
|
||||
// CentralCommandAuthInterceptor (added at AddGrpc in Program.cs) is the ONLY inbound auth
|
||||
// and fail-closes per cluster. AllowAnonymous (03/S14): this is every node's deploy ack,
|
||||
// from every cluster — same reasoning as the artifact fetch above.
|
||||
app.MapGrpcService<CentralCommandGrpcService>().AllowAnonymous();
|
||||
}
|
||||
|
||||
// Driver-node live-telemetry stream (per-cluster mesh Phase 5). Central dials in; gated by
|
||||
// TelemetryStreamAuthInterceptor (fail-closed on empty Telemetry:ApiKey).
|
||||
//
|
||||
// The node-side command endpoint (mesh gRPC command plane, Phase 1) rides the SAME
|
||||
// driver-role h2c listener rather than claiming a fourth port: central already dials this
|
||||
// node here, both directions are per-cluster-keyed, and a separate port would be one more
|
||||
// thing to open on every site firewall for no isolation gain. Mapping it is what makes
|
||||
// MeshCommandAuthInterceptor live — it is registered in Program.cs and is a pass-through
|
||||
// until a call matches this service's method prefix. It fail-closes with no
|
||||
// MeshTransport:GrpcPsk, so an operator who maps the service without setting a key gets a
|
||||
// closed endpoint, not an open one.
|
||||
if (hasDriver && telemetryListenPort > 0)
|
||||
{
|
||||
// AllowAnonymous (03/S14) on both: on a fused admin+driver node (the docker-dev central
|
||||
// pair, and any production fused central) these sit behind the SAME cookie FallbackPolicy
|
||||
// as the ConfigServe endpoints above and would be 401/redirected identically without the
|
||||
// opt-out — central's own telemetry dialer and node-command dispatch to its own pair.
|
||||
app.MapGrpcService<TelemetryStreamGrpcService>().AllowAnonymous();
|
||||
app.MapGrpcService<NodeCommandGrpcService>().AllowAnonymous();
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -695,48 +695,12 @@ if (hasAdmin)
|
||||
app.MapOtOpcUaDeployApi(app.Configuration);
|
||||
}
|
||||
|
||||
// Passive LocalDb sync endpoint. Gated on the same port check that bound the listener, so a
|
||||
// default-OFF or admin-only graph carries no sync surface whatsoever. Mapping it would be safe
|
||||
// even unauthenticated — LocalDbSyncAuthInterceptor fail-closes with no ApiKey configured — but
|
||||
// not mapping it at all is the stronger default.
|
||||
if (hasDriver && syncListenPort > 0)
|
||||
{
|
||||
app.MapZbLocalDbSync();
|
||||
}
|
||||
|
||||
// Central config-serve artifact endpoint (per-cluster mesh Phase 3). Mapped only on admin-role
|
||||
// nodes (which hold the central SQL connection) and only when a dedicated h2c port is bound. The
|
||||
// ConfigServeAuthInterceptor is the ONLY inbound auth and fail-closes with no ConfigServe:ApiKey.
|
||||
if (hasAdmin && configServeGrpcPort > 0)
|
||||
{
|
||||
app.MapGrpcService<DeploymentArtifactGrpcService>();
|
||||
|
||||
// Central command ingress (mesh gRPC command plane): the node→central deployment ack. It
|
||||
// rides the SAME listener rather than claiming a port of its own — a second cleartext h2c port
|
||||
// would buy nothing but another firewall rule, and both endpoints are admin-role and dialled by
|
||||
// the same driver nodes. Deliberately NOT gated on MeshTransport:Mode: mapping it under Dps
|
||||
// costs nothing (nothing dials it) whereas gating it would mean flipping the mode on a running
|
||||
// fleet requires a central restart before any node can ack — the same reasoning that registers
|
||||
// CentralCommunicationActor in both modes. CentralCommandAuthInterceptor (added at AddGrpc
|
||||
// above) is the ONLY inbound auth and fail-closes per cluster.
|
||||
app.MapGrpcService<CentralCommandGrpcService>();
|
||||
}
|
||||
|
||||
// Driver-node live-telemetry stream (per-cluster mesh Phase 5). Central dials in; gated by
|
||||
// TelemetryStreamAuthInterceptor (fail-closed on empty Telemetry:ApiKey).
|
||||
//
|
||||
// The node-side command endpoint (mesh gRPC command plane, Phase 1) rides the SAME driver-role
|
||||
// h2c listener rather than claiming a fourth port: central already dials this node here, both
|
||||
// directions are per-cluster-keyed, and a separate port would be one more thing to open on every
|
||||
// site firewall for no isolation gain. Mapping it is what makes MeshCommandAuthInterceptor live —
|
||||
// it is registered above and is a pass-through until a call matches this service's method prefix.
|
||||
// It fail-closes with no MeshTransport:GrpcPsk, so an operator who maps the service without setting
|
||||
// a key gets a closed endpoint, not an open one.
|
||||
if (hasDriver && telemetryListenPort > 0)
|
||||
{
|
||||
app.MapGrpcService<TelemetryStreamGrpcService>();
|
||||
app.MapGrpcService<NodeCommandGrpcService>();
|
||||
}
|
||||
// Mesh gRPC ingress: the LocalDb passive sync endpoint, the ConfigServe artifact + central
|
||||
// command endpoints, and the telemetry-stream + node-command endpoints. Extracted to
|
||||
// MeshGrpcEndpoints so a standing census test drives the SAME production mapping (see that
|
||||
// file's remarks — archreview 03/S14).
|
||||
app.MapOtOpcUaMeshGrpcServices(
|
||||
hasAdmin, hasDriver, syncListenPort, configServeGrpcPort, telemetryListenPort);
|
||||
|
||||
app.MapOtOpcUaHealth();
|
||||
app.MapOtOpcUaMetrics();
|
||||
|
||||
+241
@@ -0,0 +1,241 @@
|
||||
using Microsoft.AspNetCore.Authorization;
|
||||
using Microsoft.AspNetCore.Builder;
|
||||
using Microsoft.AspNetCore.Routing;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Shouldly;
|
||||
using Xunit;
|
||||
using ZB.MOM.WW.OtOpcUa.Host.Configuration;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.Host.IntegrationTests;
|
||||
|
||||
/// <summary>
|
||||
/// R3-01 (T1-T3) — an endpoint-metadata census over <c>MapOtOpcUaMeshGrpcServices</c>, driven in
|
||||
/// all three role shapes a node can actually carry (admin-only, driver-only, fused).
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// <b>Why this drives the PRODUCTION mapper, not a hand-copied shape.</b> The Critical this
|
||||
/// plan fixes (archreview 03/S14) survived seven phases of live gates precisely because every
|
||||
/// prior test built its own pipeline by hand — <c>MeshGrpcBoundaryTests</c>,
|
||||
/// <c>ConfigServeListenerTests</c> and <c>LocalDbSyncListenerTests</c> all reproduce the
|
||||
/// author's mental model of the mapping rather than the host's actual composition. This
|
||||
/// census calls <see cref="MeshGrpcEndpoints.MapOtOpcUaMeshGrpcServices"/> — the SAME method
|
||||
/// <c>Program.cs</c> calls — so it is structurally impossible for the guard to assert a shape
|
||||
/// production no longer has.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>Filters out grpc-dotnet's own catch-all infrastructure.</b> Every <c>MapGrpcService</c>
|
||||
/// call also registers a per-service "Unimplemented method for X" fallback, and the FIRST
|
||||
/// such call in an app additionally registers one shared "Unimplemented service" catch-all.
|
||||
/// Neither ever reaches real logic (they always answer <c>Unimplemented</c> regardless of
|
||||
/// caller identity), so their authorization metadata carries no security meaning.
|
||||
/// <see cref="RealMethodEndpoints"/> keeps only the five real, callable RPC methods, matched
|
||||
/// against the exact known route-pattern set rather than a route-pattern-shape heuristic —
|
||||
/// see that method's remarks for why a shape-based filter is NOT safe once LocalDbSync's
|
||||
/// empty-prefix <c>MapGroup</c> (T3) is in play.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>Guard-of-the-guard.</b> Every assertion below pins an EXACT expected count and the
|
||||
/// exact set of route patterns, never just "not empty" — a census that silently found zero
|
||||
/// endpoints (a typo'd role gate, a helper that never got called) would pass a bare
|
||||
/// non-empty check trivially. It cannot pass an exact-count-and-set check by accident.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
public sealed class MeshGrpcAnonymousMetadataTests
|
||||
{
|
||||
private const string ArtifactFetch = "/deployment_artifact.v1.DeploymentArtifactService/Fetch";
|
||||
private const string CentralReportApply = "/mesh_transport.v1.CentralCommandService/ReportApply";
|
||||
private const string TelemetrySubscribe = "/telemetry.v1.TelemetryStreamService/Subscribe";
|
||||
private const string NodeExecute = "/mesh_transport.v1.NodeCommandService/Execute";
|
||||
private const string LocalDbSyncSync = "/localdb_sync.v1.LocalDbSync/Sync";
|
||||
|
||||
/// <summary>The five known mesh gRPC method prefixes — the ONLY endpoints this map may anonymise.</summary>
|
||||
private static readonly string[] AllFivePrefixes =
|
||||
[LocalDbSyncSync, ArtifactFetch, CentralReportApply, TelemetrySubscribe, NodeExecute];
|
||||
|
||||
[Fact]
|
||||
public void AdminOnly_MapsExactlyTheConfigServeEndpoints()
|
||||
{
|
||||
// Admin-only: no driver role, so neither LocalDbSync nor the telemetry/node-command pair are
|
||||
// mapped even though a port is configured for them below (0 here, but the shape is the point).
|
||||
var endpoints = RealMethodEndpoints(BuildApp(
|
||||
hasAdmin: true, hasDriver: false,
|
||||
syncListenPort: 0, configServeGrpcPort: 4055, telemetryListenPort: 0));
|
||||
|
||||
endpoints.Count.ShouldBe(2);
|
||||
endpoints.Select(RoutePatternOf).ShouldBe([ArtifactFetch, CentralReportApply], ignoreOrder: true);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void DriverOnly_WithNoSyncPort_MapsExactlyTheTelemetryAndNodeCommandEndpoints()
|
||||
{
|
||||
var endpoints = RealMethodEndpoints(BuildApp(
|
||||
hasAdmin: false, hasDriver: true,
|
||||
syncListenPort: 0, configServeGrpcPort: 0, telemetryListenPort: 4056));
|
||||
|
||||
endpoints.Count.ShouldBe(2);
|
||||
endpoints.Select(RoutePatternOf).ShouldBe([TelemetrySubscribe, NodeExecute], ignoreOrder: true);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void DriverOnly_WithASyncPort_AlsoMapsTheLocalDbSyncEndpoint()
|
||||
{
|
||||
var endpoints = RealMethodEndpoints(BuildApp(
|
||||
hasAdmin: false, hasDriver: true,
|
||||
syncListenPort: 4054, configServeGrpcPort: 0, telemetryListenPort: 4056));
|
||||
|
||||
endpoints.Count.ShouldBe(3);
|
||||
endpoints.Select(RoutePatternOf).ShouldBe(
|
||||
[LocalDbSyncSync, TelemetrySubscribe, NodeExecute], ignoreOrder: true);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Fused_WithAllThreePorts_MapsAllFiveEndpoints()
|
||||
{
|
||||
// The docker-dev central pair's shape (admin,driver,cluster-MAIN) once all three dedicated
|
||||
// h2c ports are configured — the shape 03/S14 called out as breaking the deploy plane for
|
||||
// the whole fleet, not only the four sites.
|
||||
var endpoints = RealMethodEndpoints(BuildApp(
|
||||
hasAdmin: true, hasDriver: true,
|
||||
syncListenPort: 4054, configServeGrpcPort: 4055, telemetryListenPort: 4056));
|
||||
|
||||
endpoints.Count.ShouldBe(5);
|
||||
endpoints.Select(RoutePatternOf).ShouldBe(AllFivePrefixes, ignoreOrder: true);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void NeitherRoleNorAnyPort_MapsNothing()
|
||||
{
|
||||
// The default-OFF shape: every dedicated port is 0. Nothing about the mesh gRPC surface
|
||||
// should exist on a node that configured none of it.
|
||||
var endpoints = RealMethodEndpoints(BuildApp(
|
||||
hasAdmin: false, hasDriver: false,
|
||||
syncListenPort: 0, configServeGrpcPort: 0, telemetryListenPort: 0));
|
||||
|
||||
endpoints.ShouldBeEmpty();
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------------------
|
||||
// 03/S14 — every mesh endpoint carries IAllowAnonymous, so the AdminUI cookie FallbackPolicy
|
||||
// (RequireAuthenticatedUser) never reaches it; the PSK interceptors are the ONLY gate. T3
|
||||
// (LocalDbSync's empty-prefix group) is asserted separately below.
|
||||
// ---------------------------------------------------------------------------------------
|
||||
|
||||
[Fact]
|
||||
public void AdminOnly_BothConfigServeEndpoints_CarryAllowAnonymous()
|
||||
{
|
||||
var endpoints = RealMethodEndpoints(BuildApp(
|
||||
hasAdmin: true, hasDriver: false,
|
||||
syncListenPort: 0, configServeGrpcPort: 4055, telemetryListenPort: 0));
|
||||
|
||||
AnonymousPatternsOf(endpoints).ShouldBe([ArtifactFetch, CentralReportApply], ignoreOrder: true);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void DriverOnly_BothTelemetryAndNodeCommandEndpoints_CarryAllowAnonymous()
|
||||
{
|
||||
var endpoints = RealMethodEndpoints(BuildApp(
|
||||
hasAdmin: false, hasDriver: true,
|
||||
syncListenPort: 0, configServeGrpcPort: 0, telemetryListenPort: 4056));
|
||||
|
||||
AnonymousPatternsOf(endpoints).ShouldBe([TelemetrySubscribe, NodeExecute], ignoreOrder: true);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Fused_AllFiveMeshEndpoints_CarryAllowAnonymous()
|
||||
{
|
||||
// Includes LocalDbSync (T3): MapZbLocalDbSync returns IEndpointRouteBuilder, not an
|
||||
// IEndpointConventionBuilder, so .AllowAnonymous() cannot be chained onto it the way the
|
||||
// other four are — it needs the empty-prefix MapGroup wrapper instead.
|
||||
var endpoints = RealMethodEndpoints(BuildApp(
|
||||
hasAdmin: true, hasDriver: true,
|
||||
syncListenPort: 4054, configServeGrpcPort: 4055, telemetryListenPort: 4056));
|
||||
|
||||
AnonymousPatternsOf(endpoints).ShouldBe(AllFivePrefixes, ignoreOrder: true);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void LocalDbSync_AllowAnonymousGroup_LeavesTheRoutePatternByteIdentical()
|
||||
{
|
||||
// The empty-prefix MapGroup wrapper (T3) must be pure convention-carrying — it must not
|
||||
// change the gRPC route pattern LocalDbSyncAuthInterceptor and the sync client both key on.
|
||||
var endpoints = RealMethodEndpoints(BuildApp(
|
||||
hasAdmin: false, hasDriver: true,
|
||||
syncListenPort: 4054, configServeGrpcPort: 0, telemetryListenPort: 0));
|
||||
|
||||
var sync = endpoints.ShouldHaveSingleItem();
|
||||
sync.RoutePattern.RawText.ShouldBe(LocalDbSyncSync);
|
||||
sync.Metadata.GetMetadata<IAllowAnonymous>().ShouldNotBeNull();
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void AllowAnonymous_NeverLeaksBeyondTheFiveKnownMeshEndpoints()
|
||||
{
|
||||
// Scope guard (theme 2): an endpoint mapped ALONGSIDE the mesh services, through no seam this
|
||||
// extension touches, must never come out anonymous. If it did, some future edit widened the
|
||||
// AllowAnonymous convention beyond the five known prefixes — the exact failure class the
|
||||
// fallback-policy opt-out exists to bound.
|
||||
var builder = WebApplication.CreateBuilder();
|
||||
builder.Environment.ApplicationName = typeof(MeshGrpcAnonymousMetadataTests).Assembly.GetName().Name!;
|
||||
builder.Services.AddGrpc();
|
||||
var app = builder.Build();
|
||||
|
||||
app.MapOtOpcUaMeshGrpcServices(
|
||||
hasAdmin: true, hasDriver: true,
|
||||
syncListenPort: 4054, configServeGrpcPort: 4055, telemetryListenPort: 4056);
|
||||
var sentinel = app.MapGet("/sentinel", () => "ok");
|
||||
|
||||
var sentinelEndpoint = ((IEndpointRouteBuilder)app).DataSources
|
||||
.SelectMany(d => d.Endpoints)
|
||||
.OfType<RouteEndpoint>()
|
||||
.Single(e => e.RoutePattern.RawText == "/sentinel");
|
||||
|
||||
sentinelEndpoint.Metadata.GetMetadata<IAllowAnonymous>().ShouldBeNull();
|
||||
_ = sentinel; // the builder handle itself is not the assertion target
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------------------
|
||||
// harness
|
||||
// ---------------------------------------------------------------------------------------
|
||||
|
||||
private static WebApplication BuildApp(
|
||||
bool hasAdmin,
|
||||
bool hasDriver,
|
||||
int syncListenPort,
|
||||
int configServeGrpcPort,
|
||||
int telemetryListenPort)
|
||||
{
|
||||
var builder = WebApplication.CreateBuilder();
|
||||
builder.Environment.ApplicationName = typeof(MeshGrpcAnonymousMetadataTests).Assembly.GetName().Name!;
|
||||
builder.Services.AddGrpc();
|
||||
|
||||
var app = builder.Build();
|
||||
app.MapOtOpcUaMeshGrpcServices(
|
||||
hasAdmin, hasDriver, syncListenPort, configServeGrpcPort, telemetryListenPort);
|
||||
return app;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// The real, callable RPC method endpoints — excludes grpc-dotnet's per-app "Unimplemented
|
||||
/// service" catch-all and per-service "Unimplemented method for X" fallback (see class
|
||||
/// remarks). Filtered against the exact set of known method route patterns rather than a
|
||||
/// "starts with /" heuristic: inside the empty-prefix <c>MapGroup</c> the LocalDbSync map
|
||||
/// (T3) uses, grpc-dotnet's own fallback patterns ALSO gain a leading <c>/</c>, so a
|
||||
/// prefix-shaped filter would let them back in.
|
||||
/// </summary>
|
||||
private static List<RouteEndpoint> RealMethodEndpoints(WebApplication app) =>
|
||||
((IEndpointRouteBuilder)app).DataSources
|
||||
.SelectMany(d => d.Endpoints)
|
||||
.OfType<RouteEndpoint>()
|
||||
.Where(e => AllFivePrefixes.Contains(e.RoutePattern.RawText))
|
||||
.ToList();
|
||||
|
||||
private static string RoutePatternOf(RouteEndpoint e) => e.RoutePattern.RawText!;
|
||||
|
||||
/// <summary>Route patterns of the endpoints in <paramref name="endpoints"/> that carry <see cref="IAllowAnonymous"/>.</summary>
|
||||
private static List<string> AnonymousPatternsOf(IEnumerable<RouteEndpoint> endpoints) =>
|
||||
endpoints
|
||||
.Where(e => e.Metadata.GetMetadata<IAllowAnonymous>() is not null)
|
||||
.Select(RoutePatternOf)
|
||||
.ToList();
|
||||
}
|
||||
@@ -0,0 +1,395 @@
|
||||
using System.Diagnostics.CodeAnalysis;
|
||||
using System.Net;
|
||||
using Akka.Actor;
|
||||
using Akka.Hosting;
|
||||
using Grpc.Core;
|
||||
using Grpc.Net.Client;
|
||||
using Microsoft.AspNetCore.Authentication.Cookies;
|
||||
using Microsoft.AspNetCore.Builder;
|
||||
using Microsoft.AspNetCore.Server.Kestrel.Core;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Microsoft.Extensions.Hosting;
|
||||
using Shouldly;
|
||||
using Xunit;
|
||||
using ZB.MOM.WW.LocalDb.Replication;
|
||||
using ZB.MOM.WW.OtOpcUa.Cluster;
|
||||
using ZB.MOM.WW.OtOpcUa.Commons.Messages.Admin;
|
||||
using ZB.MOM.WW.OtOpcUa.Commons.Messages.Deploy;
|
||||
using ZB.MOM.WW.OtOpcUa.Commons.Messages.Mesh;
|
||||
using ZB.MOM.WW.OtOpcUa.Commons.Protos.DeploymentArtifact.V1;
|
||||
using ZB.MOM.WW.OtOpcUa.Commons.Protos.MeshTransport.V1;
|
||||
using ZB.MOM.WW.OtOpcUa.Commons.Protos.Telemetry.V1;
|
||||
using ZB.MOM.WW.OtOpcUa.Commons.Types;
|
||||
using ZB.MOM.WW.OtOpcUa.Configuration;
|
||||
using ZB.MOM.WW.OtOpcUa.Configuration.Entities;
|
||||
using ZB.MOM.WW.OtOpcUa.ControlPlane;
|
||||
using ZB.MOM.WW.OtOpcUa.ControlPlane.Communication;
|
||||
using ZB.MOM.WW.OtOpcUa.Host.Configuration;
|
||||
using ZB.MOM.WW.OtOpcUa.Runtime;
|
||||
using ZB.MOM.WW.OtOpcUa.Runtime.Telemetry;
|
||||
using ZB.MOM.WW.OtOpcUa.Security;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.Host.IntegrationTests;
|
||||
|
||||
/// <summary>
|
||||
/// R3-01 T4 — the CI-shaped guard that would have caught 03/S14: mesh gRPC calls through the
|
||||
/// REAL <c>AddOtOpcUaAuth</c> + <c>UseAuthentication</c>/<c>UseAuthorization</c> stack, not a
|
||||
/// hand-built pipeline that never installs the middleware that broke production.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// <b>The load-bearing assertion is the ALLOW.</b> A blocked call and a refused call both
|
||||
/// "fail"; only a correctly-keyed call SUCCEEDING through the real cookie
|
||||
/// <c>FallbackPolicy</c> proves the <c>.AllowAnonymous()</c> opt-outs (T2/T3) actually work in
|
||||
/// a pipeline that carries the policy at all. Every positive test here failed before T2/T3 —
|
||||
/// confirmed by running this class against the T1 commit, where every one of them was
|
||||
/// challenged by the cookie handler instead of reaching the interceptor.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>Negative control (anti-vacuity).</b> <see cref="LocalDbSyncService"/> is mapped directly
|
||||
/// here — via a bare <c>MapGrpcService</c>, deliberately NOT through
|
||||
/// <c>MapOtOpcUaMeshGrpcServices</c> — with no <c>.AllowAnonymous()</c>. A correctly-shaped
|
||||
/// call to it must be refused by the cookie challenge. If it were not, the fixture's auth
|
||||
/// stack would be inert and every positive assertion above would be proving nothing.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
public sealed class MeshGrpcAuthStackTests : IAsyncLifetime
|
||||
{
|
||||
private const string ConfigServeKey = "test-configserve-key";
|
||||
private const string NodeCommandKey = "test-node-psk";
|
||||
private const string CentralCommandKey = "test-central-psk";
|
||||
private const string TelemetryKey = "test-telemetry-key";
|
||||
private const string ClusterId = "MAIN";
|
||||
|
||||
private WebApplication _app = null!;
|
||||
private ActorSystem _system = null!;
|
||||
private int _port;
|
||||
private TaskCompletionSource<object> _centralInbox = null!;
|
||||
private TaskCompletionSource<object> _nodeInbox = null!;
|
||||
|
||||
/// <inheritdoc />
|
||||
public async ValueTask InitializeAsync()
|
||||
{
|
||||
var builder = WebApplication.CreateBuilder();
|
||||
builder.Environment.ApplicationName = typeof(MeshGrpcAuthStackTests).Assembly.GetName().Name!;
|
||||
// Production, not the WebApplicationBuilder default: forces ValidateOnBuild/ValidateScopes OFF
|
||||
// (they engage only under Development), so services never resolved without a login attempt
|
||||
// (ILdapAuthService's graph) don't need to be fully stood up for this fixture to build.
|
||||
builder.Environment.EnvironmentName = Environments.Production;
|
||||
builder.Logging.ClearProviders();
|
||||
|
||||
// DataProtection's PersistKeysToDbContext<OtOpcUaConfigDbContext> (inside AddOtOpcUaAuth)
|
||||
// resolves a SCOPED OtOpcUaConfigDbContext via CreateScope() + GetRequiredService — it has no
|
||||
// factory support. Mirrors AddOtOpcUaConfigDb's own factory+scoped-bridge shape, over InMemory.
|
||||
var dbName = Guid.NewGuid().ToString("N");
|
||||
builder.Services.AddDbContextFactory<OtOpcUaConfigDbContext>(o => o.UseInMemoryDatabase(dbName));
|
||||
builder.Services.AddScoped(sp =>
|
||||
sp.GetRequiredService<IDbContextFactory<OtOpcUaConfigDbContext>>().CreateDbContext());
|
||||
|
||||
builder.Configuration.AddInMemoryCollection(new Dictionary<string, string?>
|
||||
{
|
||||
// The whole point of this fixture: exercise the REAL cookie handler, not
|
||||
// AutoLoginAuthenticationHandler. See CookieSchemeIsTheRealHandler_NotAutoLogin.
|
||||
["Security:Auth:DisableLogin"] = "false",
|
||||
// This fixture serves cleartext h2c; a HTTPS-only cookie policy is irrelevant to what is
|
||||
// under test (the FallbackPolicy applies identically either way) and would otherwise log
|
||||
// a startup Warning for no reason.
|
||||
["Security:Cookie:RequireHttpsCookie"] = "false",
|
||||
});
|
||||
|
||||
builder.Services.AddOtOpcUaAuth(builder.Configuration);
|
||||
|
||||
// Interceptor options bound directly (NOT via AddOtOpcUaCluster, which also wires
|
||||
// AkkaClusterOptions/ConfigSourceOptions/TelemetryDialOptions validators with ValidateOnStart —
|
||||
// heavy machinery this fixture has no use for and no seed-node config to satisfy).
|
||||
builder.Services.Configure<ConfigServeOptions>(o => o.ApiKey = ConfigServeKey);
|
||||
builder.Services.Configure<MeshTransportOptions>(o => o.GrpcPsk = NodeCommandKey);
|
||||
builder.Services.Configure<TelemetryOptions>(o => o.ApiKey = TelemetryKey);
|
||||
builder.Services.AddSingleton<IMeshPskProvider>(new FixedPskProvider(ClusterId, CentralCommandKey));
|
||||
builder.Services.AddSingleton<ITelemetryLocalHub, TelemetryLocalHub>();
|
||||
|
||||
_system = ActorSystem.Create("mesh-auth-stack-" + Guid.NewGuid().ToString("N"));
|
||||
_centralInbox = new TaskCompletionSource<object>(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||
_nodeInbox = new TaskCompletionSource<object>(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||
var registry = new ActorRegistry();
|
||||
registry.Register<CentralCommunicationActorKey>(_system.ActorOf(CaptureActor.Props(_centralInbox)));
|
||||
registry.Register<NodeCommunicationActorKey>(_system.ActorOf(CaptureActor.Props(_nodeInbox)));
|
||||
builder.Services.AddSingleton(registry);
|
||||
|
||||
// The production interceptor set — the ONLY inbound auth on the gRPC side. Deliberately does
|
||||
// NOT include LocalDbSyncAuthInterceptor: LocalDbSyncService is mapped below purely as the
|
||||
// negative control and is never given a matching interceptor here.
|
||||
builder.Services.AddGrpc(o =>
|
||||
{
|
||||
o.Interceptors.Add<ConfigServeAuthInterceptor>();
|
||||
o.Interceptors.Add<CentralCommandAuthInterceptor>();
|
||||
o.Interceptors.Add<TelemetryStreamAuthInterceptor>();
|
||||
o.Interceptors.Add<MeshCommandAuthInterceptor>();
|
||||
});
|
||||
|
||||
_port = GetFreePort();
|
||||
builder.WebHost.ConfigureKestrel(k => k.ListenAnyIP(_port, o => o.Protocols = HttpProtocols.Http2));
|
||||
|
||||
_app = builder.Build();
|
||||
_app.UseAuthentication();
|
||||
_app.UseAuthorization();
|
||||
|
||||
// The production mapper — the four chainable maps carry AllowAnonymous (T2); LocalDbSync is
|
||||
// NOT mapped here (syncListenPort: 0) so it stays free for the negative control below without
|
||||
// a route collision.
|
||||
_app.MapOtOpcUaMeshGrpcServices(
|
||||
hasAdmin: true, hasDriver: true,
|
||||
syncListenPort: 0, configServeGrpcPort: 1, telemetryListenPort: 1);
|
||||
|
||||
// Negative control (T4): mapped directly, NOT through MapOtOpcUaMeshGrpcServices, and
|
||||
// deliberately with no .AllowAnonymous().
|
||||
_app.MapGrpcService<LocalDbSyncService>();
|
||||
|
||||
await _app.StartAsync(TestContext.Current.CancellationToken);
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
public async ValueTask DisposeAsync()
|
||||
{
|
||||
await _app.StopAsync(TimeSpan.FromSeconds(10));
|
||||
await _app.DisposeAsync();
|
||||
await _system.Terminate().WaitAsync(TimeSpan.FromSeconds(15));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task CookieSchemeIsTheRealHandler_NotAutoLogin()
|
||||
{
|
||||
// If this resolves AutoLoginAuthenticationHandler, every assertion below is meaningless — the
|
||||
// fallback policy would pass every request regardless of AllowAnonymous.
|
||||
var provider = _app.Services.GetRequiredService<Microsoft.AspNetCore.Authentication.IAuthenticationSchemeProvider>();
|
||||
var scheme = await provider.GetSchemeAsync(CookieAuthenticationDefaults.AuthenticationScheme);
|
||||
|
||||
scheme.ShouldNotBeNull();
|
||||
scheme.HandlerType.ShouldBe(typeof(CookieAuthenticationHandler));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task CorrectlyKeyed_ArtifactFetch_SucceedsThroughTheRealAuthStack()
|
||||
{
|
||||
var deploymentId = Guid.NewGuid();
|
||||
var blob = "R3-01-T4-artifact-bytes"u8.ToArray();
|
||||
var factory = _app.Services.GetRequiredService<IDbContextFactory<OtOpcUaConfigDbContext>>();
|
||||
await using (var db = await factory.CreateDbContextAsync(TestContext.Current.CancellationToken))
|
||||
{
|
||||
db.Deployments.Add(new Deployment
|
||||
{
|
||||
DeploymentId = deploymentId,
|
||||
RevisionHash = new string('a', 64),
|
||||
CreatedBy = "r3-01-t4",
|
||||
ArtifactBlob = blob,
|
||||
});
|
||||
await db.SaveChangesAsync(TestContext.Current.CancellationToken);
|
||||
}
|
||||
|
||||
var client = new DeploymentArtifactService.DeploymentArtifactServiceClient(Channel());
|
||||
var headers = new Metadata { { "authorization", $"Bearer {ConfigServeKey}" } };
|
||||
|
||||
using var call = client.Fetch(
|
||||
new FetchRequest { DeploymentId = deploymentId.ToString() },
|
||||
headers,
|
||||
cancellationToken: TestContext.Current.CancellationToken);
|
||||
|
||||
using var buffer = new MemoryStream();
|
||||
await foreach (var chunk in call.ResponseStream.ReadAllAsync(TestContext.Current.CancellationToken))
|
||||
chunk.Data.WriteTo(buffer);
|
||||
|
||||
buffer.ToArray().ShouldBe(blob);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task CorrectlyKeyed_CentralCommandReportApply_SucceedsThroughTheRealAuthStack()
|
||||
{
|
||||
var ack = new ApplyAck(
|
||||
new DeploymentId(Guid.NewGuid()),
|
||||
new NodeId("node-a"),
|
||||
ApplyAckOutcome.Applied,
|
||||
FailureReason: null,
|
||||
CorrelationId.NewId());
|
||||
|
||||
var client = new CentralCommandService.CentralCommandServiceClient(Channel());
|
||||
var headers = new Metadata
|
||||
{
|
||||
{ "authorization", $"Bearer {CentralCommandKey}" },
|
||||
{ "x-otopcua-cluster", ClusterId },
|
||||
};
|
||||
|
||||
await client.ReportApplyAsync(
|
||||
MeshTransportDtoMapper.ToProto(ack), headers, cancellationToken: TestContext.Current.CancellationToken);
|
||||
|
||||
var received = await _centralInbox.Task.WaitAsync(TimeSpan.FromSeconds(10), TestContext.Current.CancellationToken);
|
||||
received.ShouldBeOfType<ApplyAck>().DeploymentId.ShouldBe(ack.DeploymentId);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task CorrectlyKeyed_NodeCommandExecute_SucceedsThroughTheRealAuthStack()
|
||||
{
|
||||
var command = new RestartDriver("C1", "driver-1", "alice", Guid.NewGuid());
|
||||
var client = new NodeCommandService.NodeCommandServiceClient(Channel());
|
||||
var headers = new Metadata { { "authorization", $"Bearer {NodeCommandKey}" } };
|
||||
|
||||
await client.ExecuteAsync(
|
||||
MeshTransportDtoMapper.ToEnvelope(command), headers, cancellationToken: TestContext.Current.CancellationToken);
|
||||
|
||||
var received = await _nodeInbox.Task.WaitAsync(TimeSpan.FromSeconds(10), TestContext.Current.CancellationToken);
|
||||
received.ShouldBeOfType<RestartDriver>().DriverInstanceId.ShouldBe("driver-1");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task FalsifiabilityControl_AnUnAnonymisedServiceMapped_RefusesEvenACorrectlyShapedCall()
|
||||
{
|
||||
// LocalDbSyncService carries no AllowAnonymous in this fixture (see InitializeAsync). If this
|
||||
// call succeeds, the fixture's auth stack is not actually challenging unmarked endpoints and
|
||||
// every "succeeds through the real auth stack" assertion above proves nothing.
|
||||
var client = new ZB.MOM.WW.LocalDb.Contracts.LocalDbSync.LocalDbSyncClient(Channel());
|
||||
var headers = new Metadata { { "authorization", "Bearer irrelevant-to-this-endpoint" } };
|
||||
|
||||
using var call = client.Sync(headers, cancellationToken: TestContext.Current.CancellationToken);
|
||||
await call.RequestStream.CompleteAsync();
|
||||
|
||||
// Review (R3-01): asserting a bare Exception was vacuous — an anonymised endpoint ALSO throws
|
||||
// here (handler deps are not wired in this fixture => RpcException Unknown). Only the
|
||||
// cookie-challenge signature discriminates: the FallbackPolicy's 302 to /login is a
|
||||
// non-gRPC response, surfaced by grpc-dotnet as Internal + PROTOCOL_ERROR.
|
||||
var ex = await Should.ThrowAsync<RpcException>(
|
||||
() => call.ResponseStream.MoveNext(TestContext.Current.CancellationToken));
|
||||
ex.StatusCode.ShouldBe(StatusCode.Internal);
|
||||
ex.Status.Detail.ShouldContain("PROTOCOL_ERROR");
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------------------
|
||||
// T5 - status fidelity: a no-PSK / wrong-PSK call reports the INTERCEPTOR's verdict, never a
|
||||
// cookie challenge. This is the vacuity trap the plan calls out: ConfigServeAuthInterceptor
|
||||
// throws StatusCode.Unauthenticated, and Grpc.Net.Client also maps an HTTP 401 to
|
||||
// Unauthenticated - so "the call failed with Unauthenticated" is true on a broken build AND a
|
||||
// fixed one. The interceptor's DETAIL string is the only thing that discriminates them.
|
||||
//
|
||||
// Observed cookie-challenge signature (captured red-then-green against this same fixture, see
|
||||
// MeshGrpcAuthStackTests' positive tests): the challenge does not surface as a clean
|
||||
// Unauthenticated/PermissionDenied at all. Grpc.Net.Client reports
|
||||
// RpcException(StatusCode.Internal, "Error starting gRPC call. HttpRequestException: The HTTP/2
|
||||
// server reset the stream. HTTP/2 error code 'PROTOCOL_ERROR' (0x1)..."), because the cookie
|
||||
// handler's redirect/401 breaks the prior-knowledge h2c stream before any gRPC trailers are
|
||||
// written. So a status-only assertion here would ALSO be vacuous in the other direction: a
|
||||
// regressed build fails with StatusCode.Internal, never Unauthenticated/PermissionDenied, and a
|
||||
// caller checking only "did it throw" cannot tell a blocked call from a refused one. Recorded
|
||||
// here per the plan (T5 step 2) - this is the exact string the live gate (T10) greps for.
|
||||
// ---------------------------------------------------------------------------------------
|
||||
|
||||
[Theory]
|
||||
[InlineData(null)]
|
||||
[InlineData("Bearer the-wrong-key")]
|
||||
public async Task NoOrWrongPsk_ArtifactFetch_FailsWithTheInterceptorsDetail(string? authHeader)
|
||||
{
|
||||
var client = new DeploymentArtifactService.DeploymentArtifactServiceClient(Channel());
|
||||
var headers = new Metadata();
|
||||
if (authHeader is not null)
|
||||
headers.Add("authorization", authHeader);
|
||||
|
||||
using var call = client.Fetch(
|
||||
new FetchRequest { DeploymentId = Guid.NewGuid().ToString() },
|
||||
headers,
|
||||
cancellationToken: TestContext.Current.CancellationToken);
|
||||
|
||||
var ex = await Should.ThrowAsync<RpcException>(
|
||||
() => call.ResponseStream.MoveNext(TestContext.Current.CancellationToken));
|
||||
|
||||
ex.StatusCode.ShouldBe(StatusCode.Unauthenticated);
|
||||
ex.Status.Detail.ShouldBe("Config-serve authentication failed.");
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(null)]
|
||||
[InlineData("Bearer the-wrong-key")]
|
||||
public async Task NoOrWrongPsk_TelemetrySubscribe_FailsWithTheInterceptorsDetail(string? authHeader)
|
||||
{
|
||||
var client = new TelemetryStreamService.TelemetryStreamServiceClient(Channel());
|
||||
var headers = new Metadata();
|
||||
if (authHeader is not null)
|
||||
headers.Add("authorization", authHeader);
|
||||
|
||||
using var call = client.Subscribe(
|
||||
new TelemetryStreamRequest { CorrelationId = "r3-01-t5" },
|
||||
headers,
|
||||
cancellationToken: TestContext.Current.CancellationToken);
|
||||
|
||||
var ex = await Should.ThrowAsync<RpcException>(
|
||||
() => call.ResponseStream.MoveNext(TestContext.Current.CancellationToken));
|
||||
|
||||
ex.StatusCode.ShouldBe(StatusCode.PermissionDenied);
|
||||
ex.Status.Detail.ShouldBe("Telemetry stream authentication failed.");
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(null)]
|
||||
[InlineData("Bearer the-wrong-key")]
|
||||
public async Task NoOrWrongPsk_NodeCommandExecute_FailsWithTheInterceptorsDetail(string? authHeader)
|
||||
{
|
||||
var command = new RestartDriver("C1", "driver-1", "alice", Guid.NewGuid());
|
||||
var client = new NodeCommandService.NodeCommandServiceClient(Channel());
|
||||
var headers = new Metadata();
|
||||
if (authHeader is not null)
|
||||
headers.Add("authorization", authHeader);
|
||||
|
||||
var ex = await Should.ThrowAsync<RpcException>(() => client.ExecuteAsync(
|
||||
MeshTransportDtoMapper.ToEnvelope(command), headers,
|
||||
cancellationToken: TestContext.Current.CancellationToken).ResponseAsync);
|
||||
|
||||
ex.StatusCode.ShouldBe(StatusCode.PermissionDenied);
|
||||
ex.Status.Detail.ShouldBe("Node command authentication failed.");
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------------------
|
||||
// harness
|
||||
// ---------------------------------------------------------------------------------------
|
||||
|
||||
private GrpcChannel Channel() => GrpcChannel.ForAddress($"http://localhost:{_port}");
|
||||
|
||||
private static int GetFreePort()
|
||||
{
|
||||
using var listener = new System.Net.Sockets.TcpListener(IPAddress.Loopback, 0);
|
||||
listener.Start();
|
||||
var port = ((IPEndPoint)listener.LocalEndpoint).Port;
|
||||
listener.Stop();
|
||||
return port;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Hands out the one cluster key <see cref="CentralCommandAuthInterceptor"/> expects. Stands
|
||||
/// in for the real <c>MeshPskProvider</c> (secret-store lookups, caching, the known-cluster
|
||||
/// gate) — out of scope here, where the subject is the HTTP-layer auth stack, not the PSK
|
||||
/// resolution chain (that is <c>MeshPskProviderTests</c>' job).
|
||||
/// </summary>
|
||||
private sealed class FixedPskProvider(string clusterId, string psk) : IMeshPskProvider
|
||||
{
|
||||
public bool TryGetPsk(string clusterId2, [NotNullWhen(true)] out string? psk2)
|
||||
{
|
||||
psk2 = string.Equals(clusterId2, clusterId, StringComparison.Ordinal) ? psk : null;
|
||||
return psk2 is not null;
|
||||
}
|
||||
|
||||
public bool Matches(string clusterId2, string presented) =>
|
||||
string.Equals(clusterId2, clusterId, StringComparison.Ordinal)
|
||||
&& string.Equals(presented, psk, StringComparison.Ordinal);
|
||||
|
||||
public void SetKnownClusters(IReadOnlyCollection<string> clusterIds) { }
|
||||
|
||||
public void Invalidate(string clusterId2) { }
|
||||
|
||||
public void InvalidateAll() { }
|
||||
}
|
||||
|
||||
/// <summary>Completes an inbox with the first message it receives.</summary>
|
||||
private sealed class CaptureActor : ReceiveActor
|
||||
{
|
||||
public static Props Props(TaskCompletionSource<object> sink) =>
|
||||
Akka.Actor.Props.Create(() => new CaptureActor(sink));
|
||||
|
||||
public CaptureActor(TaskCompletionSource<object> sink) => ReceiveAny(m => sink.TrySetResult(m));
|
||||
}
|
||||
}
|
||||
+202
@@ -0,0 +1,202 @@
|
||||
using System.Diagnostics.CodeAnalysis;
|
||||
using System.Net;
|
||||
using Akka.Actor;
|
||||
using Akka.Hosting;
|
||||
using Grpc.Core;
|
||||
using Grpc.Net.Client;
|
||||
using Microsoft.AspNetCore.Authorization;
|
||||
using Microsoft.AspNetCore.Builder;
|
||||
using Microsoft.AspNetCore.Server.Kestrel.Core;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Shouldly;
|
||||
using Xunit;
|
||||
using ZB.MOM.WW.OtOpcUa.Cluster;
|
||||
using ZB.MOM.WW.OtOpcUa.Commons.Messages.Admin;
|
||||
using ZB.MOM.WW.OtOpcUa.Commons.Messages.Drivers;
|
||||
using ZB.MOM.WW.OtOpcUa.Commons.Messages.Mesh;
|
||||
using ZB.MOM.WW.OtOpcUa.Commons.Protos.MeshTransport.V1;
|
||||
using ZB.MOM.WW.OtOpcUa.Commons.Protos.Telemetry.V1;
|
||||
using ZB.MOM.WW.OtOpcUa.ControlPlane.Communication;
|
||||
using ZB.MOM.WW.OtOpcUa.Host.Configuration;
|
||||
using ZB.MOM.WW.OtOpcUa.Runtime;
|
||||
using ZB.MOM.WW.OtOpcUa.Runtime.Telemetry;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.Host.IntegrationTests;
|
||||
|
||||
/// <summary>
|
||||
/// R3-01 T6 — pins the report's "driver-only nodes are structurally unaffected" finding
|
||||
/// (archreview 03/S14): a node with no <c>admin</c> role never registers
|
||||
/// <c>AddOtOpcUaAuth</c>/<c>UseAuthentication</c>/<c>UseAuthorization</c> at all, so it has no
|
||||
/// cookie <c>FallbackPolicy</c> to opt out of in the first place — the PSK interceptors were
|
||||
/// always the ONLY gate on this shape, with or without <c>.AllowAnonymous()</c>.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// A future "register auth everywhere for consistency" change would silently reintroduce 03/S14
|
||||
/// for every site node in the fleet. This class is what turns that into a failing test instead of
|
||||
/// a live outage: it asserts BOTH that the driver-only mesh endpoints behave identically to the
|
||||
/// fused shape's (T4) AND that the reason they do — no <see cref="IAuthorizationPolicyProvider"/>
|
||||
/// registered at all — actually holds, so a change that adds auth services here fails on the
|
||||
/// second assertion even if it somehow left the first passing.
|
||||
/// </remarks>
|
||||
public sealed class MeshGrpcDriverOnlyPipelineTests : IAsyncLifetime
|
||||
{
|
||||
private const string NodeCommandKey = "test-driveronly-node-psk";
|
||||
private const string TelemetryKey = "test-driveronly-telemetry-key";
|
||||
|
||||
private WebApplication _app = null!;
|
||||
private ActorSystem _system = null!;
|
||||
private int _port;
|
||||
private ITelemetryLocalHub _hub = null!;
|
||||
private TaskCompletionSource<object> _nodeInbox = null!;
|
||||
|
||||
/// <inheritdoc />
|
||||
public async ValueTask InitializeAsync()
|
||||
{
|
||||
var builder = WebApplication.CreateBuilder();
|
||||
builder.Environment.ApplicationName = typeof(MeshGrpcDriverOnlyPipelineTests).Assembly.GetName().Name!;
|
||||
builder.Logging.ClearProviders();
|
||||
|
||||
// Deliberately NO AddOtOpcUaAuth, NO UseAuthentication, NO UseAuthorization — this IS the
|
||||
// driver-only shape (Program.cs gates all three on hasAdmin). Nothing here should need them.
|
||||
builder.Services.Configure<MeshTransportOptions>(o => o.GrpcPsk = NodeCommandKey);
|
||||
builder.Services.Configure<TelemetryOptions>(o => o.ApiKey = TelemetryKey);
|
||||
builder.Services.AddSingleton<ITelemetryLocalHub, TelemetryLocalHub>();
|
||||
|
||||
_system = ActorSystem.Create("mesh-driveronly-" + Guid.NewGuid().ToString("N"));
|
||||
_nodeInbox = new TaskCompletionSource<object>(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||
var registry = new ActorRegistry();
|
||||
registry.Register<NodeCommunicationActorKey>(_system.ActorOf(CaptureActor.Props(_nodeInbox)));
|
||||
builder.Services.AddSingleton(registry);
|
||||
|
||||
// The driver-role interceptor subset only (matches Program.cs's `if (hasDriver)` branch);
|
||||
// no ConfigServeAuthInterceptor/CentralCommandAuthInterceptor — those are admin-role.
|
||||
builder.Services.AddGrpc(o =>
|
||||
{
|
||||
o.Interceptors.Add<TelemetryStreamAuthInterceptor>();
|
||||
o.Interceptors.Add<MeshCommandAuthInterceptor>();
|
||||
});
|
||||
|
||||
_port = GetFreePort();
|
||||
builder.WebHost.ConfigureKestrel(k => k.ListenAnyIP(_port, o => o.Protocols = HttpProtocols.Http2));
|
||||
|
||||
_app = builder.Build();
|
||||
_hub = _app.Services.GetRequiredService<ITelemetryLocalHub>();
|
||||
|
||||
// hasAdmin: false — the driver-only shape. syncListenPort: 0 so LocalDbSync (admin-agnostic
|
||||
// but out of scope here) is not mapped; configServeGrpcPort: 0 because a driver-only node
|
||||
// never carries ConfigServe.
|
||||
_app.MapOtOpcUaMeshGrpcServices(
|
||||
hasAdmin: false, hasDriver: true,
|
||||
syncListenPort: 0, configServeGrpcPort: 0, telemetryListenPort: 1);
|
||||
|
||||
await _app.StartAsync(TestContext.Current.CancellationToken);
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
public async ValueTask DisposeAsync()
|
||||
{
|
||||
await _app.StopAsync(TimeSpan.FromSeconds(10));
|
||||
await _app.DisposeAsync();
|
||||
await _system.Terminate().WaitAsync(TimeSpan.FromSeconds(15));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void NoAuthorizationPolicyProviderIsRegistered()
|
||||
{
|
||||
// THE structural reason this shape is unaffected: with AddOtOpcUaAuth never called, there is
|
||||
// no FallbackPolicy to opt out of. If a future change registers authorization services on a
|
||||
// driver-only node, this goes red before any endpoint's behaviour does.
|
||||
_app.Services.GetService<IAuthorizationPolicyProvider>().ShouldBeNull();
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task CorrectlyKeyed_NodeCommandExecute_Succeeds()
|
||||
{
|
||||
var command = new RestartDriver("C1", "driver-1", "alice", Guid.NewGuid());
|
||||
var client = new NodeCommandService.NodeCommandServiceClient(Channel());
|
||||
var headers = new Metadata { { "authorization", $"Bearer {NodeCommandKey}" } };
|
||||
|
||||
await client.ExecuteAsync(
|
||||
MeshTransportDtoMapper.ToEnvelope(command), headers,
|
||||
cancellationToken: TestContext.Current.CancellationToken);
|
||||
|
||||
var received = await _nodeInbox.Task.WaitAsync(
|
||||
TimeSpan.FromSeconds(10), TestContext.Current.CancellationToken);
|
||||
received.ShouldBeOfType<RestartDriver>().DriverInstanceId.ShouldBe("driver-1");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task NoKey_NodeCommandExecute_FailsWithTheInterceptorsDetail()
|
||||
{
|
||||
var command = new RestartDriver("C1", "driver-1", "alice", Guid.NewGuid());
|
||||
var client = new NodeCommandService.NodeCommandServiceClient(Channel());
|
||||
|
||||
var ex = await Should.ThrowAsync<RpcException>(() => client.ExecuteAsync(
|
||||
MeshTransportDtoMapper.ToEnvelope(command), new Metadata(),
|
||||
cancellationToken: TestContext.Current.CancellationToken).ResponseAsync);
|
||||
|
||||
ex.StatusCode.ShouldBe(StatusCode.PermissionDenied);
|
||||
ex.Status.Detail.ShouldBe("Node command authentication failed.");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task CorrectlyKeyed_TelemetrySubscribe_Succeeds()
|
||||
{
|
||||
// Health is snapshot-style (ITelemetryLocalHub.Emit / class remarks): emitting BEFORE the
|
||||
// Subscribe call means the new subscriber's reader is primed with it immediately, with no
|
||||
// race against a live delta.
|
||||
_hub.Emit(new TelemetryItem.Health(new DriverHealthChanged(
|
||||
"C1", "driver-1", "Healthy", DateTime.UtcNow, null, 0, DateTime.UtcNow)));
|
||||
|
||||
var client = new TelemetryStreamService.TelemetryStreamServiceClient(Channel());
|
||||
var headers = new Metadata { { "authorization", $"Bearer {TelemetryKey}" } };
|
||||
|
||||
using var call = client.Subscribe(
|
||||
new TelemetryStreamRequest { CorrelationId = "r3-01-t6" },
|
||||
headers,
|
||||
cancellationToken: TestContext.Current.CancellationToken);
|
||||
|
||||
(await call.ResponseStream.MoveNext(TestContext.Current.CancellationToken)).ShouldBeTrue();
|
||||
call.ResponseStream.Current.DriverHealth.DriverInstanceId.ShouldBe("driver-1");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task NoKey_TelemetrySubscribe_FailsWithTheInterceptorsDetail()
|
||||
{
|
||||
var client = new TelemetryStreamService.TelemetryStreamServiceClient(Channel());
|
||||
|
||||
using var call = client.Subscribe(
|
||||
new TelemetryStreamRequest { CorrelationId = "r3-01-t6-nokey" },
|
||||
cancellationToken: TestContext.Current.CancellationToken);
|
||||
|
||||
var ex = await Should.ThrowAsync<RpcException>(
|
||||
() => call.ResponseStream.MoveNext(TestContext.Current.CancellationToken));
|
||||
|
||||
ex.StatusCode.ShouldBe(StatusCode.PermissionDenied);
|
||||
ex.Status.Detail.ShouldBe("Telemetry stream authentication failed.");
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------------------
|
||||
// harness
|
||||
// ---------------------------------------------------------------------------------------
|
||||
|
||||
private GrpcChannel Channel() => GrpcChannel.ForAddress($"http://localhost:{_port}");
|
||||
|
||||
private static int GetFreePort()
|
||||
{
|
||||
using var listener = new System.Net.Sockets.TcpListener(IPAddress.Loopback, 0);
|
||||
listener.Start();
|
||||
var port = ((IPEndPoint)listener.LocalEndpoint).Port;
|
||||
listener.Stop();
|
||||
return port;
|
||||
}
|
||||
|
||||
/// <summary>Completes an inbox with the first message it receives.</summary>
|
||||
private sealed class CaptureActor : ReceiveActor
|
||||
{
|
||||
public static Props Props(TaskCompletionSource<object> sink) =>
|
||||
Akka.Actor.Props.Create(() => new CaptureActor(sink));
|
||||
|
||||
public CaptureActor(TaskCompletionSource<object> sink) => ReceiveAny(m => sink.TrySetResult(m));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,66 @@
|
||||
using Shouldly;
|
||||
using Xunit;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.Host.IntegrationTests;
|
||||
|
||||
/// <summary>
|
||||
/// R3-01 T7 — a source-scan guard, following the solution-root-walk idiom
|
||||
/// <c>ZB.MOM.WW.OtOpcUa.Driver.S7.Cli.Tests.FlushLoggingConventionTests</c> uses. Complements the
|
||||
/// endpoint-metadata census (<c>MeshGrpcAnonymousMetadataTests</c>): the census catches a map
|
||||
/// added to <c>MeshGrpcEndpoints</c> without the <c>AllowAnonymous</c> convention, but it cannot
|
||||
/// catch a map added to <c>Program.cs</c> that bypasses the helper entirely — that shape never
|
||||
/// goes through <c>MapOtOpcUaMeshGrpcServices</c> at all, so the census would simply never see it.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// Would have FAILED at HEAD before T1: <c>Program.cs</c> used to carry the five map sites
|
||||
/// inline (<c>MapZbLocalDbSync()</c>, four <c>MapGrpcService<T>()</c> calls) with no
|
||||
/// <c>MapOtOpcUaMeshGrpcServices</c> call at all.
|
||||
/// </remarks>
|
||||
public sealed class MeshGrpcMapConventionTests
|
||||
{
|
||||
private static readonly string ProgramCsPath = LocateProgramCs();
|
||||
|
||||
[Fact]
|
||||
public void ProgramCs_ContainsNoDirectMeshGrpcMapSites_AndExactlyOneMapperCall()
|
||||
{
|
||||
var source = File.ReadAllText(ProgramCsPath);
|
||||
|
||||
// Guard-of-the-guard: a bad path (an empty file, or one that never had the mapper in the
|
||||
// first place) must not pass this test by finding nothing to count.
|
||||
source.ShouldNotBeNullOrWhiteSpace();
|
||||
source.ShouldContain("MapOtOpcUaMeshGrpcServices(");
|
||||
|
||||
CountOccurrences(source, "MapGrpcService<").ShouldBe(0,
|
||||
"every MapGrpcService<T>() call must live inside MeshGrpcEndpoints.MapOtOpcUaMeshGrpcServices, " +
|
||||
"not inline in Program.cs (03/S14 — a map added outside the helper is invisible to the " +
|
||||
"endpoint-metadata census).");
|
||||
CountOccurrences(source, "MapZbLocalDbSync(").ShouldBe(0,
|
||||
"the LocalDb sync map must live inside MeshGrpcEndpoints.MapOtOpcUaMeshGrpcServices, not inline " +
|
||||
"in Program.cs, for the same reason as MapGrpcService<T>() above.");
|
||||
CountOccurrences(source, "MapOtOpcUaMeshGrpcServices(").ShouldBe(1,
|
||||
"Program.cs must call the mapper exactly once; more than one call would map the mesh gRPC " +
|
||||
"services twice (a route conflict) and zero calls would drop them entirely.");
|
||||
}
|
||||
|
||||
private static int CountOccurrences(string source, string needle)
|
||||
{
|
||||
var count = 0;
|
||||
var index = 0;
|
||||
while ((index = source.IndexOf(needle, index, StringComparison.Ordinal)) >= 0)
|
||||
{
|
||||
count++;
|
||||
index += needle.Length;
|
||||
}
|
||||
|
||||
return count;
|
||||
}
|
||||
|
||||
private static string LocateProgramCs()
|
||||
{
|
||||
var dir = new DirectoryInfo(AppContext.BaseDirectory);
|
||||
while (dir is not null && !File.Exists(Path.Combine(dir.FullName, "ZB.MOM.WW.OtOpcUa.slnx")))
|
||||
dir = dir.Parent;
|
||||
dir.ShouldNotBeNull("Could not find solution root (ZB.MOM.WW.OtOpcUa.slnx).");
|
||||
return Path.Combine(dir!.FullName, "src", "Server", "ZB.MOM.WW.OtOpcUa.Host", "Program.cs");
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user