refactor(acl): delete the LDAP data-plane authenticator and its tests (#520)
Stage 2b. LdapOpcUaUserAuthenticator was left registered nowhere by stage 2a; this removes it, its three test files, and the config surface that existed only to serve it. Deleted: LdapOpcUaUserAuthenticator, LdapOpcUaUserAuthenticatorTests, LdapAuthResilienceTests, LdapGlauthOutageLiveTests (19 tests). Also removed four Security:Ldap keys — AuthTimeoutMs, MaxConcurrentAuthentications, OutageFailureThreshold, OutageCooldownSeconds — plus their validator rules and binding/validator test coverage. Verified first that the ONLY consumers were the deleted class and the validator rules asserting them; none appear in any checked-in appsettings. Leaving them would have created exactly the dead-config surface #521/#524/#525 were filed to clear up. Worth stating why that resilience machinery does not need relocating rather than just deleting. It existed because the OPC UA SDK raises impersonation SYNCHRONOUSLY under a SessionManager-wide event lock, so a hung LDAP bind there serialized every session activation server-wide — Anonymous and X509 included. Key verification is one indexed lookup plus one HMAC with no network I/O, so the hazard the wall-clock bound, concurrency cap and outage circuit guarded against no longer exists on that path. LDAP now runs only on AdminUI login, an ordinary async request path where a slow directory affects the one request making it. ConnectionTimeoutMs is RETAINED: it is projected into the shared library and is not authenticator-specific. Docs corrected rather than left to rot: - CLAUDE.md's "LDAP Authentication" section described the deleted file as the data-plane authenticator. Rewritten, including a warning not to "restore" it on seeing the LDAP wiring that AdminUI legitimately still needs. - infra/README §5b (the GLAuth outage live gate) marked RETIRED with the reason, rather than silently dropped — the harness GLAuth is still used by OTOPCUA_HARNESS_USE_LDAP=1 and is unaffected. - docs/security.md's five-key resilience table cut to the one surviving key, and a new API-key identity section added. Historical archreview/ records are left as-is; they are dated accounts. Host.IntegrationTests 211 tests (was 230, i.e. exactly the 19 removed) with the same 3 pre-existing failures verified against clean master. Host 37, Security 89. Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
This commit is contained in:
Joseph Doherty
@@ -274,7 +274,7 @@ central SQL Server.
|
||||
|
||||
> **Infra index:** [`infra/README.md`](infra/README.md) is the canonical index of all testing +
|
||||
> deployment infrastructure — the Docker host layout, driver fixtures, the env-gated **live-gate**
|
||||
> recipes (S7 blackhole, GLAuth outage, HistorianGateway LiveIntegration), the integration-test
|
||||
> recipes (S7 blackhole, HistorianGateway LiveIntegration), the integration-test
|
||||
> harness, the docker-dev rig, and the deploy setup. Start there; the sections below + `docs/v2/dev-environment.md` carry the detail.
|
||||
|
||||
> **Migrated 2026-04-28**: Docker config + host moved off this dev VM (DESKTOP-6JL3KKO) onto the shared Linux Docker host (`DOCKER`, 10.100.0.35) so the dev VM could shed WSL2/Hyper-V and have its GPU re-attached via ESXi passthrough. Docker Desktop is no longer installed here. All checked-in `appsettings.json` defaults, fixture-class default endpoints, and `e2e-config.sample.json` were rewritten to target `10.100.0.35`. See `docs/v2/dev-environment.md` for the full rewrite (header dated 2026-04-28).
|
||||
@@ -591,9 +591,39 @@ Merged to master (`26fad75c`); **live gate PASSED** (`1281aebf` — driver nodes
|
||||
Task 9 landed (`3a590a0c`). See `docs/plans/2026-07-23-mesh-phase4-cut-driver-configdb.md` and
|
||||
`docs/plans/2026-07-22-per-cluster-mesh-program.md` §Phase 4.
|
||||
|
||||
## LDAP Authentication
|
||||
## Authentication: API keys on the data plane, LDAP on AdminUI (#520)
|
||||
|
||||
The server uses LDAP-based user authentication via the `Security:Ldap` section in `appsettings.json`. When enabled, credentials are validated by LDAP bind against a GLAuth server, and LDAP group membership maps to OPC UA permissions: `ReadOnly` (browse/read), `WriteOperate` (write FreeAccess/Operate attributes), `WriteTune` (write Tune attributes), `WriteConfigure` (write Configure attributes), `AlarmAck` (alarm acknowledgment). `LdapOpcUaUserAuthenticator` (`src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/LdapOpcUaUserAuthenticator.cs`) implements `IOpcUaUserAuthenticator`, delegating the LDAP bind + group lookup to `OtOpcUaLdapAuthService` (`src/Server/ZB.MOM.WW.OtOpcUa.Security/Ldap/OtOpcUaLdapAuthService.cs`, an `ILdapAuthService`). See `docs/security.md` for the full guide.
|
||||
⚠️ **Changed 2026-07-31. The OPC UA data plane no longer uses LDAP at all.** It is machine-to-machine,
|
||||
so a client presents an OPC UA **UserName** token with the fixed username **`apikey`** and an
|
||||
`otua_<keyId>_<secret>` credential as the password. `ApiKeyOpcUaUserAuthenticator`
|
||||
(`src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/`) implements `IOpcUaUserAuthenticator`: one indexed lookup
|
||||
on `keyId` plus one peppered HMAC-SHA256, `FixedTimeEquals`, **no network I/O**. The key resolves to a
|
||||
**principal**; `NodeAcl.Principal` rows grant it permissions. `LdapOpcUaUserAuthenticator` is
|
||||
**deleted**, along with four `Security:Ldap` keys that existed only for it (`AuthTimeoutMs`,
|
||||
`MaxConcurrentAuthentications`, `OutageFailureThreshold`, `OutageCooldownSeconds`) — they bounded an
|
||||
LDAP bind running inline in the SDK's synchronous, `SessionManager`-lock-holding impersonation
|
||||
callback, and nothing on the remaining LDAP path has that property.
|
||||
|
||||
**LDAP remains, for AdminUI login only** (`Security:Ldap`, `OtOpcUaLdapAuthService` →
|
||||
`ILdapAuthService`, group→role mapping via `GroupToRole` + `LdapGroupRoleMapping`). Do not "restore"
|
||||
the data-plane authenticator when you see LDAP wiring in `Program.cs` — it is there for the AdminUI.
|
||||
|
||||
Two things to know before relying on any of this:
|
||||
|
||||
- ⚠️ **Node ACLs are still not enforced per node.** A key's grants collapse into the two existing
|
||||
server-wide role gates (`WriteOperate` on Value writes, `AlarmAck` on Part 9 methods), and
|
||||
**only `Cluster`-scoped grants count** — `ApiKeyInterimRoleMapper` deliberately under-grants, because
|
||||
honouring a narrower grant's flags while unable to honour its *scope* would turn "WriteOperate on
|
||||
Equipment X" into server-wide write authority. Read, Browse, HistoryRead and CreateMonitoredItems
|
||||
remain ungated for any admitted session, **including Anonymous**.
|
||||
- ⚠️ **A driver-only node has no key source yet.** `ConfigDbApiKeyStore` needs a ConfigDb, which Phase 4
|
||||
removed from driver-only nodes; until the artifact-backed store lands they register
|
||||
`NullOpcUaUserAuthenticator` and deny every UserName token. Note `OtOpcUaServerHostedService` takes
|
||||
`IOpcUaUserAuthenticator` as a **required** ctor parameter — leaving it unregistered fails DI and the
|
||||
node does not boot, so the deny-all must be registered explicitly.
|
||||
|
||||
Remaining stages, and the rationale for every decision above, are in
|
||||
`docs/plans/2026-07-31-apikey-acl-enforcement-design.md`. See `docs/security.md` for the full guide.
|
||||
|
||||
Dev/test LDAP is the **shared GLAuth** running on the Linux Docker host at `10.100.0.35:3893` (baseDN `dc=zb,dc=local`, plaintext/`Transport=None`). It is managed via `scadaproj/infra/glauth/` (source of truth + deploy runbook). Single bind account `cn=serviceaccount,dc=zb,dc=local` / `serviceaccount123`; all test users password `password`. The docker-dev compose binds this shared instance directly — `DevStubMode` is no longer used. The per-VM NSSM GLAuth at `C:\publish\glauth\` and the old base DNs `dc=lmxopcua,dc=local` / `dc=otopcua,dc=local` are obsolete. (The integration-test harness under `tests/.../Host.IntegrationTests/` runs its own ephemeral **GLAuth** container on port 3894 — seeded by `tests/.../Host.IntegrationTests/glauth/config.toml` with dev users alice/bob + the `cn=serviceaccount` bind account — for the opt-in `OTOPCUA_HARNESS_USE_LDAP=1` real-bind mode; distinct from the shared dev GLAuth on 3893. It replaced the retired `bitnami/openldap:2.6` in PR #451, 2026-07-15.)
|
||||
|
||||
|
||||
+56
-8
@@ -162,19 +162,67 @@ LDAP is configured under the `Security:Ldap` section (bound to `LdapOptions`, `s
|
||||
|
||||
`GroupToRole` maps LDAP group names → roles (case-insensitive); a user gets every role whose source group is in their membership. The values are the canonical control-plane role strings (`Viewer` / `Designer` / `Administrator`, plus the appsettings-only `Operator` for the `DriverOperator` policy); the same map also supplies data-plane role strings (`ReadOnly`, `WriteOperate`, `WriteTune`, `WriteConfigure`, `AlarmAck`) — see [Role grant source (data-plane)](#role-grant-source-data-plane) below. `UserNameAttribute: "sAMAccountName"` is the critical AD override — the GLAuth dev default is `cn`, which is not how AD users are looked up; use `userPrincipalName` instead if operators log in with `user@corp.example.com` form. `LdapOptionsValidator` (`src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/LdapOptionsValidator.cs`) fails startup when `Transport = None` and `AllowInsecure = false` on a real-LDAP (non-DevStub) config.
|
||||
|
||||
### LDAP resilience — timeout, concurrency, outage circuit (arch-review 03/S2)
|
||||
### API-key identity (OPC UA data plane)
|
||||
|
||||
The OPC UA SDK raises its impersonation (UserName) callback **synchronously**, inside a `SessionManager`-wide event lock (verified against `OPCFoundation.NetStandard.Opc.Ua.Server` 1.5.378). A slow or hung LDAP bind inside that callback therefore does not stall one SDK thread — it **serializes every session activation server-wide** (Anonymous and X509 included). Since the callback is irreducibly synchronous, `LdapOpcUaUserAuthenticator` bounds it at its own boundary with five `Security:Ldap` keys (all validated at startup by `LdapOptionsValidator` when `Enabled && !DevStubMode`):
|
||||
The OPC UA data plane is **machine-to-machine**, so a consuming system authenticates with an API key
|
||||
rather than an LDAP user (Gitea #520). The client presents an OPC UA **UserName** token whose username
|
||||
is the fixed literal `apikey` and whose password is the key:
|
||||
|
||||
```
|
||||
otua_<keyId>_<secret>
|
||||
```
|
||||
|
||||
`ApiKeyOpcUaUserAuthenticator` verifies it — one indexed lookup on `keyId` plus one peppered
|
||||
HMAC-SHA256 over the secret, compared with `CryptographicOperations.FixedTimeEquals`. No network I/O.
|
||||
The key resolves to a **principal**, and `NodeAcl` rows naming that principal are what grant it
|
||||
permissions.
|
||||
|
||||
| Key | Notes |
|
||||
|---|---|
|
||||
| `Security:ApiKeys:Username` | The static username clients present. Default `apikey`. Checked rather than ignored, so a typo is an explicit denial. |
|
||||
| `Security:ApiKeys:Pepper` | Server-side pepper for the stored hashes. **Supply via `Security__ApiKeys__Pepper` or `${secret:}` — never commit it.** Rotating it invalidates every existing key. Has no default: an empty pepper still hashes "successfully", so `ApiKeySecret.ComputeHash` throws on one rather than producing a working-looking deployment with no protection. |
|
||||
|
||||
Secrets are never stored — only the hash. The plaintext is shown **once** at creation and is
|
||||
unrecoverable afterwards.
|
||||
|
||||
**Every rejection returns the same opaque message** ("Invalid API key"), with the real reason logged
|
||||
server-side only, so the server cannot be used as an oracle for whether a key id exists or which half
|
||||
of a credential was wrong.
|
||||
|
||||
⚠️ **Revocation requires a deploy.** Per-cluster mesh Phase 4 leaves driver-only nodes with no
|
||||
ConfigDb, so keys reach them inside the deployment artifact exactly as `NodeAcl` rows do. Clearing
|
||||
`Enabled` takes effect on a node only once a new artifact reaches it.
|
||||
|
||||
⚠️ **Node ACLs are still not enforced per node.** Until the enforcement stages land, a key's grants are
|
||||
collapsed into the two existing server-wide role gates, and **only `Cluster`-scoped grants count** —
|
||||
a narrower grant confers nothing rather than silently widening to server-wide authority. Reads,
|
||||
browses, subscriptions and history remain ungated for any admitted session. See
|
||||
`docs/plans/2026-07-31-apikey-acl-enforcement-design.md`.
|
||||
|
||||
### LDAP resilience — mostly retired with the data-plane authenticator (#520)
|
||||
|
||||
> **Changed 2026-07-31.** The OPC UA **data plane** no longer uses LDAP at all — it authenticates with
|
||||
> API keys (see [API-key identity](#api-key-identity-opc-ua-data-plane)). `LdapOpcUaUserAuthenticator`
|
||||
> is deleted, and with it four of the five `Security:Ldap` resilience keys: **`AuthTimeoutMs`**,
|
||||
> **`MaxConcurrentAuthentications`**, **`OutageFailureThreshold`** and **`OutageCooldownSeconds`**.
|
||||
> Setting them now does nothing (options binding ignores unknown keys); remove them from any config
|
||||
> that carries them.
|
||||
|
||||
Those four existed for one specific reason. The OPC UA SDK raises its impersonation callback
|
||||
**synchronously**, inside a `SessionManager`-wide event lock (verified against
|
||||
`OPCFoundation.NetStandard.Opc.Ua.Server` 1.5.378), so a slow or hung LDAP bind in that callback did
|
||||
not stall one SDK thread — it **serialized every session activation server-wide**, Anonymous and X509
|
||||
included. That constraint is gone: key verification is one indexed lookup plus one HMAC, with no
|
||||
network I/O on the path.
|
||||
|
||||
LDAP now runs only on **AdminUI login**, an ordinary async request path where a slow directory affects
|
||||
the one request making it. One key remains:
|
||||
|
||||
| Key | Default | Bounds |
|
||||
|---|---|---|
|
||||
| `ConnectionTimeoutMs` | `10000` | Projected into the shared library's `ConnectionTimeoutMs` — bounds the socket connect **and** each per-operation search. First line of defence against an outage. |
|
||||
| `AuthTimeoutMs` | `15000` | Hard whole-flow wall-clock bound (bind **and** role-map) enforced at the authenticator boundary; a slow/hung flow denies **"Authentication timed out"** (fail-closed) and releases the SDK thread. Deliberately `> ConnectionTimeoutMs` so a healthy-but-slow flow is not cut off. |
|
||||
| `MaxConcurrentAuthentications` | `8` | Max in-flight authentications; excess denies **"Authentication backend busy"** (local backpressure — does **not** feed the circuit). Bounds accumulation of orphaned/timed-out binds during an outage. |
|
||||
| `OutageFailureThreshold` | `3` | Consecutive **system-side** failures (boundary timeout, directory-unreachable, or unexpected throw) that open the outage circuit. Any success or user-credential deny resets the streak. `0` disables the circuit. |
|
||||
| `OutageCooldownSeconds` | `15` | How long the open circuit fast-denies **"Authentication backend unavailable"** without touching LDAP; after cooldown the next call half-open-probes, and one more system failure re-opens immediately. |
|
||||
| `ConnectionTimeoutMs` | `10000` | Projected into the shared library's `ConnectionTimeoutMs` — bounds the socket connect **and** each per-operation search. Validated `> 0` at startup by `LdapOptionsValidator`. |
|
||||
|
||||
|
||||
Defaults are behaviour-neutral on a healthy directory. All new paths are **fail-closed** (they deny). A user-credential deny (wrong password) is fast when the directory is up and never trips the circuit; only a directory *outage* opens it.
|
||||
|
||||
---
|
||||
|
||||
|
||||
+9
-10
@@ -135,17 +135,16 @@ export S7_TIMEOUT_OUTAGE_STOP_CMD='ssh dohertj2@10.100.0.35 "docker unpause otop
|
||||
dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.S7.IntegrationTests --filter "FullyQualifiedName~ConnectTimeoutOutage"
|
||||
```
|
||||
|
||||
### 5b. LDAP GLAuth outage/recovery (`LdapGlauthOutageLiveTests`)
|
||||
Drives the directory-outage circuit through the production auth path against the harness GLAuth
|
||||
(§4), paused/unpaused. Also verifies the GLAuth swap binds correctly.
|
||||
### 5b. LDAP GLAuth outage/recovery — **RETIRED 2026-07-31 (#520)**
|
||||
|
||||
```bash
|
||||
ssh dohertj2@10.100.0.35 'cd ~/otopcua-harness && docker compose up -d ldap'
|
||||
export GLAUTH_LIVE_HOST=10.100.0.35 # port defaults to 3894 (GLAUTH_LIVE_PORT)
|
||||
export GLAUTH_OUTAGE_START_CMD='ssh dohertj2@10.100.0.35 "docker pause otopcua-harness-ldap-1"'
|
||||
export GLAUTH_OUTAGE_STOP_CMD='ssh dohertj2@10.100.0.35 "docker unpause otopcua-harness-ldap-1"'
|
||||
dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests --filter "FullyQualifiedName~LdapGlauthOutageLive"
|
||||
```
|
||||
`LdapGlauthOutageLiveTests` and the directory-outage circuit it drove were deleted with
|
||||
`LdapOpcUaUserAuthenticator`. The OPC UA data plane authenticates with **API keys**, not LDAP, so no
|
||||
LDAP bind runs inside the SDK's synchronous impersonation callback any more — which is the only
|
||||
reason that circuit, its wall-clock bound, and its concurrency cap existed. LDAP remains only on the
|
||||
AdminUI login path, where a slow directory stalls one request rather than serializing every OPC UA
|
||||
session activation.
|
||||
|
||||
The harness GLAuth (§4) is still used by `OTOPCUA_HARNESS_USE_LDAP=1` and is unaffected.
|
||||
|
||||
### 5c. HistorianGateway LiveIntegration soak (`Category=LiveIntegration`) — needs VPN
|
||||
Exercises the gateway read/write/alarm/EnsureTags paths against a **real AVEVA historian**. Requires
|
||||
|
||||
@@ -48,17 +48,12 @@ public sealed class LdapOptionsValidator : OptionsValidatorBase<LdapOptions>
|
||||
!(options.Transport == LdapTransport.None && !options.AllowInsecure),
|
||||
"LDAP transport is None (plaintext) but AllowInsecure is false — set Transport to Ldaps/StartTls or set AllowInsecure for dev.");
|
||||
|
||||
// S2 resilience knobs — a non-positive timeout / concurrency bound would silently disable the
|
||||
// wall-clock/concurrency protection the OPC UA data-plane authenticator relies on, so fail fast.
|
||||
// A non-positive connect/search timeout would leave the shared library unbounded against a hung
|
||||
// directory, so fail fast. The other four S2 knobs validated here (AuthTimeoutMs,
|
||||
// MaxConcurrentAuthentications, OutageFailureThreshold, OutageCooldownSeconds) went with
|
||||
// LdapOpcUaUserAuthenticator in #520 — they guarded an LDAP bind running inline on OPC UA
|
||||
// session activation, and no remaining LDAP consumer runs on that path.
|
||||
builder.RequireThat(options.ConnectionTimeoutMs > 0,
|
||||
"Ldap:ConnectionTimeoutMs must be > 0.");
|
||||
builder.RequireThat(options.AuthTimeoutMs > 0,
|
||||
"Ldap:AuthTimeoutMs must be > 0.");
|
||||
builder.RequireThat(options.MaxConcurrentAuthentications > 0,
|
||||
"Ldap:MaxConcurrentAuthentications must be > 0.");
|
||||
// An open circuit with no cooldown would re-probe LDAP on every call — pointless. Only enforce
|
||||
// a positive cooldown when the circuit is actually enabled (OutageFailureThreshold > 0; 0 disables).
|
||||
builder.RequireThat(options.OutageFailureThreshold <= 0 || options.OutageCooldownSeconds > 0,
|
||||
"Ldap:OutageCooldownSeconds must be > 0 when Ldap:OutageFailureThreshold > 0 (0 disables the circuit).");
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,260 +0,0 @@
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
using ZB.MOM.WW.Auth.Abstractions.Roles;
|
||||
using ZB.MOM.WW.OtOpcUa.OpcUaServer.Security;
|
||||
using ZB.MOM.WW.OtOpcUa.Security.Ldap;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.Host.OpcUa;
|
||||
|
||||
/// <summary>
|
||||
/// Production <see cref="IOpcUaUserAuthenticator"/> adapter that bridges OPC UA UserName
|
||||
/// tokens to the same <see cref="ILdapAuthService"/> the Admin UI cookie/JWT flows use, so a
|
||||
/// single LDAP source-of-truth governs both control-plane (Admin) and data-plane (OPC UA)
|
||||
/// session identities. Roles are resolved through the shared
|
||||
/// <see cref="IGroupRoleMapper{TRole}"/> seam from the LDAP groups returned by the directory —
|
||||
/// the same seam the login endpoint uses — and the resolved set is attached to the OPC UA
|
||||
/// session identity for the downstream data-plane ACL evaluator.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// This authenticator is registered as a singleton, but <see cref="IGroupRoleMapper{TRole}"/>
|
||||
/// (and its DbContext-backed mapping service) is scoped. A per-call DI scope is opened to
|
||||
/// resolve the mapper so the singleton never captures a scoped dependency.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>Resilience (arch-review 03/S2).</b> The OPC UA SDK raises the impersonation callback that
|
||||
/// reaches here <em>synchronously</em>, inside a <c>SessionManager</c>-wide event lock (verified
|
||||
/// against <c>OPCFoundation.NetStandard.Opc.Ua.Server</c> 1.5.378 —
|
||||
/// <c>SessionManager.ActivateSessionAsync</c> holds <c>m_eventLock</c> while invoking the
|
||||
/// <c>void</c> impersonation delegate). A slow / hung LDAP bind inside that callback therefore does
|
||||
/// not merely stall one SDK thread — it serializes <em>every</em> session activation server-wide
|
||||
/// (Anonymous and X509 included). The callback contract is irreducibly synchronous (there is no
|
||||
/// async impersonation callback), so the fix lives entirely at this boundary: a hard whole-flow
|
||||
/// wall-clock bound (<see cref="LdapOptions.AuthTimeoutMs"/>) and a bounded in-flight concurrency cap
|
||||
/// (<see cref="LdapOptions.MaxConcurrentAuthentications"/>), both fail-closed. All new paths deny.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
public sealed class LdapOpcUaUserAuthenticator : IOpcUaUserAuthenticator
|
||||
{
|
||||
private readonly ILdapAuthService _ldap;
|
||||
private readonly IServiceScopeFactory _scopeFactory;
|
||||
private readonly ILogger<LdapOpcUaUserAuthenticator> _logger;
|
||||
private readonly LdapOptions _options;
|
||||
|
||||
/// <summary>Bounds in-flight authentication flows. A boundary timeout abandons its core task, which
|
||||
/// keeps holding its slot until it actually completes — so an outage cannot accumulate unbounded
|
||||
/// orphan binds.</summary>
|
||||
private readonly SemaphoreSlim _gate;
|
||||
|
||||
/// <summary>Guards the outage-circuit state (<see cref="_consecutiveSystemFailures"/> +
|
||||
/// <see cref="_circuitOpenUntilTicks"/>).</summary>
|
||||
private readonly object _circuitLock = new();
|
||||
|
||||
/// <summary>Count of CONSECUTIVE system-side failures; reset by any success / user-side deny.</summary>
|
||||
private int _consecutiveSystemFailures;
|
||||
|
||||
/// <summary><see cref="Environment.TickCount64"/> the circuit stays open until; <c>0</c> = closed.</summary>
|
||||
private long _circuitOpenUntilTicks;
|
||||
|
||||
/// <summary>Constructs the authenticator from its dependencies and the bound LDAP options.</summary>
|
||||
/// <param name="ldap">The app LDAP auth service (shared with the Admin UI login flow).</param>
|
||||
/// <param name="scopeFactory">Opens a per-call DI scope to resolve the scoped role mapper.</param>
|
||||
/// <param name="options">The bound <see cref="LdapOptions"/> (timeout + concurrency knobs).</param>
|
||||
/// <param name="logger">The logger.</param>
|
||||
public LdapOpcUaUserAuthenticator(
|
||||
ILdapAuthService ldap,
|
||||
IServiceScopeFactory scopeFactory,
|
||||
IOptions<LdapOptions> options,
|
||||
ILogger<LdapOpcUaUserAuthenticator> logger)
|
||||
{
|
||||
_ldap = ldap;
|
||||
_scopeFactory = scopeFactory;
|
||||
_options = options.Value;
|
||||
_logger = logger;
|
||||
_gate = new SemaphoreSlim(Math.Max(1, _options.MaxConcurrentAuthentications));
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
public async Task<OpcUaUserAuthResult> AuthenticateUserNameAsync(string username, string password, CancellationToken ct)
|
||||
{
|
||||
var authTimeout = TimeSpan.FromMilliseconds(Math.Max(1, _options.AuthTimeoutMs));
|
||||
|
||||
// Directory-outage circuit: after a run of consecutive system-side failures, fail fast WITHOUT
|
||||
// touching LDAP for the cooldown window — so a sustained outage cannot serialize every session
|
||||
// activation behind a per-call stall (the SDK raises this callback under a server-wide lock).
|
||||
if (IsCircuitOpen())
|
||||
{
|
||||
_logger.LogDebug(
|
||||
"LDAP outage circuit open — fast-denying OPC UA user {User} without a directory call", username);
|
||||
return OpcUaUserAuthResult.Deny("Authentication backend unavailable");
|
||||
}
|
||||
|
||||
// Bounded in-flight concurrency: acquire a slot within our own wall-clock budget. All slots
|
||||
// busy is LOCAL backpressure, not directory evidence — deny "busy" (fail-closed) and do NOT
|
||||
// feed the outage circuit.
|
||||
if (!await _gate.WaitAsync(authTimeout, ct).ConfigureAwait(false))
|
||||
{
|
||||
_logger.LogWarning(
|
||||
"LDAP authentication backend busy (concurrency cap {Cap}) — denying OPC UA user {User}",
|
||||
_options.MaxConcurrentAuthentications, username);
|
||||
return OpcUaUserAuthResult.Deny("Authentication backend busy");
|
||||
}
|
||||
|
||||
// Release the slot when the CORE task completes — deliberately NOT in a finally here. A boundary
|
||||
// timeout below abandons the core task (it runs to completion in the background; the underlying
|
||||
// library is documented never-throw, so there are no unobserved exceptions), and it must keep
|
||||
// counting against the cap until it finishes. The point of the boundary is to release the SDK
|
||||
// thread (and m_eventLock), not to cancel the bind.
|
||||
var core = AuthenticateCoreAsync(username, password, ct);
|
||||
_ = core.ContinueWith(
|
||||
static (_, state) => ((SemaphoreSlim)state!).Release(),
|
||||
_gate,
|
||||
CancellationToken.None,
|
||||
TaskContinuationOptions.None,
|
||||
TaskScheduler.Default);
|
||||
|
||||
try
|
||||
{
|
||||
var outcome = await core.WaitAsync(authTimeout, ct).ConfigureAwait(false);
|
||||
RecordOutcome(outcome.SystemFailure, username);
|
||||
return outcome.Result;
|
||||
}
|
||||
catch (TimeoutException)
|
||||
{
|
||||
// A boundary timeout is directory evidence — it feeds the circuit.
|
||||
RecordOutcome(systemFailure: true, username);
|
||||
_logger.LogWarning(
|
||||
"LDAP authentication timed out after {Timeout} for OPC UA user {User} — denying (fail-closed)",
|
||||
authTimeout, username);
|
||||
return OpcUaUserAuthResult.Deny("Authentication timed out");
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Whether the outage circuit is currently open (fast-deny window active). Always false when
|
||||
/// the circuit is disabled (<see cref="LdapOptions.OutageFailureThreshold"/> ≤ 0).</summary>
|
||||
private bool IsCircuitOpen()
|
||||
{
|
||||
if (_options.OutageFailureThreshold <= 0) return false;
|
||||
lock (_circuitLock)
|
||||
{
|
||||
return _circuitOpenUntilTicks > Environment.TickCount64;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Updates the outage-circuit state from one authentication outcome. A system-side failure advances
|
||||
/// the consecutive-failure streak and opens the circuit once it reaches
|
||||
/// <see cref="LdapOptions.OutageFailureThreshold"/>; any non-system outcome (success or user-side
|
||||
/// deny) resets the streak and closes the circuit. Half-open behaviour is implicit: once the cooldown
|
||||
/// elapses <see cref="IsCircuitOpen"/> lets the next call through with the streak still at threshold,
|
||||
/// so a single further system failure re-opens immediately.
|
||||
/// </summary>
|
||||
/// <param name="systemFailure">Whether this outcome was a system-side (directory) failure.</param>
|
||||
/// <param name="username">The login name, for diagnostics.</param>
|
||||
private void RecordOutcome(bool systemFailure, string username)
|
||||
{
|
||||
if (_options.OutageFailureThreshold <= 0) return; // circuit disabled
|
||||
|
||||
lock (_circuitLock)
|
||||
{
|
||||
if (systemFailure)
|
||||
{
|
||||
_consecutiveSystemFailures++;
|
||||
if (_consecutiveSystemFailures >= _options.OutageFailureThreshold)
|
||||
{
|
||||
var wasOpen = _circuitOpenUntilTicks > Environment.TickCount64;
|
||||
_circuitOpenUntilTicks =
|
||||
Environment.TickCount64 + (long)Math.Max(1, _options.OutageCooldownSeconds) * 1000L;
|
||||
if (!wasOpen)
|
||||
{
|
||||
_logger.LogWarning(
|
||||
"LDAP outage circuit OPENED after {Failures} consecutive system-side failures — " +
|
||||
"fast-denying for {Cooldown}s (last user {User})",
|
||||
_consecutiveSystemFailures, _options.OutageCooldownSeconds, username);
|
||||
}
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
if (_consecutiveSystemFailures >= _options.OutageFailureThreshold)
|
||||
{
|
||||
_logger.LogInformation(
|
||||
"LDAP outage circuit CLOSED — directory recovered (probe succeeded for {User})", username);
|
||||
}
|
||||
_consecutiveSystemFailures = 0;
|
||||
_circuitOpenUntilTicks = 0;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// The unbounded core authenticate flow (bind + role-map) that the outer
|
||||
/// <see cref="AuthenticateUserNameAsync"/> wraps in a wall-clock bound + concurrency cap. Fail-closed:
|
||||
/// any backend throw becomes a denial rather than escaping into the SDK. The returned
|
||||
/// <see cref="CoreOutcome.SystemFailure"/> flags directory-side faults (a system-side
|
||||
/// <see cref="LdapAuthResult.IsSystemFailure"/> deny, or an unexpected throw) so the caller can feed
|
||||
/// the outage circuit; a user-credential deny is NOT a system failure.
|
||||
/// </summary>
|
||||
/// <param name="username">The username to authenticate.</param>
|
||||
/// <param name="password">The cleartext password.</param>
|
||||
/// <param name="ct">Cancellation token.</param>
|
||||
private async Task<CoreOutcome> AuthenticateCoreAsync(string username, string password, CancellationToken ct)
|
||||
{
|
||||
try
|
||||
{
|
||||
var result = await _ldap.AuthenticateAsync(username, password, ct).ConfigureAwait(false);
|
||||
if (!result.Success)
|
||||
{
|
||||
return new CoreOutcome(
|
||||
OpcUaUserAuthResult.Deny(result.Error ?? "Invalid credentials"), result.IsSystemFailure);
|
||||
}
|
||||
|
||||
var roles = await ResolveRolesAsync(result.Groups, result.Roles, username, ct).ConfigureAwait(false);
|
||||
return new CoreOutcome(OpcUaUserAuthResult.Allow(result.DisplayName ?? username, roles), SystemFailure: false);
|
||||
}
|
||||
catch (Exception ex) when (ex is not OperationCanceledException)
|
||||
{
|
||||
_logger.LogWarning(ex, "LDAP authentication threw for OPC UA user {User}", username);
|
||||
return new CoreOutcome(OpcUaUserAuthResult.Deny("Authentication backend error"), SystemFailure: true);
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>The result of the core authenticate flow plus whether it was a system-side (directory)
|
||||
/// failure — the signal that drives the outage circuit.</summary>
|
||||
/// <param name="Result">The auth result to return to the SDK.</param>
|
||||
/// <param name="SystemFailure">Whether the outcome was a directory-side fault.</param>
|
||||
private readonly record struct CoreOutcome(OpcUaUserAuthResult Result, bool SystemFailure);
|
||||
|
||||
/// <summary>
|
||||
/// Resolves the user's roles from their LDAP groups via the scoped
|
||||
/// <see cref="IGroupRoleMapper{TRole}"/>, unioned with any pre-resolved roles (the DevStub
|
||||
/// FleetAdmin grant). A mapper fault (e.g. a DB outage) must not deny an otherwise-authenticated
|
||||
/// session: it falls back to the pre-resolved roles, matching the login endpoint's behaviour.
|
||||
/// </summary>
|
||||
/// <param name="groups">The LDAP groups returned by the directory.</param>
|
||||
/// <param name="preResolved">Pre-resolved roles (empty on the real path; FleetAdmin under DevStub).</param>
|
||||
/// <param name="username">The login name, for diagnostics.</param>
|
||||
/// <param name="ct">Cancellation token.</param>
|
||||
private async Task<IReadOnlyList<string>> ResolveRolesAsync(
|
||||
IReadOnlyList<string> groups, IReadOnlyList<string> preResolved, string username, CancellationToken ct)
|
||||
{
|
||||
try
|
||||
{
|
||||
await using var scope = _scopeFactory.CreateAsyncScope();
|
||||
var mapper = scope.ServiceProvider.GetRequiredService<IGroupRoleMapper<string>>();
|
||||
var mapping = await mapper.MapAsync(groups, ct).ConfigureAwait(false);
|
||||
|
||||
var roles = new HashSet<string>(preResolved, StringComparer.OrdinalIgnoreCase);
|
||||
foreach (var role in mapping.Roles)
|
||||
roles.Add(role);
|
||||
return [.. roles];
|
||||
}
|
||||
catch (Exception ex) when (ex is not OperationCanceledException)
|
||||
{
|
||||
_logger.LogWarning(ex,
|
||||
"Role-map lookup failed for OPC UA user {User}; using pre-resolved baseline roles", username);
|
||||
return preResolved;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -246,7 +246,9 @@ public sealed class OpcUaApplicationHost : IAsyncDisposable
|
||||
/// (<c>GetAwaiter().GetResult()</c> under <see cref="CancellationToken.None"/>) is irreducible: the
|
||||
/// callback must complete inline. Because the SDK gives no timeout, the hard wall-clock bound + bounded
|
||||
/// concurrency + directory-outage circuit that keep a slow/hung directory from wedging the SDK live
|
||||
/// inside the authenticator (<c>LdapOpcUaUserAuthenticator</c>, arch-review 03/S2) — NOT here.
|
||||
/// inside the authenticator — NOT here. (#520 replaced the LDAP authenticator with
|
||||
/// <c>ApiKeyOpcUaUserAuthenticator</c>, whose verification does no network I/O at all, so the
|
||||
/// bound/concurrency/circuit machinery that used to live there is gone rather than relocated.)
|
||||
/// </remarks>
|
||||
/// <param name="authenticator">The user authenticator to validate credentials.</param>
|
||||
/// <param name="args">The impersonation event arguments to process.</param>
|
||||
|
||||
@@ -88,44 +88,18 @@ public sealed class LdapOptions
|
||||
/// per-operation search inside a single bind flow. Projected into the library's
|
||||
/// <see cref="LibLdapOptions.ConnectionTimeoutMs"/> via <see cref="ToLibraryOptions"/>. The
|
||||
/// default (10000) matches the library default, so behaviour is unchanged unless configured.
|
||||
/// This is the FIRST line of defence against a directory outage; the whole-flow
|
||||
/// <see cref="AuthTimeoutMs"/> is the backstop (03/S2).
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// This is now the ONLY directory-outage bound. Its former backstops — <c>AuthTimeoutMs</c>,
|
||||
/// <c>MaxConcurrentAuthentications</c>, <c>OutageFailureThreshold</c> and
|
||||
/// <c>OutageCooldownSeconds</c> — were removed with <c>LdapOpcUaUserAuthenticator</c> in #520
|
||||
/// (see the note on this type). They existed because an LDAP bind ran inline on OPC UA session
|
||||
/// activation, under a SessionManager-wide lock; nothing on the remaining AdminUI login path has
|
||||
/// that property, so they had no consumer left. Setting any of them in configuration is now
|
||||
/// silently ignored rather than an error, which is how options binding treats unknown keys.
|
||||
/// </remarks>
|
||||
public int ConnectionTimeoutMs { get; set; } = 10_000;
|
||||
|
||||
/// <summary>
|
||||
/// Hard whole-flow wall-clock bound (in milliseconds) the OPC UA data-plane authenticator
|
||||
/// (<c>LdapOpcUaUserAuthenticator</c>) applies across the ENTIRE authenticate flow — bind plus
|
||||
/// role-map — so the OPC UA SDK's synchronous impersonation callback (raised under a
|
||||
/// <c>SessionManager</c>-wide event lock) can never be wedged by a slow / hung directory.
|
||||
/// Deliberately > <see cref="ConnectionTimeoutMs"/> (default 15000) so a healthy-but-slow
|
||||
/// full flow is not cut off while an outage is bounded by the library timeout first (03/S2).
|
||||
/// </summary>
|
||||
public int AuthTimeoutMs { get; set; } = 15_000;
|
||||
|
||||
/// <summary>
|
||||
/// Maximum number of in-flight OPC UA authentication flows the data-plane authenticator admits
|
||||
/// at once. A caller that cannot acquire a slot within its budget is denied "backend busy"
|
||||
/// (local backpressure, not counted against the outage circuit). Bounds accumulation of
|
||||
/// orphaned / timed-out bind tasks during an outage (03/S2). Default 8.
|
||||
/// </summary>
|
||||
public int MaxConcurrentAuthentications { get; set; } = 8;
|
||||
|
||||
/// <summary>
|
||||
/// Number of CONSECUTIVE system-side failures (boundary timeout, directory-unreachable result,
|
||||
/// or unexpected throw) after which the data-plane authenticator's outage circuit opens and
|
||||
/// denies instantly for <see cref="OutageCooldownSeconds"/> without touching LDAP. Any success
|
||||
/// or user-side (credential) deny resets the counter. <c>0</c> disables the circuit (03/S2).
|
||||
/// Default 3.
|
||||
/// </summary>
|
||||
public int OutageFailureThreshold { get; set; } = 3;
|
||||
|
||||
/// <summary>
|
||||
/// How long (in seconds) the outage circuit stays open before a half-open probe is allowed
|
||||
/// through. Only meaningful when <see cref="OutageFailureThreshold"/> > 0. Default 15 (03/S2).
|
||||
/// </summary>
|
||||
public int OutageCooldownSeconds { get; set; } = 15;
|
||||
|
||||
/// <summary>
|
||||
/// Projects the wire fields onto the shared <c>ZB.MOM.WW.Auth.Ldap</c>
|
||||
/// <see cref="LibLdapOptions"/> the directory client consumes. App-only concerns
|
||||
|
||||
@@ -57,7 +57,7 @@ public static class ServiceCollectionExtensions
|
||||
|
||||
// Singleton — OtOpcUaLdapAuthService is stateless (the shared-library directory client it
|
||||
// wraps opens/disposes an LdapConnection per call) and must be consumable by the Singleton
|
||||
// LdapOpcUaUserAuthenticator on driver-role nodes. This is the app's ILdapAuthService: it
|
||||
// AdminUI login on admin-role nodes. This is the app's ILdapAuthService: it
|
||||
// adds the Enabled master switch + DevStubMode bypass on top of the shared ZB.MOM.WW.Auth.Ldap
|
||||
// service. TryAdd so a fused admin+driver node (which also registers it in Program.cs for the
|
||||
// driver path) ends up with exactly one descriptor regardless of registration order.
|
||||
@@ -65,7 +65,7 @@ public static class ServiceCollectionExtensions
|
||||
|
||||
// Shared ZB.MOM.WW.Auth group→role mapper seam. Wraps the existing RoleMapper.Map +
|
||||
// RoleMapper.Merge logic; consumed by AuthEndpoints.cs (/auth/login) and by
|
||||
// LdapOpcUaUserAuthenticator (OPC UA UserName session auth). Scoped to match
|
||||
// AdminUI login (the OPC UA data plane uses API keys since #520). Scoped to match
|
||||
// ILdapGroupRoleMappingService (DbContext-backed, registered Scoped) — a singleton here
|
||||
// would capture the scoped DB service.
|
||||
services.TryAddScoped<IGroupRoleMapper<string>, OtOpcUaGroupRoleMapper>();
|
||||
|
||||
@@ -1,389 +0,0 @@
|
||||
using System.Diagnostics;
|
||||
using System.Text;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using Microsoft.Extensions.Options;
|
||||
using Opc.Ua;
|
||||
using Opc.Ua.Server;
|
||||
using Shouldly;
|
||||
using Xunit;
|
||||
using ZB.MOM.WW.Auth.Abstractions.Roles;
|
||||
using ZB.MOM.WW.OtOpcUa.Host.OpcUa;
|
||||
using ZB.MOM.WW.OtOpcUa.OpcUaServer;
|
||||
using ZB.MOM.WW.OtOpcUa.Security.Ldap;
|
||||
using LdapTransport = ZB.MOM.WW.Auth.Abstractions.Ldap.LdapTransport;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.Host.IntegrationTests;
|
||||
|
||||
/// <summary>
|
||||
/// Resilience guard for <see cref="LdapOpcUaUserAuthenticator"/> (arch-review 03/S2). The OPC UA SDK
|
||||
/// raises the impersonation callback synchronously under a <c>SessionManager</c>-wide event lock, so a
|
||||
/// slow / hung LDAP bind inside it does not merely stall one SDK thread — it serializes EVERY session
|
||||
/// activation server-wide. These tests prove the authenticator enforces a hard wall-clock bound,
|
||||
/// bounded in-flight concurrency, and a directory-outage circuit at its own boundary (the SDK callback
|
||||
/// being irreducibly synchronous), all fail-closed. No real LDAP server is used — a delaying fake
|
||||
/// reproduces the outage shape offline.
|
||||
/// </summary>
|
||||
public sealed class LdapAuthResilienceTests
|
||||
{
|
||||
/// <summary>
|
||||
/// RED repro (03/S2): a backend that takes 5 s must NOT hold the caller for 5 s — the authenticator's
|
||||
/// wall-clock bound denies (fail-closed) well before then. On current code (no bound) this fails: the
|
||||
/// call returns Allow after the full 5 s.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task Authenticate_slow_backend_denies_within_AuthTimeout()
|
||||
{
|
||||
var ldap = new DelayingFakeLdap(
|
||||
TimeSpan.FromSeconds(5),
|
||||
new LdapAuthResult(true, "Alice", "alice", new[] { "configeditor" }, Array.Empty<string>(), null));
|
||||
var mapper = new FakeMapper(g => g.ToArray());
|
||||
var sut = Build(ldap, mapper, new LdapOptions { AuthTimeoutMs = 200 });
|
||||
|
||||
var sw = Stopwatch.StartNew();
|
||||
var result = await sut
|
||||
.AuthenticateUserNameAsync("alice", "secret", CancellationToken.None)
|
||||
.WaitAsync(TimeSpan.FromSeconds(10), CancellationToken.None);
|
||||
sw.Stop();
|
||||
|
||||
result.Success.ShouldBeFalse();
|
||||
result.Error.ShouldNotBeNull();
|
||||
result.Error!.ShouldContain("timed out");
|
||||
sw.Elapsed.ShouldBeLessThan(TimeSpan.FromSeconds(1.5));
|
||||
}
|
||||
|
||||
/// <summary>A fast, healthy backend is unaffected by the resilience wrapping — success + roles pass
|
||||
/// through untouched (03/S2).</summary>
|
||||
[Fact]
|
||||
public async Task Authenticate_fast_success_unaffected()
|
||||
{
|
||||
var ldap = new DelayingFakeLdap(
|
||||
TimeSpan.Zero,
|
||||
new LdapAuthResult(true, "Alice", "alice", new[] { "configeditor" }, Array.Empty<string>(), null));
|
||||
var mapper = new FakeMapper(g => g.Select(x => x == "configeditor" ? "Designer" : x).ToArray());
|
||||
var sut = Build(ldap, mapper, new LdapOptions());
|
||||
|
||||
var result = await sut
|
||||
.AuthenticateUserNameAsync("alice", "secret", CancellationToken.None)
|
||||
.WaitAsync(TimeSpan.FromSeconds(10), CancellationToken.None);
|
||||
|
||||
result.Success.ShouldBeTrue();
|
||||
result.DisplayName.ShouldBe("Alice");
|
||||
result.Roles.ShouldBe(new[] { "Designer" });
|
||||
}
|
||||
|
||||
/// <summary>With the concurrency cap saturated (a parked in-flight bind holding the only slot), a
|
||||
/// second call is denied "backend busy" within its own budget — it never waits behind the parked
|
||||
/// bind indefinitely (03/S2).</summary>
|
||||
[Fact]
|
||||
public async Task Authenticate_saturated_concurrency_denies_busy()
|
||||
{
|
||||
var ldap = new GatedFakeLdap(
|
||||
new LdapAuthResult(true, "Alice", "alice", new[] { "configeditor" }, Array.Empty<string>(), null));
|
||||
var mapper = new FakeMapper(g => g.ToArray());
|
||||
// MaxConcurrentAuthentications = 1 → the second call cannot get a slot; small AuthTimeoutMs so the
|
||||
// "busy" denial is quick and the first call's boundary also trips fast (its core stays parked,
|
||||
// holding the slot).
|
||||
var sut = Build(ldap, mapper, new LdapOptions { MaxConcurrentAuthentications = 1, AuthTimeoutMs = 300 });
|
||||
|
||||
// First call: acquires the only slot; its core parks in the gated fake.
|
||||
var first = sut.AuthenticateUserNameAsync("alice", "secret", CancellationToken.None);
|
||||
|
||||
// Wait until the fake has actually been entered (slot acquired, core parked).
|
||||
var spin = Stopwatch.StartNew();
|
||||
while (ldap.Entered == 0 && spin.Elapsed < TimeSpan.FromSeconds(5))
|
||||
await Task.Delay(10, TestContext.Current.CancellationToken);
|
||||
ldap.Entered.ShouldBe(1);
|
||||
|
||||
// Second concurrent call: the slot is held by the parked core, so acquisition times out → busy.
|
||||
var second = await sut
|
||||
.AuthenticateUserNameAsync("bob", "secret", CancellationToken.None)
|
||||
.WaitAsync(TimeSpan.FromSeconds(10), CancellationToken.None);
|
||||
|
||||
second.Success.ShouldBeFalse();
|
||||
second.Error.ShouldNotBeNull();
|
||||
second.Error!.ShouldContain("busy");
|
||||
ldap.Entered.ShouldBe(1, "the second call must be denied without ever reaching the backend");
|
||||
|
||||
// Release the parked bind so the first call's core completes cleanly (no orphaned resources).
|
||||
ldap.Release();
|
||||
await first.WaitAsync(TimeSpan.FromSeconds(10), CancellationToken.None);
|
||||
}
|
||||
|
||||
/// <summary>After <c>OutageFailureThreshold</c> consecutive system-side failures the circuit opens and
|
||||
/// the next call is fast-denied "backend unavailable" WITHOUT touching LDAP (03/S2).</summary>
|
||||
[Fact]
|
||||
public async Task Consecutive_system_failures_open_circuit_and_fast_deny()
|
||||
{
|
||||
var ldap = new DelayingFakeLdap(TimeSpan.Zero, SystemFailure());
|
||||
var mapper = new FakeMapper(g => g.ToArray());
|
||||
var sut = Build(ldap, mapper,
|
||||
new LdapOptions { OutageFailureThreshold = 3, OutageCooldownSeconds = 60, AuthTimeoutMs = 5000 });
|
||||
|
||||
for (var i = 0; i < 3; i++)
|
||||
{
|
||||
var r = await sut.AuthenticateUserNameAsync("u", "p", CancellationToken.None)
|
||||
.WaitAsync(TimeSpan.FromSeconds(10), CancellationToken.None);
|
||||
r.Success.ShouldBeFalse();
|
||||
}
|
||||
|
||||
ldap.Calls.ShouldBe(3);
|
||||
|
||||
var sw = Stopwatch.StartNew();
|
||||
var blocked = await sut.AuthenticateUserNameAsync("u", "p", CancellationToken.None)
|
||||
.WaitAsync(TimeSpan.FromSeconds(10), CancellationToken.None);
|
||||
sw.Stop();
|
||||
|
||||
blocked.Success.ShouldBeFalse();
|
||||
blocked.Error.ShouldNotBeNull();
|
||||
blocked.Error!.ShouldContain("unavailable");
|
||||
ldap.Calls.ShouldBe(3, "an open circuit must fast-deny without reaching the backend");
|
||||
sw.Elapsed.ShouldBeLessThan(TimeSpan.FromSeconds(1));
|
||||
}
|
||||
|
||||
/// <summary>A user-side (credential) deny between system failures resets the streak, so the circuit
|
||||
/// never opens — user denies are not directory evidence (03/S2).</summary>
|
||||
[Fact]
|
||||
public async Task User_side_deny_resets_the_system_failure_streak()
|
||||
{
|
||||
var ldap = new ScriptedFakeLdap(
|
||||
SystemFailure(), SystemFailure(), UserDeny(), SystemFailure(), SystemFailure());
|
||||
var mapper = new FakeMapper(g => g.ToArray());
|
||||
var sut = Build(ldap, mapper,
|
||||
new LdapOptions { OutageFailureThreshold = 3, OutageCooldownSeconds = 60, AuthTimeoutMs = 5000 });
|
||||
|
||||
// 5 scripted calls: sys, sys, user(reset), sys, sys → streak ends at 2 (< 3), circuit stays closed.
|
||||
for (var i = 0; i < 5; i++)
|
||||
{
|
||||
await sut.AuthenticateUserNameAsync("u", "p", CancellationToken.None)
|
||||
.WaitAsync(TimeSpan.FromSeconds(10), CancellationToken.None);
|
||||
}
|
||||
|
||||
// A 6th call must still REACH the backend (circuit never opened).
|
||||
await sut.AuthenticateUserNameAsync("u", "p", CancellationToken.None)
|
||||
.WaitAsync(TimeSpan.FromSeconds(10), CancellationToken.None);
|
||||
ldap.Calls.ShouldBe(6);
|
||||
}
|
||||
|
||||
/// <summary>After the cooldown elapses the circuit half-opens: the next call probes the backend and a
|
||||
/// success closes the circuit (03/S2).</summary>
|
||||
[Fact]
|
||||
public async Task Half_open_probe_after_cooldown_closes_on_success()
|
||||
{
|
||||
var ldap = new ScriptedFakeLdap(
|
||||
SystemFailure(), SystemFailure(), SystemFailure(),
|
||||
new LdapAuthResult(true, "Alice", "alice", new[] { "configeditor" }, Array.Empty<string>(), null));
|
||||
var mapper = new FakeMapper(g => g.ToArray());
|
||||
var sut = Build(ldap, mapper,
|
||||
new LdapOptions { OutageFailureThreshold = 3, OutageCooldownSeconds = 1, AuthTimeoutMs = 5000 });
|
||||
|
||||
for (var i = 0; i < 3; i++)
|
||||
await sut.AuthenticateUserNameAsync("u", "p", CancellationToken.None)
|
||||
.WaitAsync(TimeSpan.FromSeconds(10), CancellationToken.None);
|
||||
ldap.Calls.ShouldBe(3);
|
||||
|
||||
// Circuit open: this call is fast-denied without touching the backend.
|
||||
var blocked = await sut.AuthenticateUserNameAsync("u", "p", CancellationToken.None)
|
||||
.WaitAsync(TimeSpan.FromSeconds(10), CancellationToken.None);
|
||||
blocked.Error!.ShouldContain("unavailable");
|
||||
ldap.Calls.ShouldBe(3);
|
||||
|
||||
// Wait past the cooldown, then a probe reaches the backend and succeeds → circuit closes.
|
||||
await Task.Delay(TimeSpan.FromMilliseconds(1200), TestContext.Current.CancellationToken);
|
||||
|
||||
var probe = await sut.AuthenticateUserNameAsync("alice", "secret", CancellationToken.None)
|
||||
.WaitAsync(TimeSpan.FromSeconds(10), CancellationToken.None);
|
||||
probe.Success.ShouldBeTrue();
|
||||
ldap.Calls.ShouldBe(4, "the half-open probe reaches the backend");
|
||||
}
|
||||
|
||||
/// <summary><c>OutageFailureThreshold = 0</c> disables the circuit — it never opens, every call reaches
|
||||
/// the backend (03/S2).</summary>
|
||||
[Fact]
|
||||
public async Task Circuit_disabled_never_opens()
|
||||
{
|
||||
var ldap = new DelayingFakeLdap(TimeSpan.Zero, SystemFailure());
|
||||
var mapper = new FakeMapper(g => g.ToArray());
|
||||
var sut = Build(ldap, mapper,
|
||||
new LdapOptions { OutageFailureThreshold = 0, OutageCooldownSeconds = 60, AuthTimeoutMs = 5000 });
|
||||
|
||||
for (var i = 0; i < 6; i++)
|
||||
await sut.AuthenticateUserNameAsync("u", "p", CancellationToken.None)
|
||||
.WaitAsync(TimeSpan.FromSeconds(10), CancellationToken.None);
|
||||
|
||||
ldap.Calls.ShouldBe(6, "a disabled circuit never fast-denies");
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Outage-sim (offline-safe): the REAL <see cref="OtOpcUaLdapAuthService"/> + real shared library
|
||||
/// pointed at an UNROUTABLE host (a TCP blackhole — the same outage shape as "stop the GLAuth
|
||||
/// container"), driven through the REAL internal <see cref="OpcUaApplicationHost.HandleImpersonation"/>
|
||||
/// block-bridge. The activation must fail bounded (well under the SDK's default ~10-20 s stall) with
|
||||
/// <see cref="StatusCodes.BadIdentityTokenRejected"/> and no granted identity (03/S2).
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task Activation_with_unreachable_directory_fails_within_bound()
|
||||
{
|
||||
var options = new LdapOptions
|
||||
{
|
||||
Enabled = true,
|
||||
DevStubMode = false,
|
||||
Server = "10.255.255.1", // RFC-blackhole: SYN goes nowhere
|
||||
Port = 3893,
|
||||
Transport = LdapTransport.None,
|
||||
AllowInsecure = true,
|
||||
SearchBase = "dc=zb,dc=local",
|
||||
ConnectionTimeoutMs = 500,
|
||||
AuthTimeoutMs = 1000,
|
||||
};
|
||||
|
||||
var ldap = new OtOpcUaLdapAuthService(Options.Create(options), NullLogger<OtOpcUaLdapAuthService>.Instance);
|
||||
var mapper = new FakeMapper(g => g.ToArray());
|
||||
var authenticator = new LdapOpcUaUserAuthenticator(
|
||||
ldap, ScopeFactoryWith(mapper), Options.Create(options),
|
||||
NullLogger<LdapOpcUaUserAuthenticator>.Instance);
|
||||
|
||||
var token = new UserNameIdentityToken { UserName = "alice", DecryptedPassword = Encoding.UTF8.GetBytes("secret") };
|
||||
var policy = new UserTokenPolicy(UserTokenType.UserName) { PolicyId = "username_basic256sha256" };
|
||||
var args = new ImpersonateEventArgs(token, policy, new EndpointDescription());
|
||||
|
||||
var sw = Stopwatch.StartNew();
|
||||
await Task.Run(() => OpcUaApplicationHost.HandleImpersonation(
|
||||
authenticator, args, NullLogger<object>.Instance), TestContext.Current.CancellationToken)
|
||||
.WaitAsync(TimeSpan.FromSeconds(10), TestContext.Current.CancellationToken);
|
||||
sw.Stop();
|
||||
|
||||
args.Identity.ShouldBeNull();
|
||||
args.IdentityValidationError.ShouldNotBeNull();
|
||||
args.IdentityValidationError.Code.ShouldBe(Opc.Ua.StatusCodes.BadIdentityTokenRejected);
|
||||
sw.Elapsed.ShouldBeLessThan(TimeSpan.FromSeconds(3));
|
||||
}
|
||||
|
||||
private static LdapAuthResult SystemFailure() =>
|
||||
new(false, null, "u", Array.Empty<string>(), Array.Empty<string>(), "Unexpected authentication error")
|
||||
{ IsSystemFailure = true };
|
||||
|
||||
private static LdapAuthResult UserDeny() =>
|
||||
new(false, null, "u", Array.Empty<string>(), Array.Empty<string>(), "Invalid username or password")
|
||||
{ IsSystemFailure = false };
|
||||
|
||||
private static LdapOpcUaUserAuthenticator Build(
|
||||
ILdapAuthService ldap, IGroupRoleMapper<string> mapper, LdapOptions options) =>
|
||||
new(ldap, ScopeFactoryWith(mapper), Options.Create(options),
|
||||
NullLogger<LdapOpcUaUserAuthenticator>.Instance);
|
||||
|
||||
/// <summary>Builds an IServiceScopeFactory whose scopes resolve the supplied mapper.</summary>
|
||||
private static IServiceScopeFactory ScopeFactoryWith(IGroupRoleMapper<string> mapper)
|
||||
{
|
||||
var services = new ServiceCollection();
|
||||
services.AddScoped(_ => mapper);
|
||||
return services.BuildServiceProvider().GetRequiredService<IServiceScopeFactory>();
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// A fake <see cref="ILdapAuthService"/> that awaits a configurable delay (observing the token) before
|
||||
/// returning a fixed result — the offline stand-in for a slow / hung directory. Counts invocations so
|
||||
/// tests can assert the outage circuit denies WITHOUT reaching the backend.
|
||||
/// </summary>
|
||||
private sealed class DelayingFakeLdap : ILdapAuthService
|
||||
{
|
||||
private readonly TimeSpan _delay;
|
||||
private readonly LdapAuthResult _then;
|
||||
private int _calls;
|
||||
|
||||
/// <summary>Initializes the fake with a per-call delay and the result to return afterwards.</summary>
|
||||
/// <param name="delay">How long each authenticate call blocks before returning.</param>
|
||||
/// <param name="then">The result returned once the delay elapses.</param>
|
||||
public DelayingFakeLdap(TimeSpan delay, LdapAuthResult then)
|
||||
{
|
||||
_delay = delay;
|
||||
_then = then;
|
||||
}
|
||||
|
||||
/// <summary>The number of times the backend was actually invoked.</summary>
|
||||
public int Calls => Volatile.Read(ref _calls);
|
||||
|
||||
/// <summary>Awaits the configured delay, then returns the canned result.</summary>
|
||||
/// <param name="username">The username (ignored).</param>
|
||||
/// <param name="password">The password (ignored).</param>
|
||||
/// <param name="ct">Cancellation token, observed by the delay.</param>
|
||||
public async Task<LdapAuthResult> AuthenticateAsync(string username, string password, CancellationToken ct = default)
|
||||
{
|
||||
Interlocked.Increment(ref _calls);
|
||||
await Task.Delay(_delay, ct).ConfigureAwait(false);
|
||||
return _then;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// A fake <see cref="ILdapAuthService"/> that parks ASYNCHRONOUSLY (awaits a
|
||||
/// <see cref="TaskCompletionSource"/>) until the test calls <see cref="Release"/> — used to hold an
|
||||
/// in-flight bind so the concurrency cap is deterministically saturated WITHOUT blocking a thread.
|
||||
/// Records how many calls have entered (past the slot acquire).
|
||||
/// </summary>
|
||||
private sealed class GatedFakeLdap : ILdapAuthService
|
||||
{
|
||||
private readonly TaskCompletionSource _park = new(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||
private readonly LdapAuthResult _then;
|
||||
private int _entered;
|
||||
|
||||
/// <summary>Initializes the fake with the result to return once released.</summary>
|
||||
/// <param name="then">The result returned once <see cref="Release"/> is called.</param>
|
||||
public GatedFakeLdap(LdapAuthResult then) => _then = then;
|
||||
|
||||
/// <summary>Number of authenticate calls that reached the backend body.</summary>
|
||||
public int Entered => Volatile.Read(ref _entered);
|
||||
|
||||
/// <summary>Releases the parked bind so its awaiting call can complete.</summary>
|
||||
public void Release() => _park.TrySetResult();
|
||||
|
||||
/// <summary>Marks entry, awaits the parking gate, then returns the canned result.</summary>
|
||||
/// <param name="username">The username (ignored).</param>
|
||||
/// <param name="password">The password (ignored).</param>
|
||||
/// <param name="ct">Cancellation token, observed while parked.</param>
|
||||
public async Task<LdapAuthResult> AuthenticateAsync(string username, string password, CancellationToken ct = default)
|
||||
{
|
||||
Interlocked.Increment(ref _entered);
|
||||
await _park.Task.WaitAsync(ct).ConfigureAwait(false);
|
||||
return _then;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// A fake <see cref="ILdapAuthService"/> that returns a scripted sequence of results (one per call;
|
||||
/// the last entry repeats once exhausted). Counts calls so the outage-circuit tests can assert the
|
||||
/// backend was — or was NOT — reached on a given call.
|
||||
/// </summary>
|
||||
private sealed class ScriptedFakeLdap : ILdapAuthService
|
||||
{
|
||||
private readonly IReadOnlyList<LdapAuthResult> _script;
|
||||
private int _calls;
|
||||
|
||||
/// <summary>Initializes the fake with the scripted results, returned in order.</summary>
|
||||
/// <param name="script">The results to return, one per call; the last repeats once exhausted.</param>
|
||||
public ScriptedFakeLdap(params LdapAuthResult[] script) => _script = script;
|
||||
|
||||
/// <summary>The number of times the backend was invoked.</summary>
|
||||
public int Calls => Volatile.Read(ref _calls);
|
||||
|
||||
/// <summary>Returns the next scripted result (clamped to the last entry).</summary>
|
||||
/// <param name="username">The username (ignored).</param>
|
||||
/// <param name="password">The password (ignored).</param>
|
||||
/// <param name="ct">The cancellation token (ignored).</param>
|
||||
public Task<LdapAuthResult> AuthenticateAsync(string username, string password, CancellationToken ct = default)
|
||||
{
|
||||
var idx = Interlocked.Increment(ref _calls) - 1;
|
||||
return Task.FromResult(_script[Math.Min(idx, _script.Count - 1)]);
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Test group→role mapper driven by a delegate over the supplied groups.</summary>
|
||||
private sealed class FakeMapper(Func<IReadOnlyList<string>, IReadOnlyList<string>> map) : IGroupRoleMapper<string>
|
||||
{
|
||||
/// <summary>Maps groups to roles via the configured delegate; Scope is always null.</summary>
|
||||
/// <param name="groups">The LDAP groups to map.</param>
|
||||
/// <param name="ct">The cancellation token.</param>
|
||||
public Task<GroupRoleMapping<string>> MapAsync(IReadOnlyList<string> groups, CancellationToken ct)
|
||||
=> Task.FromResult(new GroupRoleMapping<string>(map(groups), Scope: null));
|
||||
}
|
||||
}
|
||||
@@ -1,190 +0,0 @@
|
||||
using System.Diagnostics;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using Microsoft.Extensions.Options;
|
||||
using Shouldly;
|
||||
using Xunit;
|
||||
using ZB.MOM.WW.Auth.Abstractions.Roles;
|
||||
using ZB.MOM.WW.OtOpcUa.Host.OpcUa;
|
||||
using ZB.MOM.WW.OtOpcUa.Security.Ldap;
|
||||
using LdapTransport = ZB.MOM.WW.Auth.Abstractions.Ldap.LdapTransport;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.Host.IntegrationTests;
|
||||
|
||||
/// <summary>
|
||||
/// R2-08 03/S2 LIVE gate — the directory-outage circuit exercised against a REAL GLAuth server
|
||||
/// (the image + config PR #451 swapped in for the retired bitnami/openldap), driven through the
|
||||
/// production auth path (<see cref="OtOpcUaLdapAuthService"/> + <see cref="LdapOpcUaUserAuthenticator"/>).
|
||||
/// The offline <see cref="LdapAuthResilienceTests"/> prove the circuit logic with fakes + an
|
||||
/// unroutable-host blackhole; this proves the same behaviour against an actual directory that is
|
||||
/// paused mid-run (a real outage) and unpaused (recovery), and doubles as the first end-to-end
|
||||
/// verification that the GLAuth swap binds correctly.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// Triple-gated so it never runs by accident and always restores the shared GLAuth:
|
||||
/// <list type="number">
|
||||
/// <item><c>GLAUTH_LIVE_HOST</c> — the GLAuth host (e.g. <c>10.100.0.35</c>); absent ⇒ skip;</item>
|
||||
/// <item><c>GLAUTH_OUTAGE_START_CMD</c> — pauses the GLAuth container (a real outage); and</item>
|
||||
/// <item><c>GLAUTH_OUTAGE_STOP_CMD</c> — unpauses it.</item>
|
||||
/// </list>
|
||||
/// A <c>try/finally</c> ALWAYS runs the stop command once the pause succeeded. Operator recipe:
|
||||
/// <code>
|
||||
/// ssh dohertj2@10.100.0.35 "cd ~/otopcua-harness && docker compose up -d ldap"
|
||||
/// export GLAUTH_LIVE_HOST=10.100.0.35
|
||||
/// export GLAUTH_OUTAGE_START_CMD='ssh dohertj2@10.100.0.35 "docker pause otopcua-harness-ldap-1"'
|
||||
/// export GLAUTH_OUTAGE_STOP_CMD='ssh dohertj2@10.100.0.35 "docker unpause otopcua-harness-ldap-1"'
|
||||
/// dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests --filter "FullyQualifiedName~LdapGlauthOutageLive"
|
||||
/// </code>
|
||||
/// </remarks>
|
||||
[Trait("Category", "Integration")]
|
||||
[Trait("Category", "LiveLdap")]
|
||||
public sealed class LdapGlauthOutageLiveTests
|
||||
{
|
||||
private const string HostEnvVar = "GLAUTH_LIVE_HOST";
|
||||
private const string PortEnvVar = "GLAUTH_LIVE_PORT";
|
||||
private const string StartCmdEnvVar = "GLAUTH_OUTAGE_START_CMD";
|
||||
private const string StopCmdEnvVar = "GLAUTH_OUTAGE_STOP_CMD";
|
||||
|
||||
private const int OutageThreshold = 3;
|
||||
private const int CooldownSeconds = 2;
|
||||
private static readonly TimeSpan AuthTimeout = TimeSpan.FromMilliseconds(2_500);
|
||||
|
||||
/// <summary>
|
||||
/// Healthy real binds succeed; pausing GLAuth makes consecutive binds fail bounded (never a
|
||||
/// multi-second SDK stall) and opens the circuit into a sub-second fast-deny; unpausing +
|
||||
/// cooldown lets a half-open probe close the circuit and binds succeed again.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task Real_glauth_outage_opens_circuit_then_recovers_on_unpause()
|
||||
{
|
||||
var host = Environment.GetEnvironmentVariable(HostEnvVar);
|
||||
if (string.IsNullOrWhiteSpace(host))
|
||||
Assert.Skip($"Set {HostEnvVar} (+ {StartCmdEnvVar}/{StopCmdEnvVar}) to run the live GLAuth outage cycle.");
|
||||
var port = int.TryParse(Environment.GetEnvironmentVariable(PortEnvVar), out var p) ? p : 3894;
|
||||
var startCmd = Environment.GetEnvironmentVariable(StartCmdEnvVar);
|
||||
var stopCmd = Environment.GetEnvironmentVariable(StopCmdEnvVar);
|
||||
if (string.IsNullOrWhiteSpace(startCmd) || string.IsNullOrWhiteSpace(stopCmd))
|
||||
Assert.Skip($"Set both {StartCmdEnvVar} and {StopCmdEnvVar} (docker pause/unpause) to run this destructive live test.");
|
||||
|
||||
var ct = TestContext.Current.CancellationToken;
|
||||
var options = new LdapOptions
|
||||
{
|
||||
Enabled = true,
|
||||
DevStubMode = false,
|
||||
Server = host!,
|
||||
Port = port,
|
||||
Transport = LdapTransport.None,
|
||||
AllowInsecure = true,
|
||||
SearchBase = "dc=zb,dc=local",
|
||||
ServiceAccountDn = "cn=serviceaccount,dc=zb,dc=local",
|
||||
ServiceAccountPassword = "serviceaccount123",
|
||||
ConnectionTimeoutMs = 2_000,
|
||||
AuthTimeoutMs = (int)AuthTimeout.TotalMilliseconds,
|
||||
OutageFailureThreshold = OutageThreshold,
|
||||
OutageCooldownSeconds = CooldownSeconds,
|
||||
};
|
||||
var authenticator = BuildAuthenticator(options);
|
||||
|
||||
// ── 1. Healthy directory: real binds succeed, wrong creds deny (user-side, not an outage). ──
|
||||
var alice = await authenticator.AuthenticateUserNameAsync("alice", "alice123", ct);
|
||||
alice.Success.ShouldBeTrue("alice should bind against the live GLAuth");
|
||||
alice.Roles.ShouldNotBeEmpty("a healthy bind returns alice's mapped group roles");
|
||||
|
||||
var bob = await authenticator.AuthenticateUserNameAsync("bob", "bob123", ct);
|
||||
bob.Success.ShouldBeTrue("bob should bind against the live GLAuth");
|
||||
|
||||
var wrong = await authenticator.AuthenticateUserNameAsync("alice", "wrong-password", ct);
|
||||
wrong.Success.ShouldBeFalse("a wrong password is a user-side deny");
|
||||
|
||||
// ── 2. Outage: pause GLAuth. Consecutive binds fail BOUNDED, then the circuit fast-denies. ──
|
||||
var paused = false;
|
||||
try
|
||||
{
|
||||
await RunShellCommandAsync(startCmd!, ct);
|
||||
paused = true;
|
||||
|
||||
for (var i = 0; i < OutageThreshold; i++)
|
||||
{
|
||||
var sw = Stopwatch.StartNew();
|
||||
var outage = await authenticator.AuthenticateUserNameAsync("alice", "alice123", ct);
|
||||
sw.Stop();
|
||||
|
||||
outage.Success.ShouldBeFalse($"bind #{i + 1} during the outage must fail closed");
|
||||
// The whole flow is wall-clock bounded — never the SDK's default ~10-20 s stall.
|
||||
sw.Elapsed.ShouldBeLessThan(AuthTimeout + TimeSpan.FromSeconds(2),
|
||||
"the authenticator must bound a hung directory call, not stall on it");
|
||||
}
|
||||
|
||||
// Circuit is now open — the next call fast-denies WITHOUT a directory round-trip (sub-second).
|
||||
var blockedSw = Stopwatch.StartNew();
|
||||
var blocked = await authenticator.AuthenticateUserNameAsync("alice", "alice123", ct);
|
||||
blockedSw.Stop();
|
||||
|
||||
blocked.Success.ShouldBeFalse();
|
||||
blocked.Error.ShouldNotBeNull();
|
||||
blocked.Error!.ShouldContain("unavailable", Case.Insensitive,
|
||||
"an open outage circuit fast-denies 'backend unavailable'");
|
||||
blockedSw.Elapsed.ShouldBeLessThan(TimeSpan.FromSeconds(1),
|
||||
"an open circuit fast-denies without a directory call");
|
||||
}
|
||||
finally
|
||||
{
|
||||
if (paused)
|
||||
await RunShellCommandAsync(stopCmd!, CancellationToken.None); // ALWAYS restore GLAuth
|
||||
}
|
||||
|
||||
// ── 3. Recovery: after the cooldown the half-open probe reaches the (restored) directory. ──
|
||||
await Task.Delay(TimeSpan.FromSeconds(CooldownSeconds) + TimeSpan.FromSeconds(1), ct);
|
||||
|
||||
var recovered = false;
|
||||
var deadline = DateTime.UtcNow + TimeSpan.FromSeconds(30);
|
||||
while (DateTime.UtcNow < deadline)
|
||||
{
|
||||
var probe = await authenticator.AuthenticateUserNameAsync("alice", "alice123", ct);
|
||||
if (probe.Success) { recovered = true; break; }
|
||||
await Task.Delay(TimeSpan.FromMilliseconds(500), ct);
|
||||
}
|
||||
recovered.ShouldBeTrue("the circuit must half-open and close once GLAuth is unpaused");
|
||||
}
|
||||
|
||||
private static LdapOpcUaUserAuthenticator BuildAuthenticator(LdapOptions options)
|
||||
{
|
||||
var ldap = new OtOpcUaLdapAuthService(Options.Create(options), NullLogger<OtOpcUaLdapAuthService>.Instance);
|
||||
return new LdapOpcUaUserAuthenticator(
|
||||
ldap, ScopeFactoryWith(new PassthroughMapper()), Options.Create(options),
|
||||
NullLogger<LdapOpcUaUserAuthenticator>.Instance);
|
||||
}
|
||||
|
||||
/// <summary>Builds an <see cref="IServiceScopeFactory"/> whose scopes resolve the supplied mapper.</summary>
|
||||
private static IServiceScopeFactory ScopeFactoryWith(IGroupRoleMapper<string> mapper)
|
||||
{
|
||||
var services = new ServiceCollection();
|
||||
services.AddScoped(_ => mapper);
|
||||
return services.BuildServiceProvider().GetRequiredService<IServiceScopeFactory>();
|
||||
}
|
||||
|
||||
/// <summary>Runs an SSH/docker one-liner through the login shell (verbatim, as in the S7 outage gate).</summary>
|
||||
private static async Task RunShellCommandAsync(string command, CancellationToken ct)
|
||||
{
|
||||
using var proc = new Process
|
||||
{
|
||||
StartInfo = new ProcessStartInfo("/bin/sh", $"-c \"{command.Replace("\"", "\\\"")}\"")
|
||||
{
|
||||
RedirectStandardOutput = true,
|
||||
RedirectStandardError = true,
|
||||
UseShellExecute = false,
|
||||
},
|
||||
};
|
||||
proc.Start();
|
||||
await proc.WaitForExitAsync(ct);
|
||||
proc.ExitCode.ShouldBe(0, $"outage command failed: {await proc.StandardError.ReadToEndAsync(ct)}");
|
||||
await Task.Delay(TimeSpan.FromSeconds(1), ct); // let the pause/unpause take effect
|
||||
}
|
||||
|
||||
/// <summary>Passes LDAP groups straight through as roles — the live gate asserts on the bind, not the mapping.</summary>
|
||||
private sealed class PassthroughMapper : IGroupRoleMapper<string>
|
||||
{
|
||||
public Task<GroupRoleMapping<string>> MapAsync(IReadOnlyList<string> groups, CancellationToken ct)
|
||||
=> Task.FromResult(new GroupRoleMapping<string>(groups, Scope: null));
|
||||
}
|
||||
}
|
||||
-152
@@ -1,152 +0,0 @@
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using Microsoft.Extensions.Options;
|
||||
using Shouldly;
|
||||
using Xunit;
|
||||
using ZB.MOM.WW.Auth.Abstractions.Roles;
|
||||
using ZB.MOM.WW.OtOpcUa.Host.OpcUa;
|
||||
using ZB.MOM.WW.OtOpcUa.Security.Ldap;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.Host.IntegrationTests;
|
||||
|
||||
/// <summary>
|
||||
/// Verifies <see cref="LdapOpcUaUserAuthenticator"/> translates app <see cref="ILdapAuthService"/>
|
||||
/// outcomes into <c>OpcUaUserAuthResult</c>, resolves roles from the directory's <em>groups</em>
|
||||
/// through the shared <see cref="IGroupRoleMapper{TRole}"/> seam (Task 1.2), unions any pre-resolved
|
||||
/// roles (the DevStub Administrator grant) in, and turns LDAP backend exceptions into a denial rather
|
||||
/// than letting them escape into the SDK.
|
||||
/// </summary>
|
||||
public sealed class LdapOpcUaUserAuthenticatorTests
|
||||
{
|
||||
/// <summary>On success the data-plane authenticator resolves roles via the mapper from the
|
||||
/// returned Groups — not from the auth result's Roles field — and grants identity.</summary>
|
||||
[Fact]
|
||||
public async Task Authenticate_LDAP_success_resolves_roles_via_mapper_from_groups()
|
||||
{
|
||||
// Library-style result: groups present, Roles empty (the real path). The mapper maps the
|
||||
// group "configeditor" -> "Designer" (canonical, Task 1.7).
|
||||
var ldap = new FakeLdap(new LdapAuthResult(true, "Alice", "alice", new[] { "configeditor" }, Array.Empty<string>(), null));
|
||||
var mapper = new FakeMapper(g => g.Select(x => x == "configeditor" ? "Designer" : x).ToArray());
|
||||
var sut = new LdapOpcUaUserAuthenticator(ldap, ScopeFactoryWith(mapper), Options.Create(new LdapOptions()), NullLogger<LdapOpcUaUserAuthenticator>.Instance);
|
||||
|
||||
var result = await sut.AuthenticateUserNameAsync("alice", "secret", CancellationToken.None);
|
||||
|
||||
result.Success.ShouldBeTrue();
|
||||
result.DisplayName.ShouldBe("Alice");
|
||||
result.Roles.ShouldBe(new[] { "Designer" });
|
||||
}
|
||||
|
||||
/// <summary>The DevStub pre-resolved roles (Administrator) survive the move to the mapper: they are
|
||||
/// unioned with the mapper output so the dev grant still reaches the OPC UA session.</summary>
|
||||
[Fact]
|
||||
public async Task Authenticate_devstub_preresolved_roles_are_unioned_with_mapper()
|
||||
{
|
||||
// DevStub-shaped result: group "dev", pre-resolved role "Administrator". Mapper maps "dev" to
|
||||
// nothing, so the union is exactly {Administrator}.
|
||||
var ldap = new FakeLdap(new LdapAuthResult(true, "dev", "dev", new[] { "dev" }, new[] { "Administrator" }, null));
|
||||
var mapper = new FakeMapper(_ => Array.Empty<string>());
|
||||
var sut = new LdapOpcUaUserAuthenticator(ldap, ScopeFactoryWith(mapper), Options.Create(new LdapOptions()), NullLogger<LdapOpcUaUserAuthenticator>.Instance);
|
||||
|
||||
var result = await sut.AuthenticateUserNameAsync("dev", "anything", CancellationToken.None);
|
||||
|
||||
result.Success.ShouldBeTrue();
|
||||
result.Roles.ShouldBe(new[] { "Administrator" });
|
||||
}
|
||||
|
||||
/// <summary>A mapper fault (e.g. DB outage) must not deny an authenticated session — it falls
|
||||
/// back to the pre-resolved roles, matching the login endpoint's behaviour.</summary>
|
||||
[Fact]
|
||||
public async Task Authenticate_mapper_fault_falls_back_to_preresolved_roles()
|
||||
{
|
||||
var ldap = new FakeLdap(new LdapAuthResult(true, "dev", "dev", new[] { "dev" }, new[] { "Administrator" }, null));
|
||||
var mapper = new FakeMapper(_ => throw new InvalidOperationException("DB down"));
|
||||
var sut = new LdapOpcUaUserAuthenticator(ldap, ScopeFactoryWith(mapper), Options.Create(new LdapOptions()), NullLogger<LdapOpcUaUserAuthenticator>.Instance);
|
||||
|
||||
var result = await sut.AuthenticateUserNameAsync("dev", "anything", CancellationToken.None);
|
||||
|
||||
result.Success.ShouldBeTrue();
|
||||
result.Roles.ShouldBe(new[] { "Administrator" });
|
||||
}
|
||||
|
||||
/// <summary>Verifies that LDAP authentication failure returns Deny result with error text.</summary>
|
||||
[Fact]
|
||||
public async Task Authenticate_LDAP_failure_returns_Deny_with_error_text()
|
||||
{
|
||||
var ldap = new FakeLdap(new LdapAuthResult(false, null, "mallory", Array.Empty<string>(), Array.Empty<string>(), "Invalid username or password"));
|
||||
var mapper = new FakeMapper(g => g.ToArray());
|
||||
var sut = new LdapOpcUaUserAuthenticator(ldap, ScopeFactoryWith(mapper), Options.Create(new LdapOptions()), NullLogger<LdapOpcUaUserAuthenticator>.Instance);
|
||||
|
||||
var result = await sut.AuthenticateUserNameAsync("mallory", "wrong", CancellationToken.None);
|
||||
|
||||
result.Success.ShouldBeFalse();
|
||||
result.Error.ShouldBe("Invalid username or password");
|
||||
}
|
||||
|
||||
/// <summary>Verifies that LDAP exceptions are converted to backend error denial results.</summary>
|
||||
[Fact]
|
||||
public async Task Authenticate_LDAP_exception_returns_backend_error_denial()
|
||||
{
|
||||
var ldap = new FakeLdap(_ => throw new InvalidOperationException("LDAP unreachable"));
|
||||
var mapper = new FakeMapper(g => g.ToArray());
|
||||
var sut = new LdapOpcUaUserAuthenticator(ldap, ScopeFactoryWith(mapper), Options.Create(new LdapOptions()), NullLogger<LdapOpcUaUserAuthenticator>.Instance);
|
||||
|
||||
var result = await sut.AuthenticateUserNameAsync("anyone", "x", CancellationToken.None);
|
||||
|
||||
result.Success.ShouldBeFalse();
|
||||
result.Error.ShouldNotBeNull();
|
||||
result.Error.ShouldContain("backend");
|
||||
}
|
||||
|
||||
/// <summary>Verifies that authentication falls back to username when LDAP omits display name.</summary>
|
||||
[Fact]
|
||||
public async Task Authenticate_falls_back_to_username_when_LDAP_omits_display_name()
|
||||
{
|
||||
var ldap = new FakeLdap(new LdapAuthResult(true, null, "alice", new[] { "ReadOnly" }, Array.Empty<string>(), null));
|
||||
var mapper = new FakeMapper(g => g.ToArray());
|
||||
var sut = new LdapOpcUaUserAuthenticator(ldap, ScopeFactoryWith(mapper), Options.Create(new LdapOptions()), NullLogger<LdapOpcUaUserAuthenticator>.Instance);
|
||||
|
||||
var result = await sut.AuthenticateUserNameAsync("alice", "x", CancellationToken.None);
|
||||
|
||||
result.Success.ShouldBeTrue();
|
||||
result.DisplayName.ShouldBe("alice");
|
||||
}
|
||||
|
||||
/// <summary>Builds an IServiceScopeFactory whose scopes resolve the supplied mapper.</summary>
|
||||
private static IServiceScopeFactory ScopeFactoryWith(IGroupRoleMapper<string> mapper)
|
||||
{
|
||||
var services = new ServiceCollection();
|
||||
services.AddScoped(_ => mapper);
|
||||
return services.BuildServiceProvider().GetRequiredService<IServiceScopeFactory>();
|
||||
}
|
||||
|
||||
/// <summary>Test fake implementation of LDAP authentication service.</summary>
|
||||
private sealed class FakeLdap : ILdapAuthService
|
||||
{
|
||||
private readonly Func<string, LdapAuthResult> _handler;
|
||||
|
||||
/// <summary>Initializes the fake with a fixed result that ignores the username.</summary>
|
||||
/// <param name="fixed_">The result to return for any authentication attempt.</param>
|
||||
public FakeLdap(LdapAuthResult fixed_) => _handler = _ => fixed_;
|
||||
|
||||
/// <summary>Initializes the fake with a handler function for custom results.</summary>
|
||||
/// <param name="handler">The handler to invoke with the username to produce a result.</param>
|
||||
public FakeLdap(Func<string, LdapAuthResult> handler) => _handler = handler;
|
||||
|
||||
/// <summary>Authenticates a user asynchronously via the handler function.</summary>
|
||||
/// <param name="username">The username to authenticate.</param>
|
||||
/// <param name="password">The password (ignored by the fake).</param>
|
||||
/// <param name="ct">Cancellation token for the operation.</param>
|
||||
public Task<LdapAuthResult> AuthenticateAsync(string username, string password, CancellationToken ct = default)
|
||||
=> Task.FromResult(_handler(username));
|
||||
}
|
||||
|
||||
/// <summary>Test fake group→role mapper driven by a delegate over the supplied groups.</summary>
|
||||
private sealed class FakeMapper(Func<IReadOnlyList<string>, IReadOnlyList<string>> map) : IGroupRoleMapper<string>
|
||||
{
|
||||
/// <summary>Maps groups to roles via the configured delegate; Scope is always null.</summary>
|
||||
/// <param name="groups">The LDAP groups to map.</param>
|
||||
/// <param name="ct">The cancellation token.</param>
|
||||
public Task<GroupRoleMapping<string>> MapAsync(IReadOnlyList<string> groups, CancellationToken ct)
|
||||
=> Task.FromResult(new GroupRoleMapping<string>(map(groups), Scope: null));
|
||||
}
|
||||
}
|
||||
@@ -63,19 +63,17 @@ public sealed class LdapOptionsBindingTests
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// The five S2 resilience keys bind from the real <c>Security:Ldap</c> section (03/S2).
|
||||
/// <c>ConnectionTimeoutMs</c> binds from the real <c>Security:Ldap</c> section. The other four S2
|
||||
/// resilience keys were removed with <c>LdapOpcUaUserAuthenticator</c> in #520 — they bounded an
|
||||
/// LDAP bind running inline on OPC UA session activation, and no remaining consumer does that.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void Binding_reads_resilience_keys()
|
||||
public void Binding_reads_the_connection_timeout_key()
|
||||
{
|
||||
var configuration = new ConfigurationBuilder()
|
||||
.AddInMemoryCollection(new Dictionary<string, string?>
|
||||
{
|
||||
["Security:Ldap:ConnectionTimeoutMs"] = "7000",
|
||||
["Security:Ldap:AuthTimeoutMs"] = "12000",
|
||||
["Security:Ldap:MaxConcurrentAuthentications"] = "5",
|
||||
["Security:Ldap:OutageFailureThreshold"] = "4",
|
||||
["Security:Ldap:OutageCooldownSeconds"] = "20",
|
||||
})
|
||||
.Build();
|
||||
|
||||
@@ -83,23 +81,13 @@ public sealed class LdapOptionsBindingTests
|
||||
|
||||
options.ShouldNotBeNull();
|
||||
options.ConnectionTimeoutMs.ShouldBe(7000);
|
||||
options.AuthTimeoutMs.ShouldBe(12000);
|
||||
options.MaxConcurrentAuthentications.ShouldBe(5);
|
||||
options.OutageFailureThreshold.ShouldBe(4);
|
||||
options.OutageCooldownSeconds.ShouldBe(20);
|
||||
}
|
||||
|
||||
/// <summary>The resilience keys carry the documented defaults when absent from config.</summary>
|
||||
/// <summary>The connection timeout carries its documented default when absent from config.</summary>
|
||||
[Fact]
|
||||
public void Resilience_key_defaults_hold_when_absent()
|
||||
public void Connection_timeout_default_holds_when_absent()
|
||||
{
|
||||
var options = new LdapOptions();
|
||||
|
||||
options.ConnectionTimeoutMs.ShouldBe(10000);
|
||||
options.AuthTimeoutMs.ShouldBe(15000);
|
||||
options.MaxConcurrentAuthentications.ShouldBe(8);
|
||||
options.OutageFailureThreshold.ShouldBe(3);
|
||||
options.OutageCooldownSeconds.ShouldBe(15);
|
||||
new LdapOptions().ConnectionTimeoutMs.ShouldBe(10000);
|
||||
}
|
||||
|
||||
/// <summary><c>ConnectionTimeoutMs</c> is projected onto the shared-library options so the
|
||||
|
||||
@@ -220,79 +220,6 @@ public sealed class LdapOptionsValidatorTests
|
||||
Sut.Validate(null, options).Failed.ShouldBeTrue();
|
||||
}
|
||||
|
||||
/// <summary>Enabled with a non-positive <c>AuthTimeoutMs</c> fails validation (03/S2).</summary>
|
||||
[Fact]
|
||||
public void Enabled_with_nonpositive_AuthTimeoutMs_fails()
|
||||
{
|
||||
var options = new LdapOptions
|
||||
{
|
||||
Enabled = true,
|
||||
Server = "ldap",
|
||||
SearchBase = "dc=x",
|
||||
Port = 389,
|
||||
Transport = LdapTransport.Ldaps,
|
||||
AuthTimeoutMs = 0,
|
||||
};
|
||||
|
||||
Sut.Validate(null, options).Failed.ShouldBeTrue();
|
||||
}
|
||||
|
||||
/// <summary>Enabled with a non-positive <c>MaxConcurrentAuthentications</c> fails validation (03/S2).</summary>
|
||||
[Fact]
|
||||
public void Enabled_with_nonpositive_MaxConcurrentAuthentications_fails()
|
||||
{
|
||||
var options = new LdapOptions
|
||||
{
|
||||
Enabled = true,
|
||||
Server = "ldap",
|
||||
SearchBase = "dc=x",
|
||||
Port = 389,
|
||||
Transport = LdapTransport.Ldaps,
|
||||
MaxConcurrentAuthentications = 0,
|
||||
};
|
||||
|
||||
Sut.Validate(null, options).Failed.ShouldBeTrue();
|
||||
}
|
||||
|
||||
/// <summary>A positive <c>OutageFailureThreshold</c> with a zero <c>OutageCooldownSeconds</c> fails —
|
||||
/// an open circuit with no cooldown would probe on every call (03/S2).</summary>
|
||||
[Fact]
|
||||
public void OutageFailureThreshold_positive_with_zero_cooldown_fails()
|
||||
{
|
||||
var options = new LdapOptions
|
||||
{
|
||||
Enabled = true,
|
||||
Server = "ldap",
|
||||
SearchBase = "dc=x",
|
||||
Port = 389,
|
||||
Transport = LdapTransport.Ldaps,
|
||||
OutageFailureThreshold = 3,
|
||||
OutageCooldownSeconds = 0,
|
||||
};
|
||||
|
||||
Sut.Validate(null, options).Failed.ShouldBeTrue();
|
||||
}
|
||||
|
||||
/// <summary>A zero <c>OutageFailureThreshold</c> (circuit disabled) passes regardless of the
|
||||
/// cooldown value (03/S2).</summary>
|
||||
[Fact]
|
||||
public void OutageFailureThreshold_zero_disables_circuit_and_passes()
|
||||
{
|
||||
var options = new LdapOptions
|
||||
{
|
||||
Enabled = true,
|
||||
Server = "ldap",
|
||||
SearchBase = "dc=x",
|
||||
Port = 389,
|
||||
Transport = LdapTransport.Ldaps,
|
||||
OutageFailureThreshold = 0,
|
||||
OutageCooldownSeconds = 0,
|
||||
};
|
||||
|
||||
Sut.Validate(null, options).Succeeded.ShouldBeTrue();
|
||||
}
|
||||
|
||||
/// <summary>Enabled with port 0 reports the port-range failure using the shared primitive wording.</summary>
|
||||
[Fact]
|
||||
public void Enabled_with_zero_port_fails()
|
||||
{
|
||||
|
||||
Reference in New Issue
Block a user