Merge R2-08 Async LDAP + HistoryRead (arch-review round 2) [PR #435]
Findings 03/S2 (LDAP bind no longer block-bridges the SDK session-activation lock — AuthTimeout/concurrency-semaphore/directory-outage circuit, fail-closed), 03/S3 (HistoryRead async-ified + bounded per-node fan-out + deadline CTS + batch limiter). All 14 tasks; full-sweep + live legs deferred. Clean merge, build clean.
This commit is contained in:
@@ -265,6 +265,9 @@ authorization uses the standard `AccessLevels.HistoryRead` bit set at materializ
|
|||||||
| `CaCertificatePath` | `null` | PEM CA file pinning the gateway TLS chain; null/empty uses the OS trust store |
|
| `CaCertificatePath` | `null` | PEM CA file pinning the gateway TLS chain; null/empty uses the OS trust store |
|
||||||
| `CallTimeout` | `00:00:30` | Per-call deadline for each unary gateway read |
|
| `CallTimeout` | `00:00:30` | Per-call deadline for each unary gateway read |
|
||||||
| `MaxTieClusterOverfetch` | `65536` | Bounded over-fetch the HistoryRead-Raw paging uses to page within an oversized same-timestamp tie cluster (retained from the prior backend) |
|
| `MaxTieClusterOverfetch` | `65536` | Bounded over-fetch the HistoryRead-Raw paging uses to page within an oversized same-timestamp tie cluster (retained from the prior backend) |
|
||||||
|
| `HistoryReadBatchParallelism` | `4` | Max nodes served concurrently within one HistoryRead batch (03/S3); `⌈N/P⌉ × CallTimeout` worst case instead of sequential |
|
||||||
|
| `MaxConcurrentHistoryReadBatches` | `16` | Process-wide concurrent HistoryRead batch cap (03/S3); flood degrades to `BadTooManyOperations`. Total in-flight reads capped at 16 × 4 = 64 |
|
||||||
|
| `HistoryReadDeadline` | `00:01:00` | Per-request HistoryRead wall-clock deadline (03/S3); a node still in flight surfaces `BadTimeout`. Non-positive = unbounded |
|
||||||
|
|
||||||
### Alarm-history path (`AlarmHistorian` section)
|
### Alarm-history path (`AlarmHistorian` section)
|
||||||
|
|
||||||
|
|||||||
@@ -674,3 +674,27 @@ Commit: `docs: ServerHistorian HistoryRead knobs + Security:Ldap resilience keys
|
|||||||
the usual test-run + review cadence. Risk concentrated in T11 (parallel fan-out — mitigated by the
|
the usual test-run + review cadence. Risk concentrated in T11 (parallel fan-out — mitigated by the
|
||||||
thread-safety audit + the paging regression suite) and T4/T5 (auth-path semantics — mitigated by the
|
thread-safety audit + the paging regression suite) and T4/T5 (auth-path semantics — mitigated by the
|
||||||
fail-closed default posture and behavior-neutral defaults).
|
fail-closed default posture and behavior-neutral defaults).
|
||||||
|
|
||||||
|
## Execution deviations (R2-08)
|
||||||
|
|
||||||
|
- **T1 RED shape.** Written against the *current* 3-arg ctor with a `Stopwatch` timing assertion (the
|
||||||
|
plan's explicitly-allowed alternative), not the intended-ctor compile-fail — so every commit compiles.
|
||||||
|
T4 then re-pointed it at the new `Options.Create(...)` ctor to turn it green.
|
||||||
|
- **Async parking fakes.** The concurrency-cap repro (`GatedFakeLdap`) and the limiter repro
|
||||||
|
(`GatedRawFake`) had to park **asynchronously** (await a `TaskCompletionSource`), not block a thread
|
||||||
|
synchronously — a synchronous block before the first `await` deadlocks the caller (the core task never
|
||||||
|
yields its `Task`). Documented inline in both fakes.
|
||||||
|
- **T6 InternalsVisibleTo.** Driving the real `internal static OpcUaApplicationHost.HandleImpersonation`
|
||||||
|
from `Host.IntegrationTests` required adding an `InternalsVisibleTo` for that test assembly to the
|
||||||
|
OpcUaServer csproj (the plan called for the internal drive but not the grant). Additive + reversible.
|
||||||
|
- **T13 fake ignores the deadline token.** `GatedRawFake.ReadRawAsync` deliberately does **not** observe
|
||||||
|
the per-request deadline `ct`: with a small `HistoryReadDeadline`, the parked first batch would
|
||||||
|
otherwise cancel its own read and free the limiter permit before the second batch checks (race). By
|
||||||
|
parking on the release gate only, the first batch holds the permit deterministically, so the second
|
||||||
|
batch reliably hits the saturated limiter → `BadTooManyOperations`.
|
||||||
|
- **T14 sweep scope.** Per the executor's memory constraint (heavy `*.IntegrationTests` / Playwright
|
||||||
|
suites leak ~16 GB and the LDAP integration harness spins an ephemeral openldap on :3894), the
|
||||||
|
whole-solution `dotnet test` was **not** run. Instead: a full-solution `dotnet build` (clean) + the
|
||||||
|
impacted **filtered** unit suites (Host.IntegrationTests LDAP-classes only, Security.Tests,
|
||||||
|
Runtime.Tests options, OpcUaServer.Tests HistoryRead). The whole-solution + live legs are deferred to a
|
||||||
|
serial heavy pass / VPN run.
|
||||||
|
|||||||
@@ -1,20 +1,20 @@
|
|||||||
{
|
{
|
||||||
"planPath": "archreview/plans/R2-08-ldap-historyread-async-plan.md",
|
"planPath": "archreview/plans/R2-08-ldap-historyread-async-plan.md",
|
||||||
"tasks": [
|
"tasks": [
|
||||||
{ "id": "R2-08-T1", "subject": "S2 stall-repro test (RED): slow LDAP backend must deny within AuthTimeout — fails on current code", "status": "pending", "blockedBy": [] },
|
{ "id": "R2-08-T1", "subject": "S2 stall-repro test (RED): slow LDAP backend must deny within AuthTimeout — fails on current code", "status": "completed", "blockedBy": [] },
|
||||||
{ "id": "R2-08-T2", "subject": "LdapOptions: ConnectionTimeoutMs/AuthTimeoutMs/MaxConcurrentAuthentications/OutageFailureThreshold/OutageCooldownSeconds + ToLibraryOptions projection + validator rules", "status": "pending", "blockedBy": [] },
|
{ "id": "R2-08-T2", "subject": "LdapOptions: ConnectionTimeoutMs/AuthTimeoutMs/MaxConcurrentAuthentications/OutageFailureThreshold/OutageCooldownSeconds + ToLibraryOptions projection + validator rules", "status": "completed", "blockedBy": [] },
|
||||||
{ "id": "R2-08-T3", "subject": "LdapAuthResult.IsSystemFailure seam + OtOpcUaLdapAuthService.Adapt mapping (ServiceAccountBindFailed => system-side)", "status": "pending", "blockedBy": [] },
|
{ "id": "R2-08-T3", "subject": "LdapAuthResult.IsSystemFailure seam + OtOpcUaLdapAuthService.Adapt mapping (ServiceAccountBindFailed => system-side)", "status": "completed", "blockedBy": [] },
|
||||||
{ "id": "R2-08-T4", "subject": "LdapOpcUaUserAuthenticator: WaitAsync wall-clock bound + SemaphoreSlim concurrency cap with release-on-core-completion (turns T1 green)", "status": "pending", "blockedBy": ["R2-08-T1", "R2-08-T2"] },
|
{ "id": "R2-08-T4", "subject": "LdapOpcUaUserAuthenticator: WaitAsync wall-clock bound + SemaphoreSlim concurrency cap with release-on-core-completion (turns T1 green)", "status": "completed", "blockedBy": ["R2-08-T1", "R2-08-T2"] },
|
||||||
{ "id": "R2-08-T5", "subject": "Directory-outage circuit: fail-fast deny after N consecutive system-side failures, half-open after cooldown", "status": "pending", "blockedBy": ["R2-08-T3", "R2-08-T4"] },
|
{ "id": "R2-08-T5", "subject": "Directory-outage circuit: fail-fast deny after N consecutive system-side failures, half-open after cooldown", "status": "completed", "blockedBy": ["R2-08-T3", "R2-08-T4"] },
|
||||||
{ "id": "R2-08-T6", "subject": "Outage-sim integration test (real library, unroutable host, via internal HandleImpersonation) + SDK m_eventLock contract doc + docs/security.md", "status": "pending", "blockedBy": ["R2-08-T4"] },
|
{ "id": "R2-08-T6", "subject": "Outage-sim integration test (real library, unroutable host, via internal HandleImpersonation) + SDK m_eventLock contract doc + docs/security.md", "status": "completed", "blockedBy": ["R2-08-T4"] },
|
||||||
{ "id": "R2-08-T7", "subject": "S3 sequential-serving repro test (RED): MaxObservedConcurrency >= 2 on an 8-node Raw batch — fails on current code", "status": "pending", "blockedBy": [] },
|
{ "id": "R2-08-T7", "subject": "S3 sequential-serving repro test (RED): MaxObservedConcurrency >= 2 on an 8-node Raw batch — fails on current code", "status": "completed", "blockedBy": [] },
|
||||||
{ "id": "R2-08-T8", "subject": "ServerHistorianOptions: HistoryReadBatchParallelism/MaxConcurrentHistoryReadBatches/HistoryReadDeadline + Validate warnings + node-manager properties + hosted-service wiring", "status": "pending", "blockedBy": [] },
|
{ "id": "R2-08-T8", "subject": "ServerHistorianOptions: HistoryReadBatchParallelism/MaxConcurrentHistoryReadBatches/HistoryReadDeadline + Validate warnings + node-manager properties + hosted-service wiring", "status": "completed", "blockedBy": [] },
|
||||||
{ "id": "R2-08-T9", "subject": "Async-ify ServeNode + Processed/AtTime arms (ct threading, OCE=>BadTimeout, behavior-neutral)", "status": "pending", "blockedBy": ["R2-08-T7", "R2-08-T8"] },
|
{ "id": "R2-08-T9", "subject": "Async-ify ServeNode + Processed/AtTime arms (ct threading, OCE=>BadTimeout, behavior-neutral)", "status": "completed", "blockedBy": ["R2-08-T7", "R2-08-T8"] },
|
||||||
{ "id": "R2-08-T10", "subject": "Async-ify ServeRawPaged (page read + tie-cluster over-fetch) + extract ServeEventsAsync", "status": "pending", "blockedBy": ["R2-08-T9"] },
|
{ "id": "R2-08-T10", "subject": "Async-ify ServeRawPaged (page read + tie-cluster over-fetch) + extract ServeEventsAsync", "status": "completed", "blockedBy": ["R2-08-T9"] },
|
||||||
{ "id": "R2-08-T11", "subject": "RunBounded fan-out in all four HistoryRead arms, single bridge per arm (turns T7 green) + upper-bound + isolation tests", "status": "pending", "blockedBy": ["R2-08-T9", "R2-08-T10"] },
|
{ "id": "R2-08-T11", "subject": "RunBounded fan-out in all four HistoryRead arms, single bridge per arm (turns T7 green) + upper-bound + isolation tests", "status": "completed", "blockedBy": ["R2-08-T9", "R2-08-T10"] },
|
||||||
{ "id": "R2-08-T12", "subject": "Per-request HistoryRead deadline CTS — hung backend surfaces BadTimeout per node", "status": "pending", "blockedBy": ["R2-08-T11"] },
|
{ "id": "R2-08-T12", "subject": "Per-request HistoryRead deadline CTS — hung backend surfaces BadTimeout per node", "status": "completed", "blockedBy": ["R2-08-T11"] },
|
||||||
{ "id": "R2-08-T13", "subject": "Process-wide concurrent-batch limiter — saturation degrades to BadTooManyOperations (lazy gate, bounded wait)", "status": "pending", "blockedBy": ["R2-08-T12"] },
|
{ "id": "R2-08-T13", "subject": "Process-wide concurrent-batch limiter — saturation degrades to BadTooManyOperations (lazy gate, bounded wait)", "status": "completed", "blockedBy": ["R2-08-T12"] },
|
||||||
{ "id": "R2-08-T14", "subject": "Docs (Historian.md + CLAUDE.md ServerHistorian table + security.md) + full touched-suite and whole-solution regression sweep + STATUS.md entry", "status": "pending", "blockedBy": ["R2-08-T5", "R2-08-T6", "R2-08-T13"] }
|
{ "id": "R2-08-T14", "subject": "Docs (Historian.md + CLAUDE.md ServerHistorian table + security.md) + full touched-suite and whole-solution regression sweep + STATUS.md entry", "status": "completed", "blockedBy": ["R2-08-T5", "R2-08-T6", "R2-08-T13"] }
|
||||||
],
|
],
|
||||||
"lastUpdated": "2026-07-12"
|
"lastUpdated": "2026-07-12"
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -176,6 +176,7 @@ what the unit leg cannot; (c) migrate the 3 xunit v2 holdouts once Akka ships an
|
|||||||
| **R2-02** | 01/S-6 (High, rank 2), 01/U-6, 01/S-8=03/S12 (rank 4), 04/C-7, 01/S-7=03/S13 | `r2/02-resilience-config-hardening` (off `1676c8f4`) | `4be13429`…`dd82e740` (17 task commits) | **Code-complete, offline-verified.** **S-6:** parser clamps timeout/retry/breaker overrides (timeout≤0→tier default, breaker==1→2, caps) with diagnostics + belt-and-braces `Build` clamp so pipeline construction can never throw Polly `ValidationException` (new `ResiliencePolicyRanges` in Core.Abstractions); brick-repro + `Build_NeverThrows_ForHostileDirectOptions` + factory-seam wiring proof; AdminUI `Validate()` range warnings. **S-8:** parser forces Write/AlarmAck retry→0 (spec invariant) **and** dispatch flips to `isIdempotent:false` (RED-first on the Runtime wiring test); non-idempotent arm caches the no-retry snapshot once/invoker (01/P-4) + tracker in-flight accounting; `WriteIdempotentAttribute` false claim corrected; retry cells disabled. **U-6:** bulkhead knob deleted end-to-end (migration-free; stored keys ignored) + `Options_properties_are_exactly_the_pipeline_wired_set` inertness guard. **C-7:** `ResilienceFormModel` JsonObject-bag round-trip preserves unknown keys; malformed JSON → `ParseFailed` lock + verbatim `ToJson` + discard button. **S-7:** per-`Create` `Interlocked` options-generation in the pipeline-cache key kills the respawn stale-pipeline race. Core resilience 121/121, AdminUI form 10/10, Analyzers 32/32, Runtime 364/364; 0 production OTOPCUA0001. **Deferred (T15/T18):** docker-dev `:9200` `/run` (warning banner, input lockout, unknown-key survival) + whole-solution sweep. |
|
| **R2-02** | 01/S-6 (High, rank 2), 01/U-6, 01/S-8=03/S12 (rank 4), 04/C-7, 01/S-7=03/S13 | `r2/02-resilience-config-hardening` (off `1676c8f4`) | `4be13429`…`dd82e740` (17 task commits) | **Code-complete, offline-verified.** **S-6:** parser clamps timeout/retry/breaker overrides (timeout≤0→tier default, breaker==1→2, caps) with diagnostics + belt-and-braces `Build` clamp so pipeline construction can never throw Polly `ValidationException` (new `ResiliencePolicyRanges` in Core.Abstractions); brick-repro + `Build_NeverThrows_ForHostileDirectOptions` + factory-seam wiring proof; AdminUI `Validate()` range warnings. **S-8:** parser forces Write/AlarmAck retry→0 (spec invariant) **and** dispatch flips to `isIdempotent:false` (RED-first on the Runtime wiring test); non-idempotent arm caches the no-retry snapshot once/invoker (01/P-4) + tracker in-flight accounting; `WriteIdempotentAttribute` false claim corrected; retry cells disabled. **U-6:** bulkhead knob deleted end-to-end (migration-free; stored keys ignored) + `Options_properties_are_exactly_the_pipeline_wired_set` inertness guard. **C-7:** `ResilienceFormModel` JsonObject-bag round-trip preserves unknown keys; malformed JSON → `ParseFailed` lock + verbatim `ToJson` + discard button. **S-7:** per-`Create` `Interlocked` options-generation in the pipeline-cache key kills the respawn stale-pipeline race. Core resilience 121/121, AdminUI form 10/10, Analyzers 32/32, Runtime 364/364; 0 production OTOPCUA0001. **Deferred (T15/T18):** docker-dev `:9200` `/run` (warning banner, input lockout, unknown-key survival) + whole-solution sweep. |
|
||||||
| **R2-10** | 03/U10=04/U-5 (Medium/Low, action #10) | `r2/10-resilience-observability` (off `46fedda3`) | `205d3980`…`<docs>` (11 task commits) | **Code-complete, offline-verified.** The resilience status tracker now has a production reader, mirroring the driver-health flow. Core truthfulness: `RecordBreakerClose` + derived `IsBreakerOpen` (breaker-open state derivable), `OnClosed` records the close, and `RecordSuccess` wired onto the invoker success paths (`ConsecutiveFailures` is a true consecutive counter, not stuck-forever). Transport: `DriverResilienceStatusChanged` Commons contract → periodic 5 s full-snapshot publisher `DriverResilienceStatusPublisherService` (driver nodes, registered in `AddOtOpcUaDriverFactories` with wiring guard `ResilienceStatusReaderRegistrationTests` — closes the built-but-never-wired class) → `driver-resilience-status` DPS topic → per-admin-node `DriverResilienceStatusBridge` → `IDriverResilienceStatusStore` → resilience chips (breaker-OPEN / consecutive-failures / in-flight / staleness) on `DriverStatusPanel.razor`, read in-process (self-HubConnection ban honored). Core.Tests resilience 31/31 (incl. deterministic breaker-close via a controllable TimeProvider), Host.IntegrationTests publisher+guard 6/6, AdminUI.Tests store 5/5; full-solution build clean. **Deferred (T10):** docker-dev live-`/run` — S7 dead-endpoint breaker-open observed on BOTH centrals + recovery-clear + staleness-dim (also the first live sighting of the #10 `circuit-breaker OPENED` log line, closing FOLLOWUP-10's behavioral gate). |
|
| **R2-10** | 03/U10=04/U-5 (Medium/Low, action #10) | `r2/10-resilience-observability` (off `46fedda3`) | `205d3980`…`<docs>` (11 task commits) | **Code-complete, offline-verified.** The resilience status tracker now has a production reader, mirroring the driver-health flow. Core truthfulness: `RecordBreakerClose` + derived `IsBreakerOpen` (breaker-open state derivable), `OnClosed` records the close, and `RecordSuccess` wired onto the invoker success paths (`ConsecutiveFailures` is a true consecutive counter, not stuck-forever). Transport: `DriverResilienceStatusChanged` Commons contract → periodic 5 s full-snapshot publisher `DriverResilienceStatusPublisherService` (driver nodes, registered in `AddOtOpcUaDriverFactories` with wiring guard `ResilienceStatusReaderRegistrationTests` — closes the built-but-never-wired class) → `driver-resilience-status` DPS topic → per-admin-node `DriverResilienceStatusBridge` → `IDriverResilienceStatusStore` → resilience chips (breaker-OPEN / consecutive-failures / in-flight / staleness) on `DriverStatusPanel.razor`, read in-process (self-HubConnection ban honored). Core.Tests resilience 31/31 (incl. deterministic breaker-close via a controllable TimeProvider), Host.IntegrationTests publisher+guard 6/6, AdminUI.Tests store 5/5; full-solution build clean. **Deferred (T10):** docker-dev live-`/run` — S7 dead-endpoint breaker-open observed on BOTH centrals + recovery-clear + staleness-dim (also the first live sighting of the #10 `circuit-breaker OPENED` log line, closing FOLLOWUP-10's behavioral gate). |
|
||||||
|
|
||||||
|
| **R2-08** | 03/S2 (High, action #8), 03/S3 (High) | `r2/08-ldap-historyread-async` (off `1a698cbb`) | `ce6d9e21`…`<docs>` (14 task commits) | **Code-complete, offline-verified.** **S2 (LDAP async/timeout):** the SDK raises the impersonation callback synchronously under a `SessionManager`-wide `m_eventLock` (verified vs. 1.5.378 — a hung bind serializes ALL activations), so the fix is entirely at the `LdapOpcUaUserAuthenticator` boundary: five new `Security:Ldap` keys (`ConnectionTimeoutMs`=10000 projected into the shared lib, `AuthTimeoutMs`=15000 hard whole-flow `WaitAsync` bound → fail-closed "timed out", `MaxConcurrentAuthentications`=8 `SemaphoreSlim` with release-on-core-completion so orphaned binds still count, `OutageFailureThreshold`=3/`OutageCooldownSeconds`=15 directory-outage circuit → fast-deny "unavailable", half-open probe) + `LdapAuthResult.IsSystemFailure` seam (mapped from `ServiceAccountBindFailed`) distinguishing outage from credential-deny + `LdapOptionsValidator` rules + `HandleImpersonation` contract doc + `docs/security.md`. Stall-repro RED→GREEN; offline outage-sim through the real `HandleImpersonation` against an unroutable host. **S3 (HistoryRead fan-out):** all four HistoryRead arms async-ified (`ServeNodeAsync`/`ServeRawPagedAsync`/`ServeEventsAsync`, OCE→`BadTimeout`) then bounded per-node fan-out (`RunBounded` semaphore + single arm-boundary bridge) + per-request deadline CTS (`HistoryReadDeadline`=60s) + process-wide lazy batch limiter (`MaxConcurrentHistoryReadBatches`=16 → flood degrades to `BadTooManyOperations`); 16×4=64 in-flight cap; three new `ServerHistorian` keys + node-manager props + hosted-service wiring + `docs/Historian.md`/CLAUDE.md tables. Sequential-serving RED→GREEN. Host.IntegrationTests (LDAP resilience) 13/13, Security.Tests 11/11, Runtime.Tests options 11/11, OpcUaServer.Tests HistoryRead 30/30. **Deferred (live):** S2 docker-dev unreachable-LDAP + GLAuth outage/recovery cycle; S3 `Category=LiveIntegration` gateway soak. |
|
||||||
| **R2-11** | 01/C-1, 01/P-1 + 05/CONV-2, UNDER-1, UNDER-6 | `r2/11-tagconfig-consolidation` (off `46fedda3`) | first `2e14fe1f` … (24 task commits) | **Code-complete, offline-verified.** **Part 1 (C-1/P-1):** single `TagConfigIntent.Parse` + `DeviceConfigIntent` in Commons replace the four byte-parity TagConfig parse copies (composer/artifact/walker/DraftValidator) — composer now parses once per tag (P-1); a permanent `Runtime.Tests/TagConfigCorpusParityTests` (30-blob corpus) guards compose≡decode; `grep "MUST parse identically" src/` = 0. **Part 2 (CONV-2):** shared strict-capable `TagConfigJson` readers in Core.Abstractions replace the six copy-pasted `ReadEnum`s; all six driver parsers gain `Inspect()` warnings + honour the `writable` key (FOCAS **forced read-only**, 05/UNDER-1); FOCAS equipment tags run the capability-matrix pre-flight (05/UNDER-6); Modbus probe parses the factory DTO shape (`timeoutMs` no longer dropped). New `EquipmentTagConfigInspector` in ControlPlane + `AdminOperationsActor` deploy-gate wiring with `Deployment:TagConfigValidationMode` (**Warn default | Error opt-in**), proven through the actor (F10b consuming test). AdminUI: `writable` checkbox in the six typed editors. **Phase-C follow-up (deferred, recorded here):** flip the runtime parsers to strict (typo'd enum ⇒ `BadNodeIdUnknown` instead of wrong-width `Good`) once fleets have run `Error` mode clean — behavior change deliberately not shipped in this plan. **Deferred (T22/T24):** docker-dev `/run` of one typed editor's Writable checkbox + whole-solution serial sweep. |
|
| **R2-11** | 01/C-1, 01/P-1 + 05/CONV-2, UNDER-1, UNDER-6 | `r2/11-tagconfig-consolidation` (off `46fedda3`) | first `2e14fe1f` … (24 task commits) | **Code-complete, offline-verified.** **Part 1 (C-1/P-1):** single `TagConfigIntent.Parse` + `DeviceConfigIntent` in Commons replace the four byte-parity TagConfig parse copies (composer/artifact/walker/DraftValidator) — composer now parses once per tag (P-1); a permanent `Runtime.Tests/TagConfigCorpusParityTests` (30-blob corpus) guards compose≡decode; `grep "MUST parse identically" src/` = 0. **Part 2 (CONV-2):** shared strict-capable `TagConfigJson` readers in Core.Abstractions replace the six copy-pasted `ReadEnum`s; all six driver parsers gain `Inspect()` warnings + honour the `writable` key (FOCAS **forced read-only**, 05/UNDER-1); FOCAS equipment tags run the capability-matrix pre-flight (05/UNDER-6); Modbus probe parses the factory DTO shape (`timeoutMs` no longer dropped). New `EquipmentTagConfigInspector` in ControlPlane + `AdminOperationsActor` deploy-gate wiring with `Deployment:TagConfigValidationMode` (**Warn default | Error opt-in**), proven through the actor (F10b consuming test). AdminUI: `writable` checkbox in the six typed editors. **Phase-C follow-up (deferred, recorded here):** flip the runtime parsers to strict (typo'd enum ⇒ `BadNodeIdUnknown` instead of wrong-width `Good`) once fleets have run `Error` mode clean — behavior change deliberately not shipped in this plan. **Deferred (T22/T24):** docker-dev `/run` of one typed editor's Writable checkbox + whole-solution serial sweep. |
|
||||||
|
|
||||||
**Report anchor drifts to fold into the next report refresh** (from the R2-02 verification pass): (1)
|
**Report anchor drifts to fold into the next report refresh** (from the R2-02 verification pass): (1)
|
||||||
|
|||||||
+7
-1
@@ -69,7 +69,10 @@ and all HistoryRead calls on historized nodes return `GoodNoData` (empty, not an
|
|||||||
"AllowUntrustedServerCertificate": false,
|
"AllowUntrustedServerCertificate": false,
|
||||||
"CaCertificatePath": null,
|
"CaCertificatePath": null,
|
||||||
"CallTimeout": "00:00:30",
|
"CallTimeout": "00:00:30",
|
||||||
"MaxTieClusterOverfetch": 65536
|
"MaxTieClusterOverfetch": 65536,
|
||||||
|
"HistoryReadBatchParallelism": 4,
|
||||||
|
"MaxConcurrentHistoryReadBatches": 16,
|
||||||
|
"HistoryReadDeadline": "00:01:00"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
@@ -84,6 +87,9 @@ and all HistoryRead calls on historized nodes return `GoodNoData` (empty, not an
|
|||||||
| `CaCertificatePath` | string\|null | `null` | PEM CA file pinning the gateway's TLS chain. Null/empty uses the OS trust store. |
|
| `CaCertificatePath` | string\|null | `null` | PEM CA file pinning the gateway's TLS chain. Null/empty uses the OS trust store. |
|
||||||
| `CallTimeout` | TimeSpan | `00:00:30` | Per-call deadline applied to each unary gateway read. A read for a tag the historian does not yet know about — e.g. a freshly-historized tag that has **not** been provisioned (see [Tag auto-provisioning](#tag-auto-provisioning-ensuretags)) — can run to this **full** deadline before the gateway errors, so an unprovisioned-tag HistoryRead can block for ~30 s. Lower this to fail faster, at the cost of truncating legitimately long reads. |
|
| `CallTimeout` | TimeSpan | `00:00:30` | Per-call deadline applied to each unary gateway read. A read for a tag the historian does not yet know about — e.g. a freshly-historized tag that has **not** been provisioned (see [Tag auto-provisioning](#tag-auto-provisioning-ensuretags)) — can run to this **full** deadline before the gateway errors, so an unprovisioned-tag HistoryRead can block for ~30 s. Lower this to fail faster, at the cost of truncating legitimately long reads. |
|
||||||
| `MaxTieClusterOverfetch` | int | `65536` | Maximum samples the server will fetch in one shot to page through a tie cluster (multiple samples sharing one `SourceTimestamp`). A cluster larger than this ceiling fails `BadHistoryOperationUnsupported`. Raise to handle abnormally large tie clusters; the default covers all normal-data cases. |
|
| `MaxTieClusterOverfetch` | int | `65536` | Maximum samples the server will fetch in one shot to page through a tie cluster (multiple samples sharing one `SourceTimestamp`). A cluster larger than this ceiling fails `BadHistoryOperationUnsupported`. Raise to handle abnormally large tie clusters; the default covers all normal-data cases. |
|
||||||
|
| `HistoryReadBatchParallelism` | int | `4` | Max historized nodes served **concurrently** within one HistoryRead batch (arch-review 03/S3). The SDK's `HistoryRead*` override surface is synchronous, so each arm bridges once at its boundary; this turns per-node serving from sequential (`N × CallTimeout` worst case) into `⌈N / P⌉ × CallTimeout`. Also caps the gateway-load multiplication a single client can command. `≤ 1` serves sequentially. |
|
||||||
|
| `MaxConcurrentHistoryReadBatches` | int | `16` | Process-wide limit on HistoryRead batches served **concurrently** (arch-review 03/S3). A flood of history clients degrades to `BadTooManyOperations` rather than exhausting the SDK request-thread pool. With `HistoryReadBatchParallelism` this caps total in-flight gateway reads at `MaxConcurrentHistoryReadBatches × HistoryReadBatchParallelism` (default 16 × 4 = 64). A batch that cannot get a slot within `min(HistoryReadDeadline, 5 s)` fails every handle `BadTooManyOperations`. |
|
||||||
|
| `HistoryReadDeadline` | TimeSpan | `00:01:00` | Per-request wall-clock deadline for a HistoryRead batch (arch-review 03/S3). A node still in flight at the deadline surfaces `BadTimeout`, so one request can never hold an SDK request thread longer than this regardless of node count. Non-positive = unbounded (a `Validate()` warning). |
|
||||||
|
|
||||||
> **Do not commit `ApiKey` to `appsettings.json`.** Set it via the environment variable
|
> **Do not commit `ApiKey` to `appsettings.json`.** Set it via the environment variable
|
||||||
> `ServerHistorian__ApiKey`, a secrets store, or a deployment-time overlay. The checked-in default is
|
> `ServerHistorian__ApiKey`, a secrets store, or a deployment-time overlay. The checked-in default is
|
||||||
|
|||||||
@@ -162,6 +162,20 @@ LDAP is configured under the `Security:Ldap` section (bound to `LdapOptions`, `s
|
|||||||
|
|
||||||
`GroupToRole` maps LDAP group names → roles (case-insensitive); a user gets every role whose source group is in their membership. The values are the canonical control-plane role strings (`Viewer` / `Designer` / `Administrator`, plus the appsettings-only `Operator` for the `DriverOperator` policy); the same map also supplies data-plane role strings (`ReadOnly`, `WriteOperate`, `WriteTune`, `WriteConfigure`, `AlarmAck`) — see [Role grant source (data-plane)](#role-grant-source-data-plane) below. `UserNameAttribute: "sAMAccountName"` is the critical AD override — the GLAuth dev default is `cn`, which is not how AD users are looked up; use `userPrincipalName` instead if operators log in with `user@corp.example.com` form. `LdapOptionsValidator` (`src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/LdapOptionsValidator.cs`) fails startup when `Transport = None` and `AllowInsecure = false` on a real-LDAP (non-DevStub) config.
|
`GroupToRole` maps LDAP group names → roles (case-insensitive); a user gets every role whose source group is in their membership. The values are the canonical control-plane role strings (`Viewer` / `Designer` / `Administrator`, plus the appsettings-only `Operator` for the `DriverOperator` policy); the same map also supplies data-plane role strings (`ReadOnly`, `WriteOperate`, `WriteTune`, `WriteConfigure`, `AlarmAck`) — see [Role grant source (data-plane)](#role-grant-source-data-plane) below. `UserNameAttribute: "sAMAccountName"` is the critical AD override — the GLAuth dev default is `cn`, which is not how AD users are looked up; use `userPrincipalName` instead if operators log in with `user@corp.example.com` form. `LdapOptionsValidator` (`src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/LdapOptionsValidator.cs`) fails startup when `Transport = None` and `AllowInsecure = false` on a real-LDAP (non-DevStub) config.
|
||||||
|
|
||||||
|
### LDAP resilience — timeout, concurrency, outage circuit (arch-review 03/S2)
|
||||||
|
|
||||||
|
The OPC UA SDK raises its impersonation (UserName) callback **synchronously**, inside a `SessionManager`-wide event lock (verified against `OPCFoundation.NetStandard.Opc.Ua.Server` 1.5.378). A slow or hung LDAP bind inside that callback therefore does not stall one SDK thread — it **serializes every session activation server-wide** (Anonymous and X509 included). Since the callback is irreducibly synchronous, `LdapOpcUaUserAuthenticator` bounds it at its own boundary with five `Security:Ldap` keys (all validated at startup by `LdapOptionsValidator` when `Enabled && !DevStubMode`):
|
||||||
|
|
||||||
|
| Key | Default | Bounds |
|
||||||
|
|---|---|---|
|
||||||
|
| `ConnectionTimeoutMs` | `10000` | Projected into the shared library's `ConnectionTimeoutMs` — bounds the socket connect **and** each per-operation search. First line of defence against an outage. |
|
||||||
|
| `AuthTimeoutMs` | `15000` | Hard whole-flow wall-clock bound (bind **and** role-map) enforced at the authenticator boundary; a slow/hung flow denies **"Authentication timed out"** (fail-closed) and releases the SDK thread. Deliberately `> ConnectionTimeoutMs` so a healthy-but-slow flow is not cut off. |
|
||||||
|
| `MaxConcurrentAuthentications` | `8` | Max in-flight authentications; excess denies **"Authentication backend busy"** (local backpressure — does **not** feed the circuit). Bounds accumulation of orphaned/timed-out binds during an outage. |
|
||||||
|
| `OutageFailureThreshold` | `3` | Consecutive **system-side** failures (boundary timeout, directory-unreachable, or unexpected throw) that open the outage circuit. Any success or user-credential deny resets the streak. `0` disables the circuit. |
|
||||||
|
| `OutageCooldownSeconds` | `15` | How long the open circuit fast-denies **"Authentication backend unavailable"** without touching LDAP; after cooldown the next call half-open-probes, and one more system failure re-opens immediately. |
|
||||||
|
|
||||||
|
Defaults are behaviour-neutral on a healthy directory. All new paths are **fail-closed** (they deny). A user-credential deny (wrong password) is fast when the directory is up and never trips the circuit; only a directory *outage* opens it.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Data-Plane Authorization
|
## Data-Plane Authorization
|
||||||
|
|||||||
@@ -47,5 +47,18 @@ public sealed class LdapOptionsValidator : OptionsValidatorBase<LdapOptions>
|
|||||||
builder.RequireThat(
|
builder.RequireThat(
|
||||||
!(options.Transport == LdapTransport.None && !options.AllowInsecure),
|
!(options.Transport == LdapTransport.None && !options.AllowInsecure),
|
||||||
"LDAP transport is None (plaintext) but AllowInsecure is false — set Transport to Ldaps/StartTls or set AllowInsecure for dev.");
|
"LDAP transport is None (plaintext) but AllowInsecure is false — set Transport to Ldaps/StartTls or set AllowInsecure for dev.");
|
||||||
|
|
||||||
|
// S2 resilience knobs — a non-positive timeout / concurrency bound would silently disable the
|
||||||
|
// wall-clock/concurrency protection the OPC UA data-plane authenticator relies on, so fail fast.
|
||||||
|
builder.RequireThat(options.ConnectionTimeoutMs > 0,
|
||||||
|
"Ldap:ConnectionTimeoutMs must be > 0.");
|
||||||
|
builder.RequireThat(options.AuthTimeoutMs > 0,
|
||||||
|
"Ldap:AuthTimeoutMs must be > 0.");
|
||||||
|
builder.RequireThat(options.MaxConcurrentAuthentications > 0,
|
||||||
|
"Ldap:MaxConcurrentAuthentications must be > 0.");
|
||||||
|
// An open circuit with no cooldown would re-probe LDAP on every call — pointless. Only enforce
|
||||||
|
// a positive cooldown when the circuit is actually enabled (OutageFailureThreshold > 0; 0 disables).
|
||||||
|
builder.RequireThat(options.OutageFailureThreshold <= 0 || options.OutageCooldownSeconds > 0,
|
||||||
|
"Ldap:OutageCooldownSeconds must be > 0 when Ldap:OutageFailureThreshold > 0 (0 disables the circuit).");
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,5 +1,6 @@
|
|||||||
using Microsoft.Extensions.DependencyInjection;
|
using Microsoft.Extensions.DependencyInjection;
|
||||||
using Microsoft.Extensions.Logging;
|
using Microsoft.Extensions.Logging;
|
||||||
|
using Microsoft.Extensions.Options;
|
||||||
using ZB.MOM.WW.Auth.Abstractions.Roles;
|
using ZB.MOM.WW.Auth.Abstractions.Roles;
|
||||||
using ZB.MOM.WW.OtOpcUa.OpcUaServer.Security;
|
using ZB.MOM.WW.OtOpcUa.OpcUaServer.Security;
|
||||||
using ZB.MOM.WW.OtOpcUa.Security.Ldap;
|
using ZB.MOM.WW.OtOpcUa.Security.Ldap;
|
||||||
@@ -16,37 +17,215 @@ namespace ZB.MOM.WW.OtOpcUa.Host.OpcUa;
|
|||||||
/// session identity for the downstream data-plane ACL evaluator.
|
/// session identity for the downstream data-plane ACL evaluator.
|
||||||
/// </summary>
|
/// </summary>
|
||||||
/// <remarks>
|
/// <remarks>
|
||||||
|
/// <para>
|
||||||
/// This authenticator is registered as a singleton, but <see cref="IGroupRoleMapper{TRole}"/>
|
/// This authenticator is registered as a singleton, but <see cref="IGroupRoleMapper{TRole}"/>
|
||||||
/// (and its DbContext-backed mapping service) is scoped. A per-call DI scope is opened to
|
/// (and its DbContext-backed mapping service) is scoped. A per-call DI scope is opened to
|
||||||
/// resolve the mapper so the singleton never captures a scoped dependency.
|
/// resolve the mapper so the singleton never captures a scoped dependency.
|
||||||
|
/// </para>
|
||||||
|
/// <para>
|
||||||
|
/// <b>Resilience (arch-review 03/S2).</b> The OPC UA SDK raises the impersonation callback that
|
||||||
|
/// reaches here <em>synchronously</em>, inside a <c>SessionManager</c>-wide event lock (verified
|
||||||
|
/// against <c>OPCFoundation.NetStandard.Opc.Ua.Server</c> 1.5.378 —
|
||||||
|
/// <c>SessionManager.ActivateSessionAsync</c> holds <c>m_eventLock</c> while invoking the
|
||||||
|
/// <c>void</c> impersonation delegate). A slow / hung LDAP bind inside that callback therefore does
|
||||||
|
/// not merely stall one SDK thread — it serializes <em>every</em> session activation server-wide
|
||||||
|
/// (Anonymous and X509 included). The callback contract is irreducibly synchronous (there is no
|
||||||
|
/// async impersonation callback), so the fix lives entirely at this boundary: a hard whole-flow
|
||||||
|
/// wall-clock bound (<see cref="LdapOptions.AuthTimeoutMs"/>) and a bounded in-flight concurrency cap
|
||||||
|
/// (<see cref="LdapOptions.MaxConcurrentAuthentications"/>), both fail-closed. All new paths deny.
|
||||||
|
/// </para>
|
||||||
/// </remarks>
|
/// </remarks>
|
||||||
public sealed class LdapOpcUaUserAuthenticator(
|
public sealed class LdapOpcUaUserAuthenticator : IOpcUaUserAuthenticator
|
||||||
ILdapAuthService ldap,
|
|
||||||
IServiceScopeFactory scopeFactory,
|
|
||||||
ILogger<LdapOpcUaUserAuthenticator> logger)
|
|
||||||
: IOpcUaUserAuthenticator
|
|
||||||
{
|
{
|
||||||
|
private readonly ILdapAuthService _ldap;
|
||||||
|
private readonly IServiceScopeFactory _scopeFactory;
|
||||||
|
private readonly ILogger<LdapOpcUaUserAuthenticator> _logger;
|
||||||
|
private readonly LdapOptions _options;
|
||||||
|
|
||||||
|
/// <summary>Bounds in-flight authentication flows. A boundary timeout abandons its core task, which
|
||||||
|
/// keeps holding its slot until it actually completes — so an outage cannot accumulate unbounded
|
||||||
|
/// orphan binds.</summary>
|
||||||
|
private readonly SemaphoreSlim _gate;
|
||||||
|
|
||||||
|
/// <summary>Guards the outage-circuit state (<see cref="_consecutiveSystemFailures"/> +
|
||||||
|
/// <see cref="_circuitOpenUntilTicks"/>).</summary>
|
||||||
|
private readonly object _circuitLock = new();
|
||||||
|
|
||||||
|
/// <summary>Count of CONSECUTIVE system-side failures; reset by any success / user-side deny.</summary>
|
||||||
|
private int _consecutiveSystemFailures;
|
||||||
|
|
||||||
|
/// <summary><see cref="Environment.TickCount64"/> the circuit stays open until; <c>0</c> = closed.</summary>
|
||||||
|
private long _circuitOpenUntilTicks;
|
||||||
|
|
||||||
|
/// <summary>Constructs the authenticator from its dependencies and the bound LDAP options.</summary>
|
||||||
|
/// <param name="ldap">The app LDAP auth service (shared with the Admin UI login flow).</param>
|
||||||
|
/// <param name="scopeFactory">Opens a per-call DI scope to resolve the scoped role mapper.</param>
|
||||||
|
/// <param name="options">The bound <see cref="LdapOptions"/> (timeout + concurrency knobs).</param>
|
||||||
|
/// <param name="logger">The logger.</param>
|
||||||
|
public LdapOpcUaUserAuthenticator(
|
||||||
|
ILdapAuthService ldap,
|
||||||
|
IServiceScopeFactory scopeFactory,
|
||||||
|
IOptions<LdapOptions> options,
|
||||||
|
ILogger<LdapOpcUaUserAuthenticator> logger)
|
||||||
|
{
|
||||||
|
_ldap = ldap;
|
||||||
|
_scopeFactory = scopeFactory;
|
||||||
|
_options = options.Value;
|
||||||
|
_logger = logger;
|
||||||
|
_gate = new SemaphoreSlim(Math.Max(1, _options.MaxConcurrentAuthentications));
|
||||||
|
}
|
||||||
|
|
||||||
/// <inheritdoc />
|
/// <inheritdoc />
|
||||||
public async Task<OpcUaUserAuthResult> AuthenticateUserNameAsync(string username, string password, CancellationToken ct)
|
public async Task<OpcUaUserAuthResult> AuthenticateUserNameAsync(string username, string password, CancellationToken ct)
|
||||||
{
|
{
|
||||||
|
var authTimeout = TimeSpan.FromMilliseconds(Math.Max(1, _options.AuthTimeoutMs));
|
||||||
|
|
||||||
|
// Directory-outage circuit: after a run of consecutive system-side failures, fail fast WITHOUT
|
||||||
|
// touching LDAP for the cooldown window — so a sustained outage cannot serialize every session
|
||||||
|
// activation behind a per-call stall (the SDK raises this callback under a server-wide lock).
|
||||||
|
if (IsCircuitOpen())
|
||||||
|
{
|
||||||
|
_logger.LogDebug(
|
||||||
|
"LDAP outage circuit open — fast-denying OPC UA user {User} without a directory call", username);
|
||||||
|
return OpcUaUserAuthResult.Deny("Authentication backend unavailable");
|
||||||
|
}
|
||||||
|
|
||||||
|
// Bounded in-flight concurrency: acquire a slot within our own wall-clock budget. All slots
|
||||||
|
// busy is LOCAL backpressure, not directory evidence — deny "busy" (fail-closed) and do NOT
|
||||||
|
// feed the outage circuit.
|
||||||
|
if (!await _gate.WaitAsync(authTimeout, ct).ConfigureAwait(false))
|
||||||
|
{
|
||||||
|
_logger.LogWarning(
|
||||||
|
"LDAP authentication backend busy (concurrency cap {Cap}) — denying OPC UA user {User}",
|
||||||
|
_options.MaxConcurrentAuthentications, username);
|
||||||
|
return OpcUaUserAuthResult.Deny("Authentication backend busy");
|
||||||
|
}
|
||||||
|
|
||||||
|
// Release the slot when the CORE task completes — deliberately NOT in a finally here. A boundary
|
||||||
|
// timeout below abandons the core task (it runs to completion in the background; the underlying
|
||||||
|
// library is documented never-throw, so there are no unobserved exceptions), and it must keep
|
||||||
|
// counting against the cap until it finishes. The point of the boundary is to release the SDK
|
||||||
|
// thread (and m_eventLock), not to cancel the bind.
|
||||||
|
var core = AuthenticateCoreAsync(username, password, ct);
|
||||||
|
_ = core.ContinueWith(
|
||||||
|
static (_, state) => ((SemaphoreSlim)state!).Release(),
|
||||||
|
_gate,
|
||||||
|
CancellationToken.None,
|
||||||
|
TaskContinuationOptions.None,
|
||||||
|
TaskScheduler.Default);
|
||||||
|
|
||||||
try
|
try
|
||||||
{
|
{
|
||||||
var result = await ldap.AuthenticateAsync(username, password, ct).ConfigureAwait(false);
|
var outcome = await core.WaitAsync(authTimeout, ct).ConfigureAwait(false);
|
||||||
|
RecordOutcome(outcome.SystemFailure, username);
|
||||||
|
return outcome.Result;
|
||||||
|
}
|
||||||
|
catch (TimeoutException)
|
||||||
|
{
|
||||||
|
// A boundary timeout is directory evidence — it feeds the circuit.
|
||||||
|
RecordOutcome(systemFailure: true, username);
|
||||||
|
_logger.LogWarning(
|
||||||
|
"LDAP authentication timed out after {Timeout} for OPC UA user {User} — denying (fail-closed)",
|
||||||
|
authTimeout, username);
|
||||||
|
return OpcUaUserAuthResult.Deny("Authentication timed out");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>Whether the outage circuit is currently open (fast-deny window active). Always false when
|
||||||
|
/// the circuit is disabled (<see cref="LdapOptions.OutageFailureThreshold"/> ≤ 0).</summary>
|
||||||
|
private bool IsCircuitOpen()
|
||||||
|
{
|
||||||
|
if (_options.OutageFailureThreshold <= 0) return false;
|
||||||
|
lock (_circuitLock)
|
||||||
|
{
|
||||||
|
return _circuitOpenUntilTicks > Environment.TickCount64;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Updates the outage-circuit state from one authentication outcome. A system-side failure advances
|
||||||
|
/// the consecutive-failure streak and opens the circuit once it reaches
|
||||||
|
/// <see cref="LdapOptions.OutageFailureThreshold"/>; any non-system outcome (success or user-side
|
||||||
|
/// deny) resets the streak and closes the circuit. Half-open behaviour is implicit: once the cooldown
|
||||||
|
/// elapses <see cref="IsCircuitOpen"/> lets the next call through with the streak still at threshold,
|
||||||
|
/// so a single further system failure re-opens immediately.
|
||||||
|
/// </summary>
|
||||||
|
/// <param name="systemFailure">Whether this outcome was a system-side (directory) failure.</param>
|
||||||
|
/// <param name="username">The login name, for diagnostics.</param>
|
||||||
|
private void RecordOutcome(bool systemFailure, string username)
|
||||||
|
{
|
||||||
|
if (_options.OutageFailureThreshold <= 0) return; // circuit disabled
|
||||||
|
|
||||||
|
lock (_circuitLock)
|
||||||
|
{
|
||||||
|
if (systemFailure)
|
||||||
|
{
|
||||||
|
_consecutiveSystemFailures++;
|
||||||
|
if (_consecutiveSystemFailures >= _options.OutageFailureThreshold)
|
||||||
|
{
|
||||||
|
var wasOpen = _circuitOpenUntilTicks > Environment.TickCount64;
|
||||||
|
_circuitOpenUntilTicks =
|
||||||
|
Environment.TickCount64 + (long)Math.Max(1, _options.OutageCooldownSeconds) * 1000L;
|
||||||
|
if (!wasOpen)
|
||||||
|
{
|
||||||
|
_logger.LogWarning(
|
||||||
|
"LDAP outage circuit OPENED after {Failures} consecutive system-side failures — " +
|
||||||
|
"fast-denying for {Cooldown}s (last user {User})",
|
||||||
|
_consecutiveSystemFailures, _options.OutageCooldownSeconds, username);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
if (_consecutiveSystemFailures >= _options.OutageFailureThreshold)
|
||||||
|
{
|
||||||
|
_logger.LogInformation(
|
||||||
|
"LDAP outage circuit CLOSED — directory recovered (probe succeeded for {User})", username);
|
||||||
|
}
|
||||||
|
_consecutiveSystemFailures = 0;
|
||||||
|
_circuitOpenUntilTicks = 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// The unbounded core authenticate flow (bind + role-map) that the outer
|
||||||
|
/// <see cref="AuthenticateUserNameAsync"/> wraps in a wall-clock bound + concurrency cap. Fail-closed:
|
||||||
|
/// any backend throw becomes a denial rather than escaping into the SDK. The returned
|
||||||
|
/// <see cref="CoreOutcome.SystemFailure"/> flags directory-side faults (a system-side
|
||||||
|
/// <see cref="LdapAuthResult.IsSystemFailure"/> deny, or an unexpected throw) so the caller can feed
|
||||||
|
/// the outage circuit; a user-credential deny is NOT a system failure.
|
||||||
|
/// </summary>
|
||||||
|
/// <param name="username">The username to authenticate.</param>
|
||||||
|
/// <param name="password">The cleartext password.</param>
|
||||||
|
/// <param name="ct">Cancellation token.</param>
|
||||||
|
private async Task<CoreOutcome> AuthenticateCoreAsync(string username, string password, CancellationToken ct)
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var result = await _ldap.AuthenticateAsync(username, password, ct).ConfigureAwait(false);
|
||||||
if (!result.Success)
|
if (!result.Success)
|
||||||
{
|
{
|
||||||
return OpcUaUserAuthResult.Deny(result.Error ?? "Invalid credentials");
|
return new CoreOutcome(
|
||||||
|
OpcUaUserAuthResult.Deny(result.Error ?? "Invalid credentials"), result.IsSystemFailure);
|
||||||
}
|
}
|
||||||
|
|
||||||
var roles = await ResolveRolesAsync(result.Groups, result.Roles, username, ct).ConfigureAwait(false);
|
var roles = await ResolveRolesAsync(result.Groups, result.Roles, username, ct).ConfigureAwait(false);
|
||||||
return OpcUaUserAuthResult.Allow(result.DisplayName ?? username, roles);
|
return new CoreOutcome(OpcUaUserAuthResult.Allow(result.DisplayName ?? username, roles), SystemFailure: false);
|
||||||
}
|
}
|
||||||
catch (Exception ex) when (ex is not OperationCanceledException)
|
catch (Exception ex) when (ex is not OperationCanceledException)
|
||||||
{
|
{
|
||||||
logger.LogWarning(ex, "LDAP authentication threw for OPC UA user {User}", username);
|
_logger.LogWarning(ex, "LDAP authentication threw for OPC UA user {User}", username);
|
||||||
return OpcUaUserAuthResult.Deny("Authentication backend error");
|
return new CoreOutcome(OpcUaUserAuthResult.Deny("Authentication backend error"), SystemFailure: true);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>The result of the core authenticate flow plus whether it was a system-side (directory)
|
||||||
|
/// failure — the signal that drives the outage circuit.</summary>
|
||||||
|
/// <param name="Result">The auth result to return to the SDK.</param>
|
||||||
|
/// <param name="SystemFailure">Whether the outcome was a directory-side fault.</param>
|
||||||
|
private readonly record struct CoreOutcome(OpcUaUserAuthResult Result, bool SystemFailure);
|
||||||
|
|
||||||
/// <summary>
|
/// <summary>
|
||||||
/// Resolves the user's roles from their LDAP groups via the scoped
|
/// Resolves the user's roles from their LDAP groups via the scoped
|
||||||
/// <see cref="IGroupRoleMapper{TRole}"/>, unioned with any pre-resolved roles (the DevStub
|
/// <see cref="IGroupRoleMapper{TRole}"/>, unioned with any pre-resolved roles (the DevStub
|
||||||
@@ -62,7 +241,7 @@ public sealed class LdapOpcUaUserAuthenticator(
|
|||||||
{
|
{
|
||||||
try
|
try
|
||||||
{
|
{
|
||||||
await using var scope = scopeFactory.CreateAsyncScope();
|
await using var scope = _scopeFactory.CreateAsyncScope();
|
||||||
var mapper = scope.ServiceProvider.GetRequiredService<IGroupRoleMapper<string>>();
|
var mapper = scope.ServiceProvider.GetRequiredService<IGroupRoleMapper<string>>();
|
||||||
var mapping = await mapper.MapAsync(groups, ct).ConfigureAwait(false);
|
var mapping = await mapper.MapAsync(groups, ct).ConfigureAwait(false);
|
||||||
|
|
||||||
@@ -73,7 +252,7 @@ public sealed class LdapOpcUaUserAuthenticator(
|
|||||||
}
|
}
|
||||||
catch (Exception ex) when (ex is not OperationCanceledException)
|
catch (Exception ex) when (ex is not OperationCanceledException)
|
||||||
{
|
{
|
||||||
logger.LogWarning(ex,
|
_logger.LogWarning(ex,
|
||||||
"Role-map lookup failed for OPC UA user {User}; using pre-resolved baseline roles", username);
|
"Role-map lookup failed for OPC UA user {User}; using pre-resolved baseline roles", username);
|
||||||
return preResolved;
|
return preResolved;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -222,6 +222,13 @@ public sealed class OtOpcUaServerHostedService : IHostedService, IAsyncDisposabl
|
|||||||
// section is absent; NodeManager is non-null here (guarded above).
|
// section is absent; NodeManager is non-null here (guarded above).
|
||||||
_server.NodeManager.MaxTieClusterOverfetch = _serverHistorianOptions.MaxTieClusterOverfetch;
|
_server.NodeManager.MaxTieClusterOverfetch = _serverHistorianOptions.MaxTieClusterOverfetch;
|
||||||
|
|
||||||
|
// Propagate the S3 HistoryRead concurrency bounds (arch-review 03/S3): bounded per-node fan-out
|
||||||
|
// within a batch, the process-wide concurrent-batch limiter, and the per-request deadline. Defaults
|
||||||
|
// (4 / 16 / 60 s) survive when the ServerHistorian section is absent.
|
||||||
|
_server.NodeManager.HistoryReadBatchParallelism = _serverHistorianOptions.HistoryReadBatchParallelism;
|
||||||
|
_server.NodeManager.MaxConcurrentHistoryReadBatches = _serverHistorianOptions.MaxConcurrentHistoryReadBatches;
|
||||||
|
_server.NodeManager.HistoryReadDeadline = _serverHistorianOptions.HistoryReadDeadline;
|
||||||
|
|
||||||
// ServiceLevel publisher needs IServerInternal — only available after Start.
|
// ServiceLevel publisher needs IServerInternal — only available after Start.
|
||||||
if (_server.CurrentInstance is { } serverInternal)
|
if (_server.CurrentInstance is { } serverInternal)
|
||||||
{
|
{
|
||||||
|
|||||||
@@ -235,6 +235,19 @@ public sealed class OpcUaApplicationHost : IAsyncDisposable
|
|||||||
/// the full SDK. Side-effects are confined to mutating <see cref="ImpersonateEventArgs"/>
|
/// the full SDK. Side-effects are confined to mutating <see cref="ImpersonateEventArgs"/>
|
||||||
/// and logging.
|
/// and logging.
|
||||||
/// </summary>
|
/// </summary>
|
||||||
|
/// <remarks>
|
||||||
|
/// <b>SDK threading contract (verified against OPCFoundation.NetStandard.Opc.Ua.Server 1.5.378).</b>
|
||||||
|
/// The SDK raises this handler SYNCHRONOUSLY — it is a plain <c>void</c> delegate, and there is no
|
||||||
|
/// async impersonation callback in this SDK version. Worse, <c>SessionManager.ActivateSessionAsync</c>
|
||||||
|
/// invokes it while holding a single <c>SessionManager</c>-wide event lock (<c>m_eventLock</c>), so a
|
||||||
|
/// handler that blocks does not merely stall one SDK request thread — it SERIALIZES every session
|
||||||
|
/// activation server-wide (Anonymous and X509 included, since even those paths must take the lock to
|
||||||
|
/// raise the handler). Consequently the <see cref="IOpcUaUserAuthenticator"/> block-bridge below
|
||||||
|
/// (<c>GetAwaiter().GetResult()</c> under <see cref="CancellationToken.None"/>) is irreducible: the
|
||||||
|
/// callback must complete inline. Because the SDK gives no timeout, the hard wall-clock bound + bounded
|
||||||
|
/// concurrency + directory-outage circuit that keep a slow/hung directory from wedging the SDK live
|
||||||
|
/// inside the authenticator (<c>LdapOpcUaUserAuthenticator</c>, arch-review 03/S2) — NOT here.
|
||||||
|
/// </remarks>
|
||||||
/// <param name="authenticator">The user authenticator to validate credentials.</param>
|
/// <param name="authenticator">The user authenticator to validate credentials.</param>
|
||||||
/// <param name="args">The impersonation event arguments to process.</param>
|
/// <param name="args">The impersonation event arguments to process.</param>
|
||||||
/// <param name="logger">The logger for diagnostic output.</param>
|
/// <param name="logger">The logger for diagnostic output.</param>
|
||||||
|
|||||||
@@ -197,6 +197,34 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
|||||||
/// </summary>
|
/// </summary>
|
||||||
public int MaxTieClusterOverfetch { get; set; } = 65536;
|
public int MaxTieClusterOverfetch { get; set; } = 65536;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Maximum historized nodes served concurrently within one HistoryRead batch (arch-review 03/S3).
|
||||||
|
/// Mirrors <c>ServerHistorianOptions.HistoryReadBatchParallelism</c>; the Host sets it at
|
||||||
|
/// <c>StartAsync</c>. <c>≤ 1</c> falls back to sequential serving. The default (4) survives when the
|
||||||
|
/// ServerHistorian section is absent.
|
||||||
|
/// </summary>
|
||||||
|
public int HistoryReadBatchParallelism { get; set; } = 4;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Process-wide cap on concurrently-served HistoryRead batches (arch-review 03/S3). A batch that
|
||||||
|
/// cannot acquire a slot within its (bounded) wait budget fast-fails every handle with
|
||||||
|
/// <c>BadTooManyOperations</c>. Mirrors <c>ServerHistorianOptions.MaxConcurrentHistoryReadBatches</c>;
|
||||||
|
/// the Host sets it at <c>StartAsync</c>. The default (16) survives when the section is absent.
|
||||||
|
/// </summary>
|
||||||
|
public int MaxConcurrentHistoryReadBatches { get; set; } = 16;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Per-request wall-clock deadline for a HistoryRead batch (arch-review 03/S3). A node still in
|
||||||
|
/// flight at the deadline surfaces <c>BadTimeout</c>. Non-positive = unbounded. Mirrors
|
||||||
|
/// <c>ServerHistorianOptions.HistoryReadDeadline</c>; the Host sets it at <c>StartAsync</c>. The
|
||||||
|
/// default (60 s) survives when the section is absent.
|
||||||
|
/// </summary>
|
||||||
|
public TimeSpan HistoryReadDeadline { get; set; } = TimeSpan.FromSeconds(60);
|
||||||
|
|
||||||
|
/// <summary>Lazily-created process-wide HistoryRead-batch limiter (see <see cref="MaxConcurrentHistoryReadBatches"/>).</summary>
|
||||||
|
private volatile SemaphoreSlim? _historyReadBatchLimiter;
|
||||||
|
private readonly object _historyReadBatchLimiterLock = new();
|
||||||
|
|
||||||
private volatile IHistoryContinuationStore _historyContinuationStore = new SessionHistoryContinuationStore();
|
private volatile IHistoryContinuationStore _historyContinuationStore = new SessionHistoryContinuationStore();
|
||||||
|
|
||||||
/// <summary>
|
/// <summary>
|
||||||
@@ -1792,19 +1820,28 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
|||||||
IDictionary<NodeId, NodeState> cache)
|
IDictionary<NodeId, NodeState> cache)
|
||||||
{
|
{
|
||||||
var session = context.OperationContext?.Session;
|
var session = context.OperationContext?.Session;
|
||||||
foreach (var handle in nodesToProcess)
|
WithHistoryReadBatchLimiter(nodesToProcess, results, errors, () =>
|
||||||
{
|
{
|
||||||
if (details.IsReadModified)
|
using var deadline = CreateDeadline();
|
||||||
|
var ct = deadline?.Token ?? CancellationToken.None;
|
||||||
|
var work = new List<Func<Task>>(nodesToProcess.Count);
|
||||||
|
foreach (var handle in nodesToProcess)
|
||||||
{
|
{
|
||||||
// We never serve modified-value history; mark this node unsupported and move on.
|
if (details.IsReadModified)
|
||||||
errors[handle.Index] = StatusCodes.BadHistoryOperationUnsupported;
|
{
|
||||||
continue;
|
// We never serve modified-value history; mark this node unsupported and move on.
|
||||||
|
errors[handle.Index] = StatusCodes.BadHistoryOperationUnsupported;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
work.Add(() => ServeRawPagedAsync(
|
||||||
|
handle, session, nodesToRead, results, errors,
|
||||||
|
details.StartTime, details.EndTime, details.NumValuesPerNode,
|
||||||
|
ct));
|
||||||
}
|
}
|
||||||
|
|
||||||
ServeRawPaged(
|
RunBounded(work, HistoryReadBatchParallelism);
|
||||||
handle, session, nodesToRead, results, errors,
|
});
|
||||||
details.StartTime, details.EndTime, details.NumValuesPerNode);
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/// <inheritdoc />
|
/// <inheritdoc />
|
||||||
@@ -1820,31 +1857,40 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
|||||||
{
|
{
|
||||||
// OPC UA ProcessingInterval is a Duration in milliseconds — convert once per batch.
|
// OPC UA ProcessingInterval is a Duration in milliseconds — convert once per batch.
|
||||||
var interval = TimeSpan.FromMilliseconds(details.ProcessingInterval);
|
var interval = TimeSpan.FromMilliseconds(details.ProcessingInterval);
|
||||||
foreach (var handle in nodesToProcess)
|
WithHistoryReadBatchLimiter(nodesToProcess, results, errors, () =>
|
||||||
{
|
{
|
||||||
// AggregateType is a per-node parallel collection (same length as nodesToRead, enforced by
|
using var deadline = CreateDeadline();
|
||||||
// the base dispatcher). handle.Index is the node's position in that collection.
|
var ct = deadline?.Token ?? CancellationToken.None;
|
||||||
var aggregateNodeId = details.AggregateType[handle.Index];
|
var work = new List<Func<Task>>(nodesToProcess.Count);
|
||||||
var aggregate = MapAggregate(aggregateNodeId);
|
foreach (var handle in nodesToProcess)
|
||||||
if (aggregate is null)
|
|
||||||
{
|
{
|
||||||
errors[handle.Index] = StatusCodes.BadAggregateNotSupported;
|
// AggregateType is a per-node parallel collection (same length as nodesToRead, enforced by
|
||||||
continue;
|
// the base dispatcher). handle.Index is the node's position in that collection.
|
||||||
|
var aggregateNodeId = details.AggregateType[handle.Index];
|
||||||
|
var aggregate = MapAggregate(aggregateNodeId);
|
||||||
|
if (aggregate is null)
|
||||||
|
{
|
||||||
|
errors[handle.Index] = StatusCodes.BadAggregateNotSupported;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Processed is SINGLE-SHOT (no continuation point). Unlike Raw, ReadProcessedDetails carries
|
||||||
|
// NO client count cap (NumValuesPerNode) — the bucket count is deterministic (window / interval)
|
||||||
|
// and the single-shot backend returns every bucket in one read, so there is no "full page ⇒
|
||||||
|
// maybe more" signal to page on. Returning the complete aggregate result with a null CP is
|
||||||
|
// spec-conformant (OPC UA Part 11 lets a server return all available data in one response).
|
||||||
|
var agg = aggregate.Value;
|
||||||
|
work.Add(() => ServeNodeAsync(handle, results, errors, (source, tagname, token) => source.ReadProcessedAsync(
|
||||||
|
tagname,
|
||||||
|
details.StartTime,
|
||||||
|
details.EndTime,
|
||||||
|
interval,
|
||||||
|
agg,
|
||||||
|
token), ct));
|
||||||
}
|
}
|
||||||
|
|
||||||
// Processed is SINGLE-SHOT (no continuation point). Unlike Raw, ReadProcessedDetails carries
|
RunBounded(work, HistoryReadBatchParallelism);
|
||||||
// NO client count cap (NumValuesPerNode) — the bucket count is deterministic (window / interval)
|
});
|
||||||
// and the single-shot backend returns every bucket in one read, so there is no "full page ⇒
|
|
||||||
// maybe more" signal to page on. Returning the complete aggregate result with a null CP is
|
|
||||||
// spec-conformant (OPC UA Part 11 lets a server return all available data in one response).
|
|
||||||
ServeNode(handle, results, errors, (source, tagname) => source.ReadProcessedAsync(
|
|
||||||
tagname,
|
|
||||||
details.StartTime,
|
|
||||||
details.EndTime,
|
|
||||||
interval,
|
|
||||||
aggregate.Value,
|
|
||||||
CancellationToken.None));
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/// <inheritdoc />
|
/// <inheritdoc />
|
||||||
@@ -1860,13 +1906,21 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
|||||||
{
|
{
|
||||||
// Snapshot the requested timestamps once — the same list is read for every node.
|
// Snapshot the requested timestamps once — the same list is read for every node.
|
||||||
var timestamps = details.ReqTimes?.ToList() ?? new List<DateTime>();
|
var timestamps = details.ReqTimes?.ToList() ?? new List<DateTime>();
|
||||||
foreach (var handle in nodesToProcess)
|
WithHistoryReadBatchLimiter(nodesToProcess, results, errors, () =>
|
||||||
{
|
{
|
||||||
ServeNode(handle, results, errors, (source, tagname) => source.ReadAtTimeAsync(
|
using var deadline = CreateDeadline();
|
||||||
tagname,
|
var ct = deadline?.Token ?? CancellationToken.None;
|
||||||
timestamps,
|
var work = new List<Func<Task>>(nodesToProcess.Count);
|
||||||
CancellationToken.None));
|
foreach (var handle in nodesToProcess)
|
||||||
}
|
{
|
||||||
|
work.Add(() => ServeNodeAsync(handle, results, errors, (source, tagname, token) => source.ReadAtTimeAsync(
|
||||||
|
tagname,
|
||||||
|
timestamps,
|
||||||
|
token), ct));
|
||||||
|
}
|
||||||
|
|
||||||
|
RunBounded(work, HistoryReadBatchParallelism);
|
||||||
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
/// <inheritdoc />
|
/// <inheritdoc />
|
||||||
@@ -1883,54 +1937,95 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
|||||||
// Snapshot the select clauses once — the same filter projects every node's events.
|
// Snapshot the select clauses once — the same filter projects every node's events.
|
||||||
var selectClauses = details.Filter?.SelectClauses ?? new SimpleAttributeOperandCollection();
|
var selectClauses = details.Filter?.SelectClauses ?? new SimpleAttributeOperandCollection();
|
||||||
|
|
||||||
foreach (var handle in nodesToProcess)
|
WithHistoryReadBatchLimiter(nodesToProcess, results, errors, () =>
|
||||||
{
|
{
|
||||||
var idString = handle.NodeId.Identifier?.ToString();
|
using var deadline = CreateDeadline();
|
||||||
if (idString is null || !_eventNotifierSources.TryGetValue(idString, out var sourceName))
|
var ct = deadline?.Token ?? CancellationToken.None;
|
||||||
|
var work = new List<Func<Task>>(nodesToProcess.Count);
|
||||||
|
foreach (var handle in nodesToProcess)
|
||||||
{
|
{
|
||||||
// Not a registered event-history source (plain folder / Null-source promotion) ⇒ unsupported.
|
var idString = handle.NodeId.Identifier?.ToString();
|
||||||
// Set both errors and results explicitly on every bad path — don't rely on the SDK base
|
if (idString is null || !_eventNotifierSources.TryGetValue(idString, out var sourceName))
|
||||||
// pre-seeding results[i], so every path is self-contained and the contract is obvious.
|
|
||||||
errors[handle.Index] = StatusCodes.BadHistoryOperationUnsupported;
|
|
||||||
results[handle.Index] = new SdkHistoryReadResult { StatusCode = StatusCodes.BadHistoryOperationUnsupported };
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
|
|
||||||
try
|
|
||||||
{
|
|
||||||
// NOT under the node-manager Lock — block-bridging the async source is safe here.
|
|
||||||
var sourceResult = _historianDataSource.ReadEventsAsync(
|
|
||||||
sourceName,
|
|
||||||
details.StartTime,
|
|
||||||
details.EndTime,
|
|
||||||
// NumValuesPerNode==0 means "no limit — return ALL values" per OPC UA
|
|
||||||
// Part 4/11, so translate it to UNBOUNDED (int.MaxValue) here. Passing the int<=0
|
|
||||||
// backend-default-cap sentinel instead would silently truncate a "give me everything"
|
|
||||||
// events read at the backend default. A positive cap passes through (clamped).
|
|
||||||
EventMaxEvents(details.NumValuesPerNode),
|
|
||||||
CancellationToken.None).GetAwaiter().GetResult();
|
|
||||||
|
|
||||||
var historyEvent = ProjectEvents(sourceResult.Events, selectClauses);
|
|
||||||
results[handle.Index] = new SdkHistoryReadResult
|
|
||||||
{
|
{
|
||||||
// No events ⇒ GoodNoData (the notifier is historized, the window just held no events).
|
// Not a registered event-history source (plain folder / Null-source promotion) ⇒ unsupported.
|
||||||
StatusCode = sourceResult.Events.Count == 0 ? StatusCodes.GoodNoData : StatusCodes.Good,
|
// Set both errors and results explicitly on every bad path — don't rely on the SDK base
|
||||||
HistoryData = new ExtensionObject(historyEvent),
|
// pre-seeding results[i], so every path is self-contained and the contract is obvious.
|
||||||
// We never issue continuation points — every read returns the full window in one shot.
|
errors[handle.Index] = StatusCodes.BadHistoryOperationUnsupported;
|
||||||
ContinuationPoint = null,
|
results[handle.Index] = new SdkHistoryReadResult { StatusCode = StatusCodes.BadHistoryOperationUnsupported };
|
||||||
};
|
continue;
|
||||||
errors[handle.Index] = ServiceResult.Good;
|
}
|
||||||
|
|
||||||
|
var sn = sourceName;
|
||||||
|
work.Add(() => ServeEventsAsync(handle, sn, details, selectClauses, results, errors, ct));
|
||||||
}
|
}
|
||||||
catch (Exception ex)
|
|
||||||
|
RunBounded(work, HistoryReadBatchParallelism);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Serve one registered event-history notifier handle: reads the source's events over the request
|
||||||
|
/// window, projects them onto the select clauses, and fills the service-level results/errors slots.
|
||||||
|
/// Per-node error isolation matches <see cref="ServeNodeAsync"/> — a backend throw becomes a Bad
|
||||||
|
/// status for THIS node only and never throws out of the batch; a deadline cancellation surfaces
|
||||||
|
/// <c>BadTimeout</c>.
|
||||||
|
/// </summary>
|
||||||
|
/// <param name="handle">The notifier node handle; <c>handle.Index</c> indexes results/errors.</param>
|
||||||
|
/// <param name="sourceName">The registered historian event-source name for this notifier.</param>
|
||||||
|
/// <param name="details">The event-read details (window + count cap).</param>
|
||||||
|
/// <param name="selectClauses">The event-filter select clauses (the fields to emit, in order).</param>
|
||||||
|
/// <param name="results">The service-level results list to fill at <c>handle.Index</c>.</param>
|
||||||
|
/// <param name="errors">The service-level errors list to fill at <c>handle.Index</c>.</param>
|
||||||
|
/// <param name="ct">Per-request deadline token.</param>
|
||||||
|
private async Task ServeEventsAsync(
|
||||||
|
NodeHandle handle,
|
||||||
|
string? sourceName,
|
||||||
|
ReadEventDetails details,
|
||||||
|
SimpleAttributeOperandCollection selectClauses,
|
||||||
|
IList<SdkHistoryReadResult> results,
|
||||||
|
IList<ServiceResult> errors,
|
||||||
|
CancellationToken ct)
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
// NOT under the node-manager Lock — awaiting the async source is safe here.
|
||||||
|
var sourceResult = await _historianDataSource.ReadEventsAsync(
|
||||||
|
sourceName,
|
||||||
|
details.StartTime,
|
||||||
|
details.EndTime,
|
||||||
|
// NumValuesPerNode==0 means "no limit — return ALL values" per OPC UA
|
||||||
|
// Part 4/11, so translate it to UNBOUNDED (int.MaxValue) here. Passing the int<=0
|
||||||
|
// backend-default-cap sentinel instead would silently truncate a "give me everything"
|
||||||
|
// events read at the backend default. A positive cap passes through (clamped).
|
||||||
|
EventMaxEvents(details.NumValuesPerNode),
|
||||||
|
ct).ConfigureAwait(false);
|
||||||
|
|
||||||
|
var historyEvent = ProjectEvents(sourceResult.Events, selectClauses);
|
||||||
|
results[handle.Index] = new SdkHistoryReadResult
|
||||||
{
|
{
|
||||||
// One node's backend failure must not throw out of the batch — surface Bad for THIS node
|
// No events ⇒ GoodNoData (the notifier is historized, the window just held no events).
|
||||||
// only. This manager carries no ILogger, so log via the SDK's static trace (see ServeNode).
|
StatusCode = sourceResult.Events.Count == 0 ? StatusCodes.GoodNoData : StatusCodes.Good,
|
||||||
|
HistoryData = new ExtensionObject(historyEvent),
|
||||||
|
// We never issue continuation points — every read returns the full window in one shot.
|
||||||
|
ContinuationPoint = null,
|
||||||
|
};
|
||||||
|
errors[handle.Index] = ServiceResult.Good;
|
||||||
|
}
|
||||||
|
catch (OperationCanceledException)
|
||||||
|
{
|
||||||
|
// The per-request deadline elapsed while reading this notifier — BadTimeout for THIS node only.
|
||||||
|
errors[handle.Index] = StatusCodes.BadTimeout;
|
||||||
|
results[handle.Index] = new SdkHistoryReadResult { StatusCode = StatusCodes.BadTimeout };
|
||||||
|
}
|
||||||
|
catch (Exception ex)
|
||||||
|
{
|
||||||
|
// One node's backend failure must not throw out of the batch — surface Bad for THIS node
|
||||||
|
// only. This manager carries no ILogger, so log via the SDK's static trace (see ServeNodeAsync).
|
||||||
#pragma warning disable CS0618 // Type or member is obsolete
|
#pragma warning disable CS0618 // Type or member is obsolete
|
||||||
Utils.LogError(ex, "OtOpcUaNodeManager: HistoryReadEvents failed for node {0}", handle.NodeId);
|
Utils.LogError(ex, "OtOpcUaNodeManager: HistoryReadEvents failed for node {0}", handle.NodeId);
|
||||||
#pragma warning restore CS0618
|
#pragma warning restore CS0618
|
||||||
errors[handle.Index] = StatusCodes.BadHistoryOperationUnsupported;
|
errors[handle.Index] = StatusCodes.BadHistoryOperationUnsupported;
|
||||||
results[handle.Index] = new SdkHistoryReadResult { StatusCode = StatusCodes.BadHistoryOperationUnsupported };
|
results[handle.Index] = new SdkHistoryReadResult { StatusCode = StatusCodes.BadHistoryOperationUnsupported };
|
||||||
}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -2037,11 +2132,131 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
|||||||
/// <param name="errors">The service-level errors list to fill at <c>handle.Index</c>.</param>
|
/// <param name="errors">The service-level errors list to fill at <c>handle.Index</c>.</param>
|
||||||
/// <param name="read">Invokes the resolved data-source read with the resolved tagname; only called
|
/// <param name="read">Invokes the resolved data-source read with the resolved tagname; only called
|
||||||
/// once the tagname is confirmed present.</param>
|
/// once the tagname is confirmed present.</param>
|
||||||
private void ServeNode(
|
/// <summary>
|
||||||
|
/// Run a batch of per-node HistoryRead work items with BOUNDED parallelism (arch-review 03/S3),
|
||||||
|
/// block-bridged ONCE at the arm boundary. <b>Contract:</b> every work item must be fully
|
||||||
|
/// self-contained — it fills its own results/errors slot and NEVER throws (all per-node handlers
|
||||||
|
/// catch every exception, including cancellation, and surface a Bad status for that node only). That
|
||||||
|
/// invariant is what makes the single <c>GetAwaiter().GetResult()</c> safe: <see cref="Task.WhenAll(Task[])"/>
|
||||||
|
/// can never fault, so it cannot throw out of the synchronous SDK override. A degree ≤ 1 (or a
|
||||||
|
/// single item) runs sequentially — no semaphore, no extra tasks.
|
||||||
|
/// </summary>
|
||||||
|
/// <param name="work">The per-node work items (index-disjoint writes into the shared results/errors).</param>
|
||||||
|
/// <param name="degree">Max concurrent work items (<see cref="HistoryReadBatchParallelism"/>).</param>
|
||||||
|
private static void RunBounded(IReadOnlyList<Func<Task>> work, int degree)
|
||||||
|
{
|
||||||
|
if (work.Count == 0) return;
|
||||||
|
if (degree <= 1 || work.Count == 1)
|
||||||
|
{
|
||||||
|
foreach (var item in work)
|
||||||
|
item().GetAwaiter().GetResult();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
using var gate = new SemaphoreSlim(degree);
|
||||||
|
var tasks = new Task[work.Count];
|
||||||
|
for (var i = 0; i < work.Count; i++)
|
||||||
|
tasks[i] = RunGatedAsync(gate, work[i]);
|
||||||
|
|
||||||
|
// Safe: every body is self-contained + never throws, so WhenAll never faults (see contract above).
|
||||||
|
Task.WhenAll(tasks).GetAwaiter().GetResult();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Run one HistoryRead arm under the process-wide concurrent-batch limiter (arch-review 03/S3). Each
|
||||||
|
/// arm waits up to <c>min(HistoryReadDeadline, 5 s)</c> for a slot; on expiry every handle in the
|
||||||
|
/// batch gets <c>BadTooManyOperations</c> (fast-fail under flood, brief waits under mild contention)
|
||||||
|
/// and <paramref name="serve"/> is skipped. On acquisition the arm body runs and the slot is
|
||||||
|
/// released in a <c>finally</c>. Combined with <see cref="HistoryReadBatchParallelism"/> this caps
|
||||||
|
/// total in-flight gateway reads at
|
||||||
|
/// <c>MaxConcurrentHistoryReadBatches × HistoryReadBatchParallelism</c>.
|
||||||
|
/// </summary>
|
||||||
|
/// <param name="nodesToProcess">The batch's handles — every slot is failed on limiter saturation.</param>
|
||||||
|
/// <param name="results">The service-level results list.</param>
|
||||||
|
/// <param name="errors">The service-level errors list.</param>
|
||||||
|
/// <param name="serve">The arm body to run once a slot is acquired.</param>
|
||||||
|
private void WithHistoryReadBatchLimiter(
|
||||||
|
List<NodeHandle> nodesToProcess,
|
||||||
|
IList<SdkHistoryReadResult> results,
|
||||||
|
IList<ServiceResult> errors,
|
||||||
|
Action serve)
|
||||||
|
{
|
||||||
|
var gate = GetHistoryReadBatchLimiter();
|
||||||
|
|
||||||
|
// Bounded admission wait: cap the block at 5 s, and never exceed the per-request deadline.
|
||||||
|
var waitMs = HistoryReadDeadline > TimeSpan.Zero
|
||||||
|
? Math.Min(HistoryReadDeadline.TotalMilliseconds, 5000.0)
|
||||||
|
: 5000.0;
|
||||||
|
|
||||||
|
if (!gate.Wait(TimeSpan.FromMilliseconds(waitMs)))
|
||||||
|
{
|
||||||
|
foreach (var handle in nodesToProcess)
|
||||||
|
{
|
||||||
|
errors[handle.Index] = StatusCodes.BadTooManyOperations;
|
||||||
|
results[handle.Index] = new SdkHistoryReadResult { StatusCode = StatusCodes.BadTooManyOperations };
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
serve();
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
gate.Release();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Lazily create (double-checked) the process-wide HistoryRead-batch limiter, sized from
|
||||||
|
/// <see cref="MaxConcurrentHistoryReadBatches"/> at first use so the Host's post-boot property set is
|
||||||
|
/// respected (the same benign wiring race as <see cref="MaxTieClusterOverfetch"/>).
|
||||||
|
/// </summary>
|
||||||
|
/// <returns>The shared batch limiter semaphore.</returns>
|
||||||
|
private SemaphoreSlim GetHistoryReadBatchLimiter()
|
||||||
|
{
|
||||||
|
var gate = _historyReadBatchLimiter;
|
||||||
|
if (gate is not null) return gate;
|
||||||
|
lock (_historyReadBatchLimiterLock)
|
||||||
|
{
|
||||||
|
return _historyReadBatchLimiter ??= new SemaphoreSlim(Math.Max(1, MaxConcurrentHistoryReadBatches));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Create the per-request deadline source for a HistoryRead arm (arch-review 03/S3): a
|
||||||
|
/// <see cref="CancellationTokenSource"/> that fires after <see cref="HistoryReadDeadline"/>, so a
|
||||||
|
/// single request can never hold an SDK request thread longer than the deadline regardless of node
|
||||||
|
/// count. Returns <c>null</c> when the deadline is non-positive (unbounded — the arm then passes
|
||||||
|
/// <see cref="CancellationToken.None"/>), matching the <c>ServerHistorianOptions</c> warning.
|
||||||
|
/// </summary>
|
||||||
|
/// <returns>The deadline source, or <c>null</c> for an unbounded (non-positive) deadline.</returns>
|
||||||
|
private CancellationTokenSource? CreateDeadline() =>
|
||||||
|
HistoryReadDeadline > TimeSpan.Zero ? new CancellationTokenSource(HistoryReadDeadline) : null;
|
||||||
|
|
||||||
|
/// <summary>Awaits one work item under a semaphore permit, releasing it when the item completes.</summary>
|
||||||
|
/// <param name="gate">The concurrency-limiting semaphore.</param>
|
||||||
|
/// <param name="work">The per-node work item.</param>
|
||||||
|
private static async Task RunGatedAsync(SemaphoreSlim gate, Func<Task> work)
|
||||||
|
{
|
||||||
|
await gate.WaitAsync().ConfigureAwait(false);
|
||||||
|
try
|
||||||
|
{
|
||||||
|
await work().ConfigureAwait(false);
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
gate.Release();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private async Task ServeNodeAsync(
|
||||||
NodeHandle handle,
|
NodeHandle handle,
|
||||||
IList<SdkHistoryReadResult> results,
|
IList<SdkHistoryReadResult> results,
|
||||||
IList<ServiceResult> errors,
|
IList<ServiceResult> errors,
|
||||||
Func<IHistorianDataSource, string, Task<HistorianRead>> read)
|
Func<IHistorianDataSource, string, CancellationToken, Task<HistorianRead>> read,
|
||||||
|
CancellationToken ct)
|
||||||
{
|
{
|
||||||
var idString = handle.NodeId.Identifier?.ToString();
|
var idString = handle.NodeId.Identifier?.ToString();
|
||||||
if (idString is null || !TryGetHistorizedTagname(idString, out var tagname))
|
if (idString is null || !TryGetHistorizedTagname(idString, out var tagname))
|
||||||
@@ -2054,9 +2269,9 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
|||||||
|
|
||||||
try
|
try
|
||||||
{
|
{
|
||||||
// HistoryRead is NOT invoked under the node-manager Lock (unlike OnWriteValue), so blocking
|
// HistoryRead is NOT invoked under the node-manager Lock (unlike OnWriteValue), so awaiting
|
||||||
// on the async source here is safe and won't freeze the address space.
|
// the async source here is safe and won't freeze the address space.
|
||||||
var sourceResult = read(HistorianDataSource, tagname!).GetAwaiter().GetResult();
|
var sourceResult = await read(HistorianDataSource, tagname!, ct).ConfigureAwait(false);
|
||||||
var historyData = ToHistoryData(sourceResult);
|
var historyData = ToHistoryData(sourceResult);
|
||||||
|
|
||||||
results[handle.Index] = new SdkHistoryReadResult
|
results[handle.Index] = new SdkHistoryReadResult
|
||||||
@@ -2071,13 +2286,20 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
|||||||
};
|
};
|
||||||
errors[handle.Index] = ServiceResult.Good;
|
errors[handle.Index] = ServiceResult.Good;
|
||||||
}
|
}
|
||||||
|
catch (OperationCanceledException)
|
||||||
|
{
|
||||||
|
// The per-request deadline elapsed while this node was in flight — surface BadTimeout for
|
||||||
|
// THIS node only (a spec-conformant history-read status), ahead of the generic catch.
|
||||||
|
errors[handle.Index] = StatusCodes.BadTimeout;
|
||||||
|
results[handle.Index] = new SdkHistoryReadResult { StatusCode = StatusCodes.BadTimeout };
|
||||||
|
}
|
||||||
catch (Exception ex)
|
catch (Exception ex)
|
||||||
{
|
{
|
||||||
// One node's backend failure (throw / timeout / cancellation) must not throw out of the
|
// One node's backend failure (throw / timeout) must not throw out of the batch — surface a
|
||||||
// batch — surface a Bad status for THIS node only. This CustomNodeManager2 carries no
|
// Bad status for THIS node only. This CustomNodeManager2 carries no ILogger (see
|
||||||
// ILogger (see ReportConditionEvent), so log through the SDK's static trace rather than
|
// ReportConditionEvent), so log through the SDK's static trace rather than swallowing silently.
|
||||||
// swallowing silently. Utils.LogError is [Obsolete] in 1.5.378 (favours an ITelemetryContext
|
// Utils.LogError is [Obsolete] in 1.5.378 (favours an ITelemetryContext this manager doesn't
|
||||||
// this manager doesn't wire) — suppress the deprecation, matching the existing pattern.
|
// wire) — suppress the deprecation, matching the existing pattern.
|
||||||
#pragma warning disable CS0618 // Type or member is obsolete
|
#pragma warning disable CS0618 // Type or member is obsolete
|
||||||
Utils.LogError(ex, "OtOpcUaNodeManager: HistoryRead failed for node {0}", handle.NodeId);
|
Utils.LogError(ex, "OtOpcUaNodeManager: HistoryRead failed for node {0}", handle.NodeId);
|
||||||
#pragma warning restore CS0618
|
#pragma warning restore CS0618
|
||||||
@@ -2115,7 +2337,7 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
|||||||
/// <param name="startTimeUtc">The request window's (inclusive) lower bound, used for a fresh read.</param>
|
/// <param name="startTimeUtc">The request window's (inclusive) lower bound, used for a fresh read.</param>
|
||||||
/// <param name="endUtc">The (inclusive) upper bound of the read window; unchanged across pages.</param>
|
/// <param name="endUtc">The (inclusive) upper bound of the read window; unchanged across pages.</param>
|
||||||
/// <param name="numValuesPerNode">The client's per-page cap; <c>0</c> means "all values, no paging".</param>
|
/// <param name="numValuesPerNode">The client's per-page cap; <c>0</c> means "all values, no paging".</param>
|
||||||
private void ServeRawPaged(
|
private async Task ServeRawPagedAsync(
|
||||||
NodeHandle handle,
|
NodeHandle handle,
|
||||||
ISession? session,
|
ISession? session,
|
||||||
IList<HistoryReadValueId> nodesToRead,
|
IList<HistoryReadValueId> nodesToRead,
|
||||||
@@ -2123,7 +2345,8 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
|||||||
IList<ServiceResult> errors,
|
IList<ServiceResult> errors,
|
||||||
DateTime startTimeUtc,
|
DateTime startTimeUtc,
|
||||||
DateTime endUtc,
|
DateTime endUtc,
|
||||||
uint numValuesPerNode)
|
uint numValuesPerNode,
|
||||||
|
CancellationToken ct)
|
||||||
{
|
{
|
||||||
var inboundCp = nodesToRead[handle.Index].ContinuationPoint;
|
var inboundCp = nodesToRead[handle.Index].ContinuationPoint;
|
||||||
|
|
||||||
@@ -2165,10 +2388,10 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
|||||||
startUtc = startTimeUtc;
|
startUtc = startTimeUtc;
|
||||||
}
|
}
|
||||||
|
|
||||||
// HistoryRead is NOT under the node-manager Lock — block-bridging the async source is safe.
|
// HistoryRead is NOT under the node-manager Lock — awaiting the async source is safe.
|
||||||
var sourceResult = HistorianDataSource
|
var sourceResult = await HistorianDataSource
|
||||||
.ReadRawAsync(tagname, startUtc, endUtc, numValuesPerNode, CancellationToken.None)
|
.ReadRawAsync(tagname, startUtc, endUtc, numValuesPerNode, ct)
|
||||||
.GetAwaiter().GetResult();
|
.ConfigureAwait(false);
|
||||||
|
|
||||||
var backendFull = HistoryPaging.IsFullPage(sourceResult.Samples.Count, numValuesPerNode);
|
var backendFull = HistoryPaging.IsFullPage(sourceResult.Samples.Count, numValuesPerNode);
|
||||||
|
|
||||||
@@ -2198,9 +2421,9 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
|||||||
// Compute in uint so +1 can never wrap: Math.Max(1,…) also guards against a zero/negative
|
// Compute in uint so +1 can never wrap: Math.Max(1,…) also guards against a zero/negative
|
||||||
// config value slipping through (Validate() warns but does not block startup).
|
// config value slipping through (Validate() warns but does not block startup).
|
||||||
var overfetchCap = (uint)Math.Max(1, MaxTieClusterOverfetch) + 1u;
|
var overfetchCap = (uint)Math.Max(1, MaxTieClusterOverfetch) + 1u;
|
||||||
var cluster = HistorianDataSource
|
var cluster = (await HistorianDataSource
|
||||||
.ReadRawAsync(tagname, startUtc, startUtc, overfetchCap, CancellationToken.None)
|
.ReadRawAsync(tagname, startUtc, startUtc, overfetchCap, ct)
|
||||||
.GetAwaiter().GetResult().Samples;
|
.ConfigureAwait(false)).Samples;
|
||||||
|
|
||||||
// Absurd burst: more ties than we're willing to buffer in memory. Preserve today's loud-fail
|
// Absurd burst: more ties than we're willing to buffer in memory. Preserve today's loud-fail
|
||||||
// for that node rather than over-fetch an unbounded cluster; the operator's remedy is a
|
// for that node rather than over-fetch an unbounded cluster; the operator's remedy is a
|
||||||
@@ -2272,6 +2495,12 @@ public sealed class OtOpcUaNodeManager : CustomNodeManager2
|
|||||||
};
|
};
|
||||||
errors[handle.Index] = ServiceResult.Good;
|
errors[handle.Index] = ServiceResult.Good;
|
||||||
}
|
}
|
||||||
|
catch (OperationCanceledException)
|
||||||
|
{
|
||||||
|
// The per-request deadline elapsed while paging this node — BadTimeout for THIS node only.
|
||||||
|
errors[handle.Index] = StatusCodes.BadTimeout;
|
||||||
|
results[handle.Index] = new SdkHistoryReadResult { StatusCode = StatusCodes.BadTimeout };
|
||||||
|
}
|
||||||
catch (Exception ex)
|
catch (Exception ex)
|
||||||
{
|
{
|
||||||
// One node's backend failure must not throw out of the batch — Bad for THIS node only.
|
// One node's backend failure must not throw out of the batch — Bad for THIS node only.
|
||||||
|
|||||||
@@ -23,6 +23,9 @@
|
|||||||
|
|
||||||
<ItemGroup>
|
<ItemGroup>
|
||||||
<InternalsVisibleTo Include="ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests"/>
|
<InternalsVisibleTo Include="ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests"/>
|
||||||
|
<!-- R2-08 (03/S2): the outage-sim integration test drives the internal static
|
||||||
|
OpcUaApplicationHost.HandleImpersonation against the real LDAP library. -->
|
||||||
|
<InternalsVisibleTo Include="ZB.MOM.WW.OtOpcUa.Host.IntegrationTests"/>
|
||||||
</ItemGroup>
|
</ItemGroup>
|
||||||
|
|
||||||
</Project>
|
</Project>
|
||||||
|
|||||||
@@ -68,6 +68,32 @@ public sealed class ServerHistorianOptions
|
|||||||
/// </summary>
|
/// </summary>
|
||||||
public int MaxTieClusterOverfetch { get; init; } = 65536;
|
public int MaxTieClusterOverfetch { get; init; } = 65536;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Maximum number of historized nodes served CONCURRENTLY within one HistoryRead batch (arch-review
|
||||||
|
/// 03/S3). The SDK's <c>HistoryRead*</c> override surface is synchronous, so each arm block-bridges
|
||||||
|
/// once at its boundary; this bound turns the per-node fan-out inside that boundary from sequential
|
||||||
|
/// (worst case <c>N × CallTimeout</c>) into <c>⌈N / P⌉ × CallTimeout</c>. It also caps the gateway-load
|
||||||
|
/// multiplication a single client can command. Default 4.
|
||||||
|
/// </summary>
|
||||||
|
public int HistoryReadBatchParallelism { get; init; } = 4;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Process-wide limit on the number of HistoryRead batches served CONCURRENTLY (arch-review 03/S3).
|
||||||
|
/// A flood of history clients degrades to <c>BadTooManyOperations</c> rather than exhausting the SDK
|
||||||
|
/// request-thread pool. Together with <see cref="HistoryReadBatchParallelism"/> this caps total
|
||||||
|
/// in-flight gateway reads at <c>MaxConcurrentHistoryReadBatches × HistoryReadBatchParallelism</c>
|
||||||
|
/// (default 16 × 4 = 64). Default 16.
|
||||||
|
/// </summary>
|
||||||
|
public int MaxConcurrentHistoryReadBatches { get; init; } = 16;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Per-request wall-clock deadline for a HistoryRead batch (arch-review 03/S3). A single request can
|
||||||
|
/// no longer hold an SDK request thread longer than this regardless of node count; a node still
|
||||||
|
/// in flight at the deadline surfaces <c>BadTimeout</c>. Non-positive = unbounded (a
|
||||||
|
/// <see cref="Validate"/> warning). Default 60 seconds.
|
||||||
|
/// </summary>
|
||||||
|
public TimeSpan HistoryReadDeadline { get; init; } = TimeSpan.FromSeconds(60);
|
||||||
|
|
||||||
/// <summary>Returns operator-facing misconfiguration warnings for an <c>Enabled</c> historian
|
/// <summary>Returns operator-facing misconfiguration warnings for an <c>Enabled</c> historian
|
||||||
/// (empty when disabled or correctly configured). Pure — the registration logs each entry.
|
/// (empty when disabled or correctly configured). Pure — the registration logs each entry.
|
||||||
/// <para>
|
/// <para>
|
||||||
@@ -100,6 +126,14 @@ public sealed class ServerHistorianOptions
|
|||||||
// so a zero/negative value is harmless (and noise-free) when the historian is disabled.
|
// so a zero/negative value is harmless (and noise-free) when the historian is disabled.
|
||||||
if (MaxTieClusterOverfetch <= 0)
|
if (MaxTieClusterOverfetch <= 0)
|
||||||
warnings.Add($"ServerHistorian:MaxTieClusterOverfetch is {MaxTieClusterOverfetch} — must be > 0; HistoryRead-Raw cannot page within an oversized tie cluster and will surface BadHistoryOperationUnsupported for those reads.");
|
warnings.Add($"ServerHistorian:MaxTieClusterOverfetch is {MaxTieClusterOverfetch} — must be > 0; HistoryRead-Raw cannot page within an oversized tie cluster and will surface BadHistoryOperationUnsupported for those reads.");
|
||||||
|
// S3 HistoryRead concurrency knobs — a non-positive value disables the corresponding bound, so warn
|
||||||
|
// (these gate the per-node fan-out / batch limiter that only run when a real data source is wired).
|
||||||
|
if (HistoryReadBatchParallelism <= 0)
|
||||||
|
warnings.Add($"ServerHistorian:HistoryReadBatchParallelism is {HistoryReadBatchParallelism} — must be > 0; HistoryRead falls back to serving each node sequentially.");
|
||||||
|
if (MaxConcurrentHistoryReadBatches <= 0)
|
||||||
|
warnings.Add($"ServerHistorian:MaxConcurrentHistoryReadBatches is {MaxConcurrentHistoryReadBatches} — must be > 0; the process-wide HistoryRead batch limiter is disabled.");
|
||||||
|
if (HistoryReadDeadline <= TimeSpan.Zero)
|
||||||
|
warnings.Add($"ServerHistorian:HistoryReadDeadline is {HistoryReadDeadline} — must be > 0; HistoryRead requests run unbounded (no per-request deadline).");
|
||||||
return warnings;
|
return warnings;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -7,4 +7,15 @@ public sealed record LdapAuthResult(
|
|||||||
string? Username,
|
string? Username,
|
||||||
IReadOnlyList<string> Groups,
|
IReadOnlyList<string> Groups,
|
||||||
IReadOnlyList<string> Roles,
|
IReadOnlyList<string> Roles,
|
||||||
string? Error);
|
string? Error)
|
||||||
|
{
|
||||||
|
/// <summary>
|
||||||
|
/// <c>true</c> when this failure is a SYSTEM-side directory fault (the directory is unreachable or
|
||||||
|
/// the service account is misconfigured) rather than a user-credential deny. Feeds the OPC UA
|
||||||
|
/// data-plane authenticator's outage circuit (03/S2): a run of consecutive system-side failures
|
||||||
|
/// opens the circuit and fails fast, whereas user-side denies never trip it. Always <c>false</c>
|
||||||
|
/// on success and on user-side denies; set only on the delegated real path for the library's
|
||||||
|
/// directory-unreachable bucket.
|
||||||
|
/// </summary>
|
||||||
|
public bool IsSystemFailure { get; init; }
|
||||||
|
}
|
||||||
|
|||||||
@@ -83,6 +83,49 @@ public sealed class LdapOptions
|
|||||||
/// </summary>
|
/// </summary>
|
||||||
public Dictionary<string, string> GroupToRole { get; set; } = new(StringComparer.OrdinalIgnoreCase);
|
public Dictionary<string, string> GroupToRole { get; set; } = new(StringComparer.OrdinalIgnoreCase);
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Bounds (in milliseconds) the shared-library directory client's socket connect AND each
|
||||||
|
/// per-operation search inside a single bind flow. Projected into the library's
|
||||||
|
/// <see cref="LibLdapOptions.ConnectionTimeoutMs"/> via <see cref="ToLibraryOptions"/>. The
|
||||||
|
/// default (10000) matches the library default, so behaviour is unchanged unless configured.
|
||||||
|
/// This is the FIRST line of defence against a directory outage; the whole-flow
|
||||||
|
/// <see cref="AuthTimeoutMs"/> is the backstop (03/S2).
|
||||||
|
/// </summary>
|
||||||
|
public int ConnectionTimeoutMs { get; set; } = 10_000;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Hard whole-flow wall-clock bound (in milliseconds) the OPC UA data-plane authenticator
|
||||||
|
/// (<c>LdapOpcUaUserAuthenticator</c>) applies across the ENTIRE authenticate flow — bind plus
|
||||||
|
/// role-map — so the OPC UA SDK's synchronous impersonation callback (raised under a
|
||||||
|
/// <c>SessionManager</c>-wide event lock) can never be wedged by a slow / hung directory.
|
||||||
|
/// Deliberately > <see cref="ConnectionTimeoutMs"/> (default 15000) so a healthy-but-slow
|
||||||
|
/// full flow is not cut off while an outage is bounded by the library timeout first (03/S2).
|
||||||
|
/// </summary>
|
||||||
|
public int AuthTimeoutMs { get; set; } = 15_000;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Maximum number of in-flight OPC UA authentication flows the data-plane authenticator admits
|
||||||
|
/// at once. A caller that cannot acquire a slot within its budget is denied "backend busy"
|
||||||
|
/// (local backpressure, not counted against the outage circuit). Bounds accumulation of
|
||||||
|
/// orphaned / timed-out bind tasks during an outage (03/S2). Default 8.
|
||||||
|
/// </summary>
|
||||||
|
public int MaxConcurrentAuthentications { get; set; } = 8;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Number of CONSECUTIVE system-side failures (boundary timeout, directory-unreachable result,
|
||||||
|
/// or unexpected throw) after which the data-plane authenticator's outage circuit opens and
|
||||||
|
/// denies instantly for <see cref="OutageCooldownSeconds"/> without touching LDAP. Any success
|
||||||
|
/// or user-side (credential) deny resets the counter. <c>0</c> disables the circuit (03/S2).
|
||||||
|
/// Default 3.
|
||||||
|
/// </summary>
|
||||||
|
public int OutageFailureThreshold { get; set; } = 3;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// How long (in seconds) the outage circuit stays open before a half-open probe is allowed
|
||||||
|
/// through. Only meaningful when <see cref="OutageFailureThreshold"/> > 0. Default 15 (03/S2).
|
||||||
|
/// </summary>
|
||||||
|
public int OutageCooldownSeconds { get; set; } = 15;
|
||||||
|
|
||||||
/// <summary>
|
/// <summary>
|
||||||
/// Projects the wire fields onto the shared <c>ZB.MOM.WW.Auth.Ldap</c>
|
/// Projects the wire fields onto the shared <c>ZB.MOM.WW.Auth.Ldap</c>
|
||||||
/// <see cref="LibLdapOptions"/> the directory client consumes. App-only concerns
|
/// <see cref="LibLdapOptions"/> the directory client consumes. App-only concerns
|
||||||
@@ -104,5 +147,6 @@ public sealed class LdapOptions
|
|||||||
UserNameAttribute = UserNameAttribute,
|
UserNameAttribute = UserNameAttribute,
|
||||||
DisplayNameAttribute = DisplayNameAttribute,
|
DisplayNameAttribute = DisplayNameAttribute,
|
||||||
GroupAttribute = GroupAttribute,
|
GroupAttribute = GroupAttribute,
|
||||||
|
ConnectionTimeoutMs = ConnectionTimeoutMs,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -113,7 +113,13 @@ public sealed class OtOpcUaLdapAuthService : ILdapAuthService
|
|||||||
if (result.Succeeded)
|
if (result.Succeeded)
|
||||||
return new(true, result.DisplayName, result.Username, result.Groups, [], null);
|
return new(true, result.DisplayName, result.Username, result.Groups, [], null);
|
||||||
|
|
||||||
return new(false, null, username, [], [], FailureToError(result.Failure));
|
// ServiceAccountBindFailed is the library's directory-unreachable / service-account-misconfigured
|
||||||
|
// bucket (its enum has no dedicated DirectoryUnavailable) — flag it as system-side so the OPC UA
|
||||||
|
// authenticator's outage circuit can distinguish an outage from a user-credential deny (03/S2).
|
||||||
|
return new(false, null, username, [], [], FailureToError(result.Failure))
|
||||||
|
{
|
||||||
|
IsSystemFailure = result.Failure is LdapAuthFailure.ServiceAccountBindFailed,
|
||||||
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
/// <summary>Folds a structured library failure code into the app's user-facing error text.</summary>
|
/// <summary>Folds a structured library failure code into the app's user-facing error text.</summary>
|
||||||
|
|||||||
@@ -0,0 +1,389 @@
|
|||||||
|
using System.Diagnostics;
|
||||||
|
using System.Text;
|
||||||
|
using Microsoft.Extensions.DependencyInjection;
|
||||||
|
using Microsoft.Extensions.Logging.Abstractions;
|
||||||
|
using Microsoft.Extensions.Options;
|
||||||
|
using Opc.Ua;
|
||||||
|
using Opc.Ua.Server;
|
||||||
|
using Shouldly;
|
||||||
|
using Xunit;
|
||||||
|
using ZB.MOM.WW.Auth.Abstractions.Roles;
|
||||||
|
using ZB.MOM.WW.OtOpcUa.Host.OpcUa;
|
||||||
|
using ZB.MOM.WW.OtOpcUa.OpcUaServer;
|
||||||
|
using ZB.MOM.WW.OtOpcUa.Security.Ldap;
|
||||||
|
using LdapTransport = ZB.MOM.WW.Auth.Abstractions.Ldap.LdapTransport;
|
||||||
|
|
||||||
|
namespace ZB.MOM.WW.OtOpcUa.Host.IntegrationTests;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Resilience guard for <see cref="LdapOpcUaUserAuthenticator"/> (arch-review 03/S2). The OPC UA SDK
|
||||||
|
/// raises the impersonation callback synchronously under a <c>SessionManager</c>-wide event lock, so a
|
||||||
|
/// slow / hung LDAP bind inside it does not merely stall one SDK thread — it serializes EVERY session
|
||||||
|
/// activation server-wide. These tests prove the authenticator enforces a hard wall-clock bound,
|
||||||
|
/// bounded in-flight concurrency, and a directory-outage circuit at its own boundary (the SDK callback
|
||||||
|
/// being irreducibly synchronous), all fail-closed. No real LDAP server is used — a delaying fake
|
||||||
|
/// reproduces the outage shape offline.
|
||||||
|
/// </summary>
|
||||||
|
public sealed class LdapAuthResilienceTests
|
||||||
|
{
|
||||||
|
/// <summary>
|
||||||
|
/// RED repro (03/S2): a backend that takes 5 s must NOT hold the caller for 5 s — the authenticator's
|
||||||
|
/// wall-clock bound denies (fail-closed) well before then. On current code (no bound) this fails: the
|
||||||
|
/// call returns Allow after the full 5 s.
|
||||||
|
/// </summary>
|
||||||
|
[Fact]
|
||||||
|
public async Task Authenticate_slow_backend_denies_within_AuthTimeout()
|
||||||
|
{
|
||||||
|
var ldap = new DelayingFakeLdap(
|
||||||
|
TimeSpan.FromSeconds(5),
|
||||||
|
new LdapAuthResult(true, "Alice", "alice", new[] { "configeditor" }, Array.Empty<string>(), null));
|
||||||
|
var mapper = new FakeMapper(g => g.ToArray());
|
||||||
|
var sut = Build(ldap, mapper, new LdapOptions { AuthTimeoutMs = 200 });
|
||||||
|
|
||||||
|
var sw = Stopwatch.StartNew();
|
||||||
|
var result = await sut
|
||||||
|
.AuthenticateUserNameAsync("alice", "secret", CancellationToken.None)
|
||||||
|
.WaitAsync(TimeSpan.FromSeconds(10), CancellationToken.None);
|
||||||
|
sw.Stop();
|
||||||
|
|
||||||
|
result.Success.ShouldBeFalse();
|
||||||
|
result.Error.ShouldNotBeNull();
|
||||||
|
result.Error!.ShouldContain("timed out");
|
||||||
|
sw.Elapsed.ShouldBeLessThan(TimeSpan.FromSeconds(1.5));
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>A fast, healthy backend is unaffected by the resilience wrapping — success + roles pass
|
||||||
|
/// through untouched (03/S2).</summary>
|
||||||
|
[Fact]
|
||||||
|
public async Task Authenticate_fast_success_unaffected()
|
||||||
|
{
|
||||||
|
var ldap = new DelayingFakeLdap(
|
||||||
|
TimeSpan.Zero,
|
||||||
|
new LdapAuthResult(true, "Alice", "alice", new[] { "configeditor" }, Array.Empty<string>(), null));
|
||||||
|
var mapper = new FakeMapper(g => g.Select(x => x == "configeditor" ? "Designer" : x).ToArray());
|
||||||
|
var sut = Build(ldap, mapper, new LdapOptions());
|
||||||
|
|
||||||
|
var result = await sut
|
||||||
|
.AuthenticateUserNameAsync("alice", "secret", CancellationToken.None)
|
||||||
|
.WaitAsync(TimeSpan.FromSeconds(10), CancellationToken.None);
|
||||||
|
|
||||||
|
result.Success.ShouldBeTrue();
|
||||||
|
result.DisplayName.ShouldBe("Alice");
|
||||||
|
result.Roles.ShouldBe(new[] { "Designer" });
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>With the concurrency cap saturated (a parked in-flight bind holding the only slot), a
|
||||||
|
/// second call is denied "backend busy" within its own budget — it never waits behind the parked
|
||||||
|
/// bind indefinitely (03/S2).</summary>
|
||||||
|
[Fact]
|
||||||
|
public async Task Authenticate_saturated_concurrency_denies_busy()
|
||||||
|
{
|
||||||
|
var ldap = new GatedFakeLdap(
|
||||||
|
new LdapAuthResult(true, "Alice", "alice", new[] { "configeditor" }, Array.Empty<string>(), null));
|
||||||
|
var mapper = new FakeMapper(g => g.ToArray());
|
||||||
|
// MaxConcurrentAuthentications = 1 → the second call cannot get a slot; small AuthTimeoutMs so the
|
||||||
|
// "busy" denial is quick and the first call's boundary also trips fast (its core stays parked,
|
||||||
|
// holding the slot).
|
||||||
|
var sut = Build(ldap, mapper, new LdapOptions { MaxConcurrentAuthentications = 1, AuthTimeoutMs = 300 });
|
||||||
|
|
||||||
|
// First call: acquires the only slot; its core parks in the gated fake.
|
||||||
|
var first = sut.AuthenticateUserNameAsync("alice", "secret", CancellationToken.None);
|
||||||
|
|
||||||
|
// Wait until the fake has actually been entered (slot acquired, core parked).
|
||||||
|
var spin = Stopwatch.StartNew();
|
||||||
|
while (ldap.Entered == 0 && spin.Elapsed < TimeSpan.FromSeconds(5))
|
||||||
|
await Task.Delay(10, TestContext.Current.CancellationToken);
|
||||||
|
ldap.Entered.ShouldBe(1);
|
||||||
|
|
||||||
|
// Second concurrent call: the slot is held by the parked core, so acquisition times out → busy.
|
||||||
|
var second = await sut
|
||||||
|
.AuthenticateUserNameAsync("bob", "secret", CancellationToken.None)
|
||||||
|
.WaitAsync(TimeSpan.FromSeconds(10), CancellationToken.None);
|
||||||
|
|
||||||
|
second.Success.ShouldBeFalse();
|
||||||
|
second.Error.ShouldNotBeNull();
|
||||||
|
second.Error!.ShouldContain("busy");
|
||||||
|
ldap.Entered.ShouldBe(1, "the second call must be denied without ever reaching the backend");
|
||||||
|
|
||||||
|
// Release the parked bind so the first call's core completes cleanly (no orphaned resources).
|
||||||
|
ldap.Release();
|
||||||
|
await first.WaitAsync(TimeSpan.FromSeconds(10), CancellationToken.None);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>After <c>OutageFailureThreshold</c> consecutive system-side failures the circuit opens and
|
||||||
|
/// the next call is fast-denied "backend unavailable" WITHOUT touching LDAP (03/S2).</summary>
|
||||||
|
[Fact]
|
||||||
|
public async Task Consecutive_system_failures_open_circuit_and_fast_deny()
|
||||||
|
{
|
||||||
|
var ldap = new DelayingFakeLdap(TimeSpan.Zero, SystemFailure());
|
||||||
|
var mapper = new FakeMapper(g => g.ToArray());
|
||||||
|
var sut = Build(ldap, mapper,
|
||||||
|
new LdapOptions { OutageFailureThreshold = 3, OutageCooldownSeconds = 60, AuthTimeoutMs = 5000 });
|
||||||
|
|
||||||
|
for (var i = 0; i < 3; i++)
|
||||||
|
{
|
||||||
|
var r = await sut.AuthenticateUserNameAsync("u", "p", CancellationToken.None)
|
||||||
|
.WaitAsync(TimeSpan.FromSeconds(10), CancellationToken.None);
|
||||||
|
r.Success.ShouldBeFalse();
|
||||||
|
}
|
||||||
|
|
||||||
|
ldap.Calls.ShouldBe(3);
|
||||||
|
|
||||||
|
var sw = Stopwatch.StartNew();
|
||||||
|
var blocked = await sut.AuthenticateUserNameAsync("u", "p", CancellationToken.None)
|
||||||
|
.WaitAsync(TimeSpan.FromSeconds(10), CancellationToken.None);
|
||||||
|
sw.Stop();
|
||||||
|
|
||||||
|
blocked.Success.ShouldBeFalse();
|
||||||
|
blocked.Error.ShouldNotBeNull();
|
||||||
|
blocked.Error!.ShouldContain("unavailable");
|
||||||
|
ldap.Calls.ShouldBe(3, "an open circuit must fast-deny without reaching the backend");
|
||||||
|
sw.Elapsed.ShouldBeLessThan(TimeSpan.FromSeconds(1));
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>A user-side (credential) deny between system failures resets the streak, so the circuit
|
||||||
|
/// never opens — user denies are not directory evidence (03/S2).</summary>
|
||||||
|
[Fact]
|
||||||
|
public async Task User_side_deny_resets_the_system_failure_streak()
|
||||||
|
{
|
||||||
|
var ldap = new ScriptedFakeLdap(
|
||||||
|
SystemFailure(), SystemFailure(), UserDeny(), SystemFailure(), SystemFailure());
|
||||||
|
var mapper = new FakeMapper(g => g.ToArray());
|
||||||
|
var sut = Build(ldap, mapper,
|
||||||
|
new LdapOptions { OutageFailureThreshold = 3, OutageCooldownSeconds = 60, AuthTimeoutMs = 5000 });
|
||||||
|
|
||||||
|
// 5 scripted calls: sys, sys, user(reset), sys, sys → streak ends at 2 (< 3), circuit stays closed.
|
||||||
|
for (var i = 0; i < 5; i++)
|
||||||
|
{
|
||||||
|
await sut.AuthenticateUserNameAsync("u", "p", CancellationToken.None)
|
||||||
|
.WaitAsync(TimeSpan.FromSeconds(10), CancellationToken.None);
|
||||||
|
}
|
||||||
|
|
||||||
|
// A 6th call must still REACH the backend (circuit never opened).
|
||||||
|
await sut.AuthenticateUserNameAsync("u", "p", CancellationToken.None)
|
||||||
|
.WaitAsync(TimeSpan.FromSeconds(10), CancellationToken.None);
|
||||||
|
ldap.Calls.ShouldBe(6);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>After the cooldown elapses the circuit half-opens: the next call probes the backend and a
|
||||||
|
/// success closes the circuit (03/S2).</summary>
|
||||||
|
[Fact]
|
||||||
|
public async Task Half_open_probe_after_cooldown_closes_on_success()
|
||||||
|
{
|
||||||
|
var ldap = new ScriptedFakeLdap(
|
||||||
|
SystemFailure(), SystemFailure(), SystemFailure(),
|
||||||
|
new LdapAuthResult(true, "Alice", "alice", new[] { "configeditor" }, Array.Empty<string>(), null));
|
||||||
|
var mapper = new FakeMapper(g => g.ToArray());
|
||||||
|
var sut = Build(ldap, mapper,
|
||||||
|
new LdapOptions { OutageFailureThreshold = 3, OutageCooldownSeconds = 1, AuthTimeoutMs = 5000 });
|
||||||
|
|
||||||
|
for (var i = 0; i < 3; i++)
|
||||||
|
await sut.AuthenticateUserNameAsync("u", "p", CancellationToken.None)
|
||||||
|
.WaitAsync(TimeSpan.FromSeconds(10), CancellationToken.None);
|
||||||
|
ldap.Calls.ShouldBe(3);
|
||||||
|
|
||||||
|
// Circuit open: this call is fast-denied without touching the backend.
|
||||||
|
var blocked = await sut.AuthenticateUserNameAsync("u", "p", CancellationToken.None)
|
||||||
|
.WaitAsync(TimeSpan.FromSeconds(10), CancellationToken.None);
|
||||||
|
blocked.Error!.ShouldContain("unavailable");
|
||||||
|
ldap.Calls.ShouldBe(3);
|
||||||
|
|
||||||
|
// Wait past the cooldown, then a probe reaches the backend and succeeds → circuit closes.
|
||||||
|
await Task.Delay(TimeSpan.FromMilliseconds(1200), TestContext.Current.CancellationToken);
|
||||||
|
|
||||||
|
var probe = await sut.AuthenticateUserNameAsync("alice", "secret", CancellationToken.None)
|
||||||
|
.WaitAsync(TimeSpan.FromSeconds(10), CancellationToken.None);
|
||||||
|
probe.Success.ShouldBeTrue();
|
||||||
|
ldap.Calls.ShouldBe(4, "the half-open probe reaches the backend");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary><c>OutageFailureThreshold = 0</c> disables the circuit — it never opens, every call reaches
|
||||||
|
/// the backend (03/S2).</summary>
|
||||||
|
[Fact]
|
||||||
|
public async Task Circuit_disabled_never_opens()
|
||||||
|
{
|
||||||
|
var ldap = new DelayingFakeLdap(TimeSpan.Zero, SystemFailure());
|
||||||
|
var mapper = new FakeMapper(g => g.ToArray());
|
||||||
|
var sut = Build(ldap, mapper,
|
||||||
|
new LdapOptions { OutageFailureThreshold = 0, OutageCooldownSeconds = 60, AuthTimeoutMs = 5000 });
|
||||||
|
|
||||||
|
for (var i = 0; i < 6; i++)
|
||||||
|
await sut.AuthenticateUserNameAsync("u", "p", CancellationToken.None)
|
||||||
|
.WaitAsync(TimeSpan.FromSeconds(10), CancellationToken.None);
|
||||||
|
|
||||||
|
ldap.Calls.ShouldBe(6, "a disabled circuit never fast-denies");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Outage-sim (offline-safe): the REAL <see cref="OtOpcUaLdapAuthService"/> + real shared library
|
||||||
|
/// pointed at an UNROUTABLE host (a TCP blackhole — the same outage shape as "stop the GLAuth
|
||||||
|
/// container"), driven through the REAL internal <see cref="OpcUaApplicationHost.HandleImpersonation"/>
|
||||||
|
/// block-bridge. The activation must fail bounded (well under the SDK's default ~10-20 s stall) with
|
||||||
|
/// <see cref="StatusCodes.BadIdentityTokenRejected"/> and no granted identity (03/S2).
|
||||||
|
/// </summary>
|
||||||
|
[Fact]
|
||||||
|
public async Task Activation_with_unreachable_directory_fails_within_bound()
|
||||||
|
{
|
||||||
|
var options = new LdapOptions
|
||||||
|
{
|
||||||
|
Enabled = true,
|
||||||
|
DevStubMode = false,
|
||||||
|
Server = "10.255.255.1", // RFC-blackhole: SYN goes nowhere
|
||||||
|
Port = 3893,
|
||||||
|
Transport = LdapTransport.None,
|
||||||
|
AllowInsecure = true,
|
||||||
|
SearchBase = "dc=zb,dc=local",
|
||||||
|
ConnectionTimeoutMs = 500,
|
||||||
|
AuthTimeoutMs = 1000,
|
||||||
|
};
|
||||||
|
|
||||||
|
var ldap = new OtOpcUaLdapAuthService(Options.Create(options), NullLogger<OtOpcUaLdapAuthService>.Instance);
|
||||||
|
var mapper = new FakeMapper(g => g.ToArray());
|
||||||
|
var authenticator = new LdapOpcUaUserAuthenticator(
|
||||||
|
ldap, ScopeFactoryWith(mapper), Options.Create(options),
|
||||||
|
NullLogger<LdapOpcUaUserAuthenticator>.Instance);
|
||||||
|
|
||||||
|
var token = new UserNameIdentityToken { UserName = "alice", DecryptedPassword = Encoding.UTF8.GetBytes("secret") };
|
||||||
|
var policy = new UserTokenPolicy(UserTokenType.UserName) { PolicyId = "username_basic256sha256" };
|
||||||
|
var args = new ImpersonateEventArgs(token, policy, new EndpointDescription());
|
||||||
|
|
||||||
|
var sw = Stopwatch.StartNew();
|
||||||
|
await Task.Run(() => OpcUaApplicationHost.HandleImpersonation(
|
||||||
|
authenticator, args, NullLogger<object>.Instance), TestContext.Current.CancellationToken)
|
||||||
|
.WaitAsync(TimeSpan.FromSeconds(10), TestContext.Current.CancellationToken);
|
||||||
|
sw.Stop();
|
||||||
|
|
||||||
|
args.Identity.ShouldBeNull();
|
||||||
|
args.IdentityValidationError.ShouldNotBeNull();
|
||||||
|
args.IdentityValidationError.Code.ShouldBe(Opc.Ua.StatusCodes.BadIdentityTokenRejected);
|
||||||
|
sw.Elapsed.ShouldBeLessThan(TimeSpan.FromSeconds(3));
|
||||||
|
}
|
||||||
|
|
||||||
|
private static LdapAuthResult SystemFailure() =>
|
||||||
|
new(false, null, "u", Array.Empty<string>(), Array.Empty<string>(), "Unexpected authentication error")
|
||||||
|
{ IsSystemFailure = true };
|
||||||
|
|
||||||
|
private static LdapAuthResult UserDeny() =>
|
||||||
|
new(false, null, "u", Array.Empty<string>(), Array.Empty<string>(), "Invalid username or password")
|
||||||
|
{ IsSystemFailure = false };
|
||||||
|
|
||||||
|
private static LdapOpcUaUserAuthenticator Build(
|
||||||
|
ILdapAuthService ldap, IGroupRoleMapper<string> mapper, LdapOptions options) =>
|
||||||
|
new(ldap, ScopeFactoryWith(mapper), Options.Create(options),
|
||||||
|
NullLogger<LdapOpcUaUserAuthenticator>.Instance);
|
||||||
|
|
||||||
|
/// <summary>Builds an IServiceScopeFactory whose scopes resolve the supplied mapper.</summary>
|
||||||
|
private static IServiceScopeFactory ScopeFactoryWith(IGroupRoleMapper<string> mapper)
|
||||||
|
{
|
||||||
|
var services = new ServiceCollection();
|
||||||
|
services.AddScoped(_ => mapper);
|
||||||
|
return services.BuildServiceProvider().GetRequiredService<IServiceScopeFactory>();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// A fake <see cref="ILdapAuthService"/> that awaits a configurable delay (observing the token) before
|
||||||
|
/// returning a fixed result — the offline stand-in for a slow / hung directory. Counts invocations so
|
||||||
|
/// tests can assert the outage circuit denies WITHOUT reaching the backend.
|
||||||
|
/// </summary>
|
||||||
|
private sealed class DelayingFakeLdap : ILdapAuthService
|
||||||
|
{
|
||||||
|
private readonly TimeSpan _delay;
|
||||||
|
private readonly LdapAuthResult _then;
|
||||||
|
private int _calls;
|
||||||
|
|
||||||
|
/// <summary>Initializes the fake with a per-call delay and the result to return afterwards.</summary>
|
||||||
|
/// <param name="delay">How long each authenticate call blocks before returning.</param>
|
||||||
|
/// <param name="then">The result returned once the delay elapses.</param>
|
||||||
|
public DelayingFakeLdap(TimeSpan delay, LdapAuthResult then)
|
||||||
|
{
|
||||||
|
_delay = delay;
|
||||||
|
_then = then;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>The number of times the backend was actually invoked.</summary>
|
||||||
|
public int Calls => Volatile.Read(ref _calls);
|
||||||
|
|
||||||
|
/// <summary>Awaits the configured delay, then returns the canned result.</summary>
|
||||||
|
/// <param name="username">The username (ignored).</param>
|
||||||
|
/// <param name="password">The password (ignored).</param>
|
||||||
|
/// <param name="ct">Cancellation token, observed by the delay.</param>
|
||||||
|
public async Task<LdapAuthResult> AuthenticateAsync(string username, string password, CancellationToken ct = default)
|
||||||
|
{
|
||||||
|
Interlocked.Increment(ref _calls);
|
||||||
|
await Task.Delay(_delay, ct).ConfigureAwait(false);
|
||||||
|
return _then;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// A fake <see cref="ILdapAuthService"/> that parks ASYNCHRONOUSLY (awaits a
|
||||||
|
/// <see cref="TaskCompletionSource"/>) until the test calls <see cref="Release"/> — used to hold an
|
||||||
|
/// in-flight bind so the concurrency cap is deterministically saturated WITHOUT blocking a thread.
|
||||||
|
/// Records how many calls have entered (past the slot acquire).
|
||||||
|
/// </summary>
|
||||||
|
private sealed class GatedFakeLdap : ILdapAuthService
|
||||||
|
{
|
||||||
|
private readonly TaskCompletionSource _park = new(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||||
|
private readonly LdapAuthResult _then;
|
||||||
|
private int _entered;
|
||||||
|
|
||||||
|
/// <summary>Initializes the fake with the result to return once released.</summary>
|
||||||
|
/// <param name="then">The result returned once <see cref="Release"/> is called.</param>
|
||||||
|
public GatedFakeLdap(LdapAuthResult then) => _then = then;
|
||||||
|
|
||||||
|
/// <summary>Number of authenticate calls that reached the backend body.</summary>
|
||||||
|
public int Entered => Volatile.Read(ref _entered);
|
||||||
|
|
||||||
|
/// <summary>Releases the parked bind so its awaiting call can complete.</summary>
|
||||||
|
public void Release() => _park.TrySetResult();
|
||||||
|
|
||||||
|
/// <summary>Marks entry, awaits the parking gate, then returns the canned result.</summary>
|
||||||
|
/// <param name="username">The username (ignored).</param>
|
||||||
|
/// <param name="password">The password (ignored).</param>
|
||||||
|
/// <param name="ct">Cancellation token, observed while parked.</param>
|
||||||
|
public async Task<LdapAuthResult> AuthenticateAsync(string username, string password, CancellationToken ct = default)
|
||||||
|
{
|
||||||
|
Interlocked.Increment(ref _entered);
|
||||||
|
await _park.Task.WaitAsync(ct).ConfigureAwait(false);
|
||||||
|
return _then;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// A fake <see cref="ILdapAuthService"/> that returns a scripted sequence of results (one per call;
|
||||||
|
/// the last entry repeats once exhausted). Counts calls so the outage-circuit tests can assert the
|
||||||
|
/// backend was — or was NOT — reached on a given call.
|
||||||
|
/// </summary>
|
||||||
|
private sealed class ScriptedFakeLdap : ILdapAuthService
|
||||||
|
{
|
||||||
|
private readonly IReadOnlyList<LdapAuthResult> _script;
|
||||||
|
private int _calls;
|
||||||
|
|
||||||
|
/// <summary>Initializes the fake with the scripted results, returned in order.</summary>
|
||||||
|
/// <param name="script">The results to return, one per call; the last repeats once exhausted.</param>
|
||||||
|
public ScriptedFakeLdap(params LdapAuthResult[] script) => _script = script;
|
||||||
|
|
||||||
|
/// <summary>The number of times the backend was invoked.</summary>
|
||||||
|
public int Calls => Volatile.Read(ref _calls);
|
||||||
|
|
||||||
|
/// <summary>Returns the next scripted result (clamped to the last entry).</summary>
|
||||||
|
/// <param name="username">The username (ignored).</param>
|
||||||
|
/// <param name="password">The password (ignored).</param>
|
||||||
|
/// <param name="ct">The cancellation token (ignored).</param>
|
||||||
|
public Task<LdapAuthResult> AuthenticateAsync(string username, string password, CancellationToken ct = default)
|
||||||
|
{
|
||||||
|
var idx = Interlocked.Increment(ref _calls) - 1;
|
||||||
|
return Task.FromResult(_script[Math.Min(idx, _script.Count - 1)]);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>Test group→role mapper driven by a delegate over the supplied groups.</summary>
|
||||||
|
private sealed class FakeMapper(Func<IReadOnlyList<string>, IReadOnlyList<string>> map) : IGroupRoleMapper<string>
|
||||||
|
{
|
||||||
|
/// <summary>Maps groups to roles via the configured delegate; Scope is always null.</summary>
|
||||||
|
/// <param name="groups">The LDAP groups to map.</param>
|
||||||
|
/// <param name="ct">The cancellation token.</param>
|
||||||
|
public Task<GroupRoleMapping<string>> MapAsync(IReadOnlyList<string> groups, CancellationToken ct)
|
||||||
|
=> Task.FromResult(new GroupRoleMapping<string>(map(groups), Scope: null));
|
||||||
|
}
|
||||||
|
}
|
||||||
+7
-6
@@ -1,5 +1,6 @@
|
|||||||
using Microsoft.Extensions.DependencyInjection;
|
using Microsoft.Extensions.DependencyInjection;
|
||||||
using Microsoft.Extensions.Logging.Abstractions;
|
using Microsoft.Extensions.Logging.Abstractions;
|
||||||
|
using Microsoft.Extensions.Options;
|
||||||
using Shouldly;
|
using Shouldly;
|
||||||
using Xunit;
|
using Xunit;
|
||||||
using ZB.MOM.WW.Auth.Abstractions.Roles;
|
using ZB.MOM.WW.Auth.Abstractions.Roles;
|
||||||
@@ -26,7 +27,7 @@ public sealed class LdapOpcUaUserAuthenticatorTests
|
|||||||
// group "configeditor" -> "Designer" (canonical, Task 1.7).
|
// group "configeditor" -> "Designer" (canonical, Task 1.7).
|
||||||
var ldap = new FakeLdap(new LdapAuthResult(true, "Alice", "alice", new[] { "configeditor" }, Array.Empty<string>(), null));
|
var ldap = new FakeLdap(new LdapAuthResult(true, "Alice", "alice", new[] { "configeditor" }, Array.Empty<string>(), null));
|
||||||
var mapper = new FakeMapper(g => g.Select(x => x == "configeditor" ? "Designer" : x).ToArray());
|
var mapper = new FakeMapper(g => g.Select(x => x == "configeditor" ? "Designer" : x).ToArray());
|
||||||
var sut = new LdapOpcUaUserAuthenticator(ldap, ScopeFactoryWith(mapper), NullLogger<LdapOpcUaUserAuthenticator>.Instance);
|
var sut = new LdapOpcUaUserAuthenticator(ldap, ScopeFactoryWith(mapper), Options.Create(new LdapOptions()), NullLogger<LdapOpcUaUserAuthenticator>.Instance);
|
||||||
|
|
||||||
var result = await sut.AuthenticateUserNameAsync("alice", "secret", CancellationToken.None);
|
var result = await sut.AuthenticateUserNameAsync("alice", "secret", CancellationToken.None);
|
||||||
|
|
||||||
@@ -44,7 +45,7 @@ public sealed class LdapOpcUaUserAuthenticatorTests
|
|||||||
// nothing, so the union is exactly {Administrator}.
|
// nothing, so the union is exactly {Administrator}.
|
||||||
var ldap = new FakeLdap(new LdapAuthResult(true, "dev", "dev", new[] { "dev" }, new[] { "Administrator" }, null));
|
var ldap = new FakeLdap(new LdapAuthResult(true, "dev", "dev", new[] { "dev" }, new[] { "Administrator" }, null));
|
||||||
var mapper = new FakeMapper(_ => Array.Empty<string>());
|
var mapper = new FakeMapper(_ => Array.Empty<string>());
|
||||||
var sut = new LdapOpcUaUserAuthenticator(ldap, ScopeFactoryWith(mapper), NullLogger<LdapOpcUaUserAuthenticator>.Instance);
|
var sut = new LdapOpcUaUserAuthenticator(ldap, ScopeFactoryWith(mapper), Options.Create(new LdapOptions()), NullLogger<LdapOpcUaUserAuthenticator>.Instance);
|
||||||
|
|
||||||
var result = await sut.AuthenticateUserNameAsync("dev", "anything", CancellationToken.None);
|
var result = await sut.AuthenticateUserNameAsync("dev", "anything", CancellationToken.None);
|
||||||
|
|
||||||
@@ -59,7 +60,7 @@ public sealed class LdapOpcUaUserAuthenticatorTests
|
|||||||
{
|
{
|
||||||
var ldap = new FakeLdap(new LdapAuthResult(true, "dev", "dev", new[] { "dev" }, new[] { "Administrator" }, null));
|
var ldap = new FakeLdap(new LdapAuthResult(true, "dev", "dev", new[] { "dev" }, new[] { "Administrator" }, null));
|
||||||
var mapper = new FakeMapper(_ => throw new InvalidOperationException("DB down"));
|
var mapper = new FakeMapper(_ => throw new InvalidOperationException("DB down"));
|
||||||
var sut = new LdapOpcUaUserAuthenticator(ldap, ScopeFactoryWith(mapper), NullLogger<LdapOpcUaUserAuthenticator>.Instance);
|
var sut = new LdapOpcUaUserAuthenticator(ldap, ScopeFactoryWith(mapper), Options.Create(new LdapOptions()), NullLogger<LdapOpcUaUserAuthenticator>.Instance);
|
||||||
|
|
||||||
var result = await sut.AuthenticateUserNameAsync("dev", "anything", CancellationToken.None);
|
var result = await sut.AuthenticateUserNameAsync("dev", "anything", CancellationToken.None);
|
||||||
|
|
||||||
@@ -73,7 +74,7 @@ public sealed class LdapOpcUaUserAuthenticatorTests
|
|||||||
{
|
{
|
||||||
var ldap = new FakeLdap(new LdapAuthResult(false, null, "mallory", Array.Empty<string>(), Array.Empty<string>(), "Invalid username or password"));
|
var ldap = new FakeLdap(new LdapAuthResult(false, null, "mallory", Array.Empty<string>(), Array.Empty<string>(), "Invalid username or password"));
|
||||||
var mapper = new FakeMapper(g => g.ToArray());
|
var mapper = new FakeMapper(g => g.ToArray());
|
||||||
var sut = new LdapOpcUaUserAuthenticator(ldap, ScopeFactoryWith(mapper), NullLogger<LdapOpcUaUserAuthenticator>.Instance);
|
var sut = new LdapOpcUaUserAuthenticator(ldap, ScopeFactoryWith(mapper), Options.Create(new LdapOptions()), NullLogger<LdapOpcUaUserAuthenticator>.Instance);
|
||||||
|
|
||||||
var result = await sut.AuthenticateUserNameAsync("mallory", "wrong", CancellationToken.None);
|
var result = await sut.AuthenticateUserNameAsync("mallory", "wrong", CancellationToken.None);
|
||||||
|
|
||||||
@@ -87,7 +88,7 @@ public sealed class LdapOpcUaUserAuthenticatorTests
|
|||||||
{
|
{
|
||||||
var ldap = new FakeLdap(_ => throw new InvalidOperationException("LDAP unreachable"));
|
var ldap = new FakeLdap(_ => throw new InvalidOperationException("LDAP unreachable"));
|
||||||
var mapper = new FakeMapper(g => g.ToArray());
|
var mapper = new FakeMapper(g => g.ToArray());
|
||||||
var sut = new LdapOpcUaUserAuthenticator(ldap, ScopeFactoryWith(mapper), NullLogger<LdapOpcUaUserAuthenticator>.Instance);
|
var sut = new LdapOpcUaUserAuthenticator(ldap, ScopeFactoryWith(mapper), Options.Create(new LdapOptions()), NullLogger<LdapOpcUaUserAuthenticator>.Instance);
|
||||||
|
|
||||||
var result = await sut.AuthenticateUserNameAsync("anyone", "x", CancellationToken.None);
|
var result = await sut.AuthenticateUserNameAsync("anyone", "x", CancellationToken.None);
|
||||||
|
|
||||||
@@ -102,7 +103,7 @@ public sealed class LdapOpcUaUserAuthenticatorTests
|
|||||||
{
|
{
|
||||||
var ldap = new FakeLdap(new LdapAuthResult(true, null, "alice", new[] { "ReadOnly" }, Array.Empty<string>(), null));
|
var ldap = new FakeLdap(new LdapAuthResult(true, null, "alice", new[] { "ReadOnly" }, Array.Empty<string>(), null));
|
||||||
var mapper = new FakeMapper(g => g.ToArray());
|
var mapper = new FakeMapper(g => g.ToArray());
|
||||||
var sut = new LdapOpcUaUserAuthenticator(ldap, ScopeFactoryWith(mapper), NullLogger<LdapOpcUaUserAuthenticator>.Instance);
|
var sut = new LdapOpcUaUserAuthenticator(ldap, ScopeFactoryWith(mapper), Options.Create(new LdapOptions()), NullLogger<LdapOpcUaUserAuthenticator>.Instance);
|
||||||
|
|
||||||
var result = await sut.AuthenticateUserNameAsync("alice", "x", CancellationToken.None);
|
var result = await sut.AuthenticateUserNameAsync("alice", "x", CancellationToken.None);
|
||||||
|
|
||||||
|
|||||||
@@ -61,6 +61,56 @@ public sealed class LdapOptionsBindingTests
|
|||||||
|
|
||||||
options.DevStubMode.ShouldBeFalse();
|
options.DevStubMode.ShouldBeFalse();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// The five S2 resilience keys bind from the real <c>Security:Ldap</c> section (03/S2).
|
||||||
|
/// </summary>
|
||||||
|
[Fact]
|
||||||
|
public void Binding_reads_resilience_keys()
|
||||||
|
{
|
||||||
|
var configuration = new ConfigurationBuilder()
|
||||||
|
.AddInMemoryCollection(new Dictionary<string, string?>
|
||||||
|
{
|
||||||
|
["Security:Ldap:ConnectionTimeoutMs"] = "7000",
|
||||||
|
["Security:Ldap:AuthTimeoutMs"] = "12000",
|
||||||
|
["Security:Ldap:MaxConcurrentAuthentications"] = "5",
|
||||||
|
["Security:Ldap:OutageFailureThreshold"] = "4",
|
||||||
|
["Security:Ldap:OutageCooldownSeconds"] = "20",
|
||||||
|
})
|
||||||
|
.Build();
|
||||||
|
|
||||||
|
var options = configuration.GetSection(LdapOptions.SectionName).Get<LdapOptions>();
|
||||||
|
|
||||||
|
options.ShouldNotBeNull();
|
||||||
|
options.ConnectionTimeoutMs.ShouldBe(7000);
|
||||||
|
options.AuthTimeoutMs.ShouldBe(12000);
|
||||||
|
options.MaxConcurrentAuthentications.ShouldBe(5);
|
||||||
|
options.OutageFailureThreshold.ShouldBe(4);
|
||||||
|
options.OutageCooldownSeconds.ShouldBe(20);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>The resilience keys carry the documented defaults when absent from config.</summary>
|
||||||
|
[Fact]
|
||||||
|
public void Resilience_key_defaults_hold_when_absent()
|
||||||
|
{
|
||||||
|
var options = new LdapOptions();
|
||||||
|
|
||||||
|
options.ConnectionTimeoutMs.ShouldBe(10000);
|
||||||
|
options.AuthTimeoutMs.ShouldBe(15000);
|
||||||
|
options.MaxConcurrentAuthentications.ShouldBe(8);
|
||||||
|
options.OutageFailureThreshold.ShouldBe(3);
|
||||||
|
options.OutageCooldownSeconds.ShouldBe(15);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary><c>ConnectionTimeoutMs</c> is projected onto the shared-library options so the
|
||||||
|
/// socket-connect + per-search timeout is bounded inside the directory client.</summary>
|
||||||
|
[Fact]
|
||||||
|
public void ToLibraryOptions_projects_ConnectionTimeoutMs()
|
||||||
|
{
|
||||||
|
var options = new LdapOptions { ConnectionTimeoutMs = 4321 };
|
||||||
|
|
||||||
|
options.ToLibraryOptions().ConnectionTimeoutMs.ShouldBe(4321);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// <summary>
|
/// <summary>
|
||||||
|
|||||||
@@ -203,6 +203,95 @@ public sealed class LdapOptionsValidatorTests
|
|||||||
result.Failures.ShouldContain("Ldap:SearchBase is required when LDAP login is enabled.");
|
result.Failures.ShouldContain("Ldap:SearchBase is required when LDAP login is enabled.");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>Enabled with a non-positive <c>ConnectionTimeoutMs</c> fails validation (03/S2).</summary>
|
||||||
|
[Fact]
|
||||||
|
public void Enabled_with_nonpositive_ConnectionTimeoutMs_fails()
|
||||||
|
{
|
||||||
|
var options = new LdapOptions
|
||||||
|
{
|
||||||
|
Enabled = true,
|
||||||
|
Server = "ldap",
|
||||||
|
SearchBase = "dc=x",
|
||||||
|
Port = 389,
|
||||||
|
Transport = LdapTransport.Ldaps,
|
||||||
|
ConnectionTimeoutMs = 0,
|
||||||
|
};
|
||||||
|
|
||||||
|
Sut.Validate(null, options).Failed.ShouldBeTrue();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>Enabled with a non-positive <c>AuthTimeoutMs</c> fails validation (03/S2).</summary>
|
||||||
|
[Fact]
|
||||||
|
public void Enabled_with_nonpositive_AuthTimeoutMs_fails()
|
||||||
|
{
|
||||||
|
var options = new LdapOptions
|
||||||
|
{
|
||||||
|
Enabled = true,
|
||||||
|
Server = "ldap",
|
||||||
|
SearchBase = "dc=x",
|
||||||
|
Port = 389,
|
||||||
|
Transport = LdapTransport.Ldaps,
|
||||||
|
AuthTimeoutMs = 0,
|
||||||
|
};
|
||||||
|
|
||||||
|
Sut.Validate(null, options).Failed.ShouldBeTrue();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>Enabled with a non-positive <c>MaxConcurrentAuthentications</c> fails validation (03/S2).</summary>
|
||||||
|
[Fact]
|
||||||
|
public void Enabled_with_nonpositive_MaxConcurrentAuthentications_fails()
|
||||||
|
{
|
||||||
|
var options = new LdapOptions
|
||||||
|
{
|
||||||
|
Enabled = true,
|
||||||
|
Server = "ldap",
|
||||||
|
SearchBase = "dc=x",
|
||||||
|
Port = 389,
|
||||||
|
Transport = LdapTransport.Ldaps,
|
||||||
|
MaxConcurrentAuthentications = 0,
|
||||||
|
};
|
||||||
|
|
||||||
|
Sut.Validate(null, options).Failed.ShouldBeTrue();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>A positive <c>OutageFailureThreshold</c> with a zero <c>OutageCooldownSeconds</c> fails —
|
||||||
|
/// an open circuit with no cooldown would probe on every call (03/S2).</summary>
|
||||||
|
[Fact]
|
||||||
|
public void OutageFailureThreshold_positive_with_zero_cooldown_fails()
|
||||||
|
{
|
||||||
|
var options = new LdapOptions
|
||||||
|
{
|
||||||
|
Enabled = true,
|
||||||
|
Server = "ldap",
|
||||||
|
SearchBase = "dc=x",
|
||||||
|
Port = 389,
|
||||||
|
Transport = LdapTransport.Ldaps,
|
||||||
|
OutageFailureThreshold = 3,
|
||||||
|
OutageCooldownSeconds = 0,
|
||||||
|
};
|
||||||
|
|
||||||
|
Sut.Validate(null, options).Failed.ShouldBeTrue();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>A zero <c>OutageFailureThreshold</c> (circuit disabled) passes regardless of the
|
||||||
|
/// cooldown value (03/S2).</summary>
|
||||||
|
[Fact]
|
||||||
|
public void OutageFailureThreshold_zero_disables_circuit_and_passes()
|
||||||
|
{
|
||||||
|
var options = new LdapOptions
|
||||||
|
{
|
||||||
|
Enabled = true,
|
||||||
|
Server = "ldap",
|
||||||
|
SearchBase = "dc=x",
|
||||||
|
Port = 389,
|
||||||
|
Transport = LdapTransport.Ldaps,
|
||||||
|
OutageFailureThreshold = 0,
|
||||||
|
OutageCooldownSeconds = 0,
|
||||||
|
};
|
||||||
|
|
||||||
|
Sut.Validate(null, options).Succeeded.ShouldBeTrue();
|
||||||
|
}
|
||||||
|
|
||||||
/// <summary>Enabled with port 0 reports the port-range failure using the shared primitive wording.</summary>
|
/// <summary>Enabled with port 0 reports the port-range failure using the shared primitive wording.</summary>
|
||||||
[Fact]
|
[Fact]
|
||||||
public void Enabled_with_zero_port_fails()
|
public void Enabled_with_zero_port_fails()
|
||||||
|
|||||||
+466
@@ -0,0 +1,466 @@
|
|||||||
|
using System.Diagnostics;
|
||||||
|
using Microsoft.Extensions.Logging.Abstractions;
|
||||||
|
using Opc.Ua;
|
||||||
|
using Opc.Ua.Server;
|
||||||
|
using Shouldly;
|
||||||
|
using Xunit;
|
||||||
|
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
||||||
|
using HistorianRead = ZB.MOM.WW.OtOpcUa.Core.Abstractions.HistoryReadResult;
|
||||||
|
using SdkHistoryReadResult = Opc.Ua.HistoryReadResult;
|
||||||
|
|
||||||
|
namespace ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Arch-review 03/S3 — the node-manager's HistoryRead arms must serve a multi-node batch with BOUNDED
|
||||||
|
/// per-node parallelism, a per-request deadline, and a process-wide concurrent-batch limiter, rather
|
||||||
|
/// than block-bridging the gateway per node sequentially on an SDK request thread. Boots the same real
|
||||||
|
/// <see cref="OtOpcUaSdkServer"/> harness as <see cref="NodeManagerHistoryReadTests"/> and drives the
|
||||||
|
/// public <c>HistoryRead</c> with a delaying fake <see cref="IHistorianDataSource"/> that records the
|
||||||
|
/// maximum observed concurrent reads (deterministic — no raw-timing dependence for the primary
|
||||||
|
/// assertions).
|
||||||
|
/// </summary>
|
||||||
|
public sealed class NodeManagerHistoryReadConcurrencyTests : IDisposable
|
||||||
|
{
|
||||||
|
private static CancellationToken Ct => TestContext.Current.CancellationToken;
|
||||||
|
|
||||||
|
private readonly string _pkiRoot = Path.Combine(
|
||||||
|
Path.GetTempPath(),
|
||||||
|
$"otopcua-historyread-conc-{Guid.NewGuid():N}");
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// RED repro (03/S3): a single Raw HistoryRead naming 8 historized nodes against a 150 ms-per-read
|
||||||
|
/// backend must serve at least two nodes concurrently. On the current sequential <c>foreach</c> the
|
||||||
|
/// max observed concurrency is pinned at 1, so this fails.
|
||||||
|
/// </summary>
|
||||||
|
[Fact]
|
||||||
|
public async Task Raw_batch_is_served_with_bounded_parallelism()
|
||||||
|
{
|
||||||
|
var (host, server) = await BootAsync();
|
||||||
|
var nm = server.NodeManager!;
|
||||||
|
var fake = new ConcurrencyTrackingHistorianDataSource(TimeSpan.FromMilliseconds(150));
|
||||||
|
nm.HistorianDataSource = fake;
|
||||||
|
|
||||||
|
var nodeIds = MaterializeHistorizedNodes(nm, count: 8);
|
||||||
|
|
||||||
|
var details = new ReadRawModifiedDetails
|
||||||
|
{
|
||||||
|
StartTime = DateTime.UtcNow.AddHours(-1),
|
||||||
|
EndTime = DateTime.UtcNow,
|
||||||
|
NumValuesPerNode = 10,
|
||||||
|
IsReadModified = false,
|
||||||
|
};
|
||||||
|
|
||||||
|
var sw = Stopwatch.StartNew();
|
||||||
|
await Task.Run(() => InvokeHistoryRead(server, nm, details, nodeIds), Ct)
|
||||||
|
.WaitAsync(TimeSpan.FromSeconds(15), Ct);
|
||||||
|
sw.Stop();
|
||||||
|
|
||||||
|
// Primary (deterministic): the batch was served with >1 node in flight at once.
|
||||||
|
fake.MaxObservedConcurrency.ShouldBeGreaterThanOrEqualTo(2);
|
||||||
|
|
||||||
|
await host.DisposeAsync();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Parallelism is bounded ABOVE by <c>HistoryReadBatchParallelism</c>: with the property set to 3 and
|
||||||
|
/// 12 nodes in flight, no more than 3 reads are ever concurrent (03/S3).
|
||||||
|
/// </summary>
|
||||||
|
[Fact]
|
||||||
|
public async Task Parallelism_is_bounded_above()
|
||||||
|
{
|
||||||
|
var (host, server) = await BootAsync();
|
||||||
|
var nm = server.NodeManager!;
|
||||||
|
nm.HistoryReadBatchParallelism = 3;
|
||||||
|
var fake = new ConcurrencyTrackingHistorianDataSource(TimeSpan.FromMilliseconds(100));
|
||||||
|
nm.HistorianDataSource = fake;
|
||||||
|
|
||||||
|
var nodeIds = MaterializeHistorizedNodes(nm, count: 12);
|
||||||
|
var details = new ReadRawModifiedDetails
|
||||||
|
{
|
||||||
|
StartTime = DateTime.UtcNow.AddHours(-1),
|
||||||
|
EndTime = DateTime.UtcNow,
|
||||||
|
NumValuesPerNode = 10,
|
||||||
|
IsReadModified = false,
|
||||||
|
};
|
||||||
|
|
||||||
|
await Task.Run(() => InvokeHistoryRead(server, nm, details, nodeIds), Ct)
|
||||||
|
.WaitAsync(TimeSpan.FromSeconds(15), Ct);
|
||||||
|
|
||||||
|
fake.MaxObservedConcurrency.ShouldBeGreaterThanOrEqualTo(2);
|
||||||
|
fake.MaxObservedConcurrency.ShouldBeLessThanOrEqualTo(3);
|
||||||
|
|
||||||
|
await host.DisposeAsync();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// One node's backend throw does not poison a parallel batch: that node surfaces
|
||||||
|
/// <c>BadHistoryOperationUnsupported</c> while the other seven return Good (per-node isolation holds
|
||||||
|
/// under parallelism — 03/S3).
|
||||||
|
/// </summary>
|
||||||
|
[Fact]
|
||||||
|
public async Task One_node_failure_does_not_poison_a_parallel_batch()
|
||||||
|
{
|
||||||
|
var (host, server) = await BootAsync();
|
||||||
|
var nm = server.NodeManager!;
|
||||||
|
|
||||||
|
var src = DateTime.UtcNow.AddSeconds(-5);
|
||||||
|
var srv = DateTime.UtcNow;
|
||||||
|
// Node index 3's tagname throws; all others return a single good sample.
|
||||||
|
var fake = new PerTagnameFake(tagname => tagname == "WW.Tag3"
|
||||||
|
? throw new InvalidOperationException("backend boom for Tag3")
|
||||||
|
: new HistorianRead(new[] { new DataValueSnapshot(1.0f, StatusCodes.Good, src, srv) }, null));
|
||||||
|
nm.HistorianDataSource = fake;
|
||||||
|
|
||||||
|
var nodeIds = MaterializeHistorizedNodes(nm, count: 8);
|
||||||
|
var details = new ReadRawModifiedDetails
|
||||||
|
{
|
||||||
|
StartTime = DateTime.UtcNow.AddHours(-1),
|
||||||
|
EndTime = DateTime.UtcNow,
|
||||||
|
NumValuesPerNode = 10,
|
||||||
|
IsReadModified = false,
|
||||||
|
};
|
||||||
|
|
||||||
|
var (results, errors) = await Task.Run(() => InvokeHistoryRead(server, nm, details, nodeIds), Ct)
|
||||||
|
.WaitAsync(TimeSpan.FromSeconds(15), Ct);
|
||||||
|
|
||||||
|
for (var i = 0; i < 8; i++)
|
||||||
|
{
|
||||||
|
if (i == 3)
|
||||||
|
{
|
||||||
|
StatusCode.IsBad(errors[i].StatusCode).ShouldBeTrue();
|
||||||
|
errors[i].StatusCode.Code.ShouldBe(StatusCodes.BadHistoryOperationUnsupported);
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
errors[i].StatusCode.Code.ShouldBe(StatusCodes.Good);
|
||||||
|
results[i].StatusCode.Code.ShouldBe(StatusCodes.Good);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
await host.DisposeAsync();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// A per-request deadline bounds a hung backend: reads that only complete on token cancellation
|
||||||
|
/// surface <c>BadTimeout</c> for every node and the call returns within the deadline (plus slack) —
|
||||||
|
/// it does not hang an SDK request thread indefinitely (03/S3).
|
||||||
|
/// </summary>
|
||||||
|
[Fact]
|
||||||
|
public async Task Deadline_bounds_a_hung_backend()
|
||||||
|
{
|
||||||
|
var (host, server) = await BootAsync();
|
||||||
|
var nm = server.NodeManager!;
|
||||||
|
nm.HistoryReadDeadline = TimeSpan.FromMilliseconds(500);
|
||||||
|
nm.HistorianDataSource = new HangingHistorianDataSource();
|
||||||
|
|
||||||
|
var nodeIds = MaterializeHistorizedNodes(nm, count: 4);
|
||||||
|
var details = new ReadRawModifiedDetails
|
||||||
|
{
|
||||||
|
StartTime = DateTime.UtcNow.AddHours(-1),
|
||||||
|
EndTime = DateTime.UtcNow,
|
||||||
|
NumValuesPerNode = 10,
|
||||||
|
IsReadModified = false,
|
||||||
|
};
|
||||||
|
|
||||||
|
var (_, errors) = await Task.Run(() => InvokeHistoryRead(server, nm, details, nodeIds), Ct)
|
||||||
|
.WaitAsync(TimeSpan.FromSeconds(10), Ct);
|
||||||
|
|
||||||
|
for (var i = 0; i < 4; i++)
|
||||||
|
errors[i].StatusCode.Code.ShouldBe(StatusCodes.BadTimeout);
|
||||||
|
|
||||||
|
await host.DisposeAsync();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// A saturated process-wide batch limiter degrades a second concurrent HistoryRead to
|
||||||
|
/// <c>BadTooManyOperations</c> (rather than exhausting the SDK request-thread pool); once the parked
|
||||||
|
/// batch completes the slot frees and a later read succeeds (03/S3).
|
||||||
|
/// </summary>
|
||||||
|
[Fact]
|
||||||
|
public async Task Saturated_limiter_rejects_with_BadTooManyOperations()
|
||||||
|
{
|
||||||
|
var (host, server) = await BootAsync();
|
||||||
|
var nm = server.NodeManager!;
|
||||||
|
nm.MaxConcurrentHistoryReadBatches = 1;
|
||||||
|
nm.HistoryReadDeadline = TimeSpan.FromMilliseconds(200);
|
||||||
|
var fake = new GatedRawFake();
|
||||||
|
nm.HistorianDataSource = fake;
|
||||||
|
|
||||||
|
var ids = MaterializeHistorizedNodes(nm, count: 2);
|
||||||
|
var details = new ReadRawModifiedDetails
|
||||||
|
{
|
||||||
|
StartTime = DateTime.UtcNow.AddHours(-1),
|
||||||
|
EndTime = DateTime.UtcNow,
|
||||||
|
NumValuesPerNode = 10,
|
||||||
|
IsReadModified = false,
|
||||||
|
};
|
||||||
|
|
||||||
|
// First batch parks in the gated fake, holding the single limiter permit.
|
||||||
|
var first = Task.Run(() => InvokeHistoryRead(server, nm, details, ids[0]), Ct);
|
||||||
|
|
||||||
|
var spin = Stopwatch.StartNew();
|
||||||
|
while (fake.Entered == 0 && spin.Elapsed < TimeSpan.FromSeconds(5))
|
||||||
|
await Task.Delay(10, Ct);
|
||||||
|
fake.Entered.ShouldBe(1);
|
||||||
|
|
||||||
|
// Second concurrent batch: the limiter is saturated → all handles BadTooManyOperations.
|
||||||
|
var (_, errors2) = await Task.Run(() => InvokeHistoryRead(server, nm, details, ids[1]), Ct)
|
||||||
|
.WaitAsync(TimeSpan.FromSeconds(5), Ct);
|
||||||
|
errors2[0].StatusCode.Code.ShouldBe(StatusCodes.BadTooManyOperations);
|
||||||
|
fake.Entered.ShouldBe(1, "the rejected batch must never reach the backend");
|
||||||
|
|
||||||
|
// Release the parked batch; the slot frees and a later read succeeds.
|
||||||
|
fake.Release();
|
||||||
|
await first.WaitAsync(TimeSpan.FromSeconds(10), Ct);
|
||||||
|
|
||||||
|
var (_, errors3) = await Task.Run(() => InvokeHistoryRead(server, nm, details, ids[1]), Ct)
|
||||||
|
.WaitAsync(TimeSpan.FromSeconds(10), Ct);
|
||||||
|
errors3[0].StatusCode.Code.ShouldBe(StatusCodes.Good);
|
||||||
|
|
||||||
|
await host.DisposeAsync();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>Materialises <paramref name="count"/> historized Float variables and returns their NodeIds.</summary>
|
||||||
|
private static NodeId[] MaterializeHistorizedNodes(OtOpcUaNodeManager nm, int count)
|
||||||
|
{
|
||||||
|
var ids = new NodeId[count];
|
||||||
|
for (var i = 0; i < count; i++)
|
||||||
|
{
|
||||||
|
var key = $"eq/tag{i}";
|
||||||
|
nm.EnsureVariable(key, parentFolderNodeId: null, displayName: $"Tag{i}", dataType: "Float",
|
||||||
|
writable: false, historianTagname: $"WW.Tag{i}");
|
||||||
|
ids[i] = nm.TryGetVariable(key)!.NodeId;
|
||||||
|
}
|
||||||
|
|
||||||
|
return ids;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>Issues one HistoryRead batch for all supplied node ids in order.</summary>
|
||||||
|
private static (IList<SdkHistoryReadResult> Results, IList<ServiceResult> Errors) InvokeHistoryRead(
|
||||||
|
OtOpcUaSdkServer server, OtOpcUaNodeManager nm, HistoryReadDetails details, params NodeId[] nodeIds)
|
||||||
|
{
|
||||||
|
var context = new OperationContext(
|
||||||
|
new RequestHeader(), secureChannelContext: null, RequestType.HistoryRead, identity: null);
|
||||||
|
|
||||||
|
var nodesToRead = nodeIds.Select(id => new HistoryReadValueId { NodeId = id }).ToList();
|
||||||
|
var results = Enumerable.Repeat<SdkHistoryReadResult>(null!, nodeIds.Length).ToList();
|
||||||
|
var errors = Enumerable.Repeat<ServiceResult>(null!, nodeIds.Length).ToList();
|
||||||
|
|
||||||
|
nm.HistoryRead(
|
||||||
|
context,
|
||||||
|
details,
|
||||||
|
TimestampsToReturn.Both,
|
||||||
|
releaseContinuationPoints: false,
|
||||||
|
nodesToRead,
|
||||||
|
results,
|
||||||
|
errors);
|
||||||
|
|
||||||
|
return (results, errors);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// A delaying fake historian source that tracks the maximum number of reads in flight at once
|
||||||
|
/// (via interlocked live-counter) so a test can prove per-node parallelism deterministically.
|
||||||
|
/// </summary>
|
||||||
|
private sealed class ConcurrencyTrackingHistorianDataSource(TimeSpan delay) : IHistorianDataSource
|
||||||
|
{
|
||||||
|
private int _current;
|
||||||
|
private int _max;
|
||||||
|
|
||||||
|
/// <summary>The highest number of concurrently in-flight reads observed.</summary>
|
||||||
|
public int MaxObservedConcurrency => Volatile.Read(ref _max);
|
||||||
|
|
||||||
|
private async Task<HistorianRead> TrackAsync(CancellationToken ct)
|
||||||
|
{
|
||||||
|
var now = Interlocked.Increment(ref _current);
|
||||||
|
UpdateMax(now);
|
||||||
|
try
|
||||||
|
{
|
||||||
|
await Task.Delay(delay, ct).ConfigureAwait(false);
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
Interlocked.Decrement(ref _current);
|
||||||
|
}
|
||||||
|
|
||||||
|
return new HistorianRead(
|
||||||
|
new[] { new DataValueSnapshot(1.0f, StatusCodes.Good, DateTime.UtcNow, DateTime.UtcNow) }, null);
|
||||||
|
}
|
||||||
|
|
||||||
|
private void UpdateMax(int observed)
|
||||||
|
{
|
||||||
|
int cur;
|
||||||
|
while ((cur = Volatile.Read(ref _max)) < observed &&
|
||||||
|
Interlocked.CompareExchange(ref _max, observed, cur) != cur)
|
||||||
|
{
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public Task<HistorianRead> ReadRawAsync(
|
||||||
|
string fullReference, DateTime startUtc, DateTime endUtc, uint maxValuesPerNode,
|
||||||
|
CancellationToken cancellationToken) => TrackAsync(cancellationToken);
|
||||||
|
|
||||||
|
public Task<HistorianRead> ReadProcessedAsync(
|
||||||
|
string fullReference, DateTime startUtc, DateTime endUtc, TimeSpan interval,
|
||||||
|
HistoryAggregateType aggregate, CancellationToken cancellationToken) => TrackAsync(cancellationToken);
|
||||||
|
|
||||||
|
public Task<HistorianRead> ReadAtTimeAsync(
|
||||||
|
string fullReference, IReadOnlyList<DateTime> timestampsUtc, CancellationToken cancellationToken) =>
|
||||||
|
TrackAsync(cancellationToken);
|
||||||
|
|
||||||
|
public Task<HistoricalEventsResult> ReadEventsAsync(
|
||||||
|
string? sourceName, DateTime startUtc, DateTime endUtc, int maxEvents,
|
||||||
|
CancellationToken cancellationToken) =>
|
||||||
|
Task.FromResult(new HistoricalEventsResult(Array.Empty<HistoricalEvent>(), null));
|
||||||
|
|
||||||
|
public HistorianHealthSnapshot GetHealthSnapshot() => NullHistorianDataSource.Instance.GetHealthSnapshot();
|
||||||
|
|
||||||
|
public void Dispose()
|
||||||
|
{
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>A fake whose reads complete ONLY on token cancellation — used to prove the per-request
|
||||||
|
/// deadline surfaces <c>BadTimeout</c> rather than hanging.</summary>
|
||||||
|
private sealed class HangingHistorianDataSource : IHistorianDataSource
|
||||||
|
{
|
||||||
|
private static async Task<HistorianRead> HangAsync(CancellationToken ct)
|
||||||
|
{
|
||||||
|
await Task.Delay(Timeout.Infinite, ct).ConfigureAwait(false);
|
||||||
|
return new HistorianRead(Array.Empty<DataValueSnapshot>(), null);
|
||||||
|
}
|
||||||
|
|
||||||
|
public Task<HistorianRead> ReadRawAsync(
|
||||||
|
string fullReference, DateTime startUtc, DateTime endUtc, uint maxValuesPerNode,
|
||||||
|
CancellationToken cancellationToken) => HangAsync(cancellationToken);
|
||||||
|
|
||||||
|
public Task<HistorianRead> ReadProcessedAsync(
|
||||||
|
string fullReference, DateTime startUtc, DateTime endUtc, TimeSpan interval,
|
||||||
|
HistoryAggregateType aggregate, CancellationToken cancellationToken) => HangAsync(cancellationToken);
|
||||||
|
|
||||||
|
public Task<HistorianRead> ReadAtTimeAsync(
|
||||||
|
string fullReference, IReadOnlyList<DateTime> timestampsUtc, CancellationToken cancellationToken) =>
|
||||||
|
HangAsync(cancellationToken);
|
||||||
|
|
||||||
|
public async Task<HistoricalEventsResult> ReadEventsAsync(
|
||||||
|
string? sourceName, DateTime startUtc, DateTime endUtc, int maxEvents,
|
||||||
|
CancellationToken cancellationToken)
|
||||||
|
{
|
||||||
|
await Task.Delay(Timeout.Infinite, cancellationToken).ConfigureAwait(false);
|
||||||
|
return new HistoricalEventsResult(Array.Empty<HistoricalEvent>(), null);
|
||||||
|
}
|
||||||
|
|
||||||
|
public HistorianHealthSnapshot GetHealthSnapshot() => NullHistorianDataSource.Instance.GetHealthSnapshot();
|
||||||
|
|
||||||
|
public void Dispose() { }
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>A fake whose raw read parks (awaits a <see cref="TaskCompletionSource"/>) until the test
|
||||||
|
/// releases it — used to hold the process-wide batch limiter permit while a second batch is issued.</summary>
|
||||||
|
private sealed class GatedRawFake : IHistorianDataSource
|
||||||
|
{
|
||||||
|
private readonly TaskCompletionSource _gate = new(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||||
|
private int _entered;
|
||||||
|
|
||||||
|
/// <summary>Number of raw reads that reached the backend body.</summary>
|
||||||
|
public int Entered => Volatile.Read(ref _entered);
|
||||||
|
|
||||||
|
/// <summary>Releases the parked read so its awaiting batch can complete.</summary>
|
||||||
|
public void Release() => _gate.TrySetResult();
|
||||||
|
|
||||||
|
public async Task<HistorianRead> ReadRawAsync(
|
||||||
|
string fullReference, DateTime startUtc, DateTime endUtc, uint maxValuesPerNode,
|
||||||
|
CancellationToken cancellationToken)
|
||||||
|
{
|
||||||
|
Interlocked.Increment(ref _entered);
|
||||||
|
// Deliberately ignore the per-request deadline token: the first batch must keep HOLDING the
|
||||||
|
// single limiter permit until the test releases it, so the second batch deterministically hits
|
||||||
|
// a saturated limiter (rather than the first batch's own deadline freeing the slot early).
|
||||||
|
await _gate.Task.ConfigureAwait(false);
|
||||||
|
return new HistorianRead(
|
||||||
|
new[] { new DataValueSnapshot(1.0f, StatusCodes.Good, DateTime.UtcNow, DateTime.UtcNow) }, null);
|
||||||
|
}
|
||||||
|
|
||||||
|
public Task<HistorianRead> ReadProcessedAsync(
|
||||||
|
string fullReference, DateTime startUtc, DateTime endUtc, TimeSpan interval,
|
||||||
|
HistoryAggregateType aggregate, CancellationToken cancellationToken) =>
|
||||||
|
NullHistorianDataSource.Instance.ReadProcessedAsync(fullReference, startUtc, endUtc, interval, aggregate, cancellationToken);
|
||||||
|
|
||||||
|
public Task<HistorianRead> ReadAtTimeAsync(
|
||||||
|
string fullReference, IReadOnlyList<DateTime> timestampsUtc, CancellationToken cancellationToken) =>
|
||||||
|
NullHistorianDataSource.Instance.ReadAtTimeAsync(fullReference, timestampsUtc, cancellationToken);
|
||||||
|
|
||||||
|
public Task<HistoricalEventsResult> ReadEventsAsync(
|
||||||
|
string? sourceName, DateTime startUtc, DateTime endUtc, int maxEvents,
|
||||||
|
CancellationToken cancellationToken) =>
|
||||||
|
NullHistorianDataSource.Instance.ReadEventsAsync(sourceName, startUtc, endUtc, maxEvents, cancellationToken);
|
||||||
|
|
||||||
|
public HistorianHealthSnapshot GetHealthSnapshot() => NullHistorianDataSource.Instance.GetHealthSnapshot();
|
||||||
|
|
||||||
|
public void Dispose() { }
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>A per-tagname fake that delegates each raw read to a caller-supplied func (which may throw
|
||||||
|
/// to simulate a per-node backend failure). Processed/AtTime/Events delegate to the null source.</summary>
|
||||||
|
private sealed class PerTagnameFake(Func<string, HistorianRead> rawHandler) : IHistorianDataSource
|
||||||
|
{
|
||||||
|
public Task<HistorianRead> ReadRawAsync(
|
||||||
|
string fullReference, DateTime startUtc, DateTime endUtc, uint maxValuesPerNode,
|
||||||
|
CancellationToken cancellationToken) => Task.FromResult(rawHandler(fullReference));
|
||||||
|
|
||||||
|
public Task<HistorianRead> ReadProcessedAsync(
|
||||||
|
string fullReference, DateTime startUtc, DateTime endUtc, TimeSpan interval,
|
||||||
|
HistoryAggregateType aggregate, CancellationToken cancellationToken) =>
|
||||||
|
NullHistorianDataSource.Instance.ReadProcessedAsync(fullReference, startUtc, endUtc, interval, aggregate, cancellationToken);
|
||||||
|
|
||||||
|
public Task<HistorianRead> ReadAtTimeAsync(
|
||||||
|
string fullReference, IReadOnlyList<DateTime> timestampsUtc, CancellationToken cancellationToken) =>
|
||||||
|
NullHistorianDataSource.Instance.ReadAtTimeAsync(fullReference, timestampsUtc, cancellationToken);
|
||||||
|
|
||||||
|
public Task<HistoricalEventsResult> ReadEventsAsync(
|
||||||
|
string? sourceName, DateTime startUtc, DateTime endUtc, int maxEvents,
|
||||||
|
CancellationToken cancellationToken) =>
|
||||||
|
NullHistorianDataSource.Instance.ReadEventsAsync(sourceName, startUtc, endUtc, maxEvents, cancellationToken);
|
||||||
|
|
||||||
|
public HistorianHealthSnapshot GetHealthSnapshot() => NullHistorianDataSource.Instance.GetHealthSnapshot();
|
||||||
|
|
||||||
|
public void Dispose() { }
|
||||||
|
}
|
||||||
|
|
||||||
|
private async Task<(OpcUaApplicationHost Host, OtOpcUaSdkServer Server)> BootAsync()
|
||||||
|
{
|
||||||
|
var host = new OpcUaApplicationHost(
|
||||||
|
new OpcUaApplicationHostOptions
|
||||||
|
{
|
||||||
|
ApplicationName = "OtOpcUa.HistoryReadConcTest",
|
||||||
|
ApplicationUri = $"urn:OtOpcUa.HistoryReadConcTest:{Guid.NewGuid():N}",
|
||||||
|
OpcUaPort = AllocateFreePort(),
|
||||||
|
PublicHostname = "localhost",
|
||||||
|
PkiStoreRoot = _pkiRoot,
|
||||||
|
},
|
||||||
|
NullLogger<OpcUaApplicationHost>.Instance);
|
||||||
|
|
||||||
|
var server = new OtOpcUaSdkServer();
|
||||||
|
await host.StartAsync(server, Ct);
|
||||||
|
return (host, server);
|
||||||
|
}
|
||||||
|
|
||||||
|
private static int AllocateFreePort()
|
||||||
|
{
|
||||||
|
using var listener = new System.Net.Sockets.TcpListener(System.Net.IPAddress.Loopback, 0);
|
||||||
|
listener.Start();
|
||||||
|
var port = ((System.Net.IPEndPoint)listener.LocalEndpoint).Port;
|
||||||
|
listener.Stop();
|
||||||
|
return port;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>Cleans up the PKI root directory.</summary>
|
||||||
|
public void Dispose()
|
||||||
|
{
|
||||||
|
if (Directory.Exists(_pkiRoot))
|
||||||
|
{
|
||||||
|
try { Directory.Delete(_pkiRoot, recursive: true); }
|
||||||
|
catch { /* best-effort cleanup */ }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -50,4 +50,55 @@ public sealed class ServerHistorianOptionsTests
|
|||||||
.Validate();
|
.Validate();
|
||||||
Assert.Contains(w, m => m.Contains("MaxTieClusterOverfetch"));
|
Assert.Contains(w, m => m.Contains("MaxTieClusterOverfetch"));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ── S3 HistoryRead concurrency knobs (arch-review 03/S3) ────────────────────────────────────
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void HistoryRead_knob_defaults()
|
||||||
|
{
|
||||||
|
var o = new ServerHistorianOptions();
|
||||||
|
Assert.Equal(4, o.HistoryReadBatchParallelism);
|
||||||
|
Assert.Equal(16, o.MaxConcurrentHistoryReadBatches);
|
||||||
|
Assert.Equal(TimeSpan.FromSeconds(60), o.HistoryReadDeadline);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Enabled_with_nonpositive_HistoryReadBatchParallelism_warns()
|
||||||
|
{
|
||||||
|
var w = new ServerHistorianOptions
|
||||||
|
{ Enabled = true, Endpoint = "https://h:5222", ApiKey = "histgw_x_y", HistoryReadBatchParallelism = 0 }
|
||||||
|
.Validate();
|
||||||
|
Assert.Contains(w, m => m.Contains("HistoryReadBatchParallelism"));
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Enabled_with_nonpositive_MaxConcurrentHistoryReadBatches_warns()
|
||||||
|
{
|
||||||
|
var w = new ServerHistorianOptions
|
||||||
|
{ Enabled = true, Endpoint = "https://h:5222", ApiKey = "histgw_x_y", MaxConcurrentHistoryReadBatches = 0 }
|
||||||
|
.Validate();
|
||||||
|
Assert.Contains(w, m => m.Contains("MaxConcurrentHistoryReadBatches"));
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Enabled_with_nonpositive_HistoryReadDeadline_warns()
|
||||||
|
{
|
||||||
|
var w = new ServerHistorianOptions
|
||||||
|
{ Enabled = true, Endpoint = "https://h:5222", ApiKey = "histgw_x_y", HistoryReadDeadline = TimeSpan.Zero }
|
||||||
|
.Validate();
|
||||||
|
Assert.Contains(w, m => m.Contains("HistoryReadDeadline"));
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Disabled_with_nonpositive_HistoryRead_knobs_is_silent()
|
||||||
|
{
|
||||||
|
var w = new ServerHistorianOptions
|
||||||
|
{
|
||||||
|
Enabled = false,
|
||||||
|
HistoryReadBatchParallelism = 0,
|
||||||
|
MaxConcurrentHistoryReadBatches = 0,
|
||||||
|
HistoryReadDeadline = TimeSpan.Zero,
|
||||||
|
}.Validate();
|
||||||
|
Assert.Empty(w);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -121,6 +121,56 @@ public sealed class OtOpcUaLdapAuthServiceTests
|
|||||||
inner.Called.ShouldBeFalse();
|
inner.Called.ShouldBeFalse();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>A system-side library failure (<c>ServiceAccountBindFailed</c> — the directory-unreachable
|
||||||
|
/// bucket) adapts to <c>IsSystemFailure == true</c> so the OPC UA authenticator's outage circuit can
|
||||||
|
/// distinguish it from a user-credential deny (03/S2).</summary>
|
||||||
|
[Fact]
|
||||||
|
public async Task ServiceAccountBindFailed_adapts_to_IsSystemFailure_true()
|
||||||
|
{
|
||||||
|
var inner = new RecordingLibService(LibLdapAuthResult.Fail(LdapAuthFailure.ServiceAccountBindFailed));
|
||||||
|
var sut = Build(
|
||||||
|
new LdapOptions { Enabled = true, DevStubMode = false, Transport = LdapTransport.Ldaps },
|
||||||
|
inner);
|
||||||
|
|
||||||
|
var result = await sut.AuthenticateAsync("alice", "secret", CancellationToken.None);
|
||||||
|
|
||||||
|
result.Success.ShouldBeFalse();
|
||||||
|
result.IsSystemFailure.ShouldBeTrue();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>User-side library failures and success adapt to <c>IsSystemFailure == false</c> — they must
|
||||||
|
/// NOT trip the outage circuit (03/S2).</summary>
|
||||||
|
[Theory]
|
||||||
|
[InlineData(LdapAuthFailure.BadCredentials)]
|
||||||
|
[InlineData(LdapAuthFailure.UserNotFound)]
|
||||||
|
public async Task User_side_failures_adapt_to_IsSystemFailure_false(LdapAuthFailure failure)
|
||||||
|
{
|
||||||
|
var inner = new RecordingLibService(LibLdapAuthResult.Fail(failure));
|
||||||
|
var sut = Build(
|
||||||
|
new LdapOptions { Enabled = true, DevStubMode = false, Transport = LdapTransport.Ldaps },
|
||||||
|
inner);
|
||||||
|
|
||||||
|
var result = await sut.AuthenticateAsync("alice", "wrong", CancellationToken.None);
|
||||||
|
|
||||||
|
result.Success.ShouldBeFalse();
|
||||||
|
result.IsSystemFailure.ShouldBeFalse();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>A successful bind is never a system failure.</summary>
|
||||||
|
[Fact]
|
||||||
|
public async Task Success_is_not_a_system_failure()
|
||||||
|
{
|
||||||
|
var inner = new RecordingLibService(LibLdapAuthResult.Success("alice", "Alice", new[] { "ReadOnly" }));
|
||||||
|
var sut = Build(
|
||||||
|
new LdapOptions { Enabled = true, DevStubMode = false, Transport = LdapTransport.Ldaps },
|
||||||
|
inner);
|
||||||
|
|
||||||
|
var result = await sut.AuthenticateAsync("alice", "secret", CancellationToken.None);
|
||||||
|
|
||||||
|
result.Success.ShouldBeTrue();
|
||||||
|
result.IsSystemFailure.ShouldBeFalse();
|
||||||
|
}
|
||||||
|
|
||||||
/// <summary>Records whether the library service was invoked and returns a canned result.</summary>
|
/// <summary>Records whether the library service was invoked and returns a canned result.</summary>
|
||||||
private sealed class RecordingLibService(LibLdapAuthResult result) : LibILdapAuthService
|
private sealed class RecordingLibService(LibLdapAuthResult result) : LibILdapAuthService
|
||||||
{
|
{
|
||||||
|
|||||||
Reference in New Issue
Block a user