Merge #844 #845 #838 credential lockout per identity, non-blocking condition events

This commit is contained in:
Joseph Doherty
2026-11-11 04:08:13 -05:00
15 changed files with 1418 additions and 209 deletions
+3 -2
View File
@@ -74,9 +74,10 @@ Until #678.6 the overlay never loaded on a node carrying a `cluster-*` role (the
| `EnabledSecurityProfiles` | list of `OpcUaSecurityProfile` | `[None, Basic256Sha256Sign, Basic256Sha256SignAndEncrypt]` | Transport-security profiles, one endpoint per entry. Must contain ≥1. Profile detail in [`security.md`](security.md). |
| `AutoAcceptUntrustedClientCertificates` | bool | `false` | Auto-trust unknown client certs on first connect (dev convenience). Not validated. See [`security.md`](security.md). |
| `PeerApplicationUris` | list of string | `[]` (empty) | Partner node `ApplicationUri`s published in `Server.ServerArray` for redundancy discovery. See [`Redundancy.md`](Redundancy.md). |
| `CredentialLockout:MaxFailedAttempts` | int | `5` | Failed UserName (API-key) activations from one client before its UserName activations are refused `BadUserAccessDenied` (#833). `≤ 0` disables. **Never refuses Anonymous.** See [`security.md`](security.md#failed-credential-lockout-opcuacredentiallockout). |
| `CredentialLockout:MaxFailedAttempts` | int | `5` | Failed UserName (API-key) activations for ONE credential identity (key id, else username) from one client before that client is refused that identity `BadUserAccessDenied` (#833, per-identity since #844 — a success clears only its own identity's count). Only definite bad credentials count; store timeouts / miss-budget refusals / authenticator faults do not (#845). `≤ 0` disables the whole lockout. **Never refuses Anonymous.** See [`security.md`](security.md#failed-credential-lockout-opcuacredentiallockout). |
| `CredentialLockout:MaxFailedAttemptsPerClient` | int | `0` (⇒ `2 × MaxFailedAttempts`) | Failed UserName activations from one client across EVERY identity before all its UserName activations are refused (#844). **A success never resets it**; it ages out through `FailureWindowSeconds`. Must be `0` or `>= MaxFailedAttempts` (validated). |
| `CredentialLockout:FailureWindowSeconds` | int | `60` | A failure more than this long after the client's previous one starts a fresh count. Must be `> 0` while the lockout is enabled (validated). |
| `CredentialLockout:SlotWaitSeconds` | int | `10` | How long a UserName activation waits for one of its client's verification slots (at most `MaxFailedAttempts − failures` verify at once; the rest queue) before that one attempt fails `BadTimeout`, uncounted. Must be `> 0` while enabled (validated). |
| `CredentialLockout:SlotWaitSeconds` | int | `10` | How long a UserName activation waits for one of its client's verification slots (at most `MaxFailedAttempts − failures` per identity, and `MaxFailedAttemptsPerClient − failures` per client, verify at once; the rest queue) before that one attempt fails `BadTimeout`, uncounted. Must be `> 0` while enabled (validated). |
| `CredentialLockout:LockoutSeconds` | int | `300` | How long a locked-out client's UserName activations are refused; an expired lockout starts a fresh count. Must be `> 0` while the lockout is enabled (validated). |
> **Transport security profiles** (the values in `EnabledSecurityProfiles` — `None`, `Basic256Sha256Sign`, `Basic256Sha256SignAndEncrypt`) and the PKI trust flow are documented in full in [`security.md`](security.md). This page does not duplicate them.
+99 -27
View File
@@ -331,37 +331,74 @@ The host now sets `MaxFailedAuthenticationAttempts = 0` on every configuration i
(`OpcUaApplicationHost.DisableSdkAuthenticationLockout`) and enforces a narrower replacement,
`FailedCredentialLockout`, in its identity-registry adapter:
- **UserName activations only.** A failed one counts; at `MaxFailedAttempts` (5) within
`FailureWindowSeconds` (60) of each other, that client's **UserName** activations are refused
`BadUserAccessDenied` for `LockoutSeconds` (300) without the key store being consulted, and a Warning is
logged once. A refusal during the lockout is not counted (it never extends itself); an **expired lockout
always starts a fresh count**, whatever `FailureWindowSeconds` is; a successful UserName activation clears the
count. **Anonymous activations never consult it**, from any client. The first refusal of each lockout episode
is logged at Information, the rest at Debug.
- **Parallel activations wait — they are never refused for parallelism.** A client gets at most
`MaxFailedAttempts − recordedFailures` verification **slots**; a UserName activation beyond that **queues** (FIFO)
until one frees, so no more than `MaxFailedAttempts` unverified guesses are ever in progress-or-failed per window,
while a legitimate consumer opening many keyed sessions at once (e.g. a ScadaBridge site node behind one NAT IP on a
`None` endpoint) just waits milliseconds and is admitted. A waiter is refused only when the key actually **locks
out** (`BadUserAccessDenied`, without reaching the authenticator), or after `SlotWaitSeconds` (default 10)
with `BadTimeout` — **not** counted as a credential failure. A cancelled activation leaves the queue and frees
any slot it held.
- **UserName activations only.** **Anonymous activations never consult it**, from any client.
- **Two counts per client (#844).** Each *definite* failure is charged to the (client, **credential identity**)
pair and to the client as a whole. The credential identity is `IOpcUaUserAuthenticator.DescribeCredential` —
for API keys `key:<keyId>` (the public lookup index, never the secret), for anything that is not a well-formed key
`user:<username>`.
- **Per credential:** at `MaxFailedAttempts` (5) failures of one identity, each within `FailureWindowSeconds` (60) of
the previous, that client is refused **that identity** `BadUserAccessDenied` for `LockoutSeconds` (300), without
the key store being consulted. A success with **that same identity** clears its count.
- **Per client:** at `MaxFailedAttemptsPerClient` failures across **every** identity (default `0` ⇒
`2 × MaxFailedAttempts` = 10), the client is refused **every** UserName activation for `LockoutSeconds`.
⚠️ **A success never resets this count** — it only ages out through `FailureWindowSeconds` or an expired lockout.
- Why both: before #844 one count per client was cleared by *any* success, so a client holding a valid key could make
four guesses at another key id, sign in with its own key, and repeat forever. The per-credential count alone would
fix that but would let a client spray four guesses at each of unboundedly many key ids; the client-wide cap that a
success cannot reset bounds the spray (≤ 10 unverified guesses per window chain, then 5 minutes locked), while a
legitimate client with one stale/revoked key and one good key loses only the stale one.
- Timing per count: a refusal during a lockout is not counted (it never extends itself); an **expired lockout always
starts a fresh count**, whatever `FailureWindowSeconds` is (an expired client-wide lockout clears every identity's
count too). A Warning is logged when a count locks (naming the scope and identity); the first refusal of each
lockout episode is logged at Information, the rest at Debug. Identities per client are bounded at 64; further ones
share one overflow count (conservative — never laxer).
- **Only a definite bad credential counts (#845).** A wrong secret, an unknown/disabled/expired key id, a malformed
credential or the wrong username counts. A denial that says nothing about the credential does **not**: a key-store
timeout (`Security:ApiKeys:StoreTimeoutMs`), a miss-budget refusal, any other store exception, an unset pepper, an
authenticator that throws, or a cancelled activation. The authenticator reports those with
`OpcUaUserAuthResult.Unavailable(...)` (`Transient = true`) and the host releases the slot uncounted — so a ConfigDb
stall can no longer lock every keyed client out for 5 minutes. The client sees the same opaque
`BadIdentityTokenRejected` either way (no oracle).
- **Parallel activations wait — they are never refused for parallelism.** An activation gets a verification
**slot** only while `identityFailures + identityInFlight < MaxFailedAttempts` **and**
`clientFailures + clientInFlight < MaxFailedAttemptsPerClient`; beyond that it **queues** until one frees (a queued
activation whose identity has room is not held behind one whose identity has none). So no more unverified guesses
than either threshold are ever in progress-or-failed per window, while a legitimate consumer opening many keyed
sessions at once (e.g. a ScadaBridge site node behind one NAT IP on a `None` endpoint) just waits milliseconds and is
admitted. A waiter is refused only when what it needs actually **locks out** (`BadUserAccessDenied`, without reaching
the authenticator), or after `SlotWaitSeconds` (default 10) with `BadTimeout` — **not** counted as a failure. A
cancelled activation leaves the queue and frees any slot it held.
- **Keyed on certificate thumbprint *and* peer address together.** That separates clients only when they present
a certificate. ⚠️ On a `None` endpoint clients commonly present none, so the key is **effectively the IP alone**:
NAT neighbours there **do** share one count (for UserName activations only — never for anonymous ones). When the
transport reports no peer address at all, every certificate-less client shares the single key `-@-`. (Observed on
docker-dev: every host client arrives as `-@192.168.97.1`.)
a certificate — on a `Sign`/`SignAndEncrypt` endpoint the certificate is proven, and two clients behind one address
with different certificates are locked out separately (pinned over the wire). ⚠️ On a `None` endpoint clients
commonly present none, so the key is **effectively the IP alone**: NAT neighbours there **do** share the client-wide
count (for UserName activations only — never for anonymous ones; the per-credential count is shared only by
neighbours presenting the same key id). When the transport reports no peer address at all, every certificate-less
client shares the single key `-@-`. (Observed on docker-dev: every host client arrives as `-@192.168.97.1`.)
- **The client `ApplicationUri` is deliberately NOT part of the key (#845, decided).** It would separate NAT
neighbours on a `None` endpoint, but there it is a free, unauthenticated string in `CreateSession` (and the
presented certificate is unproven), so an attacker would reset both counts simply by sending a new `ApplicationUri`
with every guess — turning the lockout off for exactly the client it exists to stop. Keying on it is safe only
where the certificate is proven, and there the thumbprint already separates clients. Operators who need NAT
neighbours isolated should enable a signed endpoint (each client then presents its own certificate), not widen the key.
- **Validated at host start:** with `MaxFailedAttempts > 0`, `LockoutSeconds`, `FailureWindowSeconds` and
`SlotWaitSeconds` must all be `> 0` (`OpcUaApplicationHostOptionsValidator`); `MaxFailedAttempts <= 0` disables
the lockout.
- Every denial counts, including a key-store timeout or miss-budget refusal (as the SDK's did) — a ConfigDb
outage can therefore lock a keyed client out for up to `LockoutSeconds` after recovery.
- Bounded at 4096 tracked clients (expired entries are pruned; a client arriving at a full table is not
tracked). API-key secrets are not guessable, so this is defence in depth, not the brute-force control.
`SlotWaitSeconds` must all be `> 0`, and `MaxFailedAttemptsPerClient` must be `0` or `>= MaxFailedAttempts`
(`OpcUaApplicationHostOptionsValidator`); `MaxFailedAttempts <= 0` disables the lockout.
- **Bounded at 4096 tracked clients.** Entries with nothing left to remember are pruned when the table fills; a client
arriving at a still-full table is not tracked. When a prune frees nothing, the next one is deferred until the earliest
entry could have aged out (#845) — a flood of new client keys at a full table used to rescan all 4096 entries under the
global lock on every failure. A release that frees an entry re-arms pruning immediately. API-key secrets are not
guessable, so this is defence in depth, not the brute-force control.
- **The SDK lockout is off on both configuration paths** — the configuration the host builds and one loaded from
`OpcUa:ApplicationConfigPath` (whose own `MaxFailedAuthenticationAttempts` is overwritten with `0`; pinned over the
wire with a file carrying `2`).
Pinned over the wire by `CredentialLockoutWireTests` (OpcUaServer.IntegrationTests — anonymous admitted after
six failures; a valid key refused while locked out; `MaxFailedAttempts = 0` admits after seven failures,
proving the SDK lockout is off) and unit-tested by `FailedCredentialLockoutTests`.
twelve failures; a valid key refused while client-wide locked out; own-key successes do not launder guesses at another
key id; transient denials never lock; distinct client certificates on a `Sign` endpoint are locked out separately; the
`ApplicationConfigPath` branch keeps the SDK lockout off; `MaxFailedAttempts = 0` admits after seven failures,
proving the SDK lockout is off) and unit-tested by `FailedCredentialLockoutTests`, `OpcUaApplicationHostImpersonationTests`
and `ApiKeyAuthenticationTests` (Host.Tests).
### LDAP bind flow (`OtOpcUaLdapAuthService`)
@@ -788,6 +825,41 @@ Behaviours worth knowing before enabling it:
enforcement only while the installed generation carries that reference — a hand-built authority (tests) must
supply `tagReferencingEquipment`. Inert with the flag off.
**Which permissions an event on a folder follows — checked against Part 3 (Gitea #838).** OPC UA Part 3's
`RolePermissions` model gives events their own permission, `ReceiveEvents` (PermissionType bit 11), and the pinned
SDK (2.0.1, `CustomNodeManager2.ValidateEventReceivePermissions`, documented there against Part 3 §8.55) applies it to
**both the event's EventType and its SourceNode** — not to the notifier the item happens to sit on. The notifier's own
permission only decides whether the item may be created/attached. So the spec's intent is *the folder AND the
source*, never "an event reported through a folder inherits the folder's permissions". This server keeps that shape:
the re-implemented `ValidateEventRolePermissions` runs the SDK's type + SourceNode check **first** and our decision
only on top (both must pass). For the event kinds that reach folder items:
- **Condition events** — SourceNode is the condition. Our decision is the condition rule above (the condition, or a
folder it legitimately notifies through for the session), which in this ACL model is the deliberate
generalisation of "the source": a `Tag` grant covers a native alarm's raw node and every UNS projection of it, and
an `Equipment` grant covers the alarms it owns or references.
- **A condition's Part 9 method audit events** (Acknowledge, Comment, Shelve…) — SourceNode is the condition, so they
are resolved to it and follow exactly the condition rule (plus the SDK's audit/SignAndEncrypt guard on the
Server-object path).
- **Any other event on a folder item** (none is produced today; `GeneralModelChangeEvent` and the rejected-write
`AuditWriteUpdateEvent` go to the Server object, #588) names no live condition and is **withheld** under
enforcement — fail closed, which is stricter than the folder's permission and never laxer than Part 3.
With the flag off only the SDK's own type + SourceNode `ReceiveEvents` check applies, as before.
**Event bursts never block under the node-manager lock (Gitea #838).** Each notifier with event items has an SDK
`MonitoredNode2` whose delivery task drains a **bounded** channel (4096, `FullMode = Wait`), and the synchronous
`NodeState.ReportEvent` **blocks** while that channel is full. Condition events are reported under the node
manager's `Lock`, and the delivery task needs that same `Lock` on every per-event verdict-cache miss — in the SDK's
own `GetNodeMetadata` (so **with the flag off too**, i.e. this predates #826) and in the #826 check. A burst of more
than 4096 events under one hold of `Lock`, starting at a miss (a new item, or right after a deploy invalidated the
verdicts), therefore deadlocked the node manager: `FolderEventFloodWireTests` stalled at exactly 4097 events, flag on
and off. Condition events now go through `ReportEventWithoutBlocking`: `ReportEventAsync` completes inline while the
channels have room (the normal case, unchanged); when one is full that report continues asynchronously, holding no
thread and no lock, and later reports queue behind it so events still leave **in report order**. The deferred work
holds only frozen snapshots — bounded by the burst — and drains once `Lock` is released; what a client receives is
still bounded by its item's queue (oldest discarded). Pinned by `FolderEventFloodWireTests` (6000 events under one
`Lock` hold with publishing disabled: completes, deferred reports drain to zero, the item delivers at most its
revised `QueueSize` in increasing order ending with the last event, and the next event is delivered).
**Model-change and audit events on the Server object are gated too** (Gitea #588). The node manager's
own server-wide events — `GeneralModelChangeEvent` (NodeAdded / NodeDeleted / DataTypeChanged) and the
`AuditWriteUpdateEvent` of a rejected device write — used to go straight to `Server.ReportEvent`, where