Merge fix/832-docker-dev-batch (#832 #833 #825 #823)

This commit is contained in:
Joseph Doherty
2026-11-11 02:44:01 -05:00
23 changed files with 1450 additions and 27 deletions
+4
View File
@@ -74,6 +74,10 @@ Until #678.6 the overlay never loaded on a node carrying a `cluster-*` role (the
| `EnabledSecurityProfiles` | list of `OpcUaSecurityProfile` | `[None, Basic256Sha256Sign, Basic256Sha256SignAndEncrypt]` | Transport-security profiles, one endpoint per entry. Must contain ≥1. Profile detail in [`security.md`](security.md). |
| `AutoAcceptUntrustedClientCertificates` | bool | `false` | Auto-trust unknown client certs on first connect (dev convenience). Not validated. See [`security.md`](security.md). |
| `PeerApplicationUris` | list of string | `[]` (empty) | Partner node `ApplicationUri`s published in `Server.ServerArray` for redundancy discovery. See [`Redundancy.md`](Redundancy.md). |
| `CredentialLockout:MaxFailedAttempts` | int | `5` | Failed UserName (API-key) activations from one client before its UserName activations are refused `BadUserAccessDenied` (#833). `≤ 0` disables. **Never refuses Anonymous.** See [`security.md`](security.md#failed-credential-lockout-opcuacredentiallockout). |
| `CredentialLockout:FailureWindowSeconds` | int | `60` | A failure more than this long after the client's previous one starts a fresh count. Must be `> 0` while the lockout is enabled (validated). |
| `CredentialLockout:SlotWaitSeconds` | int | `10` | How long a UserName activation waits for one of its client's verification slots (at most `MaxFailedAttempts − failures` verify at once; the rest queue) before that one attempt fails `BadTimeout`, uncounted. Must be `> 0` while enabled (validated). |
| `CredentialLockout:LockoutSeconds` | int | `300` | How long a locked-out client's UserName activations are refused; an expired lockout starts a fresh count. Must be `> 0` while the lockout is enabled (validated). |
> **Transport security profiles** (the values in `EnabledSecurityProfiles` — `None`, `Basic256Sha256Sign`, `Basic256Sha256SignAndEncrypt`) and the PKI trust flow are documented in full in [`security.md`](security.md). This page does not duplicate them.
+25 -2
View File
@@ -362,7 +362,7 @@ what the driver's cursor expects) is detected **two ways**, both triggering a
**Re-baselines without progress are bounded *and visible* (#804).** The reconnect
backoff already paced a stream that only ever re-baselines (`A_rebaseline_that_delivers_nothing…`),
but every `/current` also marked the stream delivered, so the driver read **Healthy**
for ever while no `/sample` chunk arrived — the live case is a `SampleCount` at or
for ever while no `/sample` chunk arrived — the live case is a `SampleCount`
above the Agent's `bufferSize`, refused `OUT_OF_RANGE` (`'count' must be less than N`)
on every open. After **3** consecutive re-baselines with no chunk in between
(`MaxRebaselinesWithoutProgress`, the Java driver's `MAX_REBASELINES_WITHOUT_PROGRESS`)
@@ -371,7 +371,13 @@ observations in between"` and one Warning naming the `bufferSize` cause; it stay
Degraded across further re-baselines until a chunk is delivered. Values keep flowing
off the re-baselines and `LastSuccessfulRead` keeps advancing — each `/current` is the
Agent answering — so the comms-loss watchdog does not reconnect-loop a configuration
problem. Size `SampleCount` below the Agent's `bufferSize` (in the `/probe` header).
problem. Size `SampleCount` at most the Agent's `bufferSize` (in the `/probe` header).
**It is caught before it happens (#825):** the `/probe` header's `bufferSize` is parsed
(`MTConnectProbeModel.BufferSize`, advisory — missing/malformed is null, never a parse failure); **Test
Connect** reads `sampleCount` (default 1000 when absent) and answers Ok **with a warning** (an amber
"OK with warning" chip showing the text) when it is **above** the buffer (live, cppagent 2.7: `count == bufferSize` is
accepted, `bufferSize + 1` refused `'count' must be less than {bufferSize+1}`), and the driver's **initialize**
(every deploy) logs the same Warning and puts `agentBufferSize` on the `/hosts` diagnostics bag.
**An Agent `instanceId` change means the Agent restarted**, and is checked
**before** the sequence-gap check (a restart also usually trips the gap, so
@@ -716,6 +722,23 @@ deliberately different, kept.
- **Deferred live checks:** these legs ran through the driver, **not** through a deployed OtOpcUa server and an
OPC UA client (the docker-dev rig was not used); the no-progress cap was observed via a `SampleCount` above the
Agent's `bufferSize`, not a genuinely lapped ring buffer; no real machine tool or non-cppagent Agent was exercised.
**Closed by the #823 gate below**, except the real-machine / non-cppagent caveat.
### End-to-end gate through a deployed server (#823, 2026-11-11, docker-dev)
An MTConnect driver (`SampleCount` 1000, `HeartbeatMs` 10000, `RequestTimeoutMs` 5000) against the shared fixture,
two raw tags (`fixture_x_pos` Double, `fixture_execution` String) referenced into a UNS equipment, deployed to the
central pair, and two Client.CLI subscriptions on central-1 (`ns=3` equipment folder, `ns=2` raw device). Raw and UNS
NodeIds moved together in every leg. Fixture containers were restored healthy afterwards.
| Leg | What happened on the wire |
|---|---|
| Adapter stop | Both NodeIds `BadNoCommunication` within 1 s of the adapter dying (Agent marks items UNAVAILABLE); driver stayed Healthy. Adapter start ⇒ Good 1–2 s later. |
| Agent restart (`docker restart`, ~12 s) | Stream lost and reconnected ~1 s; the stale cursor was refused `OUT_OF_RANGE`, the re-baseline saw the new `instanceId`, **rediscovery raised** — `/hosts` shows the **RE-BROWSE** chip. Values stayed Good (outage shorter than the comms-loss window). |
| Agent stopped ~2 min | Values held Good-but-stale; **Bad at 118 s** (comms-loss watchdog, connection refused). Agent start ⇒ re-init on the 10 s retry, Good ~8 s after the Agent was back. |
| Agent paused (`docker pause`) | Stream idle watchdog at 24 s (2 × 10000 + 5000); **Bad at 94 s** (watchdog); unpause ⇒ Good within ~3 s. |
| Genuine buffer overrun | A stalling TCP proxy froze the driver's transport while 32 000 observations were injected (`AllowPut`) — the Agent's `firstSequence` passed the driver's cursor (75 856 → 77 083). On release the resume was refused `OUT_OF_RANGE`, the driver re-baselined from `/current` within 1 s and resumed; values Good, no restart misdiagnosed. A raised-count stream (count 16, interval 5 s) was **not** lapped by a 266 obs/s burst: chunks are sent back-to-back while data is pending, so `interval` does not cap throughput. |
| `SampleCount` vs `bufferSize` (#825) | Test Connect rendered the amber "OK with warning" chip and a deploy logged the Warning (`/hosts` shows `agentBufferSize`). The same gate measured the Agent's real edge — `count = 16384` streams, `16385` is `OUT_OF_RANGE` — so the first build's "at or above" threshold was corrected to "above" (pinned live by `Test_Connect_warns_when_SampleCount_exceeds_the_live_agents_buffer_size`). |
## Known limitations
+50
View File
@@ -313,6 +313,56 @@ charged process-wide only — never handed a fresh budget. Clients behind one NA
guard applies to a driver-only node: its `ArtifactApiKeyStore` is two in-memory dictionary probes and is never
wrapped.
### Failed-credential lockout (`OpcUa:CredentialLockout`)
**A failed sign-in never locks a client out of anonymous access (#833).** OPC Foundation SDK 2.0.1's
`SessionManager` has its own lockout: after `ServerConfiguration.MaxFailedAuthenticationAttempts` (SDK
default **5**) failed identity validations — each within 60 s of the previous — it refuses **every**
`ActivateSession` from that client for **5 minutes** with `BadUserAccessDenied` ("Too many failed
authentication attempts"). Its check runs before the identity token is looked at, so it refused
**Anonymous** too (even a namespace-array read needs an activated session), a successful anonymous
activation did not clear it, and the authenticator was never called (no "anonymous admitted" line). Its key
is the client certificate thumbprint when one is presented, else the peer IP — so certificate-less clients
behind one NAT share a key, and on a `None` endpoint a presented certificate is not proven. That is what
the docker-dev live gate hit: four failed `multi-role` UserName activations against central-1, then every
anonymous session refused for five minutes while central-2 (which never saw the failures) kept admitting.
The host now sets `MaxFailedAuthenticationAttempts = 0` on every configuration it builds or loads
(`OpcUaApplicationHost.DisableSdkAuthenticationLockout`) and enforces a narrower replacement,
`FailedCredentialLockout`, in its identity-registry adapter:
- **UserName activations only.** A failed one counts; at `MaxFailedAttempts` (5) within
`FailureWindowSeconds` (60) of each other, that client's **UserName** activations are refused
`BadUserAccessDenied` for `LockoutSeconds` (300) without the key store being consulted, and a Warning is
logged once. A refusal during the lockout is not counted (it never extends itself); an **expired lockout
always starts a fresh count**, whatever `FailureWindowSeconds` is; a successful UserName activation clears the
count. **Anonymous activations never consult it**, from any client. The first refusal of each lockout episode
is logged at Information, the rest at Debug.
- **Parallel activations wait — they are never refused for parallelism.** A client gets at most
`MaxFailedAttempts − recordedFailures` verification **slots**; a UserName activation beyond that **queues** (FIFO)
until one frees, so no more than `MaxFailedAttempts` unverified guesses are ever in progress-or-failed per window,
while a legitimate consumer opening many keyed sessions at once (e.g. a ScadaBridge site node behind one NAT IP on a
`None` endpoint) just waits milliseconds and is admitted. A waiter is refused only when the key actually **locks
out** (`BadUserAccessDenied`, without reaching the authenticator), or after `SlotWaitSeconds` (default 10)
with `BadTimeout` — **not** counted as a credential failure. A cancelled activation leaves the queue and frees
any slot it held.
- **Keyed on certificate thumbprint *and* peer address together.** That separates clients only when they present
a certificate. ⚠️ On a `None` endpoint clients commonly present none, so the key is **effectively the IP alone**:
NAT neighbours there **do** share one count (for UserName activations only — never for anonymous ones). When the
transport reports no peer address at all, every certificate-less client shares the single key `-@-`. (Observed on
docker-dev: every host client arrives as `-@192.168.97.1`.)
- **Validated at host start:** with `MaxFailedAttempts > 0`, `LockoutSeconds`, `FailureWindowSeconds` and
`SlotWaitSeconds` must all be `> 0` (`OpcUaApplicationHostOptionsValidator`); `MaxFailedAttempts <= 0` disables
the lockout.
- Every denial counts, including a key-store timeout or miss-budget refusal (as the SDK's did) — a ConfigDb
outage can therefore lock a keyed client out for up to `LockoutSeconds` after recovery.
- Bounded at 4096 tracked clients (expired entries are pruned; a client arriving at a full table is not
tracked). API-key secrets are not guessable, so this is defence in depth, not the brute-force control.
Pinned over the wire by `CredentialLockoutWireTests` (OpcUaServer.IntegrationTests — anonymous admitted after
six failures; a valid key refused while locked out; `MaxFailedAttempts = 0` admits after seven failures,
proving the SDK lockout is off) and unit-tested by `FailedCredentialLockoutTests`.
### LDAP bind flow (`OtOpcUaLdapAuthService`)
LDAP is configured under the `Security:Ldap` section (bound to `LdapOptions`, `src/Server/ZB.MOM.WW.OtOpcUa.Security/Ldap/LdapOptions.cs`, `SectionName = "Security:Ldap"`). The app authenticator is `OtOpcUaLdapAuthService` — a thin wrapper around the shared `ZB.MOM.WW.Auth.Ldap` directory client that adds two app-only concerns the shared library deliberately does not model: the `Enabled` master switch and `DevStubMode`. Since #520 it serves the **Admin UI cookie login (`/auth/login`) only** — `LdapOpcUaUserAuthenticator` is deleted and the OPC UA UserName path authenticates API keys instead (see [API-key identity](#api-key-identity-opc-ua-data-plane)).