fix(docker-dev): wire OTOPCUA_ACL_ENABLED to the centrals; record the S18 follow-up
The R3-05 gate's own recipe correction, actioned. Security__Acl__Enabled:
"${OTOPCUA_ACL_ENABLED:-false}" was wired to site-a-1/site-a-2 only, so
`OTOPCUA_ACL_ENABLED=true docker compose up -d central-1 central-2` was a silent
no-op ("Running", no recreate, no env) — on the one node kind that carries
BoundedApiKeyStore at all, since only a fused admin+driver node wraps
ConfigDbApiKeyStore. The gate had to proceed via a scratch compose override.
The key is now on central-1 and central-2 too, same env var and same default
false, so today's rig behaviour is byte-identical and the plan's step 1 does what
it says. site-b stays hardcoded OFF as the A/B control.
Plan gains a "## S18 hard-deadline follow-up (2026-08-20)" section: the
cooperative-vs-hard root cause, the red-before-green observation (3/18 failing
with TimeoutException, and why the 15 existing deadline tests could not see it),
the fix, and the still-owed live re-gate table (expected vs. what the gate saw).
This commit is contained in:
@@ -910,3 +910,91 @@ Gate rows deleted (`NodeAcl` row, the `expiry-probe` key, nothing else had been
|
||||
recreated **without** the override (`Security__Acl__Enabled` absent from both env lists, i.e. default `false`),
|
||||
parity `b344565cad3d` ×6 re-asserted, readiness back, final deploy `1c666be3…` Sealed, anonymous read of `HR100`
|
||||
answers a value again. Rig left UP on the master image with `DisableLogin=true`.
|
||||
|
||||
## S18 hard-deadline follow-up (2026-08-20)
|
||||
|
||||
The gate's S18 finding is **fixed on `fix/r3-05-s18-hard-deadline`** (branched off master `9242630c`), plus the
|
||||
compose recipe defect the same leg exposed. Not yet live re-gated — the paused-SQL leg has to be re-run.
|
||||
|
||||
### What was wrong
|
||||
|
||||
`BoundedApiKeyStore.BoundAsync` bounded the inner store with a
|
||||
`CancellationTokenSource(TimeSpan, TimeProvider)` linked into the call. **That bounds the call only if the
|
||||
callee chooses to observe the token**, and `Microsoft.Data.SqlClient` does not while a connect or login is
|
||||
in flight on a black-holed socket. So the deadline never returned control: the gate's keyed activation ran
|
||||
**42.9 s** against a 2000 ms bound, the *"did not answer within 2000 ms"* Warning was emitted only at
|
||||
unpause, and because the whole thing sits inside the SDK's synchronous `SessionManager`-lock-holding
|
||||
impersonation callback, an unrelated already-open subscription stalled **57 s** behind it. The T3 TTL cache
|
||||
cannot help: a miss is exactly what the outage produces.
|
||||
|
||||
### Red before green — the observation, not just the claim
|
||||
|
||||
Every pre-existing deadline test drives `GatedApiKeyStore`, whose waits take the token
|
||||
(`Task.Delay(Timeout.Infinite, ct)`, `gate.WaitAsync(ct)`). A cooperative `CancelAfter` unblocks those, so
|
||||
**all 15 of them passed against the defective build** — the suite could not see the gate's failure at all.
|
||||
The new fake, `UncooperativeApiKeyStore`, ignores `ct` entirely and completes only when the test releases
|
||||
it, modelling SqlClient. Against the unmodified `BoundedApiKeyStore` the three new tests fail — with a
|
||||
test-side `WaitAsync(2 s)`, so the red is a **failure, not a hang**:
|
||||
|
||||
```
|
||||
Failed! - Failed: 3, Passed: 15, Skipped: 0, Total: 18
|
||||
An_uncancellable_store_is_abandoned_at_the_deadline [FAIL]
|
||||
An_abandoned_lookup_that_completes_later_is_not_cached [FAIL]
|
||||
An_abandoned_lookup_that_faults_later_is_observed_and_swallowed [FAIL]
|
||||
Shouldly.ShouldAssertException : … should throw System.OperationCanceledException but threw System.TimeoutException
|
||||
```
|
||||
|
||||
`TimeoutException` rather than `OperationCanceledException` **is** the finding, restated as an assertion:
|
||||
with a 200 ms bound the decorator had still not returned 2 s later.
|
||||
|
||||
### The fix
|
||||
|
||||
`BoundAsync` now races the inner call against `Task.Delay(timeout, _time, …)` — the injected clock, so the
|
||||
deadline stays on the same `TimeProvider` as the authorizer and the revalidation timer — with
|
||||
`Task.WhenAny`:
|
||||
|
||||
- **Inner wins** ⇒ stop the delay timer, dispose, `await inner` (faults and a genuine *caller* cancellation
|
||||
propagate exactly as before).
|
||||
- **Delay wins** ⇒ cancel the linked token anyway (a *cooperative* inner store should still stop), hand the
|
||||
abandoned task to `ObserveAbandoned`, meter `outcome=timeout`, log the (now honest) Warning, and throw
|
||||
`OperationCanceledException` — the same exception the authenticator's catch-all already maps to the
|
||||
standard opaque denial. **Fail-closed, unchanged: nothing here can admit.**
|
||||
- `ObserveAbandoned` attaches a continuation that disposes the CTS, **touches `finished.Exception`** (the
|
||||
observation itself, so a late SqlClient fault can never become an `UnobservedTaskException`), and logs at
|
||||
Debug. Debug rather than Warning because the timeout is already reported at Warning — a ConfigDb outage
|
||||
should not produce two lines per activation for one event.
|
||||
- **An abandoned answer is never cached.** It was taken before the deadline and would be served after it,
|
||||
so a key revoked or expired in the meantime could be admitted from a lookup nobody was still waiting for.
|
||||
`Store(...)` is simply unreachable on the timeout path; the test asserts it by call count (a cached late
|
||||
answer would make the next lookup a hit and leave `Calls` at 1).
|
||||
- `StoreTimeoutMs ≤ 0` still short-circuits to the unbounded call, byte-for-byte — the documented rollback.
|
||||
|
||||
Files: `src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/BoundedApiKeyStore.cs`,
|
||||
`tests/Server/ZB.MOM.WW.OtOpcUa.Host.Tests/OpcUa/BoundedApiKeyStoreTests.cs`, `docs/security.md`
|
||||
§"Bounding the key store". Host.Tests **137/137 green** (was 134; +3).
|
||||
|
||||
### Recipe defect fixed: `OTOPCUA_ACL_ENABLED` never reached the centrals
|
||||
|
||||
The gate's own correction note, actioned. `Security__Acl__Enabled: "${OTOPCUA_ACL_ENABLED:-false}"` was
|
||||
wired to **site-a-1 / site-a-2 only**, so
|
||||
`OTOPCUA_ACL_ENABLED=true docker compose up -d central-1 central-2` was a silent no-op — on the one node
|
||||
kind that carries `BoundedApiKeyStore` at all (only a fused admin+driver node wraps `ConfigDbApiKeyStore`).
|
||||
The key is now on **central-1 and central-2** too, same env var, **same default-`false`**, so today's rig
|
||||
behaviour is unchanged and the plan's step 1 does what it says. site-b stays hardcoded OFF as the A/B
|
||||
control.
|
||||
|
||||
### Live re-gate — still owed
|
||||
|
||||
Re-run **step 8's paused-SQL leg only** (setup steps 2-4 unchanged, and step 1 now needs no scratch
|
||||
override). Expected, versus what the gate saw:
|
||||
|
||||
| Observation | Gate (defective) | Expected now |
|
||||
|---|---|---|
|
||||
| Keyed `read` connect with SQL paused | 42.9 s, client `BadRequestTimeout` | denies in ≈ `StoreTimeoutMs` (2 s) |
|
||||
| *"did not answer within 2000 ms"* Warning | at unpause, ~40 s late | at the deadline, while SQL is still paused |
|
||||
| Already-open subscription on the same node | 57 s gap | no gap beyond the ~2 s of the activation |
|
||||
| `apikey_store_bounded{outcome="timeout"}` | 1, ~40 s late | 1, promptly |
|
||||
| Falsification control (pause with **no** activation) | streamed clean | must still stream clean |
|
||||
|
||||
⚠️ The Debug abandonment line is not visible at the rig's default log level — it is not part of the gate's
|
||||
pass criteria; the Warning and the restored subscription are.
|
||||
|
||||
Reference in New Issue
Block a user