Commit Graph

2043 Commits

Author SHA1 Message Date
Joseph Doherty 665ecd55c2 fix(focas): decode cnc_rdmacro dec_val at offset 6; vacant/fractional macros go Bad
The ODBM wire payload is [i32 mantissa][i16 length, a constant 10][i16 dec_val].
ReadMacroAsync read the decimal at offset 4, so every macro decoded without
failing but wrongly: the captured #500 = 1.0 (05 f5 e1 00 00 0a 00 08) published
0.01. FANUC normalises macros to a 9-significant-digit mantissa, so the error
was never small.

- dec_val is read at [6..7]; a payload < 8 bytes is refused, not guessed at.
- A vacant variable (dec_val 0xFFFF) publishes BadNoData, never a Good 0.
- Integer-typed tags get the exact whole value (integer arithmetic); a
  fractional macro on an integer tag is BadTypeMismatch, a whole value that
  does not fit the type BadOutOfRange. Float tags get mantissa / 10^dec.

Ported from the sister ignitionoee FOCAS driver v0.15.0 (Focas.Wire-013,
Focas-017/018); live vectors from Makino Z-34184 (31i-B, G431).
Adds a WireFixtures helper that loads the raw 31i-B captures in Fixtures/v3.
2026-10-07 13:57:11 -04:00
Joseph Doherty c657b2e03d fix(acl): make the API-key store deadline a HARD race (R3-05 S18 live-gate failure)
BoundedApiKeyStore bounded ConfigDbApiKeyStore with a
CancellationTokenSource(TimeSpan, TimeProvider) linked into the inner call. That
bounds the call only if the callee chooses to observe the token, and
Microsoft.Data.SqlClient does not while a connect/login is in flight on a
black-holed socket. The 2026-08-20 live gate paused the SQL container and watched
a keyed activation hang 42.9 s past its 2000 ms bound — the "did not answer
within 2000 ms" Warning fired only at unpause — and because this runs inside the
SDK's synchronous SessionManager-lock-holding impersonation callback, an
unrelated already-open subscription stalled 57 s behind it. That is precisely the
hazard the bound exists to prevent, and the TTL cache cannot help a miss.

BoundAsync now races the inner call against Task.Delay on the injected clock with
Task.WhenAny. Inner wins: stop the timer, await it, faults and genuine caller
cancellation propagate unchanged. Delay wins: cancel the linked token anyway (a
cooperative inner store should still stop), hand the abandoned task to
ObserveAbandoned, meter outcome=timeout, log the now-honest Warning, and throw
OperationCanceledException — the same exception the authenticator's catch-all
already maps to the standard opaque denial. Fail-closed is unchanged: nothing
here can admit.

ObserveAbandoned disposes the CTS and TOUCHES finished.Exception (the observation
itself), so a late SqlClient fault can never become an UnobservedTaskException.
Debug, not Warning — the timeout is already reported at Warning, and an outage
must not produce two lines per activation for one event.

An abandoned answer is NEVER cached: it was taken before the deadline and would
be served after it, so a key revoked or expired in the meantime could be admitted
from a lookup nobody was still waiting for. StoreTimeoutMs <= 0 still
short-circuits to the unbounded call byte-for-byte (the documented rollback).

RED BEFORE GREEN. Every pre-existing deadline test drives GatedApiKeyStore, whose
waits take the token, so a cooperative CancelAfter unblocks them and all 15
passed against the defective build — the suite could not see the gate's failure
at all. The new UncooperativeApiKeyStore ignores ct entirely and completes only
when the test releases it. Against the unmodified store the three new tests fail
3/18 with "should throw OperationCanceledException but threw TimeoutException"
under a test-side WaitAsync(2 s), so the red is a failure and not a hang — and
that exception swap IS the finding restated as an assertion.

Host.Tests 137/137.
2026-08-20 13:29:21 -04:00
Joseph Doherty e280db3753 fix(runtime): comms_latched_refs meter tag key 'driver' -> 'driver_instance' — the R3-04 bare-role-literal guard caught the R3-02 tag key at merge (cross-branch collision; neither branch saw the other) 2026-08-20 11:55:16 -04:00
Joseph Doherty 157d01c2c2 Merge branch 'fix/r3-02-stale-good-family' — R3-02: one quality contract at four seams — comms loss marks raw+UNS value variables BadNoCommunication (WriteStatuses sink member, recovery audit), Calculation + VT hold on Bad / worst-of sub-code / (value,status) dedup, MQTT initial push + ConnectionLost stale, Sparkplug birth replay (archreview 03/S15, 05/STAB-19, 05/STAB-18, 02/S14) 2026-08-20 11:49:06 -04:00
Joseph Doherty 886082fec7 fix(review): calc ReadOne reports an inputs-Bad publish; VT inputs-bad counter ticks only on a real report 2026-08-20 11:48:41 -04:00
Joseph Doherty 62c5bc3aad fix(runtime): a stopping driver child announces comms loss so restart/respawn latches its nodes Bad 2026-08-20 11:07:02 -04:00
Joseph Doherty f1b976f5e2 Merge branch 'fix/r3-04-one-role-authority' — R3-04: EffectiveRoles is the one role authority (Cluster:Roles wins over OTOPCUA_ROLES, normalised, validated, fail-start on unknown/overlay-changed roles); Program.cs + all four validators consume it; case-insensitive gossip predicates; source-scan guard (archreview 01/S-12 = 03/C5, 01/C-9) 2026-08-20 11:05:09 -04:00
Joseph Doherty 1d251cc037 Merge branch 'master' into fix/r3-02-stale-good-family
# Conflicts:
#	src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/DriverHostActor.cs
#	tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Harness/V3ArtifactSeeder.cs
2026-08-20 11:03:02 -04:00
Joseph Doherty af39e61638 fix(cluster): T7's post-overlay guard was structurally unreachable — compare live Cluster:Roles, not the PostConfigured options.Roles (review finding, R3-04)
options.Roles is unconditionally overwritten by AddOtOpcUaCluster's PostConfigure to equal
EffectiveRoles.Gossiped (T2's own structural-identity fix), so comparing options.Roles against
Gossiped could never observe a divergence — an overlay setting Cluster:Roles was silently
ignored, not fail-start. The validator now injects IConfiguration and re-reads the raw
Cluster:Roles section LIVE (reflecting any overlay applied after EffectiveRoles was first
resolved), normalized through EffectiveRoles.Resolve for reuse of its own validation.

AddOtOpcUaCluster now TryAddSingleton(configuration) so a bare ServiceCollection test harness
(not just WebApplicationBuilder) can supply IConfiguration to the validator.

Both guard tests rewritten to go through the real AddOtOpcUaCluster + PostConfigure pipeline
over a live ConfigurationManager, simulating an overlay as a second provider added AFTER
EffectiveRoles was resolved off the same object — exactly Program.cs's own sequencing. Confirmed
red beforehand: the "changes" test failed to throw against the pre-fix validator.
2026-08-20 11:01:33 -04:00
Joseph Doherty e3a72dc385 fix(runtime): virtual tags hold on Bad inputs and inherit the worst input status (archreview 02/S14) 2026-08-20 10:53:26 -04:00
Joseph Doherty 3648d3c129 fix(cluster): ClusterRoleInfo._roleLeaders stays Ordinal — Akka tracks role-leadership per exact gossiped role string (review finding, R3-04) 2026-08-20 10:52:51 -04:00
Joseph Doherty cc9b845f99 feat(engines): IVirtualTagEvaluator snapshot overload; Roslyn read cache stops fabricating Good (02/S14) 2026-08-20 10:45:11 -04:00
Joseph Doherty d79d51dd4f Merge branch 'fix/r3-01-mesh-grpc-allowanonymous' — R3-01: mesh gRPC + LocalDb sync endpoints opt out of the AdminUI cookie FallbackPolicy (PSK interceptors are the gate); real-auth-stack boundary tests + map-convention guard; rig DisableLogin parameterised (archreview 03/S14 Critical) 2026-08-20 10:43:49 -04:00
Joseph Doherty 60e5b40228 fix(calculation): read cache carries real input status + timestamps 2026-08-20 10:40:15 -04:00
Joseph Doherty 851e4cf5e0 Merge branch 'fix/r3-03-restart-resubscribe-watchdog' — R3-03: SpawnChild always sends the desired set (operator Restart + supervision restart re-provision, stale-alarm-set re-subscribe, incarnation check, 10 s backoff); watchdog grace = max(LastSuccessfulRead, connectedSince) (archreview 03/S16, S17, U6) 2026-08-20 10:40:11 -04:00
Joseph Doherty 0793b0cba8 fix(runtime): rate-limit the supervision reprovision — deferred, not dropped (review F3)
The default decider restarts a faulting child immediately and without limit, and
the hook makes every restart do ShutdownAsync+InitializeAsync+SubscribeAsync, so a
driver faulting on every value became a reconnect storm. Per-driver last-reprovision
stamp + DefaultReprovisionBackoff (10s, matched to DefaultReconnectInterval);
a held request is re-delivered by a single per-driver timer carrying the original
incarnation ref, so the F2 identity check still applies.

Seen RED with the backoff disabled: InitializeCount == 3 (second applied
immediately) where the test requires 2-then-3.

Also: the _children.Keys union-arm test (seeder gained taglessDrivers) and the
_desiredByDriver xmldoc note that it lingers on last-known-good after a failed apply.
2026-08-20 10:39:13 -04:00
Joseph Doherty 48efdac600 fix(calculation): hold evaluation on Bad inputs and propagate the worst input status; status-keyed dedup (05/STAB-19) 2026-08-20 10:37:29 -04:00
Joseph Doherty 3f8d5815b3 fix(runtime): mux adapter forwards the dependency's real status code (05/STAB-19) 2026-08-20 10:30:14 -04:00
Joseph Doherty 2b7100cf19 fix(runtime): route a reprovision by incarnation, not driver id (review F2)
A ReprovisionRequested in flight from an old incarnation could reach the NEW child
after HandleRestartDriver swapped it in. In Connecting that child DOES handle
InitializeRequested, so it started a second concurrent InitializeAsync, racing the
#527 dispose-before-overwrite.

Seen RED: InitializeCount == 2 on a child held in Connecting by a gated init
(an already-connected child swallows InitializeRequested and passes vacuously).
2026-08-20 10:27:02 -04:00
Joseph Doherty edd01fcc34 fix(runtime): a changed alarm set re-subscribes instead of latching (review F1)
The alarm subscribe gated on 'a handle exists', which is correct for an idempotent
re-push and wrong for a deploy that widens/narrows the set. Combined with the
spawn-time desired set, a respawning deploy could latch the PREVIOUS revision's
alarm refs for the life of the connection; sourceNodeIds-filtering drivers
(OpcUaClient/AbCip/Focas) would never deliver the new alarm.

Seen RED: LastSubscribedAlarmRefs stayed [alarmX] after a deploy widening to
[alarmX, alarmY].
2026-08-20 10:24:52 -04:00
Joseph Doherty e2497c908d test(cluster): guard that role reads live only in the authority (archreview theme 4) 2026-08-20 10:21:36 -04:00
Joseph Doherty 62515f3279 fix(security): pin the cached-key expiry guard, meter cache_hit after it, and close two wiring gaps (review fixes) 2026-08-20 10:18:52 -04:00
Joseph Doherty 7d98ef9332 feat(runtime): audit refs still BadNoCommunication after a reconnect (dedup-trap tripwire) 2026-08-20 10:16:01 -04:00
Joseph Doherty 4380e2e78a fix(cluster): fail start when a late config layer changes Cluster:Roles (role overlays may not set it) 2026-08-20 10:11:25 -04:00
Joseph Doherty 3314a1f98d feat(host): warn when Cluster:Roles and OTOPCUA_ROLES disagree; boot banner names the winning source 2026-08-20 10:08:03 -04:00
Joseph Doherty 97f4ea0c9e fix(runtime): comms loss marks raw + UNS value nodes BadNoCommunication, value retained (archreview 03/S15) 2026-08-20 10:07:12 -04:00
Joseph Doherty e46c1db5e5 fix(cluster): gossiped-role predicates compare case-insensitively — a miscased peer no longer un-elects the Primary 2026-08-20 10:04:16 -04:00
Joseph Doherty 04019e9414 feat(runtime): publish actor routes status-only batches to the sink 2026-08-20 10:03:25 -04:00
Joseph Doherty ec5b48bfcd feat(opcua): IOpcUaAddressSpaceSink.WriteStatuses — status-only batch write retaining the node value (03/S15) 2026-08-20 10:00:47 -04:00
Joseph Doherty 2b9a397b2d fix(runtime): re-provision a driver child after a supervision restart (archreview 03/S6, S16) 2026-08-20 09:56:05 -04:00
Joseph Doherty a4f12bccef fix(host): run monitored-item revalidation on a low-frequency timer (archreview 03/S19) 2026-08-20 09:55:01 -04:00
Joseph Doherty d97e441979 test(runtime): operator restart re-subscribes the respawned child (archreview 03/S16, U6)
Seen RED against a T3-reverted build: all 4 failed, Created[1].SubscribeCount == 0
on the headline case. 20x isolation run green after the fix was restored.
2026-08-20 09:53:38 -04:00
Joseph Doherty b7d20d14a8 fix(mqtt/sparkplug): initial push on subscribe + BadNoCommunication on connection loss 2026-08-20 09:52:15 -04:00
Joseph Doherty e3cf363703 fix(host): LocalDb sync endpoint opts out of the cookie fallback via an empty-prefix anon group
MapZbLocalDbSync returns IEndpointRouteBuilder, not an IEndpointConventionBuilder,
so .AllowAnonymous() cannot be chained onto it the way the other four mesh maps
are (T2). app.MapGroup(string.Empty).AllowAnonymous().MapZbLocalDbSync() carries
the same convention while leaving the gRPC route pattern byte-identical -
verified directly (LocalDbSync_AllowAnonymousGroup_LeavesTheRoutePatternByteIdentical).

The census's RealMethodEndpoints filter had to move off a "route pattern starts
with /" heuristic to an exact-route-pattern-set match: inside the empty-prefix
group, grpc-dotnet's own "Unimplemented service"/"Unimplemented method" fallback
endpoints also gain a leading "/", so the heuristic let them back in and
over-counted (caught by DriverOnly_WithASyncPort_AlsoMapsTheLocalDbSyncEndpoint
and LocalDbSync_AllowAnonymousGroup_LeavesTheRoutePatternByteIdentical going red
against the naive filter before this fix).

All five mesh gRPC endpoints now carry IAllowAnonymous (archreview 03/S14, T3).
2026-08-20 09:51:33 -04:00
Joseph Doherty bcec88ef70 fix(cluster): all four role-scoped validators read the authority — 'Driver' no longer skips every driver rule (archreview 01/C-9) 2026-08-20 09:51:23 -04:00
Joseph Doherty b09f78396f test(opcua): pin clock-driven expiry in the revalidation pass; correct the only-trigger remark 2026-08-20 09:50:39 -04:00
Joseph Doherty ac2f8dc801 fix(mqtt): stale observed refs BadNoCommunication on connection loss so the identical replay recovers (05/STAB-18 shape a) 2026-08-20 09:46:56 -04:00
Joseph Doherty bd80ff96ca feat(security): AclRevalidationTimer — clock-driven monitored-item revalidation (archreview 03/S19) 2026-08-20 09:46:56 -04:00
Joseph Doherty b05ebd6353 fix(host): mesh gRPC endpoints opt out of the AdminUI cookie FallbackPolicy (archreview 03/S14)
.AllowAnonymous() on DeploymentArtifactGrpcService, CentralCommandGrpcService,
TelemetryStreamGrpcService and NodeCommandGrpcService — the four MapGrpcService
maps that are chainable. On a login-enabled admin node the global cookie
FallbackPolicy (RequireAuthenticatedUser) applies to every endpoint without
explicit authorization metadata, gRPC included; the PSK bearer interceptors run
INSIDE the endpoint invocation, strictly after that middleware, so a
correctly-keyed node call was challenged before the interceptor's verdict could
matter. Safe because all four interceptors fail closed on an unset key.

The census now asserts IAllowAnonymous on all four (red before this change,
green after) plus a scope guard proving the opt-out never leaks onto an
unrelated endpoint. MapZbLocalDbSync (T3) is not chainable the same way and is
asserted absent here on purpose, so this test cannot start passing for the
wrong reason once T3 lands.
2026-08-20 09:46:30 -04:00
Joseph Doherty 792e8f422a fix(host): Program.cs derives roles from the one authority, not OTOPCUA_ROLES alone (archreview 01/S-12 = 03/C5) 2026-08-20 09:46:27 -04:00
Joseph Doherty 39a10a277a fix(security): wrap the ConfigDb key store on admin nodes + registration guard 2026-08-20 09:45:08 -04:00
Joseph Doherty 7dfd6dcc9f fix(cluster): AkkaClusterOptions.Roles is the EffectiveRoles instance, not a second derivation 2026-08-20 09:43:54 -04:00
Joseph Doherty 6048257dfe feat(observability): meter bounded API-key store lookups 2026-08-20 09:43:07 -04:00
Joseph Doherty 6dab72a479 refactor(host): extract MapOtOpcUaMeshGrpcServices (one authority for the five mesh maps)
Behaviour-preserving: Program.cs:743-784 collapses to a single call, same gates,
same order, every existing per-map rationale comment carried over. A new
endpoint-metadata census (MeshGrpcAnonymousMetadataTests) drives the SAME
production mapper across all three role shapes (admin-only / driver-only /
fused), so the guard can never assert a shape production no longer has -
archreview 03/S14, T1.
2026-08-20 09:42:49 -04:00
Joseph Doherty 7af9baab39 fix(mqtt): initial push of cached snapshots on (re)subscribe — the push-model forceRaise (05/STAB-18 shape b) 2026-08-20 09:42:25 -04:00
Joseph Doherty 7702616716 feat(cluster): EffectiveRoles — one role authority with one precedence + two-layer normalization (archreview 01/S-12, 01/C-9) 2026-08-20 09:41:54 -04:00
Joseph Doherty 37288d4df5 feat(security): short TTL read-through cache in front of the ConfigDb key store 2026-08-20 09:40:56 -04:00
Joseph Doherty 3b56e5bd01 fix(security): bound the API-key store call under the SessionManager lock (archreview 03/S18) 2026-08-20 09:38:41 -04:00
Joseph Doherty faea4ebfca feat(core): InputQuality.WorstOf — the one worst-of-input rule (archreview R3-02 contract) 2026-08-20 09:38:30 -04:00
Joseph Doherty daaeb854d6 fix(runtime): every spawn hands the child its desired subscriptions (archreview 03/S16) 2026-08-20 09:36:52 -04:00