Compare commits

...

27 Commits

Author SHA1 Message Date
Joseph Doherty d51d35bd64 Merge branch 'docs/xml-doc-cleanup'
v2-ci / unit-tests (push) Failing after 3m14s
2026-10-15 06:25:09 -04:00
Joseph Doherty 8c6067b62e docs: move subsystem gotchas out of CLAUDE.md into path-scoped .claude/rules
Keeps the root CLAUDE.md lean by lazy-loading per-area detail (drivers, topology,
data-plane auth, alarms, historian) only when matching files are touched.
2026-10-15 06:14:58 -04:00
Joseph Doherty 826312befe docs: fill missing XML docs and strip tracking IDs from src comments
Clears CommentChecker findings in src/ (3,280 -> 209, the remainder being false
positives like PLC-5/IEEE-754/M0.0) by adding missing summaries/params/returns,
normalising inheritdoc usage, and removing Gitea/review tracking IDs while keeping
the rationale they annotated. Comment-only: comment-stripped sources are identical
to the parent commit.
2026-10-15 06:14:58 -04:00
dohertj2 ebcbd81fca Merge pull request 'Ensure driver test cleanup preserves primary failures' (#730) from test/task-assertion-return-discard into master
v2-ci / unit-tests (push) Successful in 1h1m21s
2026-10-13 21:06:31 -04:00
Joseph Doherty 4f537a5652 Merge remote-tracking branch 'origin/master' into test/task-assertion-return-discard
v2-ci / unit-tests (pull_request) Successful in 1h1m9s
2026-10-13 19:03:52 -04:00
dohertj2 c89912a93b Merge pull request 'Fix DB-less ServiceLevel freshness and authorization remarks' (#724) from fix/dbless-servicelevel-freshness into master
v2-ci / unit-tests (push) Waiting to run
2026-10-13 19:03:42 -04:00
Joseph Doherty c4b4046832 test: verify missing-snapshot ServiceLevel timer behavior
v2-ci / unit-tests (pull_request) Successful in 1h0m42s
2026-10-13 17:19:50 -04:00
Joseph Doherty eed7cc7d75 test: guarantee driver fixture cleanup without masking failures 2026-10-13 16:36:38 -04:00
Joseph Doherty 5310a7e07a Discard asserted probe-task return values without joining loops 2026-10-13 16:12:31 -04:00
Joseph Doherty e1feb9fbb6 Merge remote-tracking branch 'origin/master' into fix/dbless-servicelevel-freshness
v2-ci / unit-tests (pull_request) Waiting to run
2026-10-13 15:58:35 -04:00
dohertj2 1bf8c614a9 Merge pull request 'Retain ServiceLevel requests across binding and publisher failures' (#720) from fix/servicelevel-binding-replay into master
v2-ci / unit-tests (push) Waiting to run
2026-10-13 15:58:19 -04:00
Joseph Doherty bc80b53e0e Merge ServiceLevel replay fixes for freshness integration 2026-10-13 15:11:20 -04:00
Joseph Doherty f478b844a1 docs: clarify anonymous session ACL enforcement 2026-10-13 14:29:00 -04:00
Joseph Doherty 1e71a25e3d fix: age DB-less ServiceLevel snapshots periodically 2026-10-13 14:29:00 -04:00
Joseph Doherty 8828e999c4 Merge remote-tracking branch 'origin/master' into fix/servicelevel-binding-replay
v2-ci / unit-tests (pull_request) Successful in 1h1m55s
2026-10-13 13:56:26 -04:00
dohertj2 4f7a07fbd3 Merge pull request 'Count individual skipped TRX results' (#714) from fix/trx-skip-reporting into master
v2-ci / unit-tests (push) Has been cancelled
2026-10-13 13:56:04 -04:00
Joseph Doherty da2e441bb9 Merge branch 'fix/trx-skip-reporting' of https://gitea.dohertylan.com/dohertj2/lmxopcua into fix/servicelevel-binding-replay
v2-ci / unit-tests (pull_request) Has been cancelled
2026-10-13 13:54:54 -04:00
Joseph Doherty c8e3e279b4 Retain and replay ServiceLevel across publisher binding and failures
v2-ci / unit-tests (pull_request) Waiting to run
2026-10-13 13:26:27 -04:00
Joseph Doherty 468f13cb56 Merge remote-tracking branch 'origin/master' into fix/trx-skip-reporting
v2-ci / unit-tests (pull_request) Waiting to run
2026-10-13 11:55:13 -04:00
Joseph Doherty f71fe13826 docs: record successful bounded CI baseline 2026-10-13 11:55:13 -04:00
dohertj2 6f0674296b Merge pull request 'Clarify S7 counter evidence and fixture limits' (#715) from docs/s7-counter-evidence into master
v2-ci / unit-tests (push) Has been cancelled
2026-10-13 11:54:57 -04:00
dohertj2 19bc13bf6a Merge pull request 'Bound CI concurrency and stabilize test fixtures' (#711) from fix/bootstrap-and-boundary-test-fixtures into master
v2-ci / unit-tests (push) Waiting to run
2026-10-13 11:54:15 -04:00
Joseph Doherty cd50db136e docs: clarify S7 counter evidence and fixture limits 2026-10-13 11:34:59 -04:00
Joseph Doherty eecf301200 ci: count individual skipped TRX results 2026-10-13 11:05:49 -04:00
Joseph Doherty 76bdca90e9 ci: bound build and test concurrency
v2-ci / unit-tests (pull_request) Successful in 1h1m1s
2026-10-13 10:52:12 -04:00
Joseph Doherty 8ff9f38bbd Make bootstrap cancellation test deterministic and deduplicate boundary cases 2026-10-13 09:56:37 -04:00
dohertj2 495606b9fb Merge pull request 'Clarify Galaxy native acknowledgement outcome documentation' (#708) from docs/galaxy-ack-rationale into master
v2-ci / build (push) Successful in 7m24s
v2-ci / unit-tests (push) Failing after 31m46s
2026-10-13 09:10:19 -04:00
471 changed files with 7494 additions and 5358 deletions
+56
View File
@@ -0,0 +1,56 @@
---
paths:
- "src/Core/ZB.MOM.WW.OtOpcUa.Core.ScriptedAlarms/**"
- "src/Server/ZB.MOM.WW.OtOpcUa.Runtime/ScriptedAlarms/**"
- "src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/**"
- "docs/ScriptedAlarms.md"
- "docs/AlarmTracking.md"
---
# Scripted alarm ack/shelve and Part 9 condition method behaviour
> Moved verbatim from the root `CLAUDE.md` (2026-10-13) so it loads only when working in this area.
> Root `CLAUDE.md` § "Subsystem guides" indexes it. Edit it here, not in the root file.
## Scripted Alarm Ack/Shelve
Inbound operator acknowledge/shelve for scripted alarms is fully implemented. Two surfaces converge on the `alarm-commands` DPS topic: (1) OPC UA Part 9 condition methods (Acknowledge / Confirm / AddComment / OneShotShelve / TimedShelve / Unshelve) wired in `OtOpcUaNodeManager`, gated on the `AlarmAck` role — conferred by the session's API-key principal's `NodeAcl` grants (or the `anonymous` principal's), carried by `RoleCarryingUserIdentity`; LDAP is AdminUI-only; (2) AdminUI `/alerts` per-row buttons routed through the `AdminOperationsActor` singleton (gated by `DriverOperator` policy). `ScriptedAlarmHostActor` dispatches from the topic to the engine. Client.CLI supports `ack`, `confirm`, `shelve` commands. See `docs/ScriptedAlarms.md` §"Inbound operator ack/shelve" and `docs/AlarmTracking.md`.
⚠️ **One condition event per accepted method call is OURS, not the SDK's (Gitea #587).** The SDK's own
report is gated on `AutoReportStateChanges` (left `false`), so before #587 a client Acknowledge emitted **no**
condition event — the old "SDK auto-fires its own event (E2)" comments were wrong. `WireMethodTransitionReporting`
wraps every method's SDK handler (Ack/Confirm/AddComment/Enable/Disable/OneShotShelve/TimedShelve/Unshelve) and,
on Good, reports once via `ReportConditionEvent` — **under `Lock`**, so the engine's/driver's re-projection
(routed by a fire-and-forget Tell *before* the SDK applies state) waits and then hits the delta-gate with no
delta. Things to keep: the SDK's shelve handlers never apply shelving, so the wrapper does; only a real
before/after change is reported (a re-ack is silent and restores the Confirmed/Retain the SDK flips);
`MaxTimeShelved` is set to `int.MaxValue` ms at materialise because `Create()` left it 0 and every TimedShelve
failed. ⚠️ **Native alarms support Acknowledge and AddComment only** — Confirm, the shelve methods and
Enable/Disable return `BadNotSupported` (no upstream path, no pair replication, and `NativeAlarmProjector`'s
Confirmed/Unshelved constants own those fields); a native ack re-asserts `ConfirmedState = true` because the
SDK's ack unconfirms any confirm-capable condition and ScadaBridge maps that field. Pinned over the wire by
`ConditionMethodEventTests` (exactly-once after a sentinel, folder + `i=2253`).
Behaviours that were wrong or missing and are now fixed — keep them: the SDK's `OnTimedUnshelve` is a
**no-op** (the engine's `AutoUnshelve` owns timed-shelve expiry, and `UnshelveTime` reports the real
remaining time from `AlarmConditionSnapshot.UnshelveAtUtc`); a **commented** re-ack is stored as an
`AddComment` (durable, replicated) and reports one event, an uncommented re-ack stays silent; a **native
ack is refused `BadInvalidState` off-Primary** via `NativeAlarmAckPrimaryGate` (fed by
`IDriverPrimaryRoleSource`; `DriverHostActor` publishes its Primary verdict at host start and on member
changes, so the boot window before the first redundancy snapshot is closed — the same fix covers the MQTT
Sparkplug primary-host claim). A native re-ack is still re-sent to the driver (the retry path, by design).
**Condition `Severity` is carried verbatim, clamped to 1..1000** — `ToPart9Severity`; the node manager used to re-bucket onto
the five `EventSeverity` points (so an MTConnect Fault read 900, scripted Critical 1000 read 900, native Low 200 read 300).
ScadaBridge only clamps, so exact values pass through. ⚠️ **Never select a two-state id with `AddSelectClause(type, "X/Id")`** —
the string becomes ONE QualifiedName named `X/Id`, which no event carries; the Client.CLI/Client.UI filter did this since
`a2883b82` and every alarm read inactive+unacked. Use `new SimpleAttributeOperand(type, [X, Id])` (ScadaBridge already does).
**Condition children are instance-scoped (#629).** Every child of a materialised condition carries
`{ConditionId}/{browse path}` in the condition's namespace (e.g. `…/temp_hi/ShelvingState/UnshelveTime`), assigned by
the `OtOpcUaNodeManager.New` override — the SDK default kept each child on the TYPE's ns=0 id, so a direct Read hit
the type (or `BadNodeIdUnknown`), translate failed, and children sat outside the ACL (the slash join now puts them
under their condition's scope). ⚠️ The shelve methods resolve by the `AlarmConditionType_ShelvingState_*` ids only
because `New` adopts those (a fixed set) as their `MethodDeclarationId`; calls by the `ShelvedStateMachineType_*` id
are no longer supported (pinned by a test). Condition teardown must forget the whole `PredefinedNodes` subtree
(`ForgetPredefinedSubtree`) or a removed condition's children keep answering.
+423
View File
@@ -0,0 +1,423 @@
---
paths:
- "src/Core/ZB.MOM.WW.OtOpcUa.Cluster/**"
- "src/Server/ZB.MOM.WW.OtOpcUa.ControlPlane/**"
- "src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Redundancy/**"
- "src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Telemetry/**"
- "src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Deployment/**"
- "src/Server/ZB.MOM.WW.OtOpcUa.Host/**"
- "docker-dev/**"
- "docs/Redundancy.md"
- "docs/Configuration.md"
- "docs/Telemetry.md"
---
# Cluster topology: redundancy, role authority, per-cluster meshes, MeshTransport, ConfigSource/ConfigServe, telemetry transport, LocalDb, DB-less driver nodes
> Moved verbatim from the root `CLAUDE.md` (2026-10-13) so it loads only when working in this area.
> Root `CLAUDE.md` § "Subsystem guides" indexes it. Edit it here, not in the root file.
## Redundancy
The server supports non-transparent warm/hot redundancy via the `Redundancy` section in `appsettings.json`. Two instances share the same Galaxy DB and the same mxaccessgw (under distinct `MxAccess.ClientName` values) but have unique `ApplicationUri` values. Each exposes `RedundancySupport`, `ServerUriArray`, and a dynamic `ServiceLevel` based on role and runtime health. The primary advertises a higher ServiceLevel than the secondary. See `docs/Redundancy.md` for the full guide.
**Two corrections landed 2026-07-21 — both were total-outage or wrong-node defects, and both are worth knowing before touching this area.**
- **Downing is `auto-down`, not `keep-oldest`.** `Cluster:SplitBrainResolverStrategy` (default `auto-down`) selects the provider via `ServiceCollectionExtensions.BuildDowningHocon`; an unknown value fails host start. A two-node pair running the previous `keep-oldest` + `down-if-alone` **could not survive a crash of the oldest node**: Akka's `KeepOldest.OldestDecision` only lets `down-if-alone` rescue a side holding ≥ 2 members, so the 1-vs-1 survivor downs *itself* and exits. `down-if-alone` is a 3+-node feature. The accepted trade for `auto-down` is dual-active during a genuine network partition. **Its live gate is CLOSED** — Phase 7 drill D4 (2026-07-24) ran the 1-vs-1 crash-the-oldest survival drill on the three-mesh docker-dev rig and every survivor stayed Up, never self-downing (`c50ebcf7`; `docs/plans/2026-07-24-mesh-phase7-failover-drills.md`).
- **The Primary is the oldest Up `driver` member, not the role leader.** `RedundancyStateActor` previously used `ClusterState.RoleLeader("driver")` (lowest *address*), while `ClusterSingletonManager` places singletons on the *oldest*. They agree only on a freshly-formed cluster and diverge after any restart, so the Primary-gated data plane (writes, alarm acks, alerts emit, alarm-history drain) could enable on a node not hosting the singletons. `NodeRedundancyState.IsRoleLeaderForDriver` was renamed **`IsDriverPrimary`**.
**A third bootstrap gap closed 2026-07-22 — downing was never the whole story, and the fix changed twice.** Even under `auto-down`, a node cold-starting while its peer was dead **never came Up**: Akka runs `FirstSeedNodeProcess` — the only bootstrap path that can form a *new* cluster when no peer answers `InitJoin` — exclusively when `seed-nodes[0]` is the node’s own address; every other node runs `JoinSeedNodeProcess` and retries `InitJoin` forever. **`Cluster:SeedNodes` is therefore ORDERED: every node that is one of its own seeds lists ITSELF first, partner second** (docker-dev `central-2` was swapped; site nodes list only `central-1` and are legitimately not seeds). `AkkaClusterOptionsValidator` (`AddValidatedOptions`/`ValidateOnStart` in `AddOtOpcUaCluster`) enforces it at boot — **conditionally** (self must be entry 0 only *if* self is in the list, or every site node would refuse to start), matching on `PublicHostname`+`Port`, never the `0.0.0.0` bind address. The earlier fix — a `Cluster:SelfFormAfter` timer calling `Cluster.Join(SelfAddress)` — is **deleted**: it sat outside Akka’s join handshake, so it could not tell "no seed answered" from "a seed answered and the join is in flight", and it islanded a manually-failed-over node live before a TCP reachability guard patched that one shape. Pinned by `SelfFirstSeedBootstrapTests` (real in-process clusters, incl. a falsifiability control proving peer-first ordering never forms) + `AkkaClusterOptionsValidatorTests`. See `docs/Redundancy.md` §"Bootstrap: self-first seed ordering".
**Simultaneous cold-start splits — the bootstrap guard closes it (opt-in, `Cluster:BootstrapGuard:Enabled`, default OFF).** Self-first on BOTH nodes means when both cold-start at the same instant, each runs `FirstSeedNodeProcess`, times out, and forms its OWN 1-node cluster → two Primaries in one pair (the Phase 6 live gate reproduced this on docker; two separate clusters do NOT auto-merge). The guard prevents it without losing cold-start-alone: the lexicographically **lower** `host:port` node is the preferred founder (self-first, forms immediately); the **higher** node TCP-probes the partner's Akka port up to `PartnerProbeSeconds` (25s) — **reachable ⇒ peer-first (join the founder)**, **unreachable ⇒ self-first (partner dead, form alone)**. The order is chosen BEFORE the single `JoinSeedNodes`, from an explicit reachability signal — it never re-forms mid-handshake (the `SelfFormAfter` failure mode). When on, Akka gets NO config seeds (`BuildClusterOptions` empties `ClusterOptions.SeedNodes`) and `ClusterBootstrapCoordinator` drives the join. Residual trade (accepted, operator-visible via a warning + restart-recovers): the higher node hangs if the founder dies in the probe→join window. docker-dev: **site-a = guard demo** (guard on, serialization removed); **site-b = A/B control** (guard off, `depends_on: service_healthy` serialization). The guard is the production-faithful fix (compose `depends_on` doesn't exist on real VMs). Pinned by `ClusterBootstrapGuardTests` + `ClusterBootstrapCoordinatorTests` (real 2-node clusters, incl. the higher-node-cold-start-alone case). See `docs/Redundancy.md` §"Bootstrap guard".
Assert the **effective** cluster config off a running `ActorSystem` (as `SplitBrainResolverActivationTests` does), never the typed options or the akka.conf text — several HOCON fragments compete and the losing one still reads correctly in the file.
**RESOLVED by per-cluster mesh Phase 6 (2026-07-24) — no longer per-Akka-cluster.** The "one Primary across the whole Akka mesh" limitation described above is gone; see the Phase 6 paragraph immediately below.
**Per-cluster mesh Phase 6 (2026-07-24) — the fleet is now three independent 2-node meshes, not one.** The single fleet-wide Akka mesh was split into one mesh per application `Cluster`: the central pair (`admin,driver,cluster-MAIN`), site-a (`driver,cluster-SITE-A`), site-b (`driver,cluster-SITE-B`) — same `ActorSystem` name, separated purely by per-pair self-first seeds (each node's `Cluster:SeedNodes` lists itself first, its own pair partner second, never a node from another cluster's pair). Consequences:
- **The redundancy singleton is cluster-scoped and spawned per driver node.** `RedundancyStateActor` is now a `cluster-{ClusterId}`-scoped singleton (falls back to plain `driver` for a legacy node), so each pair elects and publishes its **own** Primary on its own mesh's DPS. **Two or more Primaries exist fleet-wide, one per pair — by design, not a bug.** ⚠️ **Wiring landmine (fixed `3a4ed8dd`, caught by the live gate):** the singleton's cluster-role *scope* is computed at `AddAkka`-configurator time, so it must come from `AkkaClusterOptions` (config) — **never** by resolving `IClusterRoleInfo` from the SP there. `ClusterRoleInfo` depends on the `ActorSystem`, and that lambda runs *while the ActorSystem is being built*, so resolving it recurses into ActorSystem construction and **stack-overflows every node at boot** (compiles clean; the rehome tests passed a hand-built fake straight into the extension, mocking around the composition-root line). Keep `WithOtOpcUaClusterRedundancySingleton`/`BuildClusterRedundancySingletonOptions` taking `AkkaClusterOptions`.
- `ClusterNodeAddressReconciler` is **own-cluster-scoped** — an admin node can only reconcile `ClusterNode` rows in its own cluster; it has no gossip visibility into another mesh.
- `CentralCommunicationActor` dials each node directly over gRPC — one `NodeCommandService.Execute` call per enabled `ClusterNode` row carrying a `GrpcPort` — rather than fanning through a client/receptionist object. **Superseded by the [ClusterClient→gRPC migration](docs/plans/2026-08-06-clusterclient-to-grpc-design.md):** the Akka `ClusterClient` this bullet originally described (one per application `Cluster`, closing the `TODO(Phase 6)` Phase 2 left behind) is deleted; the per-node gRPC dial is its replacement, not an evolution of it.
- **`SplitTopologyTransportValidator` fails host start** on a node carrying a `cluster-{ClusterId}` role unless it also runs the split-safe transports: `MeshTransport:Mode=Grpc`, `Telemetry:Mode=Grpc` (driver), `TelemetryDial:Mode=Grpc` (admin). A DPS transport on a cluster-role node would try to gossip across a partition that no longer exists.
- **The compiled transport-mode defaults were deliberately NOT flipped** — `MeshTransport:Mode`/`Telemetry:Mode`/`TelemetryDial:Mode` all still default to `Dps`; a blast-radius assessment showed flipping the default breaks every integration test and appsettings profile that leaves the key unset. The validator + explicit per-deployment config is the guardrail, not a default flip.
- **Secrets replication is pair-local** — each pair replicates its own Akka DPS secrets within itself; there is no fleet-wide/cross-cluster sync after the split (sites have no SQL, so no SQL-hub, consistent with Phase 4).
See `docs/Redundancy.md` §"Per-cluster meshes (Phase 6)", `docs/Configuration.md` §"Per-cluster mesh split (Phase 6)", and `docs/plans/2026-07-24-mesh-phase6-partition-and-colocation.md`.
## Role authority (`EffectiveRoles`)
**A node's roles are resolved in exactly one place** (archreview R3-04, closing 01/S-12 = 03/C5 and
01/C-9): `EffectiveRoles.Resolve` (`ZB.MOM.WW.OtOpcUa.Cluster`), with one precedence —
`Cluster:Roles` when non-empty, else `OTOPCUA_ROLES`, else empty — and two-layer normalization: a
plain role (`admin`/`driver`/`dev`) is trimmed and lower-cased, but a cluster role's ClusterId suffix
(`cluster-SITE-A`) is kept verbatim, because lower-casing it would rename the Akka role of a running
deployment. An unknown role from **either** source fails host start. `Program.cs` resolves one
`EffectiveRoles` instance at the top of the composition root and registers it as a singleton; that
SAME instance drives both the conditional host wiring (`hasAdmin`/`hasDriver`) and
`AkkaClusterOptions.Roles` (what Akka gossips) — there is no second derivation for the two to
disagree about, which is why no runtime "do the views agree" assert exists. Before this, `Program.cs`
read `OTOPCUA_ROLES` only, so a node configured solely through `Cluster:Roles` gossiped the right
Akka roles while `Program.cs`'s own wiring mounted none of them — the inverse of the pre-#532 field
incident (`docs/deployments/wonder-app-vd03-makino-z-34184.md`).
- **A role overlay may not itself set `Cluster:Roles`.** Roles are resolved *before* the per-role
appsettings overlay loads (the overlay's filename depends on the answer), so a later layer changing
`Cluster:Roles` would leave the bound options disagreeing with the registered instance.
`AkkaClusterOptionsValidator` fails host start if this ever happens, naming both views — a no-op
today since none of the four shipped overlays carries a `Cluster` section.
- **Both sources set to different normalized role sets logs a Warning** at boot (`Cluster:Roles`
still wins outright) — the boot banner also names which source won (`source=Configured` /
`Environment` / `None`).
- **`RoleParser.GossipedRolesHave`** (case-insensitive) is defence in depth at the handful of
production sites that read LIVE GOSSIP directly (`Member.Roles`) rather than through
`EffectiveRoles` — a peer running an older, pre-R3-04 build can still gossip a miscased plain role
during a rolling upgrade. ⚠️ **Accepted residual (#620), not closed by R3-04:**
`RedundancyStateActor`'s election delegates to `ZB.MOM.WW.Health.Akka`'s `ClusterActiveNode.OldestUpMember`,
which matches via Akka's OWN ordinal `Member.HasRole`. Deliberately **not** patched — neither in the
library nor with a local case-insensitive pre-filter, which would split the Primary election from
`ClusterSingletonManager` placement (also ordinal). It only bites a mixed fixed/pre-fix rolling-upgrade
window; finish the upgrade. Full rationale: `docs/Redundancy.md` ("Accepted residual (#620)").
- **`Program.cs` wraps the composition root** (#618): a start-up failure logs Fatal, writes `FATAL:` to
stderr (the bootstrap logger is still silent before `AddZbSerilog`), flushes, and `Environment.Exit(1)`s
— leftover Akka/Kestrel threads would otherwise keep the process alive. `HostAbortedException`
(design-time tooling) passes through untouched.
- **`RoleAuthorityGuardTests`** (`tests/Core/ZB.MOM.WW.OtOpcUa.Cluster.Tests/`) is the enforcement:
a source scan failing on any production file outside `EffectiveRoles.cs`/`RoleParser.cs` that reads
`OTOPCUA_ROLES`, `Cluster:Roles` as code, or declares a bare `"admin"`/`"driver"`/`"dev"` literal —
with a small, reasoned allow-list for the handful of genuinely unrelated domain uses of those same
English words (a raw-tree node-kind label, a Sql driver's own JSON marker, an LDAP-derived AdminUI
RBAC role).
See `docs/Configuration.md` §`Roles`, `docs/ServiceHosting.md` §"Role gating", and
`archreview/plans/R3-04-one-role-authority-plan.md`.
## Mesh command transport (`MeshTransport`)
**The Akka `ClusterClient` transport described in this section's earlier history is DELETED**
(branch `feat/grpc-phase3-deletion`, [ClusterClient→gRPC migration](docs/plans/2026-08-06-clusterclient-to-grpc-design.md),
Phase 3) — code, the `akka.cluster.client` HOCON block, the receptionist extension,
`MeshTransport:CentralContactPoints`, and the `ClusterClient` mode value itself, which now **fails
validation loudly** at boot (`MeshTransportOptionsValidator` names the key and the allowed values)
rather than silently falling back to `Dps`. What remains is exactly two transports, selected by
`MeshTransport:Mode`:
- **`Dps`** (default) — DistributedPubSub, the path this repo has always used. **Pair-local only**:
it works only when central and the node share a gossip ring, which after Phase 6 means within one
2-node mesh.
- **`Grpc`** — unary gRPC calls on the `mesh_transport.v1` contract. **Required** on any node
carrying a `cluster-{ClusterId}` role; `SplitTopologyTransportValidator` refuses to boot such a
node on anything else.
Central→node commands (`deployments`, `driver-control`, `alarm-commands`) ride `NodeCommandService.Execute`,
served on each node's driver-role telemetry h2c listener (`Telemetry:GrpcListenPort`, the docker-dev
rig's `:4056` — it shares that listener with `TelemetryStreamGrpcService` rather than claiming a port
of its own). `CentralCommunicationActor` dials it **directly, one call per node** — the target set is
every enabled, non-maintenance `ClusterNode` row carrying a `GrpcPort`, dialled at `Host`+`GrpcPort`
and refreshed every `MeshTransport:ContactRefreshSeconds`. There is no client/receptionist object and
no rotation: each command goes to the specific node dialled, which is what makes per-node delivery
provable — the old `ClusterClient.SendToAll`/`Send` distinction this section used to explain no longer
applies to either direction.
Node→central deploy acks ride `CentralCommandService.ReportApply`, served on central's admin-role
config-serve h2c listener (`ConfigServe:GrpcListenPort`, the rig's `:4055` — again sharing rather than
claiming a new port). `GrpcApplyAckSender` walks `MeshTransport:CentralGrpcEndpoints` in order and
stops at the first success.
Auth is a **per-cluster PSK bearer**, both directions: `MeshTransport:GrpcPsk` (node-side, one key per
application `Cluster`) and `MeshTransport:ClusterPsks` (central-side map, consulted before the Secrets
store). `MeshCommandAuthInterceptor` (node) and `CentralCommandAuthInterceptor` (central) fail closed
on every unresolvable case — constant-time compare, no key material logged. Production keys are
`${secret:OT-MESH-PSK-<clusterId>}`; the docker-dev rig supplies three literal dev keys via env
(`dev-mesh-psk-MAIN`/`-SITE-A`/`-SITE-B`).
Things that carried over unchanged from the `ClusterClient` era, because they were never
`ClusterClient`-specific to begin with:
- **Fire-and-forget, no retry, on both transports.** A failed gRPC dispatch is logged (naming the node
and the gRPC status) and dropped in the dispatch second. No queue, no ordering guarantee across
nodes — the deploy coordinator's apply deadline and a node's boot-time replay are the only recovery.
The old `buffer-size = 0` decision this section used to describe is gone with the HOCON block that
set it; the gRPC transport has nothing to buffer in the first place — a dial either lands or doesn't.
- **Inbound commands re-emit on the node's `EventStream`, not back onto DPS.** DPS is mesh-wide, so a
DPS re-publish would deliver N copies to every subscriber. Node-side subscribers (`DriverHostActor`,
`ScriptedAlarmHostActor`) accept a command from either transport identically.
- **Do not over-claim what fire-and-forget buys.** The Phase 2 live gate observed a node applying a
deployment **44 s after it was sealed `TimedOut`** anyway, via `DriverHostActor.Bootstrap()`
replaying the orphan `Applying` row on restart. That backstop is unaffected by the ClusterClient
deletion — it was always about the deploy path, not the transport.
`redundancy-state` stays on DPS in both modes (bidirectional, and built from `Cluster.State`, which
makes it genuinely mesh-bound). See `docs/Configuration.md` §`MeshTransport`, `docs/Redundancy.md`
§"Command transport", and the migration's [design doc](docs/plans/2026-08-06-clusterclient-to-grpc-design.md) /
[plan](docs/plans/2026-08-06-clusterclient-to-grpc.md) / [tracking doc](docs/plans/2026-08-06-clusterclient-to-grpc-tracking.md) /
[live gate](docs/plans/2026-08-06-clusterclient-to-grpc-live-gate.md) for the phase-by-phase history —
those are historical records, not updated further by this sweep. `docs/plans/2026-07-22-mesh-phase2-clusterclient-transport.md`
is the original Phase 2 design doc and is likewise left as a historical record of the transport this
section now describes as deleted.
**Phase 1 landed 2026-07-22 and changed a deploy-path behaviour.** `ConfigPublishCoordinator` sources its expected-ack set from **enabled `ClusterNode` rows**, not from `Akka.Cluster.State.Members` filtered by the `driver` role — central must be able to name a deployment's nodes without sharing a gossip ring with them. Consequences worth knowing before touching deploys:
- **A configured node that is switched off now FAILS the deployment** at the apply deadline instead of letting it seal green without that node. The maintenance hatch is the **new `ClusterNode.MaintenanceMode` column**, NOT `Enabled = 0`: the live gate showed `Enabled = 0` is rejected outright by `DraftValidator.ValidateClusterTopology` (`ClusterEnabledNodeCountMismatch` — enabled count must equal `ServerCluster.NodeCount`), so on a 2-node pair — every cluster in the target topology — it can never be the hatch. `Enabled` = part of the declared topology; `MaintenanceMode` = do not expect it now. The deadline log names the silent nodes and points at the right flag.
- **Every `ClusterNode` row is now *defined* to be a driver node.** The DB has no per-node role column and deliberately did not gain one (it would drift from `Cluster:Roles`). Giving an admin-only node a `ClusterNode` row makes every deploy time out.
- **There is no such thing as a cluster-scoped deployment.** `Deployment` has no `ClusterId`, `ConfigComposer` always snapshots the whole DB, and `DeploymentArtifact.ResolveClusterScope` is *node-side* self-scoping of a fleet-wide artifact. The phase-1 plan asked for cluster-scope filtering of the ack set; it was dropped as describing a feature that does not exist.
- `ClusterNode` gained `AkkaPort` (NOT NULL, 4053) + `GrpcPort` (nullable) as **central's dial targets** for Phases 2/5. They duplicate the node's own `Cluster:Port` / `Cluster:PublicHostname` with no schema-level enforcement, so `ClusterNodeAddressReconcilerActor` (admin singleton) logs an Error on drift. **Phase 2 must revisit it** — once the meshes split, an admin node cannot see site members and every site row would report `EnabledRowNotInCluster` forever. **Resolved, and not the way this bullet originally expected:** the [ClusterClient→gRPC migration](docs/plans/2026-08-06-clusterclient-to-grpc-design.md) deleted the Akka `ClusterClient` transport that would have dialled `Host`+`AkkaPort`; the Phase 2 command dial now uses `Host`+`GrpcPort`, the same column Phase 5 telemetry already dialled — `AkkaPort` is Akka remoting only again.
## Config source (`ConfigSource` / `ConfigServe`)
Per-cluster mesh **Phase 3** (config fetch-and-cache) added a second way a **driver** node obtains its
deployed configuration, selected by `ConfigSource:Mode`: `Direct` (read `Deployment.ArtifactBlob` from
central SQL — **the default**, byte-for-byte today's behaviour) or `FetchAndCache` (fetch the artifact
bytes from central over a dedicated-h2c **gRPC stream**, verify `SHA-256 == RevisionHash`, cache in
LocalDb, and read **only** the cache — no central-SQL config read). Central (admin role) serves via
`DeploymentArtifactService` behind `ConfigServeAuthInterceptor` on `ConfigServe:GrpcListenPort`, gated by
a **shared node bearer key** (`ConfigServe:ApiKey == ConfigSource:ApiKey`, `FixedTimeEquals`,
fail-closed). Things worth knowing before touching it:
- **The serve side is wired in both modes** (mapped whenever `ConfigServe:GrpcListenPort > 0` on an
admin node), so flipping a node to `FetchAndCache` is a config change, not a redeploy — the same
dark-switch discipline as [MeshTransport](#mesh-command-transport-meshtransport).
- **`RevisionHash == SHA-256(artifact bytes)` is load-bearing.** `ConfigComposer` computes the revision
hash as the lowercase-hex SHA-256 of the exact blob, so the node verifies a fetch against the
`RevisionHash` the dispatch already carries — **no hash rides in the proto, and `DispatchDeployment`
stays payload-free.** A future change to how the revision hash is derived silently breaks every fetch.
**The blob is gzip-compressed since 2026-08-15, and the hash covers the COMPRESSED bytes** —
`ConfigComposer.Pack` gzips (`CompressionLevel.Fastest`) immediately after `SerializeToUtf8Bytes` and
hashes the result, so the invariant holds unchanged and nothing between the composer and
`DeploymentArtifact.Open` (SQL `ArtifactBlob`, the LocalDb cache chunks, the gRPC fetch stream) ever
looks inside. `Open` sniffs the gzip magic `1F 8B` — valid JSON can never start with `0x1F` — so
pre-upgrade uncompressed rows keep parsing; inflation runs under a 512 MiB guard and every
decompression failure surfaces as `JsonException`, the same family the malformed-JSON paths already
handle. Two operational consequences in `docs/Configuration.md` §`ConfigSource`: the first deploy after
upgrade **re-hashes every revision** (a one-time full re-dispatch, not a no-op), and **nodes must be
upgraded BEFORE central starts emitting compressed blobs.**
- **A null fetch is "no answer," never an empty config (#485).** All endpoints down / `NotFound` / SHA
mismatch / zero-length ⇒ the fetcher returns null ⇒ the apply FAILS, the revision does not advance,
and the node keeps serving last-known-good. It never tears down to an empty address space. At boot a
`FetchAndCache` node restores served state from the LocalDb pointer with **no** central-SQL read; an
empty/unreadable cache lands Steady-with-no-revision (the first dispatch fetches), **never Stale**.
**Malformed non-empty bytes are "no answer" too** — on cache boot, the `FetchAndCache` boot reconcile and
the subscription push alike (the lenient swallow leg is closed). A cache boot from malformed bytes does
**not** adopt the revision (`Direct` ⇒ Stale; `FetchAndCache` ⇒ no revision, the re-dispatch fetches).
- **Both pair nodes fetch independently** — no Primary gating of the fetch (same shape as both reading
SQL today; `StoreAsync` is idempotent and the cache replicates).
- **The dedicated h2c listener coexists with the LocalDb-sync listener.** A fused admin+driver node with
both ports set re-binds its existing HTTP surface exactly **once** and adds both dedicated HTTP/2-only
ports — getting this wrong either double-binds (crash) or silently unbinds the AdminUI (the Kestrel
`Listen*`-discards-`ASPNETCORE_URLS` trap the LocalDb block already warned about).
- **Phase boundary: config *reads* only.** A `FetchAndCache` node still writes its `NodeDeploymentState`
ack row to central SQL; `DbHealthProbe` and `EfAlarmConditionStateStore` still touch SQL. Cutting those
is Phase 4 — resist folding it in, or the "no SQL" scope becomes unprovable-in-parts.
On the docker-dev rig the central pair carries `ConfigServe` (`:4055`, committed dev key) and stays
hardcoded `Direct` via its admin role; the four site nodes carry `ConfigSource` (endpoints + matching
key) hardcoded `FetchAndCache` — mandatory for driver-only nodes since Phase 4, not a flip. There is
no `OTOPCUA_CONFIG_MODE` env var in the compose file. See
`docs/Configuration.md` §`ConfigSource` / `ConfigServe`, `docs/Redundancy.md` §"Command transport", and
`docs/plans/2026-07-22-mesh-phase3-config-fetch-and-cache.md`.
## Live telemetry transport (`Telemetry` / `TelemetryDial`)
Per-cluster mesh **Phase 5** (merged `35552a96`; live gate PASSED `f134935f`) added a second
transport for the four live-observability channels the AdminUI's `/alerts`, `/script-log`, and
`/hosts` panels feed on, selected by `Telemetry:Mode` (node/serve side) and `TelemetryDial:Mode`
(central/dial side): `Dps` — the four existing DPS SignalR bridges, **the default**, today's
behaviour — or `Grpc`, one gRPC server-streaming contract carrying all four channels as `oneof`
event kinds. **The direction is inverted from `ConfigSource`/`MeshTransport`: the driver node hosts
the gRPC server (dedicated Kestrel h2c port), and central is the client that dials in** — mirroring
ScadaBridge's `SiteStreamService` shape, and required so telemetry survives once the meshes split
(Phase 6) and central no longer shares a gossip ring with a site node. Things worth knowing before
touching it:
- **Migrated (4 channels):** `alerts` (`AlarmTransitionEvent`), `script-logs` (`ScriptLogEntry`),
`driver-health` (`DriverHealthChanged`), `driver-resilience-status`
(`DriverResilienceStatusChanged`) — feeding the exact same central sinks
(`IInProcessBroadcaster<...>`, `IDriverStatusSnapshotStore`, `IDriverResilienceStatusStore`) in
either mode.
- **Deferred, with reasons (do not read "seven observability topics" from the program sketch as the
delivered scope):** `redundancy-state` stays on DPS in every mode (pair-local, built from
`Cluster.State`, drives ServiceLevel + the Primary gate — genuinely mesh-bound by design, not an
observability panel); `deployment-acks` already rides the Phase 2 gRPC command transport
(`MeshTransport:Mode=Grpc`, `CentralCommandService.ReportApply`) as a command-plane reply, not
telemetry. A third, `fleet-status`, was **DELETED** rather than deferred (2026-08-06, Gitea #542):
`FleetStatusBroadcaster` → DPS → SignalR bridge → `FleetStatusHub` had **zero clients end to end**
(no browser JS, no `HubConnection` — `/fleet` polls `NodeDeploymentState` in the Config DB), its
`DriverHostStatusHeartbeat` input had no production sender, and Phase 6 left its "fleet" as just
central's own pair. Fleet visibility is `/fleet` + the four per-node telemetry dials above.
- **The node ALWAYS hosts and ALWAYS double-publishes, in both modes.** Every publish seam emits
into a node-local `ITelemetryLocalHub` **and** still publishes DPS, unconditionally; the gRPC
server binds whenever `Telemetry:GrpcListenPort > 0`, independent of `Telemetry:Mode`. Only
`TelemetryDial:Mode` on central actually switches anything: which of the two always-available
sources (DPS bridges vs. the `TelemetryDialSupervisor` dialer) feeds the sinks. This is what
makes the flip a config change, not a coordinated redeploy — same dark-switch discipline as
[MeshTransport](#mesh-command-transport-meshtransport) and [ConfigSource](#config-source-configsource--configserve).
- **Authenticated from day one, fail-closed** — a shared node bearer key (`Telemetry:ApiKey` ==
`TelemetryDial:ApiKey`), gated by `TelemetryStreamAuthInterceptor`
(`CryptographicOperations.FixedTimeEquals`, `PermissionDenied` on mismatch, an unset key rejects
every call). This **supersedes** design doc §6.3's "match ScadaBridge's unauthenticated posture
for now" — ScadaBridge itself has since closed that gap with this identical interceptor pattern.
- **Reconnect story:** central runs one reconnecting dialer per driver node (discovered from
`ClusterNode.Host`/`GrpcPort`, refreshed every `TelemetryDial:ContactRefreshSeconds`), immediate
first retry then ~5s fixed backoff, generation-stamped so a superseded stream's late error/event
is ignored, and it never permanently gives up (observability, not data plane). `TelemetryLocalHub.Evict` drops a removed driver's snapshots (`StopChild`, via `plan.ToStop`; central-side
stores are not evicted). The node-local hub
replays last-value snapshots for `driver-health`/`driver-resilience-status` on every (re)subscribe
so those stores re-prime immediately after a reconnect; `alerts`/`script-logs` are append logs and
tolerate the gap. The `/alerts`/`/script-log` pill reflects aggregate stream health (live when ≥1
node stream is up) — the same semantics the DPS pill has always had.
See `docs/Telemetry.md` for the full architecture, `docs/Configuration.md`
§`Telemetry`/`TelemetryDial` for the config keys, and
`docs/plans/2026-07-23-mesh-phase5-grpc-telemetry-stream.md` for the implementation plan and live
gate procedure.
## LocalDb pair-local store (Phases 1 + 2)
Every **driver-role** node keeps a consolidated `ZB.MOM.WW.LocalDb` SQLite database (the retired
LiteDB `LocalCache` subsystem was deleted as superseded). It holds **three replicated tables**:
`deployment_artifacts` + `deployment_pointer` (Phase 1) and `alarm_sf_events` (Phase 2).
**Phase 1** caches the
**deployed-configuration artifact** (chunked, SHA-256-verified, newest-2 retention per cluster) so a
driver node can **boot from cache when central SQL Server is unreachable** — `DriverHostActor` writes
the cache after each successful apply (`IDeploymentArtifactCache` → `LocalDbDeploymentArtifactCache`,
in `Runtime/Deployment/`) and reads it as a boot fallback, flagging running-from-cache. Wired by
`AddOtOpcUaLocalDb` in the `hasDriver` branch only (`Host/Configuration/LocalDbRegistration.cs` +
`LocalDbSetup.OnReady`, whose DDL→`RegisterReplicated` order is load-bearing); **admin-only nodes
never register it.** The cache can optionally replicate to the node's redundant pair peer over the
library's gRPC sync — **default-OFF, fail-closed bearer auth** (`LocalDbSyncAuthInterceptor`), gated
on `LocalDb:SyncListenPort` (a dedicated Kestrel h2c listener) + `LocalDb:Replication:*`. On the
docker-dev rig the **site-a pair** has replication on; central + site-b stay default-OFF (site-b is
the pin). ApiKey via `${secret:}`/env, never committed (the rig's dev key is the committed-dev-secret
exception).
**Phase 2** moved the **alarm store-and-forward buffer** off its standalone `alarm-historian.db`
and into the same database as `alarm_sf_events`, so a node that dies holding undelivered alarm
history no longer takes it to the grave. `SqliteStoreAndForwardSink` became
`LocalDbStoreAndForwardSink` (`Core.AlarmHistorian`), and the breaking config key
`AlarmHistorian:DatabasePath` was **removed** — it is still read, by `AlarmSfLegacyMigrator`, purely
to find a pre-consolidation file to copy across on first boot (renamed `.migrated` after the copy
commits). Because the buffer replicates, **only the node holding the Primary role drains it**: the
sink takes a `drainGate` delegate, Runtime supplies it from the `IRedundancyRoleView` singleton that
`DriverHostActor` publishes its `PrimaryGatePolicy` verdict to, and a gated-off node reports the new
`HistorianDrainState.NotPrimary`. Delivery is therefore **at-least-once across a failover, by
design** (no dedup layer). Row ids are a SHA-256 of the event payload rather than a GUID, so the
same event accepted on both nodes — which happens in every boot window, since
`HistorianAdapterActor` default-writes while its role is unknown — converges to one row. On the
docker-dev rig the site-a pair and site-b both run `AlarmHistorian__Enabled=true` against a
deliberately unresolvable gateway endpoint, so the buffer is always a live, non-draining queue.
**The library pin is `0.3.4`** (`Directory.Packages.props`, both `ZB.MOM.WW.LocalDb` and
`.Replication`), bumped from `0.2.1` on 2026-09-03 (was itself bumped from `0.1.3` on 2026-08-15).
The 0.2.1→0.3.4 bump is wire-compatible with 0.2.x, and the on-disk bookkeeping schema v2 → v3
upgrades in place on open, so no rig or volume action was needed. `RegisterReplicated(string)` is
unchanged and everything 0.2.x/0.3.x added is additive, **except** that `RegisterReplicated` (0.3.3)
now throws `LocalDbRegistrationException` for a table that is the parent of a row-mutating ON
DELETE FK — none of this repo's four replicated tables have such an FK, so this did not fire.
0.3.0 made the snapshot-resync flag **per-table** (see the ⚠️ below, now corrected); 0.3.2 added
validated config `Replication:KeepAlivePingDelay`/`KeepAlivePingTimeout` (defaults 60s/20s, left
unset here) and a public `LocalDbOplogGapException`, neither yet wired to a call site. **Its two
new APIs — `DeregisterReplicated` and `RegisterReplicated(table, baselineExistingRows:)` — are now
both used** (2026-08-18); the bullets below are the shape that landed, not the bump itself. Two
things are worth knowing:
- **`MaxBatchSize` no longer means what the rig's `16` was chosen to mean.** 0.2.0 made batching
**byte-budgeted** — `LocalDb:Replication:MaxBatchBytes` (2 MB default, left unset here, sized
under the 4 MB gRPC cap) bounds the wire message. `MaxBatchSize` is demoted to a secondary row
cap **and** the DB read-page size in `OplogStore.ReadBatchAboveAsync`/`SnapshotStreamer`, which
materialise a whole page in memory *before* the byte split runs. So the site-a pair keeps
`16` — it now bounds that transient allocation (≈2.7 MB at ≈171 KB artifact chunk rows), not the
message size. Every comment and doc row that called it "row-count-only against gRPC's 4 MB cap"
was corrected with the bump; do not restore that rationale.
- **Registration FOLLOWS the replication config — it is no longer unconditional (2026-08-18).**
`LocalDbSetup.OnReady` reads one predicate, `LocalDbReplicationPolicy.CapturesChanges`
(via `LocalDbSetup.ReplicationIsConfigured`):
**`LocalDb:Replication:PeerAddress` OR `LocalDb:Replication:ApiKey` is non-blank.** The **OR is
not belt-and-braces** — only the dialling half of a pair sets `PeerAddress`, so keying on it
alone would strip capture from every *passive* node (site-a-2 carries `ApiKey` +
`LocalDb:SyncListenPort` and nothing else) and the pair would converge in one direction only,
silently. ⚠️ **`SyncListenPort` deliberately does NOT join the OR:** the passive endpoint is
fail-closed on the key, so a port without an `ApiKey` rejects every inbound stream and has
nothing to capture for.
- ⚠️ **The alarm DRAIN gate uses a different predicate** — `LocalDbReplicationPolicy.SharesRowsWithPeer`:
`ApiKey` **and** (`PeerAddress` **or** `LocalDb:SyncListenPort > 0`), i.e. "a peer actually holds these
rows". Capture must be wider than the drain gate (a dialling-only node still captures); a node with no
peer must still drain, so the two predicates are deliberately not the same.
The **passive** half (no `PeerAddress`) cannot name its peer, so it stands down for its **mesh partner**
(the other member of its `cluster-{ClusterId}` role); a legacy passive half with no mesh role still
drains alongside its partner (duplicate historian rows). ⚠️ Before a node has joined its mesh the
published Primary verdict (and the live data-plane gate, which must stay the same decision) **allows** —
for seconds, up to ~25 s under the bootstrap guard — so a cold-starting node can briefly co-hold the
Sparkplug primary-host claim with its Primary peer.
- **Configured ⇒ `RegisterReplicated(table, baselineExistingRows: true)`**, per table, in the
same load-bearing DDL→register order. Baselining is what makes enabling replication on a node
that has been running *without* it actually converge: capture is CDC and the snapshot streamer
pages from `__localdb_row_version`, so pre-registration rows were in neither and a late opt-in
used to replicate only writes made after the restart. It seeds the ledger at the LWW floor
(HLC `0`, this node's id) and flags a resync — a baselined row loses to any genuine remote
write of the same key and wins only where the peer has no version at all. It is idempotent, so
it is free on every boot after the first. As of `0.3.0` (this repo pins `0.3.4`) the snapshot
flag is **per-table**, not per-database — baselining one table no longer streams every
registered table in both directions. (Prior to `0.3.0`, on the `0.2.1` pin this repo carried
until 2026-09-03, the flag was per-database and baselining anything streamed *every*
registered table in full; the old advice to trim `alarm_sf_events` before enabling replication
no longer applies at the current pin.)
- **Not configured ⇒ `DeregisterReplicated(table)` per table**, an idempotent boot-time
self-heal: it drops the capture triggers an earlier (unconditional) build left in the file and
prunes that table's oplog + row-version rows. Merely declining to register would leave every
pre-2026-08-18 file capturing forever — two INSERTs plus a `json_object` of the whole row
inside every write transaction — on precisely the node that has no peer to capture for. The
call returns `true` when it removed something, so the Information line fires **once, and only
when it actually cleaned**; a default-OFF node's ordinary boot stays silent. The logger is
Serilog's static `Log.Logger` (this runs inside the `AddZbLocalDb` singleton factory, where
resolving one from DI would nest a resolution inside a singleton construction; `Program.cs`
assigns it before anything resolves `ILocalDb`), with an optional parameter so tests observe
the line rather than infer it.
- ⚠️ **PAIR SYMMETRY: flipping registration is a BOTH-NODES-TOGETHER change.** The sync handshake
compares registered-table digests **fail-closed**, so a node that deregisters a table its peer
still replicates stops syncing with a schema mismatch. Enabling replication on an established
pair means setting the config and **restarting both nodes** — do it on both, or on neither.
- **Rollout note:** the only replicating pair in the fleet is docker-dev **site-a**, and it was
enabled from a fresh pair, so the impact of the old gap was low — central and site-b are
default-OFF and simply shed their unused triggers on their next boot. `LocalDbSetupTests`
pins both branches (registered-table set on a configured node, empty set + zero triggers +
the HLC-0 baseline ledger entry on the other) and the DDL→register ordering, unchanged.
See `docs/Configuration.md` (`LocalDb` section), `docs/Redundancy.md` (pair-local config
cache), `docs/AlarmHistorian.md`, and the runbook
`docs/operations/2026-07-20-localdb-pair-replication.md`.
## Phase 4 — cut the driver-side ConfigDb connection
Per-cluster mesh **Phase 4** finished the job Phase 3 started: a **driver-only** node (`Cluster:Roles`
has `driver`, not `admin`) now boots with **no `ConfigDb` connection string at all**. `Program.cs`
gates `AddOtOpcUaConfigDb` (and its health check) on `hasAdmin`; a fused `admin,driver` node is
unaffected (it keeps ConfigDb via its admin role). Consequences:
- **`ConfigSource:Mode` must be `FetchAndCache` on a driver-only node** — `ConfigSourceOptionsValidator`
fails host start if such a node is left on `Direct` (there is nothing to read Direct from). A fused
node may stay `Direct`.
- **ServiceLevel semantics change — client-visible.** With no ConfigDb, `DbHealthProbeActor` is not
spawned (`ServiceCollectionExtensions.WithOtOpcUaRuntimeActors` skips it when the resolved db
factory is null), and `OpcUaPublishActor` runs `dbLess`: `DbReachable` is fed constant `true` and
`Stale` comes from the redundancy-snapshot age alone. A healthy driver-only node therefore publishes
**240** (or **250** as driver Primary) even with central SQL unreachable — the survive-alone posture
— where a `Direct`/DB-backed node in the same state drops to 100/0. `ServiceLevelCalculator` itself
is unchanged; only the inputs a DB-less node feeds it changed. See `docs/Redundancy.md` §"DB-less
(driver-only) nodes".
- **Scripted-alarm Part 9 condition state moved to LocalDb.** `LocalDbAlarmConditionStateStore`
(replicated `alarm_condition_state` table, pair-local like the Phase-2 alarm S&F buffer) replaces
`EfAlarmConditionStateStore` on every driver-role node, fused central included. The DB-backed
`ScriptedAlarmState` table **was dropped** (Task 9, `3a590a0c`, migration
`20260723183050_DropScriptedAlarmStateTable`); `EfAlarmConditionStateStore` is deleted.
- **Central persists deploy acks.** `DriverHostActor` no longer writes `NodeDeploymentState` to SQL on
a driver-only node — `ConfigPublishCoordinator.PersistNodeAck` (already fed by every ApplyAck) is
the system of record.
- **LDAP group→role DB grants don't reach driver-only nodes.** They register the empty
`NullLdapGroupRoleMappingService` (no ConfigDb to read `LdapGroupRoleMapping` from), so
`OtOpcUaGroupRoleMapper` falls back to the `Security:Ldap:GroupToRole` appsettings baseline only —
the same rows a driver node never received via config either, so this is not a regression.
Merged to master (`26fad75c`); **live gate PASSED** (`1281aebf` — driver nodes run ConfigDb-free) and
Task 9 landed (`3a590a0c`). See `docs/plans/2026-07-23-mesh-phase4-cut-driver-configdb.md` and
`docs/plans/2026-07-22-per-cluster-mesh-program.md` §Phase 4.
+218
View File
@@ -0,0 +1,218 @@
---
paths:
- "src/Core/ZB.MOM.WW.OtOpcUa.Core/Authorization/**"
- "src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/**"
- "src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/**"
- "src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/**/*Acl*"
- "docs/security.md"
---
# Data-plane authentication (API keys), node ACL enforcement, anonymous principal, AdminUI LDAP
> Moved verbatim from the root `CLAUDE.md` (2026-10-13) so it loads only when working in this area.
> Root `CLAUDE.md` § "Subsystem guides" indexes it. Edit it here, not in the root file.
## Authentication: API keys on the data plane, LDAP on AdminUI (#520)
⚠️ **Changed 2026-07-31. The OPC UA data plane no longer uses LDAP at all.** It is machine-to-machine,
so a client presents an OPC UA **UserName** token with the fixed username **`apikey`** and an
`otua_<keyId>_<secret>` credential as the password. `ApiKeyOpcUaUserAuthenticator`
(`src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/`) implements `IOpcUaUserAuthenticator`: one indexed lookup
on `keyId` plus one peppered HMAC-SHA256, `FixedTimeEquals`, **no network I/O**. The key resolves to a
**principal**; `NodeAcl.Principal` rows grant it permissions. `LdapOpcUaUserAuthenticator` is
**deleted**, along with four `Security:Ldap` keys that existed only for it (`AuthTimeoutMs`,
`MaxConcurrentAuthentications`, `OutageFailureThreshold`, `OutageCooldownSeconds`) — they bounded an
LDAP bind running inline in the SDK's synchronous, `SessionManager`-lock-holding impersonation
callback, and nothing on the remaining LDAP path has that property.
**LDAP remains, for AdminUI login only** (`Security:Ldap`, `OtOpcUaLdapAuthService` →
`ILdapAuthService`, group→role mapping via `GroupToRole` + `LdapGroupRoleMapping`). Do not "restore"
the data-plane authenticator when you see LDAP wiring in `Program.cs` — it is there for the AdminUI.
Two things to know before relying on any of this:
- ⚠️ **Node ACLs are enforced only behind `Security:Acl:Enabled` (default `false`).** With the flag
**off**, a key's grants collapse into the two server-wide role gates (`WriteOperate` on Value writes,
`AlarmAck` on Part 9 methods), **only `Cluster`-scoped grants count** — `ApiKeyInterimRoleMapper`
deliberately under-grants, because honouring a narrower grant's flags while unable to honour its
*scope* would turn "WriteOperate on Equipment X" into server-wide write authority — and **read,
browse, subscribe and HistoryRead are ungated for any admitted session including Anonymous**. With the
flag **on** (`DataPlaneAuthorizer`), scoped grants decide **every** data-plane operation: write and
alarm commands and HistoryRead (stage 5), read and browse and subscription creation (stage 6).
- **The two role gates re-check the session's KEY on every call, flag off or on (Gitea #563).** Roles
are resolved once at activation, so an expired / deployment-disabled key used to keep `WriteOperate` /
`AlarmAck` on its open session until the client disconnected. `DataPlaneAuthorizer.AuthorizeKeyUsable`
runs the same `FindUsableKey` probe against the same snapshot + clock as the ACL path (one dictionary
lookup, no I/O) and denies `BadUserAccessDenied` before the role check. **Only those two gates** —
reads/subscriptions stay ungated with the flag off, by design. Anonymous carries no key and is
unaffected. No flag: a dead key keeping write authority is a defect, not a rollout choice. ⚠️ On a
fused node a key minted after the last deploy authenticates (ConfigDb) but cannot write/ack until a
deployment carries it — the flag-on path always behaved this way.
- ⚠️ **The ACL decision REPLACES the role check — the two are never chained.** Chained, a grant would
have to confer the server-wide role *as well as* pass the scope check, and only `Cluster`-scoped
grants confer a role, so every narrow grant would stay inert and the flag would appear to do
nothing. The load-bearing test is therefore an **allow** — a session carrying *no roles* writing
successfully on the strength of its grant. A deny-only test passes against a build where the ACL is
never consulted, because the role gate denies too.
- **The trie is built inside `DataPlaneAuthorityState.Apply`**, not by a second wiring at the publish
actor, and the state owns its `PermissionTrieCache` rather than being handed one. An uninstalled
trie denies everything, which is indistinguishable from working enforcement until someone checks
that an ALLOW is possible at all.
- **A deploy reaches sessions that are already open**, in both directions: each decision stamps the
*current* generation rather than the one the session activated under, and re-checks the
authenticating key — so revoking a key or a grant stops an open session immediately instead of at
the client's next reconnect. On a driver-only node the artifact is the only revocation channel.
- `UserAuthorizationState` is built **per decision walk, not per session** — and since the hot-path
change it is allocated only on a cache **miss**, so a cache hit never constructs one. Its LDAP-shaped
freshness/staleness predicates are switched off EXPLICITLY (`AuthCacheMaxStaleness = MaxValue`).
⚠️ Relying on "the stamp is always fresh" was not safe: `MembershipResolvedUtc` comes from
`DataPlaneAuthorizer`'s `TimeProvider` while `IsStale` is evaluated against
`TriePermissionEvaluator`'s, so **any divergence between the two clocks denies every request** with a
verdict indistinguishable from "no grant matched". Found by a test clock two hours ahead.
- **Stage 6's hot seams (read/browse/subscribe) go through a per-`(session, node, operation, realm)`
decision cache**, cleared wholesale when the generation advances. It is **sharded per API key**
(`ApiKeyId ?? Principal` — despite the type name, sessions sharing one key share one shard and one
budget), so a client crawling a large tree past the per-key ceiling evicts only that key's shard and
not every other key's decisions. ⚠️ It caches the **grant walk only** — key revocation and expiry are
re-checked **before** the cache lookup on every call, because expiry moves with the clock rather than
with a deployment.
- **Browse needs ancestor visibility, and it must NOT be a synthesized grant on the ancestors.** A
grant deep in the tree is reached by browsing the nodes above it, and the trie inherits downward
only, so `PermissionTrieNode.BrowseTransitPrincipals` marks the path. Planting a real `Browse` grant
on each ancestor instead would be collected by the additive downward walk and leak `Browse` onto
every sibling subtree beneath that ancestor. ⚠️ **The topmost ancestor is the single shared root
folder** (`s=OtOpcUa`, raw namespace) that BOTH realm trees hang under — its identifier is a display
name, not a path, so `DataPlaneAuthorizer.BuildScope` maps it onto the trie's **zero-segment
realm-root scope** (`V3NodeIds.SharedRootFolderId`, realm-qualified: in the UNS namespace that
identifier is an ordinary Area), and its Browse considers transit in **either** realm because it
fronts both trees. Before this every narrow Browse grant was unreachable by navigation — the Browse
gate denies the whole call when its source node is denied, so clients were stopped at the front door.
- **Revocation reaches live subscriptions** (stage 6): every other operation re-asks per request, but a
subscription is authorized once and then streams — and streaming is how machine consumers actually
read this server. `DataPlaneAuthorityState.Changed` fires `RevalidateMonitoredItems`. ⚠️ A denied
item is queued `BadUserAccessDenied` and then **detached from its `MonitoredNode2` fan-out — NOT set
`MonitoringMode.Disabled`**: disabling discards the item's queued notifications, so the client is
silenced without ever being told why (caught only by the over-the-wire test). Detaching is also
reversible, so restoring the grant resumes the feed with no client action. ⚠️ **`Changed` is no longer
the only trigger (arch-review 03/S19):** `AclRevalidationTimer`, armed by `OtOpcUaServerHostedService`
at `Security:Acl:RevalidationIntervalSeconds` (default 60 s, `0` disables, inert while
`Security:Acl:Enabled` is false), runs the same pass on the clock — because `ExpiresUtc` passes with
the clock, not with a deployment. The pass's verdict memo is **pass-local and must stay that way**;
hoisting it across ticks would serve an expiry that landed between two ticks the earlier tick's allow.
**Event/alarm items ride both triggers too (Gitea #562)** — a second map (`_eventItems`), fed by the
`SubscribeToEvents` override, holds every *direct* notifier subscription; a denied one is detached via
the SDK's own protected `SubscribeToEvents(…, unsubscribe: true)` (what a client delete does) and
re-attached with `unsubscribe: false` on restore — again never `Disabled`. Unlike a data item it is
**not told** (no value slot for a status; the feed goes quiet), and a client delete of a stood-down
item is absorbed by the override (the SDK would otherwise remove an item that is not attached).
**Server-object (`i=2253`) event subscriptions are filtered per event at delivery (Gitea #586).** The
`SubscribeToAllEvents` override attaches them to **nothing** (just records them in `_allEventsItems`);
each condition's own `OnReportEvent` (`OnConditionReportEvent`, fires once per event ⇒ still one
Server-object copy) queues the event to each recorded item whose session holds `Subscribe` on the
condition (RawPath) **or** any notifier folder it reports through — the per-folder path's question. The
root-notifier forward to the core Server object (`OnReportEvent` override) is **dropped**: it was the
ungated path, and it is the only one that served an item created before the alarm was deployed (the
SDK's `AddRootNotifier` subscribes existing all-events items with `unsubscribe: true`). Revalidation is
**implicit** — every event re-asks the authorizer, so a deploy or expiry lands on the next event with no
pass. `ConditionRefresh` on an `i=2253` item is filtered the same way. The dispatcher **must** keep
the SDK's audit guard (`AuditEventState` only while `Server.Auditing` over SignAndEncrypt/https),
because conditions report their Part 9 method audit events through themselves.
`ReconcileAllEventsMonitoring` must keep `AreEventsMonitored` raised on every condition while any
`i=2253` item exists, because `ConditionState` gates its method **audit** reports on
`EventsMonitored()` (method-driven state-change events no longer depend on it — #587, below). The
protected `SubscribeToEvents` override that no-ops all-events items is what
stops the SDK's `AddRootNotifier` (`unsubscribe: true` bug) taking that unit back. Both review
findings (review of `75e559f6`). ⚠️ The materialise path raises only the **new** condition's unit
(`RaiseAllEventsMonitoring`); a full reconcile there made a deploy of N alarms O(N²) under `Lock` with
ScadaBridge connected — keep the full reconcile to remove/rebuild/(un)subscribe.
**Model-change and audit events are gated too (Gitea #588).** The manager's own server-wide events
(`GeneralModelChangeEvent`, the rejected-write `AuditWriteUpdateEvent`) go through
`ReportToServerObject`: flag **off** ⇒ `Server.ReportEvent` as before; flag **on** ⇒ delivered per item to
`_allEventsItems` (same audience), a model change narrowed to the `Affected` nodes the session may
**Browse** (dropped when none — the builders' Messages name the node, so a narrowed copy gets a neutral
one), an audit write only with `Read` on its source node plus the SDK's audit/encryption guard. Any other
event type fails closed. Auditing is off by default (never configured in this repo).
- **ns=0 standard nodes stay ungated by design, even with `Security:Acl:Enabled`** (#617) — `ServerStatus`,
`NamespaceArray`, `ServiceLevel` etc. are served by the SDK's core node manager outside our authorizer and
clients read them to connect and to judge redundancy health; they carry no plant data. Every raw/UNS node,
event subscription and HistoryRead is still gated. `docs/security.md`.
- **Equipment-node writes: `EvaluateEquipmentWriteStructure` is the ONLY type check.** `OnWriteValue` bypasses
the SDK's own type check on every writable equipment node, so `OtOpcUaNodeManager.EvaluateEquipmentWriteStructure`
(plus `EvaluateEquipmentWriteRange`) is the whole gate. Scalars are deliberately lenient (drivers
range-check; ScadaBridge sends Double to a Float node). **Array writes** (#610): a numeric array of another
element type is **converted, checked, element by element** to the node's element type (ScadaBridge writes
CLR `int[]`/`double[]` lists) — any out-of-range, fractional-to-integer or NaN/∞ element refuses the whole
write `BadTypeMismatch` with nothing sent, so a `UInt32` > `Int32.MaxValue` can never reach a Galaxy Int32
attribute as reinterpreted bits; non-numeric mismatches are refused (`Byte[]` ≈ ByteString); wrong length ⇒
`BadTypeMismatch`; an array to a **scalar** node is refused only on Modbus/S7 (`EnsureVariable`'s
`refuseArrayOnScalar` — pass-through drivers keep their browse-committed scalar-shaped array tags writable;
a new sink wrapper must forward that argument, and the reflection guard checks only that the call arrives);
`IndexRange` writes ⇒ `BadWriteNotSupported`, `ValueRank > 1` out of scope. An
array node is writable only when the driver can write the array (`DriverArrayWriteSupport`: Modbus bounded
by `maxRegistersPerWrite`/coil cap, S7 non-Bool/non-wide types, OpcUaClient, Galaxy signed types **with a
declared `arrayLength`** — an undeclared length stays read-only because a short whole-array write would
resize the Galaxy attribute); `arrayLength >= 1` is required, and the node's `Writable && (!IsArray ||
ArrayWritable)` is projected onto the raw node and every UNS mirror.
- **Anonymous is a normal grantable principal**, not a bypass. `OpcUa:AllowAnonymous` (default `true`,
preserving today's behaviour) controls *admission*; when admitted the session authenticates as the
reserved principal `anonymous` and gets roles from `NodeAcl` rows naming it, arriving at the gates as
the same `RoleCarryingUserIdentity` a key produces. Set it `false` to make keys mandatory. A key may
**not** use the `anonymous` principal, and a failure to read anonymous's grants **denies** rather
than admitting with no roles.
- **Keys reach a driver-only node inside the deployment artifact** (stage 4). `ConfigComposer` snapshots
the `ApiKey` rows — the peppered HMAC only, never a secret, and the pepper is node config that is
deliberately *not* in the artifact — and `DeploymentArtifact.ParseDataPlaneAuthority` decodes them
node-side alongside the cluster's `NodeAcl` rows. `OpcUaPublishActor` installs both on
`IDataPlaneAuthoritySource` on every apply, **before** its `plan.IsEmpty` early-return, so a deployment
that only revokes a key still lands. A node with a ConfigDb keeps `ConfigDbApiKeyStore` (per-activation
read, wrapped — see below); a driver-only node gets `ArtifactApiKeyStore`. ⚠️ **The asymmetry is
ACTIVATION-scoped, not session-scoped — this line used to say "immediate on a ConfigDb node" and that was
wrong.** Clearing `Enabled` in the ConfigDb stops the **next session activation** on an admin/fused node
(bounded by `Security:ApiKeys:StoreCacheSeconds`, default 5 s) and takes until the next deployment on a
driver-only one — but an **already-open** session re-checks its key against the in-memory *artifact
snapshot* (`DataPlaneAuthorizer.Evaluate` → `snapshot.FindUsableKey`), never against `IApiKeyStore`, so on
**both** node kinds an `Enabled` revocation reaches an open session only at the next deployment.
**Expiry is the exception**: `FindUsableKey` compares `ExpiresUtc` against the authorizer's clock on every
call, so it is immediate everywhere — and since arch-review 03/S19 it reaches live subscriptions too, on
the host's `Security:Acl:RevalidationIntervalSeconds` timer (default 60 s; the session is **silenced with
`BadUserAccessDenied`, not closed**, and recovers on re-activation). Direct event/alarm subscriptions
are revalidated on both triggers as well (#562); with the flag **off**, expiry and revocation reach the
two role gates (#563) but not reads or subscriptions.
- ⚠️ **The activation lookup is BOUNDED (arch-review 03/S18).** The SDK raises impersonation synchronously
under a `SessionManager`-wide lock, so a ConfigDb that is *slow* rather than *refusing* stalled **every**
session activation on the server, Anonymous and X.509 included. `BoundedApiKeyStore` wraps the ConfigDb
store — and only it, never `ArtifactApiKeyStore`, which is two in-memory dictionary probes — with
`Security:ApiKeys:StoreTimeoutMs` (2000; exceeding it **denies**, via the authenticator's existing
catch-all) plus a `StoreCacheSeconds` (5) read-through cache. Faults and timeouts are never cached, a
cached hit whose `ExpiresUtc` has passed is re-read rather than served, and a timeout logs a distinct
Warning + `otopcua.opcua.apikey_store.bounded{outcome=timeout}`. Both knobs at `0` restore the old
behaviour exactly.
- ⚠️ **The auth generation is a node-local COUNTER driven by the `RevisionHash`, not derived from its
bits.** `PermissionTrieCache` treats the highest `GenerationId` as current, so a generation taken from
hash bytes would install a *lower* number about half the time and the cache would go on serving the
previous deployment's ACLs with nothing logged. Pinned by
`DataPlaneAuthorityStateTests.The_generation_only_ever_increases_even_when_the_hash_decreases`.
- Note `OtOpcUaServerHostedService` takes `IOpcUaUserAuthenticator` **and** `IDataPlaneAuthoritySource`
as **required** ctor parameters — the built-in DI container does not fall back to a default value for
an unregistered service, so both must be registered explicitly on every node (deny-all / empty on a
node with neither role) or it does not boot.
**Grants are authored through a scope picker, not free text (stage 7).** `/clusters/{id}/acls` opens
`AclScopePickerModal` over the cluster's raw + UNS trees, and selecting a node supplies `ScopeKind`
**and** `ScopeId` together — so `RawDevice` + a TagId, a cross-cluster scope, a `Cluster` grant carrying
an id, and the retired `FolderSegment` kind are all unauthorable by construction rather than merely
rejected later. ⚠️ **A `Cluster` row must produce a NULL `ScopeId`** — the deploy gate rejects one that
carries an id (`NodeAclScopeIdUnexpected`), so a picker that helpfully filled in the cluster id would
author rows that cannot be deployed. `IAclScopeService.CheckScopeAsync` mirrors
`DraftValidator.ValidateNodeAclScopes` rule for rule and **blocks the save** when a scope resolves to
nothing (re-checked on submit, since a row loaded unchanged is exactly where the operator never touches
the picker); the list page marks such rows `unresolved`. **UNS tags are deliberately not offered** — a
UNS signal is a `UnsTagReference`, and a `Tag` grant already covers the raw node *and* every UNS
projection of it. The picker is `AclScopeTree`, a separate component from `RawTree`, which already
carries two modes.
**All eight stages are done** (`d4389f57` enforcement, stage 7 authoring after it), each live-gated on
docker-dev. The rationale for every decision above is in
`docs/plans/2026-07-31-apikey-acl-enforcement-design.md`. See `docs/security.md` for the full guide.
Dev/test LDAP is the **shared GLAuth** running on the Linux Docker host at `10.100.0.35:3893` (baseDN `dc=zb,dc=local`, plaintext/`Transport=None`). It is managed via `scadaproj/infra/glauth/` (source of truth + deploy runbook). Single bind account `cn=serviceaccount,dc=zb,dc=local` / `serviceaccount123`; all test users password `password`. The docker-dev compose binds this shared instance directly — `DevStubMode` is no longer used. The per-VM NSSM GLAuth at `C:\publish\glauth\` and the old base DNs `dc=lmxopcua,dc=local` / `dc=otopcua,dc=local` are obsolete. (The integration-test harness under `tests/.../Host.IntegrationTests/` runs its own ephemeral **GLAuth** container on port 3894 — seeded by `tests/.../Host.IntegrationTests/glauth/config.toml` with dev users alice/bob + the `cn=serviceaccount` bind account — for the opt-in `OTOPCUA_HARNESS_USE_LDAP=1` real-bind mode; distinct from the shared dev GLAuth on 3893. It replaced the retired `bitnami/openldap:2.6` in PR #451, 2026-07-15.)
+307
View File
@@ -0,0 +1,307 @@
---
paths:
- "src/Drivers/**"
- "src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/**"
- "src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/**"
- "tests/Drivers/**"
- "docs/DriverLifecycle.md"
- "docs/drivers/**"
---
# Drivers: rediscovery signal, comms-loss watchdog + quality contract, AB native transport, per-driver notes
> Moved verbatim from the root `CLAUDE.md` (2026-10-13) so it loads only when working in this area.
> Root `CLAUDE.md` § "Subsystem guides" indexes it. Edit it here, not in the root file.
### Change Detection
`DeployWatcher` (`src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Galaxy/Browse/DeployWatcher.cs`) polls the gateway's deploy-event signal and raises `IRediscoverable.OnRediscoveryNeeded` when the Galaxy redeploys.
**The signal is consumed as an operator prompt, not as an address-space rebuild** (§8.2, 2026-07-27).
`DriverInstanceActor` subscribes in `PreStart` and unsubscribes in `PostStop` — the detach is
load-bearing, because the `IDriver` instance outlives the actor when the host respawns a child around
the same driver object. A raise is recorded and rides the driver-health snapshot
(`DriverHealthChanged.RediscoveryNeededUtc` / `.RediscoveryReason`) to the AdminUI `/hosts` page as a
**"re-browse" chip**.
⚠️ **It is advisory: a Galaxy redeploy still does NOT change the served address space.** v3 authors raw
tags explicitly through the `/raw` browse-commit flow, so an operator must re-browse the device and
commit. Injecting nodes at runtime would materialise nodes nobody approved and no deployment artifact
records.
Two gotchas if you touch this:
- `PublishHealthSnapshot` dedups on a health fingerprint. The rediscovery timestamp **must** stay in
that tuple, or a raise on an otherwise-unchanged Healthy driver is swallowed and never reaches the
operator. Pinned by `DriverInstanceActorRediscoverySignalTests`.
- The shared test `StubDriver.GetHealth()` returns `DateTime.UtcNow`, so its fingerprint differs on
every call and the dedup never engages — a test built on it passes vacuously. The rediscovery tests
use a stub with stable health for exactly this reason.
**#507 is closed as superseded, and the injection path is deleted.** Its retained code read
`EquipmentNode.DriverInstanceId` and `EquipmentTags`, both *structurally empty* in v3
(`AddressSpaceComposer.cs`, `DeploymentArtifact.cs`), so removing the guard would have changed a log
line and injected nothing. Gone with it: `HandleDiscoveredNodes`, `PartitionDiscoveredByDeviceHost`,
`ApplyDiscoveredPlansForDriver`, the redeploy re-inject tail, `DiscoveredNodeMapper`,
`AddressSpaceApplier.MaterialiseDiscoveredNodes`, the connect-time discovery loop
(`StartDiscovery`/`RediscoverTick`/`DiscoveredNodesReady`/`TriggerRediscovery`), and the 18
`DiscoveryInjectionDormantV3` tests — v2 characterization tests asserting an equipment-rooted graft
(`EquipmentRootNodeId == "EQ-1"`) that v3 cannot produce. `ITagDiscovery` itself stays: the `/raw`
browse picker drives it through `Commons/Browsing/DiscoveryDriverBrowser`, which never went through
the actor.
**`IRediscoverable` is the ONLY tag-change signal, and four drivers do not raise it.** A periodic
tag-set drift check briefly existed (shipped `e77c8a35`, **removed** by Gitea **#523**): every
`SupportsOnlineDiscovery` driver was re-browsed on a 5-minute timer and the result diffed against its
authored tags. It was taken back out because it generated recurring, **unconfigurable** (interval was a
constructor parameter with no appsettings key), **never-live-gated** outbound traffic to real
PLCs/CNCs/Agents — how often to browse someone's plant equipment is an operational decision, and the
design treated it as an implementation detail.
⚠️ **Consequence to know before you rely on rediscovery:** **AbCip and FOCAS browse a live backend but
implement no `IRediscoverable`, so they have no tag-change signal at all** — a PLC program re-download
or a CNC option install is invisible until someone re-browses the device by hand. Galaxy, TwinCAT,
MTConnect and MQTT raise natively and are unaffected. If that coverage matters later, reach for an
event-driven signal, or one that runs when something *else* already indicates change (a reconnect, a
deploy) — not a wall-clock timer. Removed with the feature: `TagSetDrift`/`TagSetDriftDetector`,
`DriverInstanceActor`'s drift timer, and `ITagDiscovery.AuthoredDiscoveryRefs` /
`DiscoveryStreamIncludesAuthoredTags`.
**`IHostConnectivityProbe` is consumed (Gitea #521, closed).** `DriverInstanceActor` subscribes
`OnHostStatusChanged`, pulls `GetHostStatuses()` fresh, and the host set rides `DriverHealthChanged` to the
AdminUI `/hosts` page — no SQL: the `DriverHostStatus` table is dropped, and so is
`DriverInstanceResilienceStatus` (#524; the live data goes over the `driver-resilience-status` telemetry
channel only). **Multi-host dispatch is wired too:** `DriverInstanceActor` resolves the breaker/host key
per call via `IPerCallHostResolver` (Modbus, FOCAS, AbLegacy, TwinCAT, AbCip).
### Comms-loss detection: `GetHealth()` is load-bearing (Gitea #527, closed)
Until #527, **`DriverInstanceActor.DisconnectObserved` had no production sender.** The only paths into
`Reconnecting` were a failed *initial* connect and an operator's Force Reconnect, so a driver that connected
and then lost its device stayed `Connected` indefinitely — publishing Bad for every tag, never
re-initialising, and **not recovering when the device returned** (live-confirmed: a Modbus driver against a
stopped fixture needed a node restart). `PublishHealthSnapshot` read `GetHealth()` for *reporting only*.
The `Connected` health tick now runs a watchdog that self-sends `DisconnectObserved` when a driver reports a
non-`Healthy` state **and** its `LastSuccessfulRead` has been frozen for 90 s. Things to know before touching it:
- ⚠️ **The conjunction is the design — "non-Healthy for N polls" is wrong and was the issue's own proposal.**
`Degraded` means *connected but some data is bad*, and several drivers write health **per tag inside the
read loop** (`ModbusDriver.ReadAsync`), so one unreadable address latches `Degraded` forever against a
perfectly reachable device. State-only tripping would reconnect that driver every 90 s indefinitely.
`LastSuccessfulRead` is the honest evidence — preserved across `Degraded`/`Faulted` writes, advanced by any
success. Each conjunct has a test that goes red when the *other* is deleted.
- **The mechanism rests on every driver reaching a non-Healthy state on sustained failure.** Surveyed across
all 12: nine poll drivers via `Degraded`, MQTT via live connection state, Galaxy via its reconnect
supervisor, OpcUaClient via a bad session keep-alive (#670 — it used to stay `Healthy` forever; its SDK
`SessionReconnectHandler` is deleted, a lost session publishes `BadNoCommunication` once per outage and the
watchdog's re-init is the recovery; good keep-alives + notifications advance `LastSuccessfulRead`). `CalculationDriver` reports `Healthy` while running and so can never trip — correct, as it has
no transport. Re-check this if a new driver lands.
- **The reconnect retry MUST tear down before re-initialising — FIXED in both drivers, keep it that way.**
`ModbusDriver` (`_transport`) and `MqttDriver` (`_connection`) used to overwrite an `IAsyncDisposable`
field inside `InitializeAsync` without disposing the old one, so re-initialising without `ShutdownAsync`
leaked a socket per attempt — ~360 for an hour-long outage at the 10 s retry interval. Both now dispose
first: `ModbusDriver.InitializeCoreAsync` releases `_transport` **and** the `_probeCts`/`_reprobeCts`
probe loops before rebuilding them (`c3970a2e` + the lifecycle gate in `409a58b6`), deliberately *not*
via the shared `TeardownAsync()` because that would also drop `_poll` and every active subscription with
nothing to re-drive them; `MqttDriver.InitializeCoreAsync` calls its (idempotent) `TeardownAsync()` first,
which reaches `_connection` and the primary-host connection (`2b4b3fcb`). The path is no longer latent —
the comms-loss watchdog reaches it, not just a manual Force Reconnect — so a new driver, or a new
`IAsyncDisposable` field on these two, must repeat the dispose-before-overwrite.
- The threshold is hardcoded (`DefaultCommsLossTimeout`), matching `HealthPollInterval` /
`DefaultReconnectInterval`. Unlike the #523 drift detector this generates **no** traffic on a healthy
system — it is a local `GetHealth()` read, and only reaches the network once a driver has already stopped
reading.
- **Fast path for DEFINITIVE losses (#676):** a driver implementing the opt-in `IDriverConnectionLossSource` raises
`ConnectionLost` (once per connection) when its protocol proves the connection can never recover in place — OpcUaClient:
keep-alive `BadSessionIdInvalid`/`BadSessionClosed`/`BadSessionNotActivated`/`BadServerHalted`, a SUSTAINED closed
connection (2nd `BadSecureChannelClosed`/`BadConnectionClosed` in one outage — what a real process stop produces; the
SDK never reopens it) or a subscription the SERVER reported dropped (its publishing-stall heuristic does not raise) — and `DriverInstanceActor` (attach PreStart, detach PostStop) turns it into the same disconnect only
while Connected; ambiguous failures (timeouts, `BadNoCommunication`, a stalled socket) must never raise and stay with
the watchdog's conjunction. `docs/DriverLifecycle.md`.
- **What comms loss now does downstream — one contract, six seams (archreview R3-02).** "A value consumer
never reports a quality better than its source's" is stated once in `docs/DriverLifecycle.md` §"The
quality contract" and implemented by `Core.Abstractions.InputQuality` (`WorstOf`/`IsUsable`). Three
things changed with it, all client-visible:
- ⚠️ **Value nodes of a comms-lost driver now go Bad.** `DriverHostActor.FanCommsLossToValueNodes`
stamps `BadNoCommunication` on the raw NodeId **and every referencing UNS NodeId**, **value retained
server-side**, via the new status-only sink primitive `IOpcUaAddressSpaceSink.WriteStatuses` (the host
holds no value cache, so only the node can keep the value). ⚠️ Retention is **not client-observable**:
the SDK's `NodeState.ReadAttribute` nulls the value of any Bad-status read/sample, so clients see
`Value = null` + the Bad status; the retained value is what the next Good re-publish and the
`(value, status)` dedup compare against. #481 had wired comms loss to the mux and to Part 9
conditions and stopped one seam short of the external data plane. A consumer that treats Bad as "drop
the tag" (ScadaBridge) will see more Bads — that is the fix working.
- **Nothing Good is written on reconnect** — recovery is the driver's own re-publish (the poll engine's
`forceRaise`, MQTT's new initial push). Instead the host **audits**: refs still latched 30 s after a
reconnect log a Warning and increment `otopcua.driver.comms_latched_refs`. Non-zero on that counter
means a driver reconnects without re-publishing.
- **Every dedup on the recovery path is keyed on (value, status), not value.** A device that comes back
usually comes back holding the same reading, so a value-only dedup swallows the recovery and the node
stays Bad forever — the v3-gate trap, now closed in the Calculation driver, the virtual-tag actor and
(via a connection-loss stale) MQTT's plain-mode suppression. **Any new consumer of driver values must
repeat this**; a regression test for it must replay the IDENTICAL value or it proves nothing.
- Consequences per seam: MQTT/Sparkplug stale on connection loss and push cached snapshots on every
(re)subscribe (`docs/drivers/Mqtt.md`); the Calculation driver and virtual tags **hold** while any
input is Bad and inherit the worst input's sub-code, evaluate through Uncertain, and hand scripts the
input's real `StatusCode`/`SourceTimestampUtc` (`docs/drivers/Calculation.md`, `docs/ScriptEditor.md`)
— so a script that relied on receiving `null` for a dead input no longer runs at all while it is Bad.
See `docs/DriverLifecycle.md` §"The comms-loss watchdog" and §"The quality contract".
## AB drivers: native transport (`Transport`)
AbCip and AbLegacy have a second, managed wire transport (`Driver.Ab.Transport`) beside libplctag, chosen by
the driver-config key `transport` = `Native` | `Libplctag` (string; takes effect when the driver is
**constructed** — every process start, or a deploy that recreates it, not `ReinitializeAsync`; AdminUI
"Transport" dropdown; CLIs `--transport`). "Default" is encoded by omitting the key; the AdminUI must never
re-emit a concrete default. The defaults differ per driver (final review, 2026-10-07):
- **AbCip: an absent key resolves to the node default, `Native` unless pinned.** Precedence (one place,
`AbCipTransportResolver`, shared by driver, probe and CLI): explicit per-driver key → node-level appsetting
**`Drivers:AbCip:DefaultTransport`** (`Native` | `Libplctag`, env `Drivers__AbCip__DefaultTransport`,
`AbCipDriverDefaultsOptions`; a bad value **fails host start**) → compiled `Native`. Bound unconditionally
because the driver factory AND the admin-node Test Connect probe both read it. ⚠️ So **the upgrade restart
alone moves no-key AbCip drivers to Native**; to stay on libplctag, set the node key — NOT a per-driver key
through the old AdminUI, whose typed form drops unknown keys (#576). Evidence: AbCip reads proven on a real
CompactLogix (Z-35162); scalar native writes (DINT/BOOL/REAL) proven on it once (2026-10-09, user-approved one-off, restored) — and native UDT member writes (DINT/BIT/REAL, #611 gate, same day) and a native STRING write (hw5, same day, restored byte-exact); and a native array-element write (`EM01_PowerUnit._SEQ_EP[31]`, user-approved, an element no alias targets, restored; neighbours unchanged); libplctag member and fragmented writes are `ab_server`-only. #566: a real Logix drops TCP on a connection idled past its timeout (no 0x0107/0x0203); the faulted session reopens on the next request.
- **AbLegacy: an absent key resolves to `Libplctag` for every family; `Native` is opt-in** (no node key). On
a real MicroLogix 1400 (Z-33323, idle, 2026-10-09) native read S/N/F/B/T/C/R/ST/O/I incl. `S:0`, T/C/R
sub-elements and status bits, and ST byte order, all correct, and wrote N/F8/B-bit (restored). libplctag (the default)
used to fail `S:`/`O:`/`I:` and `R` status bits there (incl. the default probe `S:0`) — its `pccc.c` needs explicit
file numbers and has no R status-bit mnemonics; since #659 `LibplctagLegacyName` translates (`S2:`/`O0:`/`I1:`,
`R6:0.CON/13`) and both transports read identically on that press. The authored grammar is unchanged. Read-side
only: a libplctag WRITE to any address the translation changes (`S:`/`O:`/`I:` incl. bit writes, R `EN…FD`, `C…UA`)
is refused `BadNotSupported` before any runtime is created (none could reach the PLC before — no write-surface
widening on the default transport); libplctag T/C status-bit writes it parses itself are unchanged.
L/A files, function files, SLC, PLC-5 and `logixpccc` remain unverified natively. Explicit Native on PLC-5/`LogixPccc` logs a once-per-instance Warning.
AbLegacy native refuses status-bit sub-element writes (`T4:0.DN`…, `BadNotSupported`).
Refusals are not comms faults: an Unsupported refusal (`BadNotSupported`) and an AbCip native **type
mismatch** (`BadTypeMismatch`, where libplctag returned Good garbage for a same-width mis-authored type) both
keep the tag handle, leave driver health alone (no `Degraded`, nothing for the comms-loss watchdog) and warn
once per tag — only genuine transport failures evict and degrade (a runtime `NotSupportedException` too, #611 —
it used to latch `Degraded`). AbCip width-checks every scalar **write** against the controller's bytes before
sending, on both transports (#611; libplctag used to patch part of its buffer and send it all). Controller-discovered
UDT members are writable once browse-committed and authored `ReadWrite` (BIT members included — the controller
applies the bit) — native member writes incl. a BIT member are live-gated on Z35162 (2026-10-09); sibling-bit preservation and the libplctag path are ab_server-only; whole-UDT writes are refused `BadNotSupported` by decision (`docs/drivers/AbCip.md` §Member writes). `DeviceSessionPool.Shared` logs through the
host logger factory (wired in `DriverFactoryBootstrap`; the FOCAS and TwinCAT factories' `Register(registry, ILoggerFactory?)` take one too, #602 CONV-3). The AbCip `browsePageDelayMs` knob has an AdminUI field (`AbCipDriverForm` "Browse page delay (ms)"). libplctag is confined to `Libplctag*.cs` adapters by
`LibplctagConfinementGuardTests`; do not reference it elsewhere. See `docs/drivers/AbCip.md` §Transport,
`docs/drivers/AbLegacy.md` §Transport + §Wire evidence, `docs/Configuration.md` §`Drivers:AbCip`, and
`docs/plans/2026-10-07-native-ab-transport-gate-results.md`.
## Other driver notes
- **S7:** connect attempts back off 1 s doubling to 30 s (one attempt per window, not one per call/poll/probe
tick); a write-side access-denied (PUT/GET disabled) latches a **sticky Faulted** that outranks later read
passes; driver-side array writes cover non-Bool narrow/`Int32`/`Float32` types only — Bool arrays are
refused (a one-byte-per-element encode would clobber neighbour bits).
- **MQTT:** Plain `writeTopic` writes publish at QoS 1 (Sparkplug NCMD/DCMD stay QoS 0); the Sparkplug
primary-host claim backs off 2 s → 30 s; writes encode the birth's own datatype (unsupported birth types
`BadNotSupported`).
- **No driver-side history (#655):** `IHistoryProvider`, `DriverCapability.HistoryRead`, `HistoryReadStatusException` and
the OpcUaClient history passthrough are deleted — nothing ever dispatched HistoryRead to a driver; it is served only by the
server-wide `IHistorianDataSource`. `isHistorized` on an OpcUaClient tag means OtOpcUa records it, not that upstream
history is proxied. A stored `ResilienceConfig` still carrying a `HistoryRead` key parses (diagnostic only).
- **FOCAS / AbLegacy:** background loops are joined (≤ 5 s) on shutdown.
- **FOCAS (#257/#266):** `cnc_statinfo` flags are a `Status/` fixed-tree subtree (`FIXED:Status/Run`…, capability-probed);
adjacent PMC tags in one poll batch coalesce into one `pmc_rdpmcrng` (`PmcCoalesce` config, default on) — a refused range
falls back to per-tag reads, a transport failure fails all members without retry. Parts count stays `PARAM:6711` (docs only).
- **FOCAS alarms (#647):** an `ALARM:` address (`*`, `<Type>`, `<Type>/<n>`, `<Type>/<lo>-<hi>`, `T<code>`, `OTHER`) +
the `alarm` object = ONE condition at the RawPath (either alone is skipped + reported). Overlapping selectors all
fire; `OTHER` = what no type selector claims. ⚠️ A failed `cnc_rdalmmsg2` THROWS (it used to return `[]` ⇒ mass
clear + un-acked re-raise) — and annotates its conditions `BadNoCommunication` once per outage (quality-only) and
degrades driver health with `LastSuccessfulRead` frozen, so an alarm-only driver trips the #527 watchdog. ⚠️ A CNC
REFUSAL (non-OK rc, transport fine) is `BadNotSupported` + one Warning and does NOT touch health — else the
watchdog reconnect-loops a production control. Health = worst of the value and alarm paths, computed together. The told
record lives on the driver INSTANCE (reconnect / in-place re-init re-raise nothing); a deploy RESPAWNS the driver,
and its Raise for a still-active alarm is a host-side re-statement (below). Type names are the 16i-era table only
(30i codes unconfirmed, not invented — use `T<code>` there). Read-only; ack server-side. `AlarmProjection.Enabled`
is a no-op. E2E: `Runtime.Tests/FocasNativeAlarmEndToEndTests` (incl. a respawn). `docs/drivers/FOCAS.md`.
- **A Raise on an already-active condition with the SAME message is a RE-STATEMENT (#647 review):** `NativeAlarmProjector` keeps its ack and
`ForwardNativeAlarm` writes no second "Activated" `/alerts` row — every native driver's told-record dies with its
instance, and a deploy respawns the driver. Only `Retrigger` (or inactive → active) un-acks, so a source whose
UPSTREAM is authoritative for the ack sends Retrigger for an unacked re-statement: OpcUaClient on first sight of a
condition (no prior told), Galaxy for an unacked active snapshot entry. **Those two also flag
`AlarmEventArgs.IsRestatement` (#656)** — history-only: a flagged Retrigger on an Active+unacked condition with the
same message AND severity, or a flagged Clear on an inactive one, writes no `/alerts` row (every Galaxy reconnect used
to duplicate them); a flagged `Unspecified` on an already-active condition (AbCip ALMA governing-limit drop-out)
moves the node's text/severity, keeps active/acked and writes no row. Active+acked is the atomic **`AlarmTransitionKind.ActiveAcknowledged`** (Galaxy snapshot
`ActiveAcked`, OpcUaClient inactive→acked-active) — never Acknowledge (kept a cold host's `Active=false`: the alarm was
invisible) nor Raise+Acknowledge (an unacked intermediate). Its mirror **`InactiveUnacknowledged`** (#657, OpcUaClient)
replaces Raise/Retrigger+Clear (an Active flicker; rows only for what changed). OpcUaClient's first sight of an inactive
condition is a flagged InactiveUnacknowledged, or flagged Clear+Acknowledge for inactive+acked (a flagged Acknowledge on an
acked condition writes no row) — never `Unspecified`, which wrote "Activated" for an inactive alarm and left a
retained-Active condition Active. `NativeAlarmProjector` returns the rows to write; unflagged
sources (FOCAS/MTConnect, live Galaxy) are unchanged. E2E: `Runtime.Tests/NativeAlarmRestatementEndToEndTests`.
- **Re-Initialize without teardown (#630):** FOCAS, AbLegacy, AbCip, TwinCAT, S7 and OpcUaClient `InitializeAsync`
first release the prior device/session state (cancel, ≤ 5 s join, dispose — incl. the AbCip alarm projection;
FOCAS's is PAUSED, its told record kept, #647) — the device-state half of `ShutdownAsync` only; the poll engine is kept and the actor re-subscribes
after every successful init. AbCip publishes fresh tag tables by reference swap (a second init used to throw
"tag name collision"); S7 swaps `Plc` under `_gate`.
- **Data and alarm subscriptions are independent in `DriverInstanceActor` (#662):** a data re-subscribe (every
deploy's bulk push to a kept driver) detaches DATA only (`DetachDataSubscription`) — it used to unhook
`OnAlarmEvent` too, silently killing the native-alarm feed until a respawn. The health-poll reconcile
re-attaches/re-subscribes alarms as well as data, so a failed alarm subscribe is retried every tick.
- **A config change is a respawn — there is NO in-place path (#665).** The `ApplyDelta`/`ApplyResult`/`DeltaApplied`
seam (host `ApplyChildDelta`, `DriverSpawnPlan.ToApplyDelta`, the actor's Connected reinit and mid-connect
`AdoptConfigDuringInit`) was unreachable since #516 and is deleted; #663/#664 were latent bugs in it. `IDriver.ReinitializeAsync`
stays as a per-driver contract the runtime never calls. Every driver's `InitializeAsync` still serves the config it is
handed (Calculation adopts, Galaxy refuses a non-equivalent one — #664), and still releases prior state (#630).
`docs/DriverLifecycle.md`.
- **S7 (#632/#638):** STRING/WSTRING/CHAR/WCHAR and DTL/DATE/TOD/TIME/S5TIME codecs (`S7ValueCodecs`; TOD/TIME/S5TIME
surface as Int32 ms); an undecodable buffer or unrepresentable write is `BadOutOfRange` with no health impact.
Optional `localTsap`/`remoteTsap`/`connectionType` override the slot TSAP (`remoteTsap` ⊕ `connectionType`). Test
Connect reads 2 bytes of `MB0` so a PUT/GET-disabled CPU fails the probe. The negotiated PDU size is logged, on
`S7Driver.GetDiagnostics()`, and (#640) on `/hosts` via the optional `IDriverDiagnostics` bag (string map on
`DriverHealthChanged.Diagnostics` / proto `has_diagnostics`+`diagnostics`; its digest is in the publish fingerprint).
- **TwinCAT (#637):** TIME/TOD publish Float64 ms, DATE/DT a UTC DateTime; a tag authored `UDInt` keeps the raw counter
(and keeps historizing — DateTime is not historized).
- **TwinCAT (#645, #310/#314):** opt-in ADS sum-read (`useSumRead`, `sumReadMaxItems` default 200 / ceiling 500, 32 KiB payload
budget). It reads raw bytes and decodes as the declared type, so only a scalar passing the strict width+kind check
(`CheckNotification`) rides a batch; value-converted pairs (DInt over INT, Real over LREAL/DINT) read singly. Only overall
ADS 1793/1794 latch the device to single reads; any other non-comms overall code (1792/1795/1797/1798...) reads that chunk singly for the poll without latching, with a once-per-device Warning; a comms failure neither.
Opt-in `diagnosticsEnabled` (default off) puts ADS state/app/online-change/cycle/exec/`HeadroomMs` on the `/hosts`
`IDriverDiagnostics` bag (≤ 4 extra reads per device per probe tick once settled; TC2 falls back to `.SYSTEMINFO` /
`.SYSTEMTASKINFOARR[1]`, costing 8 on the first tick and after each reconnect). Live-gated read-only 2026-10-08 on
Z35474 (TC3) + Z34834 (TC2, which answers sum commands) via a harness driving `TwinCATDriver` — `/hosts` end-to-end
not run (a stock host has no AMS router); the gate found ⚠️ Beckhoff's `SumRead` ctor throws for a `List<>` of entities (only an array survives
its internal cast) — `AdsTwinCATClient.CreateSumRead` owns it. Flat browse stays (VirtualTree is a strict subset
on TC3). Sum-write not ported.
- **AbLegacy (#636):** `AbLegacyAddress` (in `AbLegacy.Contracts`) is family-aware: PLC-5 `I:`/`O:` indices are OCTAL
and are emitted to libplctag as decimal (libplctag's `pccc.c` parses decimal only). MicroLogix function files parse
but read `BadNotSupported`. Indirect `N7:[N7:0]` (#646) is resolved by the driver at read/write time (index re-read
every call, never cached; bad index = `BadOutOfRange`; indirect writes are not atomic). One check (`AbLegacyAddress.Validate`) backs the
editor (family-independent) and the deploy gate (family from the tag's `DeviceConfig.PlcFamily`); an unparseable
address reads `BadNodeIdUnknown` for that tag only.
- **OpcUaClient (#635):** OperationLimits are honoured by the SDK session (`SessionClientBatched`) — do not re-add
driver-side chunking. Subscription tuning keys are range-checked at `ParseOptions`. `discoveryUrl` endpoints go
AFTER configured ones; more than one discovered server without `discoveryServerUri` fails the connect. Session
events are the OTel counter `otopcua.driver.opcuaclient.events`. `HistoryAggregateType` carries the full Part 13
set; an aggregate the gateway can't serve is `BadAggregateNotSupported` per node.
- **MTConnect (#627/#628):** TIME_SERIES reads as a variable-length `double[]` (Float64 only); a count disagreeing
with `sampleCount` is `BadDecodingError`. One MTConnect driver = one agent client = at most one `/raw` Device
(`MTConnectMultipleDevices` deploy error; `RawTreeService.CreateDeviceAsync` refuses a second).
- **MTConnect CONDITION alarms (#626):** a CONDITION raw tag is a native Part 9 condition only when its `TagConfig`
carries the standard `alarm` object (opt-in — alarm tags leave the value subscription, so auto-promotion would break
existing String tags). Normal→inactive, Warning→500, Fault→800; one condition per DataItem showing the most severe
active nativeCode (ties: ordinal nativeCode, then type); ack is server-side only. ⚠️ UNAVAILABLE never clears: an
active condition stays active+acked with a quality-only `BadNoCommunication` (same as #481 transport loss). The
driver dedups against what the server was last TOLD (level+nativeCode+nativeSeverity+qualifier — not text), records
only while a handler is attached, and drops the record of only the conditions that LEFT the set when the subscribed
alarm SET changes (#641) — never on a reconnect of the same set; a replayed `/current` must never re-raise (un-ack).
`AlarmEventArgs` gained optional `OpcUaSeverity` (exact Part 9 severity) and `QualityStatusCode`; a quality-only
event goes out as `AlarmQualityUpdate` per node and never through the snapshot projection.
- **Native condition state survives a full rebuild (#641):** `RebuildAddressSpace` captures each NATIVE condition's
live node state (client acks included), the same-apply re-materialise restores it, and `CompleteAddressSpaceApply`
(called by `OpcUaPublishActor` at the end of every apply) reports ONE event per restored off-normal condition and
discards unclaimed captures. `NativeAlarmProjector` is pruned per apply, not cleared. Native routing is an exact
`ConditionId == RawPath` match — AbCip's `#active` suffix was fixed; FOCAS routes since #647 (below).
- **Driver removal (#609):** removing a driver releases its capability-invoker pipelines and resilience
tracker, *then* evicts its telemetry snapshots. ⚠️ Both are **tombstoned**, not just cleared: the removed
child stops asynchronously and its in-flight subscribe/connect still completes through the invoker (and
emits Health) on the thread pool *after* `StopChild`, which used to resurrect the state — so a removed
driver's counters were republished forever. `Release` → tracker `Retire` and hub `Evict` ignore every later
write for that id until it is spawned again (`Create` → `Reinstate`, `SpawnChild` → `Readmit`). A
Watch/`Terminated` release would not work — the late writes land after the actor terminates.
+275
View File
@@ -0,0 +1,275 @@
---
paths:
- "src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.*/**"
- "src/Core/ZB.MOM.WW.OtOpcUa.Core.AlarmHistorian/**"
- "src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/**"
- "src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/**"
- "src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/**"
- "tests/**/*Historian*/**"
- "docs/Historian.md"
---
# Historian / HistoryRead (Gateway, SqlServer, QuestDb backends; alarm history; continuous historization; provisioning)
> Moved verbatim from the root `CLAUDE.md` (2026-10-13) so it loads only when working in this area.
> Root `CLAUDE.md` § "Subsystem guides" indexes it. Edit it here, not in the root file.
## Historian / HistoryRead
**Backend: pluggable — `ServerHistorian:Backend` = `Gateway` (default), `SqlServer` or `QuestDb`** (see the subsection below).
As of the gateway-integration cutover, the **Gateway** backend's historian read, alarm-write, and
continuous-historization paths are all served by the **`ZB.MOM.WW.HistorianGateway`** sidecar, consumed as the Gitea-feed
**`ZB.MOM.WW.HistorianGateway.Client`** gRPC package (`historian_gateway.v1`) behind a thin
`IHistorianGatewayClient` seam in `ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway`. **The bespoke
Wonderware TCP/ArchestrA sidecar projects and the vestigial `Historian.Wonderware` driver type were
retired** — there is no Wonderware backend in the tree anymore (see `docs/drivers/Historian.Wonderware.md`,
now a retired stub).
A tag is historized by adding `"isHistorized": true` to its `TagConfig` JSON blob (authored in the
raw-JSON textarea on the `/uns` TagModal); an optional `"historianTagname"` field overrides the default
historian tagname, which is the tag's driver `FullName`.
### Pluggable backends, SQL Server and HistoryUpdate (2026-10)
- **Selection:** `ServerHistorian:Backend` (`Gateway` default | `SqlServer` | `QuestDb`; unknown fails host start). `HistorianBackendRegistry` → `IHistorianBackendProvider` builds all five seams; the gateway keys are only required for `Gateway`. `SqlServer` uses `ServerHistorian:SqlServer:{ConnectionString (required), Schema=hist, CommandTimeout=30s, AutoCreateSchema=true}`.
- **SQL Server** (`Driver.Historian.SqlServer`): `hist.Sample` PK `(TagId, SourceTimeUtc)` — writes are insert-if-absent, so outbox redelivery is idempotent. Stores every value kind with its **real status**; per-sample aggregates (10, simple definitions) are pushed down to T-SQL with bucket math in **100 ns ticks**; the Part 13 time-weighted seven (TimeAverage/TimeAverage2/Total/Total2/Interpolative, #671 — ⚠️ `Total` is now the time-weighted integral, no longer `SUM` — plus the **duration-based** PercentGood/PercentBad, #680) stream window + 4 bound rows into the shared `TimeWeightedAggregator` (both SQL backends, identical results; reverse `Interpolative` anchored at the later time; window capped by `ServerHistorian:MaxTimeWeightedRawRows` ⇒ `BadTooManyOperations`, percentages included). ⚠️ **#680 status rules (client-visible):** TimeAverage/Total are Uncertain if any part of the interval uses an Uncertain value, is bridged across a Bad sample, is extrapolated past the last sample (⇒ a live tag's latest interval is Uncertain) or is not covered; TimeAverage2/Total2 keep the SDK time-based rule (Uncertain time counts as Good) but never extrapolate past the last sample — that interval is Partial, later ones `BadNoData` (a report-by-exception tag with nothing after its last sample reads `BadNoData` until its next sample arrives). `docs/Historian.md` §"Time-weighted aggregates". ⚠️ **#680 per-sample (client-visible):** `Count` counts **Good** samples only (value-based status; an all-Bad interval is generic Bad, value null) and an empty interval **inside the tag's data range** is `0`/Good (outside it still `BadNoData`); `Start`/`End` return the **raw** first/last sample of any quality — its own timestamp and status, value null when Bad; `Delta` is Uncertain only when the interval's first or last sample is unusable; ⚠️ #684: `Average`/`Minimum`/`Maximum`/`Range`/`*ActualTime` use **Good samples only** (was Good + Uncertain; an Uncertain-only interval is `BadNoData`), Min/Max-family status Uncertain on a Bad sample, an Uncertain value beyond the Good extreme (Part 13 — the SDK ignores that) or an Uncertain non-numeric sample, and only a **Good** String/DateTime sample refuses them (`BadAggregateInvalidInputs`; Delta still refuses any usable one); and every per-sample aggregate but `Average` sets **end-of-data Partial** on the interval holding the last sample (forward incl. its earlier edge, reverse strictly inside), never on a Bad result. The data-range lookup is a key seek on SQL Server and a `tag` symbol-index seek on QuestDB (#683 — `{p}sample.tag` is `SYMBOL INDEX`; the bootstrapper ADDs it to a pre-#683 table, idempotent), read BEFORE the bucket statement so a racing append can never yield a Good `Count` of 0 (a concurrent HistoryUpdate delete is the mirror case and still can, transiently). `docs/Historian.md` §"Aggregates". Docker-dev recipe, schema and aggregate semantics: `docs/Historian.md` §"SQL Server backend". Integration tests need `HISTORIAN_SQL_TEST_CONNECTION`. **Retention (#667):** opt-in `SqlServer:RetentionDays` (+ `PurgeInterval` 1h, `PurgeBatchSize` 4000) -> `HistorianRetentionService` purges old `hist.Sample` (per tag) / `hist.Event` in `DELETE TOP(n)` batches, Primary-gated on `IHistorizationRoleView`, one purger per DB via `sp_getapplock`; counter `otopcua.historian.retention.purged{table}`. Optional `IHistorianBackendProvider.CreateRetention` (default null). `docs/Historian.md` §Retention.
- **QuestDB** (`Driver.Historian.QuestDb`, `ServerHistorian:QuestDb:*`, Npgsql over PG wire; docker-dev overlay `docker-compose.questdb.yml`): same kinds/status/aggregates/HistoryUpdate as SqlServer, soft delete by tombstone (INSERT only — never `UPDATE`/`DELETE`; every read filters `NOT deleted`). ⚠️ **Timestamps are µs** (`QuestDbConnection.Round`/`BindTimestamp`; `IHistorianDataSource.TimestampResolutionTicks` = 10 steps HistoryRead paging), **pre-1970 cannot be stored** (writer drops, HistoryUpdate `BadOutOfRange`), every write waits for WAL visibility, TTL is enforced on write commits only, set Npgsql `Maximum Pool Size` < QuestDB's 64. `docs/Historian.md` §"QuestDB backend".
- ⚠️ **Reverse processed reads are Part 13/SDK-anchored (#677, client-visible):** every aggregate on SqlServer/QuestDB now anchors a reverse read at its later time — window `(end, start]`, partial interval earliest, each stamped with its later edge (was: earlier-anchored, stamped earlier edge); on forward reads Partial also marks the interval the data starts inside (not Average — the SDK's rule, forward-only); a non-default client `AggregateConfiguration` ⇒ `BadAggregateConfigurationRejected`; Gateway provisioning sends `IntegralDivisor = 1`. SDK-parity tests: `SdkAggregateParityTests`. `docs/Historian.md` §Aggregates.
- ⚠️ **Reverse / end-only RAW reads work on Gateway (#686, client-visible):** `IHistorianDataSource.SupportsReverseRaw` (SqlServer/QuestDB/Null `true`, Gateway `false`, interface default `false`, guard-tested) — the node manager sends `start > end` only to a backend that declares it; otherwise `ReverseRawReader` serves it from FORWARD reads: uncapped ⇒ ONE forward read reversed in memory; capped ⇒ a bounded walk back in segments (ms-aligned request bounds, width aimed from observed density at ≈ N + 8 rows and carried page to page in the continuation point — #689: it used to aim at half `MaxTieClusterOverfetch`, ~32k rows per page on a dense tag; a read asks for `2N + 18` rows (a discovery segment ≤ 1,026), `MaxTieClusterOverfetch` is only the tie-cluster ceiling — a full read is re-read, never answered truncated; only more than max(1, `MaxTieClusterOverfetch`) samples within ONE MILLISECOND ⇒ `BadTooManyOperations` (per ms, not per timestamp — requests are ms-rounded, and letting a segment narrow below 1 ms stepped a page through one ms a tick per read, ~10k reads); walks no later than now + 1 day and never before the Unix epoch; #691: after two empty segments it asks once for the whole remainder, and the stretch a page proved empty rides the continuation point — the last pages of a deep page-through no longer each walk ~14 reads to 1970). The raw/events tie-path oversized-cluster refusal is now `BadTooManyOperations` too (was `BadHistoryOperationUnsupported`). Uncapped Gateway raw reads (forward too) used to send `max_values = 0`, which the client/gateway reject — now unbounded (`int.MaxValue`; a ceiling refusal ⇒ `BadTooManyOperations`), and a cancelled read is no longer a backend failure. Since #669 such reads failed the node on Gateway and counted as a backend fault (the gateway rejects start > end — `GatewayHistorianDataSource.ReadRawAsync` now refuses one unsent); before #669 an end-only Gateway read returned the OLDEST N ascending — now the latest N at/before EndTime, newest first. Events follow since #691.4: a reverse/end-only capped events read on Gateway (`SupportsReverseEvents` false; Null now reports `true` and answers empty) is walked back the same way (ties sorted by EventId — the gateway orders by time only; one extra event found ⇒ continuation point) — it used to be refused `BadHistoryOperationUnsupported` (explicit reverse) or answered with the OLDEST N ascending (end-only); needs the gateway's `RuntimeDb:EventReadMaxRows` 0/unlimited (it truncates silently). Uncapped Gateway events reads (forward too) used to throw "Array dimensions exceeded" (result list pre-sized to `int.MaxValue`) — fixed. Above a truncated read the walk gallops up from the newest row seen, then bisects (never below 1 ms): a burst that ended 100 days ago ≈ 18 reads, a dense tag that stopped days ago ≈ 26. `docs/Historian.md` §"Open-ended and reverse raw reads".
- **Advertised aggregates (#682):** `IHistorianDataSource.SupportedAggregates` (Gateway 4 via `GatewayHistorianDataSource.ServedAggregates`, SqlServer/QuestDB 17 via `AggregateFamilies.Served`, Null **none**; a null set ⇒ all) is the one authority for `HistoryServerCapabilities/AggregateFunctions` (i=11201 — only its folder references are trimmed) and the up-front `BadAggregateNotSupported` in HistoryRead-Processed. ⚠️ `ServerCapabilities/AggregateFunctions` (i=2997) is deliberately NOT trimmed: it keeps the SDK's 37 because live `AggregateFilter` subscriptions still compute them all. ⚠️ Historian disabled ⇒ Processed is now `BadAggregateNotSupported`, not `GoodNoData`. `docs/Historian.md` §Advertised aggregates.
- **Raw `ReturnBounds` is honoured on every backend (#691, client-visible):** Part 11 bounding values (§4.6 Table 1 is the test oracle) are composed in the server by `HistoryBounds` from each backend's raw reads — "at or before" = capped reverse read (native on SQL backends, the #686 `ReverseRawReader` walk-back on Gateway, never AVEVA's synthetic boundary row), "at or after" = capped forward read. Bounds are stored samples of any quality, count toward `NumValuesPerNode`, start bound on the first page / end bound on the last (pending end bound rides the continuation point); missing ⇒ `Bad_BoundNotFound` (null, at the bound time; a one-sided read's open side ±1 s from the previous value). ⚠️ **Breaking for SDK-default clients:** the .NET SDK's `new ReadRawModifiedDetails()` defaults BOTH `ReturnBounds = true` AND `IsReadModified = true` — a client that cleared only `IsReadModified` now gets bounds, and one that set neither is a modified read with bounds ⇒ `Bad_InvalidArgument` (§6.5.3.3); set both explicitly (Client.Shared does; tests building details must too). No data AND no bound found ⇒ `GoodNoData`, no values (the pre-bounds answer). On Gateway the at-or-after lookup ends at now + 1 day (never a year-9999 end to AVEVA). Kill switch `ServerHistorian:HonorReturnBounds` (default `true`; `false` = pre-#691). `docs/Historian.md` §Bounding values.
- ⚠️ **Raw pages are full (#692, client-visible, every backend):** each raw page asks the backend for `N + skip + 1` rows (the boundary ties it will trim + one lookahead row) — a resumed page used to ask only N, trim the ties and come back short; at N=1 that was an EMPTY page + continuation point that then fell into the `MaxTieClusterOverfetch + 1` over-fetch (live: 8 values = 17 pages / 25 reads / 524k rows ⇒ now 8/8/≤24). No lookahead row ⇒ last page, no CP (an exact multiple no longer ends on an empty page). Exception: an oversized tie cluster — its last slice is `min(N, remaining)` with a CP, and stepping past one at the end of data can give one empty final page. The over-fetch fires only when every row past the trimmed ties sits on the boundary timestamp; a cursor inside such a cluster is marked `WithinTieCluster` (next page goes straight to the over-fetch). `SliceTieCluster` resumes only strictly before EndTime (truncated to the backend resolution) — it used to resume AT the exclusive EndTime and return samples there. Bound placeholders carry an explicit marker (`HistoryBounds.IsPlaceholder` no longer matches a stored `Bad_BoundNotFound`/null row); the N=1 bounds probe carries its walk hint + resolved end bound into the cursor. Reviewer fuzz harnesses (paged vs unpaged over random tie series) are the right test for any paging change — repo tests passed while master mismatched 1-in-160.
- ⚠️ **Strict-ceiling paging (#694):** `IHistorianDataSource.MaxRawValuesPerRead` = L = learned gateway ceiling − 16 (0 on SQL backends/Null, guard-tested; 0 until a refusal names the ceiling). `RawPageLimit.Plan(want, k, L)` sizes every raw page so page + boundary ties + lookahead ≤ L (room = L − k − 1, signed; first page L−1, resumed L−k−1; uncapped `N = 0` reads page too) — a capped read above the ceiling used to be `BadTooManyOperations`, now a ceiling-sized page + CP. One re-plan when the read taught a new limit or was refused for crowding (`HistoryReadLimitExceededException.PreWindowRows`, set only above the 16-row allowance; the page shrinks by it). No room on a resumed page ⇒ the tie-cluster over-fetch, bounded min(M, L−1); the slice size is never 0. The page path passes `retryOnLearnedLimit: false` to `ReverseRawReader` (the walk re-running at the old N cost 200–3,000 reads cold); bounds lookups keep it. ⚠️ A Gateway window starting at/before the Unix epoch counts AVEVA's synthetic row as pre-window — it used to set `reachedEnd`, so a ceiling-truncated read came back Good with no CP (forward-from-1970 = first page only; end-only past the data = the OLDEST rows as "latest"; pre-existing). The #687 crowding re-read now doubles its room up to 8 re-reads, never past a known ceiling, and its room is capped at `MaxTieClusterOverfetch` (passed to the Gateway source by the factory; past it ⇒ `BadTooManyOperations` with `PreWindowRows`). A raised ceiling is not re-probed by raw paging (pages stay small until another read re-probes or restart).
- **Real status fix:** the recorder used to stamp every sample Good (`192`); it now carries the driver's status. ⚠️ On the **Gateway** backend this changes nothing stored: the gateway discards quality (C-002), so AVEVA still stores Good.
- **Comms loss is historized as an outage marker** (`Empty` + `BadNoCommunication`, one per tag per outage) on SqlServer; dropped (`DroppedUnsupportedKind`) on Gateway. Outbox records are **format v2** — drain or delete `ContinuousHistorization:OutboxPath` before downgrading a node.
- **HistoryUpdate** (SqlServer + QuestDb; Gateway ⇒ `BadHistoryOperationUnsupported`): Insert/Replace/Update/DeleteRaw/DeleteAtTime, **Primary-only** since #674 (off-Primary ⇒ `BadInvalidState`, after authorization AND after planning — an unservable request (Remove / `IsDeleteModified` / not historized) stays `BadHistoryOperationUnsupported` off-Primary; the verdict is `IHistoryUpdateRoleView` = the data-plane verdict but **closed until this node's first redundancy snapshot**, so a pair cannot both accept in the boot window; admin-only nodes register no view ⇒ no gate; reads ungated), items with a `SourceTimestamp` past now + `ServerHistorian:HistoryUpdateMaxFutureSkew` (default 1 day, ≤0 disables) ⇒ `BadOutOfRange` in the node manager (every backend), gated in both ACL modes — flag off needs the `HistoryUpdate` role from a **Cluster-scoped** grant (+ key re-check), flag on uses `DataPlaneAuthorizer`'s HistoryUpdate operation; `WriteOperate` never implies it. The `HistoryWrite` AccessLevel bit is set only when an updater is wired at materialise time. `docs/Historian.md` §HistoryUpdate, `docs/security.md`. ⚠️ **SqlServer schema v1 is accepted (#674)** with `AutoCreateSchema=false`: writes/reads/events/retention work; modified reads + HistoryUpdate ⇒ node-level `BadHistoryOperationUnsupported` on every arm (the updater throws `NotSupportedException`, nothing sent; the `HistoryWrite` bit is still advertised — not knowable at materialise time), retention skips `SampleModified`, one Warning per boot (`SqlServerSchemaBootstrapper.SupportsModifiedTrail`; `RequiredVersion` = 1, `ModifiedTrailVersion` = 2). QuestDB: keep `cairo.ttl.use.wall.clock=true`; a QuestDB trail INSERT retried after a client timeout duplicates (no DEDUP key).
- **Modified-value history (#668, Part 11):** every applied HistoryUpdate item leaves a trail row (prior value, principal from the session's API key / `anonymous`, `HistoryModificationType`, time) — SqlServer `hist.SampleModified` (**schema v2**, written by `OUTPUT … INTO` in the same statement ⇒ atomic; ⚠️ no FK/CHECK/trigger on it, and `AutoCreateSchema=false` installs must run `002_sample_modified.sql` before upgrading), QuestDB `{p}sample_modified` (append-only, written **before** the change ⇒ a crash leaves an orphan trail row, never an untracked change). `IsReadModified=true` HistoryRead serves it as `HistoryModifiedData` via `IHistorianDataSource.ReadModifiedAsync` (default ⇒ `NotSupported` ⇒ `BadHistoryOperationUnsupported` on Gateway/Null), paged on `(SourceTime, ModifiedId)`, gated by plain HistoryRead. Live writes and retention never write it; retention/TTL purge it on source time. `IHistorianDataUpdater` methods take a `HistoryUpdateContext`. CLI `historyread --modified`. `docs/Historian.md` §Modified-value history.
### Read path (`ServerHistorian` section)
The server dispatches all OPC UA HistoryRead to the registered `IHistorianDataSource` — the
`GatewayHistorianDataSource` read client when enabled, else the `NullHistorianDataSource` default
(historized nodes return `GoodNoData` for Raw/AtTime — Processed is `BadAggregateNotSupported` since #682). Supported variants: Raw (⚠️ #687, client-visible: Gateway raw reads no longer return AVEVA's synthetic start-boundary row — asked from 1 ms early (epoch-floored) with `NumValuesPerNode + 16` rows (unclamped; 0 ⇒ `int.MaxValue`) and filtered by timestamp — so an empty window is `GoodNoData`, not a fabricated value at StartTime; the forward end is now exclusive (a point read inclusive); pre-window rows that crowd the budget trigger ONE re-read with room for them, else `BadTooManyOperations`; `ReturnBounds` is ignored on every backend; ⚠️ #690: a Gateway Bad row — a stored NULL, which histsdk sends as value 0 — now reads value **null** + Bad on Raw and AtTime, as on the SQL backends; ⚠️ #691.3: a strict gateway (`AllowStreamingBeyondCeiling=false`) that refuses a raw budget is re-asked ONCE at the ceiling its refusal names — raw and per-sample reads alike; the ceiling is then remembered per data source (`GatewayRowCeiling`: refusals count as gateway errors), re-learned from a refusal when lowered, and re-probed once at the full budget when a ceiling-bounded read comes back full (raised / streaming) — so an uncapped (`NumValuesPerNode = 0`) read of a small window is served; a ceiling-bounded stream that is not provably complete is `BadTooManyOperations`, never a truncated answer), Processed
(backend-dependent: SqlServer/QuestDb serve seventeen; Gateway twelve = `GatewayHistorianDataSource.ServedAggregates` — ⚠️ #681, client-visible: the ten per-sample ones (Average, Count, Min/Max, Range, Start/End/Delta, Min/MaxActualTime) are evaluated from RAW gateway reads by the shared `Sql.Common` evaluator — identical to the SQL backends, Calculated bits, interval-start stamps, #677 reverse; Min/Max are no longer AVEVA modes, the read moves every raw sample of the window (a month of 1 Hz ⇒ `BadTooManyOperations` at the default `MaxTimeWeightedRawRows` cap), and every gateway stream is consumed to completion (an abandoned one makes the gateway evict its shared historian session) — plus TimeAverage/Total via AVEVA modes (interval-start stamped + Calculated; ⚠️ #685: AVEVA's extra pre-window row is dropped ⇒ one row per interval in `[Start, End)`, and an Uncertain (opc 64) interval keeps its value as `UncertainDataSubNormal` — AVEVA's TWA excludes NULL-gap time where the SQL backends bridge it; per-interval values still need gateway fix historiangw#27; ⚠️ #691.2: native reads use the SQL backends' grid in both directions via forward requests only — a REVERSE read (refused by the gateway before) is anchored at its later time (#677), and the clipped interval (earliest reverse, LATEST forward — which used to be averaged past EndTime) is its own exact-span read marked Partial unless Bad; an interval wider than the window is sent as the window, interval 0 as one interval; more than 100,000 intervals is refused up front (`BadHistoryOperationUnsupported`, as on the SQL backends — forward used to be served up to the gateway's 1,000,000); until historiangw#27 lands both directions return Good-status wrong values); the other time-weighted five stay `BadAggregateNotSupported` because the gateway has no prior-sample read for Part 13 bounds; `docs/Historian.md` §Aggregate set), and AtTime over historized variable nodes (⚠️ #693, client-visible: Gateway AtTime is evaluated from RAW reads by the shared `Sql.Common` `AtTimeEvaluator` — exact stored value at a stored time, null + Bad at/after a stored NULL, Uncertain hold before a Bad neighbour, linear + Interpolated bit, `BadNoData` before the first sample — never AVEVA's own interpolation (which answered 107 Good at a NULL, null+Bad at a Good sample after it, 99.99999999944 for 100, no Interpolated bit); `GatewayAtTimeReader`: one ≤1,024-row window read for the batch (jumps to far-apart times use 64-row reads) + a walk-back for the first time's prev (gallop ×4, whole-remainder read after two empties, then real BISECTION seeded by any full read's last row — a dense tag stopped a year ago ≈ 25–31 reads; the first cut regrew segments after a full read and gave up at 65) + an 18-row probe for the last time's next — 10 dense times = 1 read (exact) / 3 (interpolated), 10 hourly = 12; read cap 64 + 16 per distinct time (a runaway guard: a time in a gap of a bursty tag needs ≈ 6 reads; 4/time refused a 1 h/day tag asked daily ×30) and a strict ceiling < 4 rows ⇒ `BadTooManyOperations` with a message naming the limit; DateTime hints stepped, non-finite/out-of-range values under an integer hint unusable; all request bounds whole ms, rows counted only strictly inside a request; the node manager passes the node's scalar type (`ReadAtTimeAsync(…, TypeCode? valueType, …)` default-interface overload, ignored by SqlServer/QuestDB/Null) and the Gateway converts raw rows to it before evaluating — Boolean stepped, integers interpolated + rounded half-away and returned as the node's integer type, floats Double; no hint ⇒ Double; ⚠️ the shared evaluator used to box interpolated Int64 as Double (`cond ? (long)x : double` widens) — fixed for SQL backends too, and Shouldly `ShouldBe` hides numeric-type bugs (assert `ShouldBeOfType`); kill switch `ServerHistorian:GatewayAtTimeFromRaw` (default `true`); `docs/Historian.md` §Gateway AtTime); Events over
alarm-owning equipment-folder event-notifier nodes. Reads are ungated (served from any redundancy node);
authorization uses the standard `AccessLevels.HistoryRead` bit set at materialization.
`ServerHistorian` appsettings keys (`ServerHistorianOptions`; `Enabled` defaults to `false`):
| Key | Default | Notes |
|---|---|---|
| `Enabled` | `false` | `true` registers the gateway read client; `false` keeps `NullHistorianDataSource` |
| `Endpoint` | `""` | Absolute gateway URI, e.g. `https://host:5222`. Scheme selects transport (`https://` = TLS, `http://` = h2c) |
| `ApiKey` | `""` | Peppered-HMAC key `histgw_<id>_<secret>` sent as `Authorization: Bearer`. **Supply via env `ServerHistorian__ApiKey` — never commit** |
| `UseTls` | `true` | Connect over TLS; must match the `Endpoint` scheme |
| `AllowUntrustedServerCertificate` | `false` | Accept a self-signed / untrusted server cert (dev / on-prem only) |
| `CaCertificatePath` | `null` | PEM CA file pinning the gateway TLS chain; null/empty uses the OS trust store |
| `CallTimeout` | `00:00:30` | Per-call deadline for each unary gateway read |
| `MaxTieClusterOverfetch` | `65536` | Bounded over-fetch the HistoryRead-Raw paging uses to page within an oversized same-timestamp tie cluster (retained from the prior backend) |
| `HonorReturnBounds` | `true` | #691 kill switch: `false` ignores HistoryRead-Raw `ReturnBounds` as before (no bounding values, no modified-read `Bad_InvalidArgument`) |
| `GatewayAtTimeFromRaw` | `true` | #693 kill switch (Gateway only): `false` = AVEVA's own AtTime interpolation passed through, as before |
| `EventReadHorizon` | `00:00:00` | #694, opt-in and LOSSY: when > 0, HistoryReadEvents never reads events older than now − horizon (fresh windows clamped, both directions, all backends; a window wholly older ⇒ `GoodNoData` unsent). Bounds the deep "rest is empty" event probe (~3.8 s live, 30 s gateway SQL timeout unfiltered). No per-source earliest-event cache by design — late S&F drains would make it stale |
| `HistoryReadBatchParallelism` | `4` | Max nodes served concurrently within one HistoryRead batch (03/S3); `⌈N/P⌉ × CallTimeout` worst case instead of sequential |
| `MaxConcurrentHistoryReadBatches` | `16` | Process-wide concurrent HistoryRead batch cap (03/S3); flood degrades to `BadTooManyOperations`. Total in-flight reads capped at 16 × 4 = 64 |
| `MaxTimeWeightedRawRows` | `1000000` | `SqlServer`/`QuestDb` (#671): raw in-window rows one time-weighted aggregate read (incl. `Total` and, since #680, PercentGood/PercentBad) may stream into the C# evaluator — and, since #681, every `Gateway` per-sample read (raw-evaluated); window query capped at this + 1, past it ⇒ `BadTooManyOperations` for that node. ≤ 0 = unbounded. Gateway (#688): the `cap + 18` budget is clamped at the gateway's default `MaxValuesCeiling` (1,000,000) while the cap is ≤ it, so the default no longer trips a strict (`AllowStreamingBeyondCeiling=false`) gateway |
| `HistoryReadDeadline` | `00:01:00` | Per-request HistoryRead wall-clock deadline (03/S3); a node still in flight surfaces `BadTimeout`. Non-positive = unbounded |
### Alarm-history path (`AlarmHistorian` section)
Alarm events are written through `GatewayAlarmHistorianWriter` (the gateway **`SendEvent`** path) behind
the durable **`LocalDbStoreAndForwardSink`** — `AlarmHistorian:Enabled=true` swaps the `NullAlarmHistorianSink`
default for the store-and-forward queue, which buffers into the node's consolidated LocalDb as the
replicated `alarm_sf_events` table (see the LocalDb section) and whose drain worker forwards batches to
the gateway and uses per-event outcomes to decide retry vs. dead-letter (never throws). **Only the node
holding the Primary role drains** (a gated node reports `HistorianDrainState.NotPrimary`). The
`AlarmHistorian` section carries only the `Enabled` gate + the queue knobs (`DrainIntervalSeconds`,
`Capacity`, `DeadLetterRetentionDays`, `BatchSize`, `MaxAttempts`) — the downstream gateway connection
(endpoint/key/TLS) is sourced from the `ServerHistorian` section. **Alarm-history `ReadEvents` requires the
target gateway deployed with `RuntimeDb:EventReadsEnabled=true`** (the C2 SQL event-read workaround).
### Continuous historization (`ContinuousHistorization` section)
When `ContinuousHistorization:Enabled=true` **and** `ServerHistorian` is configured, the Host builds a
durable, crash-safe **FasterLog** outbox (`FasterLogHistorizationOutbox`) + a gateway-backed
`IHistorianValueWriter`, and `WithOtOpcUaRuntimeActors` spawns the `ContinuousHistorizationRecorder`. The
recorder taps the per-node dependency-mux value fan-out, appends each numeric value to the outbox (the
crash boundary), and drains the outbox to the gateway's SQL live-value write path (**`WriteLiveValues`**)
with exponential backoff. The gateway connection is sourced from `ServerHistorian`; this section carries
only the recorder + outbox knobs:
| Key | Default | Notes |
|---|---|---|
| `Enabled` | `false` | `true` (with `ServerHistorian` configured) wires + spawns the recorder |
| `OutboxPath` | `""` (required when enabled) | **Directory** holding the FasterLog segment + commit files. In production set an **absolute** path on durable storage |
| `CommitMode` | `PerEntry` | `PerEntry` = fsync before the append returns (no loss window); `Periodic` = batched commits every `CommitIntervalMs`. ⚠️ `PerEntry` now means **one fsync per ingest burst, not per value** — the recorder appends a burst through `AppendBatchAsync`, which enqueues every entry then issues a single group commit; the call still does not return until that commit lands |
| `CommitIntervalMs` | `100` | Periodic-mode commit cadence; required `> 0` only under `Periodic` |
| `DrainBatchSize` | `1024` | Entries peeked + written per drain pass. **Raised from 64**: the peek is an in-memory scan and the pass is bounded-parallel (not sequential), so a wide batch no longer costs `tags × latency` — while 64 drew down ≤ 64 entries per `DrainIntervalSeconds`, letting the outbox fall permanently behind during any gateway hiccup |
| `DrainIntervalSeconds` | `2` | Steady drain cadence (and post-success reschedule) |
| `DrainConcurrency` | `8` | Historian tags one drain pass writes **concurrently** (a batch is grouped per tag; each group is its own gateway round-trip), so a sequential `tags × latency` pass becomes `⌈tags / concurrency⌉ × latency` with outstanding round-trips bounded. `1` = fully sequential. The ack is unchanged — all-or-nothing after every group reports, and the #538 quarantine still keys on "another group wrote this pass" |
| `Capacity` | `500000` | Max un-acked outbox entries before drop-oldest; `0` = unbounded. Non-zero **by default since #538** — the disk backstop for a historian refusing *every* write, the one case the per-tag quarantine deliberately cannot resolve |
| `MaxWriteAttempts` | `5` | #538: consecutive failed drain passes one tag may accumulate — counted **only across passes where other tags wrote successfully** — before its queued entries are abandoned so the FIFO head can advance |
| `MinBackoffSeconds` | `1` | Initial retry backoff after a failed drain pass |
| `MaxBackoffSeconds` | `30` | Cap on the exponential retry backoff |
| `CaptureWhilePrimaryUnreachable` | `true` | #672: a standby stood down for its mesh Primary (#537) captures as soon as that Primary is reported UNREACHABLE, instead of after auto-down — crash-failover gap ~30 s ⇒ ~10–12 s, at the cost of duplicate windows (see #537/#672 below). `false` = pre-#672 |
### Tag auto-provisioning (`IHistorianProvisioning` EnsureTags hook)
`AddressSpaceApplier.Apply()` fires a **non-blocking, fire-and-forget** `IHistorianProvisioning.EnsureTagsAsync`
hook for added historized value tags — and, since #600.1, for **changed** raw tags that flip to historized or change their resolved historian name or DataType — the gateway-backed `GatewayTagProvisioner` calls the gateway's
`EnsureTags` so a brand-new historized tag exists in the historian before the recorder's `WriteLiveValues`
land. The hook is wrapped so a faulted/synchronously-throwing provisioner can **never** block or fail a
deploy. **Gateway backend:** non-numeric (`String`/`DateTime`/`Reference`) data types are skipped (not
provisioned), the recorder likewise drops + meters non-numeric values, and continuous historization is
**numeric-analog only** (`UInt16→UInt4` is a documented fallback). The **SqlServer** backend provisions and
records every type (see "Pluggable backends" above).
The provisioner is registered by the Host's `AddHistorianProvisioning` (`Runtime.ServiceCollectionExtensions`),
built via `GatewayHistorian.CreateProvisioner`, and **gated on the same `ServerHistorian:Enabled` flag as the
read path** — `WithOtOpcUaRuntimeActors` resolves `IHistorianProvisioning` and passes it into the applier.
When the section is disabled the applier binds the no-op `NullHistorianProvisioning` and provisioning does not
run. Every dispatch logs a tally (`dispatched/requested/ensured/skipped/failed`); `dispatched=N` with
`requested=0` flags the dormant no-op. The API key must carry `historian:tags:write` for `EnsureTags`. (PR
#423 shipped `GatewayTagProvisioner` but left this wiring out, so provisioning was dormant; restored 2026-06-27.)
**DataType mapping (issue #441):** `EnsureTags` is an **upsert** (create-or-update, not skip-if-exists).
`HistorianTypeMapper` maps **`Boolean → Int2`** (Boolean historizes as a small 0/1 integer) because the
historian's `Int1` analog-tag creation path is **server-degenerate** — `EnsureTags(Int1)` stores an unusable
stub (live-probed; gateway-side resolution in HistorianGateway #11 / PR #16). Both Float and Int2 are
supported analog types, so a pre-existing Float Boolean tag retypes cleanly to Int2 on its next deploy — see
the "Boolean historization maps to Int2" section in `docs/Historian.md`.
### Gateway-side prerequisites
The target HistorianGateway OtOpcUa points at **must** run with:
- `RuntimeDb:Enabled=true` — enables the `WriteLiveValues` SQL live path (continuous historization).
- `RuntimeDb:EventReadsEnabled=true` — enables `ReadEvents` from `Runtime.dbo.Events` (alarm-history reads).
- `RuntimeDb:EventReadMaxRows=0` (its default, unlimited) — or larger than the reverse-events walk's biggest read
(`2·MaxTieClusterOverfetch + 4`). The cap truncates **silently**, so a walked-back reverse events read (#691.4) would
lose the newest events of a segment it cut.
- An API key carrying scopes **`historian:read`**, **`historian:write`**, **`historian:tags:write`**.
### Migration note (deployments upgrading from the Wonderware backend)
The `ServerHistorian` section changed shape. Rename the old Wonderware keys and supply the key via env:
| Old (Wonderware) key | New (gateway) key |
|---|---|
| `ServerHistorian:Host` + `:Port` | `ServerHistorian:Endpoint` (`https://host:5222`) |
| `ServerHistorian:SharedSecret` | `ServerHistorian:ApiKey` (**env `ServerHistorian__ApiKey`**) |
| `ServerHistorian:ServerCertThumbprint` | `ServerHistorian:CaCertificatePath` (+ `UseTls` / `AllowUntrustedServerCertificate`) |
The `AlarmHistorian` section's old Wonderware connection keys (`Host`/`Port`/`UseTls`/`ServerCertThumbprint`/`SharedSecret`)
were pruned — remove them; the SQLite knobs are retained and the downstream connection now comes from
`ServerHistorian`. See `docs/Historian.md` for the full guide.
### Live-validation gate — PASSED (was "KNOWN LIMITATION 1")
**No longer a limitation.** The cutover was live-validated against the real gateway on
`wonder-sql-vd03`: read ✅, write-persist ✅, alarm-send ✅, alarm-readback skipped as a *confirmed
protocol limitation* (the captured `CM_EVENT` wire never carries `SourceName`, so the gateway cannot
populate `Source_Object` — not a fixable bug). Evidence:
`docs/plans/2026-06-27-otopcua-historian-followups.md`.
Kept here because it is the recipe to **re-run** the gate after any gateway or backend change (VPN to
`wonder-sql-vd03`, gateway running the prerequisites above):
```bash
export HISTGW_GATEWAY_ENDPOINT=https://wonder-sql-vd03:5222 # absolute gateway URI; absent ⇒ all live tests skip
export HISTGW_GATEWAY_APIKEY=histgw_<id>_<secret> # must carry historian:read + historian:write (+ tags:write) scopes
export HISTGW_TEST_TAG=<existing-tag> # read round-trip
export HISTGW_WRITE_SANDBOX_TAG=<writable-float-tag> # e.g. HistGW.LiveTest.Sandbox — write round-trip (EnsureTags + write)
export HISTGW_BOOL_SANDBOX_TAG=<writable-bool-tag> # e.g. HistGW.LiveTest.BoolSandbox — Boolean→Int2 EnsureTags leg + retype probe (issue #441)
export HISTGW_ALARM_SOURCE=<source-name> # alarm SendEvent → ReadEvents round-trip
dotnet test --filter "Category=LiveIntegration"
```
The live suite **skips cleanly** when these env vars are absent (safe to run offline on macOS). It is the
gate the operator runs on the VPN before trusting the cutover.
### Continuous-historization value capture — wired and live-gated (was "KNOWN LIMITATION 2"; #491 CLOSED 2026-08-04)
The `ContinuousHistorizationRecorder` is fully wired (actor + FasterLog outbox + gateway value-writer +
meters). It is spawned with an **empty *initial* ref set** (`Array.Empty<string>()` in
`WithOtOpcUaRuntimeActors`) because the deployed address space — and thus the historized-ref set — is built
later at deploy time, not at actor-spawn time. **The ref-feed gap is now closed:** on every deploy the
`AddressSpaceApplier.FeedHistorizedRefs` hook computes the added/removed historized-ref delta and posts it
through `ActorHistorizedTagSubscriptionSink` → the recorder's `UpdateHistorizedRefs` message, and the
recorder keeps the full set and re-registers its dependency-mux interest. So the recorder converges to the
deployed set of historized value tags from the first deploy onward (the feed is a non-blocking, deploy-safe
`Tell`; a faulting feed can never break a deploy).
**The live gate RAN 2026-08-01 against a real HistorianGateway + a real AVEVA Historian, and it found a
production-fatal defect** — see `docs/plans/2026-08-01-continuous-historization-live-gate.md`. Three things
to know:
- ⚠️ **`ConfigureAwait(false)` in an Akka handler destroys `Self`.** `OnValueChangedAsync` touched `Self`
after awaiting `IHistorizationOutbox.AppendAsync(...).ConfigureAwait(false)`, which discards Akka's
`ActorTaskScheduler` synchronization context — so `ActorBase.Context` was gone and it threw
`NotSupportedException` on the FIRST value. The throw reached supervision, the actor **restarted**, and
because production seeds an **empty** ref set the restart discarded every historized ref: one value
captured, then silence. Fixed by capturing `IActorRef self = Self` before the await (the idiom the
`GetStatus` handler already used for `Sender`).
- ⚠️ **A fake that completes synchronously hides this entire class of bug**, and 13 tests passed because
of one. `ValueTask.CompletedTask` never yields, so the continuation ran inline with the context intact.
Worse, the obvious fix to the fake — `await Task.Yield()` — is *also* vacuous: `Yield()` posts back to
the captured context, so the caller still resumes on the actor thread. The fake must complete on a
**pool** thread (`await Task.Delay(1).ConfigureAwait(false)`), as a real per-entry fsync does. Only
reverting the production fix exposed that the first regression test proved nothing.
- **Both pair nodes historized the same value** — the recorder had **no Primary gate**, unlike the alarm
store-and-forward sink, so a redundant pair sent duplicate live values per historized tag. Filed as
**#537** and **fixed** — see the paragraph below.
**Gate COMPLETE — #491 CLOSED 2026-08-04.** Outbox append, drain, `WriteLiveValues`,
**restart-convergence**, `EnsureTags` provisioning (`ensured=2 failed=0`) and **read-back** all ✅. The
read-back leg ran against the licensed `wonder-sql-vd03` with the VPN up: two registers carrying
*distinct non-zero* constants (`8224`, `120`) each read back as its own value — a `0`-valued source
would have proved nothing (the #536 trap).
**Both gate findings are now CLOSED and separately live-gated — #537 (Primary gate) and #538 (poison
tag); evidence in the same plan doc. Two things to know before touching either.**
- **#538 — the discriminator is "did another tag write this pass?", NOT a failure count.** `DrainOnceAsync`
writes every per-tag group and acks **all-or-nothing** (deliberate: `RemoveAsync` truncates *through* an
id, so partial acking could drop older unwritten entries). One permanently-failing tag therefore pinned
the FIFO head while every *healthy* tag was re-submitted on every pass, forever — duplicate samples, an
outbox that never truncated (observed 8.3 MB; signature = the gateway's `ok` **and** `error` counters
climbing together). The drain now quarantines a tag after `MaxWriteAttempts` consecutive failures, but
⚠️ **only counts a failure on a pass where some other tag succeeded** — otherwise an ordinary gateway
restart (where everything fails) would discard the entire backlog, converting a duplicate-samples bug
into a lost-history one. A count-only quarantine passes the poison-tag test and fails nothing else, so
the load-bearing test is the **falsification control** (`A_backend_that_fails_every_tag_abandons_nothing`).
Abandonment is also deferred until *every* failing tag is over budget, so nothing is discarded on a pass
that could not have advanced the head anyway.
- **#537 — it gates the CAPTURE, not the drain, and that is the opposite of the alarm sink.** The alarm
queue is in LocalDb and *replicates*, so the Secondary holds the Primary's rows and gating the drain
loses nothing. The historization outbox is a node-local FasterLog replicating to nobody: gating only its
drain leaves the standby appending forever to a queue no one empties (unbounded disk, then a flood of
stale duplicates on promotion). ⚠️ **And the peer that matters is one recording the SAME refs, not one
holding the same rows** — a node stands down only for a Primary in its own per-cluster mesh, and a node
with no cluster-scoped role never stands down at all. Under a fleet-wide election the elected Primary is
usually running another cluster's configuration, and deferring to it ends this cluster's historization
*silently* (the drain reports success on an empty outbox forever) — the same scope mismatch that once
suspended the alarm drain on every node of the rig at once. An unknown role captures. New verdict on
`RedundancyRoleView` behind **`IHistorizationRoleView`**, registered unconditionally on driver nodes
(NOT alongside the alarm `IRedundancyRoleView`, which only exists when that section is enabled — the
#511 registration accident).
- **#672 — an UNREACHABLE recording Primary does not count (crash-failover gap).** A SIGKILLed Primary
stays a member (unreachable, not downed) for 10 s failure detection + 15 s auto-down, and the
redundancy singleton died with it, so the standby's cached snapshot kept naming it Primary and
*nobody* captured for ~30 s. `DriverHostActor` now holds its **own** `ReachabilityEvent` (+`MemberUp`)
subscription — not inside `StartPrimaryVerdictPublishing`, which early-returns with no primary-role
source — handled in **all three** behaviours, caches the last snapshot, and on a reachability change
re-derives and republishes **only** the historization verdict with unreachable peers excluded. Gap ⇒ the
failure-detector window (~10–12 s). Gated by `ContinuousHistorization:CaptureWhilePrimaryUnreachable`
(default `true`). ⚠️ **The alarm drain and `ShouldServiceAsPrimary`/`PublishPrimaryVerdict` must NOT
adopt this** (a false allow there is a dual claimant, not a duplicate) — the tests assert both stay
put. ⚠️ A **removed** member deliberately stays marked unreachable (only `MemberUp` at that address
clears it): un-marking it would re-derive from a cached snapshot still naming it Primary and close the
gate on a dead node. Cost: duplicates while a live Primary is merely unreachable and under a real
partition — idempotent on SqlServer/QuestDB only for source-timestamped drivers (Modbus stamps each
node's own poll time ⇒ duplicate rows on every backend); Gateway stores duplicates.
Re-confirmed live in 2026-06 and **since fixed (#600.1)**: `EnsureTags` provisioning used to fire only for
newly-ADDED historized tags, so flipping an existing tag's `historianTagname` (or historization, or DataType)
provisioned nothing (`2026-06-27-historian-gateway-integration-issues.md` Issue 5). It now also fires for those
changed raw tags.
+19 -22
View File
@@ -1,7 +1,6 @@
# CI for the v2 branch — runs on every push + PR to the v2-akka-fuse / master
# branches. Layered into two jobs:
# build dotnet restore + build (fast feedback on compile errors)
# unit-tests the WHOLE solution minus the env-gated tiers (self-maintaining)
# branches. One job restores and builds once, then tests the WHOLE solution
# minus the env-gated tiers (self-maintaining).
#
# Skips E2E (Category=E2E) and LiveIntegration (needs a real gateway/PLC/VPN) — both
# are env-gated tiers with no hosted-runner fixtures. (The skip-gated *.IntegrationTests
@@ -20,38 +19,35 @@ on:
branches: [v2-akka-fuse, master]
workflow_dispatch: {}
# #707: one active run per repository across push/PR triggers. Gitea 1.26 supports
# workflow concurrency; newer pending runs can supersede an older pending run.
concurrency:
group: lmxopcua-v2-ci
cancel-in-progress: false
env:
DOTNET_NOLOGO: "1"
DOTNET_CLI_TELEMETRY_OPTOUT: "1"
jobs:
build:
unit-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-dotnet@v4
with:
dotnet-version: 10.0.x
# #606: code-reviews/README.md must be regenerated from the per-module findings.md
# files; --check exits non-zero on drift without writing (see REVIEW-PROCESS.md section 5).
# #606: regenerated review summary must remain in sync with module findings.
- name: code-reviews README drift check
run: python3 code-reviews/regen-readme.py --check
- name: TRX skip guard fixtures
run: python3 scripts/ci/test-assert-not-all-skipped.py
# #707: bound project workers and avoid concurrent NuGet restore work.
- name: dotnet restore
run: dotnet restore ZB.MOM.WW.OtOpcUa.slnx
run: dotnet restore ZB.MOM.WW.OtOpcUa.slnx --disable-parallel -m:2
- name: dotnet build
run: dotnet build ZB.MOM.WW.OtOpcUa.slnx --no-restore --configuration Release
unit-tests:
needs: build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-dotnet@v4
with:
dotnet-version: 10.0.x
- name: dotnet restore
run: dotnet restore ZB.MOM.WW.OtOpcUa.slnx
run: dotnet build ZB.MOM.WW.OtOpcUa.slnx --no-restore --configuration Release -m:2
# #606: the runner's size is the context every timing-sensitive failure needs and the log never
# recorded it (load average included: the two Gitea runners may be running jobs side by side).
- name: runner capacity
@@ -69,14 +65,15 @@ jobs:
# are unreachable, so they run harmlessly here (and are counted by the skip-gate below).
- name: dotnet test (whole solution, minus E2E + LiveIntegration)
# #606: Akka TestKit time dilation for the Runtime + ControlPlane actor suites (read by their
# harnesses, tests/Server/*/Harness/TestTimeFactor.cs). This leg runs every test assembly at once
# on the runner, and their 2–8 s presence budgets — sized on an idle dev box — failed in a long
# harnesses, tests/Server/*/Harness/TestTimeFactor.cs). Their 2–8 s presence budgets
# — sized on an idle dev box — failed in a long
# tail, a different test per run. Dilation only stretches waits: a presence wait still fails if
# the message never comes, and absence windows get longer (stricter). Local runs stay at 1.
env:
AKKA_TEST_TIMEFACTOR: "3"
run: >
dotnet test ZB.MOM.WW.OtOpcUa.slnx --configuration Release --no-restore
dotnet test ZB.MOM.WW.OtOpcUa.slnx --configuration Release --no-build
-m:1 -p:TestTfmsInParallel=false --settings scripts/ci/ci.runsettings
--filter "Category!=E2E&Category!=LiveIntegration"
--logger "trx" --results-directory artifacts/trx
# 07/S-2 fail-on-skip: a whole-solution run that executed ZERO tests is a masked outage
+2 -1
View File
@@ -29,7 +29,8 @@ packages/
*.pem
# Claude Code (per-developer settings, runtime lock files, agent transcripts)
.claude/
.claude/*
!.claude/rules/
.local/
+18 -1224
View File
File diff suppressed because it is too large Load Diff
+45 -7
View File
@@ -198,15 +198,53 @@ Timers are **read-only** this phase; a write attempt returns `BadNotWritable`.
### Counter read
**Address**: `C{n}` (e.g. `C0`, `C10`). **DataType must be `Int32`**. The OPC UA node is
a scalar `Int32` whose value is the current **count** (raw word from the PLC).
a scalar `Int32` containing the unsigned **raw big-endian word**, widened to `Int32`:
`(bytes[0] << 8) | bytes[1]`. `S7.Net`'s `Counter.FromByteArray` performs no BCD
reinterpretation; for example, bytes `01 23` surface as `291`. This describes the
current decoder, not verified agreement with a PLC's displayed count.
Counters are **read-only** this phase; a write attempt returns `BadNotWritable`.
> **Known limitation — Counter BCD encoding on S7-300/400**: `S7.Net`'s
> `Counter.FromByteArray` returns the raw big-endian word without BCD decode.
> On **classic S7-300/400** the C-area word is BCD-encoded (0–999), so the
> surfaced value can differ from the actual count on that hardware. S7-1200/1500
> use IEC/DB counters (plain integers) where the raw word is correct. BCD
> reinterpretation for legacy C-area is a live-hardware-gated follow-up.
The S7comm ReadVar **C-area response encoding remains unverified on hardware**.
PLC instruction representations do not by themselves establish network encoding:
Siemens documents a binary current value for [L: Load counter](https://docs.tia.siemens.cloud/r/en-us/v21/stl-s7-300-s7-400/stl-mnemonic-s7-300-s7-400/counter-operations-s7-300-s7-400/l-load-counter-s7-300-s7-400)
and separate `CV` and `CV_BCD` outputs for [S_CD](https://docs.tia.siemens.cloud/r/en-us/v21/lad-s7-300-s7-400/counter-operations-s7-300-s7-400/simatic-counters-s7-300-s7-400/s_cd-assign-parameters-and-count-down-s7-300-s7-400?contentId=reR_xapFX_Ike2opHIvfzg).
IEC counter values stored in DB memory use the **DB address and declared integer
type**, a different read path from `C{n}`. CPU family alone does not select the
counter kind or prove its wire representation. In particular, Siemens also lists
legacy SIMATIC counters for S7-1500 in its [instruction comparison (pages 23–24)](https://cache.industry.siemens.com/dl/files/777/109986777/att_1322640/v1/s7_1500_compare_table_en.pdf?download=true).
The driver has no counter encoding selector; hardware-faithful counter support
requires independent raw-response evidence before any interpretation change.
### Counter evidence follow-up (#616)
The existing `S7_1500SpanReadTests` exercises the python-snap7 profile's synthetic
raw words (`C0 = 4711`, `C7 = 31337`). These are outside the classic SIMATIC count
range `0..999` documented by S_CD. These expectations describe raw-word handling;
they do not establish classic counter count semantics. The recorded device-0 run cannot prove nonzero
numbering. The nonzero test is environment-gated, but its options remain
`CpuType = S71500`, rack `0`, slot `0`: setting `S7_TIMER_NUMBER_ADDRESSING=1`
does not configure a classic CPU or make the profile a classic counter gate.
Before claiming hardware validation or closing #616:
- Prepare a separate target-specific semantic gate with the actual CPU type,
rack, slot and TSAP/connection settings. Keep the synthetic raw-word profile
as its own evidence of the existing decoder contract.
- Record CPU model/order number, firmware, endpoint, connection settings and
driver/S7netplus plus simulator versions where applicable.
- Observe independent PLC-side current counts and binary/BCD reference outputs
for valid classic counts such as `123`, a second distinguishing count (`42`
or `999`), and zero. Record the instruction/counter kind and DB reference
addresses; do not derive reference values from the driver's output.
- Capture the actual two-byte S7comm C-area responses alongside those references.
Do not infer response bytes from a simulator seed or expected BCD value.
- Verify several distinct nonzero timer/counter numbers against their independent
PLC values and captured requests/responses, and verify the read-only contract.
The raw-byte/count and nonzero-numbering gates remain open. Password/protection
support inherited from #303/#632 is a separate follow-up, not evidence supplied
by these counter checks.
### Deferrals (not yet implemented)
+53
View File
@@ -0,0 +1,53 @@
# Whole-solution CI
The `v2-ci` workflow restores, builds and tests in one job so the test step uses
that job's Release binaries. Restore disables parallel package work; restore and
build use at most two MSBuild workers. Testing uses one MSBuild worker, disables
parallel target-framework runs and loads `ci.runsettings`: one VSTest process and
two xUnit collection workers with conservative scheduling. Assemblies that disable
parallelization keep that setting.
The workflow retains the E2E/LiveIntegration exclusions, includes SlowUnit, keeps
Akka's test time factor at three, and requires at least 12,000 executed tests.
Runner capacity and TRX files remain available for diagnosing failures. These
limits reduce contention; they do not prove every timing failure has one cause.
There are no automatic test retries or changed assertion deadlines.
Gitea 1.26 supports the repository-wide `lmxopcua-v2-ci` concurrency group. Only
one run is active; an active run is not cancelled, but a newer pending run may
supersede an older pending run. The group does not serialize other repositories
using the same host.
To reproduce the bounded test leg after restore and Release build:
```bash
AKKA_TEST_TIMEFACTOR=3 dotnet test ZB.MOM.WW.OtOpcUa.slnx \
--configuration Release --no-build -m:1 -p:TestTfmsInParallel=false \
--settings scripts/ci/ci.runsettings \
--filter "Category!=E2E&Category!=LiveIntegration" \
--logger trx --results-directory artifacts/trx
MIN_EXECUTED=12000 bash scripts/ci/assert-not-all-skipped.sh artifacts/trx/*.trx
```
The guard uses the existing TRX `Counters/@executed` value for the aggregate
floor. Its `skipped` diagnostic counts individual `Results/UnitTestResult`
records whose outcome is `NotExecuted`, using the document's XML namespace.
Some VSTest reports set summary `notExecuted` to zero even when these records
are present. Skip records are counted per file, including repeated runs; they
are not a count of unique test methods and do not affect the executed floor.
For malformed XML, the diagnostic retains the old summary-counter fallback;
this guard does not add XML validation to its existing gate policy.
Run the deterministic script fixtures without building or running the solution:
```bash
python3 scripts/ci/test-assert-not-all-skipped.py
```
Verified hosted baseline: [Gitea run 975](https://gitea.dohertylan.com/dohertj2/lmxopcua/actions/runs/975)
completed successfully on 2026-10-13 for commit
`76bdca90e94e814e06ef10b78271e2b492b2ba32` (merged by PR #711).
Restore, Release build, README drift check, the serialized whole-solution test
step, the 12,000-test floor and TRX upload all passed. The floor counted
14,630 executed tests across 62 TRX files. This baseline predates the individual
skip-reporting correction; its printed zero skip diagnostic is not skip evidence.
+20 -1
View File
@@ -33,12 +33,31 @@ for f in "$@"; do
exit 2
fi
seen=$((seen + 1))
# The trx <Counters .../> element in <ResultSummary> carries executed + notExecuted counts.
# Keep the executed-counter floor unchanged; notExecuted is only a fallback
# for malformed XML that this guard historically accepted.
counters="$(grep -oE '<Counters[^>]*/?>' "$f" | head -1)"
ex="$(printf '%s' "$counters" | grep -oE 'executed="[0-9]+"' | grep -oE '[0-9]+' || true)"
ne="$(printf '%s' "$counters" | grep -oE 'notExecuted="[0-9]+"' | grep -oE '[0-9]+' || true)"
ex="${ex:-0}"
ne="${ne:-0}"
# VSTest can emit notExecuted="0" despite individual NotExecuted results.
# Count direct Results/UnitTestResult records in the document's namespace,
# excluding foreign elements, nested records and escaped diagnostic text.
ne="$(python3 - "$f" "$ne" <<'PY'
import sys
import xml.etree.ElementTree as ET
try:
root = ET.parse(sys.argv[1]).getroot()
except ET.ParseError:
# Skip reporting must not introduce a new malformed-XML gate failure.
print(sys.argv[2])
else:
namespace = root.tag.split("}", 1)[0] + "}" if root.tag.startswith("{") else ""
results = root.findall(f"{namespace}Results/{namespace}UnitTestResult")
print(sum(result.get("outcome") == "NotExecuted" for result in results))
PY
)"
total_executed=$((total_executed + ex))
total_skipped=$((total_skipped + ne))
echo " $(basename "$f"): executed=${ex} skipped=${ne}"
+12
View File
@@ -0,0 +1,12 @@
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<!-- #707: project scheduling is also bounded by dotnet test -m:1. -->
<RunConfiguration>
<MaxCpuCount>1</MaxCpuCount>
</RunConfiguration>
<xUnit>
<!-- Preserve each assembly's parallelization opt-out; bound enabled collections. -->
<MaxParallelThreads>2</MaxParallelThreads>
<ParallelAlgorithm>conservative</ParallelAlgorithm>
</xUnit>
</RunSettings>
+108
View File
@@ -0,0 +1,108 @@
#!/usr/bin/env python3
"""Deterministic subprocess checks for the TRX executed-floor guard."""
import os
from pathlib import Path
import subprocess
import tempfile
import unittest
SCRIPT = Path(__file__).with_name("assert-not-all-skipped.sh")
NS = "http://microsoft.com/schemas/VisualStudio/TeamTest/2010"
def trx(executed, outcomes, summary_skipped=0, namespace=NS, extra=""):
declaration = f' xmlns="{namespace}"' if namespace else ""
results = "".join(f'<UnitTestResult outcome="{outcome}"/>' for outcome in outcomes)
return (f'<TestRun{declaration}><Results>{results}{extra}</Results>'
f'<ResultSummary><Counters total="{len(outcomes)}" executed="{executed}" '
f'notExecuted="{summary_skipped}"/></ResultSummary></TestRun>')
class SkipGuardTests(unittest.TestCase):
def setUp(self):
self.directory = tempfile.TemporaryDirectory(prefix="trx-skip-fixtures-")
self.addCleanup(self.directory.cleanup)
self.root = Path(self.directory.name)
def write(self, name, content):
path = self.root / name
path.write_text(content, encoding="utf-8")
return path
def run_guard(self, *paths, minimum=None):
env = os.environ.copy()
env.pop("MIN_EXECUTED", None)
if minimum is not None:
env["MIN_EXECUTED"] = str(minimum)
return subprocess.run(["bash", str(SCRIPT), *map(str, paths)], env=env,
text=True, capture_output=True, check=False)
def assert_report(self, result, status, executed, skipped, files=1):
self.assertEqual(status, result.returncode, result.stdout + result.stderr)
self.assertIn(f"files={files} executed={executed} skipped={skipped} ", result.stdout)
def test_mixed_results_ignore_zero_summary_skip_counter(self):
# A summary-only skip count misses the three actual NotExecuted records.
path = self.write("mixed.trx", trx(827, ["Passed"] * 827 + ["NotExecuted"] * 3))
result = self.run_guard(path)
self.assert_report(result, 0, 827, 3)
self.assertIn("mixed.trx: executed=827 skipped=3", result.stdout)
def test_all_skipped_fails_default_floor_but_report_only_passes(self):
path = self.write("all-skipped.trx", trx(0, ["NotExecuted"] * 188))
self.assert_report(self.run_guard(path), 1, 0, 188)
self.assert_report(self.run_guard(path, minimum=0), 0, 0, 188)
def test_zero_tests_fails_default_floor_but_report_only_passes(self):
path = self.write("zero.trx", trx(0, []))
self.assert_report(self.run_guard(path), 1, 0, 0)
self.assert_report(self.run_guard(path, minimum=0), 0, 0, 0)
def test_agreeing_summary_and_outcomes_are_not_double_counted(self):
path = self.write("agreeing.trx", trx(1, ["Passed", "NotExecuted"], 1))
self.assert_report(self.run_guard(path), 0, 1, 1)
def test_outcomes_are_authoritative_when_summary_overstates_skips(self):
path = self.write("overstated.trx", trx(1, ["Passed", "NotExecuted"], 9))
self.assert_report(self.run_guard(path), 0, 1, 1)
def test_namespace_path_and_exact_outcome_exclude_decoys(self):
extra = ('<Other><UnitTestResult outcome="NotExecuted"/></Other>'
'<foreign:UnitTestResult xmlns:foreign="urn:other" outcome="NotExecuted"/>'
'<Output><Message>&lt;UnitTestResult outcome="NotExecuted"/&gt;</Message></Output>')
content = trx(2, ["Passed", "Failed", "NotExecuted", "notexecuted"], extra=extra)
# Prefixes on Results/UnitTestResult must work just like the default namespace.
content = content.replace('<TestRun ', f'<TestRun xmlns:t="{NS}" ')
content = content.replace('<Results>', '<t:Results>').replace('</Results>', '</t:Results>')
content = content.replace('<UnitTestResult ', '<t:UnitTestResult ')
path = self.write("namespaced.trx", content)
self.assert_report(self.run_guard(path), 0, 2, 1)
def test_unnamespaced_results_remain_supported(self):
path = self.write("legacy.trx", trx(1, ["Passed", "NotExecuted"], namespace=""))
self.assert_report(self.run_guard(path), 0, 1, 1)
def test_executed_counter_controls_aggregate_floor_at_boundary(self):
# Counters retain authority even when result records disagree with executed.
mixed = self.write("floor.trx", trx(12000, ["Passed", "NotExecuted"]))
skipped = self.write("skipped.trx", trx(0, ["NotExecuted"]))
self.assert_report(self.run_guard(mixed, skipped, minimum=12000), 0, 12000, 2, 2)
self.assert_report(self.run_guard(mixed, skipped, minimum=12001), 1, 12000, 2, 2)
def test_missing_arguments_and_file_keep_exit_two(self):
self.assertEqual(2, self.run_guard().returncode)
self.assertEqual(2, self.run_guard(self.root / "missing.trx").returncode)
def test_missing_counters_keeps_existing_exit_one(self):
path = self.write("no-counters.trx", '<TestRun><Results/></TestRun>')
self.assertEqual(1, self.run_guard(path, minimum=0).returncode)
def test_malformed_xml_keeps_summary_fallback_and_gate_status(self):
path = self.write("malformed.trx", '<TestRun><Counters executed="2" notExecuted="3"/>')
self.assert_report(self.run_guard(path, minimum=2), 0, 2, 3)
self.assert_report(self.run_guard(path, minimum=3), 1, 2, 3)
if __name__ == "__main__":
unittest.main(verbosity=2)
@@ -56,7 +56,7 @@ public class ReadCommand : CommandBase
{
// The SDK's Session.ReadValue surfaces a Bad node status (BadNodeIdUnknown,
// BadUserAccessDenied, ...) as an exception. A read command's job is to REPORT that status,
// not crash with a stack trace (#577) — session-level failures still propagate.
// not crash with a stack trace — session-level failures still propagate.
await console.Output.WriteLineAsync($"Node: {NodeId}");
await console.Output.WriteLineAsync($"Value: ");
await console.Output.WriteLineAsync($"Status: {(StatusCode)ex.StatusCode} ({StatusCodes.GetBrowseName(ex.StatusCode)})");
@@ -24,6 +24,7 @@ internal static class DisposeBridge
private static readonly TimeSpan Timeout = TimeSpan.FromSeconds(5);
/// <summary>Runs <paramref name="work" /> to completion (or the timeout), swallowing every failure.</summary>
/// <param name="work">The asynchronous work to run; it receives a token that is cancelled when the timeout elapses.</param>
public static void Run(Func<CancellationToken, Task> work)
{
using var cts = new CancellationTokenSource(Timeout);
@@ -462,7 +462,7 @@ public sealed class OpcUaClientService : IOpcUaClientService
ThrowIfDisposed();
ThrowIfNotConnected();
// The server's ShelvingState child is SDK-created with a numeric namespace-0 id, so a synthesized
// The server's ShelvingState child is SDK-created with a numeric id in namespace 0, so a synthesized
// "<condition>.ShelvingState" node id never exists. The shelve methods are called on the condition
// object itself with the AlarmConditionType_ShelvingState_* method ids, which the server dispatches
// to the condition's ShelvingState methods.
@@ -884,6 +884,8 @@ public sealed class OpcUaClientService : IOpcUaClientService
catch { return false; }
}
/// <summary>Builds the event filter (select clauses for the alarm and condition fields) used by alarm subscriptions.</summary>
/// <returns>The populated <see cref="EventFilter"/>.</returns>
internal static EventFilter CreateAlarmEventFilter()
{
var filter = new EventFilter();
@@ -58,7 +58,7 @@ public sealed class AkkaClusterOptions
/// <see cref="RoleParser.ResolveEffectiveRoles(IReadOnlyList{string}?)"/>. That fallback was
/// described here — and relied upon in the field — for a year before it was implemented; a
/// node configured only through the env var joined the cluster carrying no roles at all.
/// See Gitea #532 and <c>docs/deployments/wonder-app-vd03-makino-z-34184.md</c>.
/// See <c>docs/deployments/wonder-app-vd03-makino-z-34184.md</c>.
/// </para>
/// <para>
/// The two sources normalize differently: values bound from <c>Cluster:Roles</c> are taken
@@ -44,7 +44,7 @@ public sealed class AkkaClusterOptionsValidator : OptionsValidatorBase<AkkaClust
/// DI-constructed by <c>AddValidatedOptions</c> (a plain <c>AddSingleton</c>), so constructor
/// dependencies are safe here. Both <paramref name="effectiveRoles"/> and
/// <paramref name="configuration"/> are optional (default <see langword="null"/>) so a caller
/// with neither to hand — including every test in this file that pre-dates R3-04 — keeps
/// with neither to hand — including every test in this file that pre-dates the post-overlay guard — keeps
/// constructing this validator parameterlessly; the post-overlay guard below (choice 4) is
/// simply skipped when either is null.
/// </summary>
@@ -116,7 +116,7 @@ public sealed class AkkaClusterOptionsValidator : OptionsValidatorBase<AkkaClust
}
/// <summary>
/// R3-04 (choice 4, the residual divergence surface): <c>Program.cs</c> resolves
/// Post-overlay guard (choice 4, the residual divergence surface): <c>Program.cs</c> resolves
/// <see cref="EffectiveRoles"/> BEFORE the per-role appsettings overlay loads (the overlay's
/// filename depends on the answer), so if a role overlay — or a <c>${secret:}</c> expansion —
/// ever set <c>Cluster:Roles</c> itself, that value would silently disagree with the
@@ -46,7 +46,9 @@ public sealed class ClusterBootstrapCoordinator : IHostedService
_log = log ?? throw new ArgumentNullException(nameof(log));
}
/// <inheritdoc/>
/// <summary>Starts the bootstrap join in the background when the bootstrap guard is enabled.</summary>
/// <param name="cancellationToken">Not used; the join runs under the coordinator's own cancellation source.</param>
/// <returns>A completed task; the join itself continues asynchronously.</returns>
public Task StartAsync(CancellationToken cancellationToken)
{
if (!_options.BootstrapGuard.Enabled)
@@ -59,7 +61,9 @@ public sealed class ClusterBootstrapCoordinator : IHostedService
return Task.CompletedTask;
}
/// <inheritdoc/>
/// <summary>Cancels the background join and waits for it to finish.</summary>
/// <param name="cancellationToken">Not used; shutdown is driven by the coordinator's own cancellation source.</param>
/// <returns>A task that represents the asynchronous operation.</returns>
public async Task StopAsync(CancellationToken cancellationToken)
{
await _cts.CancelAsync().ConfigureAwait(false);
@@ -63,7 +63,7 @@ public static class ClusterBootstrapGuard
/// <summary>
/// Analyzes the configured seed list to decide this node's bootstrap role. The guard engages only
/// for the Phase-6 pair shape (exactly two seeds, one of them this node); any other shape
/// for the pair shape (exactly two seeds, one of them this node); any other shape
/// (single-seed legacy site node, three+ seeds, self absent) returns <see cref="BootstrapRole.Applies"/>
/// = false and the caller joins the configured seeds unchanged.
/// </summary>
@@ -23,7 +23,7 @@ public sealed class ClusterRoleInfo : IClusterRoleInfo, IDisposable
private readonly string? _clusterRole;
private readonly string? _clusterId;
private readonly object _lock = new();
// R3-04 (01/C-9, census row 17) — the two dictionaries below have DIFFERENT correct comparers,
// The two dictionaries below have DIFFERENT correct comparers,
// and that split is deliberate, not an oversight (review finding, 2026-08-20).
//
// _membersByRole answers "every member carrying (a spelling of) this role" — OrdinalIgnoreCase,
@@ -53,7 +53,7 @@ public sealed class ClusterRoleInfo : IClusterRoleInfo, IDisposable
// sharing a bind IP) stay distinct. Production hosts have unique DNS names so the port
// suffix is harmless redundancy.
_localNode = CommonsNodeId.Parse($"{options.Value.PublicHostname}:{options.Value.Port}");
// OrdinalIgnoreCase (Gitea #532): Cluster:Roles binds verbatim, so a node authored with 'Driver'
// OrdinalIgnoreCase: Cluster:Roles binds verbatim, so a node authored with 'Driver'
// used to answer HasRole("driver") false while gossiping a role Akka would still match against
// its own ordinal comparison — the node was a driver to the cluster and not to itself.
_localRoles = new HashSet<string>(options.Value.Roles, StringComparer.OrdinalIgnoreCase);
@@ -128,7 +128,7 @@ public sealed class ClusterRoleInfo : IClusterRoleInfo, IDisposable
{
lock (_lock)
{
// Cluster-002 (review R1 m6): seed only USABLE members. A member already Leaving / Exiting / Down
// Seed only USABLE members. A member already Leaving / Exiting / Down
// (or still Joining) at seed time is not one, and a departing member never fires a later
// MemberLeft / MemberExited / MemberDowned that HandleMemberEvent would drop it on — it would
// linger until MemberRemoved, the very delay the member-event handling exists to avoid. A
@@ -165,7 +165,7 @@ public sealed class ClusterRoleInfo : IClusterRoleInfo, IDisposable
set.Add(up.Member);
}
break;
// Cluster-002: a leaving / exiting / downed member is no longer a usable member, so
// A leaving / exiting / downed member is no longer a usable member, so
// drop it as soon as that transition lands rather than waiting for MemberRemoved
// (several seconds later). MembersWithRole now has a production caller (the
// historization scope-peer check), which must not defer to a departing partner.
@@ -210,8 +210,8 @@ public sealed class ClusterRoleInfo : IClusterRoleInfo, IDisposable
// Record the event's own address. The leader used to be resolved by looking the address up
// in Cluster.State.Members, but that snapshot is published asynchronously and can lag the
// event: a miss stored null and nothing re-ran, so RoleLeader(role) stayed null for good
// (Gitea #590). Only the address is ever read, so no Member lookup is needed.
// event: a miss stored null and nothing re-ran, so RoleLeader(role) stayed null for good.
// Only the address is ever read, so no Member lookup is needed.
_roleLeaders[evt.Role] = evt.Leader;
if (evt.Leader is not null)
next = ToNodeId(evt.Leader);
@@ -21,14 +21,18 @@ public sealed class ConfigSourceOptionsValidator : IValidateOptions<ConfigSource
/// DI-constructed by <c>AddValidatedOptions</c> (a plain <c>AddSingleton</c>), so a
/// constructor dependency is safe here. This validator's only use of configuration is this
/// node's roles, so it takes <see cref="EffectiveRoles"/> — the single role authority
/// (archreview R3-04) — directly rather than <see cref="IConfiguration"/>.
/// — directly rather than <see cref="IConfiguration"/>.
/// </summary>
/// <param name="effectiveRoles">The node's effective roles, used to decide which fetch settings are required.</param>
public ConfigSourceOptionsValidator(EffectiveRoles effectiveRoles)
{
_effectiveRoles = effectiveRoles;
}
/// <inheritdoc />
/// <summary>Validates the <see cref="ConfigSourceOptions"/> for this node's roles.</summary>
/// <param name="name">The options instance name (unused).</param>
/// <param name="options">The options to validate.</param>
/// <returns>A success result, or a failure listing every violated rule.</returns>
public ValidateOptionsResult Validate(string? name, ConfigSourceOptions options)
{
ArgumentNullException.ThrowIfNull(options);
@@ -92,8 +96,8 @@ public sealed class ConfigSourceOptionsValidator : IValidateOptions<ConfigSource
// Per-cluster mesh Phase 4: a driver-only node has no central ConfigDb connection to fall back
// on, so Direct is only ever valid on a fused admin+driver node (admin holds the SQL
// connection). EffectiveRoles is the single role authority (archreview R3-04): its precedence
// already applies the OTOPCUA_ROLES fallback (Gitea #532) and its normalization already folds
// connection). EffectiveRoles is the single role authority: its precedence
// already applies the OTOPCUA_ROLES fallback and its normalization already folds
// case, so neither an env-only driver-only node nor one spelled 'Driver' can sail through here
// on Direct to read its configuration from nothing.
var isDriver = _effectiveRoles.HasDriver;
@@ -69,6 +69,7 @@ public sealed class EffectiveRoles
/// <summary>True when <paramref name="role"/> is among <see cref="Gossiped"/>, compared case-insensitively.</summary>
/// <param name="role">The role to test for.</param>
/// <returns><see langword="true"/> when the role is present; otherwise <see langword="false"/>.</returns>
public bool Has(string role) =>
Gossiped.Any(r => string.Equals(r, role, StringComparison.OrdinalIgnoreCase));
@@ -23,7 +23,7 @@ public sealed class MeshTransportOptionsValidator : IValidateOptions<MeshTranspo
/// DI-constructed by <c>AddValidatedOptions</c> (a plain <c>AddSingleton</c>), so constructor
/// dependencies are safe here. <see cref="IConfiguration"/> is still needed for the
/// <c>ConfigServe:GrpcListenPort</c> cross-read (the admin-side obligation below) —
/// <see cref="EffectiveRoles"/> is the single role authority (archreview R3-04) this rule reads
/// <see cref="EffectiveRoles"/> is the single role authority this rule reads
/// roles through instead of re-deriving them.
/// </summary>
/// <param name="configuration">The application configuration, read for <c>ConfigServe:GrpcListenPort</c>.</param>
@@ -34,7 +34,10 @@ public sealed class MeshTransportOptionsValidator : IValidateOptions<MeshTranspo
_effectiveRoles = effectiveRoles;
}
/// <inheritdoc />
/// <summary>Validates the <see cref="MeshTransportOptions"/> for this node's roles.</summary>
/// <param name="name">The options instance name (unused).</param>
/// <param name="options">The options to validate.</param>
/// <returns>A success result, or a failure listing every violated rule.</returns>
public ValidateOptionsResult Validate(string? name, MeshTransportOptions options)
{
ArgumentNullException.ThrowIfNull(options);
@@ -71,7 +74,7 @@ public sealed class MeshTransportOptionsValidator : IValidateOptions<MeshTranspo
// dials nodes rather than being dialled, and resolves per-cluster keys at runtime through
// MeshTransportOptions.ClusterPsks / the Secrets store, so requiring a single GrpcPsk and an
// endpoint list of it would demand configuration it never reads. EffectiveRoles is the single
// role authority (archreview R3-04) — its precedence already applies the OTOPCUA_ROLES
// role authority — its precedence already applies the OTOPCUA_ROLES
// fallback and its normalization already folds case, so a node configured only through the env
// var, or spelled 'Driver', is still the driver that has to dial central.
var isDriver = _effectiveRoles.HasDriver;
@@ -24,7 +24,7 @@ public static class RoleParser
/// <summary>Returns true when <paramref name="role"/> is a cluster-scoped role — starts with
/// <see cref="ClusterRolePrefix"/> and carries a non-empty ClusterId suffix.</summary>
/// <remarks>
/// The prefix test is <b>case-insensitive</b> (Gitea #532). It was ordinal, which meant a
/// The prefix test is <b>case-insensitive</b>. It was ordinal, which meant a
/// <c>Cluster:Roles</c> entry spelled <c>Cluster-SITE-A</c> — that path binds verbatim, unlike
/// the <see cref="Parse"/> path, which lower-cases — classified as an ordinary role. The node
/// then resolved a <see langword="null"/> ClusterId while still gossiping a cluster-scoped role,
@@ -33,6 +33,7 @@ public static class RoleParser
/// <c>ClusterRoleInfo</c> did not, and got the null.
/// </remarks>
/// <param name="role">The role name to test.</param>
/// <returns><see langword="true"/> if the role is a cluster-scoped role.</returns>
public static bool IsClusterRole(string role) =>
role is not null
&& role.StartsWith(ClusterRolePrefix, StringComparison.OrdinalIgnoreCase)
@@ -47,9 +48,10 @@ public static class RoleParser
/// <c>OTOPCUA_ROLES</c> path (⇒ <c>site-a</c>) — and normalizing one side would silently rename
/// the Akka role of a running deployment (docker-dev's rig configures <c>cluster-MAIN</c> /
/// <c>cluster-SITE-A</c> verbatim today). Every consumer of a ClusterId therefore compares it
/// case-insensitively; see the identity rule on <c>AclScopeRef</c> (Gitea #533).
/// case-insensitively; see the identity rule on <c>AclScopeRef</c>.
/// </remarks>
/// <param name="role">The cluster-scoped role name.</param>
/// <returns>The ClusterId portion of the role, with the role prefix removed.</returns>
public static string ClusterIdFromRole(string role) => role[ClusterRolePrefix.Length..];
/// <summary>
@@ -72,7 +74,7 @@ public static class RoleParser
/// cluster-scoped role (a legacy / single-mesh node).
/// </summary>
/// <remarks>
/// The single home for a derivation that was open-coded at four call sites before Gitea #532
/// The single home for a derivation that was open-coded at four call sites before
/// (<c>ClusterRoleInfo</c>, the ControlPlane singleton scope, <c>SplitTopologyTransportValidator</c>
/// and <c>ConfigDbApiKeyStore</c>) — each with its own view of whether the classification was
/// case-sensitive, which is how the mixed-case gap above survived in three of the four.
@@ -91,7 +93,7 @@ public static class RoleParser
/// </summary>
/// <remarks>
/// <para>
/// ⚠️ <b>This fallback was documented for a year before it existed</b> (Gitea #532).
/// ⚠️ <b>This fallback was documented for a year before it existed</b>.
/// <c>AkkaClusterOptions.Roles</c>' own XML doc and <see cref="SplitTopologyTransportValidator"/>
/// both described it; only the validator implemented it, privately. A node configured solely
/// through <c>OTOPCUA_ROLES</c> — which drives every bit of <c>Program.cs</c>'s conditional
@@ -114,7 +116,7 @@ public static class RoleParser
/// <exception cref="ArgumentException">Either source names an unknown role.</exception>
/// <remarks>
/// Thin delegation to <see cref="EffectiveRoles.Resolve(IReadOnlyList{string}?, string?)"/> —
/// the single role authority (archreview R3-04). <see cref="EffectiveRoles"/> is the type to
/// the single role authority. <see cref="EffectiveRoles"/> is the type to
/// reach for directly; this overload is retained so existing callers of the array-returning
/// shape keep compiling.
/// </remarks>
@@ -35,7 +35,7 @@ public static class ServiceCollectionExtensions
services.AddValidatedOptions<AkkaClusterOptions, AkkaClusterOptionsValidator>(
configuration, AkkaClusterOptions.SectionName);
// R3-04 (structural identity, choice 3): EffectiveRoles is resolved ONCE and registered as a
// Structural identity: EffectiveRoles is resolved ONCE and registered as a
// singleton, off the SAME IConfiguration this call received — not off a DI-resolved
// IConfiguration, which a plain ServiceCollection (as opposed to a WebApplicationBuilder) never
// registers. Every consumer (Program.cs's hasAdmin/hasDriver, the four role-scoped validators,
@@ -54,7 +54,7 @@ public static class ServiceCollectionExtensions
// otherwise; TryAddSingleton is a no-op on a real host, where it is already registered.
services.TryAddSingleton(configuration);
// Gitea #532 (superseded by R3-04): apply the OTOPCUA_ROLES fallback the Roles property has
// Apply the OTOPCUA_ROLES fallback the Roles property has
// always DOCUMENTED, in the one place every consumer reads through. Before this, a node
// configured only through the env var — the knob that drives all of Program.cs's role wiring —
// bound an EMPTY Roles array and so joined the cluster role-less: invisible to
@@ -14,7 +14,7 @@ namespace ZB.MOM.WW.OtOpcUa.Cluster;
/// Phase 2/3/5 dark switches keep their DPS branches alive, but DPS is mesh-wide — it delivers
/// nothing between a central node and a site node that no longer share a gossip ring. The
/// marker of the split topology is a <c>cluster-{ClusterId}</c> role
/// (<see cref="RoleParser.IsClusterRole"/>); those roles exist only post-Phase-6. A node that
/// (<see cref="RoleParser.IsClusterRole"/>); those roles exist only in the per-cluster mesh topology. A node that
/// carries one <b>must</b> use the mesh-crossing transports, and this validator refuses to
/// boot it otherwise — the alternative is a node that runs, listens, and silently delivers no
/// commands or telemetry, which has no stack trace and no failing node to point at.
@@ -44,16 +44,21 @@ public sealed class SplitTopologyTransportValidator : IValidateOptions<MeshTrans
/// transport-mode cross-reads (<c>MeshTransport:Mode</c>, <c>Telemetry:Mode</c>,
/// <c>TelemetryDial:Mode</c>) — not <c>IClusterRoleInfo</c>, whose implementation needs the
/// <c>ActorSystem</c>, which does not exist yet at <c>ValidateOnStart</c> time.
/// <see cref="EffectiveRoles"/> is the single role authority (archreview R3-04) — this rule no
/// <see cref="EffectiveRoles"/> is the single role authority — this rule no
/// longer re-derives the effective, normalized role list itself.
/// </summary>
/// <param name="configuration">The host configuration used to cross-read the transport-mode settings.</param>
/// <param name="effectiveRoles">The node's effective, normalized role list.</param>
public SplitTopologyTransportValidator(IConfiguration configuration, EffectiveRoles effectiveRoles)
{
_configuration = configuration;
_effectiveRoles = effectiveRoles;
}
/// <inheritdoc />
/// <summary>Fails startup when a split-topology node is left on a transport mode that cannot work there.</summary>
/// <param name="name">The options instance name; unused.</param>
/// <param name="options">The mesh transport options; unused because every input is cross-read from configuration.</param>
/// <returns>A success result, or a failure naming the offending transport setting.</returns>
public ValidateOptionsResult Validate(string? name, MeshTransportOptions options)
{
// The options instance is intentionally unused: every value this rule depends on is cross-read
@@ -93,14 +93,18 @@ public sealed class TelemetryOptionsValidator : IValidateOptions<TelemetryOption
/// DI-constructed by <c>AddValidatedOptions</c> (a plain <c>AddSingleton</c>), so a
/// constructor dependency is safe here. This validator's only use of configuration is this
/// node's roles, so it takes <see cref="EffectiveRoles"/> — the single role authority
/// (archreview R3-04) — directly rather than <see cref="IConfiguration"/>.
/// — directly rather than <see cref="IConfiguration"/>.
/// </summary>
/// <param name="effectiveRoles">The node's resolved roles, the single role authority.</param>
public TelemetryOptionsValidator(EffectiveRoles effectiveRoles)
{
_effectiveRoles = effectiveRoles;
}
/// <inheritdoc />
/// <summary>Validates the telemetry options against this node's roles.</summary>
/// <param name="name">The named options instance, or <see langword="null"/> for the default.</param>
/// <param name="options">The options to validate.</param>
/// <returns>Success, or a failure listing every invalid setting.</returns>
public ValidateOptionsResult Validate(string? name, TelemetryOptions options)
{
ArgumentNullException.ThrowIfNull(options);
@@ -117,8 +121,8 @@ public sealed class TelemetryOptionsValidator : IValidateOptions<TelemetryOption
+ $"Expected '{TelemetryOptions.ModeDps}' or '{TelemetryOptions.ModeGrpc}'.");
}
// EffectiveRoles is the single role authority (archreview R3-04) — its precedence already
// applies the OTOPCUA_ROLES fallback (Gitea #532) and its normalization already folds case, so
// EffectiveRoles is the single role authority — its precedence already
// applies the OTOPCUA_ROLES fallback and its normalization already folds case, so
// a node configured only through the env var, or spelled 'Driver', still satisfies the
// Grpc-mode requirements below.
var isDriver = _effectiveRoles.HasDriver;
@@ -157,7 +161,10 @@ public sealed class TelemetryOptionsValidator : IValidateOptions<TelemetryOption
/// </summary>
public sealed class TelemetryDialOptionsValidator : IValidateOptions<TelemetryDialOptions>
{
/// <inheritdoc />
/// <summary>Validates the telemetry dial options.</summary>
/// <param name="name">The named options instance, or <see langword="null"/> for the default.</param>
/// <param name="options">The options to validate.</param>
/// <returns>Success, or a failure listing every invalid setting.</returns>
public ValidateOptionsResult Validate(string? name, TelemetryDialOptions options)
{
ArgumentNullException.ThrowIfNull(options);
@@ -56,7 +56,8 @@ public sealed class CapturedTreeBrowseSession(CapturedTree tree) : IBrowseSessio
return Task.FromResult<IReadOnlyList<AttributeInfo>>(attrs);
}
/// <inheritdoc />
/// <summary>Drops this session's reference to the captured tree; the tree itself may be shared.</summary>
/// <returns>A completed <see cref="ValueTask"/>.</returns>
public ValueTask DisposeAsync()
{
_tree = null; // drop this session's reference only — tree may be shared (coalescing)
@@ -68,7 +69,7 @@ public sealed class CapturedTreeBrowseSession(CapturedTree tree) : IBrowseSessio
/// <summary>
/// Surfaces a leaf's driver-native type name as the <c>dataType</c> address field, so a
/// browse-commit can stamp the driver-typed <c>TagConfig</c> with the type the controller
/// actually reported (Gitea #536).
/// actually reported.
/// </summary>
/// <param name="a">The captured attribute.</param>
/// <returns>The address fields, or null when the driver reported no native type.</returns>
@@ -80,7 +81,7 @@ public sealed class CapturedTreeBrowseSession(CapturedTree tree) : IBrowseSessio
/// reverse map would bind a 16-bit INT as a 32-bit DINT and read four bytes for two.
/// <para>
/// Null when the driver stamped nothing — the mapper then leaves the model's own default in
/// place, which is exactly the pre-#536 behaviour rather than a guess.
/// place, which is exactly the previous behaviour rather than a guess.
/// </para>
/// </remarks>
private static IReadOnlyDictionary<string, string>? NativeTypeFields(DriverAttributeInfo a)
@@ -17,6 +17,8 @@ public sealed class CapturingAddressSpaceBuilder : IAddressSpaceBuilder
public bool Truncated;
public readonly int NodeCap = nodeCap;
public readonly Dictionary<string, CapturedNode> ById = new(StringComparer.Ordinal);
/// <summary>Reserves a slot for one more node, marking the capture truncated when the cap is reached.</summary>
/// <returns><see langword="true"/> when the node may be recorded; <see langword="false"/> when the cap is reached.</returns>
public bool TryReserve()
{
if (Count >= NodeCap) { Truncated = true; return false; }
@@ -96,7 +98,10 @@ public sealed class CapturingAddressSpaceBuilder : IAddressSpaceBuilder
private sealed class CapturedVariableHandle(CapturedNode node) : IVariableHandle
{
/// <inheritdoc />
public string FullReference => node.Id;
/// <inheritdoc />
public IAlarmConditionSink MarkAsAlarmCondition(AlarmConditionInfo info)
{
node.IsAlarmMarked = true;
@@ -107,6 +112,7 @@ public sealed class CapturingAddressSpaceBuilder : IAddressSpaceBuilder
private sealed class NoopAlarmSink : IAlarmConditionSink
{
public static readonly NoopAlarmSink Instance = new();
/// <inheritdoc />
public void OnTransition(AlarmEventArgs args) { } // browse never delivers live transitions
}
}
@@ -12,7 +12,7 @@ public sealed class DiscoveryDriverBrowser : IUniversalDriverBrowser
/// 20 s per-call browse timeout, which only covers serving from memory.</summary>
public static readonly TimeSpan OpenTimeout = TimeSpan.FromSeconds(60);
/// <summary>Bound on the cleanup ShutdownAsync (R2-01: no unbounded waits — a driver wedged
/// <summary>Bound on the cleanup ShutdownAsync (no unbounded waits — a driver wedged
/// in discovery may be equally wedged in shutdown).</summary>
public static readonly TimeSpan ShutdownTimeout = TimeSpan.FromSeconds(10);
@@ -46,7 +46,11 @@ public sealed class DiscoveryDriverBrowser : IUniversalDriverBrowser
public DiscoveryDriverBrowser(IDriverFactory driverFactory, ILogger<DiscoveryDriverBrowser> logger)
: this(driverFactory, logger, OpenTimeout, ShutdownTimeout) { }
// Test seam: short timeouts.
/// <summary>Test seam: constructor with explicit open and shutdown bounds.</summary>
/// <param name="driverFactory">Factory used to construct throwaway/capture driver instances.</param>
/// <param name="logger">Logger for shutdown-timeout warnings.</param>
/// <param name="openTimeout">Upper bound on opening a capture driver and running discovery.</param>
/// <param name="shutdownTimeout">Upper bound on the cleanup shutdown of a capture driver.</param>
internal DiscoveryDriverBrowser(IDriverFactory driverFactory, ILogger<DiscoveryDriverBrowser> logger,
TimeSpan openTimeout, TimeSpan shutdownTimeout)
{
@@ -111,6 +115,10 @@ public sealed class DiscoveryDriverBrowser : IUniversalDriverBrowser
}
}
/// <summary>Builds the in-flight de-duplication key for a capture request.</summary>
/// <param name="driverType">The driver type name.</param>
/// <param name="configJson">The driver configuration JSON.</param>
/// <returns>A key combining the lower-cased driver type and a hash of the configuration.</returns>
internal static string CaptureKey(string driverType, string configJson) =>
driverType.ToLowerInvariant() + ":" +
Convert.ToHexString(System.Security.Cryptography.SHA256.HashData(
@@ -195,6 +203,10 @@ public sealed class DiscoveryDriverBrowser : IUniversalDriverBrowser
}
}
/// <summary>Applies the driver-specific browse-time patch (if any) to the configuration JSON.</summary>
/// <param name="driverType">The driver type name.</param>
/// <param name="configJson">The driver configuration JSON.</param>
/// <returns>The patched configuration JSON, or the input unchanged when the driver type has no patch.</returns>
internal static string PatchForBrowse(string driverType, string configJson)
{
if (!BrowsePatches.TryGetValue(driverType, out var patch)) return configJson;
@@ -13,7 +13,7 @@ public enum ApplyAckOutcome { Applied, Failed }
/// <param name="FailureReason">Why the apply failed; null on <see cref="ApplyAckOutcome.Applied"/>.</param>
/// <param name="CorrelationId">The dispatch's correlation id.</param>
/// <param name="Degraded">
/// Gitea #601: <see langword="true"/> when the node applied the deployment (the revision advanced) but
/// <see langword="true"/> when the node applied the deployment (the revision advanced) but
/// its OPC UA address-space apply was DEGRADED — swallowed per-node sink faults, a failed rebuild, or no
/// outcome reported in time — so the served address space may be partial. Only meaningful with
/// <see cref="ApplyAckOutcome.Applied"/>. Trailing and defaulted so an ack from a node built before
@@ -30,7 +30,7 @@ public sealed record ApplyAck(
string? DegradedReason = null)
{
/// <summary>
/// Gitea #601 — prefix of <c>NodeDeploymentState.FailureReason</c> on an <b>Applied</b> row whose
/// The prefix of <c>NodeDeploymentState.FailureReason</c> on an <b>Applied</b> row whose
/// address-space apply was degraded. Shared by central (which writes it from the ack) and a node with
/// a ConfigDb (which writes its own row), so both spell the row identically.
/// </summary>
@@ -11,11 +11,11 @@ public static class DeployTimings
/// (<c>ConfigPublishCoordinator.DefaultApplyDeadline</c>).</summary>
public static readonly TimeSpan ApplyDeadline = TimeSpan.FromMinutes(2);
/// <summary>Gitea #601 — how far inside <see cref="ApplyDeadline"/> a node's held Applied ack is released
/// <summary>How far inside <see cref="ApplyDeadline"/> a node's held Applied ack is released
/// when the address-space outcome has not arrived, leaving room for transport latency.</summary>
public static readonly TimeSpan ApplyOutcomeSafetyMargin = TimeSpan.FromSeconds(15);
/// <summary>Gitea #601 — the node's default apply-outcome timeout: <see cref="ApplyDeadline"/> −
/// <summary>The node's default apply-outcome timeout: <see cref="ApplyDeadline"/> −
/// <see cref="ApplyOutcomeSafetyMargin"/> (105 s). Long enough that a slow-but-clean rebuild of a large
/// address space is not labelled degraded, short enough to ack before central gives up.</summary>
/// <remarks>The timer is armed after the artifact read and driver reconcile, so if those alone took
@@ -38,7 +38,7 @@ namespace ZB.MOM.WW.OtOpcUa.Commons.Messages.Drivers;
/// so a driver-only node has no ConfigDb connection to write rows to. This channel already reaches
/// <c>/hosts</c>, already survives the mesh split via the Phase 5 gRPC telemetry stream, and already
/// replays a last-value snapshot on every (re)subscribe — so per-host state re-primes after a
/// reconnect without a durable store. The table was dropped; see Gitea #521.</para>
/// reconnect without a durable store. The table was dropped.</para>
/// <para><b>Value-equality caveat.</b> A record's generated <c>Equals</c> compares this list by
/// REFERENCE, so two <c>DriverHealthChanged</c> carrying equal-but-distinct lists are not equal.
/// Nothing dedups on record equality — <c>DriverInstanceActor</c> keeps its own flattened fingerprint
@@ -48,13 +48,13 @@ namespace ZB.MOM.WW.OtOpcUa.Commons.Messages.Drivers;
/// What this driver skipped when it last built its tag table, from
/// <c>ISkippedTagReporter.GetSkippedTags()</c>; null when the driver does not implement that
/// capability. All twelve production drivers do.
/// <para><b>Why this is on the health snapshot (Gitea #541).</b> A skip happens at init, is not an
/// <para><b>Why this is on the health snapshot.</b> A skip happens at init, is not an
/// error, and does not stop the driver's OTHER tags from reading — so <c>State</c> stays Healthy,
/// <c>ErrorCount5Min</c> stays 0, <c>LastError</c> stays null and <c>LastSuccessfulReadUtc</c> keeps
/// advancing while the driver ignores part or all of its configuration. On the Makino in Gitea #540
/// advancing while the driver ignores part or all of its configuration. On the Makino
/// that combination held while 20 of 22 tags were ignored. This field is what contradicts it.</para>
/// <para><b>Additive, never a state change.</b> See <c>ISkippedTagReporter</c> for why a skipped tag
/// must not latch the driver non-Healthy — it would satisfy both halves of the #527 comms-loss
/// must not latch the driver non-Healthy — it would satisfy both halves of the comms-loss
/// conjunction and reconnect-loop a healthy device.</para>
/// <para><b>Value-equality caveat</b> — the same one <paramref name="HostStatuses"/> carries: the
/// record's generated <c>Equals</c> compares the sample list by REFERENCE.</para>
@@ -62,7 +62,7 @@ namespace ZB.MOM.WW.OtOpcUa.Commons.Messages.Drivers;
/// <param name="Diagnostics">
/// Operator-facing driver diagnostics from <c>IDriverDiagnostics.GetDiagnostics()</c> (e.g. S7's
/// negotiated PDU size), bounded and in ordinal key order; null when the driver does not implement the
/// capability (Gitea #640). Additive, never a state change. Same by-reference value-equality caveat as
/// capability. Additive, never a state change. Same by-reference value-equality caveat as
/// <paramref name="HostStatuses"/>.
/// </param>
public sealed record DriverHealthChanged(
@@ -216,7 +216,7 @@ public static class MeshTransportDtoMapper
if (message.FailureReason is not null)
proto.FailureReason = message.FailureReason;
// #601 — additive fields; a clean ack leaves both unset, byte-identical to a pre-#601 encode.
// Additive fields; a clean ack leaves both unset, byte-identical to an encode that predates these fields.
proto.Degraded = message.Degraded;
if (message.DegradedReason is not null)
proto.DegradedReason = message.DegradedReason;
@@ -255,7 +255,7 @@ public static class MeshTransportDtoMapper
outcome,
proto.HasFailureReason ? proto.FailureReason : null,
CorrelationId.Parse(proto.CorrelationId),
// #601 — absent on an old node's ack, which decodes as false/null: today's clean ack.
// Absent on an old node's ack, which decodes as false/null: today's clean ack.
Degraded: proto.Degraded,
DegradedReason: proto.HasDegradedReason ? proto.DegradedReason : null);
}
@@ -77,7 +77,7 @@ public static class OtOpcUaTelemetry
public const string PollOverrunDriverInstanceTag = "driver_instance";
/// <summary>
/// Session/publish-level events of an OPC UA Client driver instance (#276): the operator's view of a
/// Session/publish-level events of an OPC UA Client driver instance: the operator's view of a
/// flapping upstream. Tagged <see cref="PollOverrunDriverInstanceTag"/> (<c>driver_instance</c>) +
/// <c>event</c> = <c>publish_response|notification|data_change|bad_status_value|publish_error|
/// keepalive_miss|keepalive_recovered|stale_notification_dropped</c>.
@@ -19,15 +19,15 @@ namespace ZB.MOM.WW.OtOpcUa.Commons.OpcUa;
/// <param name="StatusCode">
/// Status code of the condition's source data (Part 9 <c>ConditionType.Quality</c>). Carried so a
/// comms-lost native source can report a non-<c>Good</c> condition instead of the accidentally-Good
/// default (issue #477). It is a pure annotation — it never alters Active/Acked/Retain. Defaults to
/// default. It is a pure annotation — it never alters Active/Acked/Retain. Defaults to
/// <see cref="OpcUaStatus.Good"/> so scripted-alarm callers (which stay Good in v1) need not supply it.
/// Carries the full sub-code since #519, so a comms-lost condition reports
/// Carries the full sub-code, so a comms-lost condition reports
/// <see cref="OpcUaStatus.BadNoCommunication"/> rather than a bare <see cref="OpcUaStatus.Bad"/>.
/// </param>
/// <param name="UnshelveAtUtc">
/// For <see cref="AlarmShelvingKind.Timed"/>, the UTC instant the engine expires the shelve; otherwise
/// <c>null</c>. Carried so the node's Part 9 <c>ShelvingState.UnshelveTime</c> reports the real remaining
/// time instead of the SDK's ~24.8-day ceiling (Gitea #597). A timed snapshot without it (a caller that
/// time instead of the SDK's ~24.8-day ceiling. A timed snapshot without it (a caller that
/// predates the field) falls back to that ceiling — the engine still owns the expiry either way.
/// </param>
public sealed record AlarmConditionSnapshot(
@@ -44,6 +44,7 @@ public sealed class DeferredAddressSpaceSink : IOpcUaAddressSpaceSink, ISurgical
public void WriteAlarmCondition(string alarmNodeId, AlarmConditionSnapshot state, DateTime sourceTimestampUtc, AddressSpaceRealm realm)
=> _inner.WriteAlarmCondition(alarmNodeId, state, sourceTimestampUtc, realm);
/// <inheritdoc />
public void WriteAlarmQuality(string alarmNodeId, uint statusCode, DateTime sourceTimestampUtc, AddressSpaceRealm realm)
=> _inner.WriteAlarmQuality(alarmNodeId, statusCode, sourceTimestampUtc, realm);
@@ -52,7 +53,7 @@ public sealed class DeferredAddressSpaceSink : IOpcUaAddressSpaceSink, ISurgical
=> _inner.MaterialiseAlarmCondition(alarmNodeId, equipmentNodeId, displayName, alarmType, severity, realm, isNative);
/// <inheritdoc />
// Forward the WP4 multi-notifier wiring to the inner sink. Without this the native-alarm fan-out to the
// Forward the multi-notifier wiring to the inner sink. Without this the native-alarm fan-out to the
// referencing equipment folders ships INERT on every driver-role host (actors inject THIS wrapper, not the
// inner SdkAddressSpaceSink) — the F10b / PR#423 forwarding trap the reflection guard exists to catch.
public void WireAlarmNotifiers(string alarmNodeId, AddressSpaceRealm alarmRealm, IReadOnlyList<string> notifierFolderNodeIds, AddressSpaceRealm notifierFolderRealm)
@@ -100,7 +101,7 @@ public sealed class DeferredAddressSpaceSink : IOpcUaAddressSpaceSink, ISurgical
=> _inner is ISurgicalAddressSpaceSink surgical
&& surgical.UpdateFolderDisplayName(folderNodeId, displayName, realm);
// R2-07 T7 — surgical remove forwards. Same capability-sniffing pattern as UpdateTagAttributes /
// Surgical remove forwards. Same capability-sniffing pattern as UpdateTagAttributes /
// UpdateFolderDisplayName: forward when the inner sink supports the surgical capability; return false
// otherwise (before the real SdkAddressSpaceSink is swapped in, or any non-surgical inner) so the caller
// (AddressSpaceApplier) falls back to a full rebuild. Without these forwards the surgical remove path is
@@ -1,21 +1,38 @@
namespace ZB.MOM.WW.OtOpcUa.Commons.OpcUa;
/// <summary>
/// Late-binding adapter that holds an inner <see cref="IServiceLevelPublisher"/> reference
/// swappable at runtime. Mirrors <see cref="DeferredAddressSpaceSink"/>: Akka actors resolve
/// the publisher at DI time, but the production <c>SdkServiceLevelPublisher</c> only exists
/// after <c>StandardServer.Start</c>. The Host's hosted service swaps the inner once the SDK
/// is up; until then writes route through <see cref="NullServiceLevelPublisher"/>.
/// Late-binding adapter that accepts and retains the latest requested ServiceLevel until
/// the SDK publisher is available. Binding synchronously replays that request, including zero.
/// </summary>
/// <remarks>
/// Publishing and binding are serialized through the synchronous inner call. Inner publishers
/// must not call back into this adapter. Acceptance while unbound is not an SDK write confirmation.
/// A failed call retains both the requested level and the installed publisher for later recovery.
/// </remarks>
public sealed class DeferredServiceLevelPublisher : IServiceLevelPublisher
{
private volatile IServiceLevelPublisher _inner = NullServiceLevelPublisher.Instance;
private readonly object _gate = new();
private IServiceLevelPublisher? _inner;
private byte? _latest;
/// <summary>Swap the underlying publisher. Pass null to revert to the Null no-op.</summary>
/// <param name="inner">The publisher implementation to use, or null to use the null publisher.</param>
public void SetInner(IServiceLevelPublisher? inner) =>
_inner = inner ?? NullServiceLevelPublisher.Instance;
/// <summary>Bind and replay the latest request, or unbind while retaining that request.</summary>
/// <param name="inner">The publisher implementation to use, or null to unbind.</param>
public void SetInner(IServiceLevelPublisher? inner)
{
lock (_gate)
{
_inner = inner;
if (_latest is { } latest) _inner?.Publish(latest);
}
}
/// <inheritdoc />
public void Publish(byte serviceLevel) => _inner.Publish(serviceLevel);
public void Publish(byte serviceLevel)
{
lock (_gate)
{
_latest = serviceLevel;
_inner?.Publish(serviceLevel);
}
}
}
@@ -11,14 +11,14 @@ public interface IOpcUaAddressSpaceSink
/// <summary>Write a Variable node's current value + status code + source timestamp.</summary>
/// <param name="nodeId">The OPC UA node ID of the variable.</param>
/// <param name="value">The value to write.</param>
/// <param name="statusCode">The full OPC UA status code, sub-code included. Passed to the SDK verbatim
/// (#519): a driver's <see cref="OpcUaStatus.BadNoCommunication"/> reaches the client as
/// <param name="statusCode">The full OPC UA status code, sub-code included. Passed to the SDK verbatim, so
/// a driver's <see cref="OpcUaStatus.BadNoCommunication"/> reaches the client as
/// <c>BadNoCommunication</c>, not as a generic <see cref="OpcUaStatus.Bad"/>. Use
/// <see cref="OpcUaStatus"/>' named constants rather than a literal.</param>
/// <param name="sourceTimestampUtc">The source timestamp in UTC.</param>
/// <param name="realm">Which of the two v3 namespaces (<see cref="AddressSpaceRealm.Raw"/> device tree or
/// <see cref="AddressSpaceRealm.Uns"/> equipment tree) the <paramref name="nodeId"/> lives in — the sink
/// resolves the namespace index from the realm rather than parsing it out of the id string. WP3's fan-out
/// resolves the namespace index from the realm rather than parsing it out of the id string. The fan-out
/// calls this once per NodeId (raw + every referencing UNS node) with the matching realm.</param>
void WriteValue(string nodeId, object? value, uint statusCode, DateTime sourceTimestampUtc, AddressSpaceRealm realm);
@@ -81,11 +81,11 @@ public interface IOpcUaAddressSpaceSink
/// the condition was materialised under).</param>
void WriteAlarmCondition(string alarmNodeId, AlarmConditionSnapshot state, DateTime sourceTimestampUtc, AddressSpaceRealm realm);
/// <summary>#477 — annotate a materialised condition's source-data quality OUT OF BAND from any alarm
/// <summary>Annotate a materialised condition's source-data quality OUT OF BAND from any alarm
/// transition (used by the driver-connectivity path: comms lost → <see cref="OpcUaStatus.BadNoCommunication"/>,
/// restored → <see cref="OpcUaStatus.Good"/>). Sets ONLY the condition's Quality — never
/// Active/Acked/Severity/Retain (a comms-lost active alarm stays active) — and fires one Part 9 event
/// only on a change to the status code itself — since #519 that includes a move between two Bad
/// only on a change to the status code itself — that now includes a move between two Bad
/// sub-codes, which genuinely changes what a subscriber reads. A no-op for an unmaterialised /
/// non-condition node.</summary>
/// <param name="alarmNodeId">The condition node id (RawPath for a native alarm).</param>
@@ -172,9 +172,9 @@ public interface IOpcUaAddressSpaceSink
/// <param name="realm">The namespace realm the variable (and its <paramref name="parentFolderNodeId"/>) live
/// in. A historized UNS reference node registers the SAME <paramref name="historianTagname"/> as its backing
/// raw node — both NodeIds map to one tagname — so this call carries the shared tagname per realm.</param>
/// <param name="refuseArrayOnScalar">Gitea #610 review (R6): when true and the node is SCALAR, a client write
/// <param name="refuseArrayOnScalar">When true and the node is SCALAR, a client write
/// of an array value is refused inline (<c>BadTypeMismatch</c>, nothing dispatched) because the owning
/// driver's scalar encoder cannot take one (Modbus / S7). False (default) keeps the pre-#610 behaviour — the
/// driver's scalar encoder cannot take one (Modbus / S7). False (default) keeps the earlier behaviour — the
/// write is dispatched and the driver decides (OpcUaClient forwards it to the upstream server, Galaxy encodes
/// it). Fixed at creation; an existing node keeps the value it was created with.</param>
void EnsureVariable(string variableNodeId, string? parentFolderNodeId, string displayName, string dataType, bool writable, AddressSpaceRealm realm, string? historianTagname = null, bool isArray = false, uint? arrayLength = null, bool refuseArrayOnScalar = false);
@@ -186,7 +186,7 @@ public interface IOpcUaAddressSpaceSink
void RebuildAddressSpace();
/// <summary>
/// Gitea #641 — mark the END of one address-space apply pass (after <see cref="RebuildAddressSpace"/>, if
/// Mark the END of one address-space apply pass (after <see cref="RebuildAddressSpace"/>, if
/// one ran, and every materialise / notifier-wiring call that followed it). An SDK-backed sink carries each
/// native condition's live Part 9 state across a rebuild and restores it when the same condition is
/// re-materialised; this is where it reports the ONE condition event per restored condition (now that
@@ -229,7 +229,7 @@ public interface IOpcUaAddressSpaceSink
}
/// <summary>One value write within a <see cref="IOpcUaAddressSpaceSink.WriteValues"/> batch.
/// Same semantics per field as <see cref="IOpcUaAddressSpaceSink.WriteValue"/> (#519: status verbatim).</summary>
/// Same semantics per field as <see cref="IOpcUaAddressSpaceSink.WriteValue"/> (status verbatim).</summary>
public readonly record struct ValueWrite(
string NodeId, object? Value, uint StatusCode, DateTime SourceTimestampUtc, AddressSpaceRealm Realm);
@@ -237,7 +237,7 @@ public readonly record struct ValueWrite(
/// node's current VALUE is retained and only <paramref name="StatusCode"/> +
/// <paramref name="SourceTimestampUtc"/> change.</summary>
/// <param name="NodeId">The OPC UA node id of the variable, within <paramref name="Realm"/>.</param>
/// <param name="StatusCode">The full status code, sub-code included, passed to the SDK verbatim (#519).</param>
/// <param name="StatusCode">The full status code, sub-code included, passed to the SDK verbatim.</param>
/// <param name="SourceTimestampUtc">The source timestamp in UTC.</param>
/// <param name="Realm">Which of the two v3 namespaces <paramref name="NodeId"/> lives in.</param>
public readonly record struct StatusWrite(
@@ -279,6 +279,7 @@ public sealed class NullOpcUaAddressSpaceSink : IOpcUaAddressSpaceSink
/// <inheritdoc />
public void WriteAlarmCondition(string alarmNodeId, AlarmConditionSnapshot state, DateTime sourceTimestampUtc, AddressSpaceRealm realm) { }
/// <inheritdoc />
public void WriteAlarmQuality(string alarmNodeId, uint statusCode, DateTime sourceTimestampUtc, AddressSpaceRealm realm) { }
/// <inheritdoc />
@@ -2,7 +2,7 @@ namespace ZB.MOM.WW.OtOpcUa.Commons.OpcUa;
/// <summary>
/// The handful of OPC UA status codes the engine layer needs to spell, plus the Part 4 severity
/// classification helpers that replaced the retired <c>OpcUaQuality</c> 3-state enum (Gitea #519).
/// classification helpers that replaced the retired <c>OpcUaQuality</c> 3-state enum.
///
/// <para><b>Why these are bare <c>uint</c> literals.</b> Commons and Runtime deliberately do not
/// reference the OPC UA SDK — <see cref="IOpcUaAddressSpaceSink"/> exists precisely so the Runtime
@@ -10,11 +10,11 @@ namespace ZB.MOM.WW.OtOpcUa.Commons.OpcUa;
/// decision the driver assemblies make, and it carries the same single failure mode: nothing checks
/// the literal against the name. <c>StatusCodeParityTests</c> closes it by reflection, and it can
/// only see a <b>named</b> constant — so add codes here rather than writing a literal at a call
/// site, exactly as Gitea #497 concluded for the drivers.</para>
/// site, exactly as for the drivers.</para>
///
/// <para><b>Severity, not equality, is what callers should branch on.</b> A status code's top two
/// bits carry its Part 4 severity (00 = Good, 01 = Uncertain, 10/11 = Bad); everything below is the
/// sub-code. Before #519 the engine collapsed every value onto a 3-state enum here and the node
/// sub-code. Previously the engine collapsed every value onto a 3-state enum here and the node
/// manager re-expanded <c>Bad</c> to a bare <c>0x80000000</c>, so no driver's sub-code ever reached
/// a client. The sub-code now rides through to the SDK verbatim, and any code that only needs the
/// bucket uses <see cref="IsGood"/> / <see cref="IsUncertain"/> / <see cref="IsBad"/> rather than
@@ -1,7 +1,7 @@
namespace ZB.MOM.WW.OtOpcUa.Configuration.Entities;
/// <summary>
/// One OPC UA data-plane API key. The OPC UA data plane is machine-to-machine (#520), so a
/// One OPC UA data-plane API key. The OPC UA data plane is machine-to-machine, so a
/// consuming system authenticates with a key rather than an LDAP user, and the key resolves to a
/// <see cref="Principal"/> that <see cref="NodeAcl"/> rows grant permissions to.
/// </summary>
@@ -22,7 +22,7 @@ namespace ZB.MOM.WW.OtOpcUa.Configuration.Entities;
/// ⚠️ <b>Revocation requires a deploy.</b> Phase 4 leaves driver-only nodes with no ConfigDb, so
/// keys reach them inside the deployment artifact exactly as <see cref="NodeAcl"/> rows do.
/// Setting <see cref="Enabled"/> to false takes effect on a node only once a new artifact reaches
/// it. This is an operational sharp edge, not an oversight — see the #520 design doc.
/// it. This is an operational sharp edge, not an oversight.
/// </para>
/// </remarks>
public sealed class ApiKey
@@ -41,7 +41,7 @@ public sealed class OtOpcUaConfigDbContext(DbContextOptions<OtOpcUaConfigDbConte
/// <summary>Gets the DbSet of node ACLs.</summary>
public DbSet<NodeAcl> NodeAcls => Set<NodeAcl>();
/// <summary>OPC UA data-plane API keys (#520). One principal per key; secret stored as a peppered hash.</summary>
/// <summary>OPC UA data-plane API keys. One principal per key; secret stored as a peppered hash.</summary>
public DbSet<ApiKey> ApiKeys => Set<ApiKey>();
/// <summary>Gets the DbSet of configuration audit logs.</summary>
public DbSet<ConfigAuditLog> ConfigAuditLogs => Set<ConfigAuditLog>();
@@ -563,14 +563,14 @@ public sealed class OtOpcUaConfigDbContext(DbContextOptions<OtOpcUaConfigDbConte
});
}
// ConfigureDriverHostStatus is GONE (Gitea #521). The DriverHostStatus table held per-host
// ConfigureDriverHostStatus is GONE. The DriverHostStatus table held per-host
// connectivity that a publisher hosted service was supposed to upsert from each driver node. That
// publisher was never written, and per-cluster mesh Phase 4 made it unwritable as designed: ConfigDb
// is registered only on the admin role, so a driver-only node has no connection to write rows with.
// Per-host connectivity now rides DriverHealthChanged.HostStatuses to /hosts over the Phase 5
// telemetry stream, which needs no DB and survives the mesh split.
//
// ConfigureDriverInstanceResilienceStatus is GONE for the same reason (Gitea #524). That table held
// ConfigureDriverInstanceResilienceStatus is GONE for the same reason. That table held
// per-(instance, host) breaker/in-flight counters and was equally never written: its doc described a
// HostedService sampler and an AdminUI left-join on DriverHostStatus, neither of which was built. The
// live data reaches the AdminUI's DriverStatusPanel as DriverResilienceStatusChanged over the
@@ -57,7 +57,7 @@ public sealed class DraftSnapshot
public IReadOnlyList<ScriptedAlarm> ScriptedAlarms { get; init; } = [];
/// <summary>User-authored scripts (shared by VirtualTags, ScriptedAlarms, and Calculation tags). Drives
/// the WP7 Calculation gates: <c>scriptId</c> existence + the calc→calc dependency-cycle check.</summary>
/// the Calculation gates: <c>scriptId</c> existence + the calc→calc dependency-cycle check.</summary>
public IReadOnlyList<Script> Scripts { get; init; } = [];
/// <summary>Gets the list of poll groups.</summary>
public IReadOnlyList<PollGroup> PollGroups { get; init; } = [];
@@ -33,7 +33,7 @@ public static class DraftValidator
ValidateReservationPreflight(draft, errors);
ValidateEquipmentIdDerivation(draft, errors);
// v3 rules (WP4). The retired rules ValidateSameClusterNamespaceBinding (Namespace entity gone)
// v3 rules. The retired rules ValidateSameClusterNamespaceBinding (Namespace entity gone)
// and ValidateGalaxyTagFullName (blob-as-identity gone) are replaced by:
ValidateRawNameCharset(draft, errors);
ValidateHistorizedTagnameLength(draft, errors);
@@ -47,7 +47,7 @@ public static class DraftValidator
}
/// <summary>
/// #520 stage 3: every <see cref="NodeAcl"/> row's scope must name something the deployed address
/// every <see cref="NodeAcl"/> row's scope must name something the deployed address
/// space actually contains.
/// </summary>
/// <remarks>
@@ -74,7 +74,7 @@ public static class DraftValidator
/// <c>AclScopePathResolver</c>'s dictionary lookups use, so a row this rule passes is a row that
/// resolver resolves. The realm check mirrors <c>NodeAclRealms.TryCanonicalize</c> instead
/// (trimmed, case-insensitive) for the same reason. Both this rule and the resolver were
/// ordinal until Gitea #533; the cross-cluster comparison below was ordinal too, which was the
/// ordinal until that was fixed; the cross-cluster comparison below was ordinal too, which was the
/// worse half — it could reject a deploy as cross-cluster over a <c>ClusterId</c> the runtime
/// would have matched, and the two spellings genuinely occur (see
/// <c>RoleParser.ClusterIdFromRole</c>). The rule now is: identifiers compare
@@ -279,7 +279,7 @@ public static class DraftValidator
/// <summary>
/// The <c>Sql</c> driver's "a pasted literal connection string is never persisted" guarantee, enforced
/// at the deploy gate (Gitea #498).
/// at the deploy gate.
/// </summary>
/// <remarks>
/// <para>A Sql driver names its credentials indirectly — <c>connectionStringRef</c> resolves from the
@@ -294,7 +294,7 @@ public static class DraftValidator
/// Discarding a secret on read is not the same as refusing to store it.</para>
/// <para><b>Two of the three surfaces are checked here.</b> The third — a node's
/// <see cref="ClusterNode.DriverConfigOverridesJson"/> — is not reachable from
/// <see cref="DraftSnapshot"/> and is gated at its save instead (#499); see
/// <see cref="DraftSnapshot"/> and is gated at its save instead; see
/// <see cref="SqlCredentialGuard"/> for why a deploy gate is the wrong instrument for a value that
/// never enters the artifact.</para>
/// <para>Checked on <b>both</b> config surfaces a Sql driver reads: the instance's
@@ -344,7 +344,7 @@ public static class DraftValidator
}
}
/// <summary>Error code for an MTConnect driver instance that has more than one <c>/raw</c> Device (#628).</summary>
/// <summary>Error code for an MTConnect driver instance that has more than one <c>/raw</c> Device.</summary>
public const string MTConnectMultipleDevicesCode = "MTConnectMultipleDevices";
/// <summary>
@@ -352,6 +352,7 @@ public static class DraftValidator
/// use, so the two can never describe different rules.
/// </summary>
/// <param name="driverLabel">The driver instance id or name to cite.</param>
/// <returns>The validation message text.</returns>
public static string MTConnectMultipleDevicesMessage(string driverLabel) =>
$"MTConnect driver '{driverLabel}' has more than one Device. MTConnect binds all tags through one agent " +
"client per driver instance and does not route by /raw Device, and with several Devices the Device's " +
@@ -359,7 +360,7 @@ public static class DraftValidator
"supply the endpoint is ambiguous. Author one MTConnect driver per agent device, each with a single Device.";
/// <summary>
/// #628 (decision (b)): an MTConnect driver instance owns exactly one agent client, so a second <c>/raw</c>
/// An MTConnect driver instance owns exactly one agent client, so a second <c>/raw</c>
/// Device under it is a genuine ambiguity — the driver ignores <c>RawTagEntry.DeviceName</c>, and
/// <c>DriverDeviceConfigMerger</c> only merges a Device's <c>DeviceConfig</c> up to the driver config when
/// there is exactly ONE Device. The normal single-Device layout (every driver gets a default
@@ -380,7 +381,7 @@ public static class DraftValidator
}
}
/// <summary>WP7 Calculation-driver deploy gates. For every tag bound to a <c>Calculation</c> driver:
/// <summary>Calculation-driver deploy gates. For every tag bound to a <c>Calculation</c> driver:
/// <list type="number">
/// <item><b>scriptId existence</b> — the tag's <c>TagConfig.scriptId</c> must be present and resolve to
/// a <see cref="Script"/> row in the draft.</item>
@@ -634,7 +635,7 @@ public static class DraftValidator
}
// The rule itself lives on UnsSegment, which the AdminUI's authoring guards also call — a second
// copy here is exactly how the two drifted apart (Gitea #535). Only the per-level wording is local.
// copy here is exactly how the two drifted apart. Only the per-level wording is local.
private static void ValidateUnsSegments(DraftSnapshot draft, List<ValidationError> errors)
{
foreach (var a in draft.UnsAreas)
@@ -4,7 +4,7 @@ namespace ZB.MOM.WW.OtOpcUa.Configuration.Validation;
/// <summary>
/// The one place that decides whether a persisted config blob carries a literal Sql
/// <c>connectionString</c> (Gitea #498 / #499).
/// <c>connectionString</c>.
/// </summary>
/// <remarks>
/// <para>
@@ -12,7 +12,7 @@ namespace ZB.MOM.WW.OtOpcUa.Configuration.Validation;
/// rule is not cosmetic: it is what keeps a NodeId parseable and stable.
/// </para>
/// <para>
/// ⚠️ <b>This exists because the rule was duplicated and drifted (Gitea #535).</b> The deploy gate
/// ⚠️ <b>This exists because the rule was duplicated and drifted.</b> The deploy gate
/// held it privately in <c>DraftValidator</c> and the AdminUI restated it on the equipment form
/// alone, so an area or line could be named <c>Packaging</c> or <c>Line 1</c>, saved, and then
/// fail the deploy with <c>UnsSegmentInvalid</c>. Because a deployment snapshots the whole
@@ -2,7 +2,7 @@ namespace ZB.MOM.WW.OtOpcUa.Core.Abstractions;
/// <summary>
/// Shared capped-exponential backoff for poll-based / lazy-reconnect drivers, extracted from
/// the S7 driver's bespoke poll fork (05/STAB-8, CONV-1). Provides two surfaces:
/// the S7 driver's bespoke poll fork. Provides two surfaces:
/// <list type="bullet">
/// <item><see cref="ComputeDelay"/> — the pure schedule (base, 2×, 4×, … saturating at a
/// cap) used by <c>PollGroupEngine</c> to stretch its retry cadence under sustained
@@ -3,10 +3,10 @@ using System.Text.Json;
namespace ZB.MOM.WW.OtOpcUa.Core.Abstractions;
/// <summary>
/// Gitea #610 — the single authority on whether a driver can WRITE a 1-D array tag. The address-space
/// the single authority on whether a driver can WRITE a 1-D array tag. The address-space
/// composer consults it at compose time (both compose seams run the same composer, so the live-edit and
/// artifact-decode sides cannot disagree) to decide whether an authored-<c>ReadWrite</c> array tag
/// materialises as a writable OPC UA node. Before #610 every array node was forced read-only regardless of
/// materialises as a writable OPC UA node. Previously every array node was forced read-only regardless of
/// driver; that remains the answer for every (driver, element type) pair NOT listed here.
/// <para>
/// <b>The rule is "never advertise a write the driver will refuse".</b> A pair is listed only when the
@@ -125,16 +125,16 @@ public static class DriverArrayWriteSupport
}
/// <summary>
/// Gitea #610 review (R6 MINOR-1) — whether a driver of <paramref name="driverType"/> is known to REFUSE an
/// Whether a driver of <paramref name="driverType"/> is known to REFUSE an
/// array value written to one of its SCALAR tags, so the node manager may refuse such a write inline
/// (<c>BadTypeMismatch</c>, nothing dispatched) instead of storing it optimistically, dispatching it, and
/// reverting it when the scalar encoder throws.
/// <para>
/// Only Modbus and S7 qualify: their scalar encoders <c>Convert.ToXxx</c> the value, which throws on an
/// array. Every other driver keeps the pre-#610 behaviour — the write is dispatched and the driver
/// array. Every other driver keeps the previous behaviour — the write is dispatched and the driver
/// decides. That matters for the pass-through drivers: a browse-committed array attribute is committed
/// as a SCALAR tag (the <c>/raw</c> commit never sets <c>isArray</c>), the driver publishes an array into
/// it, and writing that array back worked before #610 — <c>OpcUaClient</c> forwards the
/// it, and writing that array back worked previously — <c>OpcUaClient</c> forwards the
/// <c>Variant</c> unchanged (the upstream server arbitrates shape) and Galaxy's <c>MxValueEncoder</c>
/// encodes the array. An unknown / null driver type is not listed ⇒ <see langword="false"/>.
/// </para>
@@ -53,7 +53,7 @@ namespace ZB.MOM.WW.OtOpcUa.Core.Abstractions;
/// Sparkplug births, MTConnect probe inference).
/// <para>
/// <b>It exists because <paramref name="DriverDataType"/> is lossy in exactly the direction a
/// browse-commit needs</b> (Gitea #536): <c>AbCipDataType.SInt</c>, <c>.Int</c> and <c>.DInt</c>
/// browse-commit needs</b>: <c>AbCipDataType.SInt</c>, <c>.Int</c> and <c>.DInt</c>
/// all map to <see cref="DriverDataType.Int32"/>, so a commit that reverse-mapped the
/// driver-agnostic type would bind a 16-bit INT as a 32-bit DINT. Nothing downstream of the
/// address-space build reads this — it is authoring metadata carried alongside the type the
@@ -38,7 +38,7 @@ public enum DriverCapability
AlarmAcknowledge,
/// <summary>
/// <c>IDriver.InitializeAsync</c> — connection establishment (R2-10 T10 closure: without this,
/// <c>IDriver.InitializeAsync</c> — connection establishment (without this,
/// a driver that cannot connect contributes zero breaker input and the resilience panel stays
/// quiet for the failure operators most care about). Does NOT retry — the
/// <c>DriverInstanceActor</c> reconnect timer is the retry loop, and pipeline-level retries
@@ -1,6 +1,6 @@
namespace ZB.MOM.WW.OtOpcUa.Core.Abstractions;
// OPC UA Part 11 modified-value history (Gitea #668): every HistoryUpdate a backend applies leaves an audit row
// OPC UA Part 11 modified-value history: every HistoryUpdate a backend applies leaves an audit row
// carrying the PRIOR value, who made the change and when; HistoryRead with IsReadModified=true serves them.
/// <summary>Who is performing a HistoryUpdate. Threaded into every <see cref="IHistorianDataUpdater"/> call.</summary>
@@ -1,7 +1,7 @@
namespace ZB.MOM.WW.OtOpcUa.Core.Abstractions;
// Shared history result contracts used by IHistorianDataSource and the historian gateway.
// Drivers carry no history capability (Gitea #655): HistoryRead is served server-side.
// Drivers carry no history capability: HistoryRead is served server-side.
/// <summary>Result of a HistoryRead call.</summary>
@@ -13,7 +13,7 @@ public sealed record HistoryReadResult(
/// <summary>Aggregate function for processed history reads. Mirrors OPC UA Part 13 standard aggregates.</summary>
/// <remarks>
/// The first five members are the original set every historian backend serves. The remainder (#285) are
/// The first five members are the original set every historian backend serves. The remainder are
/// the further standard Part 13 aggregates a server-side HistoryRead can forward to a backend that
/// supports them (a gateway backend maps them to its own aggregate modes). A backend that cannot serve a
/// member must throw <see cref="HistoryAggregateNotSupportedException"/> — the server maps that to
@@ -48,14 +48,14 @@ public enum HistoryAggregateType
StandardDeviationPopulation,
VariancePopulation,
/// <summary>Part 13 TimeAverage2: time-weighted average over the interval's non-Bad data only (#671).</summary>
/// <summary>Part 13 TimeAverage2: time-weighted average over the interval's non-Bad data only.</summary>
TimeAverage2,
/// <summary>Part 13 Total2: time-weighted integral (value·seconds) over the interval's non-Bad data only (#671).</summary>
/// <summary>Part 13 Total2: time-weighted integral (value·seconds) over the interval's non-Bad data only.</summary>
Total2,
}
/// <summary>Shared <see cref="HistoryAggregateType"/> sets for <see cref="IHistorianDataSource.SupportedAggregates"/> (#682).</summary>
/// <summary>Shared <see cref="HistoryAggregateType"/> sets for <see cref="IHistorianDataSource.SupportedAggregates"/>.</summary>
public static class HistoryAggregateTypes
{
/// <summary>Every defined <see cref="HistoryAggregateType"/> — the interface default.</summary>
@@ -85,7 +85,7 @@ public sealed class HistoryAggregateNotSupportedException : NotSupportedExceptio
/// <summary>
/// Thrown by a history backend when a read would exceed a configured size limit (e.g. the raw-row cap of the SQL
/// backends' time-weighted aggregates, #671). The Server HistoryRead path translates it to
/// backends' time-weighted aggregates). The Server HistoryRead path translates it to
/// <c>BadTooManyOperations</c> for that node only; it is a client-request limit, not a backend fault.
/// </summary>
public sealed class HistoryReadLimitExceededException : InvalidOperationException
@@ -94,7 +94,7 @@ public sealed class HistoryReadLimitExceededException : InvalidOperationExceptio
public long Limit { get; }
/// <summary>
/// Gitea #694: a raw read refused because rows BEFORE its window (inside the backend's look-back margin — a tie
/// a raw read refused because rows BEFORE its window (inside the backend's look-back margin — a tie
/// cluster just before a page start, the HistorianGateway's boundary row) crowded a ceiling-bounded row budget: how
/// many such rows the read saw (a lower bound when the read was itself truncated). <c>0</c> = not a crowding refusal.
/// The server re-reads such a page once, smaller by this many values (<c>RawPageLimit</c>).
@@ -127,7 +127,7 @@ public sealed record HistoricalEvent(
ushort Severity);
/// <summary>Result of a <see cref="IHistorianDataSource.ReadEventsAsync(string?, DateTime, DateTime, int, CancellationToken)"/> call.</summary>
/// <param name="Events">Events ordered by <c>EventTimeUtc</c> in the read's direction — ascending, or descending for a reverse read (#675).</param>
/// <param name="Events">Events ordered by <c>EventTimeUtc</c> in the read's direction — ascending, or descending for a reverse read.</param>
/// <param name="ContinuationPoint">Opaque token for the next call when more events are available; null when complete.</param>
public sealed record HistoricalEventsResult(
IReadOnlyList<HistoricalEvent> Events,
@@ -6,7 +6,7 @@ namespace ZB.MOM.WW.OtOpcUa.Core.Abstractions;
/// </summary>
/// <remarks>
/// This is a *server registration*, not a driver capability: drivers carry no history
/// surface (Gitea #655). The server calls it directly, so a single historian can serve
/// surface. The server calls it directly, so a single historian can serve
/// many drivers' nodes, and drivers can restart without dropping history availability.
/// All values returned use the shared <see cref="DataValueSnapshot"/> /
/// <see cref="HistoricalEvent"/> shapes; backend-specific quality / type encodings
@@ -20,7 +20,7 @@ public interface IHistorianDataSource : IDisposable
/// <remarks>
/// Forward (<c>startUtc &lt; endUtc</c>) is <c>[start, end)</c> ascending; <c>startUtc == endUtc</c> is a point read.
/// <c>startUtc &gt; endUtc</c> is the Part 11 reverse read <c>(end, start]</c> newest-first — but only a backend that
/// declares <see cref="SupportsReverseRaw"/> is ever sent one (Gitea #686); the server serves a reverse read on any
/// declares <see cref="SupportsReverseRaw"/> is ever sent one; the server serves a reverse read on any
/// other backend from bounded forward reads.
/// </remarks>
/// <param name="fullReference">The full reference of the tag to read.</param>
@@ -38,7 +38,7 @@ public interface IHistorianDataSource : IDisposable
/// <summary>
/// Whether this backend can serve a reverse (newest-first) raw read itself — <see cref="ReadRawAsync"/> with
/// <c>startUtc &gt; endUtc</c> (Gitea #686). The server checks it before every raw read. When <c>false</c> it never
/// <c>startUtc &gt; endUtc</c>. The server checks it before every raw read. When <c>false</c> it never
/// sends <c>start &gt; end</c>: it serves the reverse read — an explicit one, a resumed reverse page, and the
/// end-only capped read Part 11 reads backward from EndTime — from a series of bounded FORWARD reads walking back
/// from the later bound (<c>ReverseRawReader</c> in the OPC UA server). Default <c>false</c>, the safe direction: a
@@ -48,11 +48,11 @@ public interface IHistorianDataSource : IDisposable
/// <summary>
/// The most values one capped <see cref="ReadRawAsync"/> can currently return without failing
/// <see cref="HistoryReadLimitExceededException"/> on a BACKEND row ceiling (Gitea #694); <c>0</c> = no known limit.
/// <see cref="HistoryReadLimitExceededException"/> on a BACKEND row ceiling; <c>0</c> = no known limit.
/// The server sizes its raw pages to it — a page is at most this many values and carries a continuation point when
/// full (Part 11 allows fewer than <c>NumValuesPerNode</c> with a continuation point), instead of failing a page size
/// the backend cannot read in one go. It may change at runtime (a strict HistorianGateway's ceiling is learned from
/// its first refusal, #691.3), so it is read per read. A backend whose reads are never ceiling-bound returns <c>0</c>.
/// its first refusal), so it is read per read. A backend whose reads are never ceiling-bound returns <c>0</c>.
/// The source still never truncates: a read asking for more than this is answered in full or refused. Default
/// <c>0</c>; a backend declares it itself (a guard test pins that).
/// </summary>
@@ -94,7 +94,7 @@ public interface IHistorianDataSource : IDisposable
CancellationToken cancellationToken);
/// <summary>
/// Gitea #693: <see cref="ReadAtTimeAsync(string, IReadOnlyList{DateTime}, CancellationToken)"/> with the value type of
/// <see cref="ReadAtTimeAsync(string, IReadOnlyList{DateTime}, CancellationToken)"/> with the value type of
/// the node being read, for a backend whose stored values do not carry their type (the Gateway: AVEVA stores every
/// historized type as a numeric analog, so a Boolean comes back as a 0/1 Double). Such a backend uses the hint to apply
/// the AtTime rules exactly as the typed backends do — a Boolean (or String / DateTime) node is stepped, an integer node
@@ -156,17 +156,17 @@ public interface IHistorianDataSource : IDisposable
/// <summary>
/// Whether this backend can serve a reverse (newest-first) events read — the reverse overload of
/// <see cref="ReadEventsAsync(string?, DateTime, DateTime, int, bool, CancellationToken)"/> (Gitea #675). The server
/// <see cref="ReadEventsAsync(string?, DateTime, DateTime, int, bool, CancellationToken)"/>. The server
/// checks it BEFORE calling the source: a reverse read (explicit <c>StartTime &gt; EndTime</c>, or an end-only capped
/// read, which Part 11 reads backward) on a backend that cannot is never sent the reverse overload — since Gitea
/// #691.4 the server walks it back from bounded FORWARD reads instead (#675 refused it). Default <c>false</c>; an
/// the server walks it back from bounded FORWARD reads instead (it was once refused). Default <c>false</c>; an
/// implementation (or a wrapper) declares it itself — a guard test pins that.
/// </summary>
bool SupportsReverseEvents => false;
/// <summary>
/// Read historical alarm / event records in a chosen direction — the overload the server's HistoryReadEvents
/// paging calls (Gitea #675).
/// paging calls.
/// </summary>
/// <remarks>
/// <para>
@@ -182,7 +182,7 @@ public interface IHistorianDataSource : IDisposable
/// The default forwards a forward read to the five-argument overload and throws
/// <see cref="NotSupportedException"/> for a reverse one — a backend that cannot read newest-first (the
/// Gateway). The server never relies on that throw: it consults <see cref="SupportsReverseEvents"/> first and walks
/// such a read back from forward reads (#691.4), never returning the earliest events in ascending order as if
/// such a read back from forward reads, never returning the earliest events in ascending order as if
/// they were the answer. A backend that serves reverse natively must override BOTH members.
/// </para>
/// </remarks>
@@ -228,7 +228,7 @@ public interface IHistorianDataSource : IDisposable
long TimestampResolutionTicks => 1;
/// <summary>
/// The Part 13 aggregates this backend serves for a processed HistoryRead (Gitea #682) — the ONE authority for
/// The Part 13 aggregates this backend serves for a processed HistoryRead — the ONE authority for
/// both what the server advertises under <c>HistoryServerCapabilities/AggregateFunctions</c> (i=11201) and what
/// its HistoryRead-Processed answers <c>BadAggregateNotSupported</c> up front without calling the backend.
/// (<c>ServerCapabilities/AggregateFunctions</c>, i=2997, is not affected: it lists every aggregate the server
@@ -236,7 +236,7 @@ public interface IHistorianDataSource : IDisposable
/// </summary>
/// <remarks>
/// The default is <see cref="HistoryAggregateTypes.All"/> — "every aggregate the server can name; a member I cannot
/// serve throws <see cref="HistoryAggregateNotSupportedException"/> at read time", which is exactly the pre-#682
/// serve throws <see cref="HistoryAggregateNotSupportedException"/> at read time", which is exactly the earlier
/// behaviour, so a test fake or an out-of-tree source keeps working unchanged. It is a concrete set rather than a
/// <c>null</c> "all" sentinel so every consumer handles one shape. Every PRODUCTION backend declares its set itself
/// (a reflection guard pins that): a wrapper that forgot to forward this would otherwise advertise every aggregate.
@@ -247,7 +247,7 @@ public interface IHistorianDataSource : IDisposable
IReadOnlySet<HistoryAggregateType> SupportedAggregates => HistoryAggregateTypes.All;
/// <summary>
/// Read modified-value history — OPC UA HistoryReadRaw with <c>IsReadModified = true</c> (Part 11, Gitea #668):
/// Read modified-value history — OPC UA HistoryReadRaw with <c>IsReadModified = true</c> (Part 11):
/// every modification a HistoryUpdate applied to the tag inside the window, each carrying the PRIOR value.
/// </summary>
/// <remarks>
@@ -18,13 +18,13 @@ public enum HistoryUpdateMode
/// supplies no updater (the server then reports HistoryUpdate as unsupported).
/// </summary>
/// <remarks>
/// <b>Modified-value history (Part 11, Gitea #668).</b> Every sample an implementation inserts, replaces or deletes
/// <b>Modified-value history (Part 11).</b> Every sample an implementation inserts, replaces or deletes
/// MUST leave one modification record carrying the prior value (none for an insert), the
/// <see cref="HistoryUpdateContext.Principal"/>, the modification time and the <see cref="HistoryModificationType"/>,
/// readable through <see cref="IHistorianDataSource.ReadModifiedAsync"/>. A refused item (any non-Good code) leaves
/// no record. Live recorder writes and retention purges never leave one.
/// <para>
/// <b>Cannot update at all</b> (Gitea #674): an implementation that is wired but cannot perform any update right
/// <b>Cannot update at all</b>: an implementation that is wired but cannot perform any update right
/// now — the SqlServer backend on a schema without the modification trail — throws
/// <see cref="NotSupportedException"/> from every method without touching storage. The server reports it as a
/// node-level <c>BadHistoryOperationUnsupported</c>, the same as <see cref="IHistorianDataSource.ReadModifiedAsync"/>.
@@ -56,21 +56,21 @@ public sealed class NullHistorianDataSource : IHistorianDataSource
CancellationToken cancellationToken) => Task.FromResult(EmptyEvents);
/// <inheritdoc />
/// <remarks><c>true</c> (Gitea #686): there is no history, so the empty result is the correct answer in either
/// direction — no need for the server to walk back through forward reads to find nothing.</remarks>
// true: there is no history, so the empty result is the correct answer in either
// direction — no need for the server to walk back through forward reads to find nothing.
public bool SupportsReverseRaw => true;
/// <inheritdoc />
/// <remarks><c>0</c>: no history, no ceiling (Gitea #694).</remarks>
// 0: no history, no ceiling.
public int MaxRawValuesPerRead => 0;
/// <inheritdoc />
/// <remarks><c>true</c>, like <see cref="SupportsReverseRaw"/> (Gitea #691.4 review; <c>false</c> under #675): there is no
/// history to read in either direction — no need for the server to walk back through empty forward reads.</remarks>
// true, like SupportsReverseRaw: there is no history to read in either direction —
// no need for the server to walk back through empty forward reads.
public bool SupportsReverseEvents => true;
/// <inheritdoc />
/// <remarks>The empty result in either direction.</remarks>
// The empty result in either direction.
public Task<HistoricalEventsResult> ReadEventsAsync(
string? sourceName,
DateTime startUtc,
@@ -81,12 +81,10 @@ public sealed class NullHistorianDataSource : IHistorianDataSource
Task.FromResult(EmptyEvents);
/// <inheritdoc />
/// <remarks>
/// EMPTY (Gitea #682): with no historian configured the server advertises no aggregate functions, and a processed
/// HistoryRead answers <c>BadAggregateNotSupported</c> per node rather than <c>GoodNoData</c> — an aggregate is a
/// statistic the server cannot compute without a history, and advertising the SDK's full list on a server with no
/// historian misled capability discovery. Raw / AtTime / Events reads still return the empty <c>GoodNoData</c> result.
/// </remarks>
// EMPTY: with no historian configured the server advertises no aggregate functions, and a processed
// HistoryRead answers BadAggregateNotSupported per node rather than GoodNoData — an aggregate is a
// statistic the server cannot compute without a history, and advertising the SDK's full list on a server with no
// historian misled capability discovery. Raw / AtTime / Events reads still return the empty GoodNoData result.
public IReadOnlySet<HistoryAggregateType> SupportedAggregates => HistoryAggregateTypes.None;
/// <inheritdoc />
@@ -4,7 +4,7 @@ namespace ZB.MOM.WW.OtOpcUa.Core.Abstractions;
/// Driver capability for alarm events. Optional — only drivers whose backends expose
/// alarm conditions implement this. Currently: Galaxy (MxAccess alarms), FOCAS
/// (CNC alarms), OPC UA Client (A&amp;C events from upstream server), MTConnect
/// (CONDITION data items, Gitea #626).
/// (CONDITION data items).
/// </summary>
public interface IAlarmSource
{
@@ -88,22 +88,22 @@ public sealed record AlarmAcknowledgeRequest(
/// </param>
/// <param name="OpcUaSeverity">
/// Optional exact OPC UA severity (1..1000) for a source whose own scale is finer than the four
/// <see cref="AlarmSeverity"/> buckets (Gitea #626: MTConnect Warning = 500, Fault = 800). When set it
/// <see cref="AlarmSeverity"/> buckets (MTConnect Warning = 500, Fault = 800). When set it
/// supersedes the bucket on the projected Part 9 condition (clamped to 1..1000); <c>null</c> — every
/// pre-#626 source — keeps the bucket mapping. <see cref="Severity"/> must still carry the nearest
/// earlier source — keeps the bucket mapping. <see cref="Severity"/> must still carry the nearest
/// bucket for consumers that only read the bucket.
/// </param>
/// <param name="QualityStatusCode">
/// Optional status code of the condition's SOURCE data (Part 9 <c>ConditionType.Quality</c>), for a
/// source that can report "no value for this condition" per alarm rather than per connection (Gitea
/// #626: MTConnect <c>Unavailable</c>). <c>null</c> keeps the pre-#626 behaviour (projected Good;
/// source that can report "no value for this condition" per alarm rather than per connection (for example
/// MTConnect <c>Unavailable</c>). <c>null</c> keeps the earlier behaviour (projected Good;
/// driver-wide comms loss is annotated separately by the host). An event whose <see cref="Kind"/> is
/// <see cref="AlarmTransitionKind.Unspecified"/> and which carries a quality is a
/// <b>quality-only annotation</b> (<see cref="IsQualityOnly"/>): the host updates (via its Quality-only node path) the condition's
/// Quality without touching its Active/Acked state and publishes no <c>alerts</c> row for it.
/// </param>
/// <param name="IsRestatement">
/// Gitea #656 — true when the source is RE-STATING upstream's current state rather than reporting a transition it
/// True when the source is RE-STATING upstream's current state rather than reporting a transition it
/// watched happen: a reconnect snapshot entry (Galaxy) or the first event for a condition after a driver start or
/// deploy respawn (OpcUaClient). It governs <b>history only, never node state</b>: the host still projects the
/// state, but writes no <c>alerts</c> row when the re-statement changes nothing the condition shows (a Retrigger on
@@ -111,7 +111,7 @@ public sealed record AlarmAcknowledgeRequest(
/// Acknowledge on an already-acknowledged condition) — or
/// when a flagged <see cref="AlarmTransitionKind.Unspecified"/> update moves the message/severity of an
/// already-active condition (node updated, one condition event, active/acked kept, no row). AbCip ALMA sets the
/// flag for the last shape (the governing limit changed, no new occurrence). OpcUaClient (Gitea #657) re-states a
/// flag for the last shape (the governing limit changed, no new occurrence). OpcUaClient re-states a
/// first-seen inactive+acked condition as a flagged Clear then a flagged Acknowledge, so a host already showing
/// that writes nothing and a host retaining it Active gets a real "Cleared". <c>false</c> (the default, every source that does not
/// opt in) keeps every transition's row.
@@ -147,9 +147,9 @@ public enum AlarmSeverity { Low, Medium, High, Critical }
/// Kind of alarm state change carried by <see cref="AlarmEventArgs.Kind"/>, letting a
/// consumer derive OPC UA Part 9 active/ack state. Mirrors the driver-side
/// transition-kind enums (e.g. Galaxy's <c>GalaxyAlarmTransitionKind</c>).
/// <see cref="ActiveAcknowledged"/> (Gitea #656) means "the condition is active AND acknowledged", applied by the
/// <see cref="ActiveAcknowledged"/> means "the condition is active AND acknowledged", applied by the
/// host as ONE projection — never Raise-then-Acknowledge, which would show clients an Active+unacked intermediate.
/// <see cref="InactiveUnacknowledged"/> (Gitea #657) is its mirror: "the condition is inactive AND unacknowledged"
/// <see cref="InactiveUnacknowledged"/> is its mirror: "the condition is inactive AND unacknowledged"
/// (it went active and cleared unacknowledged), applied as ONE projection — never Raise-then-Clear, which showed
/// clients the condition Active again for an instant. The host writes rows only for what actually changed:
/// "Activated" + "Cleared" when the condition was acknowledged (a new occurrence must have happened), "Cleared"
@@ -24,8 +24,8 @@ public interface IDriver
/// <summary>
/// Apply a config change in place without tearing down the driver process.
/// <para>The runtime does not call this today: since #516 a DriverConfig change is a stop + respawn,
/// and #665 deleted <c>DriverInstanceActor</c>'s in-place <c>ApplyDelta</c> path. Kept as the in-place
/// <para>The runtime does not call this today: a DriverConfig change is a stop + respawn,
/// and <c>DriverInstanceActor</c> no longer has an in-place <c>ApplyDelta</c> path. Kept as the in-place
/// contract each driver implements and tests.</para>
/// </summary>
/// <remarks>
@@ -40,12 +40,12 @@ public interface IDriver
/// <summary>Stop the driver, close connections, release resources. Called on shutdown or driver removal.</summary>
/// <remarks>
/// <para><b>Land the terminal health value AFTER the teardown, never only before it</b> (Gitea #517).
/// <para><b>Land the terminal health value AFTER the teardown, never only before it</b>.
/// Teardown drains in-flight poll / probe loops, and those loops write health as they unwind — so a
/// driver that wrote <see cref="DriverState.Unknown"/> first and then tore down could have a late
/// loop overwrite it with <see cref="DriverState.Healthy"/> and leave THAT as the resting state,
/// with every connection already closed. This is why the fleet writes last, and it is the reason
/// #517's original suggestion — "write the transitional value before disposing" — is only safe as an
/// The original suggestion — "write the transitional value before disposing" — is only safe as an
/// ADDITIONAL write, bracketing the teardown, rather than as a move.</para>
/// <para>Bracketing matters wherever a <see cref="GetHealth"/> can land mid-teardown. It cannot on
/// this path today: <c>DriverInstanceActor</c> calls this from <c>PostStop</c>, on the actor thread,
@@ -61,7 +61,7 @@ public interface IDriver
/// <summary>Current health snapshot, polled by Core for the status dashboard and ServiceLevel.
/// <para>Must reflect the driver's ACTUAL state rather than a constant — a hard-coded
/// <see cref="DriverState.Healthy"/> reports a healthy driver after shutdown and before initialize
/// alike, which is what Gitea #517 found in <c>CalculationDriver</c>. A driver with no transport still
/// alike, which was found in <c>CalculationDriver</c>. A driver with no transport still
/// has a running / not-running distinction worth reporting.</para></summary>
/// <returns>The driver's current health snapshot.</returns>
DriverHealth GetHealth();
@@ -49,7 +49,7 @@ public interface IDriverCapabilityInvoker
/// <paramref name="isIdempotent"/> is <c>false</c>, retries are disabled regardless of the
/// configured policy; when <c>true</c>, the call runs through the Write pipeline which may
/// retry if the tier configuration permits. <b>The current production default is <c>false</c></b>
/// (R2-02/S-8): <c>DriverInstanceActor.HandleWriteAsync</c> passes <c>isIdempotent: false</c> so a
/// <c>DriverInstanceActor.HandleWriteAsync</c> passes <c>isIdempotent: false</c> so a
/// command-shaped write is never replayed; per-tag opt-in via <see cref="WriteIdempotentAttribute"/>
/// is the unshipped forward path.
/// </summary>
@@ -31,7 +31,7 @@ public interface IDriverCapabilityInvokerFactory
/// <summary>
/// Release every piece of shared resilience state held for <paramref name="driverInstanceId"/> — called
/// by the host when the instance leaves the deployed plan (#609). A concrete factory drops the
/// by the host when the instance leaves the deployed plan. A concrete factory drops the
/// instance's cached pipelines and its status-tracker snapshots, so the periodic resilience publisher
/// stops re-emitting (and re-priming new telemetry subscribers with) a decommissioned driver's last
/// breaker/failure counters. Default: no-op (a factory that holds no state).
@@ -1,9 +1,9 @@
namespace ZB.MOM.WW.OtOpcUa.Core.Abstractions;
/// <summary>
/// Optional driver capability (Gitea #676) — a driver that can tell, from its protocol, that its connection is
/// Optional driver capability — a driver that can tell, from its protocol, that its connection is
/// DEFINITIVELY gone (it can never recover in place, only a re-initialise brings it back) raises
/// <see cref="ConnectionLost"/> so the host re-initialises it at once instead of waiting out the #527 comms-loss
/// <see cref="ConnectionLost"/> so the host re-initialises it at once instead of waiting out the comms-loss
/// watchdog (~90 s of frozen <see cref="DriverHealth.LastSuccessfulRead"/>).
/// </summary>
/// <remarks>
@@ -3,7 +3,7 @@ namespace ZB.MOM.WW.OtOpcUa.Core.Abstractions;
/// <summary>
/// Optional driver capability: expose a small bag of operator-facing diagnostic key/value pairs
/// (e.g. S7's negotiated PDU size) that ride the driver-health snapshot to the AdminUI <c>/hosts</c>
/// page (Gitea #640).
/// page.
/// </summary>
/// <remarks>
/// <para>
@@ -32,11 +32,11 @@ public interface IDriverHealthPublisher
/// Authored tags the driver could not build into its poll table, from
/// <see cref="ISkippedTagReporter.GetSkippedTags"/>; null when the driver is not an
/// <see cref="ISkippedTagReporter"/>. Contradicts the "Healthy, 0 errors, reads advancing" reading
/// of a driver that is in fact ignoring part or all of its configuration (Gitea #541).
/// of a driver that is in fact ignoring part or all of its configuration.
/// </param>
/// <param name="diagnostics">
/// Bounded driver diagnostics from <see cref="IDriverDiagnostics.GetDiagnostics"/>; null when the
/// driver is not an <see cref="IDriverDiagnostics"/> (Gitea #640).
/// driver is not an <see cref="IDriverDiagnostics"/>.
/// </param>
void Publish(
string clusterId,
@@ -12,7 +12,7 @@ namespace ZB.MOM.WW.OtOpcUa.Core.Abstractions;
/// <i>inbound</i>: a write, an alarm ack, an alerts publish. A Sparkplug primary-host claim is
/// not inbound. It is a standing assertion the driver makes to the broker, on its own
/// connection, for as long as it holds one — there is no message for the host actor to
/// intercept. So the decision has to reach the driver instead (Gitea #511).
/// intercept. So the decision has to reach the driver instead.
/// </para>
/// <para>
/// <b>The decision, not the role.</b> Same discipline as the alarm-drain view in
@@ -61,7 +61,7 @@ public interface IDriverPrimaryRoleSource
/// publishing a role there is by definition no peer to collide with, and a closed default
/// would make an exclusive-identity claim silently never happen on the single-node deployment
/// that is the common case — an inert feature with no error, which is precisely the failure
/// mode Gitea #511 was filed about.
/// mode this contract exists to prevent.
/// </para>
/// <para>
/// The risk this shifts onto the wiring is real and is guarded there instead: a redundant
@@ -83,7 +83,6 @@ public sealed class UnmanagedDriverPrimaryRoleSource : IDriverPrimaryRoleSource
public bool ShouldActAsPrimary => true;
/// <inheritdoc />
/// <remarks>No-op: there is no redundancy manager to publish anything here.</remarks>
public void PublishPrimaryRole(bool shouldActAsPrimary)
{
}
@@ -7,18 +7,18 @@ namespace ZB.MOM.WW.OtOpcUa.Core.Abstractions;
/// </summary>
/// <remarks>
/// <para>
/// <b>Why this exists (Gitea #541).</b> Every driver builds its <c>RawPath → definition</c>
/// <b>Why this exists.</b> Every driver builds its <c>RawPath → definition</c>
/// table at initialise and <i>tolerates</i> a per-tag miss — an unmappable <c>TagConfig</c>, an
/// unparseable address, a capability-matrix rejection. Tolerance is the right call: one bad tag
/// must not take down the other few thousand. But a skip is not an error and happens before any
/// read, so the driver stays <c>Healthy</c>, <c>errors/5 min</c> stays 0, <c>Last error</c> stays
/// empty, and <c>Last read</c> advances live off the tags that <i>did</i> map. On the Makino in
/// Gitea #540 that combination held while the driver ignored 20 of its 22 tags, and no surface
/// a past incident that combination held while the driver ignored 20 of its 22 tags, and no surface
/// anywhere said otherwise.
/// </para>
/// <para>
/// <b>Deliberately additive to driver state, never a change of it.</b> Latching a driver
/// non-Healthy over a config problem would collide with the Gitea #527 comms-loss detector,
/// non-Healthy over a config problem would collide with the comms-loss detector,
/// which trips on <i>90 s without a successful read AND non-Healthy</i>. A driver whose tags
/// were <i>all</i> skipped has no successful reads by construction, so it would satisfy both
/// halves and reconnect-loop forever against a device that is perfectly fine. The signal
@@ -32,7 +32,7 @@ namespace ZB.MOM.WW.OtOpcUa.Core.Abstractions;
/// <para>
/// Implementations should accumulate into a <see cref="SkippedTagLedger"/> rather than
/// hand-rolling a counter: the ledger is one implementation of the reset/commit/cap rules
/// instead of one per driver, which is precisely the duplication #541 was filed about.
/// instead of one per driver, which is precisely the duplication this type removes.
/// </para>
/// </remarks>
public interface ISkippedTagReporter
@@ -4,7 +4,7 @@ namespace ZB.MOM.WW.OtOpcUa.Core.Abstractions;
/// The one worst-of-input rule every derived-value seam uses — scripted alarms, virtual tags and
/// the Calculation driver.
///
/// <para><b>The contract it implements</b> (archreview R3-02, "the stale-Good family"): a value
/// <para><b>The contract it implements</b> ("the stale-Good family"): a value
/// consumer never reports a quality better than its source's. A node derived from driver values
/// carries, at worst, the worst status among its inputs — a Bad input makes the derived node Bad
/// with that input's sub-code, an Uncertain input makes it Uncertain, and a source that has
@@ -87,8 +87,8 @@ public sealed class PollGroupEngine : IAsyncDisposable
/// internal contract-violation throw) so the owning driver can route the failure to its
/// health surface. Defensive: an <c>onError</c> handler that
/// itself throws is silently absorbed so a buggy forwarder cannot crash the poll loop.</param>
/// <param name="backoffCap">Optional cap enabling capped-exponential failure backoff
/// (05/STAB-8). When supplied, sustained reader failures stretch the retry cadence
/// <param name="backoffCap">Optional cap enabling capped-exponential failure backoff.
/// When supplied, sustained reader failures stretch the retry cadence
/// (interval, 2×, 4×, … saturating at this cap) instead of hammering a dead device every
/// tick; a successful poll resets to the base interval. <c>null</c> (default) keeps the
/// byte-identical fixed-interval cadence for consumers that have not opted in.
@@ -260,7 +260,7 @@ public sealed class PollGroupEngine : IAsyncDisposable
consecutiveFailures = 0;
}
// Only a teardown OCE (the loop's OWN token) exits the loop. A reader that throws OCE with
// the loop token un-cancelled — a driver-internal timeout CTS (the STAB-14/STAB-15 class) —
// the loop token un-cancelled — a driver-internal timeout CTS (the class of failure) —
// must NOT tear the loop down: it falls through to the generic handler, is reported, and the
// loop retries with backoff. This is the guard that makes the S7 fork migration safe.
catch (OperationCanceledException) when (ct.IsCancellationRequested) { return; }
@@ -319,7 +319,7 @@ public sealed class PollGroupEngine : IAsyncDisposable
consecutiveFailures = 0;
}
// Only teardown (the loop token) exits; a reader OCE with ct un-cancelled backs off
// like any other failure instead of killing the loop (STAB-14 engine guard).
// like any other failure instead of killing the loop.
catch (OperationCanceledException) when (ct.IsCancellationRequested) { return; }
catch (Exception ex) when (!IsFatal(ex))
{
@@ -19,7 +19,7 @@ public enum JsonEnumRead
}
/// <summary>
/// Shared TagConfig JSON field readers for the six equipment-tag parsers (05/CONV-2). Replaces the
/// Shared TagConfig JSON field readers for the six equipment-tag parsers. Replaces the
/// six byte-identical private <c>ReadEnum</c> copies with one strict-capable home beside
/// <see cref="EquipmentTagRefResolver{TDef}"/>.
/// <para><b>Unknown-key handling is a deliberate non-goal.</b> The TagConfig blob is a SHARED
@@ -62,7 +62,7 @@ public static class TagConfigJson
/// AdminUI's typed tag-config editors hold a mutable <c>JsonObject</c> key bag, not a
/// <see cref="JsonElement"/> — and an authoring gate that RESTATED the accept/reject boundary
/// would drift from the runtime the moment either side moved, in the one direction that hurts:
/// a value the modal accepts and the driver then rejects (Gitea #549). Both doors reach this
/// a value the modal accepts and the driver then rejects. Both doors reach this
/// method.
/// </para>
/// <para>
@@ -131,7 +131,7 @@ public static class TagConfigJson
=> TryReadEnum<TEnum>(o, name, out var v) == JsonEnumRead.Valid ? v : fallback;
/// <summary>
/// Strict enum read (R2-11 Phase C — the runtime flip). Absent ⇒ <paramref name="fallback"/>;
/// Strict enum read. Absent ⇒ <paramref name="fallback"/>;
/// a present, case-insensitively-parseable string ⇒ the parsed value; a present-but-invalid
/// string (a typo) ⇒ the read FAILS (returns <see langword="false"/>) so the runtime parser can
/// reject the whole tag and the driver surfaces <c>BadNodeIdUnknown</c> — instead of silently
@@ -9,7 +9,7 @@ namespace ZB.MOM.WW.OtOpcUa.Core.Abstractions;
/// <remarks>
/// Applied to tag-definition POCOs
/// (e.g. <c>ModbusTagDefinition</c>, <c>S7TagDefinition</c>, OPC UA client tag rows) at the
/// property or record level. <b>Reserved for the per-tag opt-in; NOT yet consulted</b> (R2-02/S-8) —
/// property or record level. <b>Reserved for the per-tag opt-in; NOT yet consulted</b> —
/// the dispatch site (<c>DriverInstanceActor.HandleWriteAsync</c>) currently treats every write as
/// non-idempotent, so no tag retries a write today. Threading tag-definition metadata to the dispatch
/// seam is the documented forward path; when it ships, the <c>CapabilityInvoker</c> non-idempotent arm
@@ -45,16 +45,13 @@ public sealed record AlarmHistorianEvent(
[JsonIgnore]
public Guid DeliveryId { get; init; } = Guid.NewGuid();
/// <summary>Compares the ten business fields, excluding transport delivery identity.</summary>
/// <param name="other">The event to compare.</param>
/// <returns>Whether the business fields are equal.</returns>
/// <inheritdoc />
public bool Equals(AlarmHistorianEvent? other) => ReferenceEquals(this, other) || other is not null
&& AlarmId == other.AlarmId && EquipmentPath == other.EquipmentPath && AlarmName == other.AlarmName
&& AlarmTypeName == other.AlarmTypeName && Severity == other.Severity && EventKind == other.EventKind
&& Message == other.Message && User == other.User && Comment == other.Comment && TimestampUtc == other.TimestampUtc;
/// <summary>Hashes the same business fields used by equality, excluding delivery identity.</summary>
/// <returns>The business-field hash code.</returns>
/// <inheritdoc />
public override int GetHashCode()
{
var hash = new HashCode();
@@ -106,8 +106,8 @@ public static class Part9StateMachine
if (current.Acked == AlarmAckedState.Acknowledged)
{
// A COMMENTED re-ack carries a real comment even though nothing is acknowledged (Gitea #597 review
// m1): the OPC UA node reports it — the SDK writes the comment on a re-ack — so the system of
// A COMMENTED re-ack carries a real comment even though nothing is acknowledged (review
// finding m1): the OPC UA node reports it — the SDK writes the comment on a re-ack — so the system of
// record must hold it too, or the event advertises a comment that exists only on the node that
// served the call (never persisted, never replicated to the pair peer). Recorded exactly like
// AddComment; the ack fields (LastAck*) stay those of the real acknowledge. An uncommented re-ack
@@ -387,7 +387,7 @@ public enum EmissionKind
Enabled,
Disabled,
CommentAdded,
/// <summary>#478 — the worst-of-input source quality changed bucket (Good/Uncertain/Bad) with no
/// <summary>The worst-of-input source quality changed bucket (Good/Uncertain/Bad) with no
/// accompanying Part 9 state transition. Delivered out of band via the dedicated
/// <c>WriteAlarmQuality</c> node path, never through the <c>IAlarmSource</c> fan-out (it is not a
/// state change and must not materialize or historize a condition).</summary>
@@ -97,7 +97,7 @@ public sealed class ScriptedAlarmEngine : IDisposable
new(StringComparer.Ordinal);
/// <summary>
/// #478 — last emitted worst-of-input quality bucket per alarm (0 = Good, 1 = Uncertain,
/// Last emitted worst-of-input quality bucket per alarm (0 = Good, 1 = Uncertain,
/// 2 = Bad), computed each evaluation over the refilled read cache. A change in this bucket
/// with no accompanying Part 9 state transition drives a standalone
/// <see cref="EmissionKind.QualityChanged"/> emission (a Bad input freezes the condition — no
@@ -384,7 +384,7 @@ public sealed class ScriptedAlarmEngine : IDisposable
=> _alarms.Values.Select(a => a.Condition).ToArray();
/// <summary>
/// #487 — the currently-held condition of every loaded alarm, packaged with the definition's
/// The currently-held condition of every loaded alarm, packaged with the definition's
/// severity, the rendered message template and the last-observed input quality, i.e. everything
/// a caller needs to <b>project</b> the alarm onto an OPC UA condition node without waiting for
/// the next transition.
@@ -740,7 +740,7 @@ public sealed class ScriptedAlarmEngine : IDisposable
_ => new AlarmScratch(state.Inputs, state.Logger, _clock));
RefillReadCache(scratch.ReadCache, state.Inputs);
// #478 — worst OPC UA quality across the alarm's inputs, computed BEFORE the readiness
// Worst OPC UA quality across the alarm's inputs, computed BEFORE the readiness
// short-circuit so an outright-Bad input is still observed. A bucket change with no state
// transition is delivered out of band as a QualityChanged emission (see below).
var worstStatus = WorstInputStatus(scratch.ReadCache);
@@ -881,13 +881,13 @@ public sealed class ScriptedAlarmEngine : IDisposable
// adapter honors it to suppress ONLY the durable sink write; the live alerts fan-out is
// unaffected (it is not gated on this flag).
HistorizeToAveva: state.Definition.HistorizeToAveva,
// #478 — the worst input quality at evaluation time rides the transition so the projected
// The worst input quality at evaluation time rides the transition so the projected
// full snapshot keeps quality consistent (no clobber-to-Good).
WorstInputStatusCode: worstInputStatus);
}
/// <summary>
/// #478 — build a standalone <see cref="EmissionKind.QualityChanged"/> event carrying the new
/// Build a standalone <see cref="EmissionKind.QualityChanged"/> event carrying the new
/// worst-of-input quality. Emitted when the quality bucket moved but no Part 9 transition fired
/// (a Bad input freezes the condition; a Suppressed/None transition also leaves state unchanged).
/// The host routes it to the dedicated <c>WriteAlarmQuality</c> node path (annotate quality only,
@@ -913,14 +913,14 @@ public sealed class ScriptedAlarmEngine : IDisposable
/// the <see cref="AreInputsReady"/> guard already handles by holding the condition. Counting it would
/// flash every scripted condition Bad at deploy until the first push and flood the quality path with
/// load-time annotations, so it is skipped (contributes Good). Empty / all-unpushed cache ⇒ Good (0).
/// <para><b>"Never pushed" is a specific status code, not "the value is null" (#481, closing #478
/// Finding 3).</b> This used to skip every entry whose value was null, which made a genuine runtime Bad
/// <para><b>"Never pushed" is a specific status code, not "the value is null"
/// (Finding 3).</b> This used to skip every entry whose value was null, which made a genuine runtime Bad
/// indistinguishable from cold start — and that is exactly the shape a comms-lost driver produces, so
/// the case the skip was hiding was the case that matters most. The upstream source now stamps a
/// never-pushed path <see cref="StatusNeverPushed"/> and nothing else does, so the two are tellable
/// apart. A null snapshot is still skipped: that is a cache the refill did not populate, not an
/// observation.</para>
/// <para><b>The comparison itself lives in <see cref="InputQuality.WorstOf"/></b> (archreview R3-02)
/// <para><b>The comparison itself lives in <see cref="InputQuality.WorstOf"/></b>
/// — the same rule the virtual-tag and Calculation seams now use, so "worst input wins, carrying its
/// sub-code, first-seen on a severity tie" is stated once. What stays here is the alarm-specific
/// <em>filter</em>: this engine's two skips are deliberately NOT part of the shared rule, because the
@@ -952,7 +952,7 @@ public sealed class ScriptedAlarmEngine : IDisposable
return severity >= 2 ? 2 : (int)severity;
}
/// <summary>#478 — a canonical worst StatusCode for an alarm's last-observed quality bucket, used by
/// <summary>A canonical worst StatusCode for an alarm's last-observed quality bucket, used by
/// the operator-command + shelving-timer emission paths (which don't re-read inputs) so an ack /
/// shelve while an input is Bad still carries Bad rather than resetting the condition to Good.</summary>
private uint LastWorstStatus(string alarmId)
@@ -1264,14 +1264,14 @@ public sealed record ScriptedAlarmEvent(
DateTime TimestampUtc,
string? Comment = null,
bool HistorizeToAveva = true,
// #478 — the worst OPC UA StatusCode across the alarm's input tags at evaluation time. A raw uint,
// since Core.ScriptedAlarms does not reference Commons. Since #519 the host carries it to the condition
// The worst OPC UA StatusCode across the alarm's input tags at evaluation time. A raw uint,
// since Core.ScriptedAlarms does not reference Commons. The host carries it to the condition
// node UNCHANGED — it used to be collapsed to a 3-state bucket and re-expanded, losing the sub-code.
// Default 0u == Good keeps every existing constructor call unchanged.
uint WorstInputStatusCode = 0u);
/// <summary>
/// #487 — a loaded alarm's current condition, ready to be projected onto its OPC UA node.
/// A loaded alarm's current condition, ready to be projected onto its OPC UA node.
/// Produced by <see cref="ScriptedAlarmEngine.GetProjections"/> on demand; it is <b>not</b> an
/// emission and never travels the <see cref="ScriptedAlarmEngine.OnEvent"/> path, so it can never
/// become an <c>alerts</c> row or a historian record. Deliberately a separate type from
@@ -81,7 +81,7 @@ public sealed class ScriptedAlarmSource : IAlarmSource, IDisposable
{
if (_disposed) return;
// #478 — QualityChanged is a source-quality annotation, not a Part 9 state change. It is delivered
// QualityChanged is a source-quality annotation, not a Part 9 state change. It is delivered
// out of band via the dedicated WriteAlarmQuality node path; surfacing it here would fabricate a
// phantom AlarmEventArgs that materializes / historizes a condition. Swallow it.
if (evt.Emission == EmissionKind.QualityChanged) return;
@@ -81,7 +81,7 @@ public static class ScriptSandbox
// surface (operator-authored hosting helpers, third-party packages, app code)
// is not on the list and stays unreachable.
var references = new List<MetadataReference>();
// #603: both halves are cached. CreateFromFile reads + maps the DLL, and the BCL half is ~170
// both halves are cached. CreateFromFile reads + maps the DLL, and the BCL half is ~170
// assemblies, so doing it per compile was the dominant fixed cost of every ScriptEvaluator build.
// MetadataReference is immutable and thread-safe, so sharing instances across compilations is safe.
foreach (var asm in pinnedAssemblies)
@@ -2,7 +2,7 @@ namespace ZB.MOM.WW.OtOpcUa.Core.Authorization;
/// <summary>
/// <c>Security:Acl</c> — the switch that turns per-node ACL evaluation into the authorization
/// decision on the OPC UA data plane (#520, stage 5).
/// decision on the OPC UA data plane.
/// </summary>
/// <remarks>
/// <para>
@@ -39,7 +39,7 @@ public sealed class AclOptions
/// <summary>
/// When true, node ACL grants decide write, alarm-command and HistoryRead authorization. Default
/// false, which preserves the pre-stage-5 role gates byte-for-byte.
/// false, which preserves the previous role gates byte-for-byte.
/// </summary>
public bool Enabled { get; set; }
@@ -56,7 +56,7 @@ public sealed class AclOptions
/// </para>
/// <para>
/// <c>0</c> or negative disables the timer, leaving revalidation deploy-triggered only (the
/// pre-R3-05 behaviour). The timer is <b>inert while <see cref="Enabled"/> is false</b>: the
/// previous behaviour). The timer is <b>inert while <see cref="Enabled"/> is false</b>: the
/// pass early-returns when ACL enforcement is off.
/// </para>
/// </remarks>
@@ -16,7 +16,7 @@ namespace ZB.MOM.WW.OtOpcUa.Core.Authorization;
/// wrong-node authorization rather than a visible error.
/// </para>
/// <para>
/// <b>The string-identity rule for the whole authorization path (Gitea #533), stated here because
/// <b>The string-identity rule for the whole authorization path, stated here because
/// this type is where it was first violated:</b>
/// </para>
/// <list type="bullet">
@@ -26,7 +26,7 @@ namespace ZB.MOM.WW.OtOpcUa.Core.Authorization;
/// mean the same entity, and the sources genuinely differ in case (a <c>ClusterId</c> arrives
/// lower-cased through <c>OTOPCUA_ROLES</c> and verbatim through <c>Cluster:Roles</c> — see
/// <c>RoleParser.ClusterIdFromRole</c> — and a <c>ScopeId</c> is operator free-text in the ACL
/// editor until the stage-7 picker lands).
/// editor until the scope picker lands).
/// </item>
/// <item>
/// <b>Path segments compare <c>Ordinal</c></b> — every level of <see cref="NodeAclPath"/>,
@@ -36,7 +36,7 @@ namespace ZB.MOM.WW.OtOpcUa.Core.Authorization;
/// </item>
/// </list>
/// <para>
/// Before #533 this record struct used the compiler's default equality, i.e. ordinal, while the
/// Previously this record struct used the compiler's default equality, i.e. ordinal, while the
/// trie builder filtered <c>ClusterId</c> and hashed principals <c>OrdinalIgnoreCase</c> — three
/// rules in one path. It worked only because the resolver and the builder were handed the same
/// <c>NodeAcl</c> instances, so the <c>ScopeId</c> strings were literally the same object. Feed
@@ -49,9 +49,7 @@ namespace ZB.MOM.WW.OtOpcUa.Core.Authorization;
/// <param name="ScopeId">The authored scope id.</param>
public readonly record struct AclScopeRef(NodeAclScopeKind Kind, string ScopeId)
{
/// <summary>Equality over the kind plus a case-insensitive id — see the rule on the type.</summary>
/// <param name="other">The other reference.</param>
/// <returns><see langword="true"/> when both name the same scope.</returns>
/// <inheritdoc />
public bool Equals(AclScopeRef other) =>
Kind == other.Kind && string.Equals(ScopeId, other.ScopeId, StringComparison.OrdinalIgnoreCase);
@@ -62,7 +60,7 @@ public readonly record struct AclScopeRef(NodeAclScopeKind Kind, string ScopeId)
/// <summary>
/// Produces the <c>scopePaths</c> lookup <see cref="PermissionTrieBuilder.Build"/> consumes: for each
/// ACL scope, the trie path(s) at which a grant on it must be planted (#520 stage 3).
/// ACL scope, the trie path(s) at which a grant on it must be planted.
/// </summary>
/// <remarks>
/// <para>
@@ -131,7 +129,7 @@ public sealed class AclScopePathResolver
ArgumentNullException.ThrowIfNull(references);
// Every map is re-keyed OrdinalIgnoreCase rather than used as handed in. Ids are identifiers, so
// they follow the identifier half of the rule on AclScopeRef (Gitea #533) — and doing it HERE,
// they follow the identifier half of the rule on AclScopeRef — and doing it HERE,
// rather than documenting a comparer contract for callers, is what makes the rule impossible to
// get wrong from the outside: the two producers (EF entities at the deploy gate, artifact JSON on
// the node) build these dictionaries independently and neither can now diverge from the other.
@@ -224,7 +222,7 @@ public sealed class AclScopePathResolver
// Cluster never reaches here (the caller short-circuits it), and FolderSegment is the retired
// v2 kind with no v3 entity behind it — deliberately unresolvable. The trie builder now
// short-circuits FolderSegment too, so it reports RetiredScopeKind rather than the
// indistinguishable-from-a-typo UnresolvedScope (Gitea #533); this arm stays as the answer
// indistinguishable-from-a-typo UnresolvedScope; this arm stays as the answer
// for a direct ResolveOne caller.
case NodeAclScopeKind.Cluster:
case NodeAclScopeKind.FolderSegment:
@@ -4,7 +4,7 @@ using ZB.MOM.WW.OtOpcUa.Configuration.Entities;
namespace ZB.MOM.WW.OtOpcUa.Core.Authorization;
/// <summary>Create / list / revoke OPC UA data-plane API keys (#520).</summary>
/// <summary>Create / list / revoke OPC UA data-plane API keys.</summary>
/// <remarks>
/// A service rather than logic in the Razor page, because this repo has no bUnit — anything only a
/// page can reach is untestable, and minting credentials is not where that belongs.
@@ -1,7 +1,7 @@
namespace ZB.MOM.WW.OtOpcUa.Core.Authorization;
/// <summary>
/// <c>Security:ApiKeys</c> — OPC UA data-plane API key settings (#520). The data plane is
/// <c>Security:ApiKeys</c> — OPC UA data-plane API key settings. The data plane is
/// machine-to-machine, so a consuming system presents a key rather than an LDAP user.
/// </summary>
public sealed class ApiKeyOptions
@@ -34,7 +34,7 @@ public sealed class ApiKeyOptions
/// distinct Warning plus <c>otopcua.opcua.apikey_store.bounded{outcome=timeout}</c>.
/// </summary>
/// <remarks>
/// <c>0</c> or negative disables the deadline, restoring the pre-R3-05 unbounded behaviour. Do not
/// <c>0</c> or negative disables the deadline, restoring the previous unbounded behaviour. Do not
/// do that on a node with a ConfigDb: the SDK raises impersonation synchronously under a
/// <c>SessionManager</c>-wide lock, so an unanswered lookup stalls <b>every</b> session activation
/// in the server, not just its own.
@@ -5,7 +5,7 @@ namespace ZB.MOM.WW.OtOpcUa.Core.Authorization;
/// <summary>
/// Pure helpers for the OPC UA data-plane API key format <c>otua_&lt;keyId&gt;_&lt;secret&gt;</c>
/// (#520): generation, parsing, and constant-time verification. No I/O and no DI, so the
/// — generation, parsing, and constant-time verification. No I/O and no DI, so the
/// security-critical half is unit-testable on its own.
/// </summary>
/// <remarks>
@@ -5,7 +5,7 @@ namespace ZB.MOM.WW.OtOpcUa.Core.Authorization;
/// <summary>
/// The data-plane authorization inputs a node is currently serving: the API keys that may
/// authenticate, the ACL grants naming their principals, the cluster the node belongs to, and the
/// generation those were resolved at (#520, stage 4).
/// generation those were resolved at.
/// </summary>
/// <remarks>
/// <para>
@@ -215,7 +215,6 @@ public sealed class EmptyDataPlaneAuthoritySource : IDataPlaneAuthoritySource
public DataPlaneAuthoritySnapshot Current => DataPlaneAuthoritySnapshot.Empty;
/// <inheritdoc />
/// <remarks>Never raised — this source is permanently empty, so nothing derived from it can go stale.</remarks>
public event Action<DataPlaneAuthoritySnapshot>? Changed
{
add { }
@@ -252,7 +251,7 @@ public interface IDataPlaneAuthoritySink
/// </summary>
/// <remarks>
/// <para>
/// <b>It owns the <see cref="PermissionTrieCache"/> rather than being handed one</b> (#520 stage 5).
/// <b>It owns the <see cref="PermissionTrieCache"/> rather than being handed one</b>.
/// The trie is built from the same grants and the same generation as the snapshot, inside the same
/// <see cref="Apply"/> call, so the two cannot describe different deployments. Injecting the cache
/// would have made "install the snapshot" and "install the trie" two wirings, and this repo has
@@ -282,7 +281,7 @@ public sealed class DataPlaneAuthorityState : IDataPlaneAuthoritySource, IDataPl
/// <inheritdoc />
public DataPlaneAuthoritySnapshot Current => Volatile.Read(ref _current);
/// <inheritdoc />
/// <summary>Raised after <see cref="Apply"/> publishes a new snapshot.</summary>
public event Action<DataPlaneAuthoritySnapshot>? Changed;
/// <inheritdoc />
@@ -10,7 +10,7 @@ namespace ZB.MOM.WW.OtOpcUa.Core.Authorization;
/// <remarks>
/// <para>
/// The hierarchy is <c>Cluster → Realm → …node path…</c>. The node path is <b>variable length</b>,
/// which is the whole reason this type is no longer a fixed set of level properties (#520 stage 3):
/// which is the whole reason this type is no longer a fixed set of level properties:
/// </para>
/// <list type="bullet">
/// <item>
@@ -57,7 +57,7 @@ public enum NodeHierarchyKind
/// <summary><c>Area / Line / Equipment [/ EffectiveName]</c> — the UNS realm.</summary>
Uns,
/// <summary><c>folder… / Driver / Device / group… / Tag</c> — the raw device-tree realm. (Appended, #520 stage 3.)</summary>
/// <summary><c>folder… / Driver / Device / group… / Tag</c> — the raw device-tree realm. (Appended.)</summary>
Raw,
}
@@ -69,7 +69,7 @@ public enum NodeHierarchyKind
/// <para>
/// This exists because v3 <b>has no <c>Namespace</c> entity</b>. The snapshot was retired when the
/// address space went dual-namespace ("the two OPC UA namespaces are implicit"), which left the
/// trie's level-1 gate with no configured key source at all — the pre-stage-3 walker read a
/// trie's level-1 gate with no configured key source at all — the earlier walker read a
/// <c>NamespaceId</c> that nothing in the system ever produced, so the level could only ever fail
/// to match. Deriving the segment from the realm restores the level as something real without
/// reintroducing an entity.
@@ -1,6 +1,6 @@
namespace ZB.MOM.WW.OtOpcUa.Core.Authorization;
/// <summary>Reserved principal names for the OPC UA data plane (#520).</summary>
/// <summary>Reserved principal names for the OPC UA data plane.</summary>
/// <remarks>
/// Lives in Core rather than beside the authenticator because BOTH ends need it: the OPC UA server
/// resolves an anonymous session to it, and the AdminUI authoring path must refuse to mint a key
@@ -151,7 +151,7 @@ public sealed class PermissionTrieNode
/// </summary>
/// <remarks>
/// ⚠️ The comparer is <b>case-sensitive ordinal</b>, matching <c>RawPaths.Comparer</c> and OPC UA
/// NodeId identity. It was <c>OrdinalIgnoreCase</c> before #520 stage 3, which was a silent
/// NodeId identity. It was <c>OrdinalIgnoreCase</c> in an earlier version, which was a silent
/// widening: raw sibling names are unique under an ordinal comparer, so <c>Speed</c> and
/// <c>speed</c> can both exist under one device, and a grant on either would have matched both.
/// Case-sensitive matching is also the fail-closed direction — a case-drifted ACL row now grants
@@ -167,7 +167,7 @@ public sealed class PermissionTrieNode
/// </summary>
/// <remarks>
/// Compared <c>OrdinalIgnoreCase</c>: this holds principal names, which are identifiers, and the
/// repo's rule (#533) is that identifiers compare case-insensitively while <em>path segments</em>
/// repo's rule is that identifiers compare case-insensitively while <em>path segments</em>
/// — the <see cref="Children"/> keys — compare ordinally.
/// </remarks>
public HashSet<string> BrowseTransitPrincipals { get; } = new(StringComparer.OrdinalIgnoreCase);
@@ -14,7 +14,7 @@ namespace ZB.MOM.WW.OtOpcUa.Core.Authorization;
/// life of the cache entry. Idempotent — two builds from the same rows produce equal tries.
/// </para>
/// <para>
/// ⚠️ <b>There is no longer a "deterministic test mode".</b> Before #520 stage 3 a null
/// ⚠️ <b>There is no longer a "deterministic test mode".</b> Previously a null
/// <c>scopePaths</c> made sub-cluster rows fall back to a direct child of the trie
/// root keyed on <c>ScopeId</c>. That position is unreachable by construction — level 1 of the walk
/// is the realm — so the fallback produced grants that looked placed and could never match, and
@@ -183,7 +183,7 @@ public enum PermissionTrieBuildDiagnosticReason
/// <summary>
/// The row's <see cref="NodeAclScopeKind"/> is retired and has no v3 entity behind it
/// (<see cref="NodeAclScopeKind.FolderSegment"/>), so it can never resolve whatever its
/// <c>ScopeId</c>. Split out of <see cref="UnresolvedScope"/> by Gitea #533: the two have
/// <c>ScopeId</c>. Split out of <see cref="UnresolvedScope"/> deliberately: the two have
/// different remedies — re-scope the grant to <see cref="NodeAclScopeKind.RawFolder"/> versus
/// fix an id — and conflating them told an operator with a legacy row to go looking for a typo.
/// </summary>
@@ -7,7 +7,7 @@ namespace ZB.MOM.WW.OtOpcUa.Core.Authorization;
/// </summary>
/// <remarks>
/// ⚠️ <b>The refresh machinery below is LDAP-shaped and is largely vestigial under the
/// API-key model (#520).</b> It exists because an LDAP group membership can change
/// API-key model.</b> It exists because an LDAP group membership can change
/// out-of-band in the directory, so a long-lived session had to re-resolve periodically.
/// An API-key principal set cannot change out-of-band: keys and ACL rows both ride the
/// deployment artifact (driver-only nodes have no ConfigDb since Phase 4), so the only
@@ -141,7 +141,7 @@ public static class EquipmentNodeWalker
/// <summary>
/// Pre-loaded + pre-filtered snapshot of one cluster's UNS content (Area / Line / Equipment +
/// per-equipment VirtualTags / ScriptedAlarms). v3: raw-tag references (<c>UnsTagReference</c>)
/// are materialized by the Batch-4 dual-namespace fan-out, not by this walker, so they are not
/// are materialized by the dual-namespace fan-out, not by this walker, so they are not
/// part of this snapshot.
/// </summary>
public sealed record EquipmentNamespaceContent(
@@ -30,7 +30,7 @@ public static class IdentificationFolderBuilder
public const string FolderName = "Identification";
/// <summary>
/// Canonical decision #139 field set exposed in the Identification sub-folder. Order
/// Canonical field set exposed in the Identification sub-folder. Order
/// matches the decision-log entry so any browse-order reader can cross-reference
/// without re-sorting.
/// </summary>
@@ -132,7 +132,7 @@ public sealed class CapabilityInvoker : IDriverCapabilityInvoker
{
// Build the no-retry options snapshot ONCE per invoker lifetime (01/P-4). The options are
// immutable for the invoker's lifetime — a ResilienceConfig change respawns the driver child,
// which constructs a fresh invoker (#13 respawn-on-change) — so lazily caching the snapshot is
// which constructs a fresh invoker (respawn-on-change) — so lazily caching the snapshot is
// safe and removes the per-write allocation the previous once-per-call snapshot incurred. The
// null-check-assign tolerates a benign first-call race (both threads build an equivalent record).
var noRetryOptions = _noRetryWriteOptions ??= BuildNoRetryWriteOptions();
@@ -81,7 +81,7 @@ public sealed class DriverCapabilityInvokerFactory : IDriverCapabilityInvokerFac
// too — a retained "Breaker OPEN" from a pre-respawn outage could never self-correct (only the
// discarded pipeline's OnClosed stamps the close) and would show a false OPEN chip forever.
// Reinstate first: an instance id removed by an earlier deployment (Release ⇒ Retire) and now re-added
// must record again (#609).
// must record again.
_statusTracker.Reinstate(driverInstanceId);
_statusTracker.Invalidate(driverInstanceId);
@@ -94,17 +94,15 @@ public sealed class DriverCapabilityInvokerFactory : IDriverCapabilityInvokerFac
optionsGeneration: generation);
}
// Without this the tracker kept a removed instance's snapshots forever (only Create
// invalidated, and a removed instance is never re-created), and the resilience publisher re-emitted
// them every tick — undoing the host's telemetry-hub eviction within one interval.
// The tracker retires the instance rather than merely invalidating it: the host stops the
// child asynchronously, so a capability call it still had in flight completes through the released
// invoker afterwards, and its call-start/success/complete writes re-created the entry (observed: the
// child's in-flight SubscribeBulk). A retired id ignores those writes until a later Create
// for the same id reinstates it.
/// <inheritdoc />
/// <remarks>
/// #609: without this the tracker kept a removed instance's snapshots forever (only <see cref="Create"/>
/// invalidated, and a removed instance is never re-created), and the resilience publisher re-emitted
/// them every tick — undoing the host's telemetry-hub eviction within one interval.
/// <para>The tracker <b>retires</b> the instance rather than merely invalidating it: the host stops the
/// child asynchronously, so a capability call it still had in flight completes through the released
/// invoker afterwards, and its call-start/success/complete writes re-created the entry (observed: the
/// child's in-flight SubscribeBulk). A retired id ignores those writes until a later <see cref="Create"/>
/// for the same id reinstates it.</para>
/// </remarks>
public void Release(string driverInstanceId)
{
_builder.Invalidate(driverInstanceId);
@@ -19,7 +19,7 @@ public sealed record DriverResilienceOptions
public IReadOnlyDictionary<DriverCapability, CapabilityPolicy> CapabilityPolicies { get; init; }
= new Dictionary<DriverCapability, CapabilityPolicy>();
// RecycleIntervalSeconds is GONE (Gitea #522). It configured a scheduled recycle of a Tier C
// RecycleIntervalSeconds is GONE. It configured a scheduled recycle of a Tier C
// driver's out-of-process Host — a process that no longer exists anywhere: Galaxy reaches MXAccess
// over gRPC to the external mxaccessgw sidecar (PR 7.2 retired the in-process Host/Proxy/Shared
// projects) and FOCAS has run in-process since its managed wire client landed 2026-04-24. The
@@ -64,7 +64,7 @@ public sealed record DriverResilienceOptions
/// Per-tier per-capability default policy table, per the Phase 6.1
/// Stream A.2 specification. The parser enforces no-retry on
/// <see cref="DriverCapability.Write"/> and <see cref="DriverCapability.AlarmAcknowledge"/>
/// as an invariant (R2-02/S-8) — a JSON retryCount override on those capabilities is forced back
/// as an invariant — a JSON retryCount override on those capabilities is forced back
/// to 0; per-tag opt-in via <see cref="WriteIdempotentAttribute"/> is the future relaxation.
/// </summary>
/// <param name="tier">The driver tier to get defaults for.</param>
@@ -84,7 +84,7 @@ public sealed record DriverResilienceOptions
// Initialize: no retry (the DriverInstanceActor reconnect timer is the retry loop) and a
// threshold of 2, NOT the Read-sized 5 — the breaker samples over 30 s and the reconnect
// timer fires every 10 s, so at most three init attempts fit one window; a threshold of 5
// could never open on connect failures (R2-10 T10).
// could never open on connect failures.
[DriverCapability.Initialize] = new(TimeoutSeconds: 5, RetryCount: 0, BreakerFailureThreshold: 2),
},
DriverTier.B => new Dictionary<DriverCapability, CapabilityPolicy>
@@ -94,7 +94,7 @@ public static class DriverResilienceOptionsParser
}
}
// No recycle-interval handling: the scheduled-recycle machinery was deleted in Gitea #522
// No recycle-interval handling: the scheduled-recycle machinery was deleted
// (nothing constructed it, and IDriverSupervisor — which would have performed the recycle —
// had no implementations). A "recycleIntervalSeconds" key surviving in an existing
// ResilienceConfig blob is harmless: the shape no longer declares it, and unknown JSON
@@ -16,13 +16,13 @@ namespace ZB.MOM.WW.OtOpcUa.Core.Resilience;
/// Pure in-memory (no EF Core / SQL Server dependency) so tests can exercise it directly.
/// The operator-facing reader is the DPS publisher named above, mirroring the driver-health
/// flow — not the DB-table/HostedService shape originally sketched for Phase 6.1 Stream E
/// (R2-10 built the DPS reader instead).
/// (the DPS reader was built instead).
/// </remarks>
public sealed class DriverResilienceStatusTracker
{
private readonly ConcurrentDictionary<StatusKey, ResilienceStatusSnapshot> _status = new();
// Instances retired by Retire (removed from the deployed plan) and not since Reinstated (#609). Every
// Instances retired by Retire (removed from the deployed plan) and not since Reinstated. Every
// Record* for one of these is ignored: the removed instance's child stops ASYNCHRONOUSLY, so a capability
// call it still had in flight completes through its invoker after the host's release and would otherwise
// re-create the entry — which the resilience publisher then re-emits every tick, forever. One entry per
@@ -127,7 +127,7 @@ public sealed class DriverResilienceStatusTracker
/// <summary>
/// Drop every snapshot for a driver instance that has been REMOVED from the deployed plan, and ignore
/// every later write for it until <see cref="Reinstate"/> (#609). Called by
/// every later write for it until <see cref="Reinstate"/>. Called by
/// <c>DriverCapabilityInvokerFactory.Release</c>. Unlike <see cref="Invalidate"/> (a respawn, whose new
/// invoker records straight away), the removed instance's child is still stopping — a call it had in
/// flight finishes after this returns, and without the hold its RecordCallStart / RecordSuccess /
@@ -261,7 +261,7 @@ public sealed record ResilienceStatusSnapshot
/// <c>driver-resilience-status</c> telemetry channel.
/// <para>It used to say this fed <c>DriverInstanceResilienceStatus.CurrentBulkheadDepth</c> "until
/// the full telemetry observer lands". The telemetry observer landed (Phase 5); that table never
/// had a writer and was dropped in Gitea #524.</para>
/// had a writer and was since dropped.</para>
/// </summary>
public int CurrentInFlight { get; init; }
}
@@ -54,7 +54,7 @@ public sealed class ProbeCommand : S7CommandBase
// Derive the headline verdict from BOTH the driver state AND the probe-read
// StatusCode so the operator never sees `Health: Healthy` over a Bad snapshot
// line (Driver.S7.Cli-012; mirrors Modbus.Cli's ComputeVerdict).
// line (mirrors Modbus.Cli's ComputeVerdict).
var verdict = ComputeVerdict(health.State, snapshot[0].StatusCode);
await console.Output.WriteLineAsync($"Host: {Host}:{Port}");
@@ -25,7 +25,7 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.Ab.LibplctagShared;
/// release cannot run; the tag is then rooted for the life of the process instead, which keeps its delegate
/// alive. That is no crash but an UNBOUNDED leak — one native tag (holding its libplctag session open) per failed
/// create, and the data path re-creates a NotFound tag on every read — so <c>LibplctagOrphanReleaseTests</c> / <c>LibplctagLegacyOrphanReleaseTests</c> fail, so the bump is noticed.</para>
/// <para>SHARED SOURCE (#580): this directory has no project. <c>Driver.AbCip</c> and <c>Driver.AbLegacy</c> each
/// <para>SHARED SOURCE: this directory has no project. <c>Driver.AbCip</c> and <c>Driver.AbLegacy</c> each
/// compile this file in (<c>&lt;Compile Include&gt;</c>), so each assembly has its own internal copy and its own
/// counters. It must keep a <c>Libplctag*.cs</c> name — <c>LibplctagConfinementGuardTests</c> allows libplctag
/// references only in such files.</para>
@@ -35,7 +35,7 @@ internal static class LibplctagOrphanRelease
private const BindingFlags Private = BindingFlags.NonPublic | BindingFlags.Instance;
private static readonly FieldInfo? WrapperField = typeof(Tag).GetField("_tag", Private);
// Final review M2: the field must also still be an int. A wrapper that retyped it (IntPtr, long) would make
// The field must also still be an int. A wrapper that retyped it (IntPtr, long) would make
// the cast in HandleOf throw; treating it as unreadable instead degrades to the rooting fallback ("keep
// alive"), which the pin test reports.
private static readonly FieldInfo? HandleField =
@@ -57,6 +57,8 @@ internal static class LibplctagOrphanRelease
/// <summary>The native handle the wrapper holds (0 when none was ever created, or unreadable). Never throws:
/// callers run it inside a catch block, where an exception would replace the failure being handled.</summary>
/// <param name="tag">The libplctag wrapper whose handle is read.</param>
/// <returns>The native handle, or 0 when none exists or it cannot be read.</returns>
internal static int HandleOf(Tag tag)
{
try
@@ -70,7 +72,7 @@ internal static class LibplctagOrphanRelease
}
}
/// <summary>Clears the wrapper's handle after a destroy, so a repeat release is a no-op (review r1 M2).</summary>
/// <summary>Clears the wrapper's handle after a destroy, so a repeat release is a no-op.</summary>
private static void ClearHandle(Tag tag)
{
if (HandleField is not null && WrapperField!.GetValue(tag) is { } wrapper) HandleField.SetValue(wrapper, 0);
@@ -81,6 +83,9 @@ internal static class LibplctagOrphanRelease
/// Read/Write, which initialise on demand). If it fails and leaves the tag uninitialised, the native tag it
/// created is unregistered and destroyed before the failure propagates.
/// </summary>
/// <param name="tag">The tag the operation may initialise.</param>
/// <param name="op">The libplctag operation to run.</param>
/// <returns>A task that represents the asynchronous operation.</returns>
internal static async Task RunAsync(Tag tag, Func<Task> op)
{
try
@@ -112,15 +117,19 @@ internal static class LibplctagOrphanRelease
/// Destroys the native tag behind an UNINITIALISED wrapper, unregistering its callback first. No-op for an
/// initialised tag (its own <c>Dispose</c> handles it) or one that never got a handle. Never throws.
/// </summary>
/// <param name="tag">The libplctag wrapper to release when orphaned.</param>
/// <returns>The status the native tag held just before it was destroyed (what <c>GetStatus()</c> would have
/// reported), or null when nothing was destroyed.</returns>
internal static Status? ReleaseIfOrphaned(Tag tag) => ReleaseIfOrphaned(tag, CanReadHandles);
/// <summary>
/// <see cref="ReleaseIfOrphaned(Tag)"/> with the handle-readability decision passed in, so a test can take the
/// rooting fallback (review r1 M1) without a process-wide switch (Gitea #581). Production always passes
/// rooting fallback without a process-wide switch. Production always passes
/// <see cref="CanReadHandles"/>.
/// </summary>
/// <param name="tag">The libplctag wrapper to release when orphaned.</param>
/// <param name="canReadHandles">Whether the wrapper's native handle can be read; when false the tag is rooted instead.</param>
/// <returns>The status the native tag held just before it was destroyed, or null when nothing was destroyed.</returns>
internal static Status? ReleaseIfOrphaned(Tag tag, bool canReadHandles)
{
try
@@ -2,7 +2,7 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.Ab.Transport;
/// <summary>
/// The single <see cref="AbStatus"/> to OPC UA StatusCode table shared by <c>AbCipStatusMapper</c> and
/// <c>AbLegacyStatusMapper</c> (#580) — the two drivers used to carry identical copies.
/// <c>AbLegacyStatusMapper</c> — the two drivers used to carry identical copies.
/// </summary>
public static class AbOpcUaStatusMap
{
@@ -2,6 +2,10 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.Ab.Transport;
public static class AbStatusCodes
{
/// <summary>Maps a CIP general status (and optional extended status words) to an <see cref="AbStatus"/>.</summary>
/// <param name="general">The CIP general status byte.</param>
/// <param name="extended">The extended status words, if any.</param>
/// <returns>The mapped status.</returns>
public static AbStatus FromCip(byte general, ReadOnlySpan<ushort> extended) => general switch
{
0x00 or 0x0B => AbStatus.Ok,
@@ -18,6 +22,10 @@ public static class AbStatusCodes
_ => AbStatus.BadReply,
};
/// <summary>Maps a PCCC status byte (and optional extended status) to an <see cref="AbStatus"/>.</summary>
/// <param name="sts">The PCCC STS byte.</param>
/// <param name="extSts">The extended status byte, when present.</param>
/// <returns>The mapped status.</returns>
public static AbStatus FromPccc(byte sts, byte? extSts) => sts switch
{
0x00 => AbStatus.Ok,
@@ -1,9 +1,10 @@
namespace ZB.MOM.WW.OtOpcUa.Driver.Ab.Transport;
/// <summary>Thrown by both AB transports. <see cref="Status"/> drives OPC UA status mapping and the
/// driver's "controller answered, don't throttle the device" classification (05/STAB-8).</summary>
/// driver's "controller answered, don't throttle the device" classification.</summary>
public class AbTransportException(AbStatus status, string message, Exception? inner = null)
: Exception(message, inner)
{
/// <summary>The AB status that classifies this failure.</summary>
public AbStatus Status { get; } = status;
}
@@ -12,9 +12,9 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.Ab.Transport.Cip;
/// only by the <i>next</i> send; the failing call throws its own exception. Caller cancellation never
/// faults the session (B2), so it never causes a reconnect. A connected reply saying the target no longer
/// holds the class-3 connection (general 0x01, extended 0x0107 / 0x0203 — the session itself still up) retires
/// the pair at once, whether a request or a keep-alive got it, so the next send reopens it (Gitea #566).</para>
/// the pair at once, whether a request or a keep-alive got it, so the next send reopens it.</para>
/// <para><b>Bounded.</b> Every <see cref="SendAsync"/> is bounded even if the caller passes no token
/// (ruling R9-3): opening (including the throttle wait and any wait behind another opener) is bounded by
/// — opening (including the throttle wait and any wait behind another opener) is bounded by
/// <c>MinForwardOpenInterval + 4 × Timeout</c> (connect, RegisterSession, Forward Open, fallback Forward
/// Open), and the exchange by <c>2 × Timeout</c> (waiting behind one in-flight exchange, then its own).
/// Exceeding either surfaces as <see cref="AbTransportException"/>(<see cref="AbStatus.Timeout"/>).</para>
@@ -52,6 +52,7 @@ public sealed class CipConnection : ICipChannel, IAsyncDisposable
private int _keepAlivesSent;
private int _connectionSerial = Random.Shared.Next(1, ushort.MaxValue);
/// <summary>Creates a connection for one device; nothing is opened until the first send.</summary>
/// <param name="key">The device; <see cref="DeviceKey.CipPath"/> is the route (ignored for Direct).</param>
/// <param name="options">Routing, sizes, timeouts and keep-alive settings.</param>
/// <param name="throttle">The device's process-lifetime Forward Open throttle.</param>
@@ -91,6 +92,7 @@ public sealed class CipConnection : ICipChannel, IAsyncDisposable
private TimeSpan OpenBudget => _o.MinForwardOpenInterval + 4 * _o.Timeout;
private TimeSpan ExchangeBudget => 2 * _o.Timeout;
/// <inheritdoc />
public int MaxRequestBytes => _o.Routing switch
{
CipRouting.Connected => (Volatile.Read(ref _open)?.NegotiatedSize ?? Math.Min(_o.ConnectionSize, UnconnectedMessageLimit)) - 2,
@@ -98,6 +100,7 @@ public sealed class CipConnection : ICipChannel, IAsyncDisposable
_ => UnconnectedMessageLimit,
};
/// <inheritdoc />
public async Task<ReadOnlyMemory<byte>> SendAsync(ReadOnlyMemory<byte> cipRequest, CancellationToken cancellationToken)
{
ThrowIfDisposed();
@@ -140,7 +143,7 @@ public sealed class CipConnection : ICipChannel, IAsyncDisposable
return pending.WaitAsync(ct);
}
/// <summary>The shared open. Bounded by <see cref="OpenBudget"/> (ruling R9-3); its outcome, success or
/// <summary>The shared open. Bounded by <see cref="OpenBudget"/>; its outcome, success or
/// failure, goes to every caller waiting on it.</summary>
private async Task RunOpenAsync(TaskCompletionSource<Open> result)
{
@@ -222,7 +225,7 @@ public sealed class CipConnection : ICipChannel, IAsyncDisposable
var frame = ConnectionManagerServices.ForwardOpen(p);
_diagnostics.CountForwardOpen();
var raw = await client.SendRRDataAsync(frame, _o.Timeout, ct).ConfigureAwait(false);
_diagnostics.NotifyExchange(frame, raw); // evidence tap: the granted RPIs / connection ids (#566 gate)
_diagnostics.NotifyExchange(frame, raw); // evidence tap: the granted RPIs / connection ids
return CipReply.Parse(raw);
}
@@ -270,7 +273,7 @@ public sealed class CipConnection : ICipChannel, IAsyncDisposable
/// The target answered over a session that is still up, but says the class-3 connection itself is gone (it
/// timed the connection out, or was restarted / re-downloaded and forgot it). Every later connected request
/// would get the same answer while <see cref="EncapsulationClient.IsConnected"/> stays true, so nothing else
/// would ever reopen it (Gitea #566): retire the pair now, so the next send reopens through the throttle.
/// would ever reopen it: retire the pair now, so the next send reopens through the throttle.
/// No Forward Close — the target has just said there is nothing to close.
/// </summary>
private async Task RetireClosedByTargetAsync(Open open)
@@ -333,6 +336,7 @@ public sealed class CipConnection : ICipChannel, IAsyncDisposable
/// <summary>Stops the keep-alive, sends Forward Close (Connected only) and closes the session, whose
/// own dispose sends UnRegisterSession. Later sends throw <see cref="AbStatus.Abort"/>.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
public async ValueTask DisposeAsync()
{
if (Interlocked.Exchange(ref _disposeStarted, 1) != 0) return;
@@ -7,6 +7,8 @@ public static class CipPath
{
/// <summary>Route path from "port,link[,port,link…]". Links that are not a byte value are IP
/// addresses, encoded as extended link-address port segments.</summary>
/// <param name="cipPath">The route as comma-separated port,link pairs; blank yields an empty route.</param>
/// <returns>The encoded route path segments.</returns>
public static byte[] EncodeRoute(string cipPath)
{
if (string.IsNullOrWhiteSpace(cipPath)) return [];
@@ -47,6 +49,8 @@ public static class CipPath
}
/// <summary>Splits a trailing purely numeric member ("Tag.5") into the tag name and bit index.</summary>
/// <param name="tagName">The tag name, optionally ending in a numeric bit selector.</param>
/// <returns>The tag name without the selector and the bit index, or a null bit when there is no selector.</returns>
public static (string Name, int? Bit) SplitBitSelector(string tagName)
{
var dot = tagName.LastIndexOf('.');
@@ -59,6 +63,9 @@ public static class CipPath
return (tagName, null);
}
/// <summary>Encodes a symbolic tag name (members and subscripts) as ANSI extended symbol segments.</summary>
/// <param name="tagName">The tag name, without any bit selector.</param>
/// <returns>The encoded symbolic path segments.</returns>
/// <exception cref="ArgumentException">The name is empty, has an empty member, an over-long (&gt; 255) or
/// non-ASCII member, an unterminated / non-numeric subscript, or ends in a bit selector.</exception>
public static byte[] EncodeSymbolic(string tagName)
@@ -118,6 +125,10 @@ public static class CipPath
yield return name[start..];
}
/// <summary>Encodes a logical class and instance path (8- or 16-bit segments as the values require).</summary>
/// <param name="cls">The CIP class ID.</param>
/// <param name="instance">The CIP instance ID.</param>
/// <returns>The encoded class/instance path segments.</returns>
public static byte[] EncodeClassInstance(ushort cls, uint instance)
{
var w = new CipWriter();
@@ -129,6 +140,8 @@ public static class CipPath
}
/// <summary>Prefixes the path with its size in 16-bit words.</summary>
/// <param name="segments">The encoded path segments.</param>
/// <returns>The size byte followed by the segments.</returns>
/// <exception cref="ArgumentException">The segments are not a whole number of 16-bit words, or exceed
/// the 255-word maximum the one-byte size field can carry.</exception>
public static byte[] WithPathSize(byte[] segments)
@@ -7,6 +7,7 @@ public sealed record CipReply(byte Service, byte GeneralStatus, ushort[] Extende
{
// Record equality over an array / ReadOnlyMemory is reference equality; two parses of the same wire
// bytes must compare equal, so compare the extended-status words and the data by content.
/// <inheritdoc />
public bool Equals(CipReply? other) =>
other is not null
&& Service == other.Service
@@ -14,6 +15,7 @@ public sealed record CipReply(byte Service, byte GeneralStatus, ushort[] Extende
&& ExtendedStatus.AsSpan().SequenceEqual(other.ExtendedStatus)
&& Data.Span.SequenceEqual(other.Data.Span);
/// <inheritdoc />
public override int GetHashCode()
{
var h = new HashCode();
@@ -24,8 +26,12 @@ public sealed record CipReply(byte Service, byte GeneralStatus, ushort[] Extende
return h.ToHashCode();
}
/// <summary>The reply status mapped from the general and extended status words.</summary>
public AbStatus Status => AbStatusCodes.FromCip(GeneralStatus, ExtendedStatus);
/// <summary>Parses a raw message-router reply.</summary>
/// <param name="raw">The raw reply bytes.</param>
/// <returns>The parsed reply.</returns>
public static CipReply Parse(ReadOnlyMemory<byte> raw)
{
var r = new CipReader(raw.Span);

Some files were not shown because too many files have changed in this diff Show More